Mine needed a fair whack to get them out. Don't use a punch thats too small.
When I did it, the hardest bit was getting the near side inner pad out. It didn't want to move at all, and in the end I used a large cold chisel on the backing plate, alternately tapping the top and bottom of the pad from the rear. Not sure why that one pad was so tight?!