You could probably do it from above, the switch just clips in from above - if you can find where the clips are...
Mind you, if you have the other one in your hand, you'll know where they are - might need to push them in with a small screw driver.
I'm fairly sure the connector pulls through, once you've got the switch out.