Yeah, you remember I moaned about it before? When you use the advanced edit, then click "more" at the bottom of the smilie list - the popup window needs to be wider/bigger.
If I remember, there's an X and Y variables in the page (probably in the advance edit window), but I can't remember what he said he set them to last time...