Sure you can!
As soon as you remove the battery cover the internal card is unmounted (just as when you open the flip over the external card) and you can safely remove it. You can check this in an xterm: Enter 'mount' and see that it is mounted (it's mmc2). Remove the cover, enter the 'mount' command again. mmc2 is gone. Replace the cover, the card is mounted again.
(The latter means that you can also insert a card into the internal slot when the N800 is on, I kept doing that all the time in the beginning when I had that problem where the N800 refused to boot with the card installed.)