USB bus may be more power hungry, but it's also more standard and obviously more secure (not sharing memory between modem and other things). Sizing of USB sticks is an issue, yes. I would have hoped that they are small enough to hide completely inside USB port, but it may be more difficult for antenna-equipped USB modems. Antenna reception may be challenging, too.

Edit: Ben Nanonote is fairly close to 100% FOSS, I reckon.