I got the Galaxy Nexus a week ago. I am very happy with it so far. Sure, the Note has better specs, especially the screen. The CPU and GPU matter less, because the Nexus CPU/GPU is pretty fast as it is, and for most applications it makes no difference.
But for something you carry around all the tie, the size is very important as well, and the Nexus is great in terms of size.