Assuming you are running vlc with defaults on your windows machine, can you try the following for me in a browser on that machine:

1)
http://127.0.0.1:8080/requests/browse.xml?dir=/
and paste the results.

2)
http://127.0.0.1:8080/requests/playlist.xml
Don't paste these results, I'm guessing this will be a huge amount of xml in your browser. But could you take a screenshot and post it?

I think what you are trying to tell me is that you have a large library within VLC not just on the filesystem? Sorry if I misunderstood that earlier.
This has always been a problem for remote clients because it creates a massive xml message (upwards of several MB) to parse on the phone.
If you are interested I can probably give you a fix to install on your PC to not output the 'Media Library' into the playlist that is consumed by the remote. (It doesn't involve removing your library on your PC)

cheers