Does anyone have tips or advice on how an N800 might be usable as a portable, searchable repository for a large amount of information?

Let's say I max it out with a couple of high-capacity SD cards, with the intention that as I find web pages, PDFs, or just random text snippets that are interesting to me, I want to capture them permanently on the N800 for later search and retrieval.

I've run into some road blocks in my experiments so far:

1. It's not clear what the built-in search tool is actually able to search. It seems to be able to search RTF files, but not TXT files, for example. I'm not sure if it can search PDF files. What about saved pages from Opera?

2. It's also not clear to me if the built-in search looks just at built-in memory, one of the memory cards, or all available storage.

3. The built-in search appears not to be terribly fast. Is it going to become unusable with a large volume of data?

4. I'd really like to do this "offline" so I don't need a net connection to look stuff up. So, let's not talk about web apps here.

Is anyone already doing anything like this, and if so, what works and what doesn't?

What are the best options for performance of searching?

Are there any utilities that make "capturing" easier than the somewhat tedious copy-and-paste mechanism in Hildon, followed by Notes > New Note > Paste > Save > Type File name > Click Save > etc...?

If I capture an item on the desktop, can I send it to the N800 easily with Bluetooth?

What are the most compatible / interchangeable-with-desktop file formats to use? (I use Mac OS X on the desktop)

This is kind of a theoretical question, but my N800 gets dustier and dustier thanks to my iPhone, and offline storage is one of the few things that the N800 still beats the iPhone at. Hoping to get some insightful discussion rolling.

Hmm ok i see this isnt what you were asking but i'll leave it here anyway since the article title might lure users whom this -would- be applicable

- I think the latest Mozilla supports saving html and reading offline?

- Perhaps grep exists in xterm?

- I use maemopad+ for hierarchical notes--- great for organizing info/clips

----

Wikipedia offers an offline html version intended for 3rd world countries which i used to run on my nokia 770. Its just a small subset so you might not be able to look up spur of the moment type things but its a nice reference to spend hours reading what they 'did' compile.

I think this is the main page
http://www.soschildrensvillages.org....or-schools.htm

They just updated it to 2007 version so its better than what i had at the time (by about 300 megs more).

Warning though copying all those small files takes forever you might want to copy branches at a time.

You cant search since theres no server but they index and alphabetize the contents.

Someone might be able to port Kiwi :
http://www.kiwix.org/index.php/Main_Page

That is small program running on top of static files to let you search.

sdict viewer! Search the forums, and there's even a reasonably recent copy of wikipedia (Jan 07) available for it at sdict.com

it all sounds easy if you're comfortable porting and working with linux command line apps. the porting I've done so far has been a doddle.

grep does exist as part of busybox, but (1) it finds text strings, (2) you want an indexed solution, and (3) for heavy use better to compile real grep with all options (see 'doddle' above)

I've yet to get into it, but sounds like you're after desktop searching. try googling 'linux desktop search'; easiest starting point would be something that is just command line tools underneath. the ones I remember looking into had plugins or some other mechanism to handle pdf, word doc, etc. basic idea is system indexes all your file contents, then monitors for changes or new files to keep up to date. dirs to index are configurable. you might run into hassles if there are lots of external libs/progs needed for all the file formats you want, but probably not.

good luck,

rob.

I missed the part about capturing information in my post above about sdict. It's still a great program, but no capture abilities.

For organization, I may be in the minority, but I'm very prone to large directory trees and filing things away in new folders as needed. However, all information doesn't lend itself to this approach, and sometimes filenames don't match contents, etc.

For the Sharp Zaurus, there was a program called portabase that you could use to store various bits of text. I don't think it's been ported to the maemo platform, and probably used the QT toolkit, rather than GTK+, but the underlying database was able to be read by other programs, so it should be fairly portable.

I'm thinking that since you are starting from nothing, you could just make your own database and create a simple search front end with rails. I could be talking out my *** but it would seem a simple enough prospect. If you don't know how to use any of these programs it does make a good excuse to learn them and I believe that what we are talking about is simple enough for a beginer. Again I could be wrong so go easy on me.

maybe use plucker?

Thanks for the suggestions all. I'll check some out.

As far as compiling my own stuff -- I'm a developer, I'm not afraid to! But I'm completely unfamiliar with Linux. I set up an Ubuntu virtual machine to play around with and got completely and utterly confused by the developer setup instructions on maemo.org ("install scratchbox" -- great, how? which of those 20 packages? "install xephyr" -- tons of dependency failures that I don't know how to resolve. a step-by-step is greatly needed)

Also, in another thread, a Subversion port was pointed out, which provides some interesting options for syncing the reference library with a desktop computer. Although, I'm still not sure which formats and folders the N800 can natively search without additional tools being brought in.

Don't worry about installing the scratchbox by hand, just grab the vmware appliance that has ubuntu & the scratchbox pre-installed. It's at the vmware appliance site

Thanks for the pointer to the VMWare appliance. Shouldn't there be a link to that from maemo.org?

Anyway, I got it up and running and even built the "hello world" example successfully.

Now I guess I just need to figure out how serious I am about pursuing this any further...