N800 - metalayer-crawler using 70-150+MB Memory

[penguinbait]

Nokia-N800-51:~# dpkg -S /usr/bin/metalayer-crawler
metalayer-crawler0: /usr/bin/metalayer-crawler


This process I have found twice using 70+MB Memory. I have 180MB swap setup and it was using over 150MB before I killed it

I appears to have something to do with the media player, according to /etc/init.d/metalayer-crawler0

I added exit 0 to the top of the init script, seems ok after reboot.

My guess is it is used to find media mp3,video on mmc cards. This is just a guess, but its not running on my N800 anymore.

Anyone know what this is actually for? Anyone else seen this issue also?

[penguinbait]

Anyone else able to confirm this problem?

[midiwall]

I may have this issue... using top I see 8 instances of metalayer-crawler running each claiming 3.2% of RAM. What are you using to see mem and swap usage? top on the N800 is very stripped down.

One thing... /usr/bin/metalayer-crawler will run in the foreground ("-F") and has a debug mode ("-d"). It could help to see what it's really doing.

[penguinbait]

"free" will tell you how much is used. There is a tool in the application 2006 catalog

mpv
http://www.newlc.com/IMG/deb/mpv_0.5_armel.deb

Yuo can see each process and how much memory is used

[midiwall]

Ahh... I knew about free, and I saw you mention "mpv" in the other thread, but I didn't see it in the apps catalog.

Thank you!

[midiwall]

Got it. :)

For those looking over my/our shoulder, mpv also needs the libglade v2 library which can be had with: apt-get install libglade2-0, or kick into RED PILL mode and install it from AppsManager.

[penguinbait]

Any luck? I am curious if that thing is freaking out for anyone else. I have about 1000 files on my mmc card, none of which are media related.

[midiwall]

Kinda... Right now I have two instances of it running. But, they're only taking 4M each (which still seems like a LOT).

When I started into this thread, I had 8 instances running, but I was able to watch them die off (through top).

My gut feel is that you're right - they're freaking out trying to parse your MMC card. It's one of those states where it doesn't know that it can't get anything from the file until it looks.

I'm gonna drop a bunch of random files on my card and see if I can get the crawler into a tizzy.

More in a bit.

[midiwall]

update...

I have 1128 files now in a folder on an MMC card. All of them have size (at least >4k).

I still "only" have two copies of metalayer-crawler running, each occupying 4M of RAM.

If I: /usr/bin/metalayer-crawler -F -d -c /media/mmc2/_junk

mpv shows two more instances, both are using 2M.

If I kill ALL of them (so, 4) then two will auto-restart, each taking 2M.


I'll keep playing.

[penguinbait]

Who knows maybe it was something I had installed that hosed it? I am again not sure what it does, but I found it twice on my system running 70+ mb, the second time it was very unresponsive and slow, but functional, it was using most of my 128mb swap partition + all the internal memory. If I had not had swap, it would have rebooted.

[midiwall]

I thought I'd update this one more time...

I fired up the media player at home last night and pointed it to about 9000 tracks that I have ripped on a WMC server. While it was slower than snails in molasses at showing results, and MetaCrawler seemed to be working double shifts, it didn't burn memory.


One thing that I do notice happening a LOT is a TON of maemo-launcher tasks running. Some take a meg, some take 4meg, a couple take 16meg. I think there are 10 of them running on my box right now, with the only thing running being the desktop and mpv.

[penguinbait]

/opt # find /data/ | wc -l
45821
/opt #

I forgot, I also have 45000+ files for maemoo-mapper

[midiwall]

ahhHAH! :)

[pycage]

I have also wondered about these lots of maemo-launcher processes. It turned out that maemo-launcher is a program for launching other programs faster (has to do with faster dynamic library linking, I suppose).

My guess is that maemo-launcher is a program "prelinked" with the common maemo libraries.
The launcher then loads the other program into memory and thus skips the time-consuming step of dynamic linking which you usually find on Linux-boxes.

So, actually, every maemo-launcher process you see is just an application you have started. There's nothing to worry about maemo-launcher. :)

[pycage]

I should also mention that not every process you see is a program on its own. If a program is "threaded", every thread is listed as a process as well, but the RAM is only consumed once.

[midiwall]

My guess is that maemo-launcher is a program "prelinked" with the common maemo libraries. The launcher then loads the other program into memory and thus skips the time-consuming step of dynamic linking which you usually find on Linux-boxes.yeup, understood.


So, actually, every maemo-launcher process you see is just an application you have started. There's nothing to worry about maemo-launcher. :)Yeup, but my concern is based on how I can reboot; load browser, mplayer and maybe VNC; then close everything, sit at the desktop for quite a while, and _still_ find all the launcher instances in there. I woulda' suspected them to go away as I kill off the hosted apps.


I should also mention that not every process you see is a program on its own. If a program is "threaded", every thread is listed as a process as well, but the RAM is only consumed once.Yeup, and that's something that I've actually appreciated about Linux. It helps in debugging. :)

[Xyzo]

Hi!
I also encounter this metalayer-crawler problem with my N800. Using the "top" command I can see that it is using 80~85% of the CPU! The N800 has just been switched on and I haven't started using any apps yet... What's the problem with this daemon??? Any clue?
TIA.

[penguinbait]

I never got anywhere, I disabled it and never looked back, no functionality loss, that I am aware of???

:confused:

[Xyzo]

I've just discovered bug #978 and I've applied the patch (new variable added to /etc/init.d/metalayer-crawler0): it works like a charm.
Any feedback from Nokia about this? It would be interested to have their point of view about this...

[gammer]

Thanks for the hint. metalayer-crawler was a big problem for me since I "ln -s /home/user home" in /home/user/MyDocs and the crawler then crawled into indefinite depth... Now I know how to prevent this by adapting the fixed script! Nevertheless it should not follow links, shouldn't it?

[thebeck]

My problem is I want to keep some mp3s on both cards in case one becomes corrupt, I can still listen to my music. The problem is if you have the same album on both cards it will show twice in media player so when you play the album it will play each track twice.

Is there any way to white list or black the card and or directories that get crawled?

Would it not make more sense to only crawl your media directory so the crawler doesn't choke on everything else?

Thanks.

[ed: grammar]