[Announce] modRana: a flexible GPS navigation system - Page 95 - maemo.org

Active Topics

Is there a section to talk about Java ME? Apps, etc. (0)
to General by Kalatti - 8 hrs, 25 mins ago
Extra softwares in Sailfish using CLI, repositories, etc (117)
to SailfishOS by nieldk - 2 days, 5 hrs ago
Firefox with Leste (6)
to Maemo 7 / Leste by teroyk - 3 days, 2 hrs ago
more...

Page 95 of 207

Thread Tools

白い熊	2012-03-11 , 19:31
Posts: 451 \| Thanked: 334 times \| Joined on Sep 2009	#941

Looking great man. How should I prepare to get ready for testing on Android?

Quote & Reply |

The Following User Says Thank You to 白い熊 For This Useful Post:
MartinK

MartinK	2012-03-11 , 19:40
Posts: 1,548 \| Thanked: 7,510 times \| Joined on Apr 2010 @ Czech Republic	#942

Originally Posted by 白い熊

Looking great man. How should I prepare to get ready for testing on Android?

You can try to to install Qt & PySide by following THP's guide.

Not sure about Qt Components availability though (you might need to compile them yourself

__________________
modRana: a flexible GPS navigation system
Mieru: a flexible manga and comic book reader
Universal Components - a solution for native looking yet component set independent QML appliactions (QtQuick Controls 2 & Silica supported as backends)

Quote & Reply |

The Following User Says Thank You to MartinK For This Useful Post:
zimon

MartinK	2012-03-11 , 20:47
Posts: 1,548 \| Thanked: 7,510 times \| Joined on Apr 2010 @ Czech Republic	#943

Originally Posted by Thalass

One thing i've noticed lately: If you try and use open cycle maps, the tiles all fail to download - i get the 'due to network error' tiles instead. Have OCM changed their address or something? It shouldn't be my end as this is both on wifi and 3g, and all the other maps work. Has anyone had this problem?

I know about this, you can try to use this workaround until it's fixed. I basically just need to update the map definition file & make sure the previous file is replaced by the new one (and rename & keep the old one as the user might have modified it in the meantime).

Originally Posted by Joseph9560

I saved a POI with name name "Tuscany Suites & Casino". Funny part is neither the name of the POI is listed under the POI (or say an item with blank name is listed), nor the description is shown in the POI detail. I can see that its because of "&".

Weird, I'll check it out.

Originally Posted by Joseph9560

Also while saving a route as a tracklog it is not saving the correct date. Its currently 10:51 3/7/2012 (GMT-8) but the last modified date is something like 6:48:34 8.3.2012, surely some time zone calculation error as far as I can see.

I'll check it out.

Originally Posted by Joseph9560

Also searching for something like "Tuscany Suites Hotel & Casino, 255 E. Flamingo Road, Las Vegas, Nevada 89169 " doesn't seems to return proper address. Am I right to say that search of address is performed via Google irrespective of tile provider being used? Google earth provides proper result (or results with proper one) in pc.

Yep, Google geocoder is currently used for address search. But I'd like to add other geocoders in the near future - Nominatim, Geonames, etc. Weird that it works with Google Earth - either it uses its own different private API or there are some encoding issues with the ampersand in the string ?

Quote & Reply |

The Following User Says Thank You to MartinK For This Useful Post:
Joseph9560

jafd	2012-03-12 , 10:08
Posts: 29 \| Thanked: 26 times \| Joined on Oct 2009	#944

I think I'm gonna need some help with this one.

I want to have a detailed map of a few cities pre-cached. I go to the city in question, then

1) Download -> around here -> 20 km
2) down max, up max (by the way, what do "down" and "up" mean? Does "down" mean zoom in, or zoom out?)
3) It works out 23k tiles
4) I zoom around the city and it tells me tiles must be downloaded. I don't know what area was actually downloaded.

I believe I'm wrong somewhere, but don't know where.

It would be of great help actually to have those settings illustrated. Say, here's a modrana window, scale is so and so, and the area to be covered is highlighted in blue for each mode.

By the way, sqlite interactions are prone to segfaults on amd64 python 2.7. It is for sure a regression on Python/SQLite side, but there needs to be a way to work it around.

Also, if anyone has set up a local tile server for modrana to use (say on a local network), could you please share how you did it? The openstreetmap wiki has the notorious open-source documentation problem, where old and outdated docs are intermixed with the current ones in an amusing enough way to leave one occupied for a week if not more.

Quote & Reply |

The Following User Says Thank You to jafd For This Useful Post:
MartinK

MartinK	2012-03-12 , 11:07
Posts: 1,548 \| Thanked: 7,510 times \| Joined on Apr 2010 @ Czech Republic	#945

Originally Posted by jafd

I think I'm gonna need some help with this one.

I want to have a detailed map of a few cities pre-cached. I go to the city in question, then

1) Download -> around here -> 20 km
2) down max, up max (by the way, what do "down" and "up" mean? Does "down" mean zoom in, or zoom out?)

It means the number of zoom levels "up" and "down" from the current one. But good point, it might not be a bad idea to rename it to zoom in + zoom out.

Originally Posted by jafd

3) It works out 23k tiles
4) I zoom around the city and it tells me tiles must be downloaded. I don't know what area was actually downloaded.

I believe I'm wrong somewhere, but don't know where.

What version of modRana do you have ?
Also, what tile layer are you trying to download ? Some tile providers (eq. Google) are unfortunatelly notoriously unfriendly to batch downloads. Also, were there some failed tiles during download or some error messages on stdout ? Please also check out if you have libmagic installed - modRana is using it to verify if the "tiles" it downloads are actually map images and not html error documents complaining about batch tile downloads returned by the tile provider.

Originally Posted by jafd

It would be of great help actually to have those settings illustrated. Say, here's a modrana window, scale is so and so, and the area to be covered is highlighted in blue for each mode.

I would like to include a more interactive tile download mode in the new QML GUI. Something like dragging a rectangle aver an area, then selecting zoom level range with some sliders & starting the download.

Originally Posted by jafd

By the way, sqlite interactions are prone to segfaults on amd64 python 2.7. It is for sure a regression on Python/SQLite side, but there needs to be a way to work it around.

Yeah I had quite a few issues already with sqlite with Python 2.5 that is present on the N900 (mostly related to broken threaded access to the database). Any idea what approximate sequence of events triggers the segfault ? I have a similar setup (Core i5 + Python 2.7) so I can try to reproduce that.

Originally Posted by jafd

Also, if anyone has set up a local tile server for modrana to use (say on a local network), could you please share how you did it? The openstreetmap wiki has the notorious open-source documentation problem, where old and outdated docs are intermixed with the current ones in an amusing enough way to leave one occupied for a week if not more.

I think Woody14619 tried to get a tileserver running on the N900 a while ago, don't know how far he got though. Also, there is a realtime Python map renderer called Kothic that is somehow using Mapnik - they might have some info about setting it up.

Quote & Reply |

jafd	2012-03-12 , 11:27
Posts: 29 \| Thanked: 26 times \| Joined on Oct 2009	#946

It means the number of zoom levels "up" and "down"

Like a bird flying up and down, in other words?

What version of modRana do you have ?

0.29-2

Also, what tile layer are you trying to download ?

openstreetmap; I also want to set up my own local mapnik to relieve some load on OSM servers.

I think Woody14619 tried to get a tileserver running on the N900 a while ago, don't know how far he got though.

No no no, I want Mapnik on a PC on the same LAN as N900. The thing is, I want a cache of a whole country - it's HUGE, N900 would just die from choking while processing it.

Another way would be a batch rendering with Mapnik (and not mucking around with mod_tile and friends) and mass import into SQLite, but the link to a Perl script posted a few pages ago is dead.

Any idea what approximate sequence of events triggers the segfault ? I have a similar setup (Core i5 + Python 2.7) so I can try to reproduce that.

Just try to download a 40 km area for all zoom levels around Warsaw, Poland (it estimates to ~90k of tiles) and it eventually dies. Using Python 2.7.3rc1 on Ubuntu 12.04 beta. Curious thing, on N900 it doesn't die.

I would like to include a more interactive tile download mode in the new QML GUI. Something like dragging a rectangle aver an area, then selecting zoom level range with some sliders & starting the download.

For now, a simple example would suffice. Like, here's what would be downloaded in "Around here" mode (where is "here"? GPS location or visible window center?), here's what would be downloaded in "Around view" mode (portion outside screen? Visible window's center?), here's what would get downloaded around route.

Yeah I had quite a few issues already with sqlite with Python 2.5 that is present on the N900 (mostly related to broken threaded access to the database).

From the experiences I've had with SQLite, it's better to have just a single writer thread, and queue up things you want to insert/update there. I have tried to do that myself, but got lost in the code - seems like "download" and "store" parts are not quite separate.

Last edited by jafd; 2012-03-12 at 11:40. Reason: filled in more details

Quote & Reply |

The Following User Says Thank You to jafd For This Useful Post:
MartinK

MartinK	2012-03-12 , 19:46
Posts: 1,548 \| Thanked: 7,510 times \| Joined on Apr 2010 @ Czech Republic	#947

Originally Posted by jafd

Like a bird flying up and down, in other words?

Yep, like that.

Originally Posted by jafd

No no no, I want Mapnik on a PC on the same LAN as N900. The thing is, I want a cache of a whole country - it's HUGE, N900 would just die from choking while processing it.

Yeah, I got that. But the only-one I know with some Mapnik experience is Woody.

Originally Posted by jafd

Another way would be a batch rendering with Mapnik (and not mucking around with mod_tile and friends) and mass import into SQLite, but the link to a Perl script posted a few pages ago is dead.

Well, that quite sucks - I should have mirrored the script... Well, I'll PM the author, maybe he can re-upload it.

Originally Posted by jafd

Like, here's what would be downloaded in "Around here" mode (where is "here"? GPS location or visible window center?), here's what would be downloaded in "Around view" mode (portion outside screen? Visible window's center?), here's what would get downloaded around route.

Yes, unfortunately modRana documentation is currently quite lacking... Would you be perhaps willing to write a short guide/manaul ?

Concerning the labels:
around here = download is centered around current GPS coordinates (renaming this to "around current location or "around coordinates" might make this less confusing)
around view = download is centered around a point in the middle of the currently visible map (probably should be also renamed to "around map center")

Originally Posted by jafd

From the experiences I've had with SQLite, it's better to have just a single writer thread, and queue up things you want to insert/update there.

This is roughly how it should work right now - there is a worker thread that processes tiles from a queue. Tile download threads - from automatic tile download or from batch download - supply tiles to this queue. Once the queue fills up (it can hold 50 tiles at once by default) all tiles are flushed to the database in a single transaction by the worker thread. At shutdown, the worker is notified and flushes any remaining tiles from the queue to the database.

Tile loading works in a similar manner - there is a tile loading worker thread that processes tile loading requests from a queue and loads tiles from database to a in-memory cache, guarded by a mutex. Not sure if the tile loading thread is really needed on on newer Python versions though.

Originally Posted by jafd

I have tried to do that myself, but got lost in the code - seems like "download" and "store" parts are not quite separate.

Yeah, it might be a bit convoluted.

It's structure basically like this:
mod_mapTiles - draws the map and automatically downloads visible tiles
mod_mapData - handles batch tile download
mod_storeTiles - handles tile storage, both to files and to sqlite, contains both worker threads mentioned above and all database handling logic

BTW, there is a description of the tile database structure on modRana project wiki.

Quote & Reply |

The Following User Says Thank You to MartinK For This Useful Post:
don_falcone

jafd	2012-03-12 , 21:01
Posts: 29 \| Thanked: 26 times \| Joined on Oct 2009	#948

BTW, there is a description of the tile database structure on modRana project wiki.

Yeah, I've seen that. If the blob actually contains tile files verbatim, I think it should be easy to hack together a little Python program from looking at how you store tiles, or even to modify generate_tiles.py from Mapnik to create the SQLite databases right from the start.

Some pitfalls:

1) when populating the data for the first time, I forgot to enable hstore and use -k for osm2pgsql, and was badly screwed in terms of number of retries.

2) if you want to have not one country, but, say, two bordering countries, and hope to reduce processing time by downloading corresponding datasets from, say, download-fabrik.de — think again. Right at the border, there will be duplicate nodes on those maps. They all also contain some duplicate nodes needed to create overviews, at least capitals of other countries are mapped.

This means that importing countries one by one will fail on the second country. I don't know how to resolve this, but the easier for a human (but not for the machine) way is to download the whole continent, import it and render the bounding boxes as you like.

3) the rendering time is, too, rather slow. I wonder if using PyPy would speed it up a bit. UPD It won't, unless by some miracle PyPy supports Boost bindings.

Last edited by jafd; 2012-03-12 at 21:13.

Quote & Reply |

The Following User Says Thank You to jafd For This Useful Post:
MartinK

MartinK	2012-03-13 , 14:40
Posts: 1,548 \| Thanked: 7,510 times \| Joined on Apr 2010 @ Czech Republic	#949

Beermad replied to my PM. He is searching for the script and it should be again online soon.

Quote & Reply |

jafd	2012-03-13 , 15:16
Posts: 29 \| Thanked: 26 times \| Joined on Oct 2009	#950

That's good, I already have gobs of tiles to import.

In short: don't muck around with osm2pgsql on a regular machine if aiming at a whole country: it would either be very long, or it will fail after quite a while trying, because you don't have enough RAM.

Use imposm, it's cool, it works in reasonable amounts of time, its documentation is good.

Then you will have the problem that osm.xml shipped with Mapnik doesn't work. This is because it's tuned for osm2pgsql, and imposm uses a totally different schema.

By the way, I don't know if OSM-Bright's default style is broken or it is the way in which imposm imports its data, but with this combination you won't get city boundaries. It might suck.

I have installed carto, went to OSM-Bright, converted their stylesheets to Mapnik's with carto, and now tiles are being rendered.

The conclusion: OSM tools are a mess.
Another conclusion: go buy a paper map or an unlimited data plan if you really value your time, and don't ever go places with poor cellular coverage.
Yet another conclusion: it's a nice idea to pre-render some large chunks of the world at all zoom levels and distribute the resulting SQLite files. Shame no one has done it yet.

Last edited by jafd; 2012-03-13 at 15:29.