View Single Post
Posts: 1,548 | Thanked: 7,510 times | Joined on Apr 2010 @ Czech Republic
#1847
Originally Posted by MikeHG View Post
Two questions:

1) Do normal map tiles 'expire'? It'd be ideal if the app could check whether tiles were up to date if they're older than a certain age and still being used, though if that isn't possible just having an option to flush them after a year or so would be good.
By default tiles never expire, but a timeout property can be set for a layer to set expiration for tiles corresponding to the layer.

Some layers that have frequently changing data include the timeout property by default and users can of course also set in themselv in the configuration file (which is in /home/nemo/.config/harbour-modrana on the Jolla).

Also an important note about how the timeout logic is currently implemented in modRana - it currently only prevents loading of expired tiles from local storage (so that dynamic map layers don't show stale data). No scrubbing of stale files has been implemented yet, even though the needed data for it (timeout value & tile timestamps) is available.

Originally Posted by MikeHG View Post
2) Don't copy-on-write filesystems have terrible performance for databases? I read older versions of BTRFS were notorious, and Sailfish has an oldish one...
According to the official BTRFS wiki it is still the case:
Files with a lot of random writes can become heavily fragmented (10000+ extents) causing trashing on HDDs and excessive multi-second spikes of CPU load on systems with an SSD or large amount a RAM.
Originally Posted by MikeHG View Post
I'm wondering whether creating a subvol without COW for the maps, or perhaps just tuning down the write frequency massively in the config file might help performance (not sure I'm brave enough to try the former - and I'm certainly not suggesting anyone else should, unless they know what they're doing...)
Note that even though BTRFS has the nocow mount option, it has serious drawbacks and gotchas:
Disable it by mounting with nodatacow. This implies nodatasum as well. COW may still happen if a snapshot is taken. However COW will still be maintained for existing files, because the COW status can be modified only for empty or newly created files.
And that assumes everything works correctly and you don't hit any of the miriad BTRFS bugs while doing this.

My suggestion would be to use a uSD card formated with a sane filesystem (EXT4) and store your tiles there.

You can achieve this either by changing the map storage path in:
/home/nemo/.config/harbour-modrana/user_config.conf
Or you can just symlink the folder modRana uses for map data storage by default to somewhere on the uSD card:
/home/nemo/.local/share/harbour-modrana/maps

Originally Posted by MikeHG View Post
Love the app, but hitting some weird problems on sqlite where it seems to refuse to load tiles it has clearly loaded before. Still, I realise it's early days yet for the port. I'll try to capture some debug output if I can.
Have you tried switching to the "files" storage method in Options->Map? That makes modRana store tiles as files in a folder structure. AFAIK unlike database files BTRFS is quite sane when handling many small files.


Originally Posted by MikeHG View Post
This is a log from when the database seems to 'lock up', apparently refusing to load tiles it has already downloaded. I'm not 100% sure how to trigger it, but generally scrolling around and zooming in and out in an area I haven't yet downloaded works reliably eventually. I've left it in that state for 1/2 hour probably, and it doesn't recover - though interestingly, it does seem to be able to load other tiles sometimes (e.g. if I zoom really far out) - it just seems to choke on certain particular areas of map, but after closing the program and relaunching it, and sometimes even just by zooming out then back in again, those areas of map will draw fine.
I have also hit this q couple of times - but as it seems to happen quite at random I have not yet been able to fully pin it down yet. But a lot of zooming & panning around seems to be a factor.

Originally Posted by MikeHG View Post
http://pastebin.com/dVuPYC4k

Quite a few examples of this:

Code:
Traceback (most recent call last):
  File "modules/gui_modules/gui_qt5/gui_qt5.py", line 302, in _selectImageProviderCB
    return self._imageProviders[providerId].getImage(imageId, requestedSize)
  File "modules/gui_modules/gui_qt5/gui_qt5.py", line 599, in getImage
    log.exception()
TypeError: exception() missing 1 required positional argument: 'msg'
2015-08-13 00:53:58,644 ERROR mod.gui.qt5: tile image provider: loading tile failed
2015-08-13 00:53:58,652 ERROR mod.gui.qt5: mainMap/mapnik/13/4094/2727
2015-08-13 00:53:58,655 ERROR mod.gui.qt5: (-1, -1)
2015-08-13 00:53:58,658 ERROR mod.gui.qt5: image loading failed, imageId: tile/mainMap/mapnik/13/4094/2727
Traceback (most recent call last):
  File "modules/gui_modules/gui_qt5/gui_qt5.py", line 579, in getImage
    download=False)
  File "modules/mod_mapTiles/mod_mapTiles.py", line 211, in getTile
    tileData = self._storeTiles.getTileData(lzxy)
  File "modules/mod_storeTiles.py", line 369, in getTileData
    return self._getTileDataFromSqlite(lzxy)
  File "modules/mod_storeTiles.py", line 385, in _getTileDataFromSqlite
    result = self.getTileFromDb(lookupConn, dbFolderPath, lzxy)
  File "modules/mod_storeTiles.py", line 455, in getTileFromDb
    (z, x, y))
sqlite3.OperationalError: database is locked
Interesting! Thanks a lot for the log - I think I can do something about the loockups now.

Originally Posted by MikeHG View Post
Side note:

Due to a bug, it seems it's currently impossible to send the logs via the Jolla mail client using gmail, unless you rename them (removing the #).

https://together.jolla.com/question/...ent-via-gmail/
Voted! (And others should too!)

Originally Posted by Shadowdog View Post
I guess there wont be any further updates for modRana for Meego!?? Is it possible to use offline map data, lets say from here http://download.geofabrik.de/ ?
Unfortunately no, at least not from me. Of course if someone from the community volunteers to move the MeeGo/Harmattan port forward I'm willing to help them get started. But I'm unfortunately far to overcommitted to work on it myself.
__________________
modRana: a flexible GPS navigation system
Mieru: a flexible manga and comic book reader
Universal Components - a solution for native looking yet component set independent QML appliactions (QtQuick Controls 2 & Silica supported as backends)

Last edited by MartinK; 2015-09-05 at 15:50.
 

The Following 7 Users Say Thank You to MartinK For This Useful Post: