Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

Some time ago I've posted a note about creating some custom maps for maemo-mapper from BSB nautical charts: http://talk.maemo.org/showthread.php...&highlight=bsb.

I've made a bit of development since then. A new set of scripts are written in Python, so they can be used with Windows as well. The scripts requirements are: the Python (tested with v. 2.5.2 and 2.6.6) with the Python imaging library, the GDAL tool set (tested with v. 1.6.3 and 1.7.2), optionally numpy package and pngnq tool

Here is how it's all implemented:
1) bsb2gdal.py or ozi2gdal.py -- creates geo-referenced GDAL .vrt file, optionally cropping the raster;
2) gdal4tiles.py -- creates a tile set tree directory for every chart using gdal2tiles.py from GDAL; (gdal2tiles.py also creates a couple html files, so the result can be checked with a browser).
3) tiles-merge.py -- sequentially merges a few tile sets in a single one to cover the area required;
4) tiles-opt.py -- compresses png tiles into a palleted form using pngnq tool;
5) tiles2mapper.py -- creates a sqlite3 file from a tile set tree. This file you then just copy into a maemo-mapper maps directory. You'd also need to create a reference to this file at the maemo-mapper repository settings.

For options list use "--help"

There is also kml2gdal.py script which helps geo-referencing any png or jpeg raster using Google Earth: you import a raster into the Google Earth stretch it accordingly, export it as kml file, then kml2gdal.py creates a GDAL .vrt from it.

For poi2mapper.py functionality see http://talk.maemo.org/showthread.php?p=810365

Below are the latest version of the scripts

UPDATE!

Please ignore these files. A new version of these scripts are available from here: http://code.google.com/p/tilers-tools

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

Thank you very much for the converter. Here is tiles2mappero.py script (made from your tiles2mapper.py) which can be used to produce folders suitable for mappero (former maemo-mapper). Mappero's repository/tiles settings dialogs seem to be very buggy, so this far I've failed to add new repository to it, but at least I've succeeded in replacing cached files in existing subdirectories of $HOME/MyDocs/.maps.

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

Cheers fionbio! Have you tried tiles-merge.py --gtiles. At the 1st glance it looks like it does what you needed, doesn't it?

Actually it's possible just feed these directories a http server (I tried boa), to download them into some other mapping application which is able to fetch google tiles.

BTW I understand there are a couple of ports of the original the maemo-mapper for n900: https://garage.maemo.org/frs/shownot...elease_id=3406 and http://shmuma.livejournal.com/24660.html

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

hi vadp,

this is output when i try to convert map with ozi ogf2 and tiff in folder.Filenames in .map file are the same as in folder.Any suggestion where to look for error.
Thank you,
Thomas


./ozi2gdal.py LJ_center_ortofoto.map
Traceback (most recent call last):
File "./ozi2gdal.py", line 234, in <module>
map2vrt(f)
File "./ozi2gdal.py", line 191, in map2vrt
gdalinfo=command(['gdalinfo',img_file])
File "./ozi2gdal.py", line 102, in command
process=Popen(params,stdin=PIPE, stdout=PIPE, stderr=PIPE, universal_newlines=True)
File "/usr/lib/python2.6/subprocess.py", line 623, in __init__
errread, errwrite)
File "/usr/lib/python2.6/subprocess.py", line 1141, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

Hi Tomas. Can you run it with -d switch? BTW do you have gdal binaries installed and they are in the PATH?

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

A newer version of kml2gdal.py was added to the 1st post. It has a better datum definition for the Google Earth.

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

yes , i try but same , gdal binaries are installed .

output is here:

../ozi2gdal.py -d 116-ljubljana.map
DEBUG:root:[['OziExplorer Map Data File Version 2.2'], ['116-LJUBLJANA'], ['116-LJUBLJANA.jpg'], ['1', 'Map Code', ''], ['SLOVENIJA', 'WGS 84', '0.0000', '0.0000', 'WGS 84'], ['Reserved 1'], ['Reserved 2'], ['Magnetic Variation', '', '', 'E'], ['Map Projection', 'Transverse Mercator', 'PolyCal', 'No', 'AutoCalOnly', 'No', 'BSBUseWPX', 'No'], ['Point01', 'xy', '0', '0', 'in', 'deg', '46', '7.4997', 'N', '14', '29.9322', 'E', 'grid', '', '', '', ''], ['Point02', 'xy', '4606', '6589', 'in', 'deg', '45', '59.9989', 'N', '14', '37.5532', 'E', 'grid', '', '', '', ''], ['Point03', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Point04', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Point05', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Point06', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Point07', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Point08', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Point09', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Point10', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Point11', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Point12', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Point13', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Point14', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Point15', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Point16', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Point17', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Point18', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Point19', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Point20', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Point21', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Point22', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Point23', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Point24', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Point25', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Point26', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Point27', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Point28', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Point29', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Point30', 'xy', '', '', 'in', 'deg', '', '', '', '', '', '', 'grid', '', '', '', ''], ['Projection Setup', '0.000000000', '15.000000000', '0.999900000', '500000.00', '-5000000.00', '34.000000000', '40.500000000', '', '', ''], ['Map Feature = MF ; Map Comment = MC These follow if they exist'], ['Track File = TF These follow if they exist'], ['Moving Map Parameters = MM? These follow if they exist'], ['MM0', 'Yes'], ['MMPNUM', '4'], ['MMPXY', '1', '0', '0'], ['MMPXY', '2', '4606', '0'], ['MMPXY', '3', '4606', '6589'], ['MMPXY', '4', '0', '6589'], ['MMPLL', '1', '14.498870', '46.124995'], ['MMPLL', '2', '14.625038', '46.125478'], ['MMPLL', '3', '14.625887', '45.999982'], ['MMPLL', '4', '14.500004', '45.999500'], ['MM1B', '2.116049'], ['MOP', 'Map Open Position', '0', '0'], ['IWH', 'Map Image Width/Height', '4606', '6589']]
DEBUG:root:116-ljubljana.jpg
DEBUG:root:['+proj=tmerc', '+lat_0=0.000000000', '+lon_0=15.000000000', '+k=0.999900000', '+x_0=500000.00', '+y_0=-5000000.00', '+lat_1=34.000000000', '+lat_2=40.500000000', '+datum=WGS84', '+no_defs']
DEBUG:root:proj +proj=tmerc +lat_0=0.000000000 +lon_0=15.000000000 +k=0.999900000 +x_0=500000.00 +y_0=-5000000.00 +lat_1=34.000000000 +lat_2=40.500000000 +datum=WGS84 +no_defs
DEBUG:root:[[['Point01', 'xy', '0', '0', 'in', 'deg', '46', '7.4997', 'N', '14', '29.9322', 'E', 'grid', '', '', '', ''], ['Point02', 'xy', '4606', '6589', 'in', 'deg', '45', '59.9989', 'N', '14', '37.5532', 'E', 'grid', '', '', '', ''], ['', '', '0', '6589']], [['461272.21', '109590.53'], ['471022.84', '95642.33'], ['461272.21', '95642.33']]]
Traceback (most recent call last):
File "../ozi2gdal.py", line 235, in <module>
map2vrt(f)
File "../ozi2gdal.py", line 189, in map2vrt
gcps=flatten([('-gcp', i[0][2],i[0][3],i[1][0],i[1][1]) for i in zip(refs, ref_coord)])
File "../ozi2gdal.py", line 110, in flatten
return list(itertools.chain.from_iterable(listOfLists))
AttributeError: type object 'itertools.chain' has no attribute 'from_iterable'

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

Hi Tomas, it looks like you need a more recent version of python for this: 2.6 or above.

update: I've made another version of the script, so see if it's any better for you (remove .txt from it's name)

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

Hello!
I have jpg maps with .ogf2 and .map.
At first I use ozi2gdal.py to get GDAL .vrt. After that tiles-from-gdal.py to get tiles tree with some zoom levels and html's with overlay. If I check the result in browser - it ideally fits google-map or openstreetmap.
What I can't figure out is right url format for maemo-mapper, I use:
http://myserver/maps/%0d/%d/%d.png like on openstreetmap, but in server's logs I see it tries to get other tile numers I have in tree. Zoom levels are ok, but x and y differs...
For example: my generated tree contains 8/145/177 and it's identical to oss 8/145/78 and google's x=145&y=78&z=8

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

Hi ksb! I think you can try tiles-merge.py with "--gtiles" option

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

When I add --gtiles option, it says:
Traceback (most recent call last):
File "bsb2tiles/tiles-merge.py", line 251, in <module>
for src in src_dirs:
TypeError: 'NoneType' object is not iterable
for
bsb2tiles/tiles-merge.py --gtiles riga/ resultnew/
but without --gtiles it merges trees ok!

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

Actually, I used "--gtiles" quite a while ago. Can you try this version?
You'd need to supply only a target directory then.

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

Yes, now it works, gtiles is what I need! Big thank's to You vadp!
Probably tomorrow or later, I will try to convert larger amount of maps and try to put it in sqlite.

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

Hi ksb! I wonder what s/w do you use. It's not maemo-mapper 2.x, isn't it? If so, do you know how does it stores it's map tiles?

The thing is that d/l tiles over http is a kind of tiresome

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

It's maemo mapper 2.6.2, http is to be sure it's compatible with it. Next step is to generate sqlite and put it in. And yes, download over http is very loooooong process.

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

I've uploaded newer versions of the scripts. See the 1st post here.
This should fix python 2.5 and windows compatibility

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

Hi, vadp! Can you visit n8xx.com and write a manual in Russian: how work with your scripts?

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

No problem dragonleo! I'm just making it to work better at windows

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

Thanks for the nice scripts, but I do suspect it fails at least with my ozi map data.

I've tried ozi to tile conversion with a small sample map ( 2.0 x 2.0 km 1:16000 ) from Finland and the tile hierarchy created is this.

.
./googlemaps.html
./openlayers.html
./0
./0/0
./0/0/0.png
./1
./1/0
./1/0/0.png
./1/0/1.png
./1/1
./1/1/0.png
./1/1/1.png
./tilemapresource.xml


I've enclosed the original map-file and the generated vrt-file.

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

Can you give me a link to the png as well? You'd use PM if you like.

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

Your data works fine for me. I forgot to mention these need GDAL 1.7 and above, so that could be an issue if yours is 1.6.x:

13 14 googlemaps.html openlayers.html tilemapresource.xml

./13:
4698 4699

./13/4698:
5836.png 5837.png

./13/4699:
5836.png 5837.png

./14:
9396 9397 9398

./14/9396:
11673.png 11674.png 11675.png

./14/9397:
11673.png 11674.png 11675.png

./14/9398:
11673.png 11674.png 11675.png

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

OK, that explains it all, thanks.

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

I was wrong. It worked with GDAL 1.6.3 as well. It produces the very same .vrt then.

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

Mine was 1.5.x, updated and seems to work now.

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

As discussed in a Russian forum (http://n8xx.com/topic4627-karty-dlya...brazhenii.html) one can generate a Mappero-compatible (maemo-mapper N900) tile set with

tiles-merge.py --gtiles out_map

Then

cp -RL gtiles <my card>

or

tar chzf gtiles.tgz gtiles

to move or expand to /home/user/MyDocs/.maps/<repository-name>

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

I do have some ozi maps with datum Finland Hayford, but it seems to be unsupported datum - how hard it is to add new datums to the scripts ?

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

You may try one attached.
Can you send me a couple of samples (map+raster)?

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

It seems that the calibration is off by few hundred meters. I'll try to get you the map samples.

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

Vadp
Just a quick thank you for your scripts, I am newbie and non-programmer working to put nautical charts on google earth. I am using your bsb2gdal to strip the border from rastor charts. I appreciate your hard work.

thank you once again.
tim

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

Hi tim! Thank for you nice words. I'm happy to know people found that useful. Give me a shout if you need some help.

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

At my PM janszoon is asking me for some detailed instructions. I understand quite a number of people still do not use Linux these days ;) but there is a way of doing the thing on Windows.

A guy at a Russian forum asked me for giving him instruction in Russian, so I've posted them here http://n8xx.com/post54305.html#p54305. Unfortunately I'm quite busy at the moment to prepare an English version of these. As soon as I have a bit more time I'll come back to this.

Meanwhile, perhaps Google Translate might help (although the translation is pretty much terrible):
http://translate.google.com/translat...post54305.html

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

Thanks mate. I may have to wait for you to get round to the English version as my russian is very bad, and my linux is even worse, my python is literally non exisitant.

I may be biting off more than I can chew trying to get NZ Nautical maps into Mappero, but I will try to follow the steps. Perhaps even write a noobs guide to doing. And by Noob, I mean.... What do i even do with the script, how do I access it, how do I tell it where to look and what to do etc...

Soo...Please excuse my ignorance

I'm at step one:
I have installed/extracted the programs/7zips as suggested. I have a folder full of my .kap files ready to be processed. Can I point the script to a folder, or do I need to convert each chart individually?

At the CMD I access the script with the following:
Can you please help me with the correct syntax? I would like to remove the border as suggested too.

Cheers.
(please tell me I'm not the only one who finds this daunting :eek:)

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

Did you edit bsb2tiles.bat, so the lines there look like this?
set PYTHON=<python location>
set GDAL=<gdal location>

If so, you can check if it works
The script will start, but then complain of the lack of parameters. This is fine.

Then you just type (do not hit Enter):
Then drag a .KAP file here from the Explorer's file list window, then hit Enter. You should get .vrt file.
Then (you can drag as well):
You'll get a folder with tile pyramid (so-called TMS-format). Watch for googlemaps.html there, you can fire it with a Firefox or IE.
Then:
You'll get gtiles subfolder inside the pyramid. That's what you need for the mappero (as people say ;)

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

Ok baby steps...this is probalby more a lesson in Linux than mapping but I appreciate the time / help:

How do I get from your bsb2tiles.tar.gz file (attached to original thread) to .bat ?

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

bsb2tiles.bat file attached to the Russian instructions referenced above. You would also need gdal2tiles.py from there

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

Hi Vadp.

Some progress, I got the batch file running etc and get to the step below:

I run the batch file and get a CMD window with:

Setting environment for using the GDAL and MapServer tools.
Hiding the OCI plugin library.
C:\Maps\bsb2tiles>

cd /d c:\Maps\NewZealand

C:\Maps\NewZealand>bsb2gdal.py
Usage: bsb2gdal.py [--cut] [--dest-dir=DEST_DIR] KAP_file...

bsb2gdal.py: error: No input file(s) specified

C:\Maps\NewZealand>

Yes this works

Yes I type the above then I drag the file in (add debug) and hit enter as stated but I get the following error:


Am I missing a file or similar?

Cheers.

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

I have also tried a newer version of Gdal form the russian thread. (1.7) and now I get the attached pop up error, at the same place in the process.

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

It seem the environment wasn't set properly.

Can you call 'proj' from a command prompt?

Also check the location of proj.dll. Then check the PATH environment variable( 'echo %PATH%' or even just 'set'). The location of proj and proj.dll must be among the folders listed at PATH.

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

you mean under my computer>>advanced>> environmental variables>>Path ?

Re: Custom maps for maemo-mapper from BSB, OZI charts, and uncalibrated images
 

Popup like above and then this...