maemo.org - Talk

maemo.org - Talk (https://talk.maemo.org/index.php)
-   Brainstorm (https://talk.maemo.org/forumdisplay.php?f=47)
-   -   App for vector renderer maps, using OpenGL ES 2.0 (https://talk.maemo.org/showthread.php?t=93358)

gianko 2014-06-19 11:14

App for vector renderer maps, using OpenGL ES 2.0
 
Is it possible to port on Maemo5, Harmattan, and/or SailfishOS the Mapbox GL project?

it's a on-device vector renderer, which uses OpenGL ES 2.0 technology for pixel-perfect map design, from antialiased fonts to polygon blurring, all hardware-accelerated and optimized for mobile devices — and all on the fly.

In the github page it says: An OpenGL renderer for Mapbox vector tiles, implemented in C++11, currently targeting iOS, OS X, and Ubuntu Linux.

Can compile for our platforms?

taixzo 2014-06-19 12:40

Re: App for vector renderer maps, using OpenGL ES 2.0
 
I've already started trying to compile Mapbox for harmattan. So far my issue has been that it requires gcc 4.8, but the version in the Harmattan repos is 4.4. Has anyone been successful in installing a newer gcc/g++?

MartinK 2014-06-19 21:02

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Thanks for letting us know, it looks indeed VERY interesting! :) While it works, I would say it is in quite an early stage of development, but is definitely moving in the right direction. Something like this was long overdo so it is good to see someone finally doing this.

A couple of my initial observations:

  • based on the videos and the demo on the website the performance looks very good !
  • it looks like it uses vector tiles and does some caching of them locally
  • it seems to do only basic map operations (rendering, zooming, map rotation) but does them in a pretty robust way
  • there seem to be support for styles
  • haven't seen any more advanced functionality like POI, route display, overlay with tile data, etc.
  • doesn't seem to be documented in any measurable way aside from basic build instructions
  • and even the build instructions seem to be pretty ad-hoc, not to say fragile - they are Ubuntu-centric and expect to have one specific version (4.8) of gcc, otherwise you are out of luck :) (I have gcc 4.9 on Fedora 20 and haven't yet found a way how to force it to use it)
  • the build system also looks pretty fragile - it is basically downloading & compiling some custom version of Mapnik and other libraries
  • haven't seen anything resembling packaging anywhere so far :)
  • so all in all while it is already remarkable what it can do quite a lot of work is needed to make it more robust and usable in real environment

The main question is - how do you actually use this to make a proper navigation app ? Currently it is basically just a demo that sets up a window and an OpenGL context that shows a map in it. It is open source so you could just take it and build your own application on top of it. But this would be rather tedious, as without using a proper GUI toolkit you would have to implement all the needed GUI primitives yourself, meaning that implementing any non trivial functionality would be quite a lot of work.

So basically what this needs to be actually useful is some sort of GUI toolkit integration. In this regard either Qt (as a QtQuick element or/and a QWidget) or SDL look like the best candidates. Once you have the integration, you can start building real apps with a really nice vector maps. :)

This is related to the question of how modRana can make use of the Mapbox map rendering widget.

ModRana currently has two main GUIs - the GTK2 GUI used mainly on the N900, the Qt 5 GUI used mainly on the Jolla + the semi inactive Qt 4 GUI used on Harmattan. If someone makes a QtQuick 2 element based on the Mapbox renderer (which I would say is quite likely to happen), modRana should be able to use it to render the map layer with relatively minor changes.

And if someone is crazy enough to both build it for Fremantle & to somehow integrate it with GTK2 (both are in my opinion unfortunately quite hard & unlikely to be done), it might even work with the GTK GUI with some modifications. :)

MartinK 2015-06-08 20:33

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Quote:

Originally Posted by otsaloma (Post 1472991)

Cool! Looks like it that it took them only 8 months to implement my suggestion! :)

But anyway:

Who will be the first to get it running on Sailfish OS and Fremantle ? ;-)

willwf 2016-05-05 08:08

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Thank you verry much
sbobet news

pichlo 2016-05-05 08:27

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Quote:

Originally Posted by MartinK (Post 1473023)
Who will be the first to get it running on Sailfish OS and Fremantle ? ;-)

By the looks of it, no one yet :)

m4r0v3r 2016-05-05 08:58

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Does fremantle have gcc 4.8?

freemangordon 2016-05-05 13:11

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Quote:

Originally Posted by m4r0v3r (Post 1505055)
Does fremantle have gcc 4.8?

AFAIK no, 4.7.2 is the latest available

szopin 2016-05-05 15:01

Re: App for vector renderer maps, using OpenGL ES 2.0
 
On device yeah:
https://talk.maemo.org/showthread.php?t=90542
even 5.3.0:
http://talk.maemo.org/archive/index.php/t-96420.html

MartinK 2016-05-10 18:54

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Quote:

Originally Posted by otsaloma (Post 1505337)
Qt Framework officially supported

"We are committed to Qt and Mapbox GL Native now welcomes it as a first-class platform. Mapbox Qt SDK provides QMapboxGL and QQuickMapboxGL."

https://www.mapbox.com/blog/qt-framework-support/

https://imgur.com/sq3ZVb5.gif

Well, finally! :) Took them long enough considering QML support really was a no-brainer since day one...

javispedro 2016-06-09 13:58

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Have to admit that I really like it much better than all the Mapnik styles I've tried. Is this https://www.mapbox.com/maps/outdoors/ OSM, or are they using some additional data source?

Looks too good to me to be OSM.

/me curses at the stupid build system.

I am currently toying with the Bluetooth protocol on a Garmin Fenix I got secondhand and I would enjoy something like this to render my little walks :)

MartinK 2016-06-09 15:40

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Quote:

Originally Posted by javispedro (Post 1507323)
Have to admit that I really like it much better than all the Mapnik styles I've tried. Is this https://www.mapbox.com/maps/outdoors/ OSM, or are they using some additional data source?

Looks too good to me to be OSM.

I've checked an are I have mapped for OSM and I'm pretty certain it is just filtered & styled OSM data with some height model for contours (SRTM or the other infrared image based one ?).

Quote:

Originally Posted by javispedro (Post 1507323)
/me curses at the stupid build system.

Are they still using that NPM based abomination ?

javispedro 2016-06-09 18:13

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Quote:

Originally Posted by MartinK (Post 1507327)
I've checked an are I have mapped for OSM and I'm pretty certain it is just filtered & styled OSM data with some height model for contours (SRTM or the other infrared image based one ?).

Technically, if they filtered the data somehow, they are forced to give it back with the same license, right? That's my understanding of http://www.openstreetmap.org/copyright

My perusal of the source code hasn't yet found exactly how are you supposed to feed actual map data to it.

Quote:

Originally Posted by MartinK (Post 1507327)
Are they still using that NPM based abomination ?

Seems like. But the configure-like script they ship depends on some kind of package manager they made called Mason , and for some reason it actually tries to install a bunch of libraries I already have (from a random website!).

I suspect It'd be better+easier to setup a Ubuntu VM to test it around (and see if I can replicate the output I get from their website).

otsaloma 2016-06-09 19:08

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Quote:

Originally Posted by MartinK (Post 1507327)
I've checked an are I have mapped for OSM and I'm pretty certain it is just filtered & styled OSM data with some height model for contours

Yep. It's OSM + various data sources for height.

https://www.mapbox.com/about/maps/#data-sources

Quote:

Originally Posted by javispedro (Post 1507334)
My perusal of the source code hasn't yet found exactly how are you supposed to feed actual map data to it.

The Mapbox Qt SDK is likely to be bound to Mapbox APIs, but Mapbox releases most of what they do as free software and their vector tiles are an open standard. I don't know what the whole stack to render vector maps is, but see e.g. the list of Mapbox vector tile spec implementations.

rinigus 2017-09-16 09:49

Re: App for vector renderer maps, using OpenGL ES 2.0
 
I have managed to compile Mapbox GL Native for SFOS and tested it with the small app. All seems to be working as expected. Even in online mode, its fast, allows smooth transitions and so on.

SFOS packaging: Compilation relies on recent gcc (using 6.4.0, https://build.merproject.org/package...oolbox/opt-gcc). The compiled Qt version of the library is available at https://build.merproject.org/package...maps/qmapboxgl on the basis of https://github.com/rinigus/pkg-mapbox-gl-native .

QML integration: At present, Mapbox GL Native development is targeting Qt/QML through QtLocation 5.9 plugin. As a result, direct interface with QML was removed in the beginning of March 2017.

I have made a small project that would allow us to interface Mapbox GL through QML at https://github.com/rinigus/mapbox-gl-qml . Idea is to compose the interface that can be used by mapping applications for showing and interacting with the maps. Interface between OpenGL and QML is based on the same approach as used in the current implementation for QtLocation. This looks to be compatible with Silica as well, as much as I tested.

Small SFOS app is available at https://github.com/rinigus/mapbox-demo-sfos and shows how to integrate developed QML interface into the app as well as the integration with gcc-6.4.0.

I am letting you know about the development early, so we can discuss the ways we can integrate it with the available map applications (Poor Maps, modRana, Maep, and others). There is plenty to do, but all seems to be doable at this stage and shouldn't take a while before we'll get the map applications based on OpenGL rendering. Ideally, we should keep the same source code trees for tile-based and OpenGL map applications.

Zeta 2017-09-16 13:04

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Quote:

Originally Posted by rinigus (Post 1534466)
I have managed to compile Mapbox GL Native for SFOS and tested it with the small app. All seems to be working as expected. Even in online mode, its fast, allows smooth transitions and so on.

SFOS packaging: Compilation relies on recent gcc (using 6.4.0, https://build.merproject.org/package...oolbox/opt-gcc). The compiled Qt version of the library is available at https://build.merproject.org/package...maps/qmapboxgl on the basis of https://github.com/rinigus/pkg-mapbox-gl-native .

QML integration: At present, Mapbox GL Native development is targeting Qt/QML through QtLocation 5.9 plugin. As a result, direct interface with QML was removed in the beginning of March 2017.

I have made a small project that would allow us to interface Mapbox GL through QML at https://github.com/rinigus/mapbox-gl-qml . Idea is to compose the interface that can be used by mapping applications for showing and interacting with the maps. Interface between OpenGL and QML is based on the same approach as used in the current implementation for QtLocation. This looks to be compatible with Silica as well, as much as I tested.

Small SFOS app is available at https://github.com/rinigus/mapbox-demo-sfos and shows how to integrate developed QML interface into the app as well as the integration with gcc-6.4.0.

I am letting you know about the development early, so we can discuss the ways we can integrate it with the available map applications (Poor Maps, modRana, Maep, and others). There is plenty to do, but all seems to be doable at this stage and shouldn't take a while before we'll get the map applications based on OpenGL rendering. Ideally, we should keep the same source code trees for tile-based and OpenGL map applications.

:eek:
You never stop !:D

Good work as usual !
I imagine you want to put the OpenGL rendering part inside the maps applications, and changing the server to also serve vector maps ? Would you also want to use OpenGL to render raster map (for the apps not supporting vector) on the server side ?

Seeing the progress made in QtLocation 5.9 (http://blog.qt.io/blog/2017/09/04/qtlocation-5-9/), do you think backporting the whole QtLocation to Qt5.6 would be possible and not too much work, in order to stay as compatible as possible with those future improvements ?

On my side, I am working (albeit a hundred times slower than you) on a stripped down HUD like interface for another target (Odroid C2). So far, I used OSRM (Open Source Routing Machine) as it was the first of all the engines I got to compile (OSM has a lot of dependencies in comparison...), and has an embedded server.
I played with its debug vector tile output for some tests (http://project-osrm.org/docs/v5.7.0/api/#tile-service).
I decoded them manually with QPainter so far, as they are only containing roads, not the full maps details as MapBox can render), OpenGL is my next target to allow tilting the maps to show the horizon, like you would have in other car navigation systems (like old Here Maps : http://www.geeky-gadgets.com/wp-cont...-here-maps.jpg).
You are making me think about how to move this to something that could be shared with Sailfish, despite I would have trouble to keep up with your pace, and not having Silica and other Sailfish specific components...

rinigus 2017-09-16 14:29

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Quote:

Originally Posted by Zeta (Post 1534474)
I imagine you want to put the OpenGL rendering part inside the maps applications, and changing the server to also serve vector maps ? Would you also want to use OpenGL to render raster map (for the apps not supporting vector) on the server side ?

I would start with getting QML component ready to be incorporated into the clients. It should give the best results when clients are rendering it directly. For offline, I would have to get the files to the device. I was not planning to support this rendering plugin in the server, at least not in the beginning. For those we have Mapnik already. Its maybe more productive to support the best case scenario and later adjust the plans.

Quote:

Originally Posted by Zeta (Post 1534474)
Seeing the progress made in QtLocation 5.9 (http://blog.qt.io/blog/2017/09/04/qtlocation-5-9/), do you think backporting the whole QtLocation to Qt5.6 would be possible and not too much work, in order to stay as compatible as possible with those future improvements ?

I tried once to compile QtLocation 5.9 against Qt 5.7 that I have on my desktop, but that failed on some internal Qt class (classes that they use for 'private' implementation). Didn't tried it anymore.

However, as far as I can see, much of it should be supported by Mapbox GL. Not sure about videos played on top of the map, but tilting & rotating is all there.

Quote:

Originally Posted by Zeta (Post 1534474)
On my side, I am working (albeit a hundred times slower than you) on a stripped down HUD like interface for another target (Odroid C2). So far, I used OSRM (Open Source Routing Machine) as it was the first of all the engines I got to compile (OSM has a lot of dependencies in comparison...), and has an embedded server.
I played with its debug vector tile output for some tests (http://project-osrm.org/docs/v5.7.0/api/#tile-service).
I decoded them manually with QPainter so far, as they are only containing roads, not the full maps details as MapBox can render), OpenGL is my next target to allow tilting the maps to show the horizon, like you would have in other car navigation systems (like old Here Maps : http://www.geeky-gadgets.com/wp-cont...-here-maps.jpg).
You are making me think about how to move this to something that could be shared with Sailfish, despite I would have trouble to keep up with your pace, and not having Silica and other Sailfish specific components...

If you can get Qt 5.9, that's probably the best way to go. The QML component that I want to develop as a wrapper should work on regular Qt - I am writing/testing it on regular Linux. So, that we can share. Same with the server, but it does have rather long list of dependencies indeed (that's a lot of effort to get it working).

Zeta 2017-09-16 15:34

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Quote:

Originally Posted by rinigus (Post 1534481)
I tried once to compile QtLocation 5.9 against Qt 5.7 that I have on my desktop, but that failed on some internal Qt class (classes that they use for 'private' implementation). Didn't tried it anymore.

That was what I was afraid. Too bad.

Quote:

Originally Posted by rinigus (Post 1534481)
However, as far as I can see, much of it should be supported by Mapbox GL. Not sure about videos played on top of the map, but tilting & rotating is all there.

I don't see a use case for embedding videos, I was thinking more of the MapPolyline and other types to render geometry on top of the map, like the route to follow or points of interest (like gas station in car mode). Not sure how easy it is to do with an older Qt.
I took a look at the MapBox GL before, but it is a huge piece of software, and I didn't understand all the subtleties of what it can do, and how to integrate it correctly in an application. I'll take a look again.

Can you describe your linux setup to see if I can replicate it here and maybe help ?

rinigus 2017-09-16 16:43

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Quote:

Originally Posted by Zeta (Post 1534486)
That was what I was afraid. Too bad.

I don't see a use case for embedding videos, I was thinking more of the MapPolyline and other types to render geometry on top of the map, like the route to follow or points of interest (like gas station in car mode). Not sure how easy it is to do with an older Qt.
I took a look at the MapBox GL before, but it is a huge piece of software, and I didn't understand all the subtleties of what it can do, and how to integrate it correctly in an application. I'll take a look again.

Can you describe your linux setup to see if I can replicate it here and maybe help ?

I haven't tried yet, but it seems to be easy to add routes, POIs and other objects in MapBox GL. As far as I understood, you basically add new geo layers in JSON and tell it how to render it. From your link to the blog post I understand why they made rather complicated API for QML in Qt 5.9 - they want to have it compatible with all providers.
With interfacing Mapbox GL directly, we could make a thin layer around https://github.com/mapbox/mapbox-gl-.../qmapboxgl.hpp which seems to me significantly easier.

For start, you could read QtLocation Mapbox GL Plugin API. That should give background info. Then we can test with the demo app how it all works under Linux.

Linux (and SFOS) setup actually takes not the master branch of mapbox-gl-native, but qt-staging. This is a branch followed by QtLocation plugin and it has a sane Qt build system (no need for node.js!). So, Linux setup that should work (I have a bit different, but its probably for historical reasons):

clone https://github.com/rinigus/pkg-mapbox-gl-native (recursively)

use included mapbox-gl-native-lib.pro to build the library by moving it into mapbox-gl-native folder, making dir build, and running qmake5 against mapbox-gl-native-lib.pro. That should let you build library with make

If you wish, you could install library. Otherwise, use it from that build dir (its static lib, so all should be fine).

clone https://github.com/rinigus/mapbox-gl-qml and open the project under app subfolder

Correct the path for qmapboxgl include and library dir (or even better - make symlinks that would lead to the same paths as used in the project file).

Register as a developer at mapbox.com for getting access token.

Add access token either into environment, into main() as

Code:

qputenv("MAPBOX_ACCESS_TOKEN", "HERE GOES THE TOKEN");
before registration of qml type.

Compile and run it.

Let me know if there is any trouble. I'll continue meanwhile adding QML methods. Should be all there soon, I hope.

Zeta 2017-09-16 22:17

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Quote:

Originally Posted by rinigus (Post 1534487)
Let me know if there is any trouble. I'll continue meanwhile adding QML methods. Should be all there soon, I hope.

Perfect instructions ! :cool:
I built it and ran it successfully on a Fedora workstation, with Qt5.7.1. Doing so, I found a typo on my Mapbox access token I used on previous tests, that may explain some errors I got then.

I only had to make a few changes to let it build:
Code:

diff --git a/app/mapbox-qml.pro b/app/mapbox-qml.pro
index fe7be45..d21fb3d 100644
--- a/app/mapbox-qml.pro
+++ b/app/mapbox-qml.pro
@@ -1,6 +1,6 @@
 TEMPLATE = app
 
-QT += qml quick location positioning
+QT += qml quick location positioning sql
 CONFIG += c++14
 
 SOURCES += src/main.cpp
@@ -10,7 +10,7 @@ RESOURCES += qml.qrc
 include(../mapbox-gl-qml.pri)
 
 INCLUDEPATH += ../../mapbox-gl-native/platform/qt/include ../../mapbox-gl-native/include
-LIBS += -L../../mapbox-gl-native/build -lqmapboxgl
+LIBS += -L../../mapbox-gl-native/build -lqmapboxgl -lz
 
 # Additional import path used to resolve QML modules in Qt Creator's code model
 QML_IMPORT_PATH =

The "-lz" part solves the "error adding symbols: DSO missing from command line" error I got.
The "sql" module in the QT line was needed to solve some linker error not finding some sqlite symbols (didn't write the exact message down).

It looks like that is what I need in my project.

So my next step would be to to see how I can add the route overlay on top of this map. So now, I get a lot things to read :).

Thanks for the help.

rinigus 2017-09-17 09:11

Re: App for vector renderer maps, using OpenGL ES 2.0
 
1 Attachment(s)
Testing time - I would need some help with testing. In particular, I am interested if the map widget works as intended.

Please use the provided app for testing only - that would then not deplete the map views and would be of great help for development.

So, I am looking for feedback regarding crashes and functionality. Please get the app at https://build.merproject.org/package...pbox-demo-sfos , choose your architecture, get build RPM and install using

Code:

pkcon install-local RPMfilePath
On start, you should see Helsinki map, as shown on the screenshot. You should be able to move, zoom in and out, change orientation (bearing slider), and tilt (pitch slider).

Please let me know how it went and specify
  • Device
  • SFOS version
  • Did it crash?
  • If crashed, please start in terminal.
    • Did it crash in terminal?
    • If crashed in terminal, please paste stdout
  • Is the map displayed properly?
  • Are the names rendered?
  • Does movement work?
  • Does zoom work?
  • Does rotation work?
  • Does tilting work?

If you see that someone already tested for your device/SFOS combination there is probably no need to test more.

Zeta 2017-09-17 10:19

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Using mapbox-sfos-0.1+master.20170916171905.daedaf3-10.6.1.jolla.armv7hl.rpm and mapbox-sfos-debuginfo-0.1+master.20170916171905.daedaf3-10.6.1.jolla.armv7hl.rpm
  • Device : Jolla 1
  • SFOS version : not yet updated, still on 2.1.0.11 (cat /etc/sailfish-release)
  • Did it crash? Yes
  • If crashed, please start in terminal.
    • Did it crash in terminal? Yes
    • If crashed in terminal, please paste stdout
      Only a segfault on stdout. Running through gdb on device gives thoses details (devel-su pkcon install gdb; gdb mapbox-sfos; type "r "+ Return):
      Code:

      Program received signal SIGSEGV, Segmentation fault.
      [Switching to LWP 11606]
      0x2a2333a8 in mbgl::RenderSymbolLayer::render(mbgl::PaintParameters&, mbgl::RenderSource*) ()

  • Is the map displayed properly? Before crash map is displayed OK
  • Are the names rendered? Yes
  • Does movement work? Yes
  • Does zoom work? Yes
  • Does rotation work? Yes
  • Does tilting work? Yes

Some notes:
So basically, the apps is working great but is killed after playing with it for some time (can be as low as 5 seconds, to 30 seconds), whatever the user action is (bearing, zooming...).

I have noticed a black square (a few pixels wide) on the top left of the screen. Not sure what it is, as it seems to always be there, no matter the map shown. Can provide a screenshot if needed.



Seeing those labels realign themselves while changing the bearing looks great and is really encouraging ! :cool:

Please let me know if there is a way to get you more output from gdb (call trace, local variables....). I will update the version on my Fedora desktop to see if I can see the same failure or not.

rinigus 2017-09-17 10:57

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Zeta, thank you very much for a thorough test!

On OnePlus X, same SFOS, the app is rock solid. I have not had any crashes after I changed OpenGL interface with QML to match the current implementation in QtLocation. Since I don't have any access to J1:

* would you mind to provide backtrace?

I can then search for this crash in Mapbox GL Native issues and maybe we would have to open an issue. We could also try to get the latest version of qt bindings (there seem to be one commit missing when compared to the current QtLocation version).

Let's see the backtrace and move from there.

wdehoog 2017-09-17 11:19

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Hi,

Impressive. Your demo app works flawless on my oneplus one running 2.1.1.24. And smooth.

tmi 2017-09-17 14:38

Re: App for vector renderer maps, using OpenGL ES 2.0
 
  • Device: Onda v820w V5
  • SFOS version: 2.0.5.6 (i486)
  • Did it crash? Yes
  • If crashed, please start in terminal.
    • Did it crash in terminal? Yes
    • If crashed in terminal, please paste stdout
Code:

[nemo@Sailfish ~]$ mapbox-sfos
[F] unknown:0 - This application failed to start because it could not find or load the Qt platform plugin "wayland"
in "".

Available platform plugins are: minimal.

Reinstalling the application may fix this problem.
Aborted
[nemo@Sailfish ~]$

Reinstalled mapbox with the same result.

rinigus 2017-09-17 15:02

Re: App for vector renderer maps, using OpenGL ES 2.0
 
@tmi: I will have to add i486 Sailfish 2.0.5.6 as a target. At present, only Qt 5.6 is supported on i486 since I expected that everyone moved from older releases. I'll look into it later today. It would take probably several hours of compilation on OBS though

tmi 2017-09-17 15:29

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Quote:

Originally Posted by rinigus (Post 1534530)
@tmi: I will have to add i486 Sailfish 2.0.5.6 as a target. At present, only Qt 5.6 is supported on i486 since I expected that everyone moved from older releases. I'll look into it later today. It would take probably several hours of compilation on OBS though

ATM 2.0.5.6. is the latest port for Onda v820w, but I'll be upgrading as soon as 2.1.x.x is available. I just don't know when that happens.
Personally, I can very well wait for the SFOS update so you can work on other issues if time is limited.

rinigus 2017-09-17 16:04

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Quote:

Originally Posted by tmi (Post 1534531)
ATM 2.0.5.6. is the latest port for Onda v820w, but I'll be upgrading as soon as 2.1.x.x is available. I just don't know when that happens.
Personally, I can very well wait for the SFOS update so you can work on other issues if time is limited.

OK, looks like 2.0.5.6 would require some additional effort and let's postpone it (see https://build.merproject.org/package...86_latest/i586 for error message). Probably used QML/OpenGL interaction is not compatible with Qt 5.2

Zeta 2017-09-17 18:07

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Quote:

Originally Posted by rinigus (Post 1534517)
* would you mind to provide backtrace?

It looks like I get differents outputs each time I run it... :confused:
The Jolla 1 is easily subject to OOM killer, maybe that would be such case ? Not sure where to look for this, as journalctl reports that no journal exists...

Some of these outputs, I installed the libc missing symbols at some point, but it didn't change much the output...

Code:

Program received signal SIGSEGV, Segmentation fault.
[Switching to LWP 13452]
0x4100bb22 in free () from /lib/libc.so.6
(gdb) bt
#0  0x4100bb22 in free () from /lib/libc.so.6
#1  0x434e1152 in ?? ()
#2  0x434e1152 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Code:

Program received signal SIGSEGV, Segmentation fault.
[Switching to LWP 13452]
0x4100bb22 in free () from /lib/libc.so.6
(gdb) bt
#0  0x4100bb22 in free () from /lib/libc.so.6
#1  0x434e1152 in ?? ()
#2  0x434e1152 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Code:

Program received signal SIGABRT, Aborted.
[Switching to LWP 14103]
0x40fc7be6 in ?? () from /lib/libc.so.6
(gdb) bt
#0  0x40fc7be6 in ?? () from /lib/libc.so.6
#1  0x40fd7d2e in raise () from /lib/libc.so.6
#2  0x40fdabb4 in abort () from /lib/libc.so.6
#3  0x410009f8 in ?? () from /lib/libc.so.6
#4  0x410009f8 in ?? () from /lib/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

My last try, I though it wouldn't crash after shaking it for a long time, but it finally crashed after close to 1 minute maybe, with an additional message:
Code:

*** Error in `/usr/bin/mapbox-sfos': malloc(): smallbin double linked list corrupted: 0x2ade60c0 ***
[New LWP 3120]

Program received signal SIGABRT, Aborted.
[Switching to LWP 3120]
0x40fc7be6 in ?? () from /lib/libc.so.6
(gdb) bt
#0  0x40fc7be6 in ?? () from /lib/libc.so.6
#1  0x40fd7d2e in raise () from /lib/libc.so.6
#2  0x40fdabb4 in abort () from /lib/libc.so.6
#3  0x410009f8 in ?? () from /lib/libc.so.6
#4  0x410009f8 in ?? () from /lib/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)


After a reboot (Android VM not started by default), the memory looks like this (only SSH connected, no actions locally); if it help to compare to other platforms :
Quote:

[nemo@Sailfish ~]$ free
total used free shared buffers cached
Mem: 828692 719220 109472 0 35308 419416
-/+ buffers/cache: 264496 564196
Swap: 627908 0 627908
How to know if the OOM killer is the one that closed the app ?
If not him, then it looks like some memory corruption problem, leading to segmentation fault and corrupted call stack...

Edit : anyone else with a Jolla 1 can confirm/refute what I see on my device ?

PamNor 2017-09-17 18:16

Re: App for vector renderer maps, using OpenGL ES 2.0
 
2 Attachment(s)
Testing: mapbox-sfos-0.1+master.20170916171905.daedaf3-10.6.1.jolla.armv7hl.rpm

Device: Jolla C
SFOS version: 2.1.1.26
Did it crash? No
Is the map displayed properly? Yes
Are the names rendered? No
Does movement work? Yes
Does zoom work? Yes
Does rotation work? Yes
Does tilting work? Yes

POI: When changing bearing or pitch is seems than names of POI disappear.

m4r0v3r 2017-09-17 18:26

Re: App for vector renderer maps, using OpenGL ES 2.0
 
tested on one Sony X, works well :) if routing worked this would be handy for an Uber app

elros34 2017-09-17 18:36

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Testing: mapbox-sfos 0.1+master.20170916171905.daedaf3-10.6.1.jolla

Device: Photon Q
SFOS version: 2.1.0.11
Helsinki, Punavuori, Kluuvi looks correct but Fabianinkatu and few others flick when Bearing or pitching.
There are also other artefacts: black rectangle in top left corner and black flicking lines when bearing or pitching.
gdb log: https://pastebin.com/HyeSExgU
Code:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x4637c390 (LWP 8020)]
0x43502948 in ?? ()
(gdb) bt full
#0  0x43502948 in ?? ()
No symbol table info available.
#1  0x4350f988 in ?? ()
No symbol table info available.
#2  0x4350f988 in ?? ()
No symbol table info available.
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

/proc/pid/maps gives me:
434be000-43593000 r-xp 00000000 103:05 79376 /system/lib/egl/libGLESv2_adreno200.so
I guess it's not good.

rinigus 2017-09-17 19:50

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Thank you all very much for taking time and testing. I will wait for a bit more tests to come in and then would make the table. As far as I understood, J1 could be a problem and should be tested further. We are also missing tests on Jolla Tablet.

Please test only if you have 2.1.x.y versions of SFOS - 2.0.5.6 is not supported at this stage.

Specific replies:

@Zeta: On normal systems you would usually see OOM killer in dmesg. Not sure it is there in SFOS. I have not thought that this app would take too much RAM. On my phone it takes about 100 MB RSS (250 MB virtual). Shouldn't be too much, I guess.

However, looks like there is something fishy with memory handling going on. I presume that OOM would not result in SIGSEGV in free(). Maybe someone can help us with tutorial on how to get debug symbols into your backtraces...

@PamNor: Your experience sounds to be fine and an expected one. On SFOS emulator/vm, there are no labels at all - hence the question regarding labels. Labels are expected to appear/disappear while you rotate/tilt the map since there are some parameters controlling how much can they overlap.

@m4r0v3r: no worries, will get there quite soon, I hope.

@elros34: seems like we have the same black box as in J1. You also seem to hit one of the crashes as experienced by Zeta. I'll try to run the whole thing in valgrind, let's see what will happen.

Edit: valgrind no go since it complains about illegal instruction on all tested programs.

Zeta 2017-09-17 21:03

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Quote:

Originally Posted by rinigus (Post 1534551)
@Zeta: On normal systems you would usually see OOM killer in dmesg. Not sure it is there in SFOS. I have not thought that this app would take too much RAM. On my phone it takes about 100 MB RSS (250 MB virtual). Shouldn't be too much, I guess.

However, looks like there is something fishy with memory handling going on. I presume that OOM would not result in SIGSEGV in free(). Maybe someone can help us with tutorial on how to get debug symbols into your backtraces...

I didn't see anything in dmesg, but as OOM appends regularly on Jolla 1, that was my obvious first thing to check.

Anyway, so far it seems that the problem is real and appends at least on the two weaker (CPU/GPU/RAM) phones tested, as Jolla 1 and Photon Q are both from 2012/2013 era with "only" 1GB RAM and Adreno 305/320 GPU. Not sure if that is a coincidence or not yet...

elros34 2017-09-17 21:32

Re: App for vector renderer maps, using OpenGL ES 2.0
 
I have 1GB swap enabled so it is not OOM killer.

Zeta 2017-09-17 21:34

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Quote:

Originally Posted by rinigus (Post 1534551)
Maybe someone can help us with tutorial on how to get debug symbols into your backtraces...

I got lost when trying it before, as gdb was giving commands to install them with "zypper", which appears to not be the good package manager on the Jolla (I am not far of 1 package manager per device I use, so it is sometime difficult to remember which one is used where....)
Code:

pkcon install glibc-debuginfo
Seems to do the trick.

The latest full trace and memory mapping with this are here : https://pastebin.com/Dmr34wGS
and https://pastebin.com/gs9Jv0VT


It never fail at the same place for me, but in my last tests this appends mostly around malloc/free calls.

By the way, I may have not said it before, there is a warning that I didn't take seriously before as it mostly works, but anyway, here it is:
Code:

[nemo@Sailfish ~]$ mapbox-sfos
[D] unknown:0 - Using Wayland-EGL
[W] unknown:0 - Could not find any zN.M subdirs!
[W] unknown:0 - Theme dir "/usr/share/themes/jolla-ambient/meegotouch/z1.0/" does not exist


MartinK 2017-09-18 00:11

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Quote:

Originally Posted by rinigus (Post 1534466)
I have managed to compile Mapbox GL Native for SFOS and tested it with the small app. All seems to be working as expected. Even in online mode, its fast, allows smooth transitions and so on.

Nice! Thanks a lot! :)

This should really help a lot, jut to name a few things that an Open GL based map view could provided in comparison to the custom tile based more or less 2D QML element:
  • support for proper rotation of maps (no upside down labels)
  • support for nicely animated transitions (zooming, map re-centering)
  • non-90 degree camera angle (eq. what common car navigation devices usually due in turn-by-turn navigation mode)
  • possibility for live style changes (no need to switch layers to switch on night mode or the possibility to highlight specific map elements - like roads - on the fly)

Quote:

Originally Posted by rinigus (Post 1534466)
SFOS packaging: Compilation relies on recent gcc (using 6.4.0, https://build.merproject.org/package...oolbox/opt-gcc). The compiled Qt version of the library is available at https://build.merproject.org/package...maps/qmapboxgl on the basis of https://github.com/rinigus/pkg-mapbox-gl-native .

QML integration: At present, Mapbox GL Native development is targeting Qt/QML through QtLocation 5.9 plugin. As a result, direct interface with QML was removed in the beginning of March 2017.

I have made a small project that would allow us to interface Mapbox GL through QML at https://github.com/rinigus/mapbox-gl-qml . Idea is to compose the interface that can be used by mapping applications for showing and interacting with the maps. Interface between OpenGL and QML is based on the same approach as used in the current implementation for QtLocation. This looks to be compatible with Silica as well, as much as I tested.

Yeah, that sounds to me as the way to go. We still can't use any form of Qt Location for Jolla Store apps, so depending on Qt Location from Qt 5.9 when another Qt 5 update on Sailfish OS is not even on the horizon is really a no-go.

What is the QML interface of the element like going to look like & what features will it likely support ?
Eq.: Just map centering, zooming control, etc. or more advanced stuff like POI markers any polyline display (routing trace/track logs) ? I've took a quick a for documentation of the upstream mapbox/mapbox-gl-native API and have not been really able to find a clear answer.

I guess if there is no marker/polyline support in the widget itself I should be able to use the existing polyline & POI overlays in modRana by putting them on top of the widget and adjusting them accordingly (scale, rotation, offset, etc.). But it could be quite tricky if the widget supports non-90 degree view angles & fractional scrolling. Also synchronization with animations would have to be handled so that the overlay does not lag behind during animations. Coordinate transformations could also potentially by fun in some cases. :)

Quote:

Originally Posted by rinigus (Post 1534466)
Small SFOS app is available at https://github.com/rinigus/mapbox-demo-sfos and shows how to integrate developed QML interface into the app as well as the integration with gcc-6.4.0.

I am letting you know about the development early, so we can discuss the ways we can integrate it with the available map applications (Poor Maps, modRana, Maep, and others). There is plenty to do, but all seems to be doable at this stage and shouldn't take a while before we'll get the map applications based on OpenGL rendering. Ideally, we should keep the same source code trees for tile-based and OpenGL map applications.

I'll likely integrating this map view first as an alternative option to the current fully tile based map view. At least until the features people expect to work (polylines, markers, maybe also overlays ?) can be provided by the Open GL based view as well. :)

BTW, what about offline usage ? Looking at the available documentation the widget seems to be basically locked to Mapbox for the input data for it's map rendering. How feasible does it look like making it (optionally ?) consume locally available data packs managed by OSM Scout Server and hosted @ modrana.org (as we already do with the other data packs) ?

And thanks yet again for working on all this! It's incredible what all you have accomplished so far! :)

rinigus 2017-09-18 06:51

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Quote:

Originally Posted by Zeta (Post 1534559)
I got lost when trying it before, as gdb was giving commands to install them with "zypper", which appears to not be the good package manager on the Jolla (I am not far of 1 package manager per device I use, so it is sometime difficult to remember which one is used where....)
Code:

pkcon install glibc-debuginfo
Seems to do the trick.

The latest full trace and memory mapping with this are here : https://pastebin.com/Dmr34wGS
and https://pastebin.com/gs9Jv0VT


It never fail at the same place for me, but in my last tests this appends mostly around malloc/free calls.

By the way, I may have not said it before, there is a warning that I didn't take seriously before as it mostly works, but anyway, here it is:
Code:

[nemo@Sailfish ~]$ mapbox-sfos
[D] unknown:0 - Using Wayland-EGL
[W] unknown:0 - Could not find any zN.M subdirs!
[W] unknown:0 - Theme dir "/usr/share/themes/jolla-ambient/meegotouch/z1.0/" does not exist


Looking good. Would you mind to add debug info for qmapboxgl and mapbox-sfos. As far as I can see, its packages qmapboxgl-debugsource and mapbox-sfos-debuginfo . Actually, you could add http://repo.merproject.org/obs/home:...mv7hl/armv7hl/ as a repo and then just use
Code:

pkcon install
. It will also allow you to upgrade in future, while we hunt for the bug.

If I can get these crash backtraces, we could open an issue for mapbox-gl-native and explain what has been done to make it run on SFOS. Maybe they can help.

As for crashes occurring for J1 and Photon Q: I wonder if its some OpenGL limitation of these devices. I haven't fiddled with OpenGL settings while compiling qmapboxgl, but maybe I should (have to look it up). But then not sure why it would crash on free/malloc calls...

rinigus 2017-09-18 07:07

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Quote:

Originally Posted by MartinK (Post 1534564)
...

Yeah, that sounds to me as the way to go. We still can't use any form of Qt Location for Jolla Store apps, so depending on Qt Location from Qt 5.9 when another Qt 5 update on Sailfish OS is not even on the horizon is really a no-go.

As far as I remember, qmapboxgl does use some QtLocation structures, but I could be mistaken. If this will disqualify it from current Harbour - so be it. I am sure that Jolla will change the rules regarding QtLocation when they manage to get into working on QtLocation 5.6. There is surely a pressure from their customers to get it sorted, maybe just would take some time. In meanwhile, we can distribute via OpenRepos, if needed.

Quote:

Originally Posted by MartinK (Post 1534564)
What is the QML interface of the element like going to look like & what features will it likely support ?
Eq.: Just map centering, zooming control, etc. or more advanced stuff like POI markers any polyline display (routing trace/track logs) ? I've took a quick a for documentation of the upstream mapbox/mapbox-gl-native API and have not been really able to find a clear answer.

Mapbox GL Native allows you to add all these elements via GeoJSON and styling of the data. This should allow to add POIs, routes, any polygon... See https://github.com/mapbox/mapbox-gl-...window.cpp#L96 for the start of the example. I am planning to expose similar API as well as write a small wrapper allowing you to add routes/POIs/location icon easier. So, you would not have to draw anything above it, just regular widgets as you do for interaction. Let's see where I can get in a week.

Quote:

Originally Posted by MartinK (Post 1534564)
I'll likely integrating this map view first as an alternative option to the current fully tile based map view. At least until the features people expect to work (polylines, markers, maybe also overlays ?) can be provided by the Open GL based view as well. :)

We would probably need tile-based widgets, as you all have already, as well to support sources that are not compatible with the mapbox-gl. So, you would probably end up having two alternative ways to show the maps that you would have to switch in between in the application.

Quote:

Originally Posted by MartinK (Post 1534564)
BTW, what about offline usage ? Looking at the available documentation the widget seems to be basically locked to Mapbox for the input data for it's map rendering. How feasible does it look like making it (optionally ?) consume locally available data packs managed by OSM Scout Server and hosted @ modrana.org (as we already do with the other data packs) ?

Widget seems to be locked. Its actually possible to use other data sources, as done for https://openmaptiles.org/ . Right now I am developing against Mapbox data sources, since it easiest. Already now it should be possible to use other sources, but I have not tried.

As for offline, as soon as widget is ready, I am planning to add offline support for it via OSM Scout Server. We should be able to squeeze in additional datapack for mapbox gl, I presume it will be of similar size to mapnik datapack. Hopefully, data.modrana.org will be fine to host it :) . So, the plan is to go all the way with the full online and offline support. And I don't think there are any major roadblocks on the way.

rinigus 2017-09-18 07:16

Re: App for vector renderer maps, using OpenGL ES 2.0
 
Quote:

Originally Posted by elros34 (Post 1534547)
Testing: mapbox-sfos 0.1+master.20170916171905.daedaf3-10.6.1.jolla

Device: Photon Q
SFOS version: 2.1.0.11
Helsinki, Punavuori, Kluuvi looks correct but Fabianinkatu and few others flick when Bearing or pitching.
There are also other artefacts: black rectangle in top left corner and black flicking lines when bearing or pitching.
gdb log: https://pastebin.com/HyeSExgU
Code:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x4637c390 (LWP 8020)]
0x43502948 in ?? ()
(gdb) bt full
#0  0x43502948 in ?? ()
No symbol table info available.
#1  0x4350f988 in ?? ()
No symbol table info available.
#2  0x4350f988 in ?? ()
No symbol table info available.
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

/proc/pid/maps gives me:
434be000-43593000 r-xp 00000000 103:05 79376 /system/lib/egl/libGLESv2_adreno200.so
I guess it's not good.

Edit: Looks like the same lib is used in J1. In my case (Oneplus X), I have libGLESv2_adreno.so (and others similar, without 200). Whether its an issue, don't know - I am not a specialist on OpenGL.


All times are GMT. The time now is 11:34.

vBulletin® Version 3.8.8