How to access SMS database in Sailfish? - Page 2 - maemo.org

wolke | # 11 | 2013-12-03, 22:02 | Report

human-readable import/export of SMS on sailfish is mission critical before i can start using jolla as a primary phone.

likewise with call log and contacts {MMS would be nice too}

i also dont care what format is used, as long as its human-readable, well-structured, and validatable.

however, i personally would prefer a platform-independent solution, preferably one that just reads/writes the sqlite database so that it can be done on a faster box.

as soon as i get a jolla {usa, so sad} this will be the first thing i investigate coding up.
-a sqlite solution will be complicated. it will also be brittle, since the libcomm interfacing impl and the sql table structure could change
-things aside from libcommhistory might be in the sqlite db, so this might involve risk in replacing the db.
-if the jolla code that handles the sqlite db is open and modular, it might be adaptable for this purpose

The Following 4 Users Say Thank You to wolke For This Useful Post:
hrbani, jalyst, juiceme, MartinK

jbrooks | # 12 | 2013-12-03, 22:39 | Report

Originally Posted by wolke View Post

human-readable import/export of SMS on sailfish is mission critical before i can start using jolla as a primary phone.

likewise with call log and contacts {MMS would be nice too}

i also dont care what format is used, as long as its human-readable, well-structured, and validatable.

however, i personally would prefer a platform-independent solution, preferably one that just reads/writes the sqlite database so that it can be done on a faster box.

Contacts are stored separately, but vcard export should already be possible (or is easy to write up). Some of our contacts metadata is lost in exporting, of course. We have full backups for contacts, messages, etc as well, but not in a readable format.

Originally Posted by wolke View Post

as soon as i get a jolla {usa, so sad} this will be the first thing i investigate coding up.
-a sqlite solution will be complicated. it will also be brittle, since the libcomm interfacing impl and the sql table structure could change
-things aside from libcommhistory might be in the sqlite db, so this might involve risk in replacing the db.
-if the jolla code that handles the sqlite db is open and modular, it might be adaptable for this purpose

Avoid using the database without using libcommhistory; the schema is very likely to change. All of this code is entirely opensource at https://github.com/nemomobile/libcommhistory

There is a basic JSON format in commhistory-tool for importing messages right now, primarily meant for predefined content. That may eventually be extended into a full import/export format, to replace the current fragile binary one. See https://github.com/nemomobile/libcom...tool.cpp#L1112

There's nothing to export readable call logs right now, but they are also managed by libcommhistory.

I'm happy to see so much interest. Patches are always welcome..

The Following 10 Users Say Thank You to jbrooks For This Useful Post:
jalyst, juiceme, kjmackey, lkravovicz, mattaustin, miqu, mrsellout, OVK, Watchmaker, wolke

wolke | # 13 | 2013-12-03, 22:53 | Report

this is all REALLY good news, thanks.
full json import/export would be perfect {with my n9, i am currently entirely dependent on https://github.com/wwwjfy/MeegoSMSBackupRestore}

out of curiousity, what sort of contacts metadata is non-exportable to vcard? {only things i can imagine are for tie-in with awful apps like skype or wazapp}

merlin1991 | # 14 | 2013-12-09, 16:08 | Report

I've hacked up the smsImporter today, I've tested it on nemo since I don't have a device, so no guarantee for it's functionality.

Running the binary with the exported csv as argument imports everything
Binary: http://cdnm.at/~christian/Harmattan-SMS-Boat/smsImport
Source: https://github.com/merlin1991/Harmat...port/smsIm.cpp

edit: Thanks to a tip from jbrooks it works now perfectly, no more segfaults. The new binary is available at the link above.

Last edited by merlin1991; 2013-12-09 at 22:34. Reason: segfault is fixed

The Following 11 Users Say Thank You to merlin1991 For This Useful Post:
Iipe, lkravovicz, mattaustin, mrsellout, Naranek, OVK, P@t, Watchmaker, xkkkx, ymb, zlatko

Nikos Alexandris | # 15 | 2013-12-22, 09:57 | Report

Originally Posted by merlin1991 View Post

I've hacked up the smsImporter today, I've tested it on nemo since I don't have a device, so no guarantee for it's functionality.

Running the binary with the exported csv as argument imports everything
Binary: http://cdnm.at/~christian/Harmattan-SMS-Boat/smsImport
Source: https://github.com/merlin1991/Harmat...port/smsIm.cpp

edit: Thanks to a tip from jbrooks it works now perfectly, no more segfaults. The new binary is available at the link above.

Hi and thank you for the efforts!

How can I use these tools (smsExport, smsImport) on N9 and Jolla respectively? Simply get the binaries in each device and then execute from a terminal? Any simple examples?

merlin1991 | # 16 | 2013-12-23, 19:21 | Report

Originally Posted by Nikos Alexandris View Post

How can I use these tools (smsExport, smsImport) on N9 and Jolla respectively? Simply get the binaries in each device and then execute from a terminal? Any simple examples?

Yep, simply get them on device and execute.
smsExport just prints everything to stdout so do

Code:

./smsExport > somefile.csv

And smsImport expectes a path to a file, so scp the somefile.csv onto the jolla aswell and do

Code:

./smsImport somefile.csv

The Following 4 Users Say Thank You to merlin1991 For This Useful Post:
Iipe, rainisto, xkkkx, zlatko

xillibit | # 17 | 2013-12-23, 20:13 | Report

Hello,

I'am interested too but to import SMS from symbian device, i have found a way to export the SMS in XML

How can-i see the fields to import into libcommhistory ?

Last edited by xillibit; 2013-12-23 at 20:16.

Nikos Alexandris | # 18 | 2013-12-24, 13:25 | Report

Originally Posted by merlin1991 View Post

Yep, simply get them on device and execute.
smsExport just prints everything to stdout so do

Code:

./smsExport > somefile.csv

And smsImport expectes a path to a file, so scp the somefile.csv onto the jolla aswell and do

Code:

./smsImport somefile.csv

Thanks. Trying... I see that smsExport (in N9) does not have the "x" permission granted. Trying to change that as root, doesn't work. As "user" it won't let me run it.

nieldk | # 19 | 2013-12-24, 13:46 | Report

Originally Posted by Nikos Alexandris View Post

Thanks. Trying... I see that smsExport (in N9) does not have the "x" permission granted. Trying to change that as root, doesn't work. As "user" it won't let me run it.

whats the errormessage?

/bin/chmod +x path-to/smsExport

/bin is not in PATH as standard, may be your issue

Nikos Alexandris | # 20 | 2013-12-24, 14:51 | Report

Originally Posted by nieldk View Post

whats the errormessage?

/bin/chmod +x path-to/smsExport

/bin is not in PATH as standard, may be your issue

The only place from where I could execute the binary smsExport was from /home/user. Anywhere else, e.g. under /home/user/MyDocs/N9Export I could neither alter permissions nor let it run (always permission denied).

Now, I've manage to run it (as user, via ssh), as mentioned, from "home", and it does:

Code:

./smsExport

"GDBus.Error:org.freedesktop.DBus.Error.AccessDenied: Rejected send message, 2 matched rules; type="method_call", sender=":1.163" (uid=29999 pid=4928 comm="./smsExport ") interface="org.freedesktop.Tracker1.Steroids" member="Query" error name="(unset)" requested_reply=0 destination="org.freedesktop.Tracker1" (uid=29999 pid=1324 comm="/usr/lib/tracker/tracker-store "))" 
# sms:  0 
QTrackerDirectSelectResult: QSparqlError(9, "GDBus.Error:org.freedesktop.DBus.Error.AccessDenied: Rejected send message, 2 matched rules; type="method_call", sender=":1.163" (uid=29999 pid=4928 comm="./smsExport ") interface="org.freedesktop.Tracker1.Steroids" member="Query" error name="(unset)" requested_reply=0 destination="org.freedesktop.Tracker1" (uid=29999 pid=1324 comm="/usr/lib/tracker/tracker-store "))", 4) "SELECT ?message nie:plainTextContent(?message) rdf:type(?message) nmo:communicationChannel(?message) nmo:sentDate(?message) nmo:messageId(?message) nmo:receivedDate(?message) nie:contentLastModified(?message) nmo:isSent(?message) nmo:isDraft(?message) nmo:isRead(?message) nmo:encoding(?message) nie:characterSet(?message) nie:language(?message) (SELECT GROUP_CONCAT(tracker:coalesce(nco:imID(?medium), nco:phoneNumber(?medium), ?medium), "\u001e") WHERE { ?from nco:hasContactMedium ?medium . }) (SELECT GROUP_CONCAT(tracker:coalesce(nco:imID(?medium), nco:phoneNumber(?medium), ?medium), "\u001e") WHERE { ?to nco:hasContactMedium ?medium . }) nmo:phoneMessageId(?message) WHERE { SELECT ?message ?from ?to  IF (nmo:isSent(?message) = true, ?to, ?from) AS ?target  ?message WHERE {?message nmo:from ?from ; nmo:to ?to .  ?message rdf:type nmo:SMSMessage . } }"

Thanks for caring.