[STOPPED] Transponder, a native messaging client
 

Due several issues I will not continue the development of Transponder. I will focus on my other apps instead.


Purpose
Transponder wants to fix the lack of messaging clients on Sailfish OS by providing a modular architecture which can be used with multiple providers like Matrix.org, Telegram, IRC, Hangouts, ... as long as their is a Python SDK for it.

Why Python?
I know, C++ with Qt is a nice combination and faster then Python, but Python has already a SDK for a lot of messaging providers which will make the integration and maintaining of a large group of providers possible.

Status
The development of Transponder is in it's early stages.

Donations
You can support me by buying me a coffee: PayPal.me link

Any feedback or ideas on this project are more then welcome!

Re: Transponder, a native messaging client
 

Hi,

Thanks for taking this project !

As you are developing something new, I imagine it is to overcome problems of the existing solutions (at least in desktop world) like Pidgin (https://developer.pidgin.im/wiki/ThirdPartyPlugins) or Telepathy ?

Re: Transponder, a native messaging client
 

Indeed, I want to overcome these issues by using as much as possible the offical SDKs from each provider.

Re: Transponder, a native messaging client
 

I am not an expert in the area and started reading about messaging just few days ago after porting to the latest release which disabled hangouts. Hence my understanding can be patchy.

From reading about telepathy framework and your proposal, I am wondering whether its better to focus on already developed telepathy and develop around it. From the look of it, you should be able to make new plugins for new network types and make it all accessible through the whole system. As far as I can see, its possible to make modules in python as well via PyGObject, if you wish.

Taking into account closed source components, you would probably need to implement Accounts and Messager. But the whole lot of work on making parts work together, designing API for communication and implementing it, has been done already.

There is a mention of issues regarding telepathy. Which particular issues do you have in mind?

Do I miss something vital here that makes it better approach to start developing a new framework from scratch?

Re: Transponder, a native messaging client
 

I understand your concerns but I want to keep Transponder separated from the integrated Telepathy stuff for several reasons:

• Telepathy integration doesn't play well with Harbour and I really require this.
• Depending on Telepathy implementation in SFOS, porting to UT, Plasma, ... is more difficult

Jolla's Telepathy doesn't support properly group chats, files sending like it should. Transponder should reach the state of Sailorgram (in the future)

Re: Transponder, a native messaging client
 

Also if I remember correctly, there is still the closed source UI layer, so even if community implemented all the backend stuff, someone from Jolla would have to do the UI bits to actually make it useful, which is unfortunately very unlikely to happen given similar cases in the past.

(IIRC, SIP support is basically in place backend wise, but the UI is missing and the community has been waiting for Jolla to do something about it for years and can't do anything due to UI being closed source. And there are many other similar cases.)

So while I would also, like rinigius, want to have a full featured unified messaging UX (it worked very well on the N900!) I'm afraid it's not really possible to do with Telepathy and the closed UI. At least unless there would be a concrete promise from Jolla to help with the UI bits or ideally to finally open source the relevant UI bits.

Re: Transponder, a native messaging client
 

One of the features I've been trying to integrate into Saera is the ability to read texts aloud as they come in, or to send them with dictation. Will Transponder have a method to allow other apps to communicate with it?

Re: Transponder, a native messaging client
 

Modern XMPP/Jabber support with at least Message Carbons and Message Archive Management and multiple accounts and multiuser chat.
Most of this is missing in stock Sailfish xmpp account framework making it very limited.

I know that the probably best Linux desktop xmpp application Gajim is heavily based on python but not sure it can be any use in your project...

Re: Transponder, a native messaging client
 

In the future this is possible, however I like my privacy so retrieving and sending messages would need some kind of authorization between Transponder and other apps.

Re: Transponder, a native messaging client
 

According to Plasma Mobile page they also use Telepathy. From some old docs, looks like Ubuntu Touch used Telepathy.

I don't know about the differences in implementation between Plasma/SFOS/UT. Are there any examples?

What looks to me as a main advantage is that you can hook yourself to the existing components and expand as you progress. As for Store, I would suggest to be more critical in having it and not to spend too much time in playing by its rules. They are quite limiting and should not lead to bad design.

This proposed messaging client will have to implement UI for messaging, UI for accounts, API to link multiple backends with the frontend, and battery-efficient backends. In that sense, using existing API and having to work with "just" UI is a simple undertaking, isn't it? :)

I agree with @Modulebaan in terms of keeping development easily relocatable - with significant closed source components we should be ready to move out from SFOS as a platform. So, having functional alternative UI would be handy.

@taixzo, what was the stumbling block? From all the overviews of Telepathy API it seems that it was designed to have multiple components interacting and it should support such feature out of the box.

As always, its up to the main dev to choose the direction and there are many factors in the equation, including learning-by-doing component. One aspect that I would like to bring out is that its a lot you could learn when working on bigger project that has its Q&A for the code and contributions well developed. Don't know much about telepathy, but I surely did learn much by working with the others and contributing to larger projects.

Whatever way you choose, good luck!

Re: Transponder, a native messaging client
 

FWIW, there was a lot of discussion recently on GNOME's desktop-devel-list about Telepathy and Matrix.

• https://mail.gnome.org/archives/desk...ead.html#00047
• https://mail.gnome.org/archives/desk...ead.html#00112

Messaging is not a field I'm familiar with, but it seemed to me that people have issues with Telepathy and that Telepathy's design is not considered a success -- red flags to me regarding choosing what to build on.

Re: Transponder, a native messaging client
 

Very interesting reading, thank you for sharing it!

Re: Transponder, a native messaging client
 

Well, no. From observations so far (2013-now) basically every time a community initiative hit any UI work in the closed source components only Jolla developers can do things ground to a halt and have not moved since. In some cases developers switched to alternative solutions (all the UI patches, various IM clients).

Just from top of my head:

• one category in the latest coding competition was won by patch adding GPG to the email client
• SIP is still waiting for UI integration (5 years and going strong!)
• third party applications still can't put settings pages to settings
• no multituser chat support (also quested by many users from day one)
• someone even tried to submit patches to the closed calendar UI "upstream" via the mailing list, only to be ignored
• usb tethering is still more or less hidden (by not being installed by default) for years due to missing UI elements (as discussed on a community meeting a while ago)
• AFAIK no new accounts or messaging providers have been added to the build in messaging system by a thirparty since Sailfish OS launched

So given this track-record it seems to me like a totally sensible idea to develop all these UI elements as the effert will not be blocked for years by requiring overworked Jolla developers to participate in it. I'd like to be proven wrong (and everyone is certainly welcome to chime in with any counter examples!) but it really seems to me like the only way to go.

Re: Transponder, a native messaging client
 

@MartinK, my point was exactly the same - UI for messaging and accounts would have to written within this project. There is no hope to rely on closed source Jolla components.

From reading Gnome list archive posted by @otsaloma, I would have considered any multi-protocol messaging program carefully. Looks like lead telepathy dev had a rather large number of issues with it. I guess, unless you handle it according to the lowest denominator, as for SMS.

@Modulebaan, you are involved in developing Matrix client, right? Do you still plan to develop it separately into the full-featured one or want to merge relevant code over into this project?

Re: Transponder, a native messaging client
 

@Everyone

I understand what you mean Rinigus: Telepathy would integrate much better and requires 'less' effort to develop the backend due the integration with the OS.

As MartinK pointed out: Waiting for Jolla is going to harm this project that's why I want to take the Python/QML approach. The UI is already written for messaging and contacts (it's a shared component between Transponder and Sailfinder).

I want to integrate the Matrix client into Transponder, the first release of Transponder will only support Matrix, for the next versions I plan Telegram and other providers. Matriski will be deprecated as soon as Transponder with Matrix hits stable.

The beauty of the Python plugin system is that other people easily can add new backends to Transponder. Only a bridge file needs to be written as with Otsaloma's Pan Transit app.

Re: Transponder, a native messaging client
 

Very nice project. Just out of curiosity (maybe i've overread it): is transponder working in the background (e.g. daemon)?

It is a main showstopper for me using a messaging application and needing it to have it running in foreground.

Re: Transponder, a native messaging client
 

Indeed, that's something we definitely need. No promises for the first versions but it's almost certainly it will have some kind of daemon.

Re: Transponder, a native messaging client
 

Do you plan for the matrix client to be fully functional (i.e. support exchange of pictures and videos, and have VOIP)? It would be fantastic to have all this natively rather than with Alien Dalvik. (Riot.im only partially works in my hands.)

I am already a matrix patron, and I would be happy to buy you a coffee if it helps the full functionality of matrix to land on SFOS ;)

Re: Transponder, a native messaging client
 

Sending files and stuff is on the roadmap. VOIP isn't at the moment since I lack the knowledge and time to implement this. Also the Python SDK doesn't support this.

Re: Transponder, a native messaging client
 

Are you sure that using python is a good idea?

I'm not talking performance here. I mean the maintainability. I've worked on a 50+KLOC python project and maintaining a bigger python codebase is a nightmare.

Python linters suck. Pylint is unable to properly handle classmethods, mypy had some problems too (I don't remember what kind of). The rest of linters will only scream at your style. Unless you have a giant test suite, with a really, really big test coverage and have a CI checking it all at every commit, the project will be unmaintainable. And even with this whole infra, it's a nightmare.

I'm not a fan of C++ either (Rust FTW) but when using a statically-typed language, the compiler at least pays attention that you're not doing utter nonsense.

/edit: btw. I see gajim spewing errors from time to time because the types don't match :P

besides, if shmoose were to be integrated into transponder (not discussed this yet with the main dev) - we're using swiften, so using python here would rule out the possibility of integration

Re: Transponder, a native messaging client
 

Are you sure that using swiften is a good idea?
1. It does not support simultaneous connections to several accounts.
Actually there was a question about that in swift mailing list and the answer was that: "Noted, thanks - it’s on the list and Daniel (one of our GSoC students) is looking at this over the summer." However the summer was the 2015 one... :D
2. It does not support XEP-0313: Message Archive Management. Can't imagine a multiuser chat where you see only those messages when you are present...

The best one is probably Conversations (android) and in the Linux desktop area Gajim (pygtk). Also Psi (qt5) says that XEP-0313 will be supported in version 2.0 (currently 1.3).

Re: Transponder, a native messaging client
 

Are you sure that swiften 4.0 (beta) doesn't support MAM? https://github.com/swift/swift/searc...utf8=%E2%9C%93

Re: Transponder, a native messaging client
 

Hmm, testing here Swift-4.0rc5.amd64.appimage and yes, message archive management is definitely not working.

Re: Transponder, a native messaging client
 

At work I'm one of the people who maintain & develop the Anaconda installer used by Fedora/RHEL/CentOS and other distributions. Anaconda has a sizeable codebase with a history getting back to 1999. but we don't really see any special Python related issues compared to if it was written in a different language and we can maintain it just fine.

It's certainly true that due to the awesome flexibility and dynamic nature of Python you don't get some of the benefits of static type checking and linting overall is hard to implement (though Pylint is already doing a pretty well & improving).

On the other hand I think the static type checking provided by strongly typed languages is overrated - it will find only the most trivial issues, while any real mistakes in the application logic will still show up at only at runtime. So you still need a comprehensive test suite either way & it seems to be less and less acceptable these days to develop nontrivial suftware without having a good test coverage and using CI.

Also in the Sailfish OS perspective - we have Python 3.4 (letst stable is 3.6), which compared to the ancient GCC version (4.8 I think ?) is substantially newer and should thus enable the use of resonably modern Python features, unlike with the default C/C++ compiler.

So I don't really see any real issues with using Python on this and I think it's a really good choice given the current state of the Sailfish OS toolchain.

Re: Transponder, a native messaging client
 

https://github.com/PyCQA/pylint/issues/1646
https://github.com/PyCQA/pylint/issues/1651
https://github.com/PyCQA/pylint/issues/1660
https://github.com/PyCQA/pylint/issues/1631
https://github.com/PyCQA/pylint/issues/1630

This was more than enough to drop obligatory pylint check at work. Especially the first issue.

Just as an integral of a small function over a big set, if you have a lot of small, trivial bugs, that could be easily caught - that's a big burden.

As for the static checking in C++, yes, this is often not enough. Enter Rust! :)

There's the updated GCC by rinigus.

Re: Transponder, a native messaging client
 

The discussing Python vs C++, every language has his pro and cons.

I already made some prototyping for the modules in Python which works fine. I'm also trying the implement the same thing in Qt C++ to check out which way is the best one.

Currently I'm stuck a little bit between the 2 languages since I'm considering to add 3rd party support (at least the basis to do that) and this has pro and cons for both languages.

Re: Transponder, a native messaging client
 

On 1)
I have not looked into the code about this until now. Someone will find this feature usefull, someone have only one account. It depends on the personal usage.

On 2)
MUC didn't need xep-0313 to work correctly in the client. The last recent messages of a group chat are stored on the server and will be send to clients which where offline. This is trivial implemented without the need of message archive management.

Re: Transponder, a native messaging client
 

In my opinion it is also not a thing python vs c++. When I start a project I have a look at the available frameworks and librarys which are available to handle the features I need. I think that should make the decision. It's very hard to write most things again from scratch.

Personally I would not start to work on such a framework. The linked discussion on gnome mailing list about telpathy says all about this approach.

But hey, this is open source. Everyone who has a need is free to implement things that make his life easier. I wish you all success with Transponder!

Re: Transponder, a native messaging client
 

schorsch, still, I think that we'll need to come around MAM sooner or later, anyway. I often receive some links on the phone but want to view than on the computer.

So this will wind up as writing PRs against swiften. Not saying that it's not the best solution here, I don't know the alternatives.

/edit: from the Conversations channel:

Re: Transponder, a native messaging client
 

Yes, to some extent it works. But my experience shows that this is not very reliable. Not to say, that when server is restarted, then everything is lost. Sometimes my own text is missing at all etc... Maybe it can be configured to your need when you have a full control to the server yourself...

Re: Transponder, a native messaging client
 

Please guys, don't hijack this thread with general discussions about the pros and cons of different programming languages :p There are plenty of other places for that kind of thing.

@Modulebaan is quite right to have identified messaging as a weak point on SFOS, and his proposal to do something about it is more than welcome :)

Re: Transponder, a native messaging client
 

Will this app someday be able to take care of SMS/MMS also? Does Sailfish allow to use different app for SMS/MMS?

If this is possible, will this be able to solve the big problem of not supporting group SMS/MMS in default messaging app.

Re: Transponder, a native messaging client
 

Modulebaan, will you use MartinK's universal components, so that the whole thing will work on any system that support qqc2 (such as nemo)?

Re: Transponder, a native messaging client
 

From all the comments I read here, building a framework is hard and that's not the purpose of Transponder.

The current architecture looks like this:

• Transponder QML UI
• Matrix.org daemon
• DBus interfaces to communicate
• C++ as main language, PyOtherSide to add Python for certain SDKs

I'm not familiar with those components and the UI is shared with Sailfinder. I will not use those, I'm sorry. Feel free to write the UI yourself.

Re: Transponder, a native messaging client
 

It's basically a component set I wrote so that modRana (and any other apps using Universal Components) can have a single QML UI codebase that is independent of Silica/Qt Quick Controls or any specififc QML components set in particular.

But it's not yet another component set in itself, it's rather a common API layer with backends selectable at runtime (currently Silica & Qt Quick Controls), so an application using Universal Controls still looks native (an UC Button is a Silica Button on Sailfish OS and a QQC Button on Fedora for example).

I'm using this even during development of modRana - I can easily test and debug the UI directly on my Fedora desktop via the QQC backend, while on my Sailfish X Xperia the same UI code will run with the Silica backend. I've also used Universal Controls for porting modRana to Android, where QQC are available as part of the Qt 5 port to Android.

So that's just some background to explain what the Universal Components project is about. Of course if you already have an existing UI codebase based on Silica and or if you plan to only target Sailfish OS then I guess using Universal Components might not bring many benefits to your usecase.

And also, thanks yet again for taking on the Transponder project! I think it really has a potential to improve the currently suboptimal native messaging situation on Sailfish OS.

Re: Transponder, a native messaging client
 

I think, that shmoose is hands down the best xmpp client for sailfish now if it supports at least Message Carbons XEP-0280 (needs probably to use swift 4.0 base). :)

But the XMPP discussion seems to drift offtopic here, sorry for that... :D

Re: Transponder, a native messaging client
 

Just to add to the discussion around the use of telepathy, I did quite a lot of research myself when writing PingYou for the coding competition. I didn't get very far - mostly just re-implemented account management stuff and exposed some extra options.

The main feature I implemented that isn't supported in jolla-messages was the ability to authorise subscription requests ("accept friend request"). This functionality sat alongside the closed source jolla settings menus (i.e. it was an alternative settings menu, and changes to one would be reflected in the other). I think this is the best way to work around closed source interfaces to open source components (build an alternative/complimentary interface instead of waiting for Jolla to make the changes when it's probably not a priority for them).

One of the libraries I used was telepathy-qt, which I had problems using with qml, it was written for Qt4 and there are some types that don't transfer easily to qml (or, my qml knowlege is too poor to see how to do it). I wrote a bit about how far I got with that in the development readme.

Adding new features/XEPs to telepathy is difficult on Sailfish because you would have to push the changes upstream, and get them approved by Jolla etc, which makes rapid development quite difficult. I don't know what would happen if you built your own packages and made them dependencies on openrepos (would it break other parts of the telepathy stack? seems risky). Having researched which XEPs are available currently, and the progress of people who have implemented support for quite simple XEPs (like carbons) themselves and submitted pull requests, I have to say I think the whole process is very cumbersome and it seems to take ages to get anything done.

However, it's worth noting that it is possible in theory to write a connection manager in any language (including python), so you could write telepathy-matrix and re-use the rest of the stack.

Swings and roundabouts.

Re: Transponder, a native messaging client
 

@MartinK Your components are pretty nice! I really like them but since I only target SFOS and I have already an UI ready, I'm not going to use them, keep it up though!

I really appreciate the feedback on Transponder, the initial design is completely gone at the moment :D more info and Github commits on Transponder are on the way.

Re: Transponder, a native messaging client
 

Any news on the status of this project?

Re: Transponder, a native messaging client
 

https://github.com/DylanVanAssche/harbour-transponder
Last commit January 29th ;)