[Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

A new version of Erminig-NG is available! More information in this post


The new version sports an easy-to use GUI and is ready for a more widespread testing, since it's now available in extras-devel. Please follow the link above to get more information!

This now obsolete post will stay here for historical purposes, but the old version of Erminig-NG is not available for download anymore.
************************************************** ******************************
Hi all,

I'm pleased to announce the first public (and very alpha) version of Erminig-NG, a tool that allows you to perform two-way synchronization between Fremantle Calendar and Google Calendar.

BIG FAT WARNING: this is still alpha-quality software, so it's not really for those people who are afraid to lose events in their calendars!

Currently Erminig-NG is a command-line only tool (a GUI will follow), and no .deb file is available. At the end of this post there's a .tar.gz that you can download and unpack on your device if you are brave enough to try Erminig-NG.

Please first try it on non-important calendars (or better yet, create ad-hoc calendars for the tests), since a screw-up in Erminig-NG may trash your calendars!

Software dependencies: python-gdata (available in Extras). By installing it (I won't give out the instruction about how to install python-gdata, in order to restrict a little bit the audience of crazy people wanting to try Erminig-NG), it will pull all the necessary dependencies (namely Python2.5)

How to install Erminig-NG
Download and unpack the archive file somewhere on your device
Open the file erminig_conf.py and change the first two variables (they represent your Google username and password)

How to run Erminig-NG

For the first run:

This will ask you which calendar (both local and remote) you want to keep in sync

WARNING: For the moment it is important to run Erminig-NG for the first time with the --sync-calendars option, otherwise it will miss old events to import (known bug). Similarly, adding calendars to sync after the first real sync will miss old events (only new events from the time of the last sync will be synced in the the new calendar). Uhm, this is quite convoluted as an explanation...I hope that you will get the essence of this warning though :)

This will perform the bulk of the work and keep in sync the calendars. This is the command you want to run when you wish to synchronize your calendars.

If you added a local or Google calendar, you may want to rerun Erminig-NG with the --sync-calendars, so that it will ask you if you want to sync the newly discovered calendars (with the gotcha mentioned above!)

Important: I won't take any responsibility for trashed calendars, so if you missed a date with your girlfriend/wife/boyfriend/husband/whatever, or you missed your flight, or any kind other kind of event related to calendar due to mis-synchronization I can only be sorry (and I apologize, sincerely), but it cannot go further than that.

UFAQ (Upcoming FAQ ;)))

• There's a bug, what should I do?
  Two options:
  
  - report it in this thread, if possible by providing accurate step-by-step instructions
  - Contact me by mail: lorelei@garage.maemo.org (again with accurate instructions)
  
  In the future I will open a bugs.maemo.org component for Erminig-NG, in order to keep track of everything, but first I want to see how it is received within the community before investing resources that are not mine (i.e. maemo.org's bugzilla)
  
• Should I synchronize the local calendars called "cal_ti_calendar_synced" and "cal_ti_calendar_private"?
  No idea. I think that those are the built-in default calendars, but I've never tried to sync them.
  
• I would like to reset Erminig-NG to a known state (i.e. a state where it doesn't know anything about calendars relationships)
  Remove the file ~/.erminig.db
  This will make Erminig-NG going back to "factory settings"
  Please note that this may generate duplicate events in Google or Fremantle Calendar if you perform a resync on calendar previously synced.
  
• What is the stability status of Erminig-NG ?
  It's in early alpha stage. I'm just starting to pin down the basic functionalities before moving on the the GUI part
  
• What about sync conflicts (modification of an event both on Google and on the local calendar)?
  There's no explicit handling of conflicting updates for the moment. This will be addressed in a future version
  
• Why no .deb package (or why not in Extras-devel)?
  Erminig-NG is still in early development phase, so by publishing only the .tar.gz I will try to "scare away" people who is not comfortable with command-line. This is for your own good. Don't blame me (or maybe yes). Proper packaging will come in its own time.
  
• What works, what doesn't work?
  So far, I can tell for sure that recurrent events don't work. It's not a bug (yet): it's simply not implemented yet. This will be supported in the future.
  
• How does this relate to Erminig (not -NG?)
  I'm also maintaining Erminig (two-way sync between GPE calendar and Google Calendar), and Erminig-NG will eventually replace it. It is planned to support both GPE and "Fremantle-Calendar". By rewriting Erminig from scratch I will be able to eliminate a lot of unused code in Erminig, and this will allow me to understand more in details how Google Calendar works (please remember that I'm not the original author of Erminig, I just picked up the development. Erminig-NG on the other hand will be essentially a clean-room reimplementation of Erminig)
  
• Code is ugly (not really a question, I know).
  hum. Yes. I won't deny it. It's just the first draft. It will get cleaned.

---------------------

with that said, enjoy and please report any bug you may find!

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

beeing one of those you (should) scare away from alpha software, I am going to trail this thread hoping it will hit repos soon.

It looks promising, good luck working on it!

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

Really cool, this is the type of functinality the N900 needs, for now I am using a free 3rd party server that syncs the calender via MfE and it seems to work perfectly so far, I do however look forward to this so I can do it directly within my device.

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

Can you tell whats the difference with syncing using Mail for exchange and Google calendar? That works here fine.

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

Syncing with MfE can sync only the main calendar of Google (and not all sub-calendars); likewise only one local calendar can be synced with MfE. Erminig-NG can sync any local or Google Calendar.

Additionally, if I'm not mistaken, the fact that MfE can sync Google Calendar is an "accidental feature", that just happens to work. At the moment I can't find the post that explicitly said that...

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

ahh i was waiting for such an app to appear..
(if java was a supported language on maemo i already would have picked it up, now i wait for a good development environment on windows first...)

so this app talks directly to google api and syncs it right into the calendar of the N900 including multiply calendars?
Dont know exactly the status of the google api but does it also handle/give you back the shared/public cals i have? Because i have a few that i really want to sync up with (like sports calendar of my favorite club)

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

Can you post some screen shots?

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

Oh, no GUI. gotcha.

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

Will be following this closely. I was just in the process of writing an email to see if anyone was developing this type of application.

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

Thanks! Sounds great! I'll try to test it in the next couple days...

And, what am I supposed to answer? do I paste in one of the calendar links from the google calendar site? (sorry, I haven't tried this yet, just trying to figure out what the functionality is).

Does this support syncing from multiple calendars?

I use google calendar for 3 calendars, that are shared with 3 different groups of people (read-write) by sharing a google account and password.

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

I am eagerly following this, I can't wait to see it working!

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

Sorry, I should have been more explicit. Here's what happens:

erminig will scan your list of google calendars (automatically) and for each calendar on Google it will ask you if you wish to have that calendar in sync. You have to answer yes or no (y/n). If you answer y(es), it will create a local calendar with the same name as the Google one, and will keep it in sync.

Then it will scan the list of local N900 calendars and will ask you if you wish to keep that local calendar in sync with Google. If you answer y(es), erminig will create a calendar on Google, with the same name as the one on your device, and will keep it in sync.

Yes, support for syncing multiple calendars is supported (and is a big plus compared to Mail for Exchange, or nuevasync)

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

Sounds like a smart and user-friendly way to do it.

However, how does it know what "my" calendars are? as I was mentioning, I have 3 different google accounts that I use to share stuff with people. Where do I configure these? In the contacts application?

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

Oh, ok I get it now. I misunderstood your question.

for the moment (since no GUI is available), you have to edit a config file to specify your username/password. erminig will then fetch the calendars associated with that username. It is however not possible (yet) to specify multiple username/passwords in order to fetch multiple calendars spread on multiple account.

I have to admit that I never considered this possibility, but this is a great suggestion for an enhancement of erminig.

I hope that this time I understood correctly ;)

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

Hi Lorelei,

I just tested your tool. This is what I was waiting for - thanks for programming!

I found some issues:
- I have 7 local cals. Erminig does not recognize one of them (name: privat), so I can not sync this cal. I have no idea what so special about this one cal, but this cal I also can not rename ;-)
- Erminig crashed on a cal having an umlaut (Geschäft) in the title (I'm in austria). Renamed to "Geschaeft" to work around this problem, it's a minor issue.
- I looked at the python script and saw that there is no recognition if a local calendar is already on google. Looks like a complicated thing keeping tracking of all the IDs, so it's a "nice-to-have" thing.
- A big problem is that after Erminig created 5 cals on google, three of them had wrong titles - they had the google-id as the title (the looooong string with groop.calendar.google.com at the end). Erminig could sync all the events of these cals, however it's hard to recognize the real title of these calcs in Google now. After correcting the titles in google, Erminig could not sync with them anymore. A renaming back to the google-ids as titles did not help. The only way to get Erminig to work again is to delete Erminig's database. At the moment is this the biggest problem for me - any ideas?

It's a great start - thanks again. For the issues: I'd be happy to help in debugging...


Petibub

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

i much prefer the gpe calendar over the nokia implementation. the nokia calendar doesn't seem to have the same flexibility or ease of use. i look forward to having erminig working on the n900 as well as it does on my n800. moving the calendar from the n800 to the n900 by working through google was a hassle. i am waiting for an update to m4e so i can sync the nokia calendar with google while i wait for erminig. once i have the gpe calendar synced and working on the n900 i can decide what i want to do about keeping the n800. i love the n900.

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

Petibub:

thank you for testing Erminig-NG! I appreciate!
Please find below my comments on the bugs you encountered so far:

I suspect that this is a calendar created by Mail for Exchange. I had the same issue with a calendar called "N900" on my device. I'm also unable to delete it, and I'm a bit at a loss about why it doesn't appear in the database. With the upcoming changes I'm implementing, this problem may go away.
Bug fixed, thanks. I was able to reproduce it locally. Ahem...no, it looks like it has not been fixed. I have to find another workaround. I won't spend too much time on it, since the problem is specific to using the terminal for user-interaction, which I plan to remove when the GUI is done.
In the meantime I will simply display a dialog box with the question.
That's correct. In the previous version of Erminig the approach was a bit different: you had to have a local calendar and a google calendar and then you set up manually the relationship. In Erminig-NG new calendars are created on both sides.

I got hit by the very same problem while debugging the umlaut issue. I think that they are somewhat related (but I don't understand why), but I have to do some more testing and I'll get back to you with more information!

thanks again for the bug reports!

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

I found the reason: the calendar "cal_ti_calendar_private" is called "Privat" by the GUI. So, syncing the "cal_ti_calendar_private" I have my "Privat" on google too.

In the meantime I found that some other people have this problem with Google API, not only on python but also JAVA. Also, while testing I also could reproduce this bug using google web interface :-) I could not find any pattern (not with web interface nor with gd_client.InsertCalendar() from your script. At the end, I run this
just after gd_client.InsertCalendar() (your line 122) and sometimes one of the cals had the id as the title! So I think that this is some timing issue on google's side. Maybe an .UpdateCalendar() just after .InsertCalendar() would help?

However, we are not lost because I could rename the wrong-title cals using google web interface but I had to do it just after using "--sync-calendars" and before the first real syncing. Otherwise something went wrong with the query (l. 262) because
resulted in an empty feed regardless the changes I did in the cal.

I added some additional "print"s to get more infos during syncing and I will use it in the next few days, let's see what happens :-)

Thanks again,
Petibub

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

If someone is tired of typing (like me): here is the code for an application icon which calls erminig:

Replace the XXX by your path, save the file in /usr/share/applications/hildon/ and go to the application manager to start erminig.

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

OK, to summarize the issues that petibub reported:

- calendar named "Privat", or "Private" in the English version of Maemo: to sync that one, you have to accept to sync the calendar named "cal_ti_calendar_private"

- calendars with umlauts or any other diacritic, or any other non-ascii symbol: now Erminig won't crash. It will instead display the question about syncing a calendar in a UI dialog, to avoid problems with the terminal

- recognition of calendars already in google: no fix for the moment. But this is not really a bug, more a convenience. Can be "fixed" easily only once the GUI is done.

- calendars shown with IDs instead of title: according to petibub (thank you very much for the research on that bug!) the problem looks to be on Google's side and not on erminig's. As a workaround (which I'm not sure that it is effective, but it won't do any harm), a newly created calendar is immediately updated with its real name (even if it is already correct). This may help.

----------------

Upcoming version of Erminig-NG will contain the fixes shown above, plus a big change: instead of accessing directly the calendar database, I will use the published calendar API, shis is the correct way of interacting with the Calendar DB.

SInce there's no python binding to access CCalendar DB, I will write a C++ wrapper library that will be used by Erminig-NG to access easily the Calendaring API. This should be rather straightforward (famous last words), and I hope to get that new version out by this Sunday.

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

This looks like one of the most useful applications for the N900.
Thanks for your work lorelei!
What happens with birthdays from contacts? They are displayed in the Maemo Calendar but I don't know if they could be synced because they are based on the contacts entry. Also MfE has a problem with them and they are always +- 2 days different after a sync :(
Do you have any plans to support Google Contacts sync as well? Together with Google Mail IMAP MfE wouldn't be needed to have everything in sync with Google :)

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

Sorry for the off-topic, Lorelei - I'm just curious when feydrutha is adding work by having three separate accounts...

Isn't that 'doing it wrong'? I might be missing something.

Why have three separate accounts on Google with shared passwords, when you could just have one account each (and associated email address etc) and just create three shared calendars that you can all still access in the same way?

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

I agree that the way you describe of managing shared calendars is the most natural one, and probably the designed way of doing it. The only plausible use case I can see is that if the accounts are also used to manage a mail inbox attached to these calendars.

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

I have 2 accounts on google, one personal and one work one (from google apps). The personal one has my personal calendars, the work one, my work calendars (that were added by work, not me).

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

I have a one way sync c++ app using the ccalendar c++ api and libgdata that I started before becoming ill in December. It is at http://github.com/zaheerm/mgcalsync if you want ideas on using the API.

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

Yes, they are stored in a separate database, but I think that it should be relatively easy to add support for birthdays events. I have to check with the Contacts API to see if I can get those information easily. If it's possible, then I will add this capability.

There's no plan for the moment, but it can be added. In fact I already have initial support for contact sync between Google Contacts and GPE contacts, but I never released the code. I could recycle that code for contacts sync, but for the moment I will focus on calendar functions. Once it is stable enough, I may consider adding contacts support.

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

Thanks. I will keep that in mind if I get stuck with the code. But the ccalendar API seems to be rather straightforward...

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

Well... maybe it is doing it wrong. I guess I could switch to 'doing it right' with a single account and still have read-write access to the shared calendars. Maybe I'll try this later. Anyhow I don't seem to be the only one doing things this way (like zaheerm mentioned, with his work and personal calendars).

In general people sometimes have multiple accounts of services for a variety of reasons (like 2 skype accounts for instance) and applications should try to support that, like I think the contacts/conversations app does nicely on the n900. Of course, even without this feature this is still a useful application ;-)

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

I am sorry about the kinda off topic here, but where can I find MfE so I can give it a try?

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

A few instruction here:

http://wiki.maemo.org/Sync#Synchronising_with_Google

There was also a post (several of them in fact) on the forum with some more information...

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

This project looks very promising! One of my biggest annoyances is that the N900 doesn't have a proper sync possibilty with Google Calendar incl multiple calendars.
I can't test Erminig right now but I will certainly try it asap.
But you mentioned also a GUI, when do you think this is ready for testing?

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

That's a good question. I will avoid to make firm commitments to any date, but here's the release plan, very roughly (each line corresponds to a distinct release):

- fixes for items with diacritics; transition to a C wrapper to interact with the CCalendar API (by the end of this week)

- support for birthday calendar (to be confirmed once I've checked the API), support for recurrent events, code clean-up

- GUI, plus any other bug fix for issues found up to that point. This will also be the moment when I will properly package the application, since it will be also usable by people not comfortable with the command line.

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

Thank you buddy, it worked like a charm!

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

First of all, thanks for this! I started working on the same thing (2-way multi-calendar sync), but only got as far as playing with the python-gdata and wrapping the necessary parts of the calendar-backend API with C (accessed using python ctypes). Was figuring out recurring events in the calendar-backend API when I was last working on it.

I'm gonna ditch my own effort and start playing with this. Hopefully I'll have some contributions soon.

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

Just out of curiosity, why don't you guys use mail for exchange?
So far I have been able to update both ways.

:)

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

Already answered that in this post:

http://talk.maemo.org/showpost.php?p=469828&postcount=5

I'm sure other people will have additional arguments...

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

ahem, here's why I really love putting dates in my release plans. I will not be able to release a new pre-version of Erminig-NG by the end of the weekend as I initially thought.

I somewhat crashed my development environment, and coupled with the repository outage made for a not really productive weekend. Well, in any case I got everything running again, and I'm progressing toward my goal, albeit with a few days of delays. All my apologies!

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

why not let your personal one have access to the work one (or the other way around)
Or if you really dont want that, make another account that you just use for synching and let that account have the calendars of both personal and work?

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

Thanks for the "update". :)

Re: [Announce] Erminig-NG: two-way synchronization Google Calendar <-> Fremantle Calendar
 

I agree with Quanzel that this is one of the most important applications in development. I'm just surprised that it has taken this long for someone to pick up the task.

That said, is there anything other than feedback that the community can give you to help this along?