maemo.org - Talk
Page 1 of 2 1 2
Show 40 post(s) from this thread on one page

maemo.org - Talk (https://talk.maemo.org/index.php)
-   Community (https://talk.maemo.org/forumdisplay.php?f=16)
-   -   backup of maemo resources - a community effort , keep zen no panic ... (https://talk.maemo.org/showthread.php?t=87815)

www.rzr.online.fr 2012-11-11 12:46
backup of maemo resources - a community effort , keep zen no panic ...
 
Hi,

Let me open this thread to coordinate ourselves on the future of our contents ...

It *MAY* be too early to panic, but it's better to be safe than sorry ....

There are currently several efforts to backup our contents , so I opened this thread to prevent overlapping actions ...

I also feel it would be good to have a similar thread in some other future proof forum , can you suggest some ?

Check latest status update at :

* http://wiki.merproject.org/wiki/MeeGo

So far (but check previous url ) :

* Resource(abstract) | resource-URL | status | backup-location(URL) | responsible member(IRC nick, tmo nick...) |
* wiki.maemo.org | http:wiki.maemo.org | XML backup complete | http:maemo.merlin1991.at/wiki.... | (IRC) merlin1991 |




Update :

So to make it clear dont mirror *.maemo.org / *.meego.com but mirror upcoming mirrors if you have bandwith to spare ...

ie:

http://maemo-archive.wedrop.it.nyud.net

I am not spreading an emergency plan, but as some pple are considering this is time to do some stuff , and there is no way or reason to stop them, there should be some central point for coordination ... that's why I though if no one do that, then the servers will be killed for preservation efforts ... so let's act responsibly and dont act too fast , but and dont wait things to be done if nothing is planned...

Will update this if there is a clear plan of what is going on

Related:

* http://talk.maemo.org/showthread.php...00#post1293000
* http://talk.maemo.org/showthread.php?t=87815# Please dont suck maemo.org or meego.com servers let's coordinate 1st #MeeGo #Maemo #n950club !

qwazix 2012-11-11 13:32
Re: Survival alert , backuping infrastructure
 
I think it's a good plan to have backups but please do keep in mind that "doom is imminent" may be a self fulfilling prophecy, if everybody starts apt-mirroring in bulk.

Right now, the maintenance is sparse and it mostly relies on peoples good will and free time, so putting unnecessary strain on the servers might bring the end a day earlier.

I'm not suggesting not to backup. I'm just suggesting that one person is charged with each asset and he does the backup in the most light way possible (delays between successive downloads, speed capping etc).

I'd also suggest to change the title to something less panic-inducing, like

Coordinated infrastructure backup team - come to help.

ivgalvez 2012-11-11 13:33
Re: Survival alert , backuping infrastructure
 
Quote:
Originally Posted by www.rzr.online.fr (Post 1292955)
Hi,

Let me open this thread to coordinate ourselves on the future of our contents ...

It *MAY* be too early to panic, but it's better to be safe than sorry ....

There are currently several efforts to backup our contents , so I opened this thread to prevent overlapping actions ...

I also feel it would be good to have a similar thread in some other future proof forum , can you suggest some ?
Please don't panic.

Instead of starting dozens of doomsday threads, it would have been more useful to contact with Council.

We haven't received any notification about these systems going down, so it's more probably that it's simply problems of availability.

Maemo's wiki and Garage problems during this week happened while X-Fade was travelling, now he's looking into it. However, I'm not so sure about Meego's wiki.

If everyone starts doing their own backups, it's probably that we overload servers.

During last Council, it was decided to form a group of system administrators to take care of the infrastructure once it gets migrated, so it would be better if the backups where made by them in a coordinated and organised way.

www.rzr.online.fr 2012-11-11 13:36
Re: Survival alert , backuping infrastructure
 
Read why I wrote on the 1st line : "Let me open this thread to coordinate ourselves on the future of our contents ..."

So to make it clear dont mirror *.maemo.org / *.meego.com but mirror upcoming mirrors if you have bandwith to spare ...

ie:

http://maemo-archive.wedrop.it.nyud.net

Dave999 2012-11-11 13:39
Re: Survival alert , backuping infrastructure
 
So have the council received any information of what will happen so that nokia not just pull the plug one day. It doesn't mean it will be there just becouse you don't here anything. So council will provide information related to this issue soon?

sixwheeledbeast 2012-11-11 13:41
Re: Survival alert , backuping infrastructure
 
Have confidence in your recently elected.
Panic now and cause more panic, IMO.
UK fuel strikes spring to mind...

freemangordon 2012-11-11 13:53
Re: Survival alert , backuping infrastructure
 
Quote:
Originally Posted by ivgalvez (Post 1292980)
Please don't panic.

Instead of starting dozens of doomsday threads, it would have been more useful to contact with Council.

We haven't received any notification about these systems going down, so it's more probably that it's simply problems of availability.

Maemo's wiki and Garage problems during this week happened while X-Fade was travelling, now he's looking into it. However, I'm not so sure about Meego's wiki.

If everyone starts doing their own backups, it's probably that we overload servers.

During last Council, it was decided to form a group of system administrators to take care of the infrastructure once it gets migrated, so it would be better if the backups where made by them in a coordinated and organised way.
The problem as I see it is the lack of info what exactly is going on "behind the scenes". Not to say that AFAIK merlin1991 still does not have full access to repos. And I would prefer to not overtrust Nokia to not pull the plug suddenly.

It is not about panic, we are just running out of time.

ivgalvez 2012-11-11 14:02
Re: Survival alert , backuping infrastructure
 
Quote:
Originally Posted by freemangordon (Post 1292992)
The problem as I see it is the lack of info what exactly is going on "behind the scenes". Not to say that AFAIK merlin1991 still does not have full access to repos. And I would prefer to not overtrust Nokia to not pull the plug suddenly.

It is not about panic, we are just running out of time.
You can check recent activity through Council's meeting logs, minutes soon to be published.

ivgalvez 2012-11-11 14:04
Re: Survival alert , backuping infrastructure
 
Quote:
Originally Posted by JCDenton (Post 1292986)
Maybe Council should contact people about coordinated plan for backups/migration, instead of requesting the other way around?

It's hard to not panic, when it's less than two months to doomsday. We're hearing the same things over and over again...
We are in conversations with Nemein for that. It has taken a while but fortunately communication has resumed (having a legal entity to talk through is a good tool for that).

CrashandDie 2012-11-11 14:09
Re: Survival alert , backuping infrastructure
 
Hi,

I run maemo-archive.wedrop.it. It is now officially "up and running", considering the full mirror of the following repositories is complete:

catalogue.tableteer.nokia.com
download.videolan.org
downloads.maemo.nokia.com
maemo.jacekowski.org
maemo.merlin1991.at
mohammadag.xceleo.org
repo.apps.formeego.org
repository.maemo.org
repository.sygic.com
www.amsn-project.net
www.joikushop.com

With the exception of the source repository of chinook extras and extras-devel (repository.maemo.org) due to garbage index files.

I've also removed the kolab.org repositories, which crash wget.

The cron will update/remirror everything every day at noon CEST.

qwazix 2012-11-11 14:11
Re: Survival alert , backuping infrastructure
 
Joerg_rw Crashanddie has already backuped all the repos (with the help of Joerg_rw) except meego and wiki here:

http://maemo-archive.wedrop.it/

(http://talk.maemo.org/showthread.php...61#post1292861)

so these assets are already safe. We will contact people that have expressed their will to help with specific tasks for other assets.

@rzr Please keep a list of assets and their status in the first post if possible.

joerg_rw 2012-11-11 14:15
Re: Survival alert , backuping infrastructure
 
guys, I'm seriously losing track of all those parallel threads here, and each of them a *poll*. Why *that*?
/j
(qwazix: it been crashanddie who actually did the backup, I only helped a tiny bit and posted the link here)

www.rzr.online.fr 2012-11-11 14:48
Re: Survival alert , backuping infrastructure
 
the other one is only for the wiki, that one is general ...

anyway it looks today community has a a backup of most of the "offline" things , except wiki.meego.com , I reported a bug let see where it is going ... apps4meego or ovi is an other thing

jalyst 2012-11-11 14:51
Re: Survival alert , backuping infrastructure
 
Are we sure that's absolutely everything (at least of the "offline" stuff) backed-up now (apart from what rzr mentioned), how do we know? (sorry for the dunce Qn, just a bit paranoid)

joerg_rw 2012-11-11 15:09
Re: Survival alert , backuping infrastructure
 
Quote:
Originally Posted by jalyst (Post 1293009)
Are we sure that's absolutely everything (at least of the "offline" stuff) backed-up now (apart from what rzr mentioned), how do we know? (sorry for the dunce Qn, just a bit paranoid)
Jalyst, please check what's in the wedrop.it server and compare to what you expect to find there. Any hints on missed stuff welcome. And of course a mirror started ~12h ago couldn't have anything that's been offline at that time and since.
Also a general note: Nokia never had any objections on mirroring their repos, it's been actively supported by Nokia iirc, and every of you is welcome to run his own repo mirror. Just please don't start all those projects this weekend, the original servers suffer ;-D

/j

CrashandDie 2012-11-11 17:29
Re: backup of maemo resources - a community effort , keep zen no panic ...
 
1 Attachment(s)
Attached is the full list of repositories that is mirrored.

The mirror contains the full debian packages and sources archive for:
- Community SSU (stable, testing, thumb and devel)
- Chromium and code reset (jacekowski)
- aMSN (+ tcltk)
- MohammadAG's repo (n9incepted)
- extras (scirocco, mistral, gregale, bora, chinook, diablo, fremantle)
- extras-testing (fremantle)
- extras-devel (gregale, bora, chinook, diablo, fremantle)
- community (diablo, fremantle)
- community-testing (diablo, fremantle)
- maemo ossw
- maemo 1.1 (free & non-free)
- maemo 1.1 extras
- maemo 2.0 (free & non-free)
- maemo 2.1 (free & non-free)
- maemo 2.2 (free & non-free)
- maemo 3.0 (free & non-free)
- maemo 4.0 (free & non-free)
- maemo 4.0.1 (free & non-free)
- maemo 4.1 (free & non-free)
- maemo 4.1.1 (free & non-free)
- maemo 4.1.2 (free & non-free)
- maemo 5.0 (nokia-binaries, free & non-free)
- rtcomm (bora, chinook, diablo)
- videolan (fremantle)
- Tableteer Catalogue (bora, chinook, diablo)
- ForMeego (fremantle: apps, apps-testing)
- Sygic (Fremantle)

thedead1440 2012-11-11 17:31
Re: backup of maemo resources - a community effort , keep zen no panic ...
 
how about the harmattan sdk repo? Shouldn't it be mirrored too?

aStrike 2012-11-11 23:39
Re: backup of maemo resources - a community effort , keep zen no panic ...
 
I also think that harmattan sdk repository should be mirrored. It has some really crucial packages for n9 and n950 development.

brkn 2012-11-12 01:24
Re: backup of maemo resources - a community effort , keep zen no panic ...
 
Quote:
Originally Posted by aStrike (Post 1293166)
I also think that harmattan sdk repository should be mirrored. It has some really crucial packages for n9 and n950 development.
I second that. Where can I upload the 7.6 GB? I have a fresh mirror merely a week old.

jalyst 2012-11-12 07:05
Re: Survival alert , backuping infrastructure
 
Quote:
Originally Posted by joerg_rw (Post 1293017)
Also a general note: Nokia never had any objections on mirroring their repos, it's been actively supported by Nokia iirc, and every of you is welcome to run his own repo mirror. Just please don't start all those projects this weekend, the original servers suffer ;-D
Would gladly have done this 9mth ago, now I barely have the gear (sold most of my personal stuff, long story) nor the time :(
But I'm glad to see others stepping up... That is why this community is so special....

ivgalvez 2012-11-12 10:42
Re: backup of maemo resources - a community effort , keep zen no panic ...
 
Received some news from Nemein about the recent problems in infrastructure.

lma 2012-11-13 00:10
Re: backup of maemo resources - a community effort , keep zen no panic ...
 
Quote:
Originally Posted by CrashandDie (Post 1293065)
Attached is the full list of repositories that is mirrored.
Careful, several of these (eg tableteer, nokia-binaries and various non-free repo components) are proprietary and cannot be legally redistributed without licence from the copyright holders.

Apart from everything else, creating public mirrors of these right now is almost certain to be seen as a show of very bad faith and torpedo any chances of successful negotiations between Nokia and the Hildon Foundation.

pali 2012-11-13 15:33
Re: backup of maemo resources - a community effort , keep zen no panic ...
 
Quote:
Originally Posted by CrashandDie (Post 1293065)
Attached is the full list of repositories that is mirrored.

The mirror contains the full debian packages and sources archive for:
- Community SSU (stable, testing, thumb and devel)
- Chromium and code reset (jacekowski)
- aMSN (+ tcltk)
- MohammadAG's repo (n9incepted)
- extras (scirocco, mistral, gregale, bora, chinook, diablo, fremantle)
- extras-testing (fremantle)
- extras-devel (gregale, bora, chinook, diablo, fremantle)
- community (diablo, fremantle)
- community-testing (diablo, fremantle)
- maemo ossw
- maemo 1.1 (free & non-free)
- maemo 1.1 extras
- maemo 2.0 (free & non-free)
- maemo 2.1 (free & non-free)
- maemo 2.2 (free & non-free)
- maemo 3.0 (free & non-free)
- maemo 4.0 (free & non-free)
- maemo 4.0.1 (free & non-free)
- maemo 4.1 (free & non-free)
- maemo 4.1.1 (free & non-free)
- maemo 4.1.2 (free & non-free)
- maemo 5.0 (nokia-binaries, free & non-free)
- rtcomm (bora, chinook, diablo)
- videolan (fremantle)
- Tableteer Catalogue (bora, chinook, diablo)
- ForMeego (fremantle: apps, apps-testing)
- Sygic (Fremantle)
Here is (for a long time) list of all maemo and harmattan repositories: https://wiki.maemo.org/Apt-mirror

CrashandDie 2012-11-13 19:44
Re: backup of maemo resources - a community effort , keep zen no panic ...
 
Quote:
Originally Posted by lma (Post 1293591)
Careful, several of these (eg tableteer, nokia-binaries and various non-free repo components) are proprietary and cannot be legally redistributed without licence from the copyright holders.
I'll stop mirroring any for which I get a complaint. The repositories are accessible without username and password, and did not display any warnings while cloning them, so... "Fair game". Please note that I have no problem whatsoever with removing any material that might be infringing.

Quote:
Originally Posted by lma (Post 1293591)
Apart from everything else, creating public mirrors of these right now is almost certain to be seen as a show of very bad faith and torpedo any chances of successful negotiations between Nokia and the Hildon Foundation.
How so? Quite the contrary, it shows how important we deem the archives.

Quote:
Originally Posted by pali (Post 1293842)
Here is (for a long time) list of all maemo and harmattan repositories: https://wiki.maemo.org/Apt-mirror
Aye, that's the list I initially used, and removed/commented a few that blocked the mirroring process.

misterc 2012-11-14 21:13
Re: backup of maemo resources - a community effort , keep zen no panic ...
 
i bought my last N900 (new) a little over a year ago.
N9s are still sold new, with 2 years warranty.

a lot has changed at NOKIA in the last two years (since before burning hara-kiri :mad:) but still.
if NOKIA wants to be able to even only hold what meagre market share they have left, they can't afford to shut down the software repositories for (partly) brand new items.
can they?
:confused:

suzughia 2012-11-17 09:14
Re: backup of maemo resources - a community effort , keep zen no panic ...
 
anyone has the harmattan-sdk necessary files that are downloaded from the python script?

I mean those files:

Code:
04d21308fd51acf926c757666d056080  arm-qemu-rm680_linux_i686.tar.gz
c348bb5cefb97ca973365b7783a7db98  arm-qemu-rm680_linux_x86_64.tar.gz
b557dac05ecd3445c5c49bfe340cdb12  arm-psdk-nokiameegoapi-beta-rootstrap.tgz
6f70fc3268be51d8be25f0120b5e1d25  arm-public-sdk-beta-rootstrap.tgz
14edf61c6860d078fe9aaf80cbfd319b  i386-psdk-nokiameegoapi-beta-rootstrap.tgz
c64f60bc2c0d0b89c5b5bb705da1077b  i386-public-sdk-beta-rootstrap.tgz
317b3283f4412993bcba4bd437537855  LICENSE
d17ce69699d16a10f879eb64607d218b  arm-public-sdk-beta2-rootstrap.tgz
3539f732595bd3836217c7307a57d144  i386-public-sdk-beta2-rootstrap.tgz
743a5f7c11c790291d7651c0246bc801  arm-public-sdk-beta3-rootstrap.tgz
9ac86ebcc3adfdc82e6c49bc8fa5b3c0  i386-public-sdk-beta3-rootstrap.tgz
ce392089b83dadfb47d4c7a1c0628299  arm-public-sdk-rootstrap.tgz
f2c9fc2c780d0a21f41ecd1067add387  index
b30034faef2b0bd0a211e2f7808b15cc  i386-public-sdk-rootstrap.tgz

brkn 2012-11-17 10:18
Re: backup of maemo resources - a community effort , keep zen no panic ...
 
Quote:
Originally Posted by suzughia (Post 1295209)
anyone has the harmattan-sdk necessary files that are downloaded from the python script?

I mean those files:

Code:
04d21308fd51acf926c757666d056080  arm-qemu-rm680_linux_i686.tar.gz
c348bb5cefb97ca973365b7783a7db98  arm-qemu-rm680_linux_x86_64.tar.gz
b557dac05ecd3445c5c49bfe340cdb12  arm-psdk-nokiameegoapi-beta-rootstrap.tgz
6f70fc3268be51d8be25f0120b5e1d25  arm-public-sdk-beta-rootstrap.tgz
14edf61c6860d078fe9aaf80cbfd319b  i386-psdk-nokiameegoapi-beta-rootstrap.tgz
c64f60bc2c0d0b89c5b5bb705da1077b  i386-public-sdk-beta-rootstrap.tgz
317b3283f4412993bcba4bd437537855  LICENSE
d17ce69699d16a10f879eb64607d218b  arm-public-sdk-beta2-rootstrap.tgz
3539f732595bd3836217c7307a57d144  i386-public-sdk-beta2-rootstrap.tgz
743a5f7c11c790291d7651c0246bc801  arm-public-sdk-beta3-rootstrap.tgz
9ac86ebcc3adfdc82e6c49bc8fa5b3c0  i386-public-sdk-beta3-rootstrap.tgz
ce392089b83dadfb47d4c7a1c0628299  arm-public-sdk-rootstrap.tgz
f2c9fc2c780d0a21f41ecd1067add387  index
b30034faef2b0bd0a211e2f7808b15cc  i386-public-sdk-rootstrap.tgz
You could also do a manual installation.

suzughia 2012-11-17 10:25
Re: backup of maemo resources - a community effort , keep zen no panic ...
 
Quote:
Originally Posted by brkn (Post 1295223)
You could also do a manual installation.
Yes but also the manual installation still needs those files hosted on harmattan-dev that it's offline as for now. Thanks for your help anyway!

jalyst 2012-11-18 16:57
Re: Survival alert , backuping infrastructure
 
Quote:
Originally Posted by ivgalvez (Post 1292996)
We are in conversations with Nemein for that. It has taken a while but fortunately communication has resumed (having a legal entity to talk through is a good tool for that).
So following on from here:
http://talk.maemo.org/showthread.php...18#post1294118
And your most recent update:
http://talk.maemo.org/showthread.php...57#post1294657
Are you able to advise (in this more appropriate thread) how things are progressing?
Your last update was relatively recent so no uber urgency, thanks for your time!

siva 2012-11-18 21:52
Re: backup of maemo resources - a community effort , keep zen no panic ...
 
Quote:
Originally Posted by brkn (Post 1293180)
I second that. Where can I upload the 7.6 GB? I have a fresh mirror merely a week old.
Is this the entire repo from this:

deb http://harmattan-dev.nokia.com/ harmattan/sdk free non-free ?

Any luck uploading so far, mate?

alex25 2012-11-19 17:29
Re: backup of maemo resources - a community effort , keep zen no panic ...
 
Quote:
Originally Posted by brkn (Post 1293180)
I second that. Where can I upload the 7.6 GB? I have a fresh mirror merely a week old.
It's a shame the server is still not up and still no information about when the server is up again.
I need to install the openssl packages. Can you provide me with a copy of the directory /pool/harmattan/free/o/openssl
Thanks

brkn 2012-11-19 17:32
Re: backup of maemo resources - a community effort , keep zen no panic ...
 
Quote:
Originally Posted by alex25 (Post 1295960)
It's a shame the server is still not up and still no information about when the server is up again.
I need to install the openssl packages. Can you provide me with a copy of the directory /pool/harmattan/free/o/openssl
Thanks
harmattan-openssl

brkn 2012-11-19 17:34
Re: backup of maemo resources - a community effort , keep zen no panic ...
 
Quote:
Originally Posted by siva (Post 1295684)
Is this the entire repo from this:

deb http://harmattan-dev.nokia.com/ harmattan/sdk free non-free ?

Any luck uploading so far, mate?
free, non-free, nokia-binaries. Since nobody was interested I archived those files.

student4life 2012-11-20 02:57
Re: backup of maemo resources - a community effort , keep zen no panic ...
 
Quote:
Originally Posted by brkn (Post 1295965)
free, non-free, nokia-binaries. Since nobody was interested I archived those files.
can you upload it to some server, maybe
http://maemo-archive.wedrop.it/

missing few libraries required for development

brkn 2012-11-20 09:58
Re: backup of maemo resources - a community effort , keep zen no panic ...
 
Quote:
Originally Posted by student4life (Post 1296081)
can you upload it to some server, maybe
http://maemo-archive.wedrop.it/

missing few libraries required for development
If someone sends me FTP/SSH login I will happily upload everything.

peterleinchen 2012-11-20 10:48
Re: backup of maemo resources - a community effort , keep zen no panic ...
 
You could/should contact CrashandDie.

But it could be legal reasons, why it is not mirrored (like this).

brkn 2012-11-22 09:42
Re: backup of maemo resources - a community effort , keep zen no panic ...
 
Quote:
Originally Posted by peterleinchen (Post 1296208)
You could/should contact CrashandDie.

But it could be legal reasons, why it is not mirrored (like this).
I contacted him 2 days ago. Didn't get any response though. Will post back once I have a reply from him.

CrashandDie 2012-11-22 11:45
Re: backup of maemo resources - a community effort , keep zen no panic ...
 
Hi, sorry for not replying to your PM.

This might be a weird decision, but I'm going to decline mirroring the file you've provided. I'd rather only mirror things that have been actually mirrored---not copies of mirrors of clones.

The reason for this is simple: guaranteeing data integrity. There are a number of things that could have happened; not that I mean to imply foul play, but I'd rather not host anything from third parties. This is even more the case considering that
  • the server is down for technical issues, not a conscious decision on Nokia's part, and,
  • the archive available for mirroring is incomplete.

I do not wish to create more confusion: Why would an incomplete mirror be up? Etc.

So, unless there are very convincing arguments, I'm going to decline mirroring the partial harmattan-dev backup.

brkn 2012-11-22 12:04
Re: backup of maemo resources - a community effort , keep zen no panic ...
 
Quote:
Originally Posted by CrashandDie (Post 1297192)
Hi, sorry for not replying to your PM.

This might be a weird decision, but I'm going to decline mirroring the file you've provided. I'd rather only mirror things that have been actually mirrored---not copies of mirrors of clones.

The reason for this is simple: guaranteeing data integrity. There are a number of things that could have happened; not that I mean to imply foul play, but I'd rather not host anything from third parties. This is even more the case considering that
  • the server is down for technical issues, not a conscious decision on Nokia's part, and,
  • the archive available for mirroring is incomplete.

I do not wish to create more confusion: Why would an incomplete mirror be up? Etc.

So, unless there are very convincing arguments, I'm going to decline mirroring the partial harmattan-dev backup.
I do indeed fully understand your decision and most probably would have acted the same way. I have all "Packages" files containing hashes (SHA1, SHA-256, MD5) however one can not verify them. Thus I can not prove data integrity which is the most crucial thing for packages.
Nevertheless upon request I will continue to provide files requested from community members.

jackburton 2012-11-25 13:27
Re: backup of maemo resources - a community effort , keep zen no panic ...
 
While the mirror appears incomplete, it has been quite useful for me to get the kernel sources for the N950 and other build tools. I'm using the mirror along with harmattan SDK in a chroot on the device itself http://talk.maemo.org/showthread.php?t=86158


All times are GMT. The time now is 08:26.
Page 1 of 2 1 2
Show 40 post(s) from this thread on one page

vBulletin® Version 3.8.8