More testers needed to push apps out of extras-testing

[Venomrush]

I believe that a lot of people here are not aware of the existence of this page:

http://maemo.org/packages/repository/qa/fremantle_extras-testing

Apps are getting getting backlog back to September.

Over the past few days, the number of apps entering extras-testing has been roughly 5-10 apps/version everyday. I think there's only about 10-20 people doing the commenting and voting so far => major backlog.

I don't see a lot of people voting or commenting on the apps so would appreciate if you have spare time to check out that page, write a few short comments to improve the apps which you will see in Extras.

[techdork]

I would if they'd deliver my phone already :|

[z3phyr]

i didnt wanna try the extras testing and dev apps because they said i need to know what Im doing?

[generationally]

So what exactly does testing imply?

Just checking that it installs and seems to do what it is supposed to?

Or does the tester need to check, e.g., that it is properly optified?

[kaz911]

it would help if the pages loaded within 10 secs and not 30-60 secs like now.
....

[pelago]

So what exactly does testing imply?

Just checking that it installs and seems to do what it is supposed to?

Or does the tester need to check, e.g., that it is properly optified?

See http://wiki.maemo.org/Extras-testing

[edgedemon]

A clear check list for unexperienced testers would be a good idea?
As well as a guide for how the phone should be set up as that will affect battery life?
I for one wouldn't have a clue how to tell if something has been optified?

edit - just seen previous link..

[ossipena]

See http://wiki.maemo.org/Extras-testing

http://wiki.maemo.org/Help_testing_software

those need to be somewhat merged or something, now two articles are very much duplicates of others...

e: and feel free to improve that article for example with info edgedemon reguested.

[bbin]

I think it would be helpful to give some information on how the votes are made and who can wote...

EDIT: ok thanks ossipena

[twaelti]

What about creattng a maemo-testers mailing list as an in-between of maemo-developers and maemo-users?
This might be handy to have a forum just about testing, where testers can ask if something is unclear and developers can explain, ask for more details, annnounce new apps and demand testers. Would streamline the process and improve communications, which currently happen inside the comments section of extras-testing.

[benny1967]

i wouldn't even be afraid of bricking my device while testing. it's just that i don't know how to easily check for the most basic things like is it optified, does it uninstall cleanly, are there memory issues or does it drain battery.... i read the wiki page, but it seems to be written for developers, not for end users.

so it boils down to: i don't feel competent enough to give my opinion.

[YoDude]

Benny, the thanks is for the first part of your post and not the last part. :p

Checking for optification is where there is also confusion with me. The installed "Memory" app reports the 2 gig space but does it report the rootfs space as well? How do we tell the difference? What command line tools are useful with all this? etc., etc., yabba, dabba, doo.

[benny1967]

YoDude, the thanks is mainly for the first part of your post. :D

[Relativistic]

Doesn't help that the website is being ran on a 286.
I mean come on, this is unacceptable for a multibillion euro company.

This is comical: http://wiki.maemo.org/ISP_Move
"Agreement with the new ISP has been signed. Now we are waiting for them to get the hardware to the server room." This has been the status for the last 2 months.

[benny1967]

Relativistic, this is annoying, but only a minor nuisance; you will need to invest time to test the application... hours, maybe days to be sure it does what it claims to do, that it doesn't start acting up after being shut down and restarted, etc ... only then you interact with maemo.org to vote thumbs up or down and give a comment. - That this last part is taking a minute longer than it should needn't prevent anyone from voting if he already has tested the application.

[HugoSon]

"Agreement with the new ISP has been signed. Now we are waiting for them to get the hardware to the server room." This has been the status for the last 2 months.

I totally agree - unfortunately fits into the overall behavior of Nokia regarding the N900/maemo...

[Rob1n]

Checking for optification is where there is also confusion with me. The installed "Memory" app reports the 2 gig space but does it report the rootfs space as well? How do we tell the difference? What command line tools are useful with all this? etc., etc., yabba, dabba, doo.

The "Memory" app doesn't report the rootfs space at all. From the command-line you can run "df -h" to see the free space - the two rows you'll want to look at are "rootfs" (mounted on /) and "/dev/mmcblk0p2" (mounted on /home - /opt is a symlink to /home/opt).

[adrianp]

I believe that a lot of people here are not aware of the existence of this page:

http://maemo.org/packages/repository/qa/fremantle_extras-testing

It would be useful if this page allowed some sort of sorting or filtering or at least displaying all packages in testing so one could use ones browser's search to look for packages.

I've been trying to help use and test -testing packages but find the current maemo.org web response makes the process tedious (garage often seems to take forever to display).

[real_per]

It would be a lot easier, if I could create an account https://garage.maemo.org/account/register.php :)

Or gosh, I don't know, use one of my existing 2 Maemo.org accounts, and not create a 3rd (!) (Forum, bugtracker, extras)

Also, why do some of the applications I have already installed from extras also appear in extras-testing ? Very confusing.

[mikkov]

Also, why do some of the applications I have already installed from extras also appear in extras-testing ? Very confusing.

Because they are updates for the old versions

[YoDude]

The "Memory" app doesn't report the rootfs space at all. From the command-line you can run "df -h" to see the free space - the two rows you'll want to look at are "rootfs" (mounted on /) and "/dev/mmcblk0p2" (mounted on /home - /opt is a symlink to /home/opt).

Mine reports:
rootfs use 76%
mmcblk0p2 use 6%

Is this normal?

How do I clear it up after installing then uninstalling apps from "testing"?

[blubbi]

The site is just not usable at the moment!
(http://maemo.org/packages/)

Submitting my comment took several minutes if it worked at all.

Not very encouraging to comment and vote.

It's about time the new hardware comes online...


And thanks to the connection I double and triple posted two of my comments...

Cheers
Bjoern

[Sasler]

The fact that it's impossible to find a link to http://maemo.org/packages/ doesn't help. At least I can't find it anywhere in maemo.org (http://maemo.org/).

[Flandry]

Finding Things and Getting Started
You don't need to get to /packages to do testing. If you go to the Extras-testing wiki page (http://wiki.maemo.org/Extras-testing), it links directly to the queue of packages in extras-testing, and in chronological order, to boot (the ones at the top have been waiting the longest). The link (http://maemo.org/packages/repository/qa/fremantle_extras-testing) directly there is also given in the first post. To reiterate an earlier post, you can add the testing repository like this (http://wiki.maemo.org/Help_testing_software).

Which to Test
When picking an app to test, note that those that already have 10 or more karma are "done". Don't waste your time on those. Ditto if the package is < -5 karma: such an app obviously needs to be fixed and re-released. Make sure you are testing the right version of the app: there may be an earlier version already in extras. In fact, there may even be a third version in extras-devel, but that won't show up unless you enable that repository. To check, see what version application manager is showing to be installed and make sure it matches the version in the extras-testing list..

In the case you have the extras version installed, the newest version should show up as an available upgrade if you have the extras-testing repository enabled. You should uninstall the extras version first (rather than upgrading directly) so you can properly check for optification.

Testing for Optification
"Optification" means moving anything of significant size out of the root partition and into the 2 GB /opt partition so that the very cramped 256 MB rootfs doesn't fill up (which can really ruin your day). You see disk usage by using the disk free command (df).
Mine reports:
rootfs use 76%
mmcblk0p2 use 6%

Is this normal?
Yes, that's normal. What you want to do is run
df -h | grep -e rootfs -e mmcblk0p2
before and after you install and see how much the usage of each changes. If more than, say, 300 kb (0.3 MB) are added to usage of rootfs, it's too much and the package needs to be optified (so vote it down with that comment). BTW, the above command just filters out the unnecessary lines; you can just type df -h if you want. The '-h' gives you the numbers in "human-readable" numbers like MB instead of bytes/blocks/kb.

One of the other tests is that the package uninstalls cleanly. A (partial) way to check this is to uninstall it and see if the df returns to what it was before you installed.

Commenting
Please don't let the server speed stop you from testing. Just start it doing something and do something else for a while. Your browser has tabs? In my experience, comments always go through but don't always show up right away, so just assume they work.

Plug
Anyone who already has uqm installed could start out with an easy test subject: uqm-3do-data (http://maemo.org/packages/package_instance/view/fremantle_extras-testing_free_armel/uqm-3do-data/0.6.7-0maemo/). It's not an actual program so there's a bit less to test. Check that i optified it right and that it installs and uninstalls cleanly and actually adds the 3DO audio to your uqm experience and that's most of it.

[blubbi]

The fact that it's impossible to find a link to http://maemo.org/packages/ doesn't help. At least I can't find it anywhere in maemo.org (http://maemo.org/).

Ack., there is no way to the repositories.

Really not helpful.

Kind regards
Bjoern

[epage]

My biggest complaint about some of the testing criteria is they are binary "Yes" or "No" and do not need everyone to check them. Its even worst for optification because that can and has been automated but not integrated into the process (due to adding more time to the promotion process). I feel like its a waste of my time testing for these things. Instead I focus on bugs, being able to get out of the app (Ctrl + Backspace working if nothing else), are there obvious trademark issues (like using Maemo in the name).

For battery I mainly focused on the obvious things like for games should they pause when they are running in the background? When a game is paused and in the background, should it be playing music, doing animations ,etc draining the battery?

I another issue that has been brought up before that I think is on a task list is that some apps need to be removed from Extras Testing. Two examples "Maemo Recorder" renamed to "Recorder" and "Maemo Periodic" renamed to "Copernicum" due to trademarks but both version of the app still appearing.

Though I do admit I need to go back and rate some applications..

[DannStarr]

I share the concerns of many other posters, and would love to get in on the testing of apps.

What would make me a lot happier is a clear guide on exactly how to get my device back to how it is now once i screw it up. My Rootfs partition already shows apparently 74% use which is worrying seeing as ive hardly installed anything?

[j2001m]

why as the FM Radio Player gone into that list and is now not listed at all in the main list, is there a big problem?????

[thorbo]

I think this will also be mitigated with what I mentioned yesterday:

http://talk.maemo.org/showthread.php?t=37447

I have been testing, but frankly without knowing the differences (in the case up upgraded packages) or even the basics from the developer of what may not be working, etc. it can be a little frustrating. -- Thor

[Fargus]

Finding Things and Getting Started
You don't need to get to /packages to do testing. If you go to the Extras-testing wiki page (http://wiki.maemo.org/Extras-testing), it links directly to the queue of packages in extras-testing, and in chronological order, to boot (the ones at the top have been waiting the longest). The link (http://maemo.org/packages/repository/qa/fremantle_extras-testing) directly there is also given in the first post. To reiterate an earlier post, you can add the testing repository like this (http://wiki.maemo.org/Help_testing_software).

Which to Test
When picking an app to test, note that those that already have 10 or more karma are "done". Don't waste your time on those. Ditto if the package is < -5 karma: such an app obviously needs to be fixed and re-released.

Testing for Optification
"Optification" means moving anything of significant size out of the root partition and into the 2 GB /opt partition so that the very cramped 256 MB rootfs doesn't fill up (which can really ruin your day). You see disk usage by using the disk free command (df).

Yes, that's normal. What you want to do is run
df -h | grep -e rootfs -e mmcblk0p2
before and after you install and see how much the usage of each changes. If more than, say, 300 kb (0.3 MB) are added to usage of rootfs, it's too much and the package needs to be optified (so vote it down with that comment). BTW, the above command just filters out the unnecessary lines; you can just type df -h if you want. The '-h' gives you the numbers in "human-readable" numbers like MB instead of bytes/blocks/kb.

One of the other tests is that the package uninstalls cleanly. A (partial) way to check this is to uninstall it and see if the df returns to what it was before you installed.

Commenting
Please don't let the server speed stop you from testing. Just start it doing something and do something else for a while. Your browser has tabs? In my experience, comments always go through but don't always show up right away, so just assume they work.

Plug
Anyone who already has uqm installed could start out with an easy test subject: uqm-3do-data (http://maemo.org/packages/package_instance/view/fremantle_extras-testing_free_armel/uqm-3do-data/0.6.7-0maemo/). It's not an actual program so there's a bit less to test. Check that i optified it right and that it installs and uninstalls cleanly and actually adds the 3DO audio to your uqm experience and that's most of it.

Can we get this into the Wiki rather than lost in a thread?

[stara]

After enabling extra-testing, I noticed that the yellow box started blinking. Xournal and Load-applet had new versions available. I assume from extra-testing side?

While I wait garage registration complete, very very slow, quick notes:

Xournal seems to work, thumbs up.

Load-app seems to require reboot :-/ My first reboot since getting N900 (been running it two weeks straight).

[Flandry]

Can we get this into the Wiki rather than lost in a thread?

It's a lot more time-consuming to edit the wiki, but i hacked most of that into the http://wiki.maemo.org/Help_testing_software page. I was going to just merge everything into extras-testing, but some of the material seemed a bit verbose and trivial for that page. If someone else can find a good way to consolidate everything into extras-testing and redirect the help-testing-software page to it, that would be great. Just remember that we need a concise but simple description of how to test. The extras-testing page is getting rather long, so the best bet might be to split it into one page describing the nature of the repo and how to add it, and put everything else into the qa checklist page it links to.

So much to do, so little time. Anyway, fresh pair of eyes now that i distilled that new info into the wiki would be good. We really do need a better/more complete testing guide.

[Laughing Man]

It would be helpful if the authors of those applications indicated if they were optified or not.

Edit: Some comments on those apps do tell you if it's optified or not. But it would be helpful if it was indicated in the overview itself.

[Bratag]

It would be helpful if the authors of those applications indicated if they were optified or not.

Edit: Some comments on those apps do tell you if it's optified or not. But it would be helpful if it was indicated in the overview itself.

Agree. Optified indicator would be very beneficial

[Viipottaja]

What is "optified"? I assume its not "optimized" as everyone seems to use that term.

Clearly, if I don't even know that I should not be alpha/beta testing (see , I do know a couple of geek terms though)! :D

[floffe]

What is "optified"? I assume its not "optimized" as everyone seems to use that term.

Clearly, if I don't even know that I should not be alpha/beta testing (see , I do know a couple of geek terms though)! :D

Being optified means it installs most stuff into /opt rather than / (rootfs), because the latter has very limited space available due to being on a 256MB flash chip. /opt, on the other hand, is on a 2GB partition on the internal memory card. Apps over 500kB are supposed to install mostly to /opt.

[SubCore]

What is "optified"?

there's a nice wiki page (http://wiki.maemo.org/Opt_Problem) about that topic.

the current solution in place (referred to as "optification") is the second on that page... moving everythign >500kB to /opt.

[hypoxic]

Is there any sort of list of what tests are considered to have been pased when an app makes it from extras-devel to -testing?

Like is there a battery of automated tests run by Nokia/Maemo that can confirm simple go/no go before Testers start to look at a new build? What having lots of Testers looking at builds should be focused on is finding more in depth issues that a dev might not have anticipated. Aditionally it helps find if something behaves differently than expected on your specific device and configuration. Things like, are all dependencies there or, does an install go to where ALL apps are expected to go should be simple automated entry requirements and Testers should not be expected to have to worry about wasting their valuable time on code that doesn't meet this bar.

If this system of testing 'quality gates' doesn't exist now thats fine, but it would be nice to know the Nokia/Maemo plan to provide and how we can help once the resources are available.

thanks

[RevdKathy]

Can we get this into the Wiki rather than lost in a thread?

Go right ahead!

If you don't I will later.

[Fargus]

Is there any sort of list of what tests are considered to have been pased when an app makes it from extras-devel to -testing?

Like is there a battery of automated tests run by Nokia/Maemo that can confirm simple go/no go before Testers start to look at a new build? What having lots of Testers looking at builds should be focused on is finding more in depth issues that a dev might not have anticipated. Aditionally it helps find if something behaves differently than expected on your specific device and configuration. Things like, are all dependencies there or, does an install go to where ALL apps are expected to go should be simple automated entry requirements and Testers should not be expected to have to worry about wasting their valuable time on code that doesn't meet this bar.

If this system of testing 'quality gates' doesn't exist now thats fine, but it would be nice to know the Nokia/Maemo plan to provide and how we can help once the resources are available.

thanks

If we don't actually have this at present then I think we ought to at least get a list of manual checks together and then document a standard way of achieving this. automated testing could then be built from this framework.

Does anyone have a starting point on this. Not sure I will retain my sanity editing it direct on the Wiki but I can do it offline and paste in enbulk I suppose.

[VRe]

If we don't actually have this at present then I think we ought to at least get a list of manual checks together and then document a standard way of achieving this. automated testing could then be built from this framework.

Does anyone have a starting point on this. Not sure I will retain my sanity editing it direct on the Wiki but I can do it offline and paste in enbulk I suppose.

I wrote about the process on the developer list in 1st of November. One of the issues was a missing manual QA/testing checklist and that everyone wouldn't need to test everything. Automated testing is then a bonus on top of this (and remove at least #1 and #6). As the testing interface is still the same, there is a queue, and it is not clear who has tested what - I made one text based checklist:

Testing checklist:
1. [ ] Bug database exist.
2. [ ] Licensing ok.
3. [ ] Working provided features.
4. [ ] No missing announced features.
5. [ ] Optification ok.
6. [ ] No performance problems.
7. [ ] No power management issues.
8. [ ] No illegal/dubious content.
9. [ ] No known security risks.

* Copy&paste this to the comment box.
* Put [x] for those tests you have done, elaborate on separate row if the test is FAIL.
* Vote up if there were no FAILs, if there was even one FAIL vote down.
* UI usability issues cannot be used as reason for vote down.
* Always test functionality - that is, run the program and try if it works as it should.

imaginary example:
1. [x] Bug database exist.
2. [ ] Licensing ok.
3. [x] Working provided features.
FAIL: There is choice between tabs and spaces as separators but spaces are always used (see bug: http://url/123).
FAIL: When exporting file the program crashes (see bug: http://url/456)
4. [ ] No missing announced features.
5. [x] Optification ok.
FAIL: the package uses 1512kb from root.
6. [ ] No performance problems.
7. [ ] No power management issues.
8. [ ] No illegal/dubious content.
9. [ ] No known security risks.

[VRe]

I wrote the process on the developer list in 1st of November.. one of the issues was a missing manual QA/testing checklist and that everyone wounldn't need to test everything. Automated testing is then a bonus on top of this (and remove atleast #1 and #6). As the testing interface is still the same, there is a queue, it is not clear who has tested what - I made one text based checklist:
...


I added the list to the end of http://wiki.maemo.org/Extras-testing/QA_checklist - maybe I'll be then actually used.. :)

[Flandry]

I added the list to the end of http://wiki.maemo.org/Extras-testing/QA_checklist - maybe I'll be then actually used.. :)

Thanks, that's a nice concise checklist for -testing and a nice addition to that wiki page. However, he was asking about tests that have passed before the app shows up in -testing, ie the automated tests. I'm not sure what they are at present.

Anyone who is going to start working on the wiki, please note my previous post so you don't reinvent the wheel.

One of the biggest problems we face is that people just don't read before they ask questions or even add new wiki pages. That means we end up with a lot of wasted effort and redundant content. Case in point: asking what optified means after i described it clearly a couple pages up. That innocent but already-answered question then took a few more posts that needn't have been made...

[Fargus]

It's a lot more time-consuming to edit the wiki, but i hacked most of that into the http://wiki.maemo.org/Help_testing_software page. I was going to just merge everything into extras-testing, but some of the material seemed a bit verbose and trivial for that page. If someone else can find a good way to consolidate everything into extras-testing and redirect the help-testing-software page to it, that would be great. Just remember that we need a concise but simple description of how to test. The extras-testing page is getting rather long, so the best bet might be to split it into one page describing the nature of the repo and how to add it, and put everything else into the qa checklist page it links to.

So much to do, so little time. Anyway, fresh pair of eyes now that i distilled that new info into the wiki would be good. We really do need a better/more complete testing guide.

I know I am going to regret sticking my head above the parapet like this but....

Is there anyone or group overseeing this section of the wiki at present? I know that there has been a lot of effort gone into UX and documentation recently so if anyone knows then please shout out. I'm happy to put some leg work into collating things and try to put some words together but don't want to duplicate effort or step on anyone's toes.

Regarding sections in the wiki and pages can we gather suggestions here or should we start a new thread for that?

[skalogre]

Hmm... a list of what the Automated testing covers will be good for n00bs such as myself, I agree.

Anyway, once my situation is clearer (job hunting right now so it is a bit hard to focus on all this) you can count me in, also (this sort of stuff has been my job for awhile so I'll do whatever I can to help :) ).

[matrim]

Would it be a possibility to get a group of people who's main testing responsibility is to check for optification.
When a new package is released to testing they check it, and post a comment saying if it's optified or not. This would save other people from having to do it.

Or at least we should encourage the first person testing to post a comment saying if the package is optified or not

[floffe]

Hmm... a list of what the Automated testing covers will be good for n00bs such as myself, I agree
From the wiki: https://wiki.maemo.org/Extras-devel#Promotion_checks

Short summary in user terms: It checks that the package can be installed without requiring adding additional repos, plus some copyright and naming stuff. Very basic QA right now, in other words.

[blubbi]

Why not force optification...

Put everything in opt and symlink stuff into / if it is REALLY necessary (.

What about /home/opt/maemo/{bin,libs,share} is libtool run on these folders after installing a package. If so, all stuff could go here and optification could be forced.

I just check ${PATH} and /home/opt/bin is not in it.

if /home/opt/maemo/${libs,shared} is not handled by libtool, we might should LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:/home/opt/maemo/libs"

Just some thoughts...

A maemo filesystem guide for developers would be greate. Where to put stuff..

Cheers
Bjoern

[smegheadz]

would it be possible to have an app tester application?

for example you can tell it to take a preinstall system snapshot eg memory and cpu usage, directories. could it also have a hook for the application installer to log all the install activity eg file locations. a task manager which can filter the application process like process monitor for windows giving you all the cpu and memory usage. having it log the data. all this data would make more sense to the people who develop the apps then the average user who wants to help. command line stuff is all well and good but people like simple with the ability to go advanced.

One thing that i think is important for app testing is the app enviorment details. what other apps are running app conflicts. Having some app to records this information and compress it to a nice zip file to upload would make bug tracking alot easier. could even write a parser for the bug tracking data you receive to check for consistant data like opt and rootfs usage.

sorry i'm not great at explaining whats in my head.

[hypoxic]

@smegheadz, this is a great idea and has been used successfully on windows software projects in my experience. depending on what is in your snapshot (registry, config files, installed files/locations/versions) it can be a very intensive and large output to then diff. I am speaking in windows terms, i know, as i'm still learning *nix and how its all wired.

I would envision something that that would check the above and the other environment variables (existing installed apps/versions/etc, memory footprint, CPU, etc) and then log it to a txt or xml file. then you can run the tool before and after a test app install or use and then copy it off to a desktop for analysis/diffing.

I'd be happy to help test such an app or work with any devs that are trying to plan and implement such.

However i would assume that Nokia/Maemo would already have something similar already in-house. Anybody on the 'inside' care to comment or share on what the process is for how the various quality gates are met prior to code making it to the Ovi store or any other 'official' repository?

[Zelig87]

Agree. Optified indicator would be very beneficial

The process of promoting an app from Extras-devel to Extras-testing should FAIL if the app is NOT optified.

That way all apps in Extra-testing would be OPTIFIED.

If necessary, demote all existing apps from Extras-testing back to Extras-devel and only let the OPTIFIED ones be re-promoted.

That way, the only repository that may contain a non optified app is Extras-devel.

[Flandry]

The process of promoting an app from Extras-devel to Extras-testing should FAIL if the app is NOT optified.

That way all apps in Extra-testing would be OPTIFIED.

If necessary, demote all existing apps from Extras-testing back to Extras-devel and only let the OPTIFIED ones be re-promoted.

That way, the only repository that may contain a non optified app is Extras-devel.

Uh... that's great except there's currently no automated test for this. That's why it's on the list of things for testers to check...

Many of the things in the QA list could be automated, but it's easier to get people to test than to get them to write a testing app and test it. I'm sure that such code would be welcome, though.

[ossipena]

It's a lot more time-consuming to edit the wiki, but i hacked most of that into the http://wiki.maemo.org/Help_testing_software page. I was going to just merge everything into extras-testing, but some of the material seemed a bit verbose and trivial for that page. If someone else can find a good way to consolidate everything into extras-testing and redirect the help-testing-software page to it, that would be great. Just remember that we need a concise but simple description of how to test. The extras-testing page is getting rather long, so the best bet might be to split it into one page describing the nature of the repo and how to add it, and put everything else into the qa checklist page it links to.

So much to do, so little time. Anyway, fresh pair of eyes now that i distilled that new info into the wiki would be good. We really do need a better/more complete testing guide.
that was the point for help testing software page:
being as simple instruction collection as possible.

then extras-testing can include all dev -stuff etc and new people to sw testing don't get frightened and run away..

[Sasler]

Uh... that's great except there's currently no automated test for this. That's why it's on the list of things for testers to check...

Many of the things in the QA list could be automated, but it's easier to get people to test than to get them to write a testing app and test it. I'm sure that such code would be welcome, though.

Everything that could be automated should be as soon as possible. This should be the priority now! As the user base for Maemo devices increases, there will be more developers for it too. This will mean more applications to test and this backlog will become only worse if something is not done to make the testing faster/easier.

The problem is that many of these new users have a very limited experience with Linux (like me) or no previous experience at all. Therefore something as "simple" as checking optification might already discourage potential beta testers, who might be more than qualified to check the functionality of the application and provide valuable feedback.

[Zelig87]

My point is that given being OPTIFIED is virtually a critical requirement of any app, I think whatever needs to be done to AUTOMATE that check during the promotion from Extras-devel to Extras-testing should be done ASAP.

I realise almost any test *could* be automated if enough resources were dedicated to developing an appropriate testing app, but surely given how important being OPTIFIED is, we could at least get that automated ??

I'm surprised this wasn't one of the first things done once the problem was discovered and /OPT was chosen as the solution.

Seems crazy to allow more non-optified apps into the Eco system, even if there are warning everywhere about enabling Extras-testing.

[hypoxic]

From my experience this is what is considered part of a Build Acceptance Test and comes before any kind of integration is done with the larger code base. Terminology may differ depending on your environment, but the idea is that there are a class of bugs that are so critical that it is a waste of a Tester's time to install them if they fail. Sure we could have 10-50-500 people install an app while being helpful testers and then report the bug, but if we just filled up their memory and bricked the device, I'm sure they all would have preferred that a simple go-nogo test was done before they were put into a compromised situation, and will be Much less likely to help out with Testing in the future.

This has to be a problem already solved, and likely automated by Nokia/Maemo. If I'm just dreaming, will someone who has some info please let me know.

If Nokia/Maemo hasn't got the tooling or is able to share with the Community, then we Testers need to understand better what happens between -devel and -testing and how the promotion (https://wiki.maemo.org/Extras-devel#Promotion_checks) testing occurs and how it could be improved upon.

[smegheadz]

what happened to the alpha and beta naming and v0.4 etc of apps?
it's universal. everyone generally understands that terminology.

My suggestions for making the testing of apps better are:

have a dummy hello world type app for noobs and first time testers to help which covers some of the basic functions apps could have. have a guide what to check and they can learn etc.

have an application that takes system snapshots. (see my previous post page 5) maybe have it as a bug tracker app, allowing quick feedback to maemo.org

allow people to tag an app as testing it so a user can see who and how many people have downloaded it to test and they can message each other about tests they have run etc (already implamented)

Make it easier to do things, 1 login rather then 3 for the site. it just gets annoying. people will get lazy and just not bother because they have to make another account.

when i get my n900 i'd like to help in testing apps and making it so people who just want a cool phone are able to take part without being a nix hacker, majority of people are from a windows background and haven't really scratched dos or scripting. Theres more user's then developers in the world so making it easy for them to test safely is ideal, gives people who can develop more time to do the more advanced testing and developing

[blubbi]

I am used to the Gentoo installation system.

One thing that comes in very handy is the "fake root filesystem".

Every ebuild installs itself in a fake root, replicating the directory structure there. If everything was okay during the install, this directory just gets (in simple words) copied to the live file system.

This could make testing a bit easier cause you can check if everything is optified without affecting the live filesystem. This done, copy those files to the live system and start testing the app.

Is there a similar mechanism in apt?

Cheers
Bjoern

[anidel]

After enabling extra-testing, I noticed that the yellow box started blinking. Xournal and Load-applet had new versions available. I assume from extra-testing side?

While I wait garage registration complete, very very slow, quick notes:

Xournal seems to work, thumbs up.

Load-app seems to require reboot :-/ My first reboot since getting N900 (been running it two weeks straight).

Xournal for a while had three different versions in each repository.
There was package 13 in Extras, one in Extras-Testing and one in Extras-Devel (package 21).

I promoted package 21 to Testing a month ago and that's probably what you got when enabling Testing.

Yesterday, though, I promoted it to Extras, but you've got it already installed. So no need to update there.

Aniello

[smegheadz]

i think this should be stickied to make more people aware that they can help with bringing more apps to their n900

[YoDude]

Uh... that's great except there's currently no automated test for this. That's why it's on the list of things for testers to check...

Many of the things in the QA list could be automated, but it's easier to get people to test than to get them to write a testing app and test it. I'm sure that such code would be welcome, though.

Then what's the point of the two repositories. I thought an app in the developer repo had some sort of peer review before being promoted to the testing repo. :confused:

[adrianp]

Then what's the point of the two repositories. I thought an app in the developer repo had some sort of peer review before being promoted to the testing repo. :confused:
I assumed -devel is for "release early, release often" for those comfortable with having to flash their N900s if, or more likely when, these go wrong. Once the developer is satisfied that his package is reasonably stable then it is moved to testing and then those who don't want bleading edge but are willing to have a little instability can find any major shop-stoppers before the rest of the world gets it.

Basically this is a variant of Debian, but leaving it up to the delevoper to promote to -testing and using voting to get from -testing, unlike Debian that uses severe bugs to handle migration from unstable to testing.

[mysticrokks]

ok i have no problems in testing stuff out.. unless it bricks my phone

is it easy to flash it back and what risks are involved in flashing my n900

[Venomrush]

ok i have no problems in testing stuff out.. unless it bricks my phone

is it easy to flash it back and what risks are involved in flashing my n900

Apps in Extras-testing should be safe to install without any risks of bricking.
Flashing is easy and is done under 5 minutes.

[ArnimS]

http://maemo.org/packages/package_instance/view/fremantle_extras-testing_free_armel/uqm-3do-data/0.6.7-0maemo/

this is required for Ur-Quan Masters game, which has enough votes already afaict. Works fine here. Please test.

[suihkulokki]

Note by: this thread belongs to the "community" section.

There is *still* a major lack of active voters for extras-testing applications.

Extras-testing voting instructions are available here:

http://wiki.maemo.org/Extras-testing/QA_Checklist

If even a fraction of the active forum-goers would bother once a week to check through interesting applications in the QA testing list linked from the above page, apps would not have problems passing in 10 days if ready for extras.