Active Topics
-
Porting apps to Leste (49)
to Maemo 7 / Leste by teroyk - 12 hrs, 53 mins ago -
Maemo repository (39)
to Community by teroyk - 13 hrs, 8 mins ago -
Help needed to recover from botched user config preventing boot (8)
to MeeGo / Harmattan by nphuracm - 1 day, 18 hrs ago - more...
| The Following 4 Users Say Thank You to VRe For This Useful Post: | ||
|
2009-12-17
, 18:58
|
|
Posts: 53 |
Thanked: 49 times |
Joined on Jun 2007
|
#42
|
Originally Posted by VRe
I added the list to the end of http://wiki.maemo.org/Extras-testing/QA_checklist - maybe I'll be then actually used..
I wrote the process on the developer list in 1st of November.. one of the issues was a missing manual QA/testing checklist and that everyone wounldn't need to test everything. Automated testing is then a bonus on top of this (and remove atleast #1 and #6). As the testing interface is still the same, there is a queue, it is not clear who has tested what - I made one text based checklist:
...
 |
|
2009-12-17
, 19:15
|
|
Posts: 1,559 |
Thanked: 1,786 times |
Joined on Oct 2009
@ Boston
|
#43
|
Originally Posted by VRe
Thanks, that's a nice concise checklist for -testing and a nice addition to that wiki page. However, he was asking about tests that have passed before the app shows up in -testing, ie the automated tests. I'm not sure what they are at present.
I added the list to the end of http://wiki.maemo.org/Extras-testing/QA_checklist - maybe I'll be then actually used..
Anyone who is going to start working on the wiki, please note my previous post so you don't reinvent the wheel.
One of the biggest problems we face is that people just don't read before they ask questions or even add new wiki pages. That means we end up with a lot of wasted effort and redundant content. Case in point: asking what optified means after i described it clearly a couple pages up. That innocent but already-answered question then took a few more posts that needn't have been made...
__________________
Unofficial PR1.3/Meego 1.1 FAQ
Unofficial PR1.3/Meego 1.1 FAQ
***
Classic example of arbitrary Nokia decision making. Couldn't just fallback to the no brainer of tagging with lat/lon if network isn't accessible, could you Nokia?
 |
|
2009-12-17
, 19:26
|
|
Posts: 1,217 |
Thanked: 446 times |
Joined on Oct 2009
@ Bedfordshire, UK
|
#44
|
Originally Posted by Flandry
I know I am going to regret sticking my head above the parapet like this but....
It's a lot more time-consuming to edit the wiki, but i hacked most of that into the http://wiki.maemo.org/Help_testing_software page. I was going to just merge everything into extras-testing, but some of the material seemed a bit verbose and trivial for that page. If someone else can find a good way to consolidate everything into extras-testing and redirect the help-testing-software page to it, that would be great. Just remember that we need a concise but simple description of how to test. The extras-testing page is getting rather long, so the best bet might be to split it into one page describing the nature of the repo and how to add it, and put everything else into the qa checklist page it links to.
So much to do, so little time. Anyway, fresh pair of eyes now that i distilled that new info into the wiki would be good. We really do need a better/more complete testing guide.
Is there anyone or group overseeing this section of the wiki at present? I know that there has been a lot of effort gone into UX and documentation recently so if anyone knows then please shout out. I'm happy to put some leg work into collating things and try to put some words together but don't want to duplicate effort or step on anyone's toes.
Regarding sections in the wiki and pages can we gather suggestions here or should we start a new thread for that?
 |
|
2009-12-17
, 19:26
|
|
Posts: 327 |
Thanked: 249 times |
Joined on Sep 2009
@ Λεμεσιανός, ρε!
|
#45
|
Hmm... a list of what the Automated testing covers will be good for n00bs such as myself, I agree.
Anyway, once my situation is clearer (job hunting right now so it is a bit hard to focus on all this) you can count me in, also (this sort of stuff has been my job for awhile so I'll do whatever I can to help ).
Anyway, once my situation is clearer (job hunting right now so it is a bit hard to focus on all this) you can count me in, also (this sort of stuff has been my job for awhile so I'll do whatever I can to help
).
|
|
2009-12-17
, 22:49
|
|
Posts: 121 |
Thanked: 75 times |
Joined on Oct 2009
|
#46
|
Would it be a possibility to get a group of people who's main testing responsibility is to check for optification.
When a new package is released to testing they check it, and post a comment saying if it's optified or not. This would save other people from having to do it.
Or at least we should encourage the first person testing to post a comment saying if the package is optified or not
When a new package is released to testing they check it, and post a comment saying if it's optified or not. This would save other people from having to do it.
Or at least we should encourage the first person testing to post a comment saying if the package is optified or not
|
|
2009-12-17
, 23:09
|
|
Posts: 287 |
Thanked: 127 times |
Joined on Oct 2009
@ Sweden
|
#47
|
Originally Posted by skalogre
From the wiki: https://wiki.maemo.org/Extras-devel#Promotion_checks
Hmm... a list of what the Automated testing covers will be good for n00bs such as myself, I agree
Short summary in user terms: It checks that the package can be installed without requiring adding additional repos, plus some copyright and naming stuff. Very basic QA right now, in other words.
| The Following User Says Thank You to floffe For This Useful Post: | ||
 |
|
2009-12-17
, 23:29
|
|
Posts: 288 |
Thanked: 113 times |
Joined on Dec 2009
@ Germany
|
#48
|
Why not force optification...
Put everything in opt and symlink stuff into / if it is REALLY necessary (.
What about /home/opt/maemo/{bin,libs,share} is libtool run on these folders after installing a package. If so, all stuff could go here and optification could be forced.
I just check ${PATH} and /home/opt/bin is not in it.
if /home/opt/maemo/${libs,shared} is not handled by libtool, we might should LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:/home/opt/maemo/libs"
Just some thoughts...
A maemo filesystem guide for developers would be greate. Where to put stuff..
Cheers
Bjoern
Put everything in opt and symlink stuff into / if it is REALLY necessary (.
What about /home/opt/maemo/{bin,libs,share} is libtool run on these folders after installing a package. If so, all stuff could go here and optification could be forced.
I just check ${PATH} and /home/opt/bin is not in it.
if /home/opt/maemo/${libs,shared} is not handled by libtool, we might should LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:/home/opt/maemo/libs"
Just some thoughts...
A maemo filesystem guide for developers would be greate. Where to put stuff..
Cheers
Bjoern
 |
|
2009-12-17
, 23:34
|
|
Posts: 387 |
Thanked: 566 times |
Joined on Dec 2009
@ Dublin
|
#49
|
would it be possible to have an app tester application?
for example you can tell it to take a preinstall system snapshot eg memory and cpu usage, directories. could it also have a hook for the application installer to log all the install activity eg file locations. a task manager which can filter the application process like process monitor for windows giving you all the cpu and memory usage. having it log the data. all this data would make more sense to the people who develop the apps then the average user who wants to help. command line stuff is all well and good but people like simple with the ability to go advanced.
One thing that i think is important for app testing is the app enviorment details. what other apps are running app conflicts. Having some app to records this information and compress it to a nice zip file to upload would make bug tracking alot easier. could even write a parser for the bug tracking data you receive to check for consistant data like opt and rootfs usage.
sorry i'm not great at explaining whats in my head.
for example you can tell it to take a preinstall system snapshot eg memory and cpu usage, directories. could it also have a hook for the application installer to log all the install activity eg file locations. a task manager which can filter the application process like process monitor for windows giving you all the cpu and memory usage. having it log the data. all this data would make more sense to the people who develop the apps then the average user who wants to help. command line stuff is all well and good but people like simple with the ability to go advanced.
One thing that i think is important for app testing is the app enviorment details. what other apps are running app conflicts. Having some app to records this information and compress it to a nice zip file to upload would make bug tracking alot easier. could even write a parser for the bug tracking data you receive to check for consistant data like opt and rootfs usage.
sorry i'm not great at explaining whats in my head.
 |
|
2009-12-18
, 03:13
|
|
Posts: 23 |
Thanked: 20 times |
Joined on Dec 2009
|
#50
|
@smegheadz, this is a great idea and has been used successfully on windows software projects in my experience. depending on what is in your snapshot (registry, config files, installed files/locations/versions) it can be a very intensive and large output to then diff. I am speaking in windows terms, i know, as i'm still learning *nix and how its all wired.
I would envision something that that would check the above and the other environment variables (existing installed apps/versions/etc, memory footprint, CPU, etc) and then log it to a txt or xml file. then you can run the tool before and after a test app install or use and then copy it off to a desktop for analysis/diffing.
I'd be happy to help test such an app or work with any devs that are trying to plan and implement such.
However i would assume that Nokia/Maemo would already have something similar already in-house. Anybody on the 'inside' care to comment or share on what the process is for how the various quality gates are met prior to code making it to the Ovi store or any other 'official' repository?
I would envision something that that would check the above and the other environment variables (existing installed apps/versions/etc, memory footprint, CPU, etc) and then log it to a txt or xml file. then you can run the tool before and after a test app install or use and then copy it off to a desktop for analysis/diffing.
I'd be happy to help test such an app or work with any devs that are trying to plan and implement such.
However i would assume that Nokia/Maemo would already have something similar already in-house. Anybody on the 'inside' care to comment or share on what the process is for how the various quality gates are met prior to code making it to the Ovi store or any other 'official' repository?
| The Following User Says Thank You to hypoxic For This Useful Post: | ||
Testing checklist:
1. [ ] Bug database exist.
2. [ ] Licensing ok.
3. [ ] Working provided features.
4. [ ] No missing announced features.
5. [ ] Optification ok.
6. [ ] No performance problems.
7. [ ] No power management issues.
8. [ ] No illegal/dubious content.
9. [ ] No known security risks.
* Copy&paste this to the comment box.
* Put [x] for those tests you have done, elaborate on separate row if the test is FAIL.
* Vote up if there were no FAILs, if there was even one FAIL vote down.
* UI usability issues cannot be used as reason for vote down.
* Always test functionality - that is, run the program and try if it works as it should.
imaginary example:
1. [x] Bug database exist.
2. [ ] Licensing ok.
3. [x] Working provided features.
FAIL: There is choice between tabs and spaces as separators but spaces are always used (see bug: http://url/123).
FAIL: When exporting file the program crashes (see bug: http://url/456)
4. [ ] No missing announced features.
5. [x] Optification ok.
FAIL: the package uses 1512kb from root.
6. [ ] No performance problems.
7. [ ] No power management issues.
8. [ ] No illegal/dubious content.
9. [ ] No known security risks.
Last edited by VRe; 2009-12-17 at 19:12.