Page 3 of 6 12 3 45 Last »
Reply
Thread Tools
VDVsx's Avatar
VDVsx is offline VDVsx
2010-01-25 , 17:31
Posts: 1,070 | Thanked: 1,604 times | Joined on Sep 2008 @ Helsinki
#21
Originally Posted by jukey View Post
Hi,

I really do like this suggestions! One thing seems to be important for me:




if a package is promoted the should have to be a list of changes. Is there already a possibility in the publishing interface to put a change log or at least a link to the change log location inside of it?
It would really improve the efficiency of tests if the tester knows where are new features (I know that's not all... ).
Yes, the package interface will show the changelog available in the package.

Originally Posted by jukey View Post
3 or 5 a day sounds very much. Maybe we should classify the apps. A complex application should get 3 complex points a simple new desktop theme maybe only 1 complex point. Than it would be easy to say lets test apps with in sum 5 complex points. What do you think?

This answer also goes out to the community mailing list.

Ciao Uwe
Yes, we need to ponder about the complexity of the apps.
__________________
Valério Valério
www.valeriovalerio.org
 

The Following 2 Users Say Thank You to VDVsx For This Useful Post:
Texrat's Avatar
Texrat is offline Texrat
2010-01-25 , 17:40
Posts: 11,700 | Thanked: 10,045 times | Joined on Jun 2006 @ North Texas, USA
#22
Originally Posted by VDVsx View Post
Yes, we need to ponder about the complexity of the apps.
As I suggested before, app complexity should drive several things, especially a quarantine period if we continue to have one. Lines of code (LOC) is certainly a vague indicator but useful at a high level. Perhaps type/number of libraries used could be too. We don't have to get too deep into details either-- high level approach should be good enough.
__________________
Nokia Developer Champion
Different <> Wrong | Listen - Judgment = Progress | People + Trust = Success
My personal site: http://texrat.net
 

The Following User Says Thank You to Texrat For This Useful Post:
RevdKathy's Avatar
RevdKathy is offline RevdKathy
2010-01-25 , 17:42
Posts: 2,173 | Thanked: 2,678 times | Joined on Oct 2009 @ Cornwall, UK
#23
I wonder about some sort of place where people could signify what they are currently testing? To prevent a situation where 30 people test one app and none another. Or am I being overly complex here?
__________________
Hi! I'm Kathy and I'm a Maemo Greeter! Welcome.
Useful links for newcomers: New members say hello , New users start here, Community subforum, Beginners' wiki page, Maemo5 101, Frequently Asked Questions (FAQ)
Did you know Meego.com has forums too?
 

The Following 3 Users Say Thank You to RevdKathy For This Useful Post:
Texrat's Avatar
Texrat is offline Texrat
2010-01-25 , 18:01
Posts: 11,700 | Thanked: 10,045 times | Joined on Jun 2006 @ North Texas, USA
#24
Originally Posted by RevdKathy View Post
I wonder about some sort of place where people could signify what they are currently testing? To prevent a situation where 30 people test one app and none another. Or am I being overly complex here?
An app testing status page is a must IMO.
__________________
Nokia Developer Champion
Different <> Wrong | Listen - Judgment = Progress | People + Trust = Success
My personal site: http://texrat.net
 

The Following 2 Users Say Thank You to Texrat For This Useful Post:
Sasler is offline Sasler
2010-01-25 , 18:04
Posts: 434 | Thanked: 325 times | Joined on Sep 2009
#25
I agree that a complexity category would be good. Using lines of codes would be the easiest way to do it. For example, there could be 3 categories:

LOC < X = Simple
LOC > X and < Y = Normal
LOC > Y = Complex
 

The Following User Says Thank You to Sasler For This Useful Post:
Sasler is offline Sasler
2010-01-25 , 18:09
Posts: 434 | Thanked: 325 times | Joined on Sep 2009
#26
Originally Posted by RevdKathy View Post
I wonder about some sort of place where people could signify what they are currently testing? To prevent a situation where 30 people test one app and none another. Or am I being overly complex here?
Yes, this is definitely necessary. I would even go so far that the Admins should post a list of the apps that should be tested now. Critical updates would get priority and for the rest there could be a simple FIFO (first in, first out) procedure.
 

The Following User Says Thank You to Sasler For This Useful Post:
attila77 is offline attila77
2010-01-25 , 20:55
Posts: 3,319 | Thanked: 5,610 times | Joined on Aug 2008 @ Finland
#27
Originally Posted by VDVsx View Post
What do you think ?
First of all, I'd like to emphasize how important this task is and how much I appreciate Valerio's (et al) efforts. This is just not to misunderstand what I'm saying below, you have my full support and I'll try to be a good member of the bugsquad when the time comes. Let me elaborate:

- Tester - Any community member
If you want to test just one app, you should be able to do that, without any hassle, and be a tester. If tester here means something else (as in a tester who is a more devoted tester than an ordinary tester), it should be made clear. We don't want to scare people away with special procedures or by making them think they should do something special to be a 'tester' (i.e. not to loose people to nomenclature, like seen from RevdKathy's post).

- Can demote packages when there is known blockers. (maintainer obligation)
- Can promote packages when they are stuck in the testing queue for a while without any known blocker.
The point is that IMHO it's the admin who *decides* what's a blocker based on the testing guidelines (and the actions above are just consequences of that decision). One of the major reasons to have an admin is exactly to filter the feedback so packages would not get invalid thumbs-downs because of misunderstandings.

- Receives a automatic notification for each packages that enters testing, is demoted or is promoted.
[ plug ]Not really related, but I use AppWatch with great success for this. [ /plug ]

- In each week one of the testing squad members is responsible for the elaboration of a small list of apps that should be tested. We'll start with a 5 a day (or 3 a day) approach in the beginning. Of course the testers are free to test other apps or do all the testing in just one day. The objective is have the apps tested by the end of the week.
And this is my main beef. The testing squad should certainly help stuck apps, but should also stay clear of replacing users. If the person in charge of an app does not understand what an app is supposed to do (or interested in it), it's not realistic to expect a good QA process. Organized testings also have the problem that the different testing squad members may be in different time zones, different schedules, etc (for example most of my free time is weekends - that's more than 5 days, and I will hardly be able to do anything during workdays so anything that should be done by Friday is an automatic fail for me).

Stuff that is missed out from my QAmaster proposal altogether (just mentioning it so we are in the clear if this is intentional): the cooperation of tester admins with actual developers. Testing, if issues found, should ALWAYS result in a bug report(s) and, preferably a pointer for the developer how to resolve it. Remember, the goal is not to clean out extras-testing (that's just a consequence), but to help streamline promotion to Extras with a special focus on quality.

Another thing which is probably very hard to implement, so I'll just list it as an opinion. Per user thumbs down IMHO is a bad metric and is shows very little about what has (not) been checked and how much overlap there is among the reports. Not only should we have categories, but THAT is our score/karma. I should not care if there are a 100 thumbs-ups and just one thumb down, if that thumb down is really a thumb down (blocker). So the 'karma' of the app is, in fact, not the number of people who checked it, but the number of categories it has been checked on. This works the other way round, too - it's pointless to have 10 thumbs downs because of a common issue. It should be one thumb down (in the summary), which multiple users can confirm, with a bug report, end of story.

Now, if there were folks who read through this, happy to hear comments. No pressure, no matter what solution we end up with, it can't really be worse than the current situation
__________________
Blogging about mobile linux - The Penguin Moves!
Maintainer of PyQt (see introduction and docs), AppWatch, QuickBrownFox, etc
 

The Following 8 Users Say Thank You to attila77 For This Useful Post:
VDVsx's Avatar
VDVsx is offline VDVsx
2010-01-25 , 22:42
Posts: 1,070 | Thanked: 1,604 times | Joined on Sep 2008 @ Helsinki
#28
Originally Posted by attila77 View Post
If you want to test just one app, you should be able to do that, without any hassle, and be a tester. If tester here means something else (as in a tester who is a more devoted tester than an ordinary tester), it should be made clear. We don't want to scare people away with special procedures or by making them think they should do something special to be a 'tester' (i.e. not to loose people to nomenclature, like seen from RevdKathy's post).
Yup, we need to clarify that, as I said anyone can do testing, like now, however the testers referred above will belong to a special garage group.

Originally Posted by attila77 View Post
And this is my main beef. The testing squad should certainly help stuck apps, but should also stay clear of replacing users. If the person in charge of an app does not understand what an app is supposed to do (or interested in it), it's not realistic to expect a good QA process. Organized testings also have the problem that the different testing squad members may be in different time zones, different schedules, etc (for example most of my free time is weekends - that's more than 5 days, and I will hardly be able to do anything during workdays so anything that should be done by Friday is an automatic fail for me).
I don't see any problem here, also as I said, the plan is have the applications tested by the end of the week doesn't matter when you do the testing. Of course sometimes we can arrange small meetings, but only in special cases.

Originally Posted by attila77 View Post
Stuff that is missed out from my QAmaster proposal altogether (just mentioning it so we are in the clear if this is intentional): the cooperation of tester admins with actual developers. Testing, if issues found, should ALWAYS result in a bug report(s) and, preferably a pointer for the developer how to resolve it. Remember, the goal is not to clean out extras-testing (that's just a consequence), but to help streamline promotion to Extras with a special focus on quality.
Yes of course, thumbs down must be commented by the testers, ideally with a bug report, that's one of the proposed improvements.
I'm not sure if I understood the QAMaster role, do you think someone would have time to a role like these ? Or are you talking about a paid position ?
__________________
Valério Valério
www.valeriovalerio.org
 

The Following 2 Users Say Thank You to VDVsx For This Useful Post:
attila77 is offline attila77
2010-01-25 , 23:21
Posts: 3,319 | Thanked: 5,610 times | Joined on Aug 2008 @ Finland
#29
Originally Posted by VDVsx View Post
Yes of course, thumbs down must be commented by the testers, ideally with a bug report, that's one of the proposed improvements.
I'm not sure if I understood the QAMaster role, do you think someone would have time to a role like these ? Or are you talking about a paid position ?
Ideally, a paid position, if nothing else, because of the seriousness of the task - that way things would not get delayed because someone had no time due to job/school/etc, and people stuck in testing would know they have a person to turn to who is supposed to help them (the idea of testing is not to have a hundred people individually go and hunt around about their showstoppers - if we know what the problem is, we should have a fix in place in hours if we can, not weeks !). But, even if there are no funds or possibilities for such an option, we could think about things like rotating QAmastership (say, monthly or biweekly, from the pool of testing admins/moderators) to avoid burning out and making some breathing space for volunteers. Effectively, of course, what I called QAmaster, you could say is a part of testing squad administratorship, my point being that such activity (helping developers through testing) should be considered part of the QA procedure itself (the paid position helping to secure dedicated time for this). Seriously, if one person can just double the throughput of testing by helping the developers not just to identify problems but solve their issues, it would mean a lot more applications in Extras, motivated developers, no 'calculative promotions', summa summarum, an investment IMHO well worth the money.
__________________
Blogging about mobile linux - The Penguin Moves!
Maintainer of PyQt (see introduction and docs), AppWatch, QuickBrownFox, etc
 

The Following 2 Users Say Thank You to attila77 For This Useful Post:
VDVsx's Avatar
VDVsx is offline VDVsx
2010-01-25 , 23:39
Posts: 1,070 | Thanked: 1,604 times | Joined on Sep 2008 @ Helsinki
#30
Originally Posted by attila77 View Post
Ideally, a paid position, if nothing else, because of the seriousness of the task - that way things would not get delayed because someone had no time due to job/school/etc, and people stuck in testing would know they have a person to turn to who is supposed to help them (the idea of testing is not to have a hundred people individually go and hunt around about their showstoppers - if we know what the problem is, we should have a fix in place in hours if we can, not weeks !). But, even if there are no funds or possibilities for such an option, we could think about things like rotating QAmastership (say, monthly or biweekly, from the pool of testing admins/moderators) to avoid burning out and making some breathing space for volunteers. Effectively, of course, what I called QAmaster, you could say is a part of testing squad administratorship, my point being that such activity (helping developers through testing) should be considered part of the QA procedure itself (the paid position helping to secure dedicated time for this). Seriously, if one person can just double the throughput of testing by helping the developers not just to identify problems but solve their issues, it would mean a lot more applications in Extras, motivated developers, no 'calculative promotions', summa summarum, an investment IMHO well worth the money.
Humm, I can see some pros but a lot of cons as well. I'm against a paid position as least for now, since I strongly believe that the community can take care of testing and this money can be spent in other things that we can't do. We already proved that with a broken system and disorganized testing .
A paid person will scare away a lot of the current testers.

As for rotating the QAmaster among the testing team, well the proposal includes something similar, but of course that person doesn't have total control, when we have a group of capable persons why put all the responsibility in just one person ?
__________________
Valério Valério
www.valeriovalerio.org
 

The Following 5 Users Say Thank You to VDVsx For This Useful Post:
Reply
Page 3 of 6 12 3 45 Last »

Tags
extras, squad, testing

« Previous Thread | Next Thread »

 
Forum Jump


All times are GMT. The time now is 05:08.

Contact Us - Home - Archive - Top