Reply
Thread Tools
VDVsx's Avatar
Posts: 1,070 | Thanked: 1,604 times | Joined on Sep 2008 @ Helsinki
#1
Hi,

along with the improvements in the package interface[1], we'll create a testing squad, in order kick start the testing process and provide better SW to the Maemo users.

Here is a initial proposal, suggestions welcome:

* Roles:
- Tester - Any community member
- Master/Admin - Selected testing squad members.

There will be some special permissions in the testing interface, these should be only used in extreme cases after prior note sent to the testing squad mailing list.

* Master/Admin's special permissions:
- Can demote packages when there is known blockers. (maintainer obligation)
- Can promote packages when they are stuck in the testing queue for a while without any known blocker.

* Testing Squad mailing list:
- Public mailing list where are discussed any situation/issue/doubts concerning the applications in the maemo.org repositories.
- Receives a automatic notification for each packages that enters testing, is demoted or is promoted.

* Organized testing:
- In each week one of the testing squad members is responsible for the elaboration of a small list of apps that should be tested. We'll start with a 5 a day (or 3 a day) approach in the beginning. Of course the testers are free to test other apps or do all the testing in just one day. The objective is have the apps tested by the end of the week.

After gathering some feedback, I'll post instructions how to join the team.

What do you think ?

[1] - http://wiki.maemo.org/Extras-testing...A_Improvements
__________________
Valério Valério
www.valeriovalerio.org
 

The Following 31 Users Say Thank You to VDVsx For This Useful Post:
RevdKathy's Avatar
Posts: 2,173 | Thanked: 2,678 times | Joined on Oct 2009 @ Cornwall, UK
#2
I think this is an excellent idea and will hopefully solve much of the frustration people (both devs and users) are feeling over some of the ways that the testing queue functions. Well done to the Council for coming up with something so robust.

A small question, is there room in this for people like me... an occasional user-tester? I have neither time nor expertise to do a lot of testing, but make a point of have two or three apps from either extras-testing or extras-devel. I choose ones that I shall actually use so that over the course of a couple of weeks I can make a decent evaluation. I tend to give feedback in t.m.o. threads, and then vote when appropriate.

Will there still be space for people like me to vote?
__________________
Hi! I'm Kathy and I'm a Maemo Greeter! Welcome.
Useful links for newcomers: New members say hello , New users start here, Community subforum, Beginners' wiki page, Maemo5 101, Frequently Asked Questions (FAQ)
Did you know Meego.com has forums too?
 

The Following User Says Thank You to RevdKathy For This Useful Post:
Posts: 268 | Thanked: 304 times | Joined on Oct 2009 @ Orlando, USA
#3
Great step foward.

Along with this, the definition of Blockers, especially with respect to the requirement of Bug-trackers needs to be clarified in the wiki. Is a bug-tracker required for all packages? or only for large ones? This requirement is openly ignored in many cases and the presence of a testing admin with the ability to demote a packege will help enforce the rules.

I also didn't see any changes to the quarantine period in you suggestions. Reducing the quarantine for updates would bode well among the developers who feel that the current system does not encourage quick bugfixes.
 

The Following 2 Users Say Thank You to archebyte For This Useful Post:
Posts: 3,428 | Thanked: 2,856 times | Joined on Jul 2008
#4
If I ever get my hands on an N900 I'd be all over this. Testing things is what makes this stuff fun! .

Looks like good progress in getting software moved from the Devel to the more friendly Extras repository. Excellent job!

Archbyte, regarding the quarantine - in the OP they mention the Master/Admins can promote packages. Assume a developer could contact one of them for brief, quick testing of the app before it got promoted as a "fix" or some such.. maybe? (just a thought)
__________________
If I've helped you or you use any of my packages feel free to help me out.
-----------------------------------------------------------------------------------
Maintaining:
pyRadio - Pandora Radio on your N900, N810 or N800!
 
Texrat's Avatar
Posts: 11,700 | Thanked: 10,045 times | Joined on Jun 2006 @ North Texas, USA
#5
definitely gonna need a maemo-testers list.
__________________
Nokia Developer Champion
Different <> Wrong | Listen - Judgment = Progress | People + Trust = Success
My personal site: http://texrat.net
 

The Following 5 Users Say Thank You to Texrat For This Useful Post:
Posts: 268 | Thanked: 304 times | Joined on Oct 2009 @ Orlando, USA
#6
Originally Posted by fatalsaint View Post
..regarding the quarantine - in the OP they mention the Master/Admins can promote packages. Assume a developer could contact one of them for brief, quick testing of the app before it got promoted as a "fix" or some such.. maybe? (just a thought)
@fatalsaint, I don't think the idea is to have the admins override quarantine limits. The admins would be similar to a communtiy tester but with a few more rights to override an invalid rating and of course the promotion/demotion aspect..

And IMO it would be a good idea to avoid situations of having developers make special requests directly for their apps to 'cut the line' so to speak. This would avoid potential conflict of interest situations and help the admins determine the priorities.

Last edited by archebyte; 2010-01-24 at 20:52.
 

The Following 3 Users Say Thank You to archebyte For This Useful Post:
Jaffa's Avatar
Posts: 2,535 | Thanked: 6,681 times | Joined on Mar 2008 @ UK
#7
Originally Posted by RevdKathy View Post
[I]s there room in this for people like me... an occasional user-tester? I have neither time nor expertise to do a lot of testing, but make a point of have two or three apps from either extras-testing or extras-devel.
Isn't that the Tester role? I read "Tester" as just a title (i.e. standard use-case based design) to describe what anyone can currently do: go in and vote on an app.
__________________
Andrew Flegg -- mailto:andrew@bleb.org | http://www.bleb.org
 

The Following 2 Users Say Thank You to Jaffa For This Useful Post:
Posts: 3,428 | Thanked: 2,856 times | Joined on Jul 2008
#8
Originally Posted by archebyte View Post
And IMO it would be a good idea to avoid situations of having developers make special requests directly for their apps to 'cut the line' so to speak. This would avoid potential conflict of interest situations and help the admins determine the priorities.
Good point. I can see how this would be an unwanted scenario. How would you determine whether something should skip the test phase/quarantine limit though? Any modification to source code can cause some serious problems... even a critical bug fix. You could wind up with a critical bug "fix" that produces a brick.

Well.. technically I guess that "fixes" the original problem, right .
__________________
If I've helped you or you use any of my packages feel free to help me out.
-----------------------------------------------------------------------------------
Maintaining:
pyRadio - Pandora Radio on your N900, N810 or N800!
 
Posts: 434 | Thanked: 325 times | Joined on Sep 2009
#9
Great news!

But am I to understand that once the tests are complete, it will be promoted? Or will there still be the 10 quarantine?
 
Posts: 121 | Thanked: 75 times | Joined on Oct 2009
#10
I'd add that when an application is demoted an email should also be sent directly to the maintainer.

Even just an automated email, saying your application has been demoted please check the testing squad mailing list for details.

Originally Posted by VDVsx;492276
There will be some special permissions in the testing interface, these should be only used in extreme cases after prior note sent to the testing squad mailing list.

* [B
Master/Admin's special permissions[/B]:
- Can demote packages when there is known blockers. (maintainer obligation)
- Can promote packages when they are stuck in the testing queue for a while without any known blocker.
 

The Following User Says Thank You to matrim For This Useful Post:
Reply

Tags
extras, squad, testing

Thread Tools

 
Forum Jump


All times are GMT. The time now is 09:27.