Menu

Main Menu
Talk Get Daily Search

Member's Online

    User Name
    Password

    [extras-testing QA] giving thumbs up in testing without following QA

    Reply
    Page 1 of 4 | 1   2     3   | Next | Last
    chemist | # 1 | 2010-08-20, 13:15 | Report

    Just recognized apps getting "thumbs up" from "testers" which brakes with QA list!
    What should be done about this?

    Serious issue as if you get enough assigned testers doing so you can promote the app from testing to extras.

    Example:
    http://maemo.org/packages/package_in...egweled/0.9-7/

    Edit | Forward | Quote | Quick Reply | Thanks
    The Following 2 Users Say Thank You to chemist For This Useful Post:
    benny1967, mmlado

     
    Khertan | # 2 | 2010-08-20, 14:23 | Report

    And the other way ... if you have a "niche" apps with less than 10 users your apps will never reach extras ...

    I think each app in extras should require 5000 votes ... and any negative vote block promotion. So only perfect applications will reach extras ... and we will not have any problems anymore ... as we will not have any apps

    EDIT : And everyone didn't agree with some rules setted in QA ... there is community votes ... nothing.

    Edit | Forward | Quote | Quick Reply | Thanks

    Last edited by Khertan; 2010-08-20 at 14:26.

     
    ossipena | # 3 | 2010-08-20, 14:36 | Report

    in that case it doesn't matter. even when there are 10k thumbs up, it won't be promoted because of the lack of bucktracker.

    and now you need 3 super tester votes, 10 ordinary isn't enough. that should be enough in order to get every criteria filled before promotion.

    btw try searching sio2interactive thread here, misuse has already happened.

    ps khertan set your own repo and you can be the one who decides about QA by yourself....

    Edit | Forward | Quote | Quick Reply | Thanks

     
    nicolai | # 4 | 2010-08-20, 15:00 | Report

    Originally Posted by ossipena View Post
    and now you need 3 super tester votes, 10 ordinary isn't enough. that should be enough in order to get every criteria filled before promotion.
    10 ordinary votes are enough, but you can also promote
    the packages if there are less than 10, but at least 3 super-tester votes.
    And this package got 3 super-tester votes. It is unlocked!
    Even without fulfiling the QA-criteria.

    Edit | Forward | Quote | Quick Reply | Thanks

    Last edited by nicolai; 2010-08-20 at 15:02.
    The Following 2 Users Say Thank You to nicolai For This Useful Post:
    ossipena, YoDude

     
    fms | # 5 | 2010-08-20, 15:02 | Report

    Originally Posted by chemist View Post
    Just recognized apps getting "thumbs up" from "testers" which brakes with QA list! What should be done about this?
    QA list should be shrunk. The number of required votes should be reduced from 10 to 5. Because even now, most people have switched from Extras to Extras-Testing or even Extras-Devel. Your golden goose is dead, starved by extensive restrictions on its diet. Get used to that.

    Edit | Forward | Quote | Quick Reply | Thanks
    The Following User Says Thank You to fms For This Useful Post:
    twaelti

     
    Khertan | # 6 | 2010-08-20, 15:16 | Report

    Originally Posted by ossipena View Post
    ps khertan set your own repo and you can be the one who decides about QA by yourself....
    http://khertan.net/khertan_repository

    But of course ... this mean that current version in extras are outdated, and buggy

    Edit | Forward | Quote | Quick Reply | Thanks

    Last edited by Khertan; 2010-08-20 at 15:19.
    The Following User Says Thank You to Khertan For This Useful Post:
    ossipena

     
    attila77 | # 7 | 2010-08-20, 15:21 | Report

    It’s about finding the right balance. It is inevitable that some people will not vote right and miss (to seasoned testing veterans) obvious blockers - that is why we have multiple votes and comments so we could figure out what’s going on. I personally hope that apps like kisstester will make it easier for people to leave feedback and with some automated tests, it will make it less error-prone than the current process.

    Now, since this is a crowdsourcing effort, there is no guarantee, just a best effort that very broken or incompliant does not slip through. If it does, we poke the maintainer just as we would on any other fatal bug that might not be apparent when the app was pushed. That’s why super-tester votes are important when an app has only a few votes - if we just lowered vote-requirement, that, low number could be reached "accidentally". This actually works, we have practically halved our locked package numbers since super-tester activity started - but we DO need a lot more user activity so super-tester votes become a last resort method, not a de facto way of getting to unlock.

    Khertan, there IS a community vote. I didn’t set these rules in place, some of them I agree with, some I don’t, but what we have is a very general compromise on what is necessary to provide a safe experience and provide us with a "enabled by default" state from Nokia. I very well understand the pain of not having 10 people to vote for you, I myself have such packages. Feel free to lobby for alternate solutions, and if the proposal gains traction, I will be more than happy to hand the new requirements over to Niels.

    That said, Battle Gweled is a somewhat special case - when that package was submitted, IIRC the bugtracker rule was not yet perfectly known, optification was still in it’s infancy, etc (note - even Jaffa did a thumbs up on that one ! ). I don’t know if mikkov abandoned it or not, but I did not feel the package be in such a bad shape to require a manual removal.

    In any case, we WILL have to come up with a mechanism to deal with orphaned packages (i.e. what to do if a package gets unlocked and the maintaner does not publish it for whatever reason). Thoughts welcome.

    Edit | Forward | Quote | Quick Reply | Thanks
    The Following 8 Users Say Thank You to attila77 For This Useful Post:
    benny1967, clasificado, Helmuth, Jaffa, mmlado, nidO, ossipena, Venemo

     
    attila77 | # 8 | 2010-08-20, 16:06 | Report

    Originally Posted by fms View Post
    QA list should be shrunk. The number of required votes should be reduced from 10 to 5. Because even now, most people have switched from Extras to Extras-Testing or even Extras-Devel. Your golden goose is dead, starved by extensive restrictions on its diet. Get used to that.
    One tiny remark to this. The goal of extras-testing is not to protect devel stage software from people willing to try it out, but to protect people from broken software who DON’T want to try it out. Extras-devel will ALWAYS be cooler than extras (or extras-testing). And as for actual statistics, I’ll try and see whether we can get a more detailed split from downloads so this "most people" thing can be put into perspective.

    Edit | Forward | Quote | Quick Reply | Thanks
    The Following 6 Users Say Thank You to attila77 For This Useful Post:
    benny1967, clasificado, DeargDoom, Jaffa, pelago, Venemo

     
    benny1967 | # 9 | 2010-08-20, 16:17 | Report

    I just want to stress attila's point: testing and devel are easily available for those who want to use those applications. this leaves extras for one group: for those who really, absolutly only want to use well-tested software.... or those who don't know or care about technical details (and the additional repos). both groups are best served by a strict testing-procedure; i don't see anyone gaining anything anything from relaxed QA criteria.

    Edit | Forward | Quote | Quick Reply | Thanks
    The Following 11 Users Say Thank You to benny1967 For This Useful Post:
    attila77, clasificado, DeargDoom, fnordianslip, Jaffa, mmlado, ossipena, pelago, slender, TomJ, Venemo

     
    DeargDoom | # 10 | 2010-08-20, 16:45 | Report

    Originally Posted by benny1967 View Post
    I just want to stress attila's point: testing and devel are easily available for those who want to use those applications. this leaves extras for one group: for those who really, absolutly only want to use well-tested software.... or those who don't know or care about technical details (and the additional repos). both groups are best served by a strict testing-procedure; i don't see anyone gaining anything anything from relaxed QA criteria.
    I would go further and say its useful for everyone. I imagine I am more risk averse than most people on the forum but Im happy to use useful apps in devel and testing if I know that the risks are acceptable so I generally read through the relevent threads before installing such an application.

    I trust the extras repository though and just install anything that looks interesting as I see it as being inherently low risk. I appreciate the quality control and would not like the borders between extras and testing/devel to become blurred.

    Edit | Forward | Quote | Quick Reply | Thanks
    The Following 2 Users Say Thank You to DeargDoom For This Useful Post:
    attila77, Venemo

     
    Page 1 of 4 | 1   2     3   | Next | Last
vBulletin® Version 3.8.8
Normal Logout