extras-testing QA isn't taken seriously
 

How in the hell did kobodeluxe pass QA with broken bugtracker link?!?

http://maemo.org/downloads/product/Maemo5/kobodeluxe/

bugtracker link:
http://kobodeluxe-maemo.tunk.org/

redirects to:
http://maemo.org/packages/package_in...0.5.1-5maemo6/

Re: extras-testing QA isn't taken seriously
 

And what it wrong with that?

Re: extras-testing QA isn't taken seriously
 

dear mikko, have you ever read the article I've linked to you now three times?

http://wiki.maemo.org/Bugtracker

How can I report a bug if bugtracker link leads nowhere?


e: and why aren't you mikko doing anything to change things?

http://wiki.maemo.org/Extras-testing/QA_Checklist

that clearly states that you must define a bugtracker. putting a link that redirects to where you started before clicking the link isn't clearly a bugtracker. so your app fails point number 1. so it shouldn't even be tested before issue has been solved. same rules for everyone, feel free to drive sufficient changes but now your app is slipped to extras against QA checklist.

Re: extras-testing QA isn't taken seriously
 

It is possible that the Kobo Deluxe author intended for the use of the comments section on that packages page to be used to report bugs; "tunk.org" being used to redirect to the latest packages page when a new version comes out.

It is not required to have a "bugs.maemo.org" entry as your bugtracker link. I just use "mailto://".

**** what I wrote above - I just read that wiki page...

Re: extras-testing QA isn't taken seriously
 

So redirect doesn't work for you or what?

This isn't nowhere http://maemo.org/packages/package_in...0.5.1-5maemo6/

Re: extras-testing QA isn't taken seriously
 

and where can I post the bug? to comments -section? I cant seem find that option from the bugtracker -wikipage allowed options -list....

Re: extras-testing QA isn't taken seriously
 

Yes! No real difference to talk.maemo.org thread or to your e-mail client. The list there only for as an example

Re: extras-testing QA isn't taken seriously
 

I don't think the bugtracker link is broken, it does lead to a dedicated page. Certainly not the most efficient way of bugtracking, but IMO I would be hard pressed to thumb down because of an arguable interpretation of what is considered a dedicated page or bug tracker.

PS.The title of this thread is wrong, this has nothing to do with 'seriousness', unless you're talking about the process itself.

Re: extras-testing QA isn't taken seriously
 

Yes it's not good or real bugtracker, but in my experience there will be maybe 1 bug asking for something obvious which I have already decided not to do. So it can handle the expected bugs just fine.

Re: extras-testing QA isn't taken seriously
 

I love how these guidelines let clinical wikipedians among us assault more level headed people :)

Re: extras-testing QA isn't taken seriously
 

I am talking about the process itself. Every package that has broken bugtracker have numerous thumbs up even it should be the first criteria to be correct before further testing and completing the whole QA checklist.

Re: extras-testing QA isn't taken seriously
 

It is more the fact that redirection to another package page isn't very user-friendly and can be really confusing. And i am strongly thinking that anything besides the items in the list should be wrong. Just because there are numerous options available and there are many of those that are easy or really easy to do (tmo/mailto: )

Re: extras-testing QA isn't taken seriously
 

Yeah but maybe everyone do not agree this QA checklist ... !

Re: extras-testing QA isn't taken seriously
 

I have proposed a solution to the problem that 50% of the packages are turned down because of broken bugtracker link in the end of http://wiki.maemo.org/Bugtracker

Re: extras-testing QA isn't taken seriously
 

that is something that must be brought to councils attention. I can't make the decision either no other guys from testing squad.

Re: extras-testing QA isn't taken seriously
 

so you deliberately don't follow the checklist that is a requirement for application to get to extras? what's the point of QA then?

Re: extras-testing QA isn't taken seriously
 

Requirement doesnt mean that you can leave it because you think it is stupid!
The QA checklist is there to provide a standard for extras as we expect maemo users not to be linux geeks! I for one would force more than just the things on the list... and change things in procedure. pushing from testing to extras would never be automagical. extras packages would need a bugs.maemo entry a changelog a proper description and last but not least a usability test. (yes I am crazy, but I do QM and QA for a living and have to be like that)

Re: extras-testing QA isn't taken seriously
 

Promotion isn't automatic.

I would prefer automatic bugs.maemo.org entry too. Most of the packages have proper change log, it just isn't displayed anywhere until now. And yes you are crazy :)

Re: extras-testing QA isn't taken seriously
 

Firstly , I want to make it clear, Im not against testing or QA , I understand the importance of this quite as much as anyone spending more time with S/W than with human beings.

Having said that, I believe its much "easier"(for the lack of a better word ATM) to doctrine the QA process and perform testing of the app than actually develop and maintain one, just considering the fact that is all being done for free (QA included) !
Im not against having a stricter procedure for paid apps however ( I know I'll be blasted for this :p )
Many little comments that our dear QA leave and give thumbs down too will no doubt "better" the apps, but thats also delaying many good apps and letting down developers somewhere down the line ( be honest here :) )

Im all for making this the best platform ever for mobile devices, and I seriously think we need better apps than an amazon widget or a weather guide in our portfolio and quick, not next year !
Encouraging paid apps is the other half, which is being discussed elsewhere, the problems with the infrastructure, ovi support etc etc.

All Im saying is we need to reconsider , be a little lenient towards free apps, and they are flagged down for only serious issues, and not for using wrong widgets or colors or buttons being out of place or bugtracker links missing**.
You want everybody to follow rules in a free world is going to be a bit challenging.
Apple/Android may have rules but there the dev know they get rewarded for following rules.

Again, I didnt want to change this into a QA vs DEV war or even say that QA is not needed and we need to pass all apps into extras, just saying we need to be a little more cautious in firing dev who do the software purely in the interest of the platform development or for their karma in life :D

Re: extras-testing QA isn't taken seriously
 

Hey, I thought I'm t-squad ? ;) Anyway, the point is that people who do community QA are not robots nor (mostly :) ) certified QA engineers. If everybody did everything, there would be no need for TEN thumbs-ups and ten days of quarantine, it would enough for one dude to go through the checklist. The quarantine and redundancy of votes is there exactly to find what others might have missed and to account for differences in device setups, installed apps, etc. So I ask the folks with strong engineering bloodlines to be a bit more pragmatic - it's not a "HA-HA, you forgot/misunderstood it" game. We had some suggestions on the testingsquad list to be more efficiend with the bugtracker link (thumbing down and re-quarantining just because of that is just plain overkill), but still no conclusion there.

Re: extras-testing QA isn't taken seriously
 

so you are suggesting that we must have everything done quick and dirty to extras? lame argument but how many apps have you found at app store that has those misplaced buttons etc?

Re: extras-testing QA isn't taken seriously
 

I agree with that. How is bugtracker link embedded in the package going to improve for example this one
http://maemo.org/packages/package_in...llpaper/1.1-1/ ?

Package could be promoted to extras and the maintainer could add the bugtracker link afterwards to downloads. Now he has to upload it again and wait 1-2 months.

Re: extras-testing QA isn't taken seriously
 

i agree with what you are saying and just because the app is free we should not expect it to be lower quality, however i for one would not thumbs down a devs hard work on an app just because of misplaced buttons! if the app was unusable because of this then thats another story. otherwise i would give it a thumbs up and then open an enhancement or get in touch with the dev.

i also agree that a bug tracker should be mandatory however why cant bugs.maemo.org host all apps bug tracker. as someone else said here why cant the be created automatically as part of the application release process - either to extras dev or extras testing. that way at least you are guaranteed a consistent bug system across all apps!

Re: extras-testing QA isn't taken seriously
 

Getting bug reporting onto the devices is a major goal of mine, and I can use all the help I can get (that includes you, Attila buddy!). Draft presentation below:

http://maemo-daemons.org/MeeGo_User_..._Framework.pdf

I am trying to create a working group page at the MeeGo.com wiki but failing miserably to find out how... :o

The concepts apply equally to Maemo or MeeGo.

Re: extras-testing QA isn't taken seriously
 

you were too quick to read perhaps :)
thats not what Ive said.

*EDIT*
...and I agree with jer006 tomake the bugtracker link automated during packaging if it has to be mandatory.

Re: extras-testing QA isn't taken seriously
 

I guess i better give notice here then that i've been pointing to a Talk thread in the bugtracker field of my recent apps. I actually expect to get more feedback about problems that way.

Feel free to burn/flay as desired.

Re: extras-testing QA isn't taken seriously
 

I agree with Texrat's point, If there was a built in application on the device itself which facilitated the creation of bugs/enhancements within bugzilla and an automated bug tracker created for all apps when released into extras-dev this would solve the problem.

The document you linked Texrat was perfect, I was not sure about the flowchart though for the application usage - is this optional as I dont know about surveys and usage popups!! :) I agree it will provide useful information if people take the survey but I would rather avoid nagware and popups where possible... Buzzard is where this shines though as it could capture so much additional information from the device which makes the bug entry so much more valuable to the devs - rather than relying on individual users to know how access and upload log files etc... Buzzard should also be available to users as a standalone application to log bugs in cases where the application does not crash too though.

There are too many manual steps right now which put off the average user who dabbles a bit here and there - for example the brainstorm feature is great but after creating a brainstorm you have to go and manually create the talk thread and manually link the thread back to the brainstorm. All great ideas and not exactly hard work but enough to put off the average user!

Re: extras-testing QA isn't taken seriously
 

EVERY aspect of this proposal is optional, which is why I show a configuration applet on the "user experience settings example" slide. ;)

RevdKathy provided some useful feedback to help me clarify aspects of the presentation and I will have that updated later today. Feedback always welcome!

EDIT: I have also updated the presentation based on your feedback jer006. Thanks.

Re: extras-testing QA isn't taken seriously
 

Hey, this is pretty good (not that I doubted for a second :) ). How can I help (or did you just mean general comments, in which case I need to get my coffee first :) ) ?

Re: extras-testing QA isn't taken seriously
 

Attila, you remember my email from a while back on this? Instead of a whitepaper I'm doing it in presentation format... but either way I could sure use your input on technical details!

Re: extras-testing QA isn't taken seriously
 

if rules are set up for a process like this one, following these rules is a major benefit in itself - even if you personally don't agree with these rules. not only does it guarantee a standard quality; it also shows if the rules work or not. ignoring the rules won't make them better.

as for the current rules and the bugtracker issue:
i think the current QA rules are reasonable. you might be tempted to argue if the lack of a valid bug tracker really should be a blocker.... yes, i think it should be. either the project does have a tracker; then it's so easy for the author to include it that he ha no excuse at all not to do it. or the project really doesn't have a tracker - well, then it mustn't be released in extras. you cannot just release software without letting ppl know how to report bugs.

so i don't really see the benefit if dropping this requirement. i also don't see why testers should kind of "silently ignore" the fact that it's missing, hoping that a tracker will be added in the next release or the one afterwards. if the author doesn't do it when he first gets his package in testing, he most likely never will.

there's a very clear documentation that says which criteria have to be met for a package to be promoted to extras. if these criteria aren't met, authors shouldn't try to push their work into -testing in the first place. and testerst shouldn't vote those pakages up based on nothing else than "cool application, works great!". that's not enough.

Re: extras-testing QA isn't taken seriously
 

all in all it comes down to:

• A clear set of testing requirements and rules.
• A clear set of todo's for a developer (I'm still trying to figure out how to create a bugtracker!! Wiki is a mare!!!)

Then after that actually getting both sides to follow it - only way to do that is to monitor it :D - oh the joys :D

Re: extras-testing QA isn't taken seriously
 

I reviewed the wiki again and am confused what the problem is.
It doesn't say that bugzilla has to be used, just that it's recommended. The point is to provide some location for bugs to be reported.

Let's not get confused on this point. Mikko's app seems to me to meet this criterion.

Re: extras-testing QA isn't taken seriously
 

a comment from QA on one of the packages....

"
Bugtracker link is not required in package data. Bugtracker must exist and it can be indicated otherwise too. Using XSBC-Bugtracker link is not always good idea because it's too hard to change it (requires new version of a package). http://maemo.org/downloads bug link can be changed manually when it's not included in package. http://maemo.org/downloads is the only place where the link actually matters."

this was in response to a QA that gave thumbs down for an app of mine not having a bug tracker link.
does make some sense, I agree the process in itself is not clear, and IMHO needs to be reviewed again. Giving thumbs down is not a solution.

Re: extras-testing QA isn't taken seriously
 

I don't think anyone argued for dropping the requirement. The problem is twofold. One, that 'what constitutes a bugtracker' is interpretation prone. Two, that it is something that does not affect the package itself in any way, and yet a fix is expected to pass the same 10 day quarantine, QA procedure and 10 vote-ups as a full release.

Not having a bugtracker is unacceptable, but It is just as unacceptable (I'd even say even more) to delay an app for a month or so because the rules are unclear about a single http/mail link.

Re: extras-testing QA isn't taken seriously
 

hmmm how about in my case... i have a broken bugrtracker link that doesnt want to fix itself at all... but i have put the Talk thread link in the description (here - http://maemo.org/packages/package_in...check/0.5.2-3/ )

Re: extras-testing QA isn't taken seriously
 

Your debian/control contains a duplicate "XSBC-Bugtracker: " string:

Change it to

instead.

Re: extras-testing QA isn't taken seriously
 

Ahhh but if i change it, it resets the testing phase... as the detail name of the package contains a link as well, i'd assume it should be fine? (Obviously personal interest in this matter seems to differ though)....

Re: extras-testing QA isn't taken seriously
 

I'd say put up a fixed version - it's only half a day lost, and I'm sure you'll get the votes well before the 10-day quarantine ends.

Re: extras-testing QA isn't taken seriously
 

True, meh, need to do it at home when i get time :( - may not until the weekend :( - (Although my recent thread spamming means i might hehehe)

It will reset the votes and time though i think :( - woulda been nice to get that up and going while i work on the other issues instead.