Validity criteria for bugtrackers and bugtrackers links
 

Hi,

since some developers aren't taking seriously the QA rules and there's a lot of room for different interpretations in some cases, like the validity of the bugtracker fields, the Testing Squad decided to write some additional rules to clarify dubious situations: http://wiki.maemo.org/Bugtracker

These rules aren't final and need to be approved by the community as usual, this is just a call for further discussion.

Discussion at -Community ML: http://n2.nabble.com/Validity-criter....html#a4695909

Re: Validity criteria for bugtrackers and bugtrackers links
 

With that URL I am a bit afraid that newbies might think that it is the URL for the maemo.org Bugtracker instead. It's not clear that this refers to packages.

Re: Validity criteria for bugtrackers and bugtrackers links
 

I'm ok with that. It acknowledges specific alternatives to a formal bug tracking app, which is IMO important.

Re: Validity criteria for bugtrackers and bugtrackers links
 

Yeah i agree, having the types listed is a good idea.

I'd like to see a way of linking some of the great tutorials to the pages like bugtracking etc... so the process can be followed through from beginning to end. Its the little snags like the bugtracker link that have added a week onto my first attempt. But does also prove that if we tighten things up a bit, new coders can almost have an app in extras in under a month... (Not sure if that is good, but feels good to me!)

good point Andre :D - i probably would have!

Re: Validity criteria for bugtrackers and bugtrackers links
 

I do have a slightly different opinion. as much as i agree with bug tracking etc... for small projects it isn't allways needed. And after battling through QT and Python, the last thing i wanted was to have to learn Bugtracker, get another login, remember to visit it etc... (OK it's not that bad, as it emails you) - but doesnt portray a nice image to first timers.

I think working alongside the rest of the good single-sign-on work for meego forums and making sure that users can do bugtracking on the forum sign-on would also be a good idea. (As well as all the other options).

Re: Validity criteria for bugtrackers and bugtrackers links
 

We are also working in a checklist for developers with all the links, in order to smooth the testing process: http://wiki.maemo.org/Prepare_your_a...on_for_testing

Re: Validity criteria for bugtrackers and bugtrackers links
 

Are you reading my mind? :D :D good work.

Re: Validity criteria for bugtrackers and bugtrackers links
 

I'm not sure if making it easy for developers is the point here. Maemo.org has a repository where new developers can play and learn and needn't care at all about good packaging practices.

The whole point of having -devel, -testing and then Extras is that Extras should be a high quality repo that makes it easy for users (and power users who report bugs), not primarily for developers. (It's not all that difficult for developers, though.)

I expect developers who go all the way from -testing to Extras to be serious enough about their projects and their involvement with the community that registering an account @bugzilla and learning how to use the system isn't too much for them.

I'm just very sad how many developers (good developers of useful application) just seem to insist on not following the rules. When I find a project I would love to see in Extras, I cannot vote it up because it's just not "ready". I don't know why devs do something like that, even if you tell them how to correct the problem easily.

Re: Validity criteria for bugtrackers and bugtrackers links
 

Rob1n launched a side discussion regarding the value of encouraging/requiring applications to register their data files for backup. That topic also seems quite worthy of discussion, so I've moved those posts their own thread where that topic will (hopefully) be easier to find and won't side-track this thread.

Re: Validity criteria for bugtrackers and bugtrackers links
 

remember not all are tecnically dev's though... There is alot to pickup - so getting it all right first time is a challenge... I had to read hundreds of threads, lots of wiki posts, register at a garage, bugs, get answers on irc and maemo.org, buy 3 books... etc...

So sometimes its just making sure that when they look for help its a pleasure and not a chore ;)

(Allthough you are right ... experienced dev's probably have no excuse) - i'm hoping in my second app, i'm aware of the impending bugtracker doom, and plan sooner :D

Re: Validity criteria for bugtrackers and bugtrackers links
 

I'm hoping, as the QA process and team evolve, they're able to be seen less as a barrier to, and antagonists of, Maemo development. Along those lines, might it make sense to provide developers with a easy to use, and easy to find, method of requesting a QA team review of an application while the app is still in -devel? New (or not-so-new) developers could then get (hopefully friendly, helpful, guiding) feedback on their packaging, etc. earlier. It might help avoid and reduce some of the hard feelings developers seem to be experiencing when packages fail QA, particularly for "technicality" issues.

Re: Validity criteria for bugtrackers and bugtrackers links
 

I just want to mention that one of the problems you might have seen ("xsbc-bugtracker:" at the beginning of the field) is not an indication that the developer isn't taking things seriously; it is an indication that the developer is using py2deb to package his/her product. I had to patch py2deb myself to fix this problem, and Khertan has supposedly patched other peoples' versions individually, but I'm not sure he ever released an updated version to the repository.

Re: Validity criteria for bugtrackers and bugtrackers links
 

That's the case with my stuff. Thanks for bringing that up.

Re: Validity criteria for bugtrackers and bugtrackers links
 

Yeah, he has been helpful :D - sorted mine out quickly.

He has created his own repo and the new version works well :D
Good to see developers making developers tools so to speak :D

Re: Validity criteria for bugtrackers and bugtrackers links
 

I think it is terribly ironic that the developer of the packaging sofware I use doesn't use the Extras repository.

noobmonkey: You didn't tell me, are you using py2deb or pypackager?

Re: Validity criteria for bugtrackers and bugtrackers links
 

Awesome! Thank you for this. I will be putting MaeFlight into -Testing soon hopefully.

Re: Validity criteria for bugtrackers and bugtrackers links
 

just an idea:
once there's broad consensus about what certainly is *not* a valid bugtracker, could the package interface check that, too? the way it already detects missing bug tracker links?

you certainly cannot make the machine read the wiki and check if all criteria are met... but there could be something like a blacklist based on either regular expressions or a simple string comparison.

think of the following situation: a devoloper puts package A, version 0.1 in -testing. bugtracker: talk maemo.org. - it's soon clear that talk.maemo.org is not a valid bugtracker (a specific thread might be, don't know), so there's a few thumbs down from people who know the rules. but there's also thumbs up rom those who don't care and simply think the application is cool.

a week later, the same developer puts version 0.2 in testing, along with a new package B. again, all bug tracker fields contain only talk.maemo.org. - what will happen is a race between those voting up and those voting down.... and some version may still escape to extras.

putting talk.maemo.org on the blacklist as soon as it's first seen could prevent this and wouldn't require attention of the testers wrt subsequent packages.

Re: Validity criteria for bugtrackers and bugtrackers links
 

I guess I should have put my complaints over here instead of over there. This would be a better place to discuss it.

Re: Validity criteria for bugtrackers and bugtrackers links
 

Ok, I think there's a bug here, which is quite ironic. I'm still getting reports that my bugtracker link is broken, and I think it is because the packages interface is pulling that field semi-randomly from various versions of the package, depending on factors I have yet to fathom.

For instance, earlier this morning the package overview page was showing the old broken bugtracker field, and now it is showing the correct one.

...And I just did a refresh of that page, and it is showing the broken link again!

ARGH.

EDIT: It is doing the same thing on the package instance page, too. Randomly pulling the description and bugtracker fields (but not changelog, interestingly) from old versions of the package, or the current version, depending on the whims of The Machine.

Re: Validity criteria for bugtrackers and bugtrackers links
 

The Moblin bugzilla (and maybe others) has an interesting feature: It has a "file new bug for <product>" link at the end of a page when you search for bugs in <product>. If a developer wants to make sure that users actually read existing bugs reports before filing new ones, linking directly to the query (=the search results) in the bugtracker field wouldn't, IMHO, not only be acceptable, but even better than directly linking to the "enter new bug" form.

what's your opinion?

(I think there's no such "enter new bug"-link in the search results on bugs.maemo.org)

Re: Validity criteria for bugtrackers and bugtrackers links
 

I fully agree, I was just about to write a post about the same subject. Going directly to "enter new bug" -page really doesn't encourage users to do a search first...

Re: Validity criteria for bugtrackers and bugtrackers links
 

Yes, there's a bug there, I can reproduce that behavior with your packages and with others as well.

Did you filled a bug report ?

Re: Validity criteria for bugtrackers and bugtrackers links
 

:( I hate filing bug reports...

Thankfully, I think it is this one: Bug 8694

Re: Validity criteria for bugtrackers and bugtrackers links
 

maybe slightly offtopic,sorry for that.
But ive been carrying some the thoughts for past few days and just found an opportunity to share and discuss if relevant.

Is there a way to "thumbs up" a "thumbs down" if a QA person realizes that the problem is resolved/invalid ?

Also just as we can track a thumbs down by raising a bug(if bugzilla is mandated in the future though I dont think this is happening at the moment even for packages with a bugtracker link now ) there should also be a way to track it to completion by the QA/Dev who raised/resolved it. Like DEV assigning back the bug to QA with a resolution so the "thumbs down" can either be removed or toggled to "thumbs up". ( manually or auto )

Basically trying to ensure a package doesnt get a "thumbs down" if somebody just feels it does not meet the criteria ( which may be correct/incorrect) and then carry on and may never revisit.
Keeping a framwework to resolve every reported problem on a package is needed IMHO.

If the bug isnt resolved by DEV, and the package gets to extras ( with 10 votes including enough "core" QA votes ) the bugs list may be published as part of release notes, there can be many other actions that can be discussed also.

That also raises another already raised issue about transfer of existing votes to a "new" version of the package that may fix bugs reported.

feel free to dismiss as non-relevant :)

Re: Validity criteria for bugtrackers and bugtrackers links
 

Currently only the voter can do that.

Re: Validity criteria for bugtrackers and bugtrackers links
 

might make sense that by voting, the person then allows admin or maintainer emails for updates?

In some cases it could be well voted up, but a critical bug is located at the last minute and the maintainer wants it voted down too?

Re: Validity criteria for bugtrackers and bugtrackers links
 

X-posted from the -dev list... thoughts welcome

In retrospect, I think that the mailto scheme is actually the correct one and
that the 'historical' plain mail address is the one more likely to cause
trouble in the future. Let me elaborate: if we use automatic
tools/applications (extras-assistant, appwatch, something based on texrat's
framework, you name it), defining an URL there is the right thing to do: if
it's a mailto: scheme, the browser will open the mail client, so it will work.
With putting simple mail addresses in there, any potential automated client
(including the maemo.org interface) will have to make this analysis whether
the giver string is a mail address, whether it's correct, etc, etc. I'm
inclined to say the bugtracker should be a proper URL (whether http or mailto
doesn't matter as long as it can be automatically handled with a browser). The
maemo.org interface will have to take this into account either way. Of course,
should we reach an agreement of such a change, we should probably send out a
notification mail to all package maintainers using the 'old' scheme in the
bugtracker fields (not that difficult to find with grep). Comments welcome.

Re: Validity criteria for bugtrackers and bugtrackers links
 

attila77, that makes perfect sense (and thanks for the nod).

IMO we need to design with automation in mind, even if that automation isn't available, currently practical, etc. Designing around manual methods and then trying to slap on automation creates a nighmare for the poor guys doing that work (been there done that). On the other hand, if we consider an automated solution as we design the process and incorporate the right hooks, it can still be done manually, easily, and is ready if/when automation layers can be added.