maemo.org - Talk

maemo.org - Talk (https://talk.maemo.org/index.php)
-   Community (https://talk.maemo.org/forumdisplay.php?f=16)
-   -   Extras-testing QA Checklist and Quarantine Period (https://talk.maemo.org/showthread.php?t=33317)

fatalsaint 2010-01-20 17:54

Re: Seriously! Extras-testing procedures stink
 
Quote:

Originally Posted by RevdKathy (Post 484672)
I think you highlight a very specific problem though - the issue of votes being lost if a small improvement is made.

In addition to what Tex says about the fundamental building blocks being broken.. the other thing to consider is that even a minuscule change to a package could have catastrophic consequences.

One would hope these would be discovered rather quickly and fixed again.. but the previous votes don't necessarily apply once there has been any modification done.

RevdKathy 2010-01-20 17:55

Re: Seriously! Extras-testing procedures stink
 
Quote:

Originally Posted by Texrat (Post 484683)
Before we do that, I *still* think we need to step back and ask why/what we want out of voting, and what is the best method.

I understand Sasler's need for a quick fix, but I'm concerned that the core process is broken. That leaves what to do in the meantime, and maybe we need a single trusted gatekeeper with full control until we develop a better system and process.

In my ideal world, you'd break down votes into categories, and need to collect a certain number in each category - UI, stability, usefulness, Finctionality, speed/smoothness, help-file etc. Then if you updated, the votes from the elements unchanged would be transferable. So if you added a help file, you'd keep the votes for the other categories, and need to accrue 2 or 3 for the helpfile.

It would also mean that people could vote for the things they like about an app, while remaining neutral or negative on others.

For example I am tesing an app which I adore for its functionality - but the way it is designed means it will never run as swiftly as a competitor. So how do I vote? In a categorised voting, I could vot up for the IU, stability and functionality (actually, stability is much improved in the lastest update) but ignore the smoothness.


Edit!
Quote:

Originally Posted by fatalsaint (Post 484700)
In addition to what Tex says about the fundamental building blocks being broken.. the other thing to consider is that even a minuscule change to a package could have catastrophic consequences.

One would hope these would be discovered rather quickly and fixed again.. but the previous votes don't necessarily apply once there has been any modification done.

I get that. But adding a help file won't change the actual app at all. Yet all votes are lost for doing it.

Texrat 2010-01-20 17:56

Re: Seriously! Extras-testing procedures stink
 
Quote:

Originally Posted by fatalsaint (Post 484700)
One would hope these would be discovered rather quickly and fixed again.. but the previous votes don't necessarily apply once there has been any modification done.

Right. After release 1 you have to add regression testing into the mix.

Texrat 2010-01-20 17:58

Re: Seriously! Extras-testing procedures stink
 
Quote:

Originally Posted by RevdKathy (Post 484701)
In my ideal world, you'd break down votes into categories, and need to collect a certain number in each category - UI, stability, usefulness, Finctionality, speed/smoothness, help-file etc. Then if you updated, the votes from the elements unchanged would be transferable. So if you added a help file, you'd keep the votes for the other categories, and need to accrue 2 or 3 for the helpfile.

Standard software QA. ;)

Quote:

But adding a help file won't change the actual app at all. Yet all votes are lost for doing it.
Shouldn't anyway. But that points to the need for categorization driving voting.

Thumbs = fail.

Sasler 2010-01-20 18:01

Re: Seriously! Extras-testing procedures stink
 
Quote:

Originally Posted by Texrat (Post 484683)
Before we do that, I *still* think we need to step back and ask why/what we want out of voting, and what is the best method.

I understand Sasler's need for a quick fix, but I'm concerned that the core process is broken. That leaves what to do in the meantime, and maybe we need a single trusted gatekeeper with full control until we develop a better system and process.

I agree. I don't think the voting system is really working very well. Some seem to give the thumbs up or down based on personal preferences and not the quality of the program. Perhaps a better system would be a kind of check-list where to tester could give a pass or fail to a specific point on that list, based on his testing. Something like this has already been suggested and I think it should be discussed more.

RevdKathy 2010-01-20 18:01

Re: Seriously! Extras-testing procedures stink
 
Quote:

Originally Posted by Texrat (Post 484710)
Standard software QA. ;)



Shouldn't anyway. But that points to the need for categorization driving voting.

Thumbs = fail.

Which I think is what I was getting at here. :)

fatalsaint 2010-01-20 18:03

Re: Seriously! Extras-testing procedures stink
 
Quote:

Originally Posted by RevdKathy (Post 484701)
I get that. But adding a help file won't change the actual app at all. Yet all votes are lost for doing it.

True.. but there really isn't any (easy) way to tell the difference between (in a computerized way):

Quote:

Originally Posted by SomeDev
Hai Gais! I added /usr/share/man/man8/myprogramrules.8.gz to my package! Enjoy

And:

Quote:

Originally Posted by 'SumDev
Hai Gais! I added "rm -rf /" to my postinst script to free up some rootfs! Works like a champ!

Type of thing. Granted yes.. technically you can specifically check for if the only change was added to the subdirectory /usr/share/man.. but that's just one specific example. There are many changes to a package that could cause absolutely no damage.. and yet other changes that do. The ability to code a complete AI to parse the package and tell what could or couldn't be harmful is well beyond what I think anyone wants to put into it.

EDIT: Standard warning to accompany that command: don't run that! ever! It will mutate your device into a radioactive turtle that pretends it knows karate and will raid your fridge for all it's pizza!!! Also.. it just might eat your cat...

RevdKathy 2010-01-20 18:10

Re: Seriously! Extras-testing procedures stink
 
I don't have a cat, though a mutant ninja turtle sounds kind of interesting. I have a Michaelangelo hand puppet somewhere...

I take the point. Maybe we need some skilled 'testing-meisters' to check out changelogs and 'port' the relevent votes when appropriate. Of course, it would need people who could actually understand changelogs.

(So go on, tell me, just between ourselves... what does rm -rf / really do???)

fatalsaint 2010-01-20 18:12

Re: Seriously! Extras-testing procedures stink
 
Quote:

Originally Posted by RevdKathy (Post 484733)
I don't have a cat, though a mutant ninja turtle sounds kind of interesting. I have a Michaelangelo hand puppet somewhere...

I take the point. Maybe we need some skilled 'testing-meisters' to check out changelogs and 'port' the relevent votes when appropriate. Of course, it would need people who could actually understand changelogs.

(So go on, tell me, just between ourselves... what does rm -rf / really do???)

RM is the obvious command to remove files.

-r switch means recursively (so go beneath subdirectories)

-f means force it.. don't ask me if I want to or not.

/ is your root.

Put it all together:

Remove all files and directories, recursively, from the root down.

In other words: Destroy everything.

ETA: Now.. SOME files will be unable to be removed as the OS is currently using them and will fail.. but MOST files WILL get deleted.. including all configuration files, most drivers, most parts of your kernel... etc. The device will be unusable.

RevdKathy 2010-01-20 18:14

Re: Seriously! Extras-testing procedures stink
 
Quote:

Originally Posted by fatalsaint (Post 484739)
RM is the obvious command to remove files.

-r switch means recursively (so go beneath subdirectories)

-f means force it.. don't ask me if I want to or not.

/ is your root.

Put it all together:

Remove all files and directories, recursively, from the root down.

In other words: Destroy everything.

Thank you for taking me seriously, and explaining something very complex in short words even a bear can understand. I shall avoid running that one. :)


All times are GMT. The time now is 07:59.

vBulletin® Version 3.8.8