Sprint task: Refine the karma system (community input requested) - Page 4 - maemo.org

Active Topics

Porting apps to Leste (32)
to Maemo 7 / Leste by Arno_11 - 4 days, 5 hrs ago
[ANNOUNCE] CSSU-thumb thread - stable Thumb2 on N900 (2,264)
to Maemo 5 / Fremantle by panjgoori - 6 days ago
Full linux distros on Sailfish OS (254)
to SailfishOS by qoh - 6 days, 22 hrs ago
more...

Page 4 of 19

Thread Tools

	Texrat	2010-01-13 , 21:23
	Posts: 11,700 \| Thanked: 10,045 times \| Joined on Jun 2006 @ North Texas, USA	#31

Originally Posted by penguinbait

The purpose is clear. To have a system to quantify community involvement equally across measurable tasks.

Wiki, Mailiing lists, TMO, Apps, Bugzilla and the rest.

What to do with these numbers is outside the scope of the process of creating the system to quantify the community involvement.

The hard part is determining "what equals what" fairly to everyone's satisfaction IMO.

That's a proximate purpose. RevdKathy and I are looking for ultimate purpose. So people have karma-- what does Maemo want to do with it? As discussed above... and no, it's not outside the scope of the system discussion. It comes first.

__________________
Nokia Developer Champion
Different <> Wrong | Listen - Judgment = Progress | People + Trust = Success
My personal site: http://texrat.net

Quote & Reply |

	GeneralAntilles	2010-01-13 , 21:35
	Posts: 5,478 \| Thanked: 5,222 times \| Joined on Jan 2006 @ St. Petersburg, FL	#32

Originally Posted by Texrat

That's a proximate purpose. RevdKathy and I are looking for ultimate purpose. So people have karma-- what does Maemo want to do with it? As discussed above... and no, it's not outside the scope of the system discussion. It comes first.

I think karma serves two purposes: ego and identifying contributors.

Ego isn't particularly important. It's fun, and it encourages continued and additional contribution. Identifying contributors, however, is, as it's used to hand out sponsorships, discounted devices, t-shirts, etc.

So, we want a system that identifies contributors are thoroughly and effectively as possible.

__________________
Ryan Abel

Quote & Reply |

The Following 5 Users Say Thank You to GeneralAntilles For This Useful Post:
Jaffa, jukey, mmlado, penguinbait, sjgadsby

	GeneralAntilles	2010-01-13 , 21:37
	Posts: 5,478 \| Thanked: 5,222 times \| Joined on Jan 2006 @ St. Petersburg, FL	#33

Originally Posted by penguinbait

The hard part is determining "what equals what" fairly to everyone's satisfaction IMO.

There's no way everybody is going to be satisfied with a system trying to quantify contribution. Better to focus on keeping everyones dissatisfaction to a minimum, I think.

__________________
Ryan Abel

Quote & Reply |

The Following 5 Users Say Thank You to GeneralAntilles For This Useful Post:
mmlado, penguinbait, qole, RevdKathy, sjgadsby

	qgil	2010-01-13 , 21:50
	Posts: 3,105 \| Thanked: 11,088 times \| Joined on Jul 2007 @ Mountain View (CA, USA)	#34

Originally Posted by debernardis

Nokia plans device programs the way they need and like (...). Karma has been the chosen independent variable this time

Let me re-post here my earlier post in the maemo-community list discussion to avoid further confusion:

Let's put this karma use for developer devices in its context:

- First there were a handful of handpicked developers chosen by Nokia
with the assistance of the community (Fremantle Stars discussion +
questions to the council).

- Then there were 300 devices loaned in the Maemo Summit.

- Then the DDP for maemo.org contributors with >200 karma.

- Now we will start with
http://wiki.maemo.org/Fremantle_Developer_Device_Queue since I finally
got devices (and got myself back from holidays).

- There are also Forum Nokia activities were more device loans are
involved locally: Copenhagen, BCN Long Weekend, other might come.

Based on the experience I think in Harmattan it will be done slightly
difference, avoiding NDAs completely and providing hardware to the most
interesting / active projects at each stage. Karma will still help but
as the side element that must be and not as a primary rule.

Still karma is relevant for other things like e.g. getting sponsored to
events, becoming a betatester of unreleased software etc. The specific
karma (e.g. bugs filed/commented) might become more and more relevant to
help organizing focused activities (e.g. wiki karma for a documentation
hackfest).

__________________
http://maemo.org/profile/view/qgil/ + http://qt-project.org

Quote & Reply |

The Following 2 Users Say Thank You to qgil For This Useful Post:
GeneralAntilles, Texrat

	qole	2010-01-13 , 21:53
	Moderator \| Posts: 7,109 \| Thanked: 8,820 times \| Joined on Oct 2007 @ Vancouver, BC, Canada	#35

How does that go? "A good compromise is one where nobody's happy"? Something to that effect...

__________________
qole.org --- twitter --- Easy Debian wiki page
Please don't send me a private message, post to the appropriate thread.
Thank you all for your donations!

Quote & Reply |

The Following 2 Users Say Thank You to qole For This Useful Post:
penguinbait, Texrat

	fpp	2010-01-13 , 21:57
	Posts: 2,853 \| Thanked: 968 times \| Joined on Nov 2005	#36

Originally Posted by RevdKathy

So assuming we put the current system to one side we start the debate again from scratch: what do we want to reward?

I think this whole "reward" issue, as Benny first remarked, is the entire problem.

Valerio's initiative is certainly well-meant, and may well induce improvements in the way karma is computed. But in the current context, it is doomed to fail by design, because there is no way to improve what it means.

Sorry if I don't credit everyone correctly in the synthesis below. I enjoyed reading this thread immensely, because it suddenly clarified half-thoughts I'd had in my mind about karma without knowing they were there, because I'd never put them into words :-)

PB's, GA's and Benny's contributions were the ones that rang the most bells. My way of reformulating things is that the karma issue stems from the confusion between absolute and relative.

I subscribe to the view that karma, in our context, is best viewed as a measure of the activity and involvement of a given person in the Maemo community. It even defines and compares (weights) different types and modes of involvement. In that sense, striking a better balance between those types and modes in the calculus, if the current one appears skewed, is a worthy goal. But the result is still only a relative indicator, one that allows comparisons between community members, and maybe "benchmarking" of individuals versus their own chosen "role models"... But I think it's been abundantly shown there is no reasonable way to correlate it to a person's absolute value to the community.

I also believe Benny is right that linking karma to monetary value (Kathy's "reward") was the move that lit the fuse, because it turned it into an absolute metric: 200 gets you a DDP-sponsored N900 (a 400€ value, in France... :-), 199 doesn't... that's enough to raise some serious questions about who is valuable to the community, who isn't (or less), and how that is determined.

I know it did for me, because I had never worried about karma before, in four years of modest but constant involvement in the ITT+TMO side of things. Then I realized I had only made it into DDP by 4 karma points... In the following two months, I added another 75, through simple experiments in the other parts of maemo.org that took me a lot less time and energy than writing helpful or thoughtful posts over here (except for the blog, which is not that successful).

I think that's enough proof: as long as karma determines such non-trivial perks, or is even suspected thereof, there will be karma-whores and bitter bickering, which is counter-productive in the end.

Spewing witty one-liners scores the same as spending ages on a post like this one. Ditto for honing a blog post that tells a story, vs. the 5th blaring announcement of the same news bite on the same day, complete with three lines of text and a pasted picture. And there's no way around that, because karma calculus scripts just count those things, they don't evaluate their merit... not this side of the Discontinuity, anyway :-)

So I guess karma should be promoted back to its natural relative index role inside the community, with no strings attached. The absolute decision of who gets rewarded, and for what merit, should follow a separate (if probably also flawed :-) method.

__________________
maemo blog

Last edited by fpp; 2010-01-13 at 22:02.

Quote & Reply |

The Following 5 Users Say Thank You to fpp For This Useful Post:
EIPI, frals, noventa98, qole, sjgadsby

	Texrat	2010-01-13 , 21:58
	Posts: 11,700 \| Thanked: 10,045 times \| Joined on Jun 2006 @ North Texas, USA	#37

Originally Posted by GeneralAntilles

I think karma serves two purposes: ego and identifying contributors.

Ego isn't particularly important. It's fun, and it encourages continued and additional contribution. Identifying contributors, however, is, as it's used to hand out sponsorships, discounted devices, t-shirts, etc.

So, we want a system that identifies contributors are thoroughly and effectively as possible.

Forget ego... I think most of us in this discussion are looking at karma for practical use, aren't we? That gets to the point about sponsorships and other rewards. THAT is the purpose I'm talking about.

And to that point, do we really need to abstract-cum-harmonize various contribution types, as we do now, OR is it ok to just let every activity have its own idependent rating system?

I ask that, again, because it's related to how Maemo wants to utilize it. Again: would they prefer to pull specific numbers of rewardees from various pools (developer/tester/talker/etc) OR to aggregate us and risk being top-heavy with some types and short on others?

Sorry if I failed to make that clear several posts back... but I think Maemo's needs/motives are paramount to know before we go further, especially with redesign talk...

__________________
Nokia Developer Champion
Different <> Wrong | Listen - Judgment = Progress | People + Trust = Success
My personal site: http://texrat.net

Quote & Reply |

The Following User Says Thank You to Texrat For This Useful Post:
mmlado

	penguinbait	2010-01-13 , 22:09
	Posts: 3,096 \| Thanked: 1,525 times \| Joined on Jan 2006 @ Michigan, USA	#38

Originally Posted by Texrat

That's a proximate purpose. RevdKathy and I are looking for ultimate purpose. So people have karma-- what does Maemo want to do with it? As discussed above... and no, it's not outside the scope of the system discussion. It comes first.

I disagree, what we do with it is irrelevant of how the system works. Nokia may or may not use it for X. This is irrelevant, first make a fair working system, then if it makes sense to use for DDP or sending t-shirts or voting or whatever, it will have been determined to be a fair process. I fail to see how we use the information making a difference in fairly recording everyone's contributions.

The fact is, it can and will be used for multiple things, some not even thought of yet.

The goal remains making the metric, and keeping it fair, right?

__________________
To all my Maemo friends. I will no longer be monitoring any of my threads here on a regular basis. I am no longer supporting anything I did under maemo at maemo.org. If you need some help with something you can reach me at tablethacker.com or www.facebook.com/penguinbait. I have disabled my PM's here, and removed myself from Council email and Community mailing list. There has been some fun times, see you around.

Quote & Reply |

	penguinbait	2010-01-13 , 22:11
	Posts: 3,096 \| Thanked: 1,525 times \| Joined on Jan 2006 @ Michigan, USA	#39

Originally Posted by qgil

Let me re-post here my earlier post in the maemo-community list discussion to avoid further confusion:

Which is why I posted the link to this thread on my initial post, but I think nobody read it

Quote & Reply |

	Texrat	2010-01-13 , 22:12
	Posts: 11,700 \| Thanked: 10,045 times \| Joined on Jun 2006 @ North Texas, USA	#40

Originally Posted by penguinbait

I disagree, what we do with it is irrelevant of how the system works. Nokia may or may not use it for X. This is irrelevant, first make a fair working system, then if it makes sense to use for DDP or sending t-shirts or voting or whatever, it will have been determined to be a fair process. I fail to see how we use the information making a difference in fairly recording everyone's contributions.

The fact is, it can and will be used for multiple things, some not even thought of yet.

The goal remains making the metric, and keeping it fair, right?

Why do I feel like my "mouth is moving but nothing is coming out"?

It IS important IF Maemo needs percentages of representation from various contributor types.

If so, then the system needs to be comprised of independent metrics.

If not, metrics can be abstracted/weighted/etc like we're doing now and no harm, no foul.

I think we need that answered. Otherwise we waste time on a system that may not meet Maemo's needs.

EDIT: this is not meant as a slam on anyone-- but I do data modeling/querying/reporting for a living. I've seen too many times what happens when a "solution" is designed without due consideration of downstream effects, particularly consumption. If we're back to karma for ego sake only, fine: I'll shut up. But if it's to have any meaning, I'm gonna keep harping on the needs-driven horn.

EDIT 2: lest I be seen as ridiculously pedantic...

Maybe what's best is two systems. One of raw data that captures and publishes metrics for independent activities, and one that abstracts and rolls up those independent values into a high-level solution. That should accomodate everyone and everything.

Fair enough?

__________________
Nokia Developer Champion
Different <> Wrong | Listen - Judgment = Progress | People + Trust = Success
My personal site: http://texrat.net

Last edited by Texrat; 2010-01-13 at 22:26.