1 points

expand_less expand_more

In reply to:

How hard would it be to make a modified program that tracked the up-votes and down-votes one makes with and without it (storing the comment author, the number of points the comment previously had, and one's own vote) and then printed that data in some Excel/Octave/etc.-readable format? It would be nice to know the details of my own biases.

This program is great, though. Thanks, Marcello.

It would be nice to know the details of my own biases.

Remembering, of course, correlation isn't causation :-)

See comment in full context

LessWrong anti-kibitzer (hides comment authors and vote counts)

59 points

expand_less expand_more

Mar 9, 2009

Related to Information Cascades

Information Cascades has implied that people's votes are being biased by the number of votes already cast. Similarly, some commenters express a perception that higher status posters are being upvoted too much.

If, like me, you suspect that you might be prone to these biases, you can correct for them by installing LessWrong anti-kibitzer which I hacked together yesterday morning. You will need Firefox with the greasemonkey extention installed. Once you have greasemonkey installed, clicking on the link to the script will pop up a dialog box asking if you want to enable the script. Once you enable it, a button which you can use to toggle the visibility of author and point count information should appear in the upper right corner of any page on LessWrong. (On any page you load, the authors and pointcounts are automatically hidden until you show them.) Let me know if it doesn't work for any of you.

Already, I've had some interesting experiences. There were a few comments that I thought were written by Eliezer that turned out not to be (though perhaps people are copying his writing style.) There were also comments that I thought contained good arguments which were written by people I was apparently too quick to dismiss as trolls. What are your experiences?

59 points

expand_less expand_more

Comments

sorted by

Highlighting new comments since Today at 8:12 AM

All comments loaded

Eliezer_Yudkowsky

18 points

expand_less expand_more

If the main Less Wrong codebase offered an option to hide the names of contributors and non-negative karma point totals (because negatives you may want to let your eyes just skip over), not revealing them until voted on (including a neutral vote)...

...then what would you think of only giving half a karma point for any votes made without obscuring the name and current total? (Likewise any votes changed after the reveal.) Fair? Or just annoying?

18 points

expand_less expand_more

7 points

expand_less expand_more

The half-point idea is an interesting one. I find it hard to think of genuinely good reasons for wanting to know how positive a score is before deciding whether to vote up or not, and I haven't heard any good explanations from anybody else yet either. That's to be contrasted with the multitude of bad reasons for wanting to know the score beforehand, which exploit common cognitive biases that we are all at least partially subject to. I do find the idea a bit heavy-handed, but I think it would make the community slightly less herd-like and a bit more rational.

Of course, you'd also have to change the code base so that once an initial blind vote has been made, any further changes would only register at the lower value.

As to whether it's unfair, you could explain it as getting an EXTRA half point for your willingness to make a small sacrifice for the betterment of the community at large. You could even make it 1 point versus 1.5 points.

7 points

expand_less expand_more

6 points

expand_less expand_more

that's brilliant and hilarious. We should implement more de-biasing programs by exploiting biases.

Though I'd say by this scheme it should be 1 point versus 2, just to preserve the punchline.

6 points

expand_less expand_more

5 points

expand_less expand_more

Maybe have endorcements for the program by respected members? Pictures of attractive women marvelling at Bayes on little adverts? Make it opt-out?

5 points

expand_less expand_more

5 points

expand_less expand_more

IMHO, unfair. But it would be interesting to have an option to see such a total, or to see how much karma is from info-obscured vs info-revealed votes.

5 points

expand_less expand_more

3 points

expand_less expand_more

Just annoying.

Though I frequently change my votes as I reflect on them, so even reading anonymously I'd probably end up with all devalued votes. And this process of reflection would eliminate any debiasing effect of reading anonymously.

Of course, I put a lot of stock in character, so I don't see the problem with occasionally voting up comments by better people.

3 points

expand_less expand_more

1 points

expand_less expand_more

Simply annoying from a usability point of view. It requires you to know too surely which posts you will want to vote on and which authors you'll want to know; if you care about the value of your karmic vote you'll wind up interfering with your enjoyment of the posts to preserve its value; etc.

1 points

expand_less expand_more

1 points

expand_less expand_more

Good idea Eliezer.

Then add in an autocomplete box that I can use to guess the author's name based off the comment. It's fascinating trying to pick out the fist of an author.

1 points

expand_less expand_more

11 points

expand_less expand_more

The latest version of the Anti-Kibitzer Script (click "View the original post" above this comment to lean more about it)

is integrated directly within LessWrong (no more Greasemonkey)
is controlled by a section in the user Preferences called Kibitz Options
hides names and scores from sidebars, and hides your own karma score

To activate, simply navigate to Preferences and check "Enable anti-kibitzer" - this will add a button to the upper right allowing you (without refreshing the page) to temporarily display names and scores again.

11 points

expand_less expand_more

2 points

expand_less expand_more

When the feature is active (respectively inactive), the button says "Turn Kibitzing Off" (respectively "Turn Kibitzing On"). Shouldn't it say turn "Turn Anti-Kibitzing Off" (resp. &c.)? Kibitz, to gossip, so removing the names is "anti-gossip," right?

2 points

expand_less expand_more

3 points

expand_less expand_more

Yep. This is among the bugs fixed in v0.5.

3 points

expand_less expand_more

1 points

expand_less expand_more

I don't seem to have that preference. Is it still supposed to be there?

1 points

expand_less expand_more

2 points

expand_less expand_more

Not released yet, as I alluded to elsewhere. I jumped the gun in updating the above comment to describe what's coming.

Version 0.44 is here, still packaged as a Greasemonkey script.

2 points

expand_less expand_more

11 points

expand_less expand_more

I think one of the useful things about being able to identify people by name is that you frequently end up entertaining a contrary opinion you would have otherwise dismissed because it's held by someone you respect. Such events are probably more significant to personal development than discovering a gem in the rough you would have otherwise overlooked. Hiding karma probably has more utility than hiding names.

11 points

expand_less expand_more

6 points

expand_less expand_more

How hard would it be to make a modified program that tracked the up-votes and down-votes one makes with and without it (storing the comment author, the number of points the comment previously had, and one's own vote) and then printed that data in some Excel/Octave/etc.-readable format? It would be nice to know the details of my own biases.

This program is great, though. Thanks, Marcello.

6 points

expand_less expand_more

1 points

expand_less expand_more

It would be nice to know the details of my own biases.

Remembering, of course, correlation isn't causation :-)

1 points

expand_less expand_more

4 points

expand_less expand_more

Actually, I find I have the exact opposite problem. I almost never vote. Partly that's because I read Less Wrong through an RSS feed that doesn't even show the vote totals. I only ever vote if, like now, I've gone to the actual site in order to comment.

Even then, I find that I am comparing the quality of Less Wrong posts and comments against the entire corpus of what I read on a daily basis, some of which is great, and some of which is dreck.

So, I tend to only vote when the quality of what is written is extremely good -- enough so that I want to 'reward' it -- or extremely bad, so that I want to punish. The vast majority is in the middle and so I don't bother to vote.

4 points

expand_less expand_more

3 points

expand_less expand_more

Thankyou Marcello!

I'm not particularly worried about my voting being biassed. In fact, seeing the dynamics of 'karma' at play, and feeling the impact of the social forces so explicitly is a useful experience.

Nevertheless, the 'social game' is always a distraction that takes up valuable resources. I expect I'll turn on your script fairly often, just so I can turn off the noise. I find it emotionally draining to go ahead and write posts that I know will be dissaproved of, yet still consider worth saying. All over an utterly insignficant and easily manipulated number on a server somewhere. Yet, I suppose that is just more visceral experience of the costs of rationality.

If only you could write one of those for real life. It'd make things so much less tiresome.

3 points

expand_less expand_more

3 points

expand_less expand_more

I think the latter experience is telling. my experience of trolls tells me that they get labeled as trolls because they say things that make people uncomfortable because they suspect they might be true. If the so called troll was just saying completely wacky things it would be simple to dismiss them. Someone coming into a forum and shouting white power or some such nonsense is not the same as a troll.

3 points

expand_less expand_more

2 points

expand_less expand_more

This is really great. Thanks for making it available.

My only suggestion is to hide the "Turn kibitzing On/Off" icon after a few seconds and only show it when the user hovers over that area.

The reason I make this suggestion is that while I slowly scroll down the page while reading, the icon jumps all over the place in the upper right corner as it repositions itself 10 times a second.

Even if it didn't jump around, I'd still rather have it hidden except when needed. IMHO, it stands out against the dark background and mars the otherwise very nice LW aesthetic.

2 points

expand_less expand_more

2 points

expand_less expand_more

That's a bug of sorts in the script, and easily fixable. In fact, I've already done so; have an updated version.

2 points

expand_less expand_more

1 points

expand_less expand_more

I'd still prefer it be hidden altogether after a few seconds until I hover in that area, but this is a big improvement. Thanks.

1 points

expand_less expand_more

1 points

expand_less expand_more

Your version is now the official version 0.3. However, the one thing you changed (possibly unintentionally) was to make Kibitzing default to on. I changed it back to defaulting to off, because it's easy to click the button if you're curious, but impossible to un-see who wrote all the comments, if you didn't want to look.

1 points

expand_less expand_more

1 points

expand_less expand_more

I'm significantly torn on whether to enable this. I understand the downsides of seeing authors (and am confident that I'm engaging in at least some of them), but I have one issue with it. Knowing authors can improve my ability to rapidly and effectively process posts. There's at least one author who makes very good points, but sometimes glosses over issues that turn out to be either quite complicated or openings to criticism of the post. I've found these omissions both important and quite hard to find - at the moment, its worth it to me to leave author names active just to be aware that I need to read these posts with a different style of criticism than I normally engage in.

In short, there are sometimes positive outcomes of knowing authors, if only for general efficiency increases.

1 points

expand_less expand_more

0 points

expand_less expand_more

I have my anti-kibitzer on, I've had it on for two days. I too, read certain posters more carefully than others, but now, rather than deciding who to read carefully by status, I decide who to read carefully by over-viewing the contents of their posts. Of course, you want to give more resources and time to a great master of the art, than to a moderate master. But deciding who is who by status, or letting status weight in as much as it does in humans, is almost as bad as not having any time management at all. It's like time managing, where you also falsely think that some independent variable has something to do with the content.

0 points

expand_less expand_more

1 points

expand_less expand_more

It appears as though activating anti-kibitzer causes the lesswrong, Singularity Institute, and Future of Humanity logos appear in the wrong places in the header.

1 points

expand_less expand_more

0 points

expand_less expand_more

I believe this is specifically anti-kibitzer and Firefox 4 in combination; I've had AK on for a while, and only noticed the problem when I upgrade from FF3. Other breakages include block quotation in comments and vote link bolding.

0 points

expand_less expand_more

Oscar_Cunningham

1 points

expand_less expand_more

When I see a lesswrong post in my RSS reader, it still shows the author of the post. Is there a way I can avoid this? Does anyone else experience the same problem?

1 points

expand_less expand_more

1 points

expand_less expand_more

You might be able to do something with Yahoo Pipes.

1 points

expand_less expand_more

Oscar_Cunningham

0 points

expand_less expand_more

And... Done!

http://pipes.yahoo.com/pipes/pipe.info?_id=0ebe8882acda67e183017fa5f27014c1

0 points

expand_less expand_more

1 points

expand_less expand_more

I didn't think I needed this until the current kerfuffle over gender bias came up. Still unsure if it's the most effective approach to avoiding my own biases (i.e. judging argument and author together), but it's an awesome experiment.

1 points

expand_less expand_more

1 points

expand_less expand_more

A couple notes: I still see names in "Recent Comments", and I still see names in comment permalink pages. Still, good stuff!

Edit: Also, it hides my name and my scores, which seems rather useless from an objectivity perspective...

1 points

expand_less expand_more

Vladimir_Golovin

1 points

expand_less expand_more

Just installed it, works like a charm -- I'll keep it this way. If only it could hide the sidebar as well.

Also, thanks to this post, I'm now aware of the fact that there are posts other than those listed on the homepage! I have no prior experience with Reddit, so I assumed that the front page is completely community-driven, but it turns out that it is for 'PROMOTED' posts only!

I spotted the word 'anti-kibitzer' in the sidebar, but was unable to find the post on the front page -- that's how I discovered the 'NEW' and 'POPULAR' links under the site banner.

1 points

expand_less expand_more

2 points

expand_less expand_more

I've upgraded the LW anti-kibitzer so that it hides the taglines in the recent comments sidebar as well. (Which is imperfect, because it also hides which post the comment was about, but better will have to wait until the server starts enclosing all the kibitzing pieces of information in nice tags.) No such hack was possible for the recent posts sidebar.

2 points

expand_less expand_more

Eliezer_Yudkowsky

1 points

expand_less expand_more

Well, to kibitz the anti-kibitzing, it looks to me like:

by Marcello

would match pretty easily against something that looked for

by (.+)

and deleted it, similarly on the Recent Posts but without the , checking for the identity of the two matched strings is optional (I forget how to do this offhand with REs).

1 points

expand_less expand_more

3 points

expand_less expand_more

That particular hack looks like a bad idea. What if somebody actually put a bold-face link into a post or comment? However, your original suggestion wan't as bad. All non-relative links to user pages get blocked by the anti-kibitzer. (Links in "Top contributors" and stuff in comments seem to be turned into relative links if they point inside LW.) It's gross, but it works.

Version 0.2 is now up. It hides everything except the point-counts on the recent posts (there was no tag around those.) (Incidentally, I don't have regular expressions because by the time my script gets its hands on the data, it's not a string at all, but a DOM-tree. So, you'd have to specify it in XPath.)

I think trying to do any more at this point would be pointless. Most of the effort involved in getting something like this to be perfect would be gruesome reverse engineering, which would all break the minute the site maintainers change something. The right thing to do(TM) would be to get the people at Tricycle to implement the feature (I hereby put the code I wrote into the public domain, yada yada.) Then we don't have to worry about having to detect which part of the page something belongs to because the server actually knows.

3 points

expand_less expand_more

4 points

expand_less expand_more

I hereby put the code I wrote into the public domain, yada yada.

A great way to put your code into the public domain would be to put it up somewhere like (ideally exactly like, because they're very awesome) http://github.com/.

If anyone else wants to modify your code the git (and GitHub) workflow is very good - anyone can fork your repository, and if you want to accept their changes that's easy for you to do.

See the GitHub Guides if you're interested.

4 points

expand_less expand_more

2 points

expand_less expand_more

by (.+)

by <a href="http://lesswrong.com/user/([^"]+)"><strong>([^<]+)</strong></a> would be better (you could also make your .+ non-greedy with .+?), but us (Trike) providing more classes and ids is better still. We're looking into it.

2 points

expand_less expand_more

Vladimir_Golovin

1 points

expand_less expand_more

Uh, now it's on the homepage. I'm completely confused. What determines if (and when) a post makes it to the homepage -- human intervention or some kind of algorithm?

1 points

expand_less expand_more

2 points

expand_less expand_more

Editors include Robin Hanson and Eliezer Yudkowsky; they promote articles to the "Promoted" page.

2 points

expand_less expand_more

Tyrrell_McAllister

1 points

expand_less expand_more

Thanks for making this available. Plus, thanks for the opportunity to learn about Greasemonkey :).

1 points

expand_less expand_more

1 points

expand_less expand_more

Thank you. Trying it now and it's working perfectly. It's like a dream come true.

1 points

expand_less expand_more

3 points

expand_less expand_more

Looks great to me! Unfortunately I know via Recent Comments and Recent Posts the identity of the author of the above comment... ;)

3 points

expand_less expand_more

0 points

expand_less expand_more

The "Only works in Firefox" note in the preferences is incorrect; it's working fine for me in Chrome.

0 points

expand_less expand_more