In reply to:

Upvoted for admirable restraint in not linking or naming.

Several times the troll mentioned he was forced to slow down posting by the site. Was this because of low karma? If so, can we just penalise people more for massively negative karma?

Apparently there is a 10 minute countdown between comments if you have negative karma.

On Debates with Trolls

One of the best achievements of the LessWrong community is our high standard of discussion. More than anywhere else, people here are actively trying to interpret others charitatively, argue to the point, not use provocative or rude language, apologise for inadvertent offenses while not being overtly prone to take offense themselves, avoid their own biases and fallacies instead seeking them in others, and most importantly, find the truth instead of winning the argument. Maybe the greatest attribute of this approach is its infectivity - I have observed several newcomers to change their discussing habits for better in few weeks. However, not everybody is susceptible to the LW standards and our attitude produces somewhat bizarre results when confronted with genuine trolls.

Recent posts about epistemology1 have all generated large number of replies; in fact, the discussions were among the largest in the last few months. People have commented there (yes, I too am guilty) even if it was clear that the author of the posts doesn't actually react to our arguments. After he was rude and had admitted to do it on purpose. After commiting several fallacies, after generating an unreasonable amount of text of mediocre to low quality, after saying that he is neither trying to convince anyone nor he is willing to learn anything nor he aims for agreement. In short, perhaps all symptoms of trolling were present, and still, people were repeatedly patiently explaining what's wrong with the author's position. Which reaction is, I must admit, sort of amazing - but on the other hand, it is hard to deny that the whole discussion was detrimental to the quality of LW content and was mostly a waste of time.

So, here is the question: why didn't we apply the don't feed the troll meme, as would probably happen much sooner on most forums? I have several hypotheses on that.

1. We are unable to recognise trolls for lack of training. The first hypothesis is quite improbable, given that the concerned troll was downvoted to oblivion2, but still possible. There are not many trolls on LW and perhaps it is difficult to believe that someone is actively seeking that sort of confrontation. I have never understood the psychology of trolls - I try to avoid combative arguments instinctively and find it hard to imagine why somebody would intentionally try to create one. Perhaps a manifestation of the typical mind fallacy combines with compartmentalisation here: although we consciously know that there are trolls out there (as this is hard to ignore), when meeting one our instict tells us that the person cannot be so much different from us.

2. We are unwilling to deal with trolls. The second theory is that although we know that a person isn't sincere, we cherish our standards of discussion so strongly that we still try to respond kindly and maintain a civil debate, or at least one side of the debate. If it is the case, it is not automatically a bad policy. Our rationality is limited and we always operate under the threat of self-serving biases. Some quasi-deontological rule of kindness in debates, even if it is an overkill, may be useful in the same way presumption of innocence is useful in justice.

3. Sunken costs. Once the debate has started, our initial investments feel binding. It is unsettling to quit an argument admitting that it was completely useless and we have lost an hour of our life for nothing. Sunken costs fallacy is well known and widespread, there is no reason to expect we are immune.

4. Best rebuttal contest. An interesting fact is that not only the number of replies was fairly large, but also lot of replies were strongly upvoted. It leads me to suspect that those replies weren't in fact aimed at the opponent in the discussion, but rather intended to impress the fellow LessWrongers. Once the motivation is not "I want to convince my interlocutor" but rather "I can craft an extraordinarily elegant counter-argument which until now didn't appear", the attitude of the opponent doesn't matter. The debate becomes an exercise in arguing, a potentially useful practice maybe, but with many associated dangers.

5. Trollish arguments are fun. I include this possibility mainly for completeness since I don't much believe that significant number of LW users enjoy pointless arguments. But still, there is something fascinating in fallacious arguments. They are frustrating to follow, for sure, especially for a rationalist, but I cannot entirely leave out of consideration the appeal of seeing biases and fallacies in real life, as opposed to mere reading about them in a Kahneman and Tversky paper.

Whatever of the above hypotheses is correct, or even if none of them is correct, I don't doubt that on reflection most of us would prefer to have less irrational discussions. The karma system works somehow, but slowly, and cannot prevent the trollish discussions from gaining momentum if people continue in their present voting patterns. One of the problems lies in upvoting the rebuttals which gives additional motivation for people to participate. There seem to be two main strategies of voting: "I want to see more/less of this" and "this deserves more/less karma than it presently has". The first strategy seems marginally better for dealing with trolls, but both strategies should work better when applied in context. Even a brilliant reply should not be upvoted when placed in an irrational debate: first, it is mostly wasting of resources, and more, we certainly want to see less irrational debates. I don't endorse downvoting good replies, if only because the troll could interpret it as support for his cause. But leaving them on zero seems to be a correct policy.

 


1 I am not going to link to them because I don't want to generate more traffic there; one of those posts figures already on the 4th place when you Google lesswrong epistemology. Neither I write down the precise topic or the name of the author explicitly, which I hope decreases the probability of his appearing here.

2 In fact, the downvoting, even if massive, came relatively late, with the person in question being able to post on the main site after several days.

 

Comments

sorted by
magical algorithm
Highlighting new comments since Today at 1:06 AM
Select new highlight date
All comments loaded

So, here is the question: why didn't we apply the don't feed the troll meme, as would probably happen much sooner on most forums?

To your question, my own question: Why would we apply the "don't feed the troll" meme? There is no reason to believe it works. There's no academic study of least trollish places on the web, nor are there reams of data with which we can analyze best responses. Interestingly, the most trollish places on the internet tend to rely entirely on the "don't feed the troll" meme. While the least visibly trollish places on the web all use distributed moderation systems along with an active defense by all the posters there to combat trolling.

Using 4chan's method of ignoring trolls when we could be following Stackoverflow's method of obliterating trolls is insane. It's like asking for advice on passing our driving test from the girl who failed the test 12 times. Sure she's got more experience than anyone, but that's only because she so bad at it. We have no reason to believe that "don't feed the trolls" works. Going further, I'd say we have good evidence to believe it doesn't work. We should first determine what deters and eliminates trolls best, instead of aspiring to an untested ideal.

"Don't feed the trolls" is just a meme that may or may not work. Nothing more and nothing less.

I agree that 'don't feed the trolls' is an untested folk theory that we shouldn't necessarily assume works, but your argument seems to assume that the expected result of not feeding trolls is no trolls at all, which I'm fairly sure is a strawman. The point of not feeding the trolls is, as far as I can tell, to minimize how disruptive they are when they appear. It's more analogous to "if you cut yourself, apply first aid and consider seeing a doctor" than "if you want to avoid scurvy, make sure to get some vitamin C in your diet".

The poster in question at least wouldn't be immediately obvious as a troll to outsiders reading only a small part of the discussion, more proactive ways to deal with posters like that would seem to carry a serious risk of making less wrong appear even more cultish (group-think).

For whatever measures we take we should first consider how much ammunition against less wrong they offer, how likely they are to cost us genuinely valuable contributions due to seeming closed to dissent, and whether the expected magnitude of intended effects is worth that.

That includes using the word "troll". After the various facts about their behavior and motivations there is no additional fact as to whether they are a troll. Using the word troll might easily lead people who only took a quick look to come away with the impression that we generally dismiss non-bayesians as trolls, whereas talking about how to prevent endless discussions not aimed at resolving disagreements seems less dangerous that way.

That includes using the word "troll". After the various facts about their behavior and motivations there is no additional fact as to whether they are a troll. Using the word troll might easily lead people who only took a quick look to come away with the impression that we generally dismiss non-bayesians as trolls, whereas talking about how to prevent endless discussions not aimed at resolving disagreements seems less dangerous that way.

Indeed. I'm not even sure the user in question was a troll by intention (even if they were one functionally) - being persistent and dense beyond reason is a highly plausible trait of participants in Internet philosophy discussions, after all, particularly when the participant has their very own site all about what they're talking about.

That is, the label "troll" assigns intent in a way that is not actually all that relevant to the problem, which is the behaviour, when you can accurately describe the problematic behaviour.

"I prefer trolls to cranks, because trolls sometimes rest." - Alexandre Dumas (fils) (loose translation)

Your first suggestion, "we are unable to recognize trolls", is a valid one. I personally didn't think of the downvoted user as a troll, just as someone who supported an incorrect epistemology and had a couple of confused ideas. It was only after said user failed to learn from several clear, well-written replies that it became clear that ver presence on LW was a bad thing.

It wasn't immediately clear to me either. But the discussions had continued well after said user failed to learn from that replies. And I have written at least two responses to his posts when I already knew it was going nowhere, and it required non-negligible willpower to give up after that.

A really good troll would be able to maintain a high karma level even while wasting Users' time on trivial matters -- and this forum has been able to fend off even that kind, so I would not regard it as worrisome that an occasional less-obvious troll like User:[withheld] succeeds.

I don't think the "trolls" started that way - in fact claiming to be trolling (or that they don't care, etc) is a common defensive response, especially on the internet, after which appearances and cognitive dissonance do lead to some actual trolling.

They were just people who disagreed with with the majority, with the unfortunate need to frame everything in terms of their favorite topic. Sure their arguments were well below normal LW standards, but we have high standards. They were normal, pre-rationality-training arguers, and not talking to normal people doesn't appeal to me at all.

For what it's worth -- I had one exchange with the person in question pretty early on, decided based on their response that the conversation wasn't going to go anywhere useful, and dropped it.

But I mostly refrained from downvoting them initially because there were people I respected who were continuing the discussion with every indication that it was being productive, and I value productive discussion even if I'm not getting anything out of it personally. (There are a lot of exchanges on this site that kinda sound like gibberish to me, and in at least some cases I'm fairly confident that this is because I don't understand the issue well enough to participate usefully in the discussion even as an observer, not because the participants are in fact spouting gibberish.)

After a very short while I stopped reading any of the comments on that thread except to sample them every once in a while to see if they were going anywhere, and it seemed pretty clear that they weren't. At that point I started downvoting everyone involved on the grounds that I want less high-volume discussion that makes no progress.

No general lesson here, just another data point.

How about 6.: arguing for the lurkers' benefit? That argument was voluminous, repetitious, and random sampling of the low-voted comments turned out to only be useful for verifying that their scores weren't capricious, but a few of the upvoted comments were worth my time to find and read; if a few dozen other people felt the same then they might have also been worth the authors' time to write.

As someone who was probably doing more feeding than most, I'd like to apologise here.

In my case it was primarily the 'didn't recognise troll' problem, I'm not very good at distinguishing the more eloquent and seemingly reasonable type of troll from honest commenters who disagree with me. I also have a strong aversion to just walking out on a debate without reaching any kind of agreement, mainly because it annoys me a lot when other people do it.

I will try not to fall into a similar pattern again, but since I'm not all that good at noticing it any 'don't feed the troll' PMs are appreciated.

I'm in a similar situation - I can, technically, recognize that someone fits the profile of 'troll', but my brain doesn't like to actually use that information for anything. (It's not just an issue with trolls, it's a general inability to use different scripts for people based on categorizing them.) What I've found works, in my case, is to be more aware of subtleties in peoples' behavior, rather than trying to categorize them. I still wind up feeding trolls sometimes (and in fact tend to enjoy doing so in those cases), but if someone is being logically rude or otherwise offensive, that's a thing worth noticing and reacting to whether they're 'a troll' or not.

Another trick that's useful sometimes is backing out of the argument a bit and looking at the bigger picture. Sometimes that shows patterns that aren't otherwise obvious, and that can make it clear that the person isn't worth continuing to deal with. For example, in the most recent case, the individual was defying the evidence in an irrational and not very obvious way. Once I noticed that, it seemed obvious to me that without a more thorough understanding of when that is and isn't a reasonable thing to do, they weren't going to stop doing it to any evidence that we gave them, and thus the argument at hand was not going to resolve anything, so it was pointless. It's a lot easier to walk away in a situation like that, when you can see that it's impossible to actually get the goal you were aiming for.

1. We are unable to recognise trolls for lack of training.

2. We are unwilling to deal with trolls.

3. Sunken costs.

4. Best rebuttal contest.

5. Trollish arguments are fun.

It's good to recognize how tempting these are. In fact, my impression was that the exact same mental bugs were motivating the "troll" in this case to continue with the conversation. As far as motivation for participating in that specific conversation was concerned, the only difference between the "troll" and his interlocutors was that the interlocutors knew for sure that they had a large approving audience.

There's another issue here- a lot of my comments in reply (almost all of them) were upvoted. Some were upvoted quite high. I interpreted this as a sign that people were interested in the subject when it seems in retrospect I should have interpreted those upvotes more as "agree" or "well-argued" and not "want more". Being more clear about what people mean when they upvote might help.

Replying to people who are wrong or systematically wrong is not a problem, so long as we keep the good Less Wrong tradition of addressing the statements with excruciating seriousness, like this.

The problem appears when people who systematically produce wrong or low-quality content don't slow down in modes in which they get downvoted. Global Karma level and 10 minute delay don't address this problem directly.

Perhaps downvotes should act as a cooler measure, temporary ban points, for example:

  • Count the total Karma K of all comments published within the last 2 days that have total negative Karma.
  • If K is less than -10, and the last comment with total negative Karma was made at time T, you're not allowed to comment before time T+(|K|*30 min).

This would be too harsh sometimes, but it would automatically prevent escalation of negatively-judged discussions.

That system would also ban people for some time after they set up a poll. Perhaps it would be better to let K bet the total Karma of all comments, not just negative Karma comments, so good comments could offset bad ones.

Upvoted for admirable restraint in not linking or naming.

Several times the troll mentioned he was forced to slow down posting by the site. Was this because of low karma? If so, can we just penalise people more for massively negative karma?

Apparently there is a 10 minute countdown between comments if you have negative karma.

I'm unfamiliar with this particular instance, but I've engaged trolls on Fark before, for these reasons:

1) Some trolls just give up when taken calmly seriously at face value, as opposed to getting hit with indignation.

2) Bored at work.

3) They provide such wonderful strawmen against which to clarify my own thoughts and sharpen my arguments.

4) In some discussions (political ones especially), Poe's Law applies, oftentimes real people hold real opinions that are as blunt as trolls' opinions.

In some discussions (political ones especially), Poe's Law applies, oftentimes real people hold real opinions that are as blunt as trolls' opinions.

In the discussed case I have little doubts that the said troll indeed held most of the opinions he has expressed here. To clarify, by "troll" I mean someone who argues for fun of arguing and without caring a bit about the standards of a reasonable discussion; a troll needn't to pretend holding opinions which he in fact doesn't share. (My use of "not being sincere" in the OP might be misleading in this respect.)

I recommend numbering your hypotheses, for easy reference.

There are some more constructive reasons for arguing with the incorrigible. One is to persuade the audience, rather than your interlocutor -- anything you say on a public stage is addressed as much to the general readership as to the individual being explicitly addressed. Another is to practise your skill at arguing the material.

Not that these justify going on and on indefinitely, but they are worth putting in the balance.

... all symptoms of trolling were present, and still, people were repeatedly patiently explaining what's wrong with the author's position. Which reaction is, I must admit, sort of amazing ...

Amazing only if you assume that educating that trollish author is the purpose of the response. Frequently, though, one responds imagining an audience much larger than a single troll. Sometimes one writes experimentally, for oneself, seeking feedback from the community as to whether one's own viewpoint finds resonance with other people.

it is hard to deny that the whole discussion was detrimental to the quality of LW content and was mostly a waste of time.

Some people think it is hard to deny that participating in sex while using a condom is a waste of time. Yet I have heard people deny it.

I don't much believe that significant number of LW users enjoy pointless arguments.

Yeah. Right.

My guess is that you are pretty close to the truth with your "best rebuttal contest" and "fun" hypotheses. You are participating in a forum with several hundred people. You can't expect that all of them will share your own austere tastes in intellectual entertainment.

Good post, thanks.

I note you don't list as a hypothesis "even though the interlocutor was a troll, their arguments had enough merit to be worth the effort of rebutting". I didn't read most of the threads; I take it if I had, I'd know that wasn't a good hypothesis? Or does this come under the "Best rebuttal contest" heading?

It was rather an unstated assumption that the arguments have not enough merit. Of course a troll can make few good arguments, and perhaps there were few in those threads, but most of the debate wasn't centered around reasonable arguments.

The worth rebutting property itself can possibly be further reduced with respect to the motivation of the debater. Arguments may be rebutted for pure intellectual curiosity, just as "let's look whether I can find a hole in that reasoning", they may be rebutted to ensure that other people don't fall victim to fallacious deductions, or they may be rebutted to show one's intellectual superiority or other social signalling purposes. Those are different mechanisms and an argument worth rebutting for one reason may not be worth the same for other reasons (in fact, presence of the signalling reasons depends more on the context than on the argument itself).

I don't endorse the rebutting for signalling, as it is in fact what the trolls generally aim for. When other motivations are present, a better way is to create a separate post to discuss the problem, ideally after waiting for a while until the troll disappears. Instant rebutting will likely be contaminated by trollish distractions and thus suboptimally productive.

We could hide downvoted posts from "recent posts" (make them available only to those who already know the url) and hide downvoted comments from "recent comments". And hide all children of downvoted posts/comments from "recent comments" too. That would discourage feeding.

And hide all children of downvoted posts/comments from "recent comments" too.

Undesired side effect.

  1. Trollish arguments are fun.

As something of a troll, depending on what is meant by that at least, it is often quite interesting to debate with people that do not hold your point of view and are capable of making decent arguments. Too often one runs into arguments where both sides don't know what they are talking about in the slightest, which is terribly frustrating.

As for the Karma, I have had for a while enough karma to make a top level post, getting it isn't particularly hard. I would have much higher karma if I didn't occasionally post controversial comments.

If I were determined to actually troll I would create a group of accounts and post thoughtful comments for a day or so to build up some karma from outside sources. Then I would up vote one account using the dozen or so other accounts I had to give it plenty of Karma with which to make top level posts even with a decent amount of down votes. To slow the drop off I could continue up-voting that account with the other ones, however this wouldn't be worth while. Better would be to craft on the other accounts a series of posts attacking the top level troll post so that these other accounts get upvoted by tearing into the troll. This could create serial accounts to dominate the recent posts threads over a decently long period of time. I am sure there are ways in which this process could be optimized from the trolls point of view.

I have no intention of ever posting a top level post that is highly controversial. I am not that kind of troll.

Also one could probably create an semiautomatic spammer as well that had the purpose of defeating the karma requirements without much difficulty.

Basically create a core of accounts that is the size of the minimal number of Karma needed to make a top level post. Then one of those accounts needs to get one Karma, which hopefully can't be automated. Then one has to create some banal discussion that includes one post from each of the core accounts which then gets upvoted in a cycle.

Then start creating spam accounts, have it post randomly something like "I agree" on something and have the core accounts upvote that one post so the spam account can post the spam on the top page. Rinse and repeat.

It would be important to avoid detection to have a scripted discussion between the core accounts and to not have the spam accounts post on that discussion except according to the random algorithm that is being used.

There are probably ways to make this simpler to create and there may be easy ways to defeat such a program. This is just an idea.

Responding to curi was good training. I found myself discovering, for myself, what levels parts of my understanding were at - automatically seeing exactly where he went wrong, or being able to derive from my web of beliefs a principle that was violated... and I could go on.

Point is, I fed the trolls because it was good exercise.