Over the summer, Eliezer suggested (approximately, I am repeating this from memory) the following method for making an important decision:

1. write down a list of all of the relevant facts on either side of the argument.
2. assign numerical weights to each of the facts, according to how much they point you in one direction or another.
3. burn the piece of paper on which you wrote down the facts, and go with your gut.

This was essentially the method I used in coming to my (probably slightly low) estimate of the probability that Knox and Sollecito were innocent. It just felt like they were innocent, and I saw essentially no reason to suspect they were guilty. I will note that the 'pro-guilt' site that komponisto linked to was just horribly devoid of anything that I might consider evidence (if anything, that site did more to convince me of Knox's innocence than the pro-innocence site), and I did spend probably about 10 minute trying to find some evidence that they had missed, but completely failed.

On a different not, as I said at the time, 0.95 and 0.05 were just proxies for "pretty damn sure" and "pretty damned unlikely" - I have very little idea what 5% probability feels like, and I'm sure that if arbitrary scientific convention had settled on some different number for significance, I'd have picked that one instead. I have made some progress since a year ago on calibrating my estimates of small probabilities, but I absolutely do not think that I would be wrong approximately 1 time in 20 when making predictions to which I assign a probability of 0.95.

Despite the fact that my opinion on the case has hardly changed at all, these posts -- and thinking about the case in general -- were a tremendous learning experience for me. Some of the lessons include:

• Less Wrong is good at getting the right answer. Believe it or not, the strong survey consensus in favor of innocence -- prior to my second post -- came as something of a pleasant surprise to me. You don't find this in many other places, despite the fact that the case is a no-brainer. I had assumed there would be more wishy-washiness and probabilities close to 50% than there turned out to be. (There was some of this, but less than I expected.)

• People in general are bad at getting the right answer. As shown by the original verdict, not to mention all the numerous pro-guilt commentators on internet forums and elsewhere. What's surprising about that? Not much, perhaps, but I would say that one thing that is important about it is that it shows that huge, glaring errors of judgement are not restricted to Far Mode. Even on a mundane question such as this, people are susceptible to strange cognitive biases that can severely distort their assessment of evidence.

• Confidence should depend on how much you know. I erred twice during the original sequence of posts on the case: once when I implied that participants in my survey ought to have come up with numbers like mine (0.001), and then again when I reacted to the indignation that followed by downgrading my own confidence. In truth, because (even before the verdict) I had followed the case more than most LW readers, my level of confidence should have been higher than most readers'. For someone who spends ten minutes reading the two sites I linked to, 0.9 probability of innocence (as opposed to 0.99) is perfectly reasonable; whereas someone like me should expect to be more confident, even if only on account of the meta-information that any additional information not on those sites isn't particularly relevant.

(More comments possibly to be added later...)

See my added comment. I did not assign a probability of 15%. I said that if you assigned a probability higher than 15%, it meant you had a really major problem with crediting the opinions of other people and the authority of idiots. My probability that Knox and Sollecito were guilty was "that's privileging the hypothesis", i.e., "I see no real evidence in its favor so same as prior probability", i.e., "really damned' low".

The Knox thread was one of the first steps in my getting interested in predictions in general. It was a slow process and is still ongoing, but it has had me spend time on various calibration exercises, on PredictionBook, on the Crowdcast instance dedicated to the Good Judgment project, on Inkling Markets because I saw a few arbitrage opportunities there that sounded like fun. I'm not as into predictions as gwern appears to be, but they're growing on me.

All that and I'm still not very sure what to think of the Knox case. Yes, if our predictions were being scored I'd be getting a non-trivial penalty from my 50% chance of her guilt - that is, if we take the outcome of the appeals process as an arbitration of the prediction, and judge, for the purposes of scoring, that she was "in fact" innocent. (I'm not saying I have much doubt now about her innocence: I'm saying that we won't ever know for sure, and part of the point of these prediction exercises is to allow us to better deal with that permanent uncertainty.)

On the other hand, some of the people listed above would be taking a much more serious hit. One thing I've learned from my various exercises is that you can't expect to be right all the time - sometimes, with minimal knowledge of the relevant facts, a 50% prediction is in fact not so bad.

Then again, some of us were also apparently very confident in the answer that "in fact" turned out to be the correct one. Then again, we all get lucky from time to time - that too is the nature of the beast...

My intention is to continue to learn, to continue to get better at predicting, to become better calibrated and more discriminating over time.

I want to see the regression of LW karma (or the log of LW karma) on probability-Amanda-is-guilty!

You ought to leave Eliezer out of the equation, or assign him a Karma value equal to Yvain's, or else he'll dominate the regression.

Strange thing about this is, if I've calculated it right, the average probability estimate of Guede's guilt is only ~87%. It seems to me that if this were your real probability estimate of his guilt, and you were on the jury at the guy's trial, you would be obligated to vote innocent. If you operate on the basis that a 13% chance of innocence is not a reasonable doubt, about thirteen out of every hundred people who go to jail will be innocent. That is (let me check) more than one in ten, which strikes me as rather a lot. I think my own estimate of Guede's guilt is above 99%, so I would vote guilty, but I'm surprised the average here is so low.

I updated my 60% guilt for Knox/Sollecito almost immediately after reading the follow-up article. As I noted on that page, my 60% judgement was a clear case of anchoring. I started with the pro-guilt evidence and only managed an 80% guilt after reading only their evidence. That I was then only able to re-adjust down to 60% was absurd. Since I was (and still am) fairly weak as a rationalist, I probably should have withheld any kind of assessment until after I had read the pro-innocence website, rather than try to make an early assessment and update it with the evidence I knew would be coming.

I didn't put a number on my update at the time, I just went from "There isn't enough evidence to convict" to "Why were they still suspects?". If I had to put a number on it, I'd say this was a <5% chance of guilt.

Since the acquittal, given the judge's statements about the DNA evidence and the handling thereof (I didn't read the full assessment by the independent expert), I'd have to adjust it now to <1%.

ETA: I just went through the independent analysis, and the conclusions were even stronger than I thought they were.

I thought that the conclusions were of the nature: "this DNA cannot be conclusively matched to anyone".

In fact, they were of the nature: "There is no organic material present."

They did find a few grains of starch on the bra clasp, but that was it. How did the Scientific Police screw it up that badly? They got a shakey DNA match where no DNA existed!

I have to update my assessment yet again, to the lowest I'm willing to put a number on, and that's <0.1% chance that either of them are guilty. There is literally zero evidence that either of them were involved.

Amazing job putting this together.

I had remembered LW as being a hive of Amanda fans, but either I’m succumbing to hindsight bias or people updated significantly after those articles.

My .13 probability in the first thread definitely went down following further discussion and, in particular, komponisto's second post. In the last year I've been comfortable using "She's definitely innocent" talking about the case with non-LWers.