(The following happened to me in an IRC chatroom, long enough ago that I was still hanging around in IRC chatrooms. Time has fuzzed the memory and my report may be imprecise.)
So there I was, in an IRC chatroom, when someone reports that a friend of his needs medical advice. His friend says that he's been having sudden chest pains, so he called an ambulance, and the ambulance showed up, but the paramedics told him it was nothing, and left, and now the chest pains are getting worse. What should his friend do?
I was confused by this story. I remembered reading about homeless people in New York who would call ambulances just to be taken someplace warm, and how the paramedics always had to take them to the emergency room, even on the 27th iteration. Because if they didn't, the ambulance company could be sued for lots and lots of money. Likewise, emergency rooms are legally obligated to treat anyone, regardless of ability to pay. (And the hospital absorbs the costs, which are enormous, so hospitals are closing their emergency rooms... It makes you wonder what's the point of having economists if we're just going to ignore them.) So I didn't quite understand how the described events could have happened. Anyone reporting sudden chest pains should have been hauled off by an ambulance instantly.
And this is where I fell down as a rationalist. I remembered several occasions where my doctor would completely fail to panic at the report of symptoms that seemed, to me, very alarming. And the Medical Establishment was always right. Every single time. I had chest pains myself, at one point, and the doctor patiently explained to me that I was describing chest muscle pain, not a heart attack. So I said into the IRC channel, "Well, if the paramedics told your friend it was nothing, it must really be nothing—they'd have hauled him off if there was the tiniest chance of serious trouble."
Thus I managed to explain the story within my existing model, though the fit still felt a little forced...
Later on, the fellow comes back into the IRC chatroom and says his friend made the whole thing up. Evidently this was not one of his more reliable friends.
I should have realized, perhaps, that an unknown acquaintance of an acquaintance in an IRC channel might be less reliable than a published journal article. Alas, belief is easier than disbelief; we believe instinctively, but disbelief requires a conscious effort.
So instead, by dint of mighty straining, I forced my model of reality to explain an anomaly that never actually happened. And I knew how embarrassing this was. I knew that the usefulness of a model is not what it can explain, but what it can't. A hypothesis that forbids nothing, permits everything, and thereby fails to constrain anticipation.
Your strength as a rationalist is your ability to be more confused by fiction than by reality. If you are equally good at explaining any outcome, you have zero knowledge.
We are all weak, from time to time; the sad part is that I could have been stronger. I had all the information I needed to arrive at the correct answer, I even noticed the problem, and then I ignored it. My feeling of confusion was a Clue, and I threw my Clue away.
I should have paid more attention to that sensation of still feels a little forced. It's one of the most important feelings a truthseeker can have, a part of your strength as a rationalist. It is a design flaw in human cognition that this sensation manifests as a quiet strain in the back of your mind, instead of a wailing alarm siren and a glowing neon sign reading "EITHER YOUR MODEL IS FALSE OR THIS STORY IS WRONG."
Part of the sequence Mysterious Answers to Mysterious Questions
Next post: "Absence of Evidence Is Evidence of Absence"
Previous post: "The Virtue of Narrowness"
Considering that medical errors apparently kill more people than car accidents each year in the United States, I suspect the establishment is not in fact infallible.
Citation needed? I know I'm coming to this rather late, but a quick check of the 2010 CDC report on deaths in the US gives "Complications of medical and surgical care" as causing 2,490 deaths whereas transport accidents causing 37,961 deaths (35,332 of which were classified a 'motor vehicle deaths'). The only other thing I can see that might be medical errors put under a different heading is "Accidental poisoning and exposure to noxious substances" at 33,041 which combines to still fewer deaths than transport accidents even without removing those poisonings which are not medical errors. (This poisoning category appears to have a lot of recreational drug overdoses judging by the way it sharply increases in the 15-24 age group then drops off after 54 whereas time-spent-in-hospital is presumably increasing with age.)
On the other hand, a 2012 New York Times Op-Ed claims 98,000 deaths from medical errors a year. This number is so much larger than what the CDC reports that I must be misreading something. That would be about 1 in 20 people who die in the US die due to medical error. Original source from 1999). Actually checking that source, 98,000 deaths/year is the upper bound number given (lower bound of 44,000 deaths/year). The report also recommends a 50% reduction in these deaths within 5 years (so by 2004) - and Wikipedia mentions a 2006 study claiming that they successfully preventing 120,000 deaths in an 18 month time period but I can't find this study. A 2001 followup here appears to focus on suggestions for improvements rather than on giving new data to our question. 3 minutes on Google Scholar didn't turn up any recent estimates. This entire sub-field appears to rely very heavily upon that one source - at least in the US.
Also of interest is "Actual Causes of Death in the US" which classifies deaths by 'mistake made' (so to speak) - the top killer being tobacco use, then poor diet/low exercise, alcohol, microbial agents, toxic agents, car accidents, firearms, sexual behaviors, and illicit drug use. Medical errors didn't show high up on this list, despite it being the only source in the Wikipedia article on the original article.
Edit: also some places that cite the 1999 study accuse the CDC of not reporting these deaths as their own category. This appears to have changed given the category I reported above. The fact that there has been substantial uproar about medical error since the 1999 article and a corresponding increase in funding for studying it makes me unsurprised that the CDC would start reporting.