We need a new paradigm for doing medicine. I make the case by first speaking about the problems of our current paradigm of evidence-based medicine.
The status quo of evidence-based medicine
While biology moves forward and the cost of genetic-sequencing dropped a lot faster than Moore's law the opposite is true for the development of new drugs. In the current status quo the development of new drugs rises exponentially with Eroom's law. While average lifespan increased greatly about the last century in Canada the average life span at age 90 increased only 1.9 years over the last century. In 2008 the Centers for Disease Control and Prevention reported that life expectancy in the US declined from 77.9 to 77.8 years. After Worldbank data Germany increased average lifespan by two years over the last decade which is not enough for the dream of radical lifespan increases in our lifetime.
When it costs 80 million to test whether an intervention works and most attempts show that the intervention doesn't work we have a problem. We end up paying billions for every new intervention.
Eric Ries wrote "The Lean Startup". In it he argues that it's the job of a startup to produce validated learning. He proposes that companies that work with small batch sizes can produce more innovation because they can learn faster how to build good products. The existing process in medicine doesn't allow for small batch innovation because the measuring stick for whether an intervention works is too expensive.
In addition the evidence-based approach rests on the assumption that we don't build bespoke interventions for every client. As long as a treatment doesn’t generalize about multiple different patients, it’s not possible to test it with a trial. In principle a double-blind trial can't give you evidence that a bespoke intervention that targets the specific DNA profile of a patient and his co-morbidity works.
The ideal of prediction-based medicine
The evidence-based approach also assumes that practitioners are exchangeable. It doesn't model the fact that different physical therapist or psychologists have different skill levels. It doesn't provide a mechanism to reward highly skilled practitioners but it treats every practitioner that uses the same treatment intervention the same way.
Its strong focus on asking whether a treatment beats a placebo in double-blind studies makes it hard to compare different treatments against each other. In the absence of an ability to predict the effect sizes of different drugs with the literature the treatment that wins on the market is often the treatment that's best promoted by a pharmaceutical company.
How could a different system work? What's the alternative to making treatment decisions based on big and expensive studies that provide evidence?
I propose that a treatment provider should provide a patient with the credence that the treatment provider estimates for treatment outcomes that are of interest to the client.
If Bob wants to stop smoking and asks doctor Alice whether the treatment Alice provides will result in Bob not smoking in a year, Alice should provide him with her credence estimation. In addition Alice’s credence estimations can be entered in a central database. This allows Bob to see Alice’s Brier score that reflects the ability of Alice to predict the effects of her treatment recommendations.
In this framework Alice’s expertise isn't backed up by having gotten an academic degree and recommending interventions that are studied with expensive gold-standard studies. Her expertise is backed by her track record.
This means that Alice can charge money based on the quality of her skills. If Alice is extremely good she can make a lot of money with her intervention without having to pay billions for running trials.
Why don't we pay doctors in the present system based on their skills? We can't measure their skills in the present paradigm, because we can't easily compare the outcomes of different doctors. Hard patients get send to doctors with good reputations and as a result every doctor has an excuse for getting bad outcomes. In the status quo he can just assert that his patients were hard.
In prediction-based medicine a doctor can write down a higher credence for a positive treatment outcome for an easy patient than a hard patient. Patients can ask multiple doctors and are given good data to choose the treatment that provides the best outcome for which they are willing to pay.
In addition to giving the patient a more informed choice about the advantages of different treatment options this process helps the treatment provider to increase his skills. They learn about where they make errors in the estimation of treatment outcomes.
The provider can also innovate new treatments in small batches. Whenever he understands a treatment well enough to make predictions about its outcomes he's in business. He can easily iterate on his treatment and improve it.
The way to bring prediction-based medicine into reality
I don't propose to get rid of evidence-based medicine. It has its place and I don't have any problem with it for the cases where it works well.
It works quite poorly for body work interventions and psychological interventions that are highly skill based. I have seen hypnosis achieve great effects but at the same time there are also many hypnotists who don't achieve great effects. In the status quo a patient who seeks hypnosis treatment has no effective way to judge the quality of the treatment before he's buying.
A minimal viable product might be a website that's Uber for body workers and hypnotists. The website lists the treatment providers. The patient can enter his issue and every treatment provider can offer his credence of solving the issue of the patient and the price of his treatment.
Before getting shown the treatment providers, a prospective patient would take a standardized test to diagnose the illness. The information from the standardized test will allow the treatment providers make better predictions about the likelihood that they can cure the patient. Other standardized tests that aren’t disease specific like the OCEAN personality index can also be provided to the patient.
Following the ideas of David Burn’s TEAM framework, the treatment provider can also tell the patient to take tests between treatments sessions to keep better track of the progression of the patient.
When making the purchasing decision the patient agrees to a contract that includes him paying a fine, if he doesn’t report the treatment outcome after 3 months, 6 months and 1 year. This produces a comprehensive database of claims that allows us to measure how well the treatment providers are calibrated.
Various Quantified Self gadgets can be used to gather data. Many countries have centralized electronic health records that could be linked to a user account.
The startup has a clear business model. It can take a cut of every transaction. It has strong network effects and it's harder for a treatment provider to switch because all his prediction track record is hosted on the website.
Thanks to various people from the Berlin Lesswrong crowd who gave valuable feedback for the draft of this article.
That doesn't surprise me. Training and degrees are easily observable; outcomes (or, rather, how much of an outcome is attributable to each teacher) are not. It's harder to pay people based on something less observable.
Thanks!
But I'm not sure it matters. What Bill Gates says in that quotation might well be consistent with what I wrote — it's hard to be sure because he's quite vague.
For instance, he says that a teacher in the top quartile increases their class' performance "by over 10 percent in a single year". I can believe that, but maybe all it means is that a top-quartile teacher increases their class' scores by 11% a year, while a bottom-quartile teacher increases their class' scores by 9% a year. That would hardly refute the idea that teacher effects are small on average!
Maybe. I did a bit more searching on Google Scholar but didn't uncover a more recent review article with relevant statistics. (I did find one study of a thousand students in HSGI schools in "school year 2013-2014", which found that 12% of the variance in GPA was at the teacher level.)
However, while the study I cited uses only pre-2004 data, I don't see much reason to think a newer study would reveal a big increase in teacher-level variance in outcomes. The low proportion of variance attributable to teachers has been true, as far as I know, for as long as people have investigated it (at least 46 years). I'm doubtful that an act which demands high-stakes testing, improvements in average state-wide test scores, and state-wide standards for teachers has changed that, or that a charity has changed that.
It's possible, but I don't see evidence that the paper I cited has this flaw. Its new analysis was based on about 100 Tennessee schools included in Project STAR, and the earlier analyses it summarized (table 1) used "samples of poor or minority students", "nationally representative samples of students", and "a large sample of public school students in Texas".
Fair point — my thought experiment only addresses diagnosis and treatment recommendation, not treatment administration. I think there would be more doctor-level variation among the latter...although not much more in absolute terms.
Probably true in most places.
Agreed that you'd get more variance, but I suspect it wouldn't be much more (subject to this hypothetical's exact details).
This is certainly true. I've seen too many stories of people with rare genetic conditions and other diseases successfully working out aetiologies to think otherwise. But those are unusual cases where people tended to invest lots of effort into figuring their cases out. I don't see them as signs that we can improve the quality-to-cost ratio of healthcare in general by looking very carefully at the details of each case.
Sure. But the ratios in my example are more important than the exact numbers.
A good question, and a good answer to the question!
It'd be immediately possible in the US: eliminate de facto immigration barriers for foreign doctors. Those barriers are, I'd guess, lower in other developed countries (hence why doctors in the UK, Germany, Canada, etc. earn less than US doctors) but I expect doctors' salaries there could also be reduced a bit by further relaxing immigration restrictions for foreign doctors.
Another option is for patients to go to the cheaper doctors: medical tourism.
I understand him to be speaking about them increasing 10% more than non-top quartile teachers.
Eg. enough to circumvent the US-Asia difference in two years and also enough to circumvent the Black-White difference in four years as suggested in the answer to the Stackexchange question.
It's worth noting that Medical Error is the third leading cause of death in the US http://www.bmj.com/content/353/bmj.i2139
The article doesn't only describe immigration barriers but also barriers of credentialism. The non-US degree often isn't enough to work in the US as a doctor because it's quality is in doubt. If there would be hard evidence that those doctors perform as well as US doctors the ability to seek rents via credentialism will be reduced.
I looked at that study. It seems their best predictors were e (i) Fall algebra EOC (End Of Course) scores, (ii) English language learner (ELL) status, (iii) Black student status, and (iv) Hispanic student status.
Of course you do a better prediction of the student performance for a standardized test when you look at the last standardized test they took than when you look at whether or not they had a very good teacher for a single school year. The 12% variance under that setting might be compatible with the claims that Gates makes. 12% variance per year might compound over multiple years to bridge the gap between the US and Asia in two years and US Black White gap in 4 years.