The Unfriendly Superintelligence next door

Markets are powerful decentralized optimization engines - it is known.  Liberals see the free market as a kind of optimizer run amuck, a dangerous superintelligence with simple non-human values that must be checked and constrained by the government - the friendly SI.  Conservatives just reverse the narrative roles.

In some domains, where the incentive structure aligns with human values, the market works well.  In our current framework, the market works best for producing gadgets. It does not work so well for pricing intangible information, and most specifically it is broken when it comes to health.

We treat health as just another gadget problem: something to be solved by pills.  Health is really a problem of knowledge; it is a computational prediction problem.  Drugs are useful only to the extent that you can package the results of new knowledge into a pill and patent it.  If you can't patent it, you can't profit from it.

So the market is constrained to solve human health by coming up with new patentable designs for mass-producible physical objects which go into human bodies.  Why did we add that constraint - thou should solve health, but thou shalt only use pills?  (Ok technically the solutions don't have to be ingestible, but that's a detail.)

The gadget model works for gadgets because we know how gadgets work - we built them, after all.  The central problem with health is that we do not completely understand how the human body works - we did not build it.  Thus we should be using the market to figure out how the body works - completely - and arguably we should be allocating trillions of dollars towards that problem.

The market optimizer analogy runs deeper when we consider the complexity of instilling values into a market.  Lawmakers cannot program the market with goals directly, so instead they attempt to engineer desireable behavior by ever more layers and layers of constraints.  Lawmakers are deontologists.

As an example, consider the regulations on drug advertising.  Big pharma is unsafe - its profit function does not encode anything like "maximize human health and happiness" (which of course itself is an oversimplification).  If allowed to its own devices, there are strong incentives to sell subtly addictive drugs, to create elaborate hyped false advertising campaigns, etc.  Thus all the deontological injunctions.  I take that as a strong indicator of a poor solution - a value alignment failure.

What would healthcare look like in a world where we solved the alignment problem?

To solve the alignment problem, the market's profit function must encode long term human health and happiness.  This really is a mechanism design problem - its not something lawmakers are even remotely trained or qualified for.  A full solution is naturally beyond the scope of a little blog post, but I will sketch out the general idea.

To encode health into a market utility function, first we create financial contracts with an expected value which captures long-term health.  We can accomplish this with a long-term contract that generates positive cash flow when a human is healthy, and negative when unhealthy - basically an insurance contract.  There is naturally much complexity in getting those contracts right, so that they measure what we really want.  But assuming that is accomplished, the next step is pretty simple - we allow those contracts to trade freely on an open market.

There are some interesting failure modes and considerations that are mostly beyond scope but worth briefly mentioning.  This system probably needs to be asymmetric.  The transfers on poor health outcomes should partially go to cover medical payments, but it may be best to have a portion of the wealth simply go to nobody/everybody - just destroyed.

In this new framework, designing and patenting new drugs can still be profitable, but it is now put on even footing with preventive medicine.  More importantly, the market can now actually allocate the correct resources towards long term research.

To make all this concrete, let's use an example of a trillion dollar health question - one that our current system is especially ill-posed to solve:

What are the long-term health effects of abnormally low levels of solar radiation?  What levels of sun exposure are ideal for human health?

This is a big important question, and you've probably read some of the hoopla and debate about vitamin D.  I'm going to soon briefly summarize a general abstract theory, one that I would bet heavily on if we lived in a more rational world where such bets were possible.

In a sane world where health is solved by a proper computational market, I could make enormous - ridiculous really - amounts of money if I happened to be an early researcher who discovered the full health effects of sunlight.  I would bet on my theory simply by buying up contracts for individuals/demographics who had the most health to gain by correcting their sunlight deficiency.  I would then publicize the theory and evidence, and perhaps even raise a heap pile of money to create a strong marketing engine to help ensure that my investments - my patients - were taking the necessary actions to correct their sunlight deficiency.  Naturally I would use complex machine learning models to guide the trading strategy.

Now, just as an example, here is the brief 'pitch' for sunlight.

If we go back and look across all of time, there is a mountain of evidence which more or less screams - proper sunlight is important to health.  Heliotherapy has a long history.

Humans, like most mammals, and most other earth organisms in general, evolved under the sun.  A priori we should expect that organisms will have some 'genetic programs' which take approximate measures of incident sunlight as an input.  The serotonin -> melatonin mediated blue-light pathway is an example of one such light detecting circuit which is useful for regulating the 24 hour circadian rhythm.

The vitamin D pathway has existed since the time of algae such as the Coccolithophore.  It is a multi-stage pathway that can measure solar radiation over a range of temporal frequencies.  It starts with synthesis of fat soluble cholecalciferiol which has a very long half life measured in months. [1] [2]

The rough pathway is:

  • Cholecalciferiol (HL ~ months) becomes 
  • 25(OH)D (HL ~ 15 days) which finally becomes 
  • 1,25(OH)2 D (HL ~ 15 hours)

The main recognized role for this pathway in regards to human health - at least according to the current Wikipedia entry - is to enhance "the internal absorption of calcium, iron, magnesium, phosphate, and zinc".  Ponder that for a moment.

Interestingly, this pathway still works as a general solar clock and radiation detector for carnivores - as they can simply eat the precomputed measurement in their diet.

So, what is a long term sunlight detector useful for?  One potential application could be deciding appropriate resource allocation towards DNA repair.  Every time an organism is in the sun it is accumulating potentially catastrophic DNA damage that must be repaired when the cell next divides.  We should expect that genetic programs would allocate resources to DNA repair and various related activities dependent upon estimates of solar radiation.

I should point out - just in case it isn't obvious - that this general idea does not imply that cranking up the sunlight hormone to insane levels will lead to much better DNA/cellular repair.  There are always tradeoffs, etc.

One other obvious use of a long term sunlight detector is to regulate general strategic metabolic decisions that depend on the seasonal clock - especially for organisms living far from the equator.  During the summer when food is plentiful, the body can expect easy calories.  As winter approaches calories become scarce and frugal strategies are expected.

So first off we'd expect to see a huge range of complex effects showing up as correlations between low vit D levels and various illnesses, and specifically illnesses connected to DNA damage (such as cancer) and or BMI.  

Now it turns out that BMI itself is also strongly correlated with a huge range of health issues.  So the first key question to focus on is the relationship between vit D and BMI.  And - perhaps not surprisingly - there is pretty good evidence for such a correlation [3][4] , and this has been known for a while.

Now we get into the real debate.  Numerous vit D supplement intervention studies have now been run, and the results are controversial.  In general the vit D experts (such as my father, who started the vit D council, and publishes some related research[5]) say that the only studies that matter are those that supplement at high doses sufficient to elevate vit D levels into a 'proper' range which substitutes for sunlight, which in general requires 5000 IU day on average - depending completely on genetics and lifestyle (to the point that any one-size-fits all recommendation is probably terrible).

The mainstream basically ignores all that and funds studies at tiny RDA doses - say 400 IU or less - and then they do meta-analysis over those studies and conclude that their big meta-analysis, unsurprisingly, doesn't show a statistically significant effect.  However, these studies still show small effects.  Often the meta-analysis is corrected for BMI, which of course also tends to remove any vit D effect, to the extent that low vit D/sunlight is a cause of both weight gain and a bunch of other stuff.

So let's look at two studies for vit D and weight loss.

First, this recent 2015 study of 400 overweight Italians (sorry the actual paper doesn't appear to be available yet) tested vit D supplementation for weight loss.  The 3 groups were (0 IU/day, ~1,000 IU / day, ~3,000 IU/day).  The observed average weight loss was (1 kg, 3.8 kg, 5.4 kg). I don't know if the 0 IU group received a placebo.  Regardless, it looks promising.

On the other hand, this 2013 meta-analysis of 9 studies with 1651 adults total (mainly women) supposedly found no significant weight loss effect for vit D.  However, the studies used between 200 IU/day to 1,100 IU/day, with most between 200 to 400 IU.  Five studies used calcium, five also showed weight loss (not necessarily the same - unclear).  This does not show - at all - what the study claims in its abstract.

In general, medical researchers should not be doing statistics.  That is a job for the tech industry.

Now the vit D and sunlight issue is complex, and it will take much research to really work out all of what is going on.  The current medical system does not appear to be handling this well - why?  Because there is insufficient financial motivation.

Is Big Pharma interested in the sunlight/vit D question?  Well yes - but only to the extent that they can create a patentable analogue!  The various vit D analogue drugs developed or in development is evidence that Big Pharma is at least paying attention.  But assuming that the sunlight hypothesis is mainly correct, there is very little profit in actually fixing the real problem.

There is probably more to sunlight that just vit D and serotonin/melatonin.  Consider the interesting correlation between birth month and a number of disease conditions[6].  Perhaps there is a little grain of truth to astrology after all.

Thus concludes my little vit D pitch.  

In a more sane world I would have already bet on the general theory.  In a really sane world it would have been solved well before I would expect to make any profitable trade.  In that rational world you could actually trust health advertising, because you'd know that health advertisers are strongly financially motivated to convince you of things actually truly important for your health.

Instead of charging by the hour or per treatment, like a mechanic, doctors and healthcare companies should literally invest in their patients long-term health, and profit from improvements to long term outcomes.  The sunlight health connection is a trillion dollar question in terms of medical value, but not in terms of exploitable profits in today's reality.  In a properly constructed market, there would be enormous resources allocated to answer these questions, flowing into legions of profit motivated startups that could generate billions trading on computational health financial markets, all without selling any gadgets.

So in conclusion: the market could solve health, but only if we allowed it to and only if we setup appropriate financial mechanisms to encode the correct value function.  This is the UFAI problem next door.


Comments

sorted by
magical algorithm
Highlighting new comments since Today at 6:56 AM
Select new highlight date
Rendering 50/68 comments  show more

Interesting read. In the same vein. What I was imagining is a computational market, relying on the ability to do lots of complex trades at high speeds, and AI/ML. But much of that difference is explained by the 20 years.

I also reviewed some of his prototype code for a combinatorial prediction market around 10 years ago. I agree that these are promising ideas and I liked this post a lot.

As someone living in a universal/governmental healthcare country, I think we are doing this. If I am healthy and working, I am an asset for the state, pay taxes / social security. If I am ill or disabled, they gotta pay me. If I am dead, I don't pay them.

Of course it is not ideal first of all because of the usual problem of government: politicians, bureaucrats don't get a dividend from the profits of the state, they are not incentivized to maximize profitability. Secondarily, there are some incentive pitfalls like I am cheaper for them dead than pulling disability pay. Once it would look likely I will never work again their incentive would be providing zero healthcare.

So while it is not ideal, the basic idea of people paying something to an organization every month or year when they are healthy and working, and the healthcare costs are paid by that organization so they want to keep their client healthy and working and paying is there, and it can be tweaked. Part of the story is that people even when retired should keep paying. From this angle life insurance is better than social security.

However I think there is no workaround for the fact that once you have 5 years to live and extending that to 10 years costs a lot, whatever tax or insurance premiums you would pay in the second 5 years would not cover it and thus the organization has no incentive to extend your life. This can only be done by strict contracts or by politics. A third option is kids.

Let's go a big sci-fi here, we make a pill that extends female fertility up to about 60 easy, and thus we can assume most people will have kids again because it can just as treated like an early retirement.

The point is, if people have kids, you can treat families as immortal or long-lived persons. You can work out a scheme that if dad's life is not extended the kids will take their insurance elsewhere.

There's research suggesting that in developing countries, increased healthcare spending doesn't improve health outcomes like longevity. Don't know how good the research is though.

"In some domains, where the incentive structure aligns with human values, the market works well. In our current framework, the market works best for producing gadgets. It does not work so well for pricing intangible information, and most specifically it is broken when it comes to health."

Health care, as an example of a problem "market"?

In the US, health care is largely just a shake down by various licensed monopolies engaged in government enabled and enforced rent seeking.

What would healthcare look like in a world where we solved the alignment problem?

We'd be free.

In a US that had been free for the last 20 years, google would be analyzing the data from my weekly, if not daily microfluidic samples, from real time continuous information provided by sensors throughout the day, from my dna, and doing the same for millions of other people. We'd share our data so that google could make better recommendations for achieving our health goals, and with the mass of data everyone shares, google would be able to make those predictions.

The only market "alignment" required is allowing a market in health care to actually exist.

In the US, health care is largely just a shake down by various licensed monopolies engaged in government enabled and enforced rent seeking.

Yes, and? A market exists where a market exists. Any moral or ideological properties that might make a market "pure" or "impure" are just attributed to it by ideologues.

In a US that had been free for the last 20 years, google would be analyzing the data from my weekly, if not daily microfluidic samples, from real time continuous information provided by sensors throughout the day, from my dna, and doing the same for millions of other people. We'd share our data so that google could make better recommendations for achieving our health goals, and with the mass of data everyone shares, google would be able to make those predictions.

You seem to be confusing Google with a Friendly singleton. Of course, once you get into Singletons, it's easier to have a democratic government function as one, anyway.

A market exists where a market exists. Any moral or ideological properties that might make a market "pure" or "impure" are just attributed to it by ideologues.

The properties are what make it a market, a chimpanzee, or a turnip.

“The beginning of wisdom is to call things by their proper name.” ― Confucius

You seem to be confusing Google with a Friendly singleton. Of course, once you get into Singletons, it's easier to have a democratic government function as one, anyway.

Google and government have different powers and incentives.

If the US were "free" you'd have a gazillion snake oil salesmen,

And I'd be free to buy from them, or not.

And if they committed fraud in the sale, I would have legal recourse.

Historically, this didn't work out well. You know, back when the snake oil salesmen were literal and selling real snake oil, cocaine, and various low-dose toxic extracts. (I believe similar things happen in China today, but it's more slanted toward traditional medicine and thus less likely to be toxic.)

I wonder what Yvain/Scott would have to say about the vitamin D pitch.

I had read that article and wanted to explicitly mention it but couldn't re-find it in time, he has so many blogs post that mention vit D.

Yvain's view seems similar to the viewpoint I critiqued in the article. He doesn't care what dose the study is using, ethnicity/genetics, or if it is likely to raise OHD levels sufficiently at all.

The meta-analysis studies he links to are of the kind I critiqued - it's basically gerrymandering. These issues should be decided by largescale public ML competitions, not dark age statistics.

Did you read the New England paper? They specifically measured serum levels, did a very thorough sub group analysis (including ethnicity and many other possible factors), broke down the supplementation by dosage. And all within a 36,000 person double blinded randomized control trial.

I don't think I need to point out the danger in creating a just-so story in favor of your belief then approvingly citing sources that agree with your conclusion, while slamming sources that don't give you the results you want. Granted, the problem of certain medications only working on some groups is somewhat difficult and this is a very common complaint when studies show a null effect. Perhaps a medication really does only work on elderly Hispanic women - and its not just an instance of p-value hacking. However, when a massive amount of data shows that there is no discernible effect of supplementation across many groups, you have to decide when to set aside a hypothesis until there is a good reason to consider it again.

(of course, this is just the colorectal cancer case, but I think it illustrates that the medical research community is well aware of the issues you bring up. Epidemiology is hard and no one has ever claimed otherwise).

Did you read the New England paper? They specifically measured serum levels, did a very thorough sub group analysis (including ethnicity and many other possible factors), broke down the supplementation by dosage.

Could you cite the paper you mean?

Did you read the New England paper?

Not sure. Is it the colon cancer one? I read the breast cancer one linked on his post. I dont believe/expect that vit D is going to make an impact for every illness or type of cancer. I expect the relationship will be complex, and will depend heavily on dietary factors and genetics (for example there is strong evidence for various climdate adaptations - far northern peoples need much less vit D, etc.) I picked the breast cancer paper because I have a vague memory of hearing a very convincing talk from a breast cancer doctor about the evidence for the bio mechanisms between the low vit D and breast cancer.

From the summary of the breast cancer paper yvain linked to:

Methods Postmenopausal women (N = 36 282) who were enrolled in a Women's Health Initiative clinical trial were randomly assigned to 1000 mg of elemental calcium with 400 IU of vitamin D3 daily or placebo for a mean of 7.0 years

Results Invasive breast cancer incidence was similar in the two groups (528 supplement vs 546 placebo; hazard ratio = 0.96; 95% confidence interval = 0.85 to 1.09). In the nested case–control study, no effect of supplement group assignment on breast cancer risk was seen. Baseline 25-hydroxyvitamin D levels were modestly correlated with total vitamin D intake (diet and supplements) (r = 0.19, P < .001) and were higher among women with lower BMI and higher recreational physical activity (both P < .001). Baseline 25-hydroxyvitamin D levels were not associated with breast cancer risk in analyses that were adjusted for BMI and physical activity (Ptrend = .20).

So they used only 400 IU, and even then the breast cancer incidence was 528 vs 546 for 400 IU vs 0 IU. There was no effect only when they "adjusted for BMI" - of course - as some other studies have suggested, vit D has a potential weight loss effect. The 400 IU didn't make a huge diff on OHD levels.

This study is supportive of vit D potentially having a significant effect on breast cancer, and it is reasonable evidence against the null hypothesis (no effect). Instead here is their conclusion:

Conclusions Calcium and vitamin D supplementation did not reduce invasive breast cancer incidence in postmenopausal women. In addition, 25-hydroxyvitamin D levels were not associated with subsequent breast cancer risk. These findings do not support a relationship between total vitamin D intake and 25-hydroxyvitamin D levels with breast cancer risk.

That is not justifiable, nor is it even an accurate summary of their own data.

Yes I meant the colon cancer paper (I should have stated at the beginning rather than the end), but the breast cancer paper was from the same experiment, and as such also addresses you initial concerns about effects on different subgroups and measuring serum levels.

So they used only 400 IU, and even then the breast cancer incidence was 528 vs 546 for 400 IU vs 0 IU.

The difference in incidence was not statistically significant. Random chance could easily result in this level of variation. Regarding the dosage, yes you can only make conclusions about what you measure. This trial was started in the 90's, and since then there have been some recommendations for higher doses. However, the authors are not unaware of this and point out that many women in the trial were supplementing with additional Vitamin D, and after analyzing that, there was still no detectable effect:

We cannot assess whether a higher dosage would have changed the outcome of the current study. However, our findings provide some evidence against that hypothesis. Because approximately half of the women were taking an additional 400 IU of nonprotocol vitamin D supplement daily, actual vitamin D supplement intake was greater than 800 IU daily for a substantial number of participants in the supplement group. Nonetheless, no effects on risk of breast cancer overall or in sensitivity analyses that were adjusted for study adherence were observed.

Perhaps a still higher dose is need to see any effect. But this would require positing a very non-linear model. Is it possible? Sure, but this complaint can always be made.

Regarding BMI, the authors are also aware of this argument - see the quote below. I'd like to point out that the BMI numbers were baseline which is not evidence that vitamin D supplementation causes weight loss. It is consistent with the hypothesis that healthier lifestyles result in both higher vitamin D levels (due to sun exposure with exercise) and lower BMI.

Levels of 25-hydroxyvitamin D at baseline were statistically significantly higher among lean women and/or those with more recreational activity than overweight or obese or less active women. Based on the associations of these breast cancer risk factors with 25-hydroxyvitamin D, 25-hydroxyvitamin D could be a potential mediator of lifestyle influence on breast cancer. Alternatively, lifestyle choices could have led to more sunlight exposure, with higher 25-hydroxyvitamin D levels and lower breast cancer risk as independent processes. The finding that analyses adjusted for BMI and physical activity did not identify an association between baseline 25-hydroxyvitamin D levels and breast cancer risk suggests that the association between 25-hydroxyvitamin D and breast cancer seen in some observational studies could be confounded to some degree by such factors.

You can argue that the question isn't yet settled. I think that is defensible - maybe we really do need much higher doses. But you cannot argue that the researchers haven't done their homework or that they aren't aware of what their data is capable or incapable of showing.

So they used only 400 IU, and even then the breast cancer incidence was 528 vs 546 for 400 IU vs 0 IU.

The difference in incidence was not statistically significant.

This is primitive frequentist statistics.

If we could repeat this same experiment with a very similar distribution of patients and other factors (or if the trial was still ongoing), and it came down to a bet on which group would have higher cancer deaths - would you accept an even money bet that the placebo group would have less cancer deaths? ie - would you bet against me?

Whether 3% more deaths in the placebo group is significant or not for a sample size of 1000, and an average supplement level which is only 10% of what the vit D experts believe is required to replace sunlight depends completely on your priors and your statistical model. There is nothing innate or inherently correct about the standard puny little models used to determine 'statistical significance'.

Thinking in terms of prediction markets helps clarify these issues - the best model is the one that ends up sitting on the big heap pile of money.

Random chance could easily result in this level of variation.

Sure - or maybe not. It is still evidence that favors a vit D effect for breast cancer.

However, the authors are not unaware of this and point out that many women in the trial were supplementing with additional Vitamin D, and after analyzing that, there was still no detectable effect:

Half of the women reported taking an additional 400 IU, which - even with full compliance - only raises the average up to 600 IU.

The main key point is that the range of measured OHD levels was between 30 to 67 ish and didn't have a strong correlation to the supplement - they failed to raise the OHD levels. That is key. All of the studies which show a health effect also significantly shift the OHD levels. If your supplement isn't shifting OHD levels much, why would think it would do much of anything?

The general theory predicts health effects in 1.) certain genetic subpopulations, who 2.) have unnaturally low levels (say <30 for caucasians), and 3. then raise those levels up to evolutionary ideal for their genetics (perhaps 50-60 for caucasians). The exact numbers are just hyperparameters - the point of the studies should be to refine them into a predictive model.

But you cannot argue that the researchers haven't done their homework or that they aren't aware of what their data is capable or incapable of showing.

The completely honest conclusion for their abstract should have been something like this:

Conclusions Calcium and low doses of vitamin D supplementation did not significantly reduce (> 3%) invasive breast cancer incidence in postmenopausal women. In addition, 25-hydroxyvitamin D levels were not associated with subsequent breast cancer risk after we adjusted for body mass index. It is difficult to make definitive conclusions from these findings.

Compare that to the conclusion they actually wrote. Then the telephone game begins, and people begin quoting this study as one which somehow 'proves' vit D supplementation has no effect on breast cancer . ...

If you want to bet there the VITAL study going on: http://www.vitalstudy.org/VitalSigns.html 25,000 subjects 2000 UI Vitamin D3 and upcoming mortality data.

I also created a prediction book entry a while ago: http://predictionbook.com/predictions/14426

I'm sorry, I'm really not interested in getting into an ideological debate or whether frequentist or bayesian statistics is "better" - if you think that frequentist methods are worthless, then the inferential gap is too wide to begin to bridge.

encode health into a market utility function, first we create financial contracts with an expected value which captures long-term health. We can accomplish this with a long-term contract that generates positive cash flow when a human is healthy, and negative when unhealthy - basically an insurance contract.

That's broadly the structure of taxpayer supported public healthcare system. The government is incentivised to keep people alive and paying tax, and disincentivised to treat people unnecessarily. I can vouch that under that kind of system, the authorities aren't shy about promoting preventative measures.

I generally agree that the government is more value aligned, but it is also much less efficient.

In particular it has trouble with fine grained credit assignment. It works well for running a military, it doesn't work as well in the modern era for research - where anybody with some smarts, the internet, and luck can come up with an idea that is potentially worth billions - without any credentials.

yep, alive and paying tax or dead quickly if they're going to die.

There's a reason there was a big push for anti-smoking measures in recent years. Cancer treatments got better, people with lung cancer suddenly lived longer and cost more so the incentive to stop people smoking grew.

Better aligned but not perfectly.

Also it is largely a command economy setup.

Better aligned but not perfectly.

Better, of course, is still better than worse. "Perfectly aligned", I'd say, is an overly strong claim to make for anything in this mortal world, ahaha, that was not explicitly designed and verified to be perfectly aligned, or at least aligned in-the-limit of increasing resources.

Also it is largely a command economy setup.

There are countries with non-command economy public healthcare. I lived in one (Israel).

It's also important to note that markets, in the sense of allocatively-efficient combinatorial price-seeking and price-taking as such, are not the magic super sauce of Western economies as we know them. As far as economists were able to study, the magic super sauce is actually just allowing firms to enter and exit business as they please, enabling economic agents to plan reliably, and containing liability when things go wrong.

Command economies violate that first guideline, which takes away the ability of an economy to explore a combinatorially-large solution space in parallel.

The last problem was once "solved" via limited liability corporations, but nowadays bailout/hostage capitalism and the increasing financialization of Western economies has created a previously-unrecognized creature: distinctly capitalist economies defined by catering to a small number of agents who can punish everyone else with toxic liabilities when they don't get their upside.

The middle problem is where different societies have usually had to fudge things. A strong sense of private property often helps, but an overly strong institution of private property allows hostage-taking (as described above with banks, or as is the case with natural-resource rents, or as witnessed in any Bay Area urban planning meeting). Things like collective bargaining, universal health care, or Flexicurity seem to run against the grain of private property, but actually help increase the total planning reliability across society, and certainly people like having them. There doesn't seem to be a secret sauce for building institutional structures that give people the Freedom To Optimize.

There's a great chart of the popularity of tobacco, and of various modes of consuming it, on the first page of this paper:

http://cancercontrol.cancer.gov/brp/tcrb/monographs/8/m8_2.pdf

There's also a timeline of related events on page 16.

Peak tobacco use corresponded with the period where scientific studies were establishing that smoking causes cancer and various other diseases. It was several years into the decline of smoking popularity before "nonsmoker's rights" (i.e. measures against smoking in public) started taking off.

While I'm glad to see someone writing up critiques of markets and capitalism in language LW will understand, really really incredibly glad in fact, I do have one nit to pick:

Liberals see the free market as a kind of optimizer run amuck, a dangerous superintelligence with simple non-human values that must be checked and constrained by the government - the friendly SI. Conservatives just reverse the narrative roles.

Not only is politics spiders, this is a simplistic strawman of both positions at once. The article would be better for cutting out the meta-level mention of politics and sticking to its object-level claims of fact.

But overall, you have all my upvotes and I wish to subscribe to your newsletter.

I like the idea of how to use markets to 'solve' health and esp. the illustrative example. Reminds me of David Friedman's approach so solve law (see Anarchy and Efficient Law).

One nitpick regarding the post: It is really two: The health market and the vitamin D story. These are even clearly consecutive. I also do not see a clear connection between them except in your narrative. Why not post separatedly?

ADDED: And the title is misleading. It only relates to the opener, not the meat of the post.

Thanks for this post -- very interesting.

One question:

In general, medical researchers should not be doing statistics. That is a job for the tech industry.

Are you claiming that the tech industry is better at stats, or could be better at stats, if it were somehow decentralized? If the former, what's your evidence?

Standard statistics is just primitive machine learning. Any question that you have about data is best answered by a well structured machine learning system/market, not old-school statistics. This is especially important for big complex health questions.

To make this more concrete:

First, a huge amount of health data is collected. Really all of the health data we have should be put into public repositories (anonymized/protected by the gov). The gov should then fund largescale prediction contests based on this data (kaggle is one implementation, prediction markets are more advanced form). For example, predicting how many people in X demographic cluster receiving Y interventions will receive a breast cancer diagnosis in the next year, or next month, etc.

The models that make the best predictions can then be used to predict farther into the future, to predict the outcome of new interventions, etc.

First, a huge amount of health data is collected. Really all of the health data we have should be put into public repositories (anonymized/protected by the gov).

One problem with that idea is that it is really hard to effectively anonymize a large data set.

The health care system is not so unfriendly it will kill us all. Maybe it is time to stop using unfriendly to mean a wide variety of conditions.

It's unfriendly in the sense that its values are not aligned with ours. I thought that's what the U in UFAI meant. Luckily, it's either not so powerful or not so misaligned that it will kill us all. But I think it's more or less reasonable to say it's unfriendly.

To encode health into a market utility function, first we create financial contracts with an expected value which captures long-term health. We can accomplish this with a long-term contract that generates positive cash flow when a human is healthy, and negative when unhealthy - basically an insurance contract.

This comment is a follow-up to both this post, and Buy Health, Not Health Care.

Possible job title in a saner version of the future: Lifestyle Advisor.

"Paenacea Health, America's leading Life Maintenance Organization seeks Lifestyle Advisors to work in our New York, Los Angeles, Phoenix, and Austin field offices. Lifestyle Advisors grow to develop relationships with 40-80 members, and will be responsible for contacting members on a weekly basis to encourage them to adopt better health practices. Upon joining our team, Lifestyle Advisors will undergo a three week training program, in which candidates will be familiarized with our approach to assessing the weighted importance of different behavioral changes to the life expectancy extension of our members.

All Paenacea Health, we truly value our members' health-- which means that all members are assigned a Lifestyle Advisor. Ideal candidates for this position will have excellent interpersonal skills, and will be able to successfully coach Paenacea members through each step of processes like quitting smoking or establishing an exercise routine. Candidates are expected to have the persistence and vision necessary to proactively and insistently initiate contact with members who have not adopted adequate health practices within a short period of joining Paenacea Health..."

I really like this as an example of incentive misalignment, and I think it should be in the community's core materials on that subject.

That said, the ideas about healthcare contracts to fix the alignment problem could use some fleshing out. So here's my attempt at fleshing them out.

General properties of the problem:

  • We want an open market in healthcare ideas. Anybody should be able to invest in whatever idea they have.

  • Ideas which do not work should cost the investor, ideas which do work should reward the investor.

  • The ideas must be tried on some actual people. Those people, unavoidably, bear some risk, and must be incentivized to take that risk.

Solution: First, the cost to incentivize people to try ideas must be paid by someone. The investors are a natural choice here. So imagine that the investors offer payment to people to try their ideas. To avoid selection issues, the investors may specify that people must fulfill certain criteria in order to be eligible. Anyone who meets the criteria can sign up for whatever the investors are pushing. This would be sort of like an open market for clinical trials, except that it can scale to any number of people and to long times.

For their investment, the investors would get a contract. Under that contract, they get some regular payments for as long as their participants see the benefits the investors claimed and don't see serious problems. That payment ultimately needs to come from the participants; all participants pay to participate in the market as a whole. If the market is government-managed, then everyone is a participant and their payment comes from taxes.

As an example, consider a cure for prostate cancer. The investors put out their cure at some price, paying people to try their particular idea. Only those with prostate cancer are eligible. All participants get paid by the investors. As soon as a participant's cancer clears, the investors get paid a big chunk of money.Whenever the participant has health issues later in life, the investors have to pay some money back, regardless of what those later issues are. Presumably, more serious issues would be more expensive, and the original contract might specify an extra payment if the prostate cancer returns. (There would be a secondary market for investors to insure against normal later-life health risks, but investors would still need to handle more-than-normal problems, and would get paid for less-than-normal problems).

Now an even more interesting example: a new diet. The problem with monetizing a new diet is that, once word gets out, anyone can just try it without having to pay. Thus the prevalence of pills: pills can be patented and monetized. But with this market structure, the investors are PAYING people to participate. Anyone who's trying the diet will WANT to sign up, because they get paid. The investors can then reap the benefits (if any) over the person's lifetime.

Finally, a preventative example. Once again, we have a prostate cancer treatment, but this time it is preventative rather than a cure. This time, prostate cancer is not required for participation. The investors pay the participants for their effort, and make money for every prostate-cancer-free year of every participant's life. Again, there is more complexity around paying for any later problems, insuring those payments, etc... but that's all standardized stuff and the associated costs are known in advance.

As far as I understand the Microsoft-Band can already measure the amount of UV radiation a person perceives. If we are lucky that will become the standard for smart watches and we will get a lot of new data about sun exposure.

As far as Vitamin D research goes, a bunch of Quantified Self folks have found that for them taking vitamin D3 in the morning has good effects on sleep while taking it in the evening has bad effects on sleep.

The studies on Vitamin D don't track timing at all. A bunch of Vitamin D studies likely fail to produce good effects because the Vitamin D is taken at the wrong time.

An app that knows your sleep data and your sun exposure could calculate the amount of Vitamin D that you should take and when you should take it..

There is probably more to sunlight that just vit D and serotonin/melatonin.

Unfortunately it seems like sunlight causes wrinkles

Liberals see the free market as a kind of optimizer run amuck, a dangerous superintelligence with simple non-human values that must be checked and constrained by the government - the friendly SI. Conservatives just reverse the narrative roles.

I like this analogy. So basically, how do you want to balance the power between your two overlords, one much much smarter than you but with non-human values, and the other much dumber than you but with human (mostly) values.

Who says the state is dumb? It created the market, after all.

Who says the state is dumb? It created the market, after all.

What does that mean? Across history, there is a whole spectrum of complexity of both markets and governmental arrangements, and against that backdrop it's not clear that it means anything to talk about "the state" or "the market", still less one creating the other. You might as well say that "the market" created "the state", and I expect that a historian could argue either thesis with equal facility.

And some people would like to make it sit down and write "I will not conjure up what I can't control" a thousand times for this. But I, for one, welcome our efficient market overlords!