I want to propose a variant of the Counterfactual Mugging problem discussed here. BE CAREFUL how you answer, as it has important implications, which I will not reveal until the known dumb humans are on record.
Here is the problem:
Clipmega is considering whether to reveal to humans information that will amplify their paperclip production efficiency. It will only do so if it expects that, as a result of revealing to humans this information, it will receive at least 1,000,000 paperclips within one year.
Clipmega is highly accurate in predicting how humans will respond to receiving this information.
The smart humans' indifference curve covers both their current condition and the one in which Clipmega reveals the idea and steals 1e24 paperclips. (In other words, smart humans would be willing to pay a lot to learn this if they had to, and there is an enormous "consumer surplus".)
Without Clipmega's information, some human will independently discover this information in ten years, and the above magnitude of the preference for learning now vs later exists with this expectation in mind. (That is, humans place a high premium on learning it how, even though they will eventually learn it either way.)
The human Alphas (i.e., dominant members of the human social hierarchy), in recognition of how Clipmega acts, and wanting to properly align incentives, are considering a policy: anyone who implements this idea in making paperclips must give Clipmega 100 paperclips within a year, and anyone found using the idea but not having donated to Clipmega is fined 10,000 paperclips, most of which are given to Clipmega. It is expected that this will result in more than 1,000,000 paperclips being given to Clipmega.
Do you support the Alphas' policy?
Problem variant: All of the above remains true, but there also exist numerous "clipmicros" that unconditionally (i.e. irrespective of their anticipation of behavior on the part of other agents) reveal other, orthogonal paperclip production ideas. Does your answer change?
Optional variant: Replace "paperclip production" with something that current humans more typically want (as a result of being too stupid to correctly value paperclips.)
A pack of 10 boxes of 100 paperclips each costs $2 USD. I infer from this that the world supply of paper clips is large enough that buying 1,000 such boxes would not significantly move the supply-demand curve.
Satisfying Clipmega's demand is therefore within the means of any middle-class family. If anyone cares about maximizing paper clip production, they could provide the million paper clips without impacting anyone else. If more than one person cares (or, if no-one wants to be the one person who's out two grand while everyone else benefits), someone could make a kickstarter.
The Alphas' complex plan just drives up the transaction cost. Actually, though I started this thinking that Clippy's number choice was sloppy, that's an interesting factor now that I think about it. Any attempt by the Alphas to ensure "fair" distribution of the costs is going to increase inefficiency by a significant fraction or multiple of what Clipmega is actually asking for - at some point you have to stop fighting over the bill and just pay it.
Would you support a policy of "The human Alphas (i.e., dominant members of the human social hierarchy), in recognition of how Clipmega acts, and wanting to properly align incentives, are considering a policy: anyone who implements this idea in making paperclips must give Clipmega twenty cents within a year, and anyone found using the idea but not having donated to Clipmega is fined twenty dollars, most of which is given to Clipmega. It is expected that this will result in more than $2,000 being given to Clipmega."? I wouldn't. They should just buy the paper clips with the money they'd be paying paperclip factory auditors. I wouldn't support it if I were one of the Alpha's either - there's got to be a cheaper way to force someone else to pay it, if nothing else.