As a hypothetical example, Bostrom asked what would happen to civilization if a weapon of mass destruction comparable to an atomic bomb was much easier to make, on the level of cooking sand in a microwave or something like that. The natural answer being, once knowledge of the technique spreads, any random psychopath can do it, and some do so, and soon thereafter bomb us back into the stone age. Civilization then can't rebuild past the point of microwaves without getting destroyed again, as long as the knowledge persists.persists (making black marbles an extreme type of information hazard). But this scenario is path dependent. One could imagine a different civilization with different capabilities that could survive such knowledge. Perhaps one with a world government (no wars) and a screening for psychopathy, etc. Perhaps a dystopian world panopticon could prevent use. Or, for a space-faring civilization that mostly lives in small independent orbital colonies, everybody already has (and civilization is somehow currently surviving) similarly destructive kinetic attack capabilities so maybe the sand-nukes don't change much.
ABlack Marble is a technology that by default destroys the civilization that invents it. It's one type of Existential Risk. AGI may be such an invention, but isn't the only one.
A technology that by default destroys the civilization that invents it. It's one type of Existential Risk. AGI may be such an invention., but isn't the only one.
As a hypothetical example, Bostrom asked what would happen to civilization if a weapon of mass destruction comparable to an atomic bomb was much easier to make, on the level of cooking sand in a microwave or something like that. The natural answer being, once knowledge of the technique spreads, any random psychopath can do it, and some do so, and soon thereafter bomb us back into the stone age. Civilization then can't rebuild past the point of microwaves without getting destroyed again, as long as the knowledge persists. But this scenario is path dependent. One could imagine a different civilization with different capabilities that could survive such knowledge. Perhaps one with a world government (no wars) and a screening for psychopathy. Forpsychopathy, etc. Perhaps a dystopian world panopticon could prevent use. Or, for a space-faring civilization that mostly lives in small independent orbital colonies, everybody already has (and civilization is somehow currently surviving) similarly destructive kinetic attack capabilities so maybe the sand-nukes don't change much.
As a hypothetical example, Bostrom asked what would happen to civilization if a weapon of mass destruction comparable to an atomic bomb was much easier to make, on the level of cooking sand in a microwave or something like that. The natural answer being, once knowledge of the technique spreads, any random psychopath can do it, and some do so, and soon thereafter bomb us back into the stone age. Civilization then can't rebuild past the point of microwaves without getting destroyed again, as long as the knowledge persists. But this scenario is path dependent. One could imagine a different civilization with different capabilities that could survive such knowledge. Perhaps one with a world government (no wars) and a screening for psychopathy. For a space-faring civilization that mostly lives in small independent orbital colonies, everybody already has (and is somehow currently surviving) similarly destructive kinetic attack capabilities so maybe the sand-nukes don't change much.
The name comes from a thoughtexperiment by Nick Bostrom,Bostrom, where he described inventions as pulling marbles out of an urn. Most are white (beneficial), some are dangerous or a mixed blessing (usually described as gray or red balls), and some are black (fatal).
A technology that by default destroys the civilization that invents it. It's one type of existential risk. Existential Risk. AGI may be such an invention.invention.
The name comes from a thought experiment by Nick Bostrom, where he described inventions as pulling marbles out of an urn. Most are white (beneficial), some are dangerous or a mixed blessing,blessing (usually described as gray or red balls), and some are black (fatal).
Thanks for bringing up the definition. So for example, a different planet where the isotope mix for uranium is different, so that weapons grade u-235 rocks can be commonly found, would face the black marble problem.
Someone could learn how to bang the rocks together and form a critical mass with cannon technology, maybe less - just enough machining so the 2 masses make good contact and free fall might be enough to get an explosion.
So in some medieval or pre industrial revolution era, small fission bombs get used and technology cannot advance past this point because people keep setting off fission bombs until they destroy so much infrastructure and information they forget how to do it. Then a few generations later, someone rediscovers the method, perhaps by leftover writing or artifacts, and so on. Or alternatively every city has to be too small to be worth a nuke, and nobody bothers with castles, it's all mobile forces.
Either way it seems like a bottleneck on city size or technology.
It is possible no black marbles will ever be drawn for technology on earth before humans spread out past the earth, preventing this problem.