One of the subskills mentioned in Eliezer's Security Mindset post is mitigating assumption risk–that is, the risk of losing utility because some of your assumptions are wrong. There are two main ways to do this:
- Gain more information about whether your assumptions hold
- Make the assumption irrelevant (such as the hashing passwords example)
Here are a bunch more examples:
- Repeating back what someone said in your own words, to check understanding
- Adding a margin of safety when estimating how much load a bridge can bear
- Using Statistical models that make fewer assumptions, or have fatter tails
- Exposing your work to attack in low-risk situations, such as comedians testing new material in small clubs, or Netflix's Chaos Monkey
- Emphasizing fast adaption to unexpected circumstances over better forecasting
- Putting spare capacity in steps in your process that aren't the bottleneck
- Testing code frequently while refactoring to check functionality doesn't unintentionally change
- Doing an analysis in different ways on different datasets, and only trusting them when the conclusions match
Keeping necessary materials and short-term inputs stored locally, to avoid exposure to supply chains or service outages.
Build and use applications that don't rely on always-on internet.
Download files, including music and videos, whenever possible, in addition to any cloud storage.
Locate resource sources and/or stored reserves close to you in physical space. Keep reserves of key materials, and of tradable resources.
Gather capital that cannot be confiscated or lost, especially human capital.
Avoid debt, especially debt that is not carefully bounded, is high interest or that could potentially balloon in size.
Stress test to trigger failure modes, so you know where those modes are, and also so you know what the consequences of failure are in context (and to remind yourself that they're not so bad).
Avoid or minimize prioritizing tasks by urgency or deadline when possible.
Avoid priortizing tasks by value beyond a certain threshold, so long as you are confident that you have sufficient resources to complete all tasks above that threshold.
Don't entirely hold off on engaging in high-value places even if you would prefer a higher-quality response. (e.g. the problem where you don't respond to an email because it deserves a good response, so it never gets one at all).
(This seems fun and useful, these were my first brainstorms)
There's definitely a lot of meat to dig into. Your mention of legal reminds me that having backups for when legal options fail, or the law is actively turned against you (for any reason) are also important.
What would you do if all of your accounts are frozen and you can't use any credit cards or other electronic sources of money? This could happen due to something like identity theft, so it's not even assuming legal trouble, let alone legal trouble you deserve.
What would you do if you needed to be off the grid entirely?
And the central poin... (read more)