EA - On the Vulnerable World Hypothesis by Catherine

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: On the Vulnerable World Hypothesis, published by Catherine on August 1, 2022 on The Effective Altruism Forum. This is a cross-post from my Substack. Summary Nick Bostrom presents the Vulnerable World Hypothesis (VWH): if technological progress continues without safeguards, we should expect the means and information necessary to cause a catastrophic event to become sufficiently available for malicious actors to cause such an event. He argues that to the extent that the VWH is true, we should consider implementing ubiquitous real-time worldwide surveillance (henceforth just surveillance) to mitigate the risk of these catastrophic events. I have some misgivings about this argument, which I divide into three parts. First, I consider the premises and assumptions of the VWH. I argue that: not many people want to cause global catastrophes it's not clear that we must assume continuing technological development Second, I consider the costs and benefits of surveillance. I start with a best case scenario, where I assume surveillance is: implemented by good actors extremely effective (if not perfect) democratically agreed upon global and argue that even under these conditions, surveillance: limits free thought disincentivises weirdness potentially erodes trust norms and diminishes the value of trust I also make a more general argument against systems with a single point of failure. Third, I relax these assumptions one by one. Relaxing the good actors assumption increases totalitarianism risk and (more weakly) misuse risk Relaxing the effectiveness assumption suggests surveillance would not be worth its costs, and may lead to a false sense of security Relaxing the democratic assumption increases totalitarianism risk and would be bad in itself Relaxing the global assumption reduces surveillance's effectiveness My main concerns are that: It's not clear that surveillance is the best response to the VWH Even if the best response is surveillance, its costs probably outweigh its benefits: Even the best case scenario has distinct costs and seems very unlikely In particular, it seems really hard to achieve global democratic surveillance Relaxing the assumptions of the best case scenario increases the costs There's a real risk that surveillance increases the likelihood of totalitarianism, including totalitarian lock-in So as things stand, I don't think ubiquitous real-time worldwide surveillance is a good idea. A quick disclaimer/some context I wrote this post in a semi-academic style, and I'm worried that this signals undue seriousness or confidence in my thoughts. I'm really not very confident in what I've said! Everything here was written up quickly after some initial thoughts and discussion with others. At times I make claims and hardly defend them; at other times I defend my claims, but I expect given some time to reflect on them, I'd change my mind. Interpret this as 'some thoughts I had and wanted to write up', rather than 'a carefully reasoned critique of an academic paper'. I'd like to hear criticism and corrections, particularly if you think they undermine key parts of my argument. Thinking about the VWH Bostrom defines the VWH as follows: Bostrom later stipulates that 'civilisational destruction' here refers to: Such events would be considered global catastrophic risks (GCRs) under most definitions of GCRs. Given this (and for simplicity) , I'll refer to them as global catastrophes. Bostrom presents the VWH as just that - a hypothesis - and states explicitly that he thinks its truth is still an open question. For my purposes it's helpful to restate the VWH as an argument with three premises, remembering that Bostrom would not commit to it: P1: The information necessary to cause an event which is itself a global catastrophe, or likely to lead to a global catastrophe, is sufficie...