New paper by Nick Bostrom: AI Creation and the Cosmic Host
Quote: "There may well exist a normative structure, based on the preferences or concordats of a cosmic host, and which has high relevance to the development of AI."
The paper argues there is likely a “cosmic host”: Powerful natural or supernatural agents (e.g., advanced civilizations, simulators, deities) whose preferences shape cosmic-scale norms. This is motivated by the simulation argument, large/infinite-universe and multiverse hypotheses.
The paper also suggests the cosmic host may actually want us to build superintelligence. A superintelligence would be more capable of understanding and adhering to cosmic norms than humans are, potentially making our region of the cosmos more aligned with the host's preferences. Therefore, the host might favor a timely development of AI, as long as it becomes a good cosmic citizen.
Paper:
https://nickbostrom.com/papers/ai-creation-and-the-cosmic-host.pdfThis is a serious version of the following tongue-in-cheek derivation of a God-like coalition via acausal trade by Scott Alexander:
https://slatestarcodex.com/2018/04/01/the-hour-i-first-believed/If you have been involved with rationalists for as long as I have, none of this will be surprising. If you follow the basic premises of rationality to their logical end, things get weird. As muflax once wrote:
"I hate this whole rationality thing. If you actually take the basic assumptions of rationality seriously (as in Bayesian inference, complexity theory, algorithmic views of minds), you end up with an utterly insane universe full of mind-controlling superintelligences and impossible moral luck, and not a nice “let’s build an AI so we can fuck catgirls all day” universe. The worst that can happen is not the extinction of humanity or something that mundane – instead, you might piss off a whole pantheon of jealous gods and have to deal with them forever, or you might notice that this has already happened and you are already being computationally pwned, or that any bad state you can imagine exists. Modal fucking realism."