Empire of AI: Dreams and Nightmares in Sam Altman's OpenAI

Hao describes her own experience living in San Francisco, where income inequality is very high. While tech workers like Hao live and work in abundance, unhoused people, or people experiencing addiction are left without support. She argues that this is emblematic of how “the tech industry could profess big, bold visions about changing the world […] while ignoring the very problems at its door” (227).

It was within this context that the Effective Altruism (EA) movement took root in Silicon Valley. EA is a philosophy that argues, in part, that the most effective way to help others is to make a lot of money and use it to fund non-profit work focused on the biggest global problems, with many EA adherents identifying AGI as the single greatest danger facing humanity. EA ideology profoundly impacted AI “safety” discourse. A key vector of this was the imprisoned former billionaire Sam Bankman-Fried (SBF) who was a proponent of both EA and AI safety and spent lavishly to promote the ideology before being imprisoned for financial fraud. AI safety discourse became largely focused on one’s understanding of “probability of doom” or p(doom), the term used to describe “how likely you think it is that AGI will lead to catastrophic outcomes” (232) such as human extinction.