To set the record straight: I once gave a ~2% probability for the classic AGI-doom paperclip-maximizer-like scenario. I have a much higher probability for an existential catastrophe in which AI is causally involved in one way or another — there are many possible existential catastrophes (nuclear war, pandemics, runaway climate change…), and many bad people who would cause or fail to prevent them, and I expect AI will soon be involved in just about everything people do! But making a firm prediction would require hashing out what it means for AI to play a “critical causal role” in the catastrophe — for example, did Facebook play a “critical causal role” in Trump’s victory in 2016? I’d say it’s still not obvious, but in any case, Facebook was far from the only factor.
I would argue that there is pretty solid journalistic level evidence that it was a contributing factor (and not by accident either--it happened on purpose). Given how close the election was...
Also, the GREENS took some votes from Hillary Clinton. And guess who had lunch with their candidate? Vladimir Putin. Misha Frier: "Jill Stein launched the 2024 White House bid as a Green Party candidate. Is she getting any support from Mother Russia again?
Back in 2015, after the Annexation of Crimea and the launch of the war in Donbas, Jill Stein was at a gala dinner in Moscow to celebrate the 10th birthday of the Russian TV propaganda network Russia Today. She was there at the invitation of Vladimir Putin and sat behind the same table with him and two former KGB agents, Gromov (chief of propaganda) and Ivanov (Putin's chief of staff), at the gala dinner.
In 2016, Stein's presidential campaign was heavily promoted by RT. I mean what better person to ask for endorsement of an environmentalist candidate that a dictator who bases his power on oil and gas sales to the West?
To be clear, your 2% risk is from being paperclipped (inner alignment failure + fast local takeoff?) and the probability above 2% comes from exacerbating non-AGI risks? Does that mean you don't have much room for non-paperclip AI catastrophe, like rapid value drift from substituting ourselves out of the economy or fast local takeoff from a badly outer-aligned AGI, etc?
To set the record straight: I once gave a ~2% probability for the classic AGI-doom paperclip-maximizer-like scenario. I have a much higher probability for an existential catastrophe in which AI is causally involved in one way or another — there are many possible existential catastrophes (nuclear war, pandemics, runaway climate change…), and many bad people who would cause or fail to prevent them, and I expect AI will soon be involved in just about everything people do! But making a firm prediction would require hashing out what it means for AI to play a “critical causal role” in the catastrophe — for example, did Facebook play a “critical causal role” in Trump’s victory in 2016? I’d say it’s still not obvious, but in any case, Facebook was far from the only factor.
I would argue that there is pretty solid journalistic level evidence that it was a contributing factor (and not by accident either--it happened on purpose). Given how close the election was...
https://en.wikipedia.org/wiki/Facebook%E2%80%93Cambridge_Analytica_data_scandal
Also, the GREENS took some votes from Hillary Clinton. And guess who had lunch with their candidate? Vladimir Putin. Misha Frier: "Jill Stein launched the 2024 White House bid as a Green Party candidate. Is she getting any support from Mother Russia again?
Back in 2015, after the Annexation of Crimea and the launch of the war in Donbas, Jill Stein was at a gala dinner in Moscow to celebrate the 10th birthday of the Russian TV propaganda network Russia Today. She was there at the invitation of Vladimir Putin and sat behind the same table with him and two former KGB agents, Gromov (chief of propaganda) and Ivanov (Putin's chief of staff), at the gala dinner.
In 2016, Stein's presidential campaign was heavily promoted by RT. I mean what better person to ask for endorsement of an environmentalist candidate that a dictator who bases his power on oil and gas sales to the West?
Stein didn't win, but she contributed to Donald Trump winning the election that year. Her vote totals in the crucial states of Pennsylvania, Wisconsin and Michigan arguably denied Hillary Clinton an Electoral College victory." - Also CNN https://edition.cnn.com/2016/11/10/politics/gary-johnson-jill-stein-spoiler/index.html
No AI, just ol` KGB-I.
To be clear, your 2% risk is from being paperclipped (inner alignment failure + fast local takeoff?) and the probability above 2% comes from exacerbating non-AGI risks? Does that mean you don't have much room for non-paperclip AI catastrophe, like rapid value drift from substituting ourselves out of the economy or fast local takeoff from a badly outer-aligned AGI, etc?
No, all those other things go into the beyond-2% zone.
Why 2% Scott? Or rather, how did you get to the figure?
I find it a little bit funny that you give it basically the exact same probability as P=NP :
"I’d give it a 2 to 3% chance that P equals NP"
https://www.technologyreview.com/2021/10/27/1037123/p-np-theoretical-computer-science