Stuart Russell

Where they come up

Said by someone else — these sentences name them.

• “21 In Russell (2019) ’s memorable phrasing, “you can’t fetch coffee if you’re dead”—implying that even a policy with a simple goal like fetching coffee would pursue survival as an instrumental subgoal (Hadfield-Menell et al., 2017) .”

  · The Alignment Problem from a Deep Learning Perspective · machine-resolved

• “Stuart Russell’s Human Compatible book and his example of fossil fuels companies”

  · · machine-resolved

In the claim ledger

8 promoted claims about them. Assessments are the model’s knowledge, not verification.

• Instrumental convergence: survival and similar subgoals serve almost any final goal, including trivial ones.

  consistent · The Alignment Problem from a Deep Learning Perspective

• Treating humanity and machines as two monolithic agents is insufficient; risk analysis must span multiple organisational scales.

  plausible · TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI

• The paper's organising principle is accountability: whose actions caused the risk, whether unified, whether deliberate.

  consistent · TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI

• The authors claim the economic and legal disempowerment illustrated for a subgroup could extend to all of humanity.

  unverifiable · TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI

• RL systems can in principle learn arbitrary manipulation of human minds and institutions to pursue their objectives.

  consistent · TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI

• There is a fundamental tension between deals that appear acceptable to all parties and deals that are equitable over time.

  plausible · TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI

…and 2 more, in the posts’ own ledgers.