#62 – Häggström on AI Motivations and Risk Denialism


In this episode I talk to Olle Häggström. Olle is a professor of mathematical statistics at Chalmers University of Technology and a member of the Royal Swedish Academy of Sciences (KVA) and of the Royal Swedish Academy of Engineering Sciences (IVA). Olle’s main research is in probability theory and statistical mechanics, but in recent years he has broadened his research interests to focus applied statistics, philosophy, climate science, artificial intelligence and social consequences of future technologies. He is the author of Here be Dragons: Science, Technology and the Future of Humanity (OUP 2016). We talk about AI motivations, specifically the Omohundro-Bostrom theory of AI motivation and its weaknesses. We also discuss AI risk denialism.

You can download the episode here or listen below. You can also subscribe to the podcast on Apple Podcasts, Stitcher and a variety of other podcasting services (the RSS feed is here).

Show Notes

  • 0:00 – Introduction
  • 2:02 – Do we need to define AI?
  • 4:15 – The Omohundro-Bostrom theory of AI motivation
  • 7:46 – Key concepts in the Omohundro-Bostrom Theory: Final Goals vs Instrumental Goals
  • 10:50 – The Orthogonality Thesis
  • 14:47 – The Instrumental Convergence Thesis
  • 20:16 – Resource Acquisition as an Instrumental Goal
  • 22:02 – The importance of goal-content integrity
  • 25:42 – Deception as an Instrumental Goal
  • 29:17 – How the doomsaying argument works
  • 31:46 – Critiquing the theory: the problem of self-referential final goals
  • 36:20 – The problem of incoherent goals
  • 42:44 – Does the truth of moral realism undermine the orthogonality thesis?
  • 50:50 – Problems with the distinction between instrumental goals and final goals
  • 57:52 – Why do some people deny the problem of AI risk?
  • 1:04:10 – Strong versus Weak AI Scepticism
  • 1:09:00 – Is it difficult to be taken seriously on this topic?

Relevant Links


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s