Week 1 - The present and Future of AI
How could AI transform society over the next few decades?
AI capabilities have rapidly advanced in recent years, across all types of models, and this trend looks likely to continue. This might bring about transformative AI: systems that would bring us into a new, qualitatively different future. In addition, in the short-term it looks likely that existing AI systems will be deployed much more broadly.
In this session, you'll gain an appreciation of the rapid advancements we’ve seen so far. You'll then evaluate predictions for when new AI capabilities will emerge, focusing on the idea that at some point in the near future (years to decades), we may develop AI systems that can outperform humans at most tasks (noting that all predictions about the future are wildly uncertain). Finally, you'll consider the societal impacts of current and future AI capabilities, including the potential to solve global problems as well as cause catastrophic harm.
Optional background material:
A Brief History of AI (Roser, 2022) (15 mins)
A short introduction to machine learning (Ngo, 2021) (20 mins)
Gradient descent, how neural networks learn (3Blue1Brown, 2017) (20 mins)
But what is a GPT? Visual intro to transformers (3Blue1Brown, 2024) (30 mins)
Starting exercise:
Before you start reading through the curriculum, take five minutes to write down: Why did you decide to sign up for the course? What risks do you foresee as AI models become more powerful? What questions do you have that you’re hoping the course to answer? This will help you to start the readings with clear goals in mind, enabling you to focus on aspects most relevant to your interests and concerns.
Core readings:
AI is transforming our world - how to make it go well? (Roser, 2022) (20 mins)
This article defines transformative AI, and provides some pointers for thinking about what the future might look like. This will be useful for the exercises and session discussions.
On the opportunities and risks of foundation models (Bommasani et al., 2021) (only pages 3-6) (10 mins)
Bommasani et al. provide a high-level overview of foundation models, how they're trained, and how they fit into the broader field of ML.
Scaling Laws for Neural Language Models (Kaplan et al., 2020) (only pages 2-4) (10 mins)
Kaplan et al. explain how the performance of language models improves predictably with increases in model size, dataset size, and computational resources, following a power law. The authors argue that these trends span several orders of magnitude, while other architectural details such as network width or depth have minimal effects within a wide range.
AI Timelines (Roser, 2023) (15 mins)
This article discusses how we might go about forecasting future developments in the capabilities of AI models and the time when the first generally intelligent model will be built. It also discusses expert opinions on the subject.
AGI Safety from First Principles (Ngo, 2020) (from section 1 to end of 2.1) (20 mins)
Ngo describes the difference between narrow and general AI systems and why we might expect general AI systems to be developed. This reading helps to introduce what we mean by 'AGI' and some claims about why it might be a technical possibility.
Four Background Claims (Soares, 2015) (15 mins)
This reading introduces high-level claims about AI systems, including whether they're possible to build and how they might shape the future. Note we'll explore claim 4 in more detail next session. This piece was written before current state-of-the-art transformer models were invented. You could contrast what we know about modern ML and its potential to achieve generality with the claims made in this article.
Further readings:
This article argues that in the field of AI, general methods that scale with increased compute outperform (expert) human-knowledge-based approaches.
AI and compute: how much longer can computing power drive AI progress? (Lohn and Musser, 2022) (30 mins)
This and the next two readings focus on forecasting progress in AI, via looking at trends in compute and algorithms, and surveying expert opinions.
Thousands of AI Authors on the Future of AI (Grace et al., 2024) (30 mins)
Biological Anchors: A Trick That Might Or Might Not Work (Alexander, 2022) (30 mins)
This reading provides a critical review of one popular method for forecasting progress in AI: biological anchors.
AI: racing towards the brink (Harris and Yudkowsky, 2018) (110 mins) (audio here)
Transcript of a podcast conversation between Sam Harris and Eliezer Yudkowsky. It covers many of the topics from this week and next week.
AI Trends (Epoch, 2023) (5 mins)
How Predictable Is Language Model Benchmark Performance? (Owen, 2023) (5 mins)
Training Compute-Optimal Large Language Models (Hoffmann et al., 2022) (only sections 1 and 2) (15 mins)
This paper from DeepMind provides a correction to the original scaling laws outlined in the paper by Kaplan et al.
Resource spotlight: AI Safety Fundamentals
AI Safety Fundamentals is a project by the Cambridge-based non-profit BlueDot Impact that offers introductory courses on AI safety. This curriculum is heavily influenced by their courses. Once you’ve finished this course, you can go to the Alignment 201 curriculum on their page for further readings, or to the Governance curriculum to learn more about AI policy.
Exercises:
As discussed in Ngo (2020), Legg and Hutter define intelligence as “an agent’s ability to achieve goals in a wide range of environments”: a definition of intelligence in terms of the outcomes it leads to. An alternative approach is to define intelligence in terms of the cognitive skills (memory, planning, etc) which intelligent agents used to achieve their desired outcomes. What are the key cognitive skills which should feature in such a definition of intelligence? Which definition do you find more useful?
If you had to forecast when the first generally intelligent AI system is developed, what would be your guess? Why?