**Main points in favor of this grant**
- Dan Hendrycks' AI safety textbook seems great, but principally serves as an introduction to the field, rather than an in-depth overiew of current technical AI safety research directions, which is the intent of this project. Periodically updated "topics courses" could serve as an equivalent source of value, but these might be bound to particularly universities and updatable on a slower timescale than an online textbook. I'm also enthused by Markov's plans to eventually integrate interactive content and live content from sources like Metaculus, Our World in Data, Stampy, and more.
- I believe that the AI safety research field should grow 10-100x over the next 10-20 years and AI safety student groups should be a strong driver of this growth. Currently, I think AI safety student groups need more "plug-and-play" curricula to best prepare members for progression into research, engineering, and policy roles, especially at universities without dedicated AI safety courses like that based on Hendrycks' textbook. I think BlueDot Impact's AI Safety Fundamentals courses are great, but I don't see why BlueDot and CAIS should be the only players in this space and think there is some benefit from healthy competition/collaboration.
- Charbel has road-tested content from the early stages of this textbook project with several AI safety university groups and courses with apparently good feedback.
- I've been impressed with Charbel's LessWrong posts and nuanced takes on AI safety research agendas.
- The online version of the textbook will be free and open-source (MIT License), which I think is important for introductory AI safety fieldbuilding materials to be maximally impactful.
**Donor's main reservations**
- I think that the optimal form of this project is a continually updated online resource that periodically integrates new papers and research paradigms and therefore this project will eventually need long-term funding and permanent home. However, I believe that my grant will greatly assist Charbel and Markov in producing a proof-of-concept sufficient to secure long-term funding or institutional support. Additionally, the textbook MVP seems likely to be high-value for the near-term regardless of whether the project continues. Lastly, if the textbook is high-value and Charbel and Markov are unable to secure long-term funding, I'm sure it will be useful for established curriculum developers like BlueDot Impact.
- I wonder if this project should actually be converted into a wiki once the MVP is developed. Markov has previously worked with Stampy and has mentioned that they might want to integrate some Stampy articles into the online textbook. However, even if this project is ideally a wiki, building a viable MVP seems crucial to securing long-term funding and core content for iterating upon.
- I don't know Markov or the proposed editor, Professor Vincent Corruble, very well, which slightly decreases my confidence in the textbook quality. However, Markov comes highly recommended by Charbel, has previously worked as at Rational Animations in charge of AI safety, and has produced good-according-to-me content for the textbook so far. Professor Corruble is an Associate Professor Sorbonne Université and a UC Berkeley CHAI affiliate, which indicate he has the technical expertise to oversee the computer science aspects of the the textbook. I additionally recommend that Charbel and Markov enlist the support of further editors with experience in AI safety strategy and AI governance, as I believe these are critical aspects of the textbook.
**Process for deciding amount**
I chose to donate enough to fund the minimum amount for this project to proceed because:
- I want full-time work on this textbook to commence immediately to minimize its time-to-impact and Markov is unable to do this until he receives confirmed funding for 6 months;
- I think it is relatively unlikely that this project will be funded by Open Philanthropy or the LTFF and I have greater risk tolerance for projects like this;
- I have 5x the grant budget I had last year and I think this project is probably more impactful than I would have considered necessary for a counterfactual $7.8k regrant made last year based on the projects I funded;
- I didn't give more than the minimum amount as I feel my marginal funding is high-value for other projects and I think Charbel and Markov can likely secure additional funding from other sources (including other Manifund regrantors) if necessary.
**Conflicts of interest**
I don't believe there are any conflicts of interest to declare.