Forethought

Forethought does some of the best AI related futurism and strategy work, and I think it's very valuable that they're thinking about this. They seem at least somewhat funding constrained, which is unfortunate, and it seems that marginal donations will cause them to do more work.

I asked Forethought to apply. I think Forethought does great work, and am happy to support it.

Theory of change: I largely view Forethought's theory of change as doing high-level strategic thinking about what should be done in the world, given that we will plausibly see Transformative AI in the near future. What questions are people missing? How should we prioritize? What mistakes are we making? etc.

Empirically, their work on AI-enabled coups seems to have been somewhat influential, and important people at Anthropic and DeepMind think highly of them. So, I think they are well-placed to have an impact with this work.

They have a bunch of extremely smart and capable people, and I expect these people are well placed to think about fucky, confusing things and get somewhere.

The track record of this kind of work seems fantastic to me. I'm not confident there is still significant impact to be had, but it's plausible enough that it seems like it should clearly have a non-trivial investment.

I have not personally engaged much with Forethought's work (it's not super relevant to what I personally am doing, and I'm very behind on paper reading). So, this is less a personal endorsement based on an inside view, and more that this just seems obviously reasonable. Multiple people I respect seem to think well of Forethought.

Why me? Why am I, as a regranter, making this grant? I was a bit hesitant about this, but based on chatting to grantmakers it didn't seem like Forethought was likely to have its funding gap filled by large donors, yet both Open Philanthropy and Longview said positive things when I asked about giving to Forethought. For various reasons that I don't think are particularly important OpenPhil can't give money to Forethought directly, but still endorses them.

Funding gap: Forethought has a significant funding gap for their goal of 18 months runway at an aggressive growth rate, though they also have enough runway that I see this donation as marginally helping them be comfortable with an aggressive growth rate and increasing their runway, rather than something critical or deeply counterfactual. But I also think it is good to not insist on being the donor of last resort.

I also haven't found anything better to spend my regranting budget on, and so think I should err more towards funding things and away from procrastination / holding out for something better, especially with the prospect of a bunch of Anthropic employee funding in future. But that was a fairly marginal factor and I endorse the grant on the merits.

Forethought’s work has been some of the biggest influence on my safety research work, raising several new areas of research I and some of my collaborators hadn’t previously considered or prioritized. Keep up the amazing work!