Epoch AI

I'm re-granting because I've been very impressed with the quality of Epoch's work. It's clearly far and away better than any competition when it comes to actually understanding what's going on with AI (and being publicly communicated), and several people I know who are part of Epoch strike me as highly competent. I'm also a fan of their new weekly newsletter. I haven't carefully checked the details of much of my work myself, but the one example where I thought I found an error (using total rather than active parameters of an MoE model), they then produced a newsletter with a compelling argument for why their way made more sense, which I was impressed by.

I see the main theory of impact of Epoch as broadly helping key decisionmakers in society (policy-makers, businesses, the AI safety community, etc) be well informed about AI: what has already happened, what is happening, what will happen. I'm a bit less confident in how well their projects do on the dissemination front. I see them go viral fairly often, though I am totally in a massive bubble. I've seen some scattered examples like Satya Nadella having Epoch graphs in a slide deck, which seem pretty promising!

Is this theory of impact important?

I think that AI is clearly a big deal, at this point, having an important impact on society, and expect this to accelerate. In addition to the big issues of misuse and misalignment, there's also just a lot of social and policy change that needs to happen to handle this well. By default, society changes slowly, and tech policy often makes little sense. High quality information is far from sufficient to fix this, but it helps!

I have some concern that this also has negative externalities by increasing the number of people who realise AI is a big deal and decide to race (in a similar sense to how, eg, scaling laws got more people realising the potential of AI). But my guess is that this isn't that big a deal. I think that "AI is a big deal" is pretty widely believed at this point, so that ship has largely sailed. Another concern is that this increases competitive pressures by making it clearer to people when they're losing the race. But on the other hand, this helps the person in first place realise it, which is high value (see eg the Cold War "missile gap"). And I expect key actors like AGI labs already have much better awareness of what's going on than most, so Epoch adds less value to them.

Overall, this isn't my area of expertise so I'm not confident, but my best guess is that Epoch is doing great work.

I'm only donating a token amount as I don't think this is my comparative advantage as a funder, but I would encourage others to donate, and want Epoch to have as much funding as they can productively use.

I think Epoch has done truly outstanding work on core trends in AI progress in the past few years. I'm also excited by their recent foray into benchmarking in the form of FrontierMath. I think highly of core team members involved in the project. I found our initial discussions about this project very promising.

Better benchmarks that help us forecast time to AGI (and especially time to relevant capabilities, such as automated AI research) and do so in a highly credible and scientific way are very valuable for informing policymakers and catalyzing important policy efforts.

Donor's main reservations

It's a pilot, it might not work.

Epoch has other funding—but not for this effort, and benchmarking is especially expensive (API calls, labelers).

Process for deciding amount

I reviewed a proposed budget. (Confidential, more on request from Manifund.)

Conflicts of interest

Please disclose e.g. any romantic, professional, financial, housemate, or familial relationships you have with the grant recipient(s).

No COIs.