Horizon Institute for Public Service

Horizon Institute for Public Service runs a fellowship where it places people into positions in governments and think tanks. It claims to be reasonably successful. (I do not have much of an opinion as to how much credit Horizon Institute deserves for its fellows’ accomplishments.)

Horizon Institute has received an undisclosed amount of funding from Open Philanthropy (along with some other big foundations).

Do Horizon fellows care about x-risk, and does their work reduce x-risk in expectation? Politico alleges that the Horizon Institute is a clandestine plot to get governments to care more about x-risk. I’m not a fan of clandestine plots, but that aside, should I expect Horizon fellows to reduce x-risk?

Most of their work is not legible, so I’m skeptical by default. Caring about x-risk is not enough to make me trust you. Some people take totally the wrong lessons from concerns about x-risk (especially AI risk) and end up increasing it instead. Case in point: OpenAI, DeepMind, and Anthropic all had founders who cared about AI x-risk, and two of those (OpenAI + Anthropic) were founded with the explicit mission of preventing extinction. And yet OpenAI is probably the #1 worst thing that has ever happened in terms of increasing x-risk, and DeepMind and Anthropic aren’t much better.

I reviewed all the highlighted accomplishments of fellows that looked relevant to AI:

• AI risk management standards for NIST. Only marginally relevant to x-risk, but not bad.
• An article on how we shouldn’t worry about x-risk (!!).
• Auditing tools for AI equity. Unrelated to x-risk.
• Detecting AI fingerprints. Marginally related to x-risk.
• Autonomous cyber defense. Increasing the capabilities of cybersecurity AI is plausibly net negative.
• An article on the EU AI Act. Non-vacuous and discusses AI risk (not exactly x-risk, but close). Vaguely hints at slowing AI development.

In my judgment after taking a brief look, 3/6 highlighted writings were perhaps marginally useful for x-risk, 1/6 was irrelevant, and 2/6 were likely harmful. None were clearly useful.

Zvi Mowshowitz wrote:

In my model, one should be deeply skeptical whenever the answer to ‘what would do the most good?’ is ‘get people like me more money and/or access to power.’

I agree, but even beyond that, the Horizon fellows don’t seem to be “people like me”. They include people who are arguing against caring about x-risk.

I believe the world would be better off if Horizon Institute did not exist (credence: 55%).

And if I’m wrong about that, it still looks like Horizon fellows don’t do much work related to x-risk, so the expected value of Horizon Institute is low.