SaferAI

I’ve been impressed by Simeon and his track record, including here. Simeon is stepping down as leader to start a company, which happened post-SFF, so they would need to be reevaluated in light of this before any substantial donation.

Donate here, or contact them at contact@safer-ai.org.

Main points in favor of this grant

SaferAI have rapidly since their start in 2023 established themselves as a reputable and informed source on AI risk management and standards. They have assembled a team with diverse relevant experience, and built connections with key stakeholders in the policy community to effectively disseminate their research output.

Donor's main reservations

SaferAI’s rapid growth coupled with an inexperienced leadership team is likely to put a strain on management capacity. This risk is exacerbated by the team taking on a diverse range of projects (standards, ratings, direct policy advising) that, while complementary, will impose additional overhead on the leadership team to coordinate. The team intends to explore a more focused approach over the next 6 months to mitigate this risk.

Process for deciding amount

In principle I would be excited to fund at least the next $300k of SaferAI’s funding gap based on their described marginal uses of funding. My overall remaining grantmaking budget is around $125k; so I am allocating the bulk of my remaining budget (and around 25% of my total Manifund budget) to SaferAI. I wish to retain some remaining discretionary funds for small, high-value projects. I have not conducted an extensive search for possible projects in this space so I make no claim that SaferAI is the highest impact project at the margin, but I believe it clears the more general bar for funding of technical AI governance projects.

Conflicts of interest

None; I have no significant relationship with any of the SaferAI team. I know Siméon best but this has been limited to infrequent (~twice a year) chats at the sides of conferences.