ControlAI

ControlAI is the most x-risk-focused of the 501(c)(4)s, and the only one that advocates for a pause on AI development. They started operations in the UK, and this year they have expanded to the US.

Some thoughts:

• ControlAI’s open letter calling for an international treaty looks eminently reasonable.
• ControlAI had success getting UK politicians to support their statement on AI risk.
• They wrote a LessWrong post about what they learned from talking to policy-makers about AI risk, which was a valuable post that demonstrated thoughtfulness.
• I liked ControlAI last year, but at the time they only operated in the UK, so they weren’t a finalist. This year they are expanding internationally.

ControlAI is tentatively my favorite non-tax-deductible org because they’re the most transparent and the most focused on x-risk.

Control AI runs advocacy campaigns on AI risk.

Its current campaign) proposes slowing AI development such that no one develops superintelligence for at least the next 20 years, then using this time to establish a robust system for AI oversight. The campaign includes a non-vacuous proposal for the organizational structure of a regulatory body.

Control AI has a paper on AI policy that appears reasonable:

• It acknowledges that voluntary commitments from AI companies are insufficient.
• It proposes establishing international regulatory body that (1) imposes a global cap on computing power used to train an AI system and (2) mandates safety evaluations.
• It proposes that regulators should have the authority to halt model deployment on a model that they deem excessively dangerous.

The campaign’s proposal is similar. It lays out the most concrete plan I’ve seen for how to get to a place where we can solve AI alignment.

I listened to a podcast with Andrea Miotti, co-founder of Control AI. He mostly covered standard arguments for caring about AI x-risk, but he also made some insightful comments that changed my thinking a bit.

I like the concept of Control AI’s latest campaign, but I don’t know how much impact it will have.

Control AI’s past campaigns (example) have received media coverage (example) and their policy objectives have been achieved, although it’s not clear how much of a causal role Control AI played in achieving those objectives, or what Control AI actually did. Control AI clearly deserves some credit, or else news outlets wouldn’t cite it.

Control AI might be as impactful as other advocacy orgs that I like, but I have more uncertainty about it, so it’s not a top candidate. It would be fairly easy to change my mind about this.

I couldn’t find any information about Control AI’s funding situation, and I didn’t inquire because it wasn’t one of my top candidates.