The Future Society, Inc.

I’ve seen credible sources saying they do good work, and that they substantially helped orient the EU AI Act to at least care at all about frontier general AI. The EU AI Act was not a good bill, but it could easily have been a far worse one, doing much to hurt AI development while providing almost nothing useful for safety.

We should do our best to get some positive benefits out of the whole thing. And indeed, they helped substantially improve the EU Code of Practice, which was in hindsight remarkably neglected otherwise.

They’re also active around the world, including the USA and China.

Donate here, or contact them here.

The Future Society seeks to align AI through better governance. I reviewed some of its work, and it looks almost entirely irrelevant to x-risk.

Of The Future Society’s recent publications, the most concrete is “List of Potential Clauses to Govern the Development of General Purpose AI Systems” (pdf). Some notes on this report:

• The Future Society collected recommendations from industry staff, independent experts, and engineers from frontier labs. Engineers from frontier labs should not be trusted to produce recommendations, any more than petroleum engineers should be trusted to set climate change policy.
• The proposals for mitigating harmful behavior are mostly vacuous and in some cases harmful. They largely amount to: keep building dangerous AI, but do a good job of making it safe.
• “Use the most state-of-the-art editing techniques to erase capabilities and knowledge that are mostly useful for misuse.” That’s not going to work. (Palisade Research has demonstrated that it’s easy to remove safeguards from LLMs.)
• “Use state-of-the-art methods and tools for ensuring safety and trustworthiness of models, such as mechanistic interpretability.” This sentence makes me think the authors don’t have a good understanding of AI safety. The state of the art in mechanistic interpretability is nowhere close to being able to ensure the trustworthiness of models. We still have virtually no idea what’s going on inside large neural networks.
• The report proposes using the same industry-standard risk management model that the Center for Long-Term Resilience proposed. The same criticisms apply—this model is obvious enough that you don’t need to propose it, and severely insufficient for mitigating extinction risks.
• The report proposes “air gapping & sandboxing, no internet access” for powerful models. I feel like I shouldn’t need to explain why that won’t work.

Another report (pdf) submitted in response to the EU AI Act discussed seven challenges of “general-purpose AI”. The second challenge is “generalization and capability risks, i.e. capability risks, societal risks and extinction risks”. There is no further discussion of extinction risk, and this is the only place that the word “extinction” appears in any of The Future Society’s materials. (The word “existential” appears a few times, but existential risks are not discussed.)