Focal at CMU

This is an area MIRI and the old rationalist crowd thought about a lot back in the day. There are a lot of ways for advanced intelligences to cooperate that are not available to humans, especially if they are capable of doing things in the class of sharing source code or can show their decisions are correlated with each other.

With sufficient capability, any group of agents should be able to act as if it is a single agent, and we shouldn’t need to do the game theory for them in advance either. I think it’s good things to be considering, but one should worry that even if they do find answers it will be ‘into the void’ and not accomplish anything. Based on my technical analysis I wasn’t convinced Focal was going to sufficiently interesting places with it, but I’m not at all confident in that assessment.

They note they’re also interested in the dynamics prior to Ai becoming superintelligent, as the initial conditions plausibly matter a lot.

To donate, reach out to Vincent directly at conitzer@cs.cmu.edu to be guided through the donation process.