Nick Hay

Design and implement simulations of human cultural acquisition as both an analog of and testbed for AI alignment.

We’re funding Nick Hay to do AI alignment research as a visiting scholar at CHAI, advised by Andrew Critch and Stuart Russell. He will focus on creating environments to test AI’s ability to culturally acquire desirable properties. The idea is to design ML environments where the goal is to “fit in” with a group of existing agents in the environment—e.g. learn what the different roles are that the other agents are playing and be able to join the group to fulfill one of those roles.

This is probably the grant I’m most excited about in this payout report. Nick has a solid ML background, having previously completed an ML PhD at Berkeley under Stuart Russell. He also has great advisors in Critch and Russell, and is working on an excitingly novel project.

In evaluating new benchmarks, one of the main things I look for is whether they are likely to be hard for an interesting reason, such that I expect solving them to yield useful alignment insights. I think that is very much true for what Nick is trying to build here—fitting in with a group is an ill-defined task that’s likely to require learning from feedback in a complex way that I expect will shine light on real, difficult problems in AI alignment.