Marc-Everin Carauleanu

Writing a paper/blogpost on cognitive and evolutionary insights for AI alignment.

Marc’s project is to attempt to understand the evolutionary development of psychological altruism in humans—i.e. the extent to which people intrinsically value others—and understand what sorts of evolutionary pressures led to such a development.

Marc was pretty unknown to all of us when he applied and didn’t seem to have much of a prior track record of AI safety research. Thus, this grant is somewhat speculative. That being said, we decided to fund Marc for a number of reasons.

First, I think Marc’s proposed project is very inside-view exciting, and demonstrates a good sense of research taste that I think is likely to be indicative of Marc potentially being a good researcher. Specifically, evolution is the only real example we have of a non-human-level optimization process producing a human-level optimizer, which I think makes it very important to learn about. Furthermore, understanding the forces that led to the development of altruism in particular is something that is likely to be very relevant if we want to figure out how to make alignment work in a multi-agent safety setting.

Second, after talking with Marc, and having had some experience with Bogdan-Ionut Cirstea, with whom Marc will be working, it seemed to me like both of them were very longtermism-focused, smart, and at least worth giving the chance to try doing independent AI safety research.

Third, the small amount of money requested for this grant meant that our bar for funding was lower than for other similar but larger grants. This was not a very large factor in my thinking, however, as I don’t believe that the AI safety space is overall very funding-constrained—such that if we can find good opportunities for funding, it’s likely we’ll be able to raise the necessary money.