Amon Elders

Doing a PhD in computer science with a focus on AI safety.

Amon applied for funding to support a PhD position in artificial intelligence working with Prof. Michael Osborne. Receiving funding was important to securing an offer, since there is limited PhD funding available for non-UK citizens in the UK after Brexit. Moreover, the funding sources that are available have application deadlines earlier in the year, and so waiting for these would have delayed the start of the PhD by a year.

Amon graduated from UCL with an MS in CS & ML (with distinction). His research experience includes a year-long RA position at the Italian Institute of Technology in Genoa with his UCL supervisors, which resulted in a third-author publication in AIES, and an MS Thesis on learning to rank. Amon also has relevant professional experience, having worked as an ML engineer at Spark Wave, and as an undergraduate co-founded the EA society at the University of Amsterdam.

Amon is particularly interested in working on robustness and distributional shift in his PhD, though he is also considering other topics. While his research direction is currently tentative, he is familiar with the key aspects of long-term AI safety and is interested in pursuing this direction further both during and after his PhD.

Michael Osborne seems like a good advisor given Amon's interests. Michael has a deep background in Bayesian ML which is relevant for robustness and distributional shift, and Michael also has an interest in the social impact of machine learning. Additionally, Michael Osborne is also advising Michael Cohen, a PhD scholar at FHI. While Michael Cohen is pursuing a different concrete research agenda to Amon's current interests, they have a similar high-level focus, so I expect being in the same group will help lead to fruitful discussions.

I generally think there's a strong case for funding talented people to pursue PhDs on relevant topics. For one, this is a great way of training relevant researchers: I think PhDs generally do a good job of teaching relevant research skills and are a valuable credential. In addition, there's often scope to produce directly valuable research during a PhD, although the opportunity here varies between labs.

A big reason why I'm often hesitant to fund PhDs is adverse selection. There are many other funding sources for PhDs: government fellowships, university bursaries, private endowments, advisors’ research grants, etc. Most of the people who we would want to fund can easily get funding from elsewhere. So the applications we see will be disproportionately those whom other funders have chosen to reject. However, Amon was simply ineligible for the majority of those funding sources, so this consideration is much weaker in his case. He ended up getting 5 PhD offers including offers from Oxford, Cambridge, UCL, and Imperial.

The main concern I have with this grant is that Amon may end up working on topics that I do not believe will have an impact on the long-term future. This could be fine if he later pivots to focus more on AI safety post-PhD, but research direction often can be "sticky" (see similar discussion in the write-up for

Sören Mindermann below). However, Amon does seem to currently be on a promising path, and he’s currently keeping up with AI safety research at other labs.