Johannes Heidecke

Supporting aspiring researchers of AI alignment to boost themselves into productivity

From the application:

(1) We would like to apply for a grant to fund an upcoming camp in Madrid that we are organizing. The camp consists of several weeks of online collaboration on concrete research questions, culminating in a 9-day intensive in-person research camp. Participants will work in groups on tightly-defined research projects in strategy and technical AI safety. Expert advisors from AI Safety/Strategy organizations will help refine proposals to be tractable and relevant. This allows for time-efficient use of advisors’ knowledge and research experience, and ensures that research is well-aligned with current priorities. More information: [https://aisafetycamp.com/](https://aisafetycamp.com/)

(2) The field of AI alignment is talent-constrained, and while there is a significant number of young aspiring researchers who consider focussing their career on research on this topic, it is often very difficult for them to take the first steps and become productive with concrete and relevant projects. This is partially due to established researchers being time-constrained and not having time to supervise a large number of students. The goals of AISC are to help a relatively large number of high-talent people to take their first concrete steps in research on AI safety, connect them to collaborate, and efficiently use the capacities of experienced researchers to guide them on their path.

(3) We send out evaluation questionnaires directly after the camp and in regular intervals after the camp has passed. We measure impact on career decisions and collaborations and keep track of concrete output produced by the teams, such as blog posts or published articles.

We have successfully organized two camps before and are in the preparation phase for the third camp taking place in April 2019 near Madrid. I was the main organizer for the second camp and am advising the core team of the current camp, as well as organizing funding.

An overview of previous research projects from the first 2 camps can be found here:

[https://aisafetycamp.com/2018/06/05/aisc-1-research-summaries](https://aisafetycamp.com/2018/06/05/aisc-1-research-summaries)/

[https://aisafetycamp.com/2018/12/07/aisc2-research-summaries/](https://aisafetycamp.com/2018/12/07/aisc2-research-summaries/)

We have evaluated the feedback from participants of the first two camps in the following two documents:

[https://docs.google.com/document/d/1f8wvsvQTv4wdBaggCaK8aKC5gFdIHUDcihnmVkZPM6I/edit?usp=sharing](https://docs.google.com/document/d/1f8wvsvQTv4wdBaggCaK8aKC5gFdIHUDcihnmVkZPM6I/edit?usp=sharing)

[https://docs.google.com/document/d/18v2e-S3iZrOPbE7d9n26sUs1K6CkUAvezRvRj_xlcj8/edit?usp=sharing](https://docs.google.com/document/d/18v2e-S3iZrOPbE7d9n26sUs1K6CkUAvezRvRj_xlcj8/edit?usp=sharing)

My thoughts and reasoning

I’ve talked with various participants of past AI Safety camps and heard broadly good things across the board. I also generally have a positive impression of the people involved, though I don’t know any of the organizers very well.

The material and testimonials that I’ve seen so far suggest that the camp successfully points participants towards a technical approach to AI Alignment, focusing on rigorous reasoning and clear explanations, which seems good to me.

I am not really sure whether I’ve observed significant positive outcomes of camps in past years, though this might just be because I am less connected to the European community these days.

I also have a sense that there is a lack of opportunities for people in Europe to productively work on AI Alignment related problems, and so I am particularly interested in investing in infrastructure and events there. This does however make this a higher-risk grant, since I think this means this event and the people surrounding it might become the main location for AI Alignment in Europe, and if the quality of the event and the people surrounding it isn’t high enough, this might cause long-term problems for the AI Alignment community in Europe.

Concerns

• I think organizing long in-person events is hard, and conflict can easily have outsized negative effects. The reviews that I read from past years suggest that interpersonal conflict negatively affected many participants. Learning how to deal with conflict like this is difficult. The organizers seem to have considered this and thought a lot about it, but the most likely way I expect this grant to have large negative consequences is still if there is some kind of conflict at the camp that results in more serious problems.
• I think it’s inevitable that some people won’t get along with organizers or other participants at the camp for cultural reasons. If that happens, I think it’s important for these people to have some other way of getting connected to people working on AI Alignment. I don’t know the best way to arrange this, but I would want the organizers to think about ways to achieve it.

I also coordinated with Nicole Ross from CEA’s EA Grants project, who had considered also making a grant to the camp. We decided it would be better for the LTF Fund team to make this grant, though we wanted to make sure that some of the concerns Nicole had with this grant were summarized in our announcement:

• AISC could potentially turn away people who would be very good for AI Safety or EA, if those people have negative interactions at the camp or if they are much more talented than other participants (and therefore develop a low opinion of AI Safety and/or EA).
• Some negative interactions with people at the camp could, as with all residential programs, lead to harm and/or PR issues, (for example, if someone at the camp were sexually harassed). Being able to handle such issues thoughtfully and carefully is a hard task, and additional support or advice may be beneficial.

This seems to roughly mirror my concerns above.

I would want to engage with the organizers a fair bit more before recommending a renewal of this grant, but I am happy about the project as a space for Europeans to get engaged with alignment ideas and work on them for a week together with other technical and engaged people.

Broadly, the effects of the camp seem very likely to be positive, while the (financial) cost of the camp seems small compared to the expected size of the impact. This makes me relatively confident that this grant is a good bet.