Holly Elmore

I made Holly an offer of $2.5k yesterday. This won't necessarily be the last offer I make to this project; I'm giving a smaller amount in order to get money out the door more quickly and to provide a credible signal to other regrantors.

Here's my reasoning about this grant as of the offer:

1) Overall, I feel excited about this proposal on the meta level.

• As per my profile, I want to make offers to "grants that might otherwise fall through the gaps. (E.g. [...] politically challenging for other funders [...].)" My impression is that this proposal is a great example of that.
• Also as per my profile, I want to "[o]ccasionally make small regrants as credible signals of interest rather than endorsements. (To improve [...] funder-chicken dynamics.)" I think that this application has some funder-chicken going on -- regrantors seem to be positive, but it's a slightly nervey proposal to give to. (Maybe that's for good reason -- there's more downside potential here -- but I don't feel like that's what's driving concerns, since I (and others) might expect any significant downside to come after the period in which Holly tests her fit.)
• I often see Holly (online) say interesting and reasonable things that go against her crowd. (E.g. XYZ.) I'm interested in what she'll have to say about AI.
• I have the weak (positive) impression that Harvard EA became noticeably less well-organized after Holly left, and an even weaker (positive) impression that this was due to Holly being a great organizer (rather than pandemic, or being Holly's fault for not enabling a strong successor, or something else).
• Unfortunately, unlike for most other grants I will make, I don't think I'll be able to get helpful information from my network about the attractiveness of the grant.
• I am uncomfortable with how in-group this grant feels. Holly and I have only briefly overlapped professionally; there's definitely no COI. But "fund an OEB PhD to advocate for AI moratorium" feels like something I'd be more dismissive of if I didn't know that Holly was so deep into our shared memesphere. In some ways this situation feels appropriate on the substance -- sharing ideas and professional connections with Holly gives me positive context. In other ways it feels like my judgement might be compromised. (See my last object-level point below.)

2) I feel confused about how I should value this proposal on the object level.

• I feel somewhat uncomfortable with the role that Conjecture plays in AI discourse. The main thing I'm concerned about is something like "important decision-makers get turned off after hearing unpersuasive and possibly-also-misleading messaging." Some of this is substance, some of this is style. Advocating for a moratorium sounds like shared substance. This makes me nervous about Holly's advocacy.
• On the other hand, the bull case for Conjecture's advocacy is something like "Overton window-shifting." I am skeptical of this case, but people I trust are more excited about it. I think Holly is better-placed to take advantage of this case in some ways (e.g. more relatable and careful speaker) and worse-placed in other ways (e.g. doesn't have the ML credibility that Connor has through Eleuther).
• Related to my final meta-level concern: I have the sense that the LessWrong memesphere is becoming less and less relevant to how AI goes, and so am decreasingly excited in proposals that seem to speak to or on behalf of that memesphere.

Overall:

• I've thought about this grant for too long without making much progress.
• Seems like the most productive thing to do is to provide early support to Holly and a credible signal to other regrantors.
• Holly and/or other grantmakers might be more likely to provide me with information that helps me think about whether to give more.

---

Holly Elmore organizing people for a frontier AI moratorium, Aug 2023

@Holly_Elmore Thanks for picking this up; I totally agree that you're not reflecting some "LessWrong perspective." My previous wording was clumsy. My thinking might be too. I'll have another try at pointing at this cluster of concerns.

- I'm having a conversation with a senior US policymaker or journalist (maybe not focused on AI) 1 year from now. I say to them: "Holly did X work that attempted to encourage Y person/organization to do Z." Then they say: "who is Y?" Unfortunately, actor Y is not very influential. - An organizer with more experience in politics/media or without the shared presentation/ideas might have picked up on this and better targeted their efforts. - Perhaps Holly has targeted Y for bad taste reasons (Y is more sexy, or better-known to LW types) or for good efficiency reasons (Holly has second-hand connections at Y, perhaps borrowed from shared-memesphere-connections, which made it easier to engage with Y than an actor with similar influence and lesser connections). - A senior journalist engages with Holly 3 months from now. Associating the idea of a moratorium, or even Holly in particular, as coming from a particular memesphere, the journalist takes her less seriously, or seeks to contrast her more strongly with other voices, or comes to see her view as representative of the memesphere in ways that are unhelpful for others' lobbying, or [other bad outcome]. - After writing, I'm not sure this concern makes sense. The reasoning seems pretty tortured, and the outcomes are not that bad anyway. - The better version of this might look more like my Conjecture comment in the next paragraph: people who Holly engages with might persistently take AIS concerns less seriously if pushing a message that they think is crazy.

Regarding Conjecture, I have the sense (I think second-hand from public Matt Clifford communications and a bunch of conversations with [redacted prominent AI safety researcher]) that policymakers get turned off very quickly when they hear the message that everyone's going to die. My big concern is that this turning off persists over time.

I agree that "the question of whether to regulate/slow/stop AI transcends many of the technical details" on the substance. I guess I have a sense that, at least pre-Hinton leaving, AI safety worries felt to many people like they came from amateurs who had misunderstood the nature of the technology, which is part of what turned them off from these concerns. Maybe I think this has an unhelpful interaction with arguing for a window-shifting policy.

After writing out the above, I think I see that my concerns -- even if they do make sense for conversations with policymakers and media elites -- might not port over to political organizing. I could probably get clearer on this if I had a clearer sense of the political organizing activities you might pursue. Could you describe these in more detail?