Peli Grietzer

Peli is a frugal philosopher who cowrote one of my favourite essays on alignment.

The idea: He wants to invent virtue post-training, inspired by e.g. the actual normative practice of fields like maths, which often appeals to self-instantiation. It's an open question whether current capabilities allow for instilling stable loops like this and I'm glad someone is trying.

Counterfactual: The current project struck out from the usual funders, I guess because theory is de-emphasised now, or because virtue ethics is usually unoperationalisable, or because he's not good at sales, or because they don't know the following about him.

Track record: Besides the (great) above essay: I've worked with Peli on two technical ML research papers and was impressed with his experimental skill, design skill, and precision. He also did some invited technical replication work on Turner 2023. He's competent at ML experiments at the level of a decent PhD student in the field. He is very used to working totally independently from inception to .

Concerns: If I didn't know the above, the doc would worry me in how totally non-ML-technical it is (it's intentionally doing some original philosophical work as prerequisite for that part). He's also been looking for collaborators and not finding much success; I hope that giving him independent funding makes this search an easier pitch.

I expect distribution to be the weakest part of the project. The inferential distance might be too high for the narrower part of the ingroup audience to bridge to him, even with him presenting them good data. But places like PAW, AIES and HAAISS will certainly engage. He's capable of doing the conference paper grind but doesn't seem motivated by it. Maybe a collaborator could bring the will to actively disseminate the results. But if distribution is the worst risk for a speculative ambitious project then we're in a good spot.

Cost-effectiveness: Very high, $40k / FTE for taking an intriguing idea and bringing it to testability.

Conflict of interest: As noted, Peli has worked on several projects at my company Arb and I've known him for years on Twitter.