www.alignmentforum.org/users/rubi-j-hudson · individual · Manifund
Technical AI safety researchOther
| UK AI Security Institute | $589,500 | 87% | 1 |
| Keri Warr | $40,000 | 6% | 1 |
| Evan Hubinger | $33,000 | 5% | 1 |
| Ryan Kidd | $10,000 | 1% | 1 |
| raber3 | $1,450 | 0% | 1 |
| gnutella | $1,000 | 0% | 1 |
| Vincent Weisser | $200 | 0% | 1 |
| Carolina Oliveira | $150 | 0% | 1 |
Evan Hubinger · Manifund · full post · Sep 6, 2023
Avoiding Incentives for Performative Prediction in AI, Sep 2023
Read more ▾Show less ▴**Main points in favor of this grant**
I am excited about more work along the lines of the existing "Incentivizing honest performative predictions with proper scoring rules" paper. I think that there are serious safety problems surrounding predictors that select their predictions to influence the world in such a way as to make those predictions true ("self-fulfilling prophecies") and I am excited about this work as a way to discover mechanisms for dealing with those sorts of problems. "Conditioning Predictive Models" discusses these sorts of issues in more detail. Rubi is a great person to work on this as he was an author on both of those papers.
**Donor's main reservations**
I think my main reservations here are just around Rubi's opportunity costs, though I think this is reasonably exciting work and I trust Rubi to make a good judgement about what he should be spending his time working on. The most likely failure mode here would probably be that the additional work here doesn't turn up anything else new or interesting that wasn't already surfaced in the "Incentivizing honest performative predictions with proper scoring rules" paper.
**Process for deciding amount**
I think that $33k is a reasonable amount given the timeframe and work.
**Conflicts of interest**
Rubi was a previous mentee of mine in SERI MATS and a coauthor of mine on "Conditioning Predictive Models."
| Date ↓ | Funder | Via | Amount | Cause | Source | Purpose |
|---|---|---|---|---|---|---|
| Nov 7, 2026 | gnutella | Manifund | $1,000 | Other | manifund | Principled Agents |
| Oct 21, 2026 | Smitty | Manifund | $100 | Other | manifund | Principled Agents |
| Oct 20, 2026 | David Williams-King | Manifund | $50 | Other | manifund | Principled Agents |
| Oct 20, 2026 | Ryan Kidd | Manifund | $10,000 | Other | manifund | Principled Agents |
| Oct 20, 2026 | Carolina Oliveira | Manifund | $150 | Other | manifund | Principled Agents |
| Oct 20, 2026 | raber3 | Manifund | $1,450 | Other | manifund | Principled Agents |
| Oct 20, 2026 | Keri Warr | Manifund | $40,000 | Other | manifund | Principled Agents |
| Feb 19, 2026 | UK AI Security Institute | Principles of Intelligence | ~$589,500* | Technical AI safety research | uk_aisi | Constructing robust incentivizes for AI agents to act cautiously — led by Rubi Hudson. — Alignment Project |
| Oct 1, 2023 | Vincent Weisser | Manifund | $200 | Alignment methods | manifund | Avoiding Incentives for Performative Prediction in AI |
| Sep 11, 2023 | Evan Hubinger | Manifund | $33,000 | Alignment methods | manifund | Avoiding Incentives for Performative Prediction in AI |
* UK AISI reported a total of 27m GBP over 60 projects, so estimating each of them at the average grant size.