individual · Manifund
Technical AI safety research
| Long-Term Future Fund | $40,000 | 87% | 1 |
| Neel Nanda | $6,000 | 13% | 1 |
Neel Nanda · Manifund · full post · May 8, 2025
Hallucination Detector, May 2025
Read more ▾Show less ▴I've supervised Javier & Oscar for a paper before, and they've done great work. I think this project seems promising - to my knowledge there's been little work on long-form hallucination detection, I expect there's low hanging fruit here, and the results seem decent so far. This is a fairly small grant, and will help unblock the project, so feels like a no-brainer to me
I think that detecting hallucinations in long-form content is a good downstream task to improve monitoring methods on (and find insights that translate to higher stakes tasks like control or deception), in addition to being near-term useful for making models safer. Improving factuality is likely to also help with making systems more profitable, which is slightly accelerationist, but I am generally pro work that legibly demonstrates how safety is incentivised by profit-seeking.
CoI Note: I am a (very hands off) supervisor for this project and will likely be a co-author on the paper resulting from this. But my quality bar for spending time on a project is notably higher than my bar for regrants, so I don't feel too concerned
| Date ↓ | Funder | Via | Amount | Cause | Source | Purpose |
|---|---|---|---|---|---|---|
| May 16, 2025 | Neel Nanda | Manifund | $6,000 | Interpretability | manifund | Hallucination Detector |
| Jan 2024 | Long-Term Future Fund | EA Funds | $40,000 | Interpretability | ea_funds | 4-month stipend to research the mechanisms of refusal in chat LLMs |