kunvarthaman.com · individual · Manifund
Technical AI safety research
| Marcus Abramovitch | $8,500 | 77% | 1 |
| Long-Term Future Fund | $2,500 | 23% | 1 |
| Ambreen Deol | $8,500 | 87% | 1 |
| Michaël Rubens Trazzi | $500 | 5% | 1 |
| shivam dubey | $400 | 4% | 1 |
| Joss Oliver | $300 | 3% | 1 |
| Zhonghao He | $100 | 1% | 1 |
Marcus Abramovitch · Manifund · full post · Dec 10, 2023
Exploring feature interactions in transformer LLMs through sparse autoencoders, Dec 2023
Read more ▾Show less ▴About time I wrote up my reasoning for this grant:
Main points in favor of this grant
Kunvar has set out an ambitious research agenda for himself which would lead to some quite exciting papers. This donation also will get a lot of Mech Interp per dollar spent if successful (like 10x more per dollar than most other stuff I come across given the little compute and low overhead)
The main reason for funding this though is that I know Kunvar is very smart due to many interactions I have with him and I doubt this will be as apparent to other people so I am pretty uniquely able to make this grant.
**Donor's main reservations**
Independent alignment work without any mentorship doesn't have a fantastic track record. That said, I am expecting this to launch Kunvar into something like MATS or another program to get him some mentorship on this and he has mentioned collaborating on these projects with others.
**Process for deciding amount**
This was the amount needed to cover the compute and the minimum salary (which was super low). If more were needed, I could make that happen but this just gets the project off the ground.
**Conflicts of interest**
None.
| Date ↓ | Funder | Via | Amount | Cause | Source | Purpose |
|---|---|---|---|---|---|---|
| Apr 2024 | Long-Term Future Fund | EA Funds | $2,500 | Evals | ea_funds | Seeking funds to present “Benchmark Inflation” at ICML 2024, my paper on making AI progress measures more accurate |
| Dec 7, 2023 | Marcus Abramovitch | Manifund | $8,500 | Interpretability | manifund | Exploring feature interactions in transformer LLMs through sparse autoencoders |
| Date ↓ | Recipient | Via | Amount | Cause | Source | Purpose |
|---|---|---|---|---|---|---|
| Jul 15, 2026 | shivam dubey | Manifund | $400 | Security | manifund | Finding and Defending the Attention Circuits That Make LLMs Jailbreakable |
| Sep 22, 2024 | Joss Oliver | Manifund | $300 | Foundations | manifund | Travel funding to the International Conference on Algorithmic Decision Theory. |
| Mar 12, 2024 | Ambreen Deol | Manifund | $8,500 | Other | manifund | Investigating the Effects of IF in the reversal of Type 2 Diabetes Mellitus. |
| Feb 23, 2024 | Michaël Rubens Trazzi | Manifund | $500 | Advocacy | manifund | Making 52 AI Alignment Video Explainers and Podcasts |
| Dec 11, 2023 | Zhonghao He | Manifund | $100 | Interpretability | manifund | Mapping neuroscience and mechanistic interpretability |