Trace

Iván Arcuschin Moreno

individual · LinkedIn · Manifund

Technical AI safety researchAI safety

Received
$79,000
from 3 funders
Grants
3
Funders
3
Years active
2024–26

Grants over time

$0$25K$50K$75K$100K202420252026
Long-Term Future FundNeel NandaBlueDot Impact

Biggest funders

Long-Term Future Fund$67,00085%1
Neel Nanda$11,00014%1
BlueDot Impact$1,0001%1

Reviews

Neel Nanda · Manifund · full post · Nov 19, 2024

Mechanistic Interpretability research for unfaithful chain-of-thought (1 month), Nov 2024

Read more ▾

I'm fully funding this. I think that understanding, detecting and potentially mitigating chain of thought unfaithfulness is a very important problem, especially with the rise of o1 models. I think the approach taken here is reasonable. I think Arthur is fairly good at supervising projects, and that under him Jett and Ivan have a decent shot of making progress, and that enabling this to start a month earlier is clearly a good idea. I know Arthur well, and got a moderate amount of data on Jett and Ivan's ability to execute on this project in my MATS training program.

Conflicts of interest: Jett and Ivan were in my MATS training program (but no longer are, and I would not be involved in this project going forwards). I manage Arthur and he works on my interpretability team at GDM (but he is doing MATS in a purely personal capacity, and if anything I'm incentivised to *reduce *his amount of extra-curricular distractions :P ). Overall I don't feel particularly concerned about conflicts of interest here.

Grants received

Date ↓FunderViaAmountCauseSourcePurpose
Jun 2026BlueDot Impact$1,000AI safetybluedotDirecting an MSc thesis on how unverbalized biases enter LLM's chain-of-thoughts: finding the 'bias anchor' sentences and the attention path from the model's input to its output.
Nov 19, 2024Neel NandaManifund$11,000InterpretabilitymanifundMechanistic Interpretability research for unfaithful chain-of-thought (1 month)
Jan 2024Long-Term Future FundEA Funds$67,000Interpretabilityea_funds4-month stipend for MATS extension on mechanistic interpretability benchmark + 2-month stipend for career switch