individual · Twitter · Manifund
Technical AI safety researchAI governance, policy, and advocacy
| Foresight Institute | $150,000 | 83% | 1 |
| Marcus Abramovitch | $15,000 | 8% | 1 |
| Evan Hubinger | $15,000 | 8% | 1 |
| Vincent Weisser | $150 | 0% | 1 |
| Adrian Regenfuss | $110 | 0% | 1 |
Marcus Abramovitch · Manifund · full post · Dec 28, 2023
Activation vector steering with BCI, Jul 2023
Read more ▾Show less ▴Main points in favor of this grant
When I talked with Lisa, she was clearly able to articulate why the project is a good idea. Often people struggle to do this
Lisa is smart and talented and wants to be expanding her impact by leading projects. This seems well worth supporting.
Davidad is fairly well-known to be very insightful and proposed the project before seeing the original results.
Reviewers from Nonlinear Network gave great feedback on funding Lisa for two projects she proposed. She was most excited about this one and, with rare exceptions, when a person has two potential projects, they should do the one they are most excited about.
I think we need to get more tools in our arsenal for attacking alignment. With some early promising results, it seems very good to build out activation vector steering.
Donor's main reservations
I don’t feel I’m a good judge of whether or not this is worth doing. I think I judge talent well, but I don't have nearly enough alignment background or neurotech background to judge this. This is far more of a bet on the people than on the project. I also don't think many people would be qualified to judge the project.
It's expensive.
I somewhat worry that Lisa won't be full-time on the project and/or that this might distract her from her other work. She did say she had broad support from her current workplace to pursue this in tandem.
Process for deciding amount
The project is in discussion with Foresight to see if it's possible to do a scaled-down version that isn't as expensive. My $15k should go towards getting the ball rolling with the expectation of a few more people to get this at least to the scaled-down stage but preferably the full proposed project.
Conflicts of interest
None
---
Activation vector steering with BCI, Dec 2023
I reached out to Lisa for a progress update on this:
-There really is an important minimum funding to make this project viable that hasn't been achieved yet and so they haven't started. Money is sitting in Lisa's account.
-She will talk with Davidad soon and decide if they are going to make a push for more funding, pivot to something else that could be lower cost or return the funds (expecting by end of March)
I like the honesty here and hope we can get Lisa funded to do a large project (whether this one of another one on areas she is interested in). I'm very "bullish" on Lisa still. I think she's someone that can/will just make something happen that should happen without needing too much permission. I think she also has a rather unique and needed blend of technical understanding + ability to do all the little organizing things that are needed for something to get off the ground.
| Date ↓ | Funder | Via | Amount | Cause | Source | Purpose |
|---|---|---|---|---|---|---|
| May 12, 2026 | Foresight Institute | ~$150,000* | Security, Technical governance | foresight | Security Level 5 Task Force We write standards, create PoCs and advise AI labs to increase the probability that frontier AI labs are protected against nation state attacks by 2028. This is critical to avoid unintended proliferation of unsafe models to malicious state and non-state actors, and to contain LoC risks to buy time to develop technical alignment solutions. — AI for Science & Safety Nodes — 2026 cohort | |
| Apr 16, 2024 | Adrian Regenfuss | Manifund | $110 | Interpretability | manifund | Activation vector steering with BCI |
| Sep 19, 2023 | Vincent Weisser | Manifund | $150 | Interpretability | manifund | Activation vector steering with BCI |
| Jul 31, 2023 | Marcus Abramovitch | Manifund | $15,000 | Interpretability | manifund | Activation vector steering with BCI |
| Jul 31, 2023 | Evan Hubinger | Manifund | $15,000 | Interpretability | manifund | Activation vector steering with BCI |
* Estimated typical grant size; Foresight Institute did not report grant-level amounts.