joel-becker.com · individual · Twitter · LinkedIn · Manifund · LessWrong · EA Forum
| Other | $149,980 | 37% | 1 |
| AI safety meta | $82,200 | 20% | 3 |
| AI safety fieldbuilding | $70,000 | 17% | 2 |
| Technical AI safety research | $60,250 | 15% | 3 |
| AI governance, policy, and advocacy | $21,550 | 5% | 3 |
| Other existential risk | $10,000 | 2% | 1 |
| AI safety adjacent | $7,200 | 2% | 1 |
| Lyptus Research | $149,980 | 37% | 1 |
| Sean Peters | $73,000 | 18% | 2 |
| Sage | $71,700 | 18% | 1 |
| Parker Whitfill | $38,000 | 9% | 1 |
| ChinaTalk | $17,550 | 4% | 1 |
| Theia Vogel | $17,500 | 4% | 1 |
| Good Ancestors Policy | $10,000 | 2% | 1 |
| ALERT | $8,000 | 2% | 1 |
arenal · Manifund · full post · Dec 1, 2023
Estimating annual burden of airborne disease (last mile to MVP), Oct 2023
Read more ▾Show less ▴Props to Austin for filling the minimum funding before I had a chance to ask any questions, haha. But for the sake of future/>min funding and general clarity:
Initial thoughts:
Questions:
- I'm sort of surprised that this needed to be pursued through independent funding--this is such a think-tank-friendly project! Richard works at the Center for Health Security and even produced an analysis in collaboration with Institute for Progress that seems super related to this! What's the missing expertise that Qally's brings to the table, and why isn't CHS backing this piece of the project? - The think-tank-friendliness of it reduces the failure risk from the team not having time to write up results--possibly could rope in the dedicated policy advocates at IfP or Rethink for advocacy-style writing? - Are you also considering long-term costs of other viruses? Just curious--I know this is much more speculative, but my impression is that COVID just had the most attention on it for a while and now there's increasing scientific interest in possible long-term effects of flu. That would also emphasize the importance of endemic disease. But could be way too complicated, just wondering if it was discussed! - You reference researching other contributions to disease burden as a possible extension of this project--what contributions would those be, if the bulk of the project has already been completed?
---
Estimating annual burden of airborne disease (last mile to MVP), Dec 2023
Offering a grant at this point after Vivian reached out to me and told me about a couple of customers for the estimate that are excited to have this sooner rather than later.
None of the possible conflicts of interest really gave me reservations but for the sake of disclosure: Joel and I are friends, and (although Vivian and Richard aren't the direct recipients), Vivian and I collaborate closely, and I am likely to begin working with Richard soon on a related project. As mentioned in my comment below, this estimate is, in fact, something that I might wind up using.
Joel Becker · Manifund · full post · Oct 31, 2023
Estimating annual burden of airborne disease (last mile to MVP), Oct 2023
Read more ▾Show less ▴Thanks for the questions @NunoSempere.
Do you already have some decision-makers who could use these estimates to make different decisions?
Waiting on some emails that will help me answer this; will get back to you soon.
How valuable do you think that this project is without the long covid estimate?
I just put this question to Vivian, who says: "I think without the long covid estimate it probably just never gets published."
Who is actually doing this work? Vivian and Richard, or Joel and Aron?
Joel and Aron. (If "this work" refers to building the long COVID estimate.)
Why are you doing stuff $3.6k a time, rather than having set up some larger project with existing biosecurity grantmakers?
Very possible that I've made a silly decision somewhere, but here's how it looks from my side:
- Earlier this year I applied to LTFF and Lightspeed for ~general funding for Qally's. These applications were rejected. - I think rejection was the right call -- even if you think higher-effort quantitative analyses are worthwhile, it's preferable for them to come with demand signals rather than subsidizing my discretionary offerings. - I didn't apply to ~any other projects nor sources of funding because of my US work visa stuff. - Mostly this is 'getting a US work visa attached to your own new organization requires a lot of time.' - To a lesser extent this is 'the expected benefit of finding new projects is lower than usual, because I would only proceed with these projects if I was approved for the visa.' - My visa petition was approved ~4 weeks ago; I have started to seriously look for projects in that time. One project that came up was this one. It currently doesn't have a source of funding, so I put up a Manifund proposal. - All that said, I would, of course, like to work >$3.6k at a time. A few things on this: - I hope that this application will attract more than the minimum! - It's plausible that this will be a larger project. There are a bunch of other things to work on for this project, which I currently have cached as 'things Vivian or Richard will do' or 'things that aren't necessary for MVP.' - I almost feel like I don't know other existing biosecurity grantmakers...! Linch has moved away from this sort of thing. Open Philanthropy staff are in the business of larger amounts of money. My uninvestigated understanding from this proposal is that Convergent Research (where Vivian has recently worked) aren't able to fund adjacent projects. So I posted on Manifund and notified @GavrielK. (I'm apparently allowed to fund myself, but don't want to.) - There is another project that I might take on, which I expect would be ~full-time for many months.
---
Estimating annual burden of airborne disease (last mile to MVP), Oct 2023
Thank you for the comment @GavrielK!
On one of your thoughts:
I've personally received conflicting opinions from people in the field about how much policymakers care about/pay attention to in-depth analyses that they didn't actually commission. I like that this analysis format will result in a solid headline number that can be communicated simply.
I think the skepticism from people in the field is warranted. And I agree it's attractive that this project will result in a headline number -- this is definitely what Vivian and Richard have in mind. But I think there's a related risk of the project here: because the (at least MVP) analysis will be less in-depth, the headline number will be easier to undermine.
On your questions:
I'm sort of surprised that this needed to be pursued through independent funding [...] What's the missing expertise that Qally's brings to the table, and why isn't CHS backing this piece of the project?
I think the missing expertise is: having the time to do the work! My understanding is that the project hasn't moved for a few months for this reason. That said, since posting this project on Manifund, I'm less confident that Vivian would not contribute (further than they already have) to the nitty gritty of the long COVID estimates; I guess this is mostly upside from your perspective.
As for why CHS isn't backing this project, I think there's a mix of (1) limited JHCHS staff capacity, and (2) part of the original motivation of this project being Vivian receiving mentorship from Richard. (I am happy to explore the possibility of funding from JHCHS at some point.)
The think-tank-friendliness of it reduces the failure risk from the team not having time to write up results--possibly could rope in the dedicated policy advocates at IfP or Rethink for advocacy-style writing?
Could you possibly expand on what roping in dedicated policy advocates would involve? Asking them to produce a document that reads like a policy report, given our less friendly write-up? And perhaps asking them to share this document with contacts in government and media?
Are you also considering long-term costs of other viruses? Just curious--I know this is much more speculative, but my impression is that COVID just had the most attention on it for a while and now there's increasing scientific interest in possible long-term effects of flu. That would also emphasize the importance of endemic disease. But could be way too complicated, just wondering if it was discussed!
Aron and I are not, at least not right now. But Vivian and Richard have so far considered: common cold, influenza, and TB.
You reference researching other contributions to disease burden as a possible extension of this project--what contributions would those be, if the bulk of the project has already been completed?
The other components that Vivian has marked as needing to be revisited are: mainline estimates of direct medical costs of common cold and of influenza, lost productivity from influenza, and learning loss due to common cold and influenza. They already have low and high estimates for each of these, but I think hasn't thought about which of these estimates is more appropriate (which in turn would affect their averaging for the mainline estimate).
Ozzie Gooen · Long-Term Future Fund · full post · May 27, 2021
$42,000
Read more ▾Show less ▴Conducting research into forecasting and pandemics during an economics Ph.D.
Joel Becker is a first-year Ph.D. student at NYU who is pursuing forecasting research. The grant will act as a replacement salary for teaching for three years, to allow Joel to pursue research instead.
He intends to work on a few projects, including (text taken from the application):
I think Joel could plausibly become a successful academic: He’s been part of the Philanthropy Advisory Fellowship at Harvard, and will soon join the Global Priorities Fellowship of the Forethought Foundation. I get the impression from this that he’s fairly committed to improving the long-term future.
Joel seemed to me to currently be working on somewhat standard academic forecasting topics. This has the benefits of having accessible advisors and an easier time integrating with other work. But it has the downside that the current academic forecasting topics are likely different than would be ideal for longtermists. My intuition is that much of the most valuable work for generic forecasting will be a combination of theoretical foundations and advanced software implementations, and the existing economics work is fairly separate from this. For example, much of the literature explores laboratory experimentation with what seem to me to be unrealistic and inhibited scenarios.
I’ve read some of Joel’s in-progress work, and he seems to get some things right (asking better questions than other academic work, and using more modern software), but this work is very early-stage.
The big question is what direction Joel goes after doing this research. I could see the research helping to build valuable credentials and skills. This could put him in a position to spearhead bold and/or longtermism-centric projects later on.
Lyptus Research · Manifund · full post · Apr 13, 2026
Lyptus Research: Bridge Funding, Apr 2026
Read more ▾Show less ▴Here's my updated sense of where Sean and co. are at, extremely heavily influenced by colleagues, especially Nate Rush:
Coming out of this:
I'm delighted to recommend funding Lyptus' maximum here; I hope they can shift work to more ambitious ends that I think would be more helpful, and I hope they ask for more funding now.
---
Lyptus Research: Bridge Funding, Apr 2026
(Noting that after conversation with Nate it's now less obvious to me that Lyptus should focus on dramatically harder cyber tasks in particular. Although I continue to think that there is a very, very, very high premium on dramatically harder tasks in general. Want to leave the focus more flexible depending on Lyptus' taste.)
---
Lyptus Research: Bridge Funding, Apr 2026
Recommended an additional $50k to Lyptus on same thesis as the below. I'm still curious about Lyptus absorbing more than this, but I only had another $50k in my Manifund pot! I've reached out to some funders about considering funding Lyptus further. I hope my comment below can serve as some evidence. For now, without more regrantor recommendation budget, I'm bowing out. Wishing Sean et al. the best of luck!!
Mox · Manifund · full post · Mar 9, 2026
Mox: 2026 Fundraiser, Mar 2026
Read more ▾Show less ▴(COI: I currently live with Austin Chen and Rachel Shu.)
I had been planning to donate $20k-40k to Manifund or Mox out of personal donation budget; alas I think this might lead to too significant COI for my own future possible involvements with Manifund or Mox. Instead, I'm laying out a case for funding here!
Mostly my excitement about Mox routes through (1) great event experiences I've had at Mox and (2) the dumb obvious case for there to exist an AI safety hub in San Francisco.
On (1): I've spoken at a Golden Gate Institute event focused on my METR research, co-hosted a hackathon with/for METR, and co-hosted 2 very large music events.
I'm not sure what the counterfactual for the GGI was because I didn't organize it, but presumably GGI would have used some worse event space (in particular much smaller + with less relevant default crowd).
I think there's a ~50% chance the METR hackathon simply wouldn't have happened without Mox. (In the end I think the hackathon ended up not being so helpful, unfortunately, although this is very clearly on us and not Mox.)
The large music events are my favorite thing in the world. We could put them on elsewhere, but I think they'd be ~50% of the current crowd size (and we probably have room to comfortably 1.5x the current crowd at Mox). I don't want to overstate how helpful these events are for building a community around Mox, but my understanding is that attendees who have made donations to support the event or to support Mox for hosting for the event have this as a pretty primary motivation.
On (2), I basically think the claim below which Mox make above is correct.
In a single year on a shoestring budget, we've become a primary nexus for the AI safety community and other EA-adjacent work in San Francisco.
Now, in some ways I think this reflects a sad state of affairs! It seems to me obvious that the AI safety community should have more presence in SF. In particular, I think Constellation should move to SF. If a Constellation office was set up separate from Mox in SF I expect that, by default, it would become the primary nexus for safety work. Then there's some question of what the comparative advantage of each place would be, and so on. But for now it's clearly true that Mox is the only game in town in SF.
I think the dumb case for some sort of AIS space to exist in SF is obvious -- good work often benefits or emerges from in-person connections, in expensive cities it's helpful to have dedicated in-person space to make this happen; SF is the center of AI work. Now, I think there are some obvious things to enable this vision which I don't see Mox doing much of (e.g. happy hours with staff from major AI companies). But I have enough faith in the basic vision + confidence in the founders to be excited to back Mox for their funding goals.
Sage · Manifund · full post · Dec 18, 2025
AI Digest, Jul 2025
Read more ▾Show less ▴Per my own work, I feel very excited about means of evaluating AI systems that are more "in the wild"/not having the constraints of existing benchmark-style tasks (self-contained, algorithmically scorable, etc.). From this perspective "AI village" is a very exciting project; I would like for this donation to be internally earmarked for the AI village (though not legally restricted; I want to leave room for the team's judgement, just to let them know that this is my vote).
(I think I'm personally relatively less excited about AI digest, even though it looks very well executed, basically on neglectedness grounds.)
There's a decent chance I put more of my pot to this project later.
COI: nothing serious; I'm personal friends with Misha Yagudin and Eli Lifland, two of the founders of the organization that AI Digest is a project of.
---
AI Digest, Oct 2025
Donated another $20k for $40k total.
I think it's crazy that we live in a world where AIs are well on their way to saturating ~all challenging technical benchmarks and yet they can't pull off basic experiments. It seems to me that exploring this harder would be very informative for learning about the capability profile of AIs. Right now it's unclear what we should learn from AI village because (1) I don't know how much I should trust quality of the set-up/elicitation, and (2) the effective sample size is tiny.
One blocker for this happening is limited staff time. Another blocker -- totally speculative on my part, but my guess is shared between both other grantmakers and Sage/AI digest staff themselves -- is viewing AI digest as primarily a comms play and not a science play. I think AI village should primarily be seen as a science play (that currently isn't doing a great job at science but has a tonne of potential).
I know @Austin is a bit concerned that this grant opportunity is not Manifund's comparative advantage/can easily be considered by other funders. I disagree -- it seems to me that traditional funders (and perhaps the staff behind AI village themselves) are being far too unambitious/unexcited about this project (on the basis of it being a comms play with limited existing comms traction), and I'm thrilled to put some of my Manifund pot towards nudging this situation in a better direction.
---
AI Digest, Dec 2025
Recommended my remaining regrantor budget, $31.7k, for $71.7k total.
I continue to stand behind my last comment (and hopefully will write more later). I think that something like AI village is one of the most interesting science (not comms!!!) plays in AI today. I'm disappointed that this continues to be a non-consensus pick.
Sean Peters · Manifund · full post · Dec 2, 2025
3 Months Career Transition into AI Safety Research & Software Engineering, Sep 2025
Read more ▾Show less ▴I asked Sean to put up this proposal and have fully funded his request.
The case for this grant is extremely straightforward: my sense (from ~1.5 years at METR) is that of all the independent external people building directly on top of METR work, Sean is the person whose work my colleagues have been most excited about. Sean appears to be smart, agentic, and focused. My sense is that he's working ~1.5 days per week on these projects basically due to funding constraints, which seems crazy; I'm very excited to support him increasing this to 4.
I could say some more specific things (excited for Sean's connections to pen testers, excited for him to contribute to sabotage monitoring projects, excited for his past work measuring time horizons in offensive cybersecurity , etc.) but mainly my decision comes down to my colleagues' rare degree of excitement for whatever Sean has in store next. I'm thrilled to bet on that.
---
Evaluating Model Attack Selection and Offensive Cyber Horizons, Dec 2025
Evaluation science uses deep statistics. For example, in the attack selection methodology I'm working with hierarchical Bayesian modeling, adaptive sampling, MCMC, etc.
Noting that a lot of stats we use at METR is intentionally pretty simple. E.g. I think we've never done Bayes, despite having many members of staff who would feel intuitively on board with that. Partly this is for communication clarity, partly because it gives less surface area to mess up.
---
Evaluating Model Attack Selection and Offensive Cyber Horizons, Dec 2025
Donating on identical thesis to previous grant -- I wish I could have gone in for more size then!
I haven't thought about expenses for RE; from my perspective, I'm also thinking of this funding as a bet on Sean.
Parker Whitfill · Manifund · full post · Nov 30, 2025
General Career Support, Nov 2025
Read more ▾Show less ▴Noting two very prominent downsides of this grant up front:
However, some upsides:
Overall, I'm excited to suggest this grant!
---
General Career Support, Nov 2025
Another thing to say: it's not at all obvious to me that econ PhD work is the best use of Parker's time from my POV. I am not suggesting this grant because I want to support the econ PhD. I am suggesting this grant because I think the world would benefit from Parker having more freedom with his time, and this grant does that locally.
Theia Vogel · Manifund · full post · Nov 5, 2025
Fund thebes' model tinkering, Nov 2025
Read more ▾Show less ▴I asked Theia to put this proposal up, on the recommendation of my excellent and very well-read colleague Charles Foster.
The thesis here is: get small grants to high-context model tinkerers who are AI risk skeptics -> they take time away from their jobs to do indie research and writing projects -> they blog about what they find -> those blog posts get read by folks within the AIS world and AIS-skeptic world -> everybody converges to truer beliefs.
I'm excited to support Theia as part of a bet on this thesis!
Alexa Pan · Manifund · full post · Dec 13, 2023
Read more ▾Show less ▴I've made a $1.5k offer to this project; @RenanAraujo will do the same. (@Austin could you possibly raise the maximum funding limit, just in case others are interested in going beyond the $3k that we agreed with Alexa?)
**Main points in favor of this grant**
Renan and I put out a call to an invite-only scholarship program, the "Aurora Scholarship," to 9 individuals recommended by a source we trust. We were aiming to support people who are nationals of or have lived in China with a $2,400-$4,800 scholarship for a research project in a topic related to technical AI safety or AI governance. The project should last for approximately 4-8 weeks (i.e. we aim to offer $600/week at 20h/week).
Our hope is that scholars might use the experience and signaling value of these projects to counterfactually advance through to the next stage in their chosen career pipeline (e.g., PhD acceptance, think-tank placement), and that this program will strengthen the Chinese AI safety community. The program is loosely inspired by this CAIS program (but note we're not affiliated with CAIS and this does not mean CAIS endorses this program), especially in the sense that it is a requirement to join the program that each scholar has to seek their own supervisor.
Alexa was one of our excellent applicants.
Matt Sheehan is a well-respected China analyst who obviously believes in this project. I'm looking forward to seeing how Alexa gets on, and am excited for her to benefit from mentor feedback and association.
**Donor's main reservations**
I feel ill-equipped to judge the object-level benefits (and therefore downsides) of AI governance projects related to China, which worries me a little. But it seems unlikely that a small project driven by a promising undergraduate (and Concordia affiliate) and mentored by a respected, mainstream, US-based China analyst could result in serious downside.
Targeting the Aurora Scholarship invitees in the way we have has greater possible downside. We think this is fairly low, and have taken steps to lower this further (e.g. by not including junior researchers currently located in China).
**Process for deciding amount**
In her application, Alexa suggested to us that this project would take "median 15h/week" for "4-6 weeks." I took the maximum of rounding up one of these numbers (for possible expenses/planning fallacy). Max($30 x 20 x 5, $30 x 15 x 6) = $3,000.
**Conflicts of interest**
None.
Zhonghao He · Manifund · full post · Dec 11, 2023
Mapping neuroscience and mechanistic interpretability , Dec 2023
Read more ▾Show less ▴I've made a $1.75k offer to this project. @RenanAraujo and I asked Zhonghao to put this project up, having agreed to fund him for our "Aurora Scholarship" program. (This was supposed to be $4.8k shared between us, but @Austin pipped us to the first $1.2k! I would advocate raising the maximum grant amount if possible @Austin; Zhonghao noted that he would be working on this project for more hours than we had pre-committed to fund, and @NeelNanda comment strengthens the case for potentially funding the remaining hours.)
**Main points in favor of this grant**
Renan and I put out a call to an invite-only scholarship program, the "Aurora Scholarship," to 9 individuals recommended by a source we trust. We were aiming to support people who are nationals of or have lived in China with a $2,400-$4,800 scholarship for a research project in a topic related to technical AI safety or AI governance. The project should last for approximately 4-8 weeks (i.e. we aim to offer $600/week at 20h/week).
Our hope is that scholars might use the experience and signaling value of these projects to counterfactually advance through to the next stage in their chosen career pipeline (e.g., PhD acceptance, think-tank placement), and that this program will strengthen the Chinese AI safety community. The program is loosely inspired by this CAIS program (but note we're not affiliated with CAIS and this does not mean CAIS endorses this program), especially in the sense that it is a requirement to join the program that each scholar has to seek their own supervisor.
Zhonghao was one of our excellent applicants.
**Donor's main reservations**
I think the project itself is very low downside.
Targeting the Aurora Scholarship invitees in the way we have has greater possible downside. We think this is fairly low, and have taken steps to lower this further (e.g. by not including junior researchers currently located in China).
**Process for deciding amount**
As above.
**Conflicts of interest**
None.
ChinaTalk · Manifund · full post · Dec 4, 2023
Read more ▾Show less ▴@JordanSchneider thanks Jordan.
Those all sound like potentially cool activities. One big question I have is: what are the ChinaTalk theories of change, and should I expect some of these to be significantly more impactful than others? (I am asking for information, not asking you to change. I don't want to ruin the ChinaTalk magic by suggesting a change in focus!)
Here are some goals you might have in mind:
(I'll note that three of your five suggestions above sound to me like "mentor a new generation", not "enrich public discourse." I feel a little confused about this -- the impression I get from listening to the show is that you're especially concerned about the former, but your application is focused on the latter.)
With the possible goals in mind, here are some more focused questions that it would be great to get your take on:
---
Support for Deep Coverage of China and AI, Aug 2023
I've made ChinaTalk an offer of $10k. This is not necessarily the last offer I will make to ChinaTalk.
In fact, I am currently very excited about ChinaTalk absorbing at least $50k from Manifund (plausibly more). I will advocate for this outcome to other regrantors. At the same time, I am very excited to hear any good reasons against funding ChinaTalk further.
I am agnostic about how this funding should be structured (see Austin's previous comment) -- I will leave Austin and Jordan to figure out what makes most sense.
Below, I explain my thinking around this grant -- my early confusions, the call that gave me some clarity on these confusions, and my current perspective.
Early confusions
When I began considering this grant, I was cautiously excited but very confused. Reasons for excitement included:
- I am a long-time ChinaTalk listener. I thought it was fantastic -- detailed conversations with a wide range of experts, on important topics, that are difficult to find elsewhere, with a charismatic host. - China-US relations, China-tech issues, and Chinese views on and actions in AI are all very important. - I can expand, but for now will treat it as obvious. - One point that I haven't seen elsewhere: some paths to AI disaster run through military competition incentivizing state actors to hand over critical or obviously-potentially-dangerous systems to AIs. Having more transparency about what China might do here seems great. - At least according to Jordan, "in the entire United States today, there are no more than 700 China researchers." This level of neglectedness seems almost unbelievable. - In the area I can somewhat independently evaluate, AI, Jordan's understanding and views have always seemed reasonable to me.
Reasons for confusion include:
- Uncertainty about ChinaTalk's paths to impact. (See my earlier comment.) - What are they? - What is Jordan optimizing for? Policy influence, better information environment, developing expertise, mentoring China experts...? - What should Jordan optimize for? In other words, how should I think about the relative impact of these paths? - How might Jordan optimize more strongly for particular paths to impact? - Jordan's proposal is not written in a way that I find helpful. - In particular, he seems to be trying to find out which discrete project will most easily get funded. Among other things, this makes it difficult for me to understand his beliefs about what is most important/impactful. - Also, it's too unfocused. - Why aren't others jumping to support ChinaTalk? How is it possible that the best English-language media I can find about China has been run on $35k/year? - Perhaps philanthropy doesn't support ChinaTalk for reasons that would not affect my own excitement. - Grantmakers lack China expertise, leading to a vicious cycle whereby they do not fund work on China that would help them gain expertise. - Grantmakers are unexcited about small media companies in general. - Grantmakers will not consider grants this small. - Grantmakers are too nervous to make grants in this area for political reasons. - Jordan doesn't have a PhD. - Despite the podcast suggesting that Jordan has a wide network, Jordan has very limited networks in philanthropy. (His proposal not being well-optimized for Manifund regrantors is some evidence for this.) - Perhaps philanthropy doesn't want to support ChinaTalk for reasons that from my perspective would be persuasive if true. - I am wrong that Jordan is a smart expert working on important topics with reasonable views. (I am a novice; more serious people recognize that his expertise aren't there, or his views aren't reasonable, or something else.) - I am wrong to infer that others get value from the podcast from the fact that I enjoy the podcast. - Perhaps ChinaTalk takes very little work. - If ChinaTalk has garnered little philanthropic support, and Jordan is an expert in an important, neglected area who "meets regularly with senior US officials to discuss and advise on US-China relations and technology policy," then why isn't Jordan in an important government job?
Notice that I am largely ignoring Jordan's particular proposals regarding what he might do with the money. It doesn't feel like these would be pivotal for my decision to fund.
Call with Jordan
I spoke with Jordan about his proposal. Our chat made me feel significantly more confident in my optimism. I found out, for instance, that:
- I shouldn't worry so much about distinctions between ChinaTalk's paths to impact. - Jordan is credibly dedicated to helping prevent really bad outcomes like World War III. Within that, he optimizes for activities that give him energy. This seems extremely reasonable to me. - The paths are synergistic with one another: meeting with senior officials helps with connections, which helps with the information environment, which helps with encouraging a new generation of experts, etcetc. - The reason why Jordan's proposal is not written in a way that I would find easier to read is that he's not very engaged with the EA memesphere. On balance a good thing with respect to this grant, from my perspective! - Whilst poking him on this, I discovered an especially shocking fact. Luke Muehlhauser was quoted as being very enthusiastic about ChinaTalk and Jordan personally, but too busy to consider grants of this size. If I were looking for Manifund funding for ChinaTalk, such a quote would be in the first few lines of my proposal. I took the absence of this quote in his application as credible evidence in favor of "despite other talents, Jordan is bad at writing proposals for the Manifund audience" and against "Jordan is mildly manipulating me with his powers of persuasion." - There are good answers to all of my concerns about ChinaTalk having received so little funding support. - Philanthropists are scared to touch China, in part because of lack of expertise and in part for political reasons. - Advertisers can be nervous for similar reasons. - EAs in particular are scared to touch anything vaguely shaped like an AI race. But there is reason to think this might be silly in the China case. More importantly, Jordan is a force pushing for a more transparent information environment, not for racing. - Jordan only recently started his funding push. Previously he was employed or looking for jobs. And it wasn't clear whether a small media business like ChinaTalk should be a philanthropy-backed entity; Jordan was hoping to support this work through subscriptions only. - The private list of senior US officials is indeed impressive. - Fancy and influential people in the China and tech policy space are big fans.
I left the call feeling like the grant was too good to be true. So I asked two people I really trust for their views:
Current perspective
All in all, here's my current understanding of the situation:
If all of the above is true, then ChinaTalk is to my eye perhaps the best grant opportunity on Manifund to date.
Finally, note that many of my confusions and reasons for excitement might also apply to other China X AI projects. I plan to seriously explore this further, and have some early leads from Jeffrey Ding as well as from Jordan. By default, I will take low excitement about other projects in this space as a reason to be even more excited about ChinaTalk (although I am cautiously optimistic that I will be able to find exciting projects).
---
Support for Deep Coverage of China and AI, Dec 2023
I've offered ChinaTalk an extra $5k. I have become more bearish since writing the below, but I still directionally stand by it.
Jordan's recent conversations with Jason Matheny and Dylan Patel are marvelous. He continues to dive deeper into AI, and he continues to have difficulty funding his important niche for understandable but frustrating ~political reasons. I am really, really excited to continue to support ChinaTalk.
(Written hastily; happy to expand.)
ALERT · Manifund · full post · Dec 4, 2023
Make ALERT happen, Dec 2023
Read more ▾Show less ▴I've made a $5k offer to this project. I might end up offering more later. Below written hastily.
**Main points in favor of this grant**
**Donor's main reservations**
**Process for deciding amount**
I would prefer to offer a larger amount, but my Manifund budget is pretty constrained. Any amount I offer to this project will be based on some vague notion of 'fair share' between the remaining projects I want to fund before Dec 31.
**Conflicts of interest**
First, Nuno is my friend. Second, one of my projects (Dendritic) is taking up a part of the ALERT mantle in a different way (monitoring + emergency response for bio in particular). There's no immediate conflict with Dendritic, but it doesn't seem implausible to me that there could be some Nuno-Dendritic collaboration in future given the overlap.
Joel Becker · Manifund · full post · Oct 31, 2023
Estimating annual burden of airborne disease (last mile to MVP), Oct 2023
Read more ▾Show less ▴Thanks for the questions @NunoSempere.
Do you already have some decision-makers who could use these estimates to make different decisions?
Waiting on some emails that will help me answer this; will get back to you soon.
How valuable do you think that this project is without the long covid estimate?
I just put this question to Vivian, who says: "I think without the long covid estimate it probably just never gets published."
Who is actually doing this work? Vivian and Richard, or Joel and Aron?
Joel and Aron. (If "this work" refers to building the long COVID estimate.)
Why are you doing stuff $3.6k a time, rather than having set up some larger project with existing biosecurity grantmakers?
Very possible that I've made a silly decision somewhere, but here's how it looks from my side:
- Earlier this year I applied to LTFF and Lightspeed for ~general funding for Qally's. These applications were rejected. - I think rejection was the right call -- even if you think higher-effort quantitative analyses are worthwhile, it's preferable for them to come with demand signals rather than subsidizing my discretionary offerings. - I didn't apply to ~any other projects nor sources of funding because of my US work visa stuff. - Mostly this is 'getting a US work visa attached to your own new organization requires a lot of time.' - To a lesser extent this is 'the expected benefit of finding new projects is lower than usual, because I would only proceed with these projects if I was approved for the visa.' - My visa petition was approved ~4 weeks ago; I have started to seriously look for projects in that time. One project that came up was this one. It currently doesn't have a source of funding, so I put up a Manifund proposal. - All that said, I would, of course, like to work >$3.6k at a time. A few things on this: - I hope that this application will attract more than the minimum! - It's plausible that this will be a larger project. There are a bunch of other things to work on for this project, which I currently have cached as 'things Vivian or Richard will do' or 'things that aren't necessary for MVP.' - I almost feel like I don't know other existing biosecurity grantmakers...! Linch has moved away from this sort of thing. Open Philanthropy staff are in the business of larger amounts of money. My uninvestigated understanding from this proposal is that Convergent Research (where Vivian has recently worked) aren't able to fund adjacent projects. So I posted on Manifund and notified @GavrielK. (I'm apparently allowed to fund myself, but don't want to.) - There is another project that I might take on, which I expect would be ~full-time for many months.
---
Estimating annual burden of airborne disease (last mile to MVP), Oct 2023
Thank you for the comment @GavrielK!
On one of your thoughts:
I've personally received conflicting opinions from people in the field about how much policymakers care about/pay attention to in-depth analyses that they didn't actually commission. I like that this analysis format will result in a solid headline number that can be communicated simply.
I think the skepticism from people in the field is warranted. And I agree it's attractive that this project will result in a headline number -- this is definitely what Vivian and Richard have in mind. But I think there's a related risk of the project here: because the (at least MVP) analysis will be less in-depth, the headline number will be easier to undermine.
On your questions:
I'm sort of surprised that this needed to be pursued through independent funding [...] What's the missing expertise that Qally's brings to the table, and why isn't CHS backing this piece of the project?
I think the missing expertise is: having the time to do the work! My understanding is that the project hasn't moved for a few months for this reason. That said, since posting this project on Manifund, I'm less confident that Vivian would not contribute (further than they already have) to the nitty gritty of the long COVID estimates; I guess this is mostly upside from your perspective.
As for why CHS isn't backing this project, I think there's a mix of (1) limited JHCHS staff capacity, and (2) part of the original motivation of this project being Vivian receiving mentorship from Richard. (I am happy to explore the possibility of funding from JHCHS at some point.)
The think-tank-friendliness of it reduces the failure risk from the team not having time to write up results--possibly could rope in the dedicated policy advocates at IfP or Rethink for advocacy-style writing?
Could you possibly expand on what roping in dedicated policy advocates would involve? Asking them to produce a document that reads like a policy report, given our less friendly write-up? And perhaps asking them to share this document with contacts in government and media?
Are you also considering long-term costs of other viruses? Just curious--I know this is much more speculative, but my impression is that COVID just had the most attention on it for a while and now there's increasing scientific interest in possible long-term effects of flu. That would also emphasize the importance of endemic disease. But could be way too complicated, just wondering if it was discussed!
Aron and I are not, at least not right now. But Vivian and Richard have so far considered: common cold, influenza, and TB.
You reference researching other contributions to disease burden as a possible extension of this project--what contributions would those be, if the bulk of the project has already been completed?
The other components that Vivian has marked as needing to be revisited are: mainline estimates of direct medical costs of common cold and of influenza, lost productivity from influenza, and learning loss due to common cold and influenza. They already have low and high estimates for each of these, but I think hasn't thought about which of these estimates is more appropriate (which in turn would affect their averaging for the mainline estimate).
Good Ancestors Policy · Manifund · full post · Sep 20, 2023
Good Ancestors Policy expenses, Sep 2023
Read more ▾Show less ▴Main points in favor of this grant
In my regrantor bio, I wrote that my “edge as a regrantor [...] comes from having an unusually large professional network.” This grant is primarily a bet on that network — when I asked people I trust about projects they are unusually excited about, the Good Ancestors Project received a very strong recommendation.
The case in favor seems clean:
- Greg Sadler is a very senior public servant who deeply understands Australian government. This means he is more likely to have considerable influence. - He has already had some reasonably significant wins; above pre-funding expectations. - Without additional funding, he might soon return to his previous position. - This makes it more likely that the impact of this grant is counterfactual. - I take Greg’s revealed preferences (wanting to continue with GAP if possible) to reveal that he believes that returning to his previous position would be a less impactful option. - The effect on incentives from not funding senior professionals who make bold moves into more impactful work (and exceed expectations for this work) when the funding winds change seems awful. - Greg has tentatively agreed that “[i]f GAP stops operating before 30 June 2024 (or some similar date) it will return this grant.” - I leave the exact details to be worked out with Austin. - This was somewhat important to me. Manifund dollars will likely be necessary but insufficient for GAP to continue; I wanted to make sure that the funding wouldn’t be wasted if GAP wasn’t able to raise remaining funds. - I do not sense that there is much negative selection going on here.
Donor's main reservations
My main reservation is presumably the same reservation that other grantmakers have had — the Australian government is not an unusually important actor with regards to making advanced AI safe or ending pandemics. To me, this seems like a pretty severe reservation; I worry that I am not triaging sufficiently hard.
The incentives point above weighed on me quite heavily when thinking about this grant. At one point (not now) I thought it would be the pivotal consideration. But I distrust my reasoning here for two reasons:
Lastly, I’m a little concerned that making this regrant will prevent me from making offers that I am even more excited about in the near future.
Process for deciding amount
Less of a process, more of a cloud of reasons:
Conflicts of interest
Please disclose e.g. any romantic, professional, financial, housemate, or familial relationships you have with the grant recipient(s).
None.
Robert Long · Manifund · full post · Sep 9, 2023
Empirical research into AI consciousness and moral patienthood, Sep 2023
Read more ▾Show less ▴Main points in favor of this grant
I have been trying to nudge Rob in this direction since earlier this year.
Earlier this year I was involved in a startling conversation. Rob Long had been speaking about the chances that we will see conscious AIs in the coming years. (And I had started to grok this possibility.) Now, he was talking about research collaborations he might aim for in future. Rob had empirical projects in mind that could only be done with access to frontier models. Should he bug a colleague-of-colleague to work at [top AI lab X]? Should he ask [collaborator Y at top AI lab Z] about the possibilities at his employer? Rob’s conclusion was: not right now. Rob already had his plate full with other work, the request might be annoying and, besides, Rob had already had a similar request to similar people declined-ish a couple of months ago.
This situation struck me as being preposterous. Here is one of the world’s top experts on AI consciousness, claiming a nerve-wracking chance of AI consciousness in the not-too-distant future, with fairly strong professional links at top AI labs and ~shovel-ready ideas for empirical projects, preparing a not-terribly-costly ask (give me a work desk, ~0 engineering time, and model access to work on these research questions)... and he is unsure whether he should make the ask?!
It seemed to me that the right question to ask was more like “should I try to start a research group as soon as possible?”. (Of course there are many reasons why starting a research group might be a bad idea. But, even if that were the case, Rob should at the very least be asking to work at places that would enable him to work on his empirical projects.)
I want Rob to move the needle on empirical AI consciousness projects harder and faster. In the short-term (see below), this means doing less ‘public-facing work and thinking about his professional opportunities,’ and more ‘thinking through a theory of change for AI consciousness research, spending more time on empirical research with existing collaborators (e.g. Ethan Perez), and pushing for ways he can continue this research in the near future.’
Donor's main reservations
First, I don’t think Rob needs funding in some sense. But I’m not super concerned about this. People should be paid for important work and, besides, I’m partly trying to set up a good incentive environment for future grantees.
Second, I think that I can only counterfactually change a fraction of Rob’s professional activities. Currently, his work falls under the following buckets:
Bucket (1) is the sort of work that I want Rob to be doing more of: activities that directly move the needle on empirical work in his area of expertise.
I instinctively feel less excited about bucket (2), because this paper will not involve empirical AI consciousness research. But I don’t want to impose on Rob’s pre-existing commitment to this project. Also, the issues of the paper have some overlap with writing a strategy doc. (Though this overlap should not be overstated, as academic work is optimized for different things than a strategy document).
Bucket (3) I think Rob should be doing less of. The public-facing work mentioned above does not obviously move the needle on empirical work — and to the extent it does (e.g. indirectly via field-building or career capital), I would feel better if Rob undertook this work after having reflected more on his theory of change for AI consciousness research, rather than as a natural consequence of the release of his recent paper. And, unlike for bucket (2), giving up on some bucket (3) commitments feels low-downside — Rob is not going to be a less interesting podcast guest in 1 years time!
Bucket (4) feels like a waste of time that I want Rob to avoid.
My understanding is that buckets (3) and (4) add up to a bit less than half of Rob’s time at the moment.
Third, empirical projects in AI consciousness feels like a tricky area where I am extremely out-of-my-depth. I am strongly relying on Rob being a ‘reasonable expert who won’t make dumb decisions that make me regret this grant.’ That said, I feel very good about relying on Rob in this way.
Process for deciding amount
Time buckets (3) and (4) add up to 20 hrs/wk * 5 weeks = 100 hours time. Rounding up to 120 (for possible underestimation, and professional expenses), at $60/hour, I will provide $7,200 funding. I'm leaving up to $12,000 as a funding goal, in case anyone wants to fund the remainder of Rob's time during the 5 weeks.
Conflicts of interest
Please disclose e.g. any romantic, professional, financial, housemate, or familial relationships you have with the grant recipient(s).
I was housemates with Rob for a couple of months in early 2023, which is how I found out about this grant.
Holly Elmore · Manifund · full post · Aug 10, 2023
Holly Elmore organizing people for a frontier AI moratorium, Aug 2023
Read more ▾Show less ▴I made Holly an offer of $2.5k yesterday. This won't necessarily be the last offer I make to this project; I'm giving a smaller amount in order to get money out the door more quickly and to provide a credible signal to other regrantors.
Here's my reasoning about this grant as of the offer:
1) Overall, I feel excited about this proposal on the meta level.
2) I feel confused about how I should value this proposal on the object level.
Overall:
---
Holly Elmore organizing people for a frontier AI moratorium, Aug 2023
@Holly_Elmore Thanks for picking this up; I totally agree that you're not reflecting some "LessWrong perspective." My previous wording was clumsy. My thinking might be too. I'll have another try at pointing at this cluster of concerns.
- I'm having a conversation with a senior US policymaker or journalist (maybe not focused on AI) 1 year from now. I say to them: "Holly did X work that attempted to encourage Y person/organization to do Z." Then they say: "who is Y?" Unfortunately, actor Y is not very influential. - An organizer with more experience in politics/media or without the shared presentation/ideas might have picked up on this and better targeted their efforts. - Perhaps Holly has targeted Y for bad taste reasons (Y is more sexy, or better-known to LW types) or for good efficiency reasons (Holly has second-hand connections at Y, perhaps borrowed from shared-memesphere-connections, which made it easier to engage with Y than an actor with similar influence and lesser connections). - A senior journalist engages with Holly 3 months from now. Associating the idea of a moratorium, or even Holly in particular, as coming from a particular memesphere, the journalist takes her less seriously, or seeks to contrast her more strongly with other voices, or comes to see her view as representative of the memesphere in ways that are unhelpful for others' lobbying, or [other bad outcome]. - After writing, I'm not sure this concern makes sense. The reasoning seems pretty tortured, and the outcomes are not that bad anyway. - The better version of this might look more like my Conjecture comment in the next paragraph: people who Holly engages with might persistently take AIS concerns less seriously if pushing a message that they think is crazy.
Regarding Conjecture, I have the sense (I think second-hand from public Matt Clifford communications and a bunch of conversations with [redacted prominent AI safety researcher]) that policymakers get turned off very quickly when they hear the message that everyone's going to die. My big concern is that this turning off persists over time.
I agree that "the question of whether to regulate/slow/stop AI transcends many of the technical details" on the substance. I guess I have a sense that, at least pre-Hinton leaving, AI safety worries felt to many people like they came from amateurs who had misunderstood the nature of the technology, which is part of what turned them off from these concerns. Maybe I think this has an unhelpful interaction with arguing for a window-shifting policy.
After writing out the above, I think I see that my concerns -- even if they do make sense for conversations with policymakers and media elites -- might not port over to political organizing. I could probably get clearer on this if I had a clearer sense of the political organizing activities you might pursue. Could you describe these in more detail?
Marcel van Diemen · Manifund · full post · Aug 8, 2023
The Base Rate Times, Aug 2023
Read more ▾Show less ▴I've made Marcel an offer of $2.5k. This is not necessarily the last offer I make to this project; I'm giving a smaller amount in order to get money out the door more quickly and to provide a credible signal to other regrantors.
This grant seems fairly straightforward to evaluate. On the one hand,
On the other hand,
Overall, I'm happy to give partial funding to this project, on the substance and as a signal to others.
Miti Saksena · Manifund · full post · Aug 8, 2023
Optimizing clinical Metagenomics and Far-UVC implementation., Aug 2023
Read more ▾Show less ▴@GavrielK (Note to others: I have a major COI with Miti.)
I think your write-up is very clear. But I am somewhat surprised that you have ~nothing negative to say, which makes me think that it could be more useful if you said more (true, salient, and) negative things.
Even if you are totally positive about the project, could you say something about how other bio opportunities you've looked at feel below this bar? I am imagining things like "cost too much relative to Manifund pool," "seem better-suited to larger funders with more context on their project," and other stuff that I think you imply in the write-up but don't say outright.
| Date ↓ | Funder | Via | Amount | Cause | Source | Purpose |
|---|---|---|---|---|---|---|
| Aug 21, 2024 | Ross Rheingans-Yoo | Manifund | $200 | Global health and development | manifund | Estimating annual burden of airborne disease (last mile to MVP) |
| Dec 5, 2023 | arenal | Manifund | $3,400 | Global health and development | manifund | Estimating annual burden of airborne disease (last mile to MVP) |
| Dec 5, 2023 | Austin Chen | Manifund | $3,600 | Global health and development | manifund | Estimating annual burden of airborne disease (last mile to MVP) |
| Dec 5, 2023 | b | Manifund | $1,000 | Global health and development | manifund | Estimating annual burden of airborne disease (last mile to MVP) |
| Oct 2022 | EA Infrastructure Fund | EA Funds | $8,480 | Other existential risk | ea_funds | Part-time salary for myself and my PA to work on SHELTER (Safe Haven to Evade Long-Term Extinction Risk) Weekend |
| Oct 2022 | Long-Term Future Fund | EA Funds | $27,248 | Other existential risk | ea_funds | 5-month salary plus expenses to support civilizational resilience projects arising from SHELTER Weekend |
| Jan 2021 | Long-Term Future Fund | EA Funds | $42,000 | Strategy & forecasting | ea_funds | Replacement salary for teaching during economics Ph.D., freeing time for conduct research into forecasting and pandemics |
| Date ↓ | Recipient | Via | Amount | Cause | Source | Purpose |
|---|---|---|---|---|---|---|
| Apr 28, 2026 | Lyptus Research | Manifund | $149,980 | Other | manifund | Lyptus Research: Bridge Funding |
| Jan 9, 2026 | Parker Whitfill | Manifund | $38,000 | Career transitions | manifund | General Career Support |
| Jan 4, 2026 | Sean Peters | Manifund | $41,000 | Evals | manifund | Evaluating Model Attack Selection and Offensive Cyber Horizons |
| Dec 18, 2025 | Sage | Manifund | $71,700 | Epistemic infrastructure | manifund | AI Digest |
| Nov 5, 2025 | Theia Vogel | Manifund | $17,500 | Interpretability | manifund | Fund thebes' model tinkering |
| Sep 15, 2025 | Sean Peters | Manifund | $32,000 | Career transitions | manifund | 3 Months Career Transition into AI Safety Research & Software Engineering |
| Jan 9, 2024 | Marcel van Diemen | Manifund | $2,500 | Epistemic infrastructure | manifund | The Base Rate Times |
| Dec 15, 2023 | ChinaTalk | Manifund | $17,550 | International governance | manifund | Support for Deep Coverage of China and AI |
| Dec 14, 2023 | Alexa Pan | Manifund | $1,500 | International governance | manifund | Explainer and analysis of CNCERT/CC (国家互联网应急中心) |
| Dec 12, 2023 | ALERT | Manifund | $8,000 | Strategy & forecasting | manifund | Make ALERT happen |
| Dec 11, 2023 | Zhonghao He | Manifund | $1,750 | Interpretability | manifund | Mapping neuroscience and mechanistic interpretability |
| Dec 5, 2023 | Good Ancestors Policy | Manifund | $10,000 | Other existential risk | manifund | Good Ancestors Policy expenses |
| Oct 5, 2023 | Robert Long | Manifund | $7,200 | AI welfare | manifund | Empirical research into AI consciousness and moral patienthood |
| Aug 21, 2023 | Holly Elmore | Manifund | $2,500 | Advocacy | manifund | Holly Elmore organizing people for a frontier AI moratorium |