Daniel Eth

Hiring staff to carry out longtermist academic legal research and increase the operational capacity of the organization.

The Legal Priorities Project (LPP) applied for funding to hire Suzanne Van Arsdale and Renan Araújo to conduct academic legal research, and Alfredo Parra to perform operations work. All have previously been involved with the LPP, and Suzanne and Renan contributed to the LPP’s research agenda. I’m excited about this grant for reasons related to LPP as an organization, the specific hires they would use the grant for, and the proposed work of the new hires.

Overall, I think the LPP is exciting enough to warrant funding in general, at least for the time being. The goal of the LPP is to generate longtermist legal research that could subsequently be used to influence policymakers in a longtermist direction. Personally, while I don’t think this theory of change is crazy, I’m also not completely sold on it (I have perhaps a more cynical view of policy, where principled academic arguments are rarely causal in policies being enacted, even if they are claimed as justifications). I do, however, think this theory of change is plausible enough that it’s at least worth trying (if done well), especially since the indirect effects (discussed below) also seem most likely positive.

Insofar as the LPP’s theory of change makes sense, I think they’re likely to go about it well. Since they’re a younger organization, they do not yet have a large body of output. Their main piece of output so far is their research agenda. As part of my evaluation, I skimmed the agenda in full and more closely read a couple sections (including the sections on longtermism and biorisk), and I found the work to be at a high level of quality. More specifically, I thought it was well written (thorough without being verbose, expressed concepts clearly – especially for an audience that might not be familiar with longtermism), demonstrated good understanding of longtermism and related concerns (e.g., infohazards), and presented "weird" longtermist concepts in a manner unlikely to be off-putting to those who might be put off by much of the rhetoric around such concepts elsewhere. This work makes me hopeful that future work will also be high quality.

Additionally, the LPP team includes many individuals with stellar academic credentials. I’m usually skeptical of leaning too heavily on academic credentials in making assessments (I tend to consider them only one of many reasonable signals for intelligence and competence), but I’d imagine that such credentials are particularly important here given LPP’s theory of change. My sense is that policymakers and other legal scholars are much more inclined to take academic ideas seriously if they’re promoted by those with prestigious credentials.

Furthermore, I think it often makes sense to fund newer organizations without long track records if there is a plausible argument in favor (and no apparent large downside risk). The upside of such grants can be large, and the wasted effort and resources if they don’t work out will probably be only a minor downside. I think such arguments are also stronger for organizations such as the LPP that are operating in newer areas, or with a theory of change different than what has come before, as their progress (or lack thereof) provides valuable information about not only the organization, but also potentially the whole area.

In addition to the direct effects from the LPP pursuing its theory of change, I’d imagine the indirect effects here would also most likely be positive. Given my impression of the LPP’s ability to promote longtermist ideas faithfully and in a non-offputting manner, and additionally the fact that it would be lending academic credibility to such ideas, my sense is that the LPP would, if anything, be good for the reputation of longtermism more broadly. Additionally, the LPP might allow for those involved to gain career capital.

Regarding the specific hires, I am also enthusiastic – due to their past work and educational credentials. Suzanne was the main author of the “Synthetic Biology and Biorisk” section of the LPP’s research agenda, and Renan was a co-author of the “Longtermism” section, both of which impressed me. Additionally, both have impressive academic credentials (Suzanne has a J.D. from Harvard, Renan has an M.Sc. from LSE), which I believe is valuable for reasons I outlined above. While I have less of a sense of the quality of Alfredo’s work than that of Suzanne and Renan (and, as a researcher myself, I trust my judgment less on questions related to operations than to research), it’s a positive signal that Alfredo has a Ph.D. from TU Munich and previously worked for four years in operations at EAF/CLR, and the fact that he has been involved with the LPP so far and they’re interested in hiring him is a further signal. I also think that operations work is important, and I am fairly inclined to defer to (seemingly competent) organizations when they claim a particular individual that they are familiar with would be a good operations hire.

Finally, I support this grant because I support the proposed work for the hires. According to the application, Suzanne will work on the topic of dual-use research, and Renan’s work will focus on protection of future generations via constitutional law. Both of these areas strike me as sensible areas of research for academic legal work. While Suzanne’s topic does contain potential downsides (such as infohazards) if handled poorly, her section within the research agenda on “Synthetic Biology and Biorisk” contains a subsection on “Information Hazards” which shows she is at least aware of this risk, and the section as a whole demonstrates a nuanced level of thinking that makes me less worried about these potential downsides (and I further update against being worried here given that the LPP as an institution strikes me as presenting an environment where such issues are handled well). Alfredo’s proposed work includes setting up “robust internal systems”. While vague, this work also seems valuable for the organization.

Continuing research projects in AI forecasting, AI strategy and forecasting.

Tegan applied for funding to continue independent research relevant for AI strategy and AI timelines. The LTFF has twice before funded Tegan to perform independent research. I supported this grant primarily due to Tegan’s research output from these previous two grants, and despite some skepticism about an independent-research environment being the best option for Tegan.

Tegan’s main outputs from her previous grants are a report comparing primate and bird brain architectures in terms of cognition, and a draft of a report that both compares the efficiency of biological evolution vs. deep learning in terms of improving cognitive abilities and sheds light on whether there is a common ordering between biological evolution and deep learning in terms of which cognitive tasks are more difficult to learn. I had previously read the former report, and I skimmed the latter for this evaluation.

My overall evaluation of these two outputs is as follows. These research questions are very ambitious (in a good way), and would be valuable to learn about. I think these questions are also in areas that are unlikely to see significant work from academia; both projects require making somewhat fuzzy or subjective judgments with very limited data and generalizing from there, and my impression is that many academics would dismiss such work as too speculative or unscientific. Additionally, the interdisciplinary nature of this work means there are fewer individuals in academia who would be well-positioned to tackle such research. Furthermore, work along these lines is neglected within the EA community. Despite the fact that biological evolution is the only process we know of that has produced human-level intelligence, there has been very little work within EA to study evolution in hopes of gaining insights about transformative AI. That Tegan’s research focuses on questions that are important and neglected makes me more optimistic about the value of the sorts of projects she will pursue, as well as her research judgement.

Regarding research methodology, my sense is Tegan is going about this in a mostly reasonable way. My inside view is that she’s handling the fuzzy, subjective nature of this research well, her evidence largely supports her conclusions, and she appreciates the limitations of her research. The biggest weakness of both projects, in my mind, is that they suffer from very few data points. Admittedly, both projects are in areas where not much data seems to exist (and that which does exist is fuzzy and may be difficult to use). But the fact remains that with few data points, strong conclusions cannot be drawn, and the research itself ends up less informative than it would be with more data. While finding such data may be difficult, I have a reasonably strong prior that at least some further relevant data exists and that the benefits to the research from finding such data would outweigh the search costs.

In light of all this, I would like to see more research along the lines that Tegan has performed previously, and I am in favor of her continuing to pursue such research. Having said that, I think there are some potential downsides to pursuing long-term independent research. Since this will be the third grant that the LTFF will give to Tegan for independent research, I think it’s worth considering these potential downsides, as well as potential mitigation techniques. (I also think this is a valuable exercise for other long-term independent researchers, which is part of my motivation for spelling out these considerations below.)

The largest pitfall in my mind for long-term independent researchers is one’s research becoming detached from the actual concerns of a field and thereby producing negligible value. Tegan seems to have avoided this pitfall so far, thanks to her research judgment and understanding of the relevant areas, and I see no evidence that she’s headed towards it in the future.

Another potential pitfall of independent research is a general lack of feedback loops, both for specific research projects and for the individual’s research skills. One way that independent researchers may be able to produce stronger feedback loops for their work is by sharing more intermediate work. While Tegan has shared (and received feedback from) some senior longtermist researchers on some of her intermediate work, I think she would probably benefit from sharing intermediate work more broadly, such as on the EA forum.

Finally, independent research can struggle to get as much traction as other work (keeping quality constant), as it’s less likely to be connected to organizations or networks where it will naturally be passed around. My sense is that Tegan’s research hasn’t gotten as much attention as it “deserves” given its level of quality, and that many who would find value in the research aren’t aware of it. Fixing such a dynamic generally requires a more active promotion strategy from the researcher. Again, I think posting more intermediate work could help here, as it would create more instances where others see the work, learn about what the researcher is working on, and perhaps even offer feedback.