Sydney AI Safety Hub (SASH)

Some of this review is copied from my recent regrant to TARA.

**Main points in favor of this grant**

• I’ve visited several Australian AI safety and EA conferences this year (and for the past 5 years) and I think that the local talent base is quite strong. MATS has accepted ~12 Australian scholars (~4% of all alumni) and they did great! Australians can get work visas for the UK and USA (E-3) relatively easily, which means they can emigrate to the SF Bay Area and London for AI safety work.
• I’ve been impressed by Yanni’s operations and marketing ability in regards to TARA, the AI Safety ANZ Careers Conference 2024, and AI Safety ANZ’s recent seminar/workshop series.
• Australia might have a role as a "middle-power" between the US and China in facilitating international AI treaties. The Sydney AI Safety Hub and its members might help support the formation of an Australian AI Safety Institute or otherwise support the Australian government and industry bodies in furthering AI safety. Additionally, Australia, as a member of Five Eyes and a NATO partner, might have a role in tracking foreign AGI projects and building international coalitions for AI safety.
• I think that physical coworking spaces for AI safety have a strong track record for spawning useful projects (e.g., ARC Evals/METR in Constellation, Apollo Research in MATS London Office/LISA), facilitating talent flow into AI safety organizations, and supporting emerging and independent researchers with a strong community. Sydney has a few AI safety organizations (e.g., Gradient Institute, Harmony Intelligence) about which a hub can form and is proximal to talent sources like Sydney University and Sydney's corporate tech scene.
• $5k to secure an office space for two months seems really cheap! I think that $30k for a year of office space is similarly cheap and I would be surprised if this is not funded.

**Donor's main reservations**

• My grant is quite small and will only support the office for two months. AI Safety ANZ will need to secure permanent funding from a larger funder soon.
• Sydney University, which is next door to SASH, is ranked 69th internationally for computer science. This is not an impressive ranking relative to the top US or UK universities, many of which do not have nearby AI safety offices (exceptions: AISST office near Harvard/MIT, Constellation/FAR Labs near UC Berkeley, LISA near ICL, the late Trajan House near Oxford). However, I think Australia ought to have an AI safety office given the strength of AI safety talent I regularly see coming from Australia, and Sydney seems like the natural location (though Canberra or Melbourne are also possible, due to their proximity to Seth Lazar's and Dan Murfet's labs, respectively).
• It's possible that there is not enough strong talent or new orgs in Sydney to justify an office the size of SASH. To my knowledge, there is no dedicated AI safety research group at the University of Sydney. TARA should bring in some talent, however, and I encourage Yanni to help incubate any new orgs/projects that emerge in SASH.

**Process for deciding amount**

I decided to fund the entire amount because it's relatively small and AI Safety ANZ urgently needs funding to secure the office space for the next two months.

**Conflicts of interest**

I am an Australian citizen and have personal and professional ties to Australian AI safety and EA communities. Yanni recently invited me to present at the AI Safety ANZ Careers Conference 2024, although I received no financial compensation. I am an Advisor to AI Safety ANZ, Yanni’s organization, though I receive no financial compensation for this role. This year, I presented at the Australian AI Safety Forum 2024 and I have previously presented at EAGxAustralia 2022 and 2023, for which I received travel and accommodation funding.