Dario Amodei's Brain Trust Of Bad Ideas

  by Jordan Schachtel via American Greatness,

Anthropic CEO Dario Amodei wants to shape the future of artificial intelligence - and, by extension, the future of civilization. Yet in doing so, he seeks input and advice from an ideologically uniform group of individuals and institutions.

In a series of sprawling essays on his website, Amodei has outlined his expectations for a world transformed by powerful AI. His forecasts range from curing diseases and accelerating economic growth to creating catastrophic biological threats and destabilizing the balance of power between the United States and China.

These essays are presented as the work of a sober-minded technologist trying to navigate humanity through an unprecedented transition.

In the acknowledgments section, Amodei inadvertently reveals insight into the ideas forming his worldview.

Across four of his major essays - "Machines of Loving Grace," "The Urgency of Interpretability," "The Adolescence of Technology," and "Policy on the AI Exponential" - Amodei thanks a recurring cast of advisers and intellectual collaborators. The list includes Tom McGrath, Martin Wattenberg, Chris Olah, Ben Buchanan, Kevin Esvelt, Parag Mallick, Stuart Ritchie, Matt Yglesias, Erik Brynjolfsson, Jim McClave, Allan Dafoe, Mariano-Florentino Cuéllar, Richard Fontaine, Buddy Shah, Vas Narasimhan, Nick Beckstead, Jason Matheny, and Brad Carson.

Amodei thanks this group for their contributions to his ideas, comments on his drafts, and help to strengthen his thinking.

Of course, these acknowledgments do not prove some kind of uniformity of worldview. However, they help us map his intellectual world.

And that world is remarkably homogeneous.

It includes, unsurprisingly, Anthropic insiders but also establishment academics, liberal policy experts, national-security think tankers, NGO executives, and prominent figures intertwined with the Effective Altruism (EA) movement.

Here are some of the more notable characters Dario Amodei has cited:

• Matt Yglesias is a high-profile liberal journalist.

• Ben Buchanan, an Anthropic adviser, served as a special adviser for AI at the White House Office of Science and Technology Policy.

• Allan Dafoe is a prominent name in the Effective Altruism community, and he established one of its AI academic centers at Oxford.

• Nick Beckstead is a high-profile figure in the Effective Altruism network.

• Jason Matheny has deep, longstanding ties to the EA world, and he's now president of the RAND Corporation.

• Richard Fontaine runs the left-leaning Center for a New American Security (CNAS), a major Democratic Party policymaker incubator in D.C.

• Mariano-Florentino Cuéllar is the chief global affairs official at Anthropic and has long been a staple in the Democratic Party legal world.

• Chris Olah is an Anthropic cofounder.

• Stuart Ritchie is the research communications lead at Anthropic.

• James "Jim" McClave is an early Anthropic investor who has long been embedded in the EA community. McClave reportedly made one of the largest 2022 contributions to Future Forward, a super PAC that supported Joe Biden.

• Neil "Buddy" Shah is the CEO of the Clinton Health Access Initiative.

• Brad Carson is a former Democratic congressman who served in senior positions in the Obama administration.

What is missing from this collection of contributors?

First and most obvious, there is no identifiable right-leaning intellectual, academic, or policymaker in the mix. Dario Amodei does not appear to consult with any White House-aligned voices, and few, if any, critics of centralized power or anyone skeptical of unverifiable international agreements with China.

We can't find anyone publicly mentioned within Amodei's brain trust who seems to recognize that global, U.N.-like institutions frequently fail or become quickly corrupted by a group of credentialed "experts" who inevitably just seek to increase their own power and status.

His brain trust is indeed diverse in professional specialization, but it's painfully narrow in ideological diversity.

That likely explains the liberal-internationalist-meets-Effective-Altruist blend that runs through his writing and media appearances. Amodei combines the Effective Altruist obsession with extinction-level AI risks with the liberal internationalist worldview, which posits that an "expert" class can design more useful international institutions outside of America's sovereign borders.

In "Machines of Loving Grace," putting aside the hyper-pagan notion of a machine God, Amodei is quite optimistic about AI being used to advance a variety of fields that can result in human flourishing. The essay considers a near future in which powerful AI systems are making decades of scientific progress in just months or years.

In "The Adolescence of Technology," however, Amodei concentrates on the darker side of the same exponential curve. He discusses autonomous AI, bioweapons, economic displacement, and authoritarian inputs that could threaten much of humanity. Consulting with his brain trust, Amodei is determined to construct systems capable of guiding us through this dangerous period of "adolescence."

In "The Urgency of Interpretability," he calls for transparency laws and strict export controls on AI chips to China. He argues, in consultation with his brain trust, that America could create enough breathing room to make powerful systems more understandable before geopolitical competition makes caution impossible.

Amodei isn't wrong to point out that AI companies should definitely test their systems before public release.

And it's wise to continue dedicating resources to researching how to understand these novel systems.

Yet Amodei's institutional reflex is visible throughout his writing. When he encounters something that is determined by his West Coast bubble as a risk, the "expert" brain trust fills in the gaps on policy and morality.

In his latest proposals, Amodei has floated something of a U.N.-style AI standards body and a Paris Climate Deal 2.0 with China.

Amodei told CBS, "The worry I have - I am concerned that one single government could abuse this technology just as easily as a single company could."

He added, "But I think a combination of democratically elected governments - I don't know about handing over, but some kind of oversight, some kind of joint governance."

As for the China deal, he says the agreement must be verifiable. He even concedes that an AI pause is unlikely because the incentive to cheat would be enormous. Yet he still presses as if this is the best option. If that sounds like Hillary Clinton and Barack Obama, it's not some accident of interpretation. He is using the liberal internationalist playbook.

As the Paris climate agreement already showed, the Chinese Communist Party is not a good-faith participant in international agreements. It is an adversarial regime seeking technological, military, and geopolitical supremacy over the United States.

A climate change-style agreement is hard enough to verify. With AI, it's even more difficult. Do we really think China is going to hand American inspectors a list of its most capable data centers, military facilities, intelligence-gathering sites, and a whole variety of other secret networks that are critical to its national AI program?

Amodei can see the problem, as evidenced in his writing, but his narrow intellectual brain trust prevents him from following it to its logical conclusion. He recognizes the behavior of adversarial states while retaining faith in the institutions that assume cooperation among them. And he wants to create new global government institutions that will supposedly solve these problems.

This is the reality of living and operating in the San Francisco tech bubble. Dario Amodei and his ideologically narrow group have introduced a series of highly elaborate ideas that have not been stress-tested by reality and history.

Everyone helping him answer complex questions comes from the same constellation of institutions, liberal policy shops, Effective Altruist-connected organizations, and technocratic philanthropy. His U.N. for AI and Paris 2.0 "solutions" reflect the blind spots in his thought process.

Before determining that he has all of the firepower to solve all of the big problems in AI, Dario Amodei should first consider expanding his own context window.

https://www.zerohedge.com/ai/dario-amodeis-brain-trust-bad-ideas