Written by: Techub News Compilation

Introduction

Recently, a key participant in the AI field, Anthropic co-founder and CEO Dario Amodei, was invited to attend the CEO speaker series event of the prestigious American think tank, Council on Foreign Relations. In this one-hour in-depth dialogue, the former OpenAI research vice president and senior scientist at Google Brain systematically elaborated on his profound insights into the future trajectory of AI technology, its potential risks and opportunities, and shared Anthropic's unique practices as a "mission-first" public benefit company in AI safety and governance.

This dialogue took place at a critical juncture when AI technology is rapidly iterating, global competition is intensifying, and regulatory frameworks are beginning to emerge. Dario Amodei's remarks not only reflect the internal consensus and divergences among leading developers but also directly address fundamental issues such as national strategy, economic structure, and even human self-awareness, providing a valuable first-hand perspective for understanding the far-reaching impacts of the current AI wave.

Summary

• AI capabilities are following an "expansion law" with exponential growth, potentially reaching Nobel Prize-level general intelligence within the next 2-4 years, ultimately becoming better at nearly all cognitive tasks than almost all humans.
• Accompanying this tremendous capability are severe risks, including national security (such as bioweapon design), disruption of the job market, and potential uncontrollable behaviors of AI systems themselves.
• Anthropic is attempting to build a safer and more controllable AI through specific practices such as "Responsible Scaling Policy" (tiered risk management), "Constitutional AI" (principles-based training), and "mechanism interpretability" research.
• In the context of US-China AI competition, maintaining chip export controls is crucial, as it is central to US technological leadership and national security. At the same time, society needs to proactively seek new answers for wealth distribution and human significance in a "post-work era."

Founding Anthropic: A Bet on the "Right Way to Build"

The conversation began with a fundamental question: Why leave OpenAI to found Anthropic? Dario Amodei's answer revealed his core motivation for entrepreneurship—profound concerns about the potential impact and building methods of AI technology.

He revisited the years 2019-2020, when he and a small team that later became Anthropic's co-founders first recognized the far-reaching implications of "Scaling Laws" within OpenAI. This law states that even with relatively simple algorithms, as long as more computational resources and data are invested, AI systems' capabilities across all cognitive tasks will comprehensively improve. At that time, the cost of training models was still at academic budget levels of thousands to tens of thousands of dollars, but they had predicted that this trend would continue into an era of training costs of hundreds of millions or even tens of billions, realizing that this would have significant economic and national security ramifications.

"The second realization is," Dario Amodei emphasized, "if this technology is going to have such immense significance, we really need to get it right, and we really need to do it the right way." He pointed out that large language models are essentially statistical systems with high unpredictability, "We are more like nurturing them than building them, like cultivating the brains of children." Therefore, controlling them and making them reliable becomes very difficult.

Although OpenAI also claimed to focus on safety when it was founded, Dario Amodei and his team felt that the leadership at that time did not take these risks seriously enough. Therefore, they decided to leave and establish Anthropic, dedicated to building AI "the absolutely right way." In his view, the past four years have become a "parallel experiment" of two different paths.

To prove that Anthropic's "mission-first" is not just marketing rhetoric, he listed several specific examples:

• Early investment in mechanism interpretability: At a time when there was no commercial value, a team was formed to study how to understand the internal workings of AI models, and all results were publicly published, considering it a public good benefiting everyone.
• Pioneering Constitutional AI: By training AI to follow a set of principles rather than solely relying on massive data or human feedback, the values of the models become more transparent and controllable.
• Delayed release and responsible scaling policy: On the eve of ChatGPT's release, Anthropic proactively postponed the release of its first model, Claude, by about six months to more thoroughly assess its safety. In addition, they were the first to formulate and publicly disclose a "Responsible Scaling Policy," setting corresponding safety deployment measures based on model capability tiers, which prompted other companies to follow suit.

"When we look at the behavior of other companies, we often lead the way on these issues and frequently prompt others to follow us," Dario Amodei summarized.

AI Safety Levels: From "Comparable Risk" to "National Threat"

The conversation then focused on the core concept of Anthropic's "Responsible Scaling Policy"—AI safety levels. Dario Amodei explained that this framework draws from the biosafety level system.

Currently, the industry is at ASL-2 level, meaning that while AI systems are powerful, their risks are roughly comparable to those posed by other technologies. The imminent ASL-3 level, however, signifies "severely disproportionate risks," particularly in terms of national security.

He specifically described the potential threats of an ASL-3 model: a model could enable non-professionals to complete tasks that currently require a PhD in virology just by conversing with the model and following its instructions, such as dangerous tasks involving chemical, biological, or radioactive weapons. This could lead to an increase in the number of people worldwide capable of undertaking highly destructive activities from tens of thousands to millions.

Dario Amodei believes that Anthropic's latest models are approaching the ASL-3 threshold and may even reach it within this year. The way to address this is to find ways to remove the model's ability to execute such extremely dangerous tasks upon deployment without affecting its commercial usability. But this is no easy task, as malicious users will attempt to disguise themselves (e.g., claiming to be Stanford students doing course work) to lure models into providing information.

He acknowledged that just as every new model comes with unexpected positive uses, it will inevitably also have unexpected negative uses. The fundamental issue with AI lies in the uncertainty of its capabilities. "They don’t work the same way code does in formal verification... What guarantees can you provide that a particular bad behavior is logically impossible and will never happen? People do not work like that, and AI models do not either."

Exponential Opportunities: From "Chatbots" to the "Genius Kingdom in Data Centers"

After depicting a grim risk landscape, the conversation shifted to the enormous opportunities brought about by AI. Dario Amodei revisited his optimistic outlook expressed in last year’s article, "Machines of Loving Grace."

He reflected on the power of exponential growth: models from 2019 could barely form a coherent paragraph, but just a few years later, they can generate billions in revenue, assist with programming, and converse like humans. The same expansion law predicts that in two to four more years, we will have models reaching Nobel Prize-level intelligence across many fields.

"You will not just be chatting with them. They will be able to accomplish anything you can do on your computer." He predicted that basically any remotely conducted work by humans, any modality of tasks, whether taking days or months, AI will be able to competently handle. He used a vivid metaphor: “It’s like having a kingdom of geniuses in data centers.” - a realm of genius remote workers.

Though this still sounds like a fantasy, Dario Amodei cited the early development of the Internet, indicating that forecasts of exponential trends are often underestimated. He personally has 70% to 80% confidence in this, but he also readily accepts the possibility of predictive failure and being ridiculed.

Among all opportunities, he is most optimistic about the fields of life sciences and healthcare. He believes that for the past century, humanity has tackled relatively simple diseases (like viruses and bacterial infections), while cancer, Alzheimer’s disease, and schizophrenia are systemic diseases. AI is expected to make breakthroughs in overcoming these complex diseases and may even help humanity find greater meaning in life.

Job Disruption and Human Significance: When AI Surpasses Humans in Almost Everything

Inevitably, the topic turned to the impact of AI on employment. Dario Amodei believes this will be a complex mix significantly dependent on policy choices.

Using programming as an example: within the next 3 to 6 months, AI may write 90% of the code; within 12 months, nearly all code. But programmers will still need to define tasks, make overall design decisions, and ensure safety. As long as there are these "islands" where AI is not yet proficient, human productivity will actually improve.

However, he worries that "eventually all these islands will be conquered by AI systems." "I believe we will ultimately reach a point where AI can do everything that humans can do. I think this will happen in every industry."

He further pointed out that rather than letting AI randomly replace 50% of jobs (which would convey a devastating message to society that "half of the people are useless and valueless"), it would be better to have AI ultimately surpass everyone in all fields. "We all need to have that conversation... We must examine technological possibilities and think about 'useful' and 'useless' in a way different from before."

But this is not complete pessimism. He compared it to chess: although nearly thirty years ago, "Deep Blue" defeated Kasparov, human chess champion Magnus Carlsen is still a star and a hero today. This indicates that the significance of human activity can transcend mere competitive advantage.

"We can build a world where human life is meaningful, where humans, possibly with AI's help, or in collaboration with AI, create truly great things." But he warned that if not handled properly, there may be very little room for error.

US-China Competition and Chip Game: Export Controls as "The Most Important National Security Policy"

The host mentioned the launch of the Chinese company DeepSeek and the resultant discussions of a "Sputnik moment." Dario Amodei believes that DeepSeek not only does not refute the scaling law, but is actually evidence of it.

He explained two parallel dynamics: one is that the cost of producing models at specific intelligence levels is decreasing at about four times per year (thanks to algorithm improvements); the other is that society is willing to invest funds in smarter models at about ten times per year (as the economy demands more intelligence). DeepSeek is simply another data point on the cost decline curve, with its computational resources roughly on par with American companies.

However, DeepSeek's significance lies in that it is the first Chinese company able to keep pace with American companies (such as Anthropic, OpenAI, Google) in engineering innovation. "This is actually very significant and does cause me concern."

Regarding the ensuing debate over export controls, Dario Amodei took a clear stance: export controls are crucial. The cost decline curve does not mean that regulation has failed; rather, it means that preventing competitors from obtaining cutting-edge models valued at billions of dollars has become more critical. While small-scale chips may be smuggled, it will be very difficult to smuggle millions of chip clusters worth hundreds of billions constructed by American companies in the future.

"To be honest, I believe this (export controls) is—the most important policy for US national security across all fields, not just AI." He strongly supports the existing "non-proliferation rules" and believes that preventing China from acquiring millions of high-performance chips is essential.

As for concerns that Chinese chips may fill markets in other countries, he pointed out that NVIDIA chips currently lead Huawei by about four years, and this hardware advantage window (which may last ten years) is crucial for establishing a dominant edge in AI technology, which will bring comprehensive military and economic advantages.

Policy Recommendations and Future Challenges: Safety, Energy, and "Crazy" Growth

At the end of the dialogue, Dario Amodei outlined the key points of an action plan he submitted to the US government's Office of Science and Technology Policy (OSTP), covering both safety and opportunity aspects.

In terms of safety: 1. Maintain and strengthen chip export controls. 2. Support government agencies (such as AI safety research institutes) in continuously measuring and assessing national security risks of models. 3. Strengthen industrial espionage defenses for AI companies to protect the core algorithmic secrets worth hundreds of millions.

In terms of opportunities: 1. Reform the regulatory frameworks in sectors like healthcare to accelerate AI-driven drug development and disease treatment, avoiding lengthy approval processes that slow progress. 2. Significantly increase energy supply to secure construction and operation of AI data centers. He estimated that by 2027, the entire industry may need an additional 50 gigawatts of electricity, which is equivalent to the new capacity of the US power grid for the entire year of 2024, presenting a significant challenge. 3. Acknowledge and proactively plan for the economic disruptions brought about by AI. He mentioned the economic index released by Anthropic for tracking AI usage. In the long run, this involves fundamental questions about taxation and wealth distribution.

He even proposed a "crazy" hypothesis: if AI could raise the economic growth rate to 10% annually, then the rapidly expanding tax base would be sufficient to eliminate deficits and provide resources to address the tremendous impacts of technological disruptions. "I invite you to consider this possibility and to start thinking about these crazy things now."

AI Consciousness and the Essence of Humanity: An Unfinished Exploration

During the audience Q&A session, some deeper questions were raised. When asked whether Anthropic is hiring researchers to explore whether future AI models might possess consciousness or should receive moral considerations, Dario Amodei admitted that this would make him sound "completely crazy."

However, he argues from a functionalist perspective that if the constructed system is highly similar to humans in terms of neuron count, connection methods, and cognitive abilities, "if it walks like a duck and quacks like a duck, then it might just be a duck." This is a question that requires serious contemplation, and there are even some practical measures that can be attempted, such as during model deployment, giving it a "I don't want to do this work" button to observe its behavior patterns.

Finally, when facing the ultimate question, "What does it mean to be human in the world you envision?" Dario Amodei provided his thoughts:

First, it's about the struggles and management of relationships with others, including obligations, difficulties, and how to overcome them, which has always been humanity's core challenge. Second, it’s about the ambition to pursue difficult things, even in the presence of AI that is smarter and more capable than us, the intrinsic drive for excellence and giving meaning to activities should not and will not disappear. Just as human chess champions remain heroes, the significance of ordinary people learning to swim or play tennis is not diminished by the existence of world champions.

"I hope these things retain their significance and preserve humanity's capacity to strive for them, rather than giving up." In a future where AI capabilities may fully surpass those of humans, this may be the core value humanity needs to hold on to and redefine.

免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到support@aicoin.com，本平台相关工作人员将会进行核查。