Artificial Intelligence – Regulatory Capture

Industry leaders cry wolf again

• Hot on the heels of yet another warning of killer robots massacring the human race, Anthropic has called on the industry to slow its development, which at best looks like regulatory capture and, at worst, like an attempt to deliver better profits right before the IPO.
• This kind of event is nothing new, as GPT-2, o1, Fable and “thinking models” were all considered too dangerous to release, but then turned out to be simply more of the same, albeit with good improvements in performance.  
• This latest brouhaha started when an employee resigned from Anthropic and “blew the whistle” on superintelligence development at Anthropic and OpenAI, claiming that they are ”gambling with our lives”.
• The issue here is what’s known as recursive self-improvement, which means that the models get so good that they can train themselves to reach superintelligence.
• However, the employee failed to provide any evidence of what he had seen or experienced that drove this opinion, which immediately makes me suspicious, as we have seen this playbook before.
• Furthermore, there are plenty of groups and countries that would be quite happy to see the US shoot itself in the foot on AI.
• This was followed up by a 3,800-word essay from Dario Amodei, CEO of Anthropic, calling for a coordinated slowdown in AI development so that the appropriate safeguards can be put in place.
• The other contenders for AI leadership immediately chimed in with their agreement, but there was nothing from the fast followers such as Meta or any of the Chinese labs.
• I think that this is because the leaders have the most to gain from a coordinated slowdown, because something like this will inevitably lead to some form of government involvement and regulatory capture.
• One only has to look at the testimonies given before Congress by some of the technology companies to see that politicians know nothing about AI, and any regulator that is set up will lean heavily on the leading AI companies for guidance on how to regulate.
• This will give the leading companies effective control of the regulator, meaning that they can make life very hard for anyone who threatens their business.
• The biggest problem here is open source, which is a community of developers from all over the world, especially China, who are going to pay no attention at all to any rules.
• This could lead to open source being banned by an AI regulator for non-compliance, cutting off a major source of competition for the frontier labs.
• However, the really cynical interpretation is that if everyone were to pause or slow model development, then R&D costs would fall temporarily, meaning that everyone would become more profitable for a period of time.
• Long-term growth would, of course, suffer, but as the market is more concerned with the next quarter, this would be a way to flatter profits and cash flow to ensure a successful IPO.
• This comes just as the FT (see here) is reporting that Anthropic is telling investors that it is currently profitable, which strikes me as a strange coincidence.
• The net result is that I remain very sceptical about the motivations that lie behind these sorts of pronouncements, as all of these models are still demonstrably very far from any form of superintelligence.
• This is what the visible evidence tells me, particularly as previous claims of models being “too dangerous” have repeatedly come to nothing.
• Many people who work at these companies are much smarter than me, and so I am sure that they are aware of these shortcomings, meaning that there must be another reason for these sorts of statements and the tidal wave of press coverage and hand-wringing that inevitably follows.
• I think that it is unlikely that anything is going to change, as the Chinese are not going to slow down, meaning that the USA will be giving away the lead that it has if it unilaterally hobbles itself in the AI race.
• This is also not the way to have a successful IPO, and if these were genuine concerns, I would expect Anthropic to delay its IPO until such time that the risks are clarified and the appropriate actions taken.
• I see no such delay, and it is by the company’s actions as opposed to its words that it should be judged.