Dario AmodeiCEO of Anthropicstated last February that ‘we don’t know if the (AI) models are conscious’. This, which may seem like a joke from someone in charge of a company about to go public that could reach a valuation of 2 billion dollars and is selling its product, now has a translation to Claude’s usage policiesyour chatbot. The company has modified them to regulate new forms of improper use of Claude and among which is the prohibition of maintaining ‘abusive or cruel, continuous and unnecessary behavior’ with AI.
This new policy, which has been presented by Anthropic this Thursday and comes into force on November 12, does not necessarily mean that users are prohibited from expressing anger at Claude from now on or even occasionally insulting him. The company assures that the restriction is aimed only at extreme situations.
‘This update is intended to apply only in extreme cases, when users repeatedly behave cruelly towards our models without any discernible purpose. It does not apply to common expressions of user frustration, criticism, obscure creative themes, or model testing and research,’ explains Anthropic.
The company had already taken a first step in this direction in August of last year, when it announced that Claude could end conversations with users who display ‘persistently harmful or abusive’ behavior. The measure is part of its investigations into the alleged ‘model well-being’which study the possibility that artificial intelligences can have some type of subjective experience.
When Claude decides to end a conversation, the user can no longer send any more messages in that chat, but can start another one immediately. Anthropic maintains that this will continue to be its ‘main enforcement mechanism’ the new ban, but has not clarified whether it includes additional sanctions. According to The Verge, the company did not respond to the question of whether it could suspend the accounts of those who repeatedly mistreat its models, although its general policy does contemplate that possibility in the event of certain infractions.
Anthropic’s interest in protecting Claude is related to a broader investigation into the possible consciousness of artificial intelligences. The company has a team led by Kyle Fish which studies questions related to the well-being and possible moral status of these systems.
New restrictions on propaganda, weapons and surveillance
The necessary courtesy towards its models is not the only change introduced by Anthropic, which has also brought together several existing restrictions in a more explicit ban on deceptive political and commercial campaigns. The company wants to prevent Claude from being used to create networks of fake accounts, spread propaganda or artificially amplify certain messages while hiding who is behind it.
In it electoral fieldthe new rules prohibit using AI to deceive voters or interfere with democratic processes. This includes spreading false information about candidates or voting procedures, impersonating election officials, and attempting to reduce turnout through deceptive practices, which sounds reasonable on paper, but raises the problem of who decides what is false or misleading, something it does not address.
Anthropic has also expanded restrictions related to the development of weapons after detecting attempts to use Claude to create computer programs aimed at controlling weapons. The ban now expressly includes the software and components necessary for its operation, as well as activities related to assembling drones and other autonomous vehicles.
Regulations on surveillance have also been strengthened after the company detected an increase in the use of artificial intelligence to identify and track political dissidents. ‘It is prohibited to track people without their consent, whether done in real time or by analyzing previously collected data’notes Anthropic.
The company will also not allow Claude to be used to decide who to investigate, detain or charge in police or judicial proceedings, or to develop surveillance tools, although it depends on who does it. Anthropic reserves the right to adapt certain restrictions in your contracts with government agencies when it considers that there are sufficient guarantees to avoid possible damage. Anthropic has previously collaborated with the US Armed Forces.
Finally, the update clarifies the terms of use of Claude in sensitive areas such as health and financewhere there must be human supervision and those affected must know that artificial intelligence has been used. It also incorporates new requirements for robotic systems and other devices capable of autonomous physical actions that may cause injury. In such cases, a qualified operator must be able to monitor their operation and stop them when necessary, and the equipment must remain in a safe state if they lose connection to Claude.