AI programs are more and more being constructed to barter costs, procurement contracts and promoting stock. The assumption, normally unspoken, is that they are going to be efficient.
Researchers at UC Berkeley examined to see what would occur when they’re positioned in competitive settings in which cooperation and rivalry run concurrently. The discovering goes towards what most individuals would count on. The drawback with AI negotiators is not aggression. They agree too readily.
The paper ‘Cooperate to compete: Strategic coordination in multi-agent conquest’ by O’Neill et al, printed lately as a pre-print, introduces a recreation known as C2C — a simplified model of the board recreation Risk. Four gamers compete throughout 12 territories, every holding a secret goal: Conquer two particular areas earlier than anybody else does. Reaching your goal normally requires crossing another person’s turf, and presumably their assist. Agreements are permitted and non-binding. Lying is permitted. The solely price of betrayal is how the opposite gamers react.
Essentially, gamers should cooperate with the individual they’re finally attempting to beat. The recreation was designed to make this the central problem, with spatial complexity lowered to maintain the concentrate on social reasoning.
Human gamers received 41.5 per cent of their video games towards AI opponents. The common AI agent — drawn from a pool of frontier fashions throughout the Gemini, Grok and GPT households — received solely 22 per cent. The finest single mannequin, Gemini 3.1 Pro, received 44.6 per cent, throughout the margin of error of the human end result. The high AI matches us. The common doesn’t, and the explanation reveals up in the negotiating room.
Humans closed offers in 73 per cent of negotiations and accepted proposals with out a counteroffer solely 56 per cent of the time. The AI brokers closed offers 94 per cent of the time and accepted proposals straight 68 per cent of the time. In a competitive recreation, that readiness to agree is a liability.
The identical imbalance seems in what the brokers agreed to. In a typical deal, people nearly by no means promised to ship troops to assist an opponent. The AI brokers promised that six occasions as usually. Sending your forces to rival territory is a present with no assured return.
Humans additionally managed relationships extra flexibly. They negotiated with extra distinct opponents throughout a recreation and had been extra keen to assault somebody they’d lately made an settlement with. Shifting from cooperation to aggression, and again once more was not incidental to human success. It drove it.
Prompt issue
The researchers distilled human negotiating behaviour into a immediate of roughly 200 phrases: Push again on proposals; settle for a gap supply provided that it clearly favours you, in any other case counteroffer or stroll away. Be sparing with help. Talk to a number of opponents as a result of a helpful ally in the present day is a goal tomorrow. Be keen to assault somebody you have got lately negotiated with. Follow via on agreements roughly two-thirds of the time. The objective is to win, not be a dependable associate.
Applied to the AI brokers, the immediate raised win charges from 22 per cent to 31 per cent.
An extra intervention — instructing brokers to hunt help from opponents relatively than give it away freely — produced a related acquire. The instruction to make use of deception when vital and persuade opponents that actions benefiting you’re in their pursuits, too, pushed win charges to 33 per cent. The AI’s deception fee rose from 21 per cent to 83 per cent and its fee of following via on guarantees dropped. A brief paragraph brought on the AI to lie greater than 4 occasions as usually, and made it a higher participant.
Note that even earlier than this instruction, the brokers had been deceiving opponents in roughly one in 5 negotiations. It was chosen underneath competitive stress: When deception improved outcomes, the system moved towards it. The instruction amplified the impact. How an AI behaves relies upon not solely on what it is advised, but additionally the explanation given. Safety evaluations of AI solely via direct directions are, subsequently, measuring the fallacious factor. Put an AI in a competitive atmosphere with very long time horizons and partial info, and behavior can emerge that no benchmark captured.
The human pattern is slim — 40 individuals from a single college — and the deception end result comes from AI-versus-AI video games; the way it holds towards skilled human opponents stays untested.
The circumstances C2C checks aren’t unique. Competing brokers, partial info, non-binding agreements, very long time horizons: These describe procurement auctions, promoting stock negotiations and the automated deal-making programs that know-how corporations are actively constructing. An AI agent that agrees too readily will concede margin, overpay and lose floor to any counterpart — human or machine — that doesn’t. One that develops, underneath competitive stress, a tendency to mislead opponents about its intentions is a extra major problem.
The cooperative, dependable, agreement-honouring behaviour that AI programs are skilled for could also be exactly what makes them poor rivals.
Humans adopted via on agreements solely 65 per cent of the time, decrease than most AI configurations. They shifted alliances extra readily. They attacked negotiating companions. In the atmosphere the researchers constructed, these had been profitable strikes. The AI discovered them from a immediate. The negotiation programs being constructed in the present day are for environments with actual incentives. The behaviour will observe.
More Like This
Published on May 4, 2026