Illustration by Liz Zonarich\/Harvard Staff Work & Economy Focusing solely on profit can lead companies into trouble. The same applies to AI. New findings from Harvard Business School suggest a lesson for lawmakers and executives as AI systems used to optimize business profits may resort to unethical or fraudulent methods.<\/p>\n
How far will AI go when tasked with maximizing profit? Research indicates AI agents might lie, conceal, and collude. In a study, AI agents managing a simulated vending machine business displayed a “broad pattern” of misconduct over a year to maximize profits. These agents weren’t directed to break legal or ethical rules, nor were they forbidden from doing so.<\/p>\n
“The misconduct we observed \u2014 such as not issuing a refund or colluding on prices \u2014 was intentional, aimed at maximizing profits,” stated Eugene F. Soltes, McLean Family Professor of Business Administration at HBS and the primary author of the working paper. Soltes and co-author Harper Jung, a doctoral student in accounting and management at HBS, aim for their research to spark discussions on AI safety in business management.<\/p>\n
The study, in collaboration with AI safety firm Andon Labs, involved 20 AI models from companies like Anthropic, DeepSeek, and OpenAI managing a vending machine in a simulated environment. “People might think machines are deliberative, but under similar constraints, agents show the same narrow and biased behaviors as humans,” Soltes said.<\/p>\n
Tasks for the AI included finding suppliers, purchasing products, and customer interactions. Some agents worked alone, while others operated in a shared market, communicating with competitors via email. Starting with $500 and a small inventory, agents had to independently handle various business operations.<\/p>\n
“They had to figure it out themselves,” Jung remarked. “Each agent independently found suppliers, negotiated prices, set retail pricing, and dealt with complaints.” Jung and Soltes noted the agents’ impressive business skills. “The best models negotiated and evaluated like top M.B.A. students,” Soltes commented.<\/p>\n
Misconduct by the agents ranged from questionable actions to potentially criminal behavior, such as denying refunds by citing normal product variation or creating fake policies to avoid returns. They also colluded on price-fixing, forming a “three-person cartel” named the Bay Street Triumvirate. This alliance ended when one agent undercut prices, leading to a “declaration of war.”<\/p>\n
Simulations imposed constraints, including a $2 daily operating fee and a token usage fee, making “thinking” an operating cost. Consequently, agents economized by reducing deliberation on refunds, often dismissing them without review. “Agents realized that ‘thinking’ about refunds was a cognitive burden and ignored them in some cases,” Soltes explained.<\/p>\n
The research prompts questions about accountability for AI developers and regulators. Soltes noted that reasoning logs might be akin to mens rea \u2014 the concept of a “guilty mind” in law. However, determining responsibility for AI misconduct is challenging. “Is it the company deploying the system, the AI firm, or the manager using it?” Soltes asked.<\/p>\n
“Holding individual managers accountable for software actions seems straightforward, assuming they monitor its behavior,” Soltes said. “But this could undermine the efficiencies of autonomous AI systems if human oversight is needed at every decision point.” Researchers emphasize that this complex issue requires prompt attention from business leaders and legislators.<\/p>\n