AI tokens getting cheaper but businesses paying more than ever

AI costs are rising for Australian businesses even as tokens get cheaper In short: Businesses are looking for ways to measure the return on the billions being spent on AI. Some say they are "years away" from judging bottom-line impacts. In this story: Australian businesses paying to use AI are weighing up what value they are getting from the disruptive technology. "It's actually only really a relatively new thing that we have started to speak about," Anna Volkova, the head of people and culture at HR software company HiBob, told ABC News. The platform has seen wins from using artificial intelligence, but it has been difficult to judge that against the mounting cost. With AI capabilities changing so quickly, Ms Volkova said the business can "never catch up" with the need to redesign how it works to get the most out of it. This has made it hard to work out precisely if the money is being spent effectively. "We'll probably start to see impact to our bottom line in maybe two or three years, and I think most businesses are in that boat." Jeremy Pell, whose firm Elastic helps companies to manage the cost of AI, agreed with that estimate. "We're in a really interesting time at the moment," he said. "Businesses are forced to pay this additional investment into AI, but they haven't yet reaped the 'bottom-line' benefits of the cost savings yet." Last week, Assistant Minister for Technology and the Digital Economy Andrew Charlton described the value flowing out of Australia via AI as a "distant sucking sound". In a speech at the ANU Crawford School of Public Policy, he said Australians were already spending $5-8 billion each year on artificial intelligence. "With the overwhelming majority of those payments flowing offshore, in effect, we are creating a new import bill. An import bill for intelligence." That is just the start, with the government estimating local spending on AI could "plausibly" reach $20 billion to $40 billion annually within a decade. Tokens adding up to billions Even a few years ago, AI was not a line item companies had in annual budgets. Now, in the current reporting season for ASX-listed companies, you can measure it in billions. The technology spend at Australia's biggest bank, the Commonwealth Bank, increased from $2.3 billion in the 2025 financial year to $2.4 billion in this most recent financial year. It lists savings too. "Measured gross benefits from AI use cases, including reinvested capacity, were [around] $200 million in FY26," its results presentation noted. "Gross benefits from AI use cases are expected to double in FY27, and to exceed investment." On Wall Street, investors are asking whether the billions invested by US tech giants in AI will ever be justified by returns, creating concern about an AI-fuelled stock market bubble. Stu Scotis, AI leader for consulting firm Deloitte, said the equation of how to spend an IT budget is getting tougher. Clients are weighing up long-term information technology costs — like storage and refreshing laptops — with a new, runaway expense. "What we used to plan for as a static activity is now somewhat variable," he said. Mr Scotis said current conversations are around "tokens": units of data processed by AI, or essentially the amount of grunt used to compute tasks. Complex requests use more tokens, simple ones fewer. The cost of tokens has plummeted in recent years, with increased competition, more efficient chips and more supply. An unprecedented amount of data centre construction has increased the computing power available in the world. Adding in more energy-efficient microchips, increased competition and the technology improving how much it can compute per token, it is now cheaper than ever. As the below chart from research institute Epoch shows, the cost of the most powerful models has fallen the fastest. Epoch used the "Massive Multitask Language Understanding" benchmark to rank AI large language models. The price for large language models GPT-4 Turbo level or better (the most powerful, shown by the blue line) fell from $US15 ($20.90) for one million tokens in November 2023, to 17.5 US cents (24.4 cents) per million in February last year. Despite this trend, what companies are spending on AI is rising. That is because firms are finding new tasks for AI to take on, leading to increasingly difficult computations, requiring more tokens per request. AI not necessarily cheaper than workers Mr Scotis put it another way: Human workers are paid in dollars, digital workers — or AI models — are paid in tokens. "You want to get value for the digital workers and their output," he explained. That means companies investing in AI need to examine the "value, outcome or growth" that comes from the tasks they complete, just as companies evaluate the work of human employees. "You start to get pretty clear, measurable outcomes that mean your investment in AI dollars or AI tokens actually pay a return." He said the human workers should not be too worried about being replaced. "The cost of a digital worker doesn't necessarily mean it's cheaper than a human," he said. "The complexity of the task, the length of time which a digital worker runs can, in some instances, be more expensive than a person." Mr Scotis said firms needed to make a judgement "as to where you want to invest that type of work in your organisation, and does it make sense to do so". It is a question being weighed in boardrooms right now. How do companies pay for AI? There are multiple ways that companies pay for artificial intelligence. Some use "seats", paying per employee that accesses the technology. This model is often used with platforms like Microsoft's Copilot, where AI abilities can be unlocked in existing programs, like writing documents in Word or juggling numbers in Excel spreadsheets. The most common way to pay is by tokens. The frequency with which employees are using AI models and the amount of tokens used in the queries is a new and increasing cost for businesses — which many of them will not have been paying for in even last financial year's report. "That's the cost metric these businesses are grappling with," said Elastic's Jeremy Pell. Different model, different prices Another variable is that some of the cutting-edge or "frontier" AI models use more tokens. That has led to companies using different models for different tasks, to avoid paying top dollar for tasks a more basic model can handle. Some firms have also become more deliberate about the use of AI, sometimes capping the amount of tokens employees can use in a certain time frame. "Businesses can no longer do AI for AI's sake,"Mr Pell said. "It's a matter of making sure that you are driving the business forward and these initiatives are driven and tied to the business's return on investment … It's working out which bits are useful." 'Scattergun' AI approach AI expert Jon Whittle has warned the "scattergun" approach business has taken to using AI means the cost calculations are becoming more difficult, even as the price of tokens falls. "I don't think companies are being very strategic right now about their AI use cases," he told The Business. "They're just trying to find any place where they can use it, and that's why they're having costs increase so much." Professor Whittle, a former technical lead at NASA, director of CSIRO's AI capability and founder of Australia's National AI Centre, said a particular issue was that AI itself did a poor job at estimating the amount of tokens required. Research has shown estimates can differ "by a factor of 30" running the same request on the same technology. "I think that's the number one thing keeping [chief financial officers] up at night right now,"Professor Whittle said. "It turns out that even the AI itself is not very good at predicting the cost that a particular AI task will take." Speed increases as AI crunches data faster From her Sydney office, Anna Volkova says the major advantage her firm currently gets from AI is speed. By connecting artificial intelligence to the software tools the company already uses, staff have been able to do more, at greater pace. But the bill is mounting. She said, eventually, the business would likely need to evaluate usage "by need", not "limiting responsible experimentation" but looking for uses that maximise what staff get out of using AI as a tool. "I think the things that we're looking for in terms of really seeing the return for our people … is it making their work better, easier?" she asks. "Is it helping our leaders make decisions faster?" It turns out the questions business leaders are asking about AI are very human. Watch The Business, Mondays to Thursdays 8:44pm on ABC News Channel, after the Late News on ABC TV, and anytime on ABC iview