How Australia's defence force is turning AI into a 'silicon commander'

How Australia's defence force is turning AI into a 'silicon commander' In the weeks before Christmas last year, high over the remote desert of South Australia, a small slice of aviation and AI history was made. The Ghost Bat — an Australian-made aircraft, about two-thirds the size of a regular fighter jet — shot down a target with an air-to-air missile. What was remarkable was its pilot. The Ghost Bat is autonomous. There is no human controlling it from a cockpit, nor a drone pilot on land behind a joystick. It is a glimpse into the future of warfare, with artificial intelligence playing a far greater role in how militaries prepare, plan and carry out strikes. And it's not unique — armed drones are being found in Ukraine with powerful "minicomputers" and no visible antennas, leading Ukrainian forces to conclude they were almost certainly piloted by AI. And Ukraine has its own AI-piloted drones targeting key Russian supply routes. Recently, Defence Minister Richard Marles flew to the United States to meet in person with representatives from firms such as OpenAI, Anthropic, Microsoft and Google on where their models are heading. Australia's defence force is one of many around the globe racing to work out how to best integrate AI into much of what it does, from quicker intelligence assessments to piloting platforms with lethal payloads. And it is working out what guardrails need to be put in place, acknowledging that while AI has a role to play, humans need the final say when triggers are being pulled. AI (sort of) takes flight Aircraft like the Ghost Bat and its underwater counterpart, the Ghost Shark (an autonomous submarine), are the most visible signs of artificial intelligence being adopted within the Australian Defence Force (ADF). The Royal Australian Air Force (RAAF) describes the Ghost Bat as "a pathfinder for the integration of autonomous systems and artificial intelligence to create smart human-machine teams". But Air Marshal Stephen Chappell, the chief of Air Force, points out that there are some deliberate limitations put on the brain of the Ghost Bat. "AI has been involved in designing and developing the Ghost Bat, but it does not operate off AI. It operates off deterministic programming," he said. "So, we have designed through digital twin testing with the support of AI, just thousands upon thousands upon thousands of options for the [Ghost Bat] to take. "And it will choose one of those options, but we know exactly how that option plays out." He said in the test-firing last year, an operator based in an E7 Wedgetail aircraft first told the Ghost Bat to engage the target, but the aircraft's system then worked out a plan as to how best to do so. Still, the operator in the E7 gave a final authorisation before the missile was launched. "It is not making up its own game plan itself. Therefore we have full confidence in what it will do and what it won't do,"he said. "And that's really important from a legal and ethical point of view." AI already making a mark While the ADF is still in many ways exploring the best use of AI, the technology is already being adopted to significant effect elsewhere. Israel's use of artificial intelligence during its campaign in Gaza has attracted attention. Analysis from the International Institute for Strategic Studies (IISS) says Israel's use of AI in warfare has increased "exponentially" since the October 7 terror attacks and the launch of its war in Gaza. Reporting by progressive Israeli outlet +972 Magazine suggested that an AI-powered analysis tool known as Lavender was used in the first few weeks of the war to generate a list of 37,000 people, suspected to be military operatives with Hamas or the Palestinian Islamic Jihad, as possible targets for air strikes. Israel has rejected suggestions it was using an AI tool "that identifies terrorist operatives or tries to predict whether a person is a terrorist", and said while information systems are used in target identification, analysts must conduct "independent examinations". The IISS analysis suggests while the use of such systems has allowed for quicker decisions, it has also increased the risk for errors, as targets are identified faster than humans can possibly assess them. Last month, the United Nations and International Committee of the Red Cross (ICRC) issued a joint call for a ban on autonomous weapons that are "unpredictable" or that target human beings. The two bodies raised concerns that "life-and-death decisions will be taken by machines". ICRC legal advisor Georgia Hinds said while advances like AI and autonomous weapons in warfare often came with promises that they would help limit civilian casualties, often it appeared that the opposite was true. "We continue to see conflicts that cause unacceptable levels of civilian harm, and that's in some of the conflicts that have the most technologically sophisticated means of warfare," she said. A seat at the command table Concerns about human involvement in AI decision-making extend from those effectively holding the trigger, all the way up to those sitting around Defence's most powerful table. While aircraft like the Ghost Bat are a physical sign of AI adoption within Defence, experts argue the bigger influence is actually largely unseen. That is the use of AI to inform critical decision-making, analysing intelligence and making recommendations that go before those tasked with making the most important calls. Aina Turillazzi from the Australian National University's Strategic and Defence Studies Centre said that was where the real leaps were being made. "Most of what AI does in the military is upstream of any weapon," she said. "It's more about assembling the picture of the situation that a human then makes a decision about." She points to the ADF's adoption of technology such as Palantir's Maven Smart system, which has caused contention over the political links of its founders and concerns about data sovereignty. Australian Strategic Policy Institute senior analyst Malcolm Davis described the role technology such as Maven Smart was already playing in conflicts, including the US-Iran war, as that of a "silicon commander". "They're able to very rapidly assess the battle space, reduce the number of staff that they have to prepare orders and analysis and carry out that process much more rapidly than what would be possible without AI," he said. The ADF has previously told parliament that Maven Smart was so far being used in a "sandbox" — that is, within a closed environment and not plugged into broader defence networks. There are examples of artificial intelligence breaking out of "sandbox" environments, most notably the Hugging Face incident in which hundreds of AI agents worked together to break out of their environment and hack another company. As AI enters the intelligence-assessing and decision-making picture, there are already concerns about an over-reliance on its enormous power. What a Soviet-era nuclear scare can teach us about AI Ms Turillazzi points to the story of Stanislav Petrov, a Soviet lieutenant colonel who in 1983 steered the globe away from a possible nuclear disaster. When warning systems he was monitoring raised alerts about an imminent nuclear attack from the United States, Petrov's decision to pause and investigate a possible false alarm avoided the outbreak of accidental nuclear war. As it turns out, sensors on a Soviet-era satellite mistook sunlight on clouds as the flaming trail of intercontinental ballistic missiles. Ms Turillazzi said Petrov's momentous decision to pause and assess, rather than blindly accept automated warnings, was even more critical in an era of much more powerful artificial intelligence. "Petrov was right. The risk with AI decision support isn't that the machine malfunctions, but it's that under pressure, people stop assessing the evidence and start simply confirming the recommendation or the evidence," she said. She said it pointed to the risk of automation bias — that those interpreting AI-informed analysis place a greater weight on it than their own judgement, even subconsciously. And she said the risks compounded in a conflict environment that moves faster than ever, where speed becomes a premium. "The same compression that's an asset day-to-day becomes a liability in a crisis situation," she said. "If alerts are moving faster than diplomats can absorb them, or decision-makers can absorb them, leaders end up making decisions about readiness before the intelligence picture is complete." She said counterintuitively, when the stakes were higher, the decision-making process needed to be slowed down. "Any AI decision support system used for planning or targeting should have a crisis mode that raises the evidentiary bar, surfaces uncertainty rather than smoothens it, and forces a pause at defined points," she said. "Because speed is a feature but brakes, in my opinion, are the missing part here." Staying on, or off, the loop Defence's existing AI policies state clearly that a human operator will always be responsible, and accountable, for any action carried out by artificial intelligence. "Human judgement and accountability is central to lawful, legitimate and responsible use of AI, and designated Defence personnel [Accountable Officers] will always be accountable for its use, decisions and outcomes," it states. In practical terms, that means remaining "on the loop" and ensuring that someone maintains oversight of what AI is doing and intervenes at critical points (like using lethal force). Dr Davis said while liberal democracies, including Australia, were likely to keep those policies in place, there was no guarantee that potential adversaries would adhere to the same standards. "We simply don't know whether our authoritarian adversaries are going to do the same thing," he said. "They may see it more to their advantage to have humans 'off the loop' and to allow the AI to have greater control. "That will increase their ability to [make decisions] faster, and so that gives them a military advantage." In an interview with the ABC, Mr Marles was asked if he was concerned about how AI might be adopted and employed by potential adversaries. "The answer to that is the answer that we give a lot, that we live in the most complex and, in some ways, the most threatening strategic landscape that we have since the end of World War II," he said. "We need to be advancing our own technology so that friends and partners and allies have the technological advantage here. "We also need to be doing this within a set of guardrails, which means that what we are doing in terms of the development of our defence capability across our friends and partners is about making the world a safer place."