CPS cited 'non-existent' legal cases generated by AI, judge claims

CPS cited 'non-existent' legal cases generated by AI, judge claims The Crown Prosecution Service was forced to apologise after court documents prepared by AI were found to have 'hallucinated' a number of fake legal cases A senior judge has warned of the risks of using AI to prepare for court hearings after a number of documents were found to contain references to "non-existent" legal cases. Mr Justice Sweeting made the comments on Wednesday where he dismissed the appeals of two people challenging decisions to extradite them to Romania to serve sentences for offences committed there. The Crown Prosecution Service (CPS) opposed the appeals at the High Court, but at the end of the ruling Mr Justice Sweeting revealed that two legal authorities cited by the CPS had been 'hallucinated' by an AI and did not actually exist. He continued that the two cases were then cited in another document, before the issue was raised with the CPS prior to a hearing in February. The CPS acknowledged and apologised for the error, stating that it had not sought to mislead the court and that inquiries were being undertaken into how the error was made. The judge said that, in a letter in March, the CPS accepted that the references to the cases were inaccurate and “explained that they were likely to have originated from the use of artificial intelligence”. “However, the CPS identified the critical failing as the fact that the reviewing lawyer did not properly check the accuracy of the document before it was filed and served,” the judge said. He continued: “The position of the CPS was therefore that, while the immediate source of the error may have been the use of generative artificial intelligence, the operative cause was human error in the failure to verify the authorities relied upon in formal submissions placed before the Court. “The CPS emphasised that this was not a deliberate attempt to mislead, but rather an isolated incident arising from inadequate checking of written work.” Mr Justice Sweeting also said that the CPS had conducted a full internal review into the incident and reviewed 78 other cases linked to the same lawyer, which identified no concerns and led it to believe there was a low risk of it happening again. He concluded: “As far as the appeal was concerned, the errors came to light prior to the hearing and therefore had no impact on argument or the court’s judgment. “I have accepted the apology given on behalf of the CPS and the assurance that there was no attempt to mislead.” He continued: “It would be naive to assume that there will not be an increasing use of artificial intelligence in legal work in future; indeed, that may be both necessary and beneficial. “The episode highlights the risks of its use without appropriate oversight, particularly for legal research.” In a statement, a spokesperson for the CPS, which does not use AI in legal decisions or when determining charging outcomes, said: “We apologised to the court for this error made in legal submissions. “As soon as this was identified, we carried out an internal review to establish how this occurred and to check no other cases were affected and that any learning was identified to prevent a recurrence. “The judge has made clear the error had no impact on legal arguments or the court’s eventual judgment and accepted our apology.”