While Deloitte Is Getting Roasted For ‘Artificial Incompetence,’ KPMG Makes Its Errors the Old-Fashioned Way

As you may have heard and subsequently made fun of, Aussie Deloitte got caught using a free-wheeling AI in an IT systems review that cost an Australian government department $440k (approx. $290k USD) and is now refunding $98,000 of that fee in penance. Deloitte’s original report included made-up quotes from a federal court judgment and referenced academic papers that don’t exist. The firm’s revised report added a disclosure that Azure OpenAI helped in writing the report.

Labor senator Deborah O’Neill, who couldn’t possibly be characterized as a fan of Big 4 firms’ work in general, told AFR that “a partial refund looks like a partial apology for substandard work.”

“Deloitte has a human intelligence problem,” she said to AFR. “This would be laughable if it wasn’t so lamentable … too often, as our parliamentary inquiries have shown, these consulting firms win contracts by promising their expertise, and then when the deal is signed, they give you whatever [staff] costs them the least.” Spitting facts as always, that plucky senator.

Greens senator Barbara Pocock, also not a big cheerleader for professional services firms, had some things to say too. “This isn’t just artificial incompetence, it’s wilful negligence. The case for outsourcing government work is to pay for a level of expertise that is not available in the public sector,” she said. “Yet the work they produce is costly corner cutting that bypasses the experts.”

“Deloitte’s work would not pass a first-year university assessment test … If contractors outsourced to do government work are further outsourcing that work to AI – it needs to be checked – that’s not too much to ask.”

Anyway, while AFR continues to dig into the details to ensure egg remains on Deloitte’s face for as long as possible (shame is a great motivator for good behavior after all), another firm is taking some jabs for its own error-ridden report, a mistake that likely would have quietly gone mostly unnoticed had Deloitte not brought error-ridden consulting reports into the spotlight.

It seems KPMG has trouble getting its citations right, too. Except in this case, KPMG insists it was humans who screwed it up and not a group of humans incapable of triple-checking AI output:

It was paid more than $60,000 to complete a review for the National Health and Medical Research Council in 2023 – which it duly did, spewing out a 64-page “policy scan” on “information about research integrity arrangements outside Australia”.

It did not invest as much time scanning its own such arrangements, though. There was a phantom reference in the report. The book it referred to existed, and the authors it cited were real people. But they never co-wrote the book chapter that KPMG claimed. The chapter wasn’t even real, despite the consultants earnestly including (fake) page numbers for it.

KPMG was alerted to the problem (by the so-called authors of the so-called chapter) last January. It investigated, spoke to the government, and quietly updated the report.

Absolutely not through AI, it turns out. KPMG said it didn’t use AI for any of the report. It instead pointed to human error.

We buying this? Slapping fake page numbers on something you know will be seen by the client seems…exceptionally sloppy.

Seems a good use of AI right now would be to run a bunch of consulting reports through it to identify errors, made-up academic papers, and fabricated quotes. For the LOLs, that is.