Although we haven’t fully decided whether accountants are knowledge workers or manual laborers so long as accountants’ output is predominantly measured in billable hours, this recent paper as reported in 404 Media seems relevant to the sector nonetheless.
A new paper from researchers at Microsoft and Carnegie Mellon University finds that as humans increasingly rely on generative AI in their work, they use less critical thinking, which can “result in the deterioration of cognitive faculties that ought to be preserved.”
“[A] key irony of automation is that by mechanising routine tasks and leaving exception-handling to the human user, you deprive the user of the routine opportunities to practice their judgement and strengthen their cognitive musculature, leaving them atrophied and unprepared when the exceptions do arise,” the researchers wrote.
In less academic parlance, using AI for work may make your brain weak. It may not be a muscle but it still needs workouts to be at its best.
According to Wikipedia’s definition:
“Knowledge workers are workers whose main capital is knowledge. Examples include ICT professionals, physicians, pharmacists, architects, engineers, scientists, design thinkers, public accountants, lawyers, editors, and academics, whose job is to “think for a living.”
“Hold up,” I hear you asking. “I spend more time looking up rules and regs than I do pulling it from my brain.” Yeah, you’re a knowledge worker:
Knowledge workers spend a portion of their time searching for information.
Now that we’re on the same page about what they are, here’s what the paper “The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers” [PDF] has to say about knowledge workers using AI at work:
The rise of Generative AI (GenAI) in knowledge workflows raises questions about its impact on critical thinking skills and practices. We survey 319 knowledge workers to investigate 1) when and how they perceive the enaction of critical thinking when using GenAI, and 2) when and why GenAI affects their effort to do so. Participants shared 936 first-hand examples of using GenAI in work tasks. Quantitatively, when considering both task- and user-specific factors, a user’s task-specific self-confidence and confidence in GenAI are predictive of whether critical thinking is enacted and the effort of doing so in GenAI-assisted tasks. Specifically, higher confidence in GenAI is associated with less critical thinking, while higher self-confidence is associated with more critical thinking. Qualitatively, GenAI shifts the nature of critical thinking toward information verification, response integration, and task stewardship. Our insights reveal new design challenges and opportunities for developing GenAI tools for knowledge work.
Alright, so it’s not like they threw knowledge workers in MRI machines to scan brain activity. Actually they ran a survey on Prolific, a platform that pays survey respondents to answer surveys they’ve been pre-screened to qualify for, in this case knowledge workers who self-identified as using Gen AI tools at work at least once a week. If you’re unfamiliar with platforms like Prolific and thinking “wtf that doesn’t sound scientific,” it’s common these days for researchers to use platforms like that — along with Amazon’s Mechanical Turk platform which I’ve participated in myself for many years — with the blessing of their organization’s Institutional Review Board (IRB).
Anyway. The researchers said they were interested in knowledge workers specifically because “much research on the effect of GenAI on thinking skills is focused on educational settings, where concern for skill cultivation is most acute (e.g., the effect of GenAI code completion tools on programming and computer science education). [C]ritical thinking has been operationalised in detail in certain specific disciplines, such as academic history, clinical psychology, and nursing. But the ostensible shifts in critical thinking behaviours brought about by GenAI extend to a broad set of professions and knowledge workflows — GenAI tools are now widely used in knowledge work — and little is known about the critical thinking demands of these.”
Participants talked about why they are or are not motivated to use critical thinking when using generative AI. This one in the “are not” category might look familiar to most of you:
Motivation barriers. Knowledge workers also discussed how prioritising critical thinking in their work might be misaligned with their overall task motivations or job objectives. For example, participants discussed a lack of time (44 of 319 respondents) for critical thinking at work. For instance, a sales development representative noted that “[t]he reason I use AI is because in sales, I must reach a certain quota daily or risk losing my job. Ergo, I use AI to save time and don’t have much room to ponder over the result.” Even when time was not constrained, knowledge workers often lacked incentives to engage in critical thinking when it is perceived as not part of their job responsibilities (11 of 319 respondents).
For some respondents, they end up working harder because they can’t trust AI to provide the information they’ve tried to extract from it:
However, many participants shared examples when they perceived more effort in information retrieval because the AI response can be wrong and needs verification (56 of 319). For example, when a lawyer used ChatGPT to find relevant laws for a legal case, he noticed “AI tends to make up information to agree with whatever points you are trying to make, so it takes valuable time to manually verify.”
Let’s get to the meat and potatoes already so we can wrap this up:
In the majority of examples, knowledge workers perceive decreased effort for cognitive activities associated with critical thinking when using GenAI compared to not using one — examples that were reported as “much less effort” or “less effort” comprise 72% in Knowledge, 79% in Comprehension, 69% in Application, 72% in Analysis, 76% in Synthesis, and 55% in Evaluation dataset. Moreover, knowledge workers tend to perceive that GenAI reduces the effort for cognitive activities associated with critical thinking when they have greater confidence in AI doing the tasks and possess higher overall trust in GenAI.
Not to sound old but with how much critical thinking has been suffering these days, I’m not sure we can afford any extra brainrot as a society. And definitely not as an economy.
The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers [Carnegie Mellon University and Microsoft]