top of page

Between Hype and Fear of Chatbots

What AI Chatbots Are Actually Doing to Work

By Tommy Cooke, powered by coffee and curiousity

May 16, 2025

Key Points:


  1. The labour market impact of AI chatbots is far smaller than headlines suggest, with minimal changes to work time, earnings, or job displacement


  2. AI’s true influence lies not in replacing workers, but in reorganizing workflows, expectations, and trust especially when organizations prepare for adoption


  3. Without a supportive culture, training, and governance, AI tools reflect existing flaws rather than deliver transformation, meaning leadership must shape readiness


For years now, the narrative around AI chatbots in the workplace has swung wildly between utopia and dystopia. Depending on who you ask, chatbots are either eliminating jobs or boosting productivity at unprecedented rates. But recent research offers a sobering counterpoint: the reality is far less dramatic.


To find out what was really going on, researchers surveyed 25,000 people working in 7,000 places, covering 11 jobs that were previously believed to be on the path to destruction due to AI, over 2023 and 2024.. The survey results were published in a study last year, exploring how AI has affected the labour market. The result? Not very much—not yet.


Headlines tend to focus on how AI replaces jobs and transforms sectors. Yet, the study finds that the average time saved from using AI chatbots was just 2.8 percent of total work time. No meaningful impact on earnings. No significant shift in hours worked. No sweeping revolution or impending doom.


This kind of insight isn’t great news, but it certainly isn’t bad news either. Rather, it’s a clarifying moment and it’s one we need to pay attention to.


Are We Asking the Wrong Questions About AI?


The majority of mainstream discourse about AI has been driven by scale: “How many jobs will be lost?” or “How much time will be saved?” But these questions flatten the real path of technologically-fueled change. They assume impact must be immediate, measurable, and massive to matter.


Between the hype and fear is a more mundane story. It’s a real story, and it’s not about replacement. It’s about reorganization of workflows, of expectations, and of trust. When we ask if AI is "changing the world of work," we have to ask what kind of change we're actually expecting.


Take this example from a recent MIT-Stanford study: customer support agents who used generative AI saw measurable gains, especially with junior agents. They closed tickets faster and improved customer satisfaction. But this was a narrow use case, supported by extensive prompts and training. And even then, senior agents saw little benefit. Why? Because the AI was trained on transcripts written by those same senior agents. The AI amplified what already worked. It didn’t invent a new system or innovate a new way of working. This tells us something critical: AI is not replacing expertise. It's merely supplementing it.


AI Perception Problems Are Real Problems that Must be Addressed


There’s another undercurrent in the reality of chatbot impact on the labour market. Once more, this undercurrent doesn’t often show up often in mainstream discourse, nor do we see it in cited labour stats: the social cost of using AI.

More specifically, a separate study by researchers at Duke and Princeton found that employees who used AI tools like ChatGPT were perceived as less competent, less hardworking, and even lazier by their peers.


Even if someone uses AI effectively to save time and focus on higher-order work, they may still be socially punished for it. In corporate environments where appearances matter, that stigma can be enough to stop adoption altogether, especially if organizations fail to create the right messaging and workplace culture around AI.


Let’s keep in mind that we are talking about a social issue and not a cultural one.


Reading Between the Lines of Recent AI Studies on the Labour Market


There is a lesson that is not explicitly articulated within the studies referenced thus far: uncertainty and underinvestment shapes AI impact. In our experience, most organizations do not train staff on how to use AI well, lack internal policies to govern safe or appropriate use, rely on ad hoc champions rather than structured leadership, and neglect to reconfigure workflows to accommodate AI as a partner.


Dropping a chatbot into a workplace doesn’t magically enhance it. Rather, it will often mirror its flaws.


My point is simple: the companies seeing real value from AI (even if modest) are doing the preparatory work that matters behind the scenes. They're adapting processes, not just adding tools. They're establishing clear boundaries, strong incentives, and internal trust. They're shifting AI from being a novelty into something more like organizational infrastructure.


Moving From Transaction to Transformation


So, what should you as a business leader do with this information? Here’s a simple takeaway: don’t expect AI to change your business on its own. Start changing your business so AI fits into it.


This means moving from transactions (throwing AI at a task) to transformations (rebuilding how work is done). It also means slowing down and doing the essential work of governance, policy, training, communication, and culture building.


Here’s what we suggest:


Create a clear internal stance. Make your organization’s expectations for AI use visible and affirming. Let people know they won’t be punished for using AI and they won’t be rewarded for hiding it.


Focus on real use cases. Don’t adopt AI for its own sake. Instead, identify where it can reliably augment work, especially for less experienced employees or for low-risk internal processes.


Track impacts quietly and consistently. You may not see big numbers at first. That’s okay. Instead, look for friction reduction, faster onboarding, or error detection. The wins may be small, but they compound.

 

Build a governance wrapper. Define what “responsible AI” means for your organization and operationalize it with processes, documentation, and regular review. Don’t just govern the tool. Govern the ecosystem around AI itself.


AI Opportunities Hiding in Plain Sight


The study I opened with may seem like it dampens AI excitement. But there is another way of reading it. My takeaway from the study is that opportunities for growth and change are dependent upon leaders shaping their organizations.

Rather than wait for a chatbot to change your organization, change your organization so that the chatbot fits.


That’s the real work. And it’s where the value lives.

bottom of page