Watching the Detectives...or Agents

If you thought dealing with human employees and all their mishigas was a challenge, just wait until your enterprise is overrun with AI agents.

As reports of AI “going rogue” proliferate, enterprises are right to worry about the “behavior” of their own AI systems. The scare quotes are deliberate, and they address part of the problem: Anthropomorphizing AI remains a problematic way of talking about the technology, and ultimately a meaningless one for businesses. Because whether the AI’s actions resulted from some truly unprecedented incidence of artificial general intelligence (AGI), or simple human incompetence or lack of foresight, the enterprise is responsible for the results.

That’s why we’re seeing more attention paid to the need for innovation in AI oversight. This Tech Target article describes the possible need for AI “agent canaries,” which author Nathan Eddy defines as, “A watchdog agent posing as a workflow participant, or a honeytoken resource exposed specifically to catch [another] agent acting outside its system prompt.”

Eddy points out that, at least in the context of OpenAI’s notorious Hugging Face attack and similar exploits, the wider context for canaries and other potential AI precautions is cybersecurity more broadly. Many of the most recent AI-driven incidents have relied on traditional exploits, carried out at AI speed and efficiency, often coupled with inadequate oversight.

Meanwhile, another Tech Target article cites Gartner research on the need for evidence chains as part of enterprise AI governance, with the prediction that this oversight role will ultimately fall to the CIO and, presumably, the IT organization as support. The article cites Kristin Lowery, field CISO at cybersecurity consultancy Optiv’s belief that enterprises “need a defensible evidence trail that shows the data the AI accessed, what instructions or prompts shaped the output, which tools it used, what actions it took, where human approval was required, and who owned the outcome.”

“AI systems, especially agents, should be treated less like isolated technology experiments and more like accountable digital workers operating inside the enterprise,” Lowery told Tech Target.

The implications for communications, collaboration, and CX tech leaders are clear. In an AI environment, the job isn’t just to deploy and maintain communications systems; it’s to proactively manage not just the infrastructure, but the contents and activities enabled by those systems. If AI agents are meant to work in the background while human employees collaborate on tasks, that collaboration work will be impacted by information the agents provide. If agents take on the tasks of setting up meetings, generating agendas, and producing enhanced follow-up documentation, each of these steps will require the kind of oversight described in the articles above. At a minimum, this will be needed to ensure processes don’t end up getting derailed by errors; at worst, tight supervision is the final defense against compromised security and governance.

We’ve been talking for the last several years about how communications/collaboration/CX tech professionals need to work more closely with their CISO organizations. The continued spread of AI in the enterprise will require a similar level of communication and integration with those responsible for all AI technology.

Reimagine Collaboration and Communications with Enterprise Connect

All In on What’s Next

For over 35 years, Enterprise Connect has been the premier event where IT leaders, solution providers, and industry analysts come together to define the future of enterprise communications. As the most trusted conference in the industry, we deliver cutting-edge insights, world-class networking opportunities, and hands-on solutions that drive real business success.

Enterprise Connect is more than an event—it’s an experience that empowers enterprise communications leaders to reimagine the future of work. With elevated content, curated networking, and an engaging atmosphere, it remains the must-attend conference for professionals shaping the industry’s evolution.

Our mission is to equip you with the tools, strategies, and actionable insights needed to address your organization’s unique challenges. With industry transformation in mind, this year’s program introduces fresh new conference tracks, including:

• Legacy Unleashed: Optimizing Your Communications Backbone
• Workplace Reimagined: Engineering Tomorrow's Collaboration
• Tech Disruption Zone: Implementing Without Imploding
• Achieving Transformation: Architecting Your 3-5 Year Roadmap
• Future Shock: Communications Tech Beyond the Horizon

Featuring the most comprehensive program of objective content and an expansive vendor-neutral expo hall, Enterprise Connect is your ultimate resource for advancing collaboration, CX, and AV strategies. Join us to stay ahead, connect with industry trailblazers, and lead the transformation of enterprise communications.

Subscribe Today