Companies warned over the use of AI chatbots in customer service

Businesses which introduce AI chatbots to help with customer service are being warned of the potential security risks associated with the technology.

Cyber and Fraud Centre Scotland, a social enterprise, said chatbots and virtual assistants could create new opportunities for criminals if not configured correctly.

The organisation has expanded its cybersecurity testing to help clients identify vulnerabilities related to the use of chatbots.

It said attackers could attempt to manipulate an AI assistant using carefully constructed prompts, causing it to ignore its instructions, disclose sensitive information or interact with connected systems in unintended ways.

Thaïs Ramdani, of Cyber and Fraud Centre Scotland, said: “Organisations are understandably keen to explore what AI can do for their customers and help with efficiencies in teams. But security needs to be part of that conversation.

“Adding an AI assistant to a website isn’t simply adding another customer service tool. Depending on how it’s been configured, the AI assistant could have access to confidential information or documents which risk being exposed.”

The Centre’s new AI and Web Application Security Assessment service involves a specialist team assessing traditional web application vulnerabilities alongside security risks specific to the way AI has been implemented.

Ramdani added: “Our team essentially approach the technology from an attacker’s point of view – what can we make it do that it wasn’t intended to do?

“Finding these weaknesses through controlled testing gives our clients the opportunity to address them before someone else finds them.”