Tamlin Magee is a freelance technology journalist and culture writer.

Once a signifier of luxury or novelty, custom products and services are now within reach of the everyday consumer, thanks to the rise of artificial intelligence, algorithm-led platforms, and the COVID-era digitisation boom. Today, everyone wants to be the main character of their brand experience. 

Personal Worlds

When digital assistants such as Apple’s Siri hit the market in 2011, they were heralded as a new dawn for computer-human interaction. But digital assistants have hardly become a source of wonder for consumers in the years since – with the majority of chatbots only capable of answering rudimentary questions, rather than navigating the intricacies of human speech. That’s until OpenAI released ChatGPT in November 2022. Using a chunk of the internet as its source material, the chatbot has stunned consumers and tech experts alike for its ability to produce coherent answers to requests as complex as ‘are NFTs dead’, and ‘tell a funny joke about the tax risks of international remote work’. 

CNBC

 As the world gets to grips with new advances in conversational AI, brands are grappling to decipher what role these tools could play in their organisations – specifically, how they communicate with their audiences.

It’s extremely unlikely these tools will ever win a Nobel Prize for literature or replace the Rembrandt collection at the Rijksmuseum, but the outputs range from OK to occasionally highly impressive, and as such, some businesses are already foregoing the usual marketing routes and turning to machines, especially for content generation. 

Sifted

 These early snippets of machine-generated marketing copy may prove to be the thin end of the wedge for how brands communicate with their audiences – with AI already driving developments in web search, customer service, and troubleshooting.

Despite the sometimes impressive results that AI-powered platforms produce, the technology is still clearly at an early stage. The kinds of results that the new tranche of AI tools surface are, after all, only as good as their inputs, and when the inputs are the whole internet, there’s a lot of misinformation to contend with. Conversational AI has “no clue whether what it produces is misguided, incorrect, or even dangerous – it has no safeguards,” says Lucas Galan, head of data science product at CODE Worldwide. 

Interview with Lucas Galan conducted by author

 With trust in technology rapidly declining around the world – hitting an all-time low in 17 of the 27 countries surveyed in Edelman’s Trust Barometer 2021 – the careful implementation of AI advances will be crucial to avoid further denting consumer trust. 

Cnet

 How can brands realise the creative potential of Natural Language Processing (NLP), and what practical and ethical considerations stand in the way of implementing these tools with success?

Almost no one likes chatbots. Since about 2016, brands have raced to incorporate them across their websites, on Facebook, and on messaging services like WhatsApp. Although 93% of consumers now expect chat functionality on websites or apps, only 63% have had satisfactory experiences. And although everyone expects automated customer service voice assistants, only a quarter of people actually prefer them. 

Modern Retail

The equation does, however, change when AI is brought into the mix: 45% of Americans say they’d prefer to have inquiries handled by AI if it led to faster results. 

The Robot Report

Expectations here are high, and the majority of people – at 61% – believe that AI shouldn’t make any mistakes whatsoever. 

Science Focus

 What’s more, while consumers are happy to leave quick, logical, rational queries to machines, if a subject is delicate or requires a human touch, then customers will want to 

 for all the accompanying nuance that machines aren’t capable of. Consumers do tend to trust chatbots for simple enquiries, but are quickly frustrated when they feel that a human could solve their problems faster: only 23% of customers would settle disputes or pay bills via an automated chatbot option, but over half are happy to ask chatbots about basic product details. 

Userlike

To date, most chatbots or automated customer service options have operated within very specific parameters; that means they’re more reliable at responding to certain queries, but are highly limited in the responses they can offer. “A reason chatbots are quite poor at the moment is partly because businesses are using out-of-the box solutions, which are limited in meeting particular contexts,” says search and analytics Professor Paul Clough from the University of Sheffield. “They’re probably using quite simplistic methods and tools, which don’t necessarily respond very well to the variation that we normally have in human language.” 

Interview with Professor Paul Clough conducted by author

 At the other extreme end of the spectrum are Chat GPT-like technologies which can write impressive screeds but go off the rails in their replies just as easily.

As AI is increasingly intertwined with customer service, though, this could be changing. ‘Sentiment analysis’ seeks to imbue machine interactions with a degree of emotional intelligence, and although machines are unlikely to ever experience emotions the ways we humans do, they’re getting better at approximating them. Some advocates in this field of ‘emotional AI’ suggest that machines that seem more empathetic – a linguistically complex concept to nail down, but loosely defined as understanding another’s emotional experience – could have uses in medicine, for example in helping to better decode and communicate the feelings of dementia sufferers to their caregivers. 

Forbes

 Outside of medicine, schools in Hong Kong are even using artificial intelligence programs to measure the micro-movements on students’ faces in order to better understand their emotional states. 

Wired

 However, critics such as behaviour and data scientist Pragya Agarwal have warned that these systems are based on “flawed science”, because emotional states are reduced to 1s and 0s without the multitude of nuance or subtlety that humans experience. 

If you really could have Asked Jeeves a question, maybe Google wouldn’t have put 

. That’s what plucky AI-powered start-ups like the ad-free, subscription-based search like Neeva AI are betting on, launching a ChatGPT-esque feature that will respond to questions with conversational text, instead of page upon page of web links. Former Google engineers were impressed with its ability to answer simple prompts, or put its hands up when it couldn’t 

, seem to mash together traditional search with the ChatGPT experience. And Microsoft recently committed to a multibillion-dollar OpenAI investment, on the back of plans to roll ChatGPT into the company’s own search engine, 

In a nutshell, the selling point of AI-powered search is to better understand the intent behind a query, allowing for more accurate and – crucially – personally relevant results. Neeva’s engine can, for example, provide summaries of pages in a way that’s relevant to the user’s specific request. In the future, it could also help to present information from multiple sources in a more engaging way, illustrating quick visualisations to better present results. The personalisation aspect is really key: brands that personalise experiences can drive 40% more revenue than those that don’t. 

McKinsey & Company

 Yet half of all brands are failing to personalise experiences. 

Twilio

 AI means there’s an enormous opportunity to capture nuance, subtext, and intent, then mix up results and present them back to the user in a way that’s relevant for not just their query, but their persona and communication preferences.

Omar H. Fares, lecturer at the School of Retail Management, Toronto Metropolitan University, sees enormous potential in search that’s integrated with conversational AI. But he warns that conversational AI can struggle with newer situations where information about a given topic is only just beginning to emerge. “If AI is given the wrong information or the system is trained on the wrong thing, it will generate the wrong outcome,” says Fares. “This is one of the key risks of AI as we move forward and we’re not there yet, in terms of matching expectations and what it can actually deliver.” 

Interview with Omar H. Fares conducted by author

 And although ChatGPT was trained on 570 gigabytes and over 300 billion words, processing up-to-date, accurate data and then parsing it for the best results is “very much a challenge from a systems perspective”, Fares says. 

 Galan agrees, saying that AI has “no relationship with the truth, they don’t care which answer is the best. It’s not optimised for it – it is merely working on the capacity for volume of data.” 

 That’s why beta AI search engines like Consensus provide results that, according to Grid magazine, range from “wrong to incoherent” and even occasionally dangerous, such as claiming that vaccines cause autism. 

Grid

Algorithms don’t just frivolously dictate the kinds of products we’re served on Amazon or help Netflix tell us the content it thinks we’ll like. They are increasingly making decisions that have an unquestionable real-world impact, often for the worse. Take the controversial algorithm that predicted the grades of students in Britain during the pandemic, which was allegedly biased against school-goers from poorer economic backgrounds. 

The Verge

 In fact, it is often under-represented or marginalised groups who tend to experience the negative effects of AI first: in the wake of riots in the UK’s capital in 2011, London’s Metropolitan Police used AI to create a ‘gangs violence Matrix’ intelligence tool – it was implemented without transparency and would flag innocent young ethnic minorities as potential criminals to the police. 

Byline Times

In 2016, author and data scientist Cathy O’Neill highlighted in her book, Weapons of Math Destruction, that prejudiced algorithms may not have been created with ill intent, but that the subconscious or unconscious bias of the coders designing the systems had undesirable and unintended consequences. Meanwhile, critics such as Pragya Agarwal have more recently warned that left unchecked and without oversight, AI systems could strengthen existing inequalities and “further disadvantage those who are already marginalized”. Agarwal cited a disastrous case of facial recognition AI – when an analysis from 400 NBA games using two emotion-recognition tools – Face, and Face API by Microsoft – which assigned more negative emotions on average to Black players, including when they were smiling. As well as being tremendously offensive, the consequences of tools such as these “reaffirm research showing that black men have to project more positive emotions in the workplace, because they are stereotyped as aggressive and threatening”. 

It’s nearly a decade since O’Neill published her book, and there is more awareness around these issues than at that time. Government bodies are making headway, especially in Europe with the development of the AI Act, which addresses algorithmic bias. In Britain, the ‘Algorithmic Transparency Standard’ – a list of protocols so the public sector can design algorithms more transparently, and an accompanying template to inform the public about the use of their algorithms – is in use by a handful of organisations, like the Hampshire and Thames Valley Police and the Food Standards Agency’s Food Rating Scheme. 

Currently, customer service chatbots tend to be out-of-the-box offerings with little conversational or engagement value. But, says Professor Clough, given the right investment – hypothetically, Microsoft buying OpenAI, for example – there could be an explosion in AI-driven customer support chatbots. “I can imagine it being used for frontline customer support people who are able to query against ChatGPT to get answers to various questions, possibly beyond the scope of what they know about their product or service,” Professor Clough says, “and I could see it being used where brands might want to learn more about customers in some way, or how to engage with different types of customers. You could use it to generate responses, and you could use it to summarise a whole bunch of customer dialogue or conversation, or you could use it for real-time translation.” 

 Think of the wealth of information on tech support forums online, and imagine being able to get this distilled and summarised for your specific needs: an AI-driven kind of customisation could obliterate the need for trawling through ancient web forum threads or staying on hold for some indeterminate length of time, and instantly provide the answers the customer needs.

Currently, ChatGPT is not trained on the most up to date information – that is, it’s not real-time, and therefore its abilities are somewhat limited to summaries, visualisations, and instructions. Admittedly, these may quickly change the way users experience information for the better, with more personalised results that are pertinent to the individual user. However, says Fares, an AI-powered search engine that is trained on up-to-date, accurate information, could have potentially profound implications for businesses of all stripes. Entertain the thought: an AI-powered search that helps consumers find the best prices for any given product, reliably, and at any time. Businesses that compete on price alone could find themselves being left behind by brands that offer more complete packages around convenience, personalisation, heritage, and quality. “If your competition point is price, that’s OK, but as marketers we might need to think about how we redefine our value propositions,” Fares says. 

If brands want to differentiate from price alone they could do worse than to work on their trust credentials. At the moment, says Professor Clough, technologies like ChatGPT are ultimately a black box – the ‘how’ and the ‘why’ of its inner workings are just generalities rather than specifics. Professor Clough says that to win consumer trust, businesses need to build ‘explainability’ into their systems: precisely how decisions are being reached, the people who are ultimately accountable for those decisions, and being open to discussions around how those decisions are reached. “With a machine, who is accountable?” Professor Clough says. “This is going to be absolutely necessary for people to be able to trust the outputs of these tools and the people using them, building them, and developing with them. This is super important for things like AI in healthcare applications: maybe a doctor is using the output of AI to make a diagnosis – but how did it really understand this patient problem? Consumers will want to know: why am I being recommended this product? There are elements of things like GDPR and other regulations that say people have a right to explanation – but I think enforcing that will help.” 

 Larger tech brands are wising up to this imperative – TikTok, for example, has introduced a feature that tells users why its algorithm recommends certain videos in their feeds in a bid to boost transparency. 

Gizmodo

 of Americans say they’d prefer to have inquiries handled by AI if it led to faster results

 of respondents would settle disputes or pay bills via an automated chatbot, while 

are happy to ask chatbots about product information

Brands that personalise experiences can drive 

Customer-facing roles could be inexorably transformed by AI, but despite advances in emotional AI, the human touch is proving difficult to replicate

AI could change the face of search and how we experience information, but there are roadblocks towards real-time processing

Brands will need to strive towards openness and transparency if consumers are expected to trust machine decision-making

Professor Paul Clough is a professor of search and analytics at the University of Sheffield. He heads the information retrieval research group at the University of Sheffield’s Information School, where he teaches and researches data management, information storage, and retrieval.

Lucas Catalan Galan is the head of data science product at CODE Worldwide and a self-taught programmer and AI expert with over ten years in building innovative technology solutions and focusing on their applications. Galan has been involved in designing award-winning AI solutions for the likes of Formula One, Reuters Institute, and Public Health England.

Omar H. Fares is a lecturer at the Ted Rogers School of Retail Management. He researches consumers’ interaction with novel digital innovations and the impact of digital innovations on retailers. Fares has worked for leading institutions such as Walmart, CIBC, and Fairstone Bank.

Technology

Apps and software

Interview with Lucas Galan conducted by the author

Interview with Professor Paul Clough conducted by the author

Interview with Omar H. Fares conducted by the author

Gov.uk

When OpenAI released ChatGPT to the public, the text generator made a clear case for the sheer disruptive force of AI – making it apparent that such technology will change the way we experience information and engage with brands. But major challenges remain before AI elbows humans out of the way.

Personal Worlds

Logically: fighting the infodemic with AI

Trust an AI? The science of the ‘word-of-machine’ effect

Lensa AI's 'magic avatars' spur debate over AI ethics

AI-generated fashion show showcases beauty in ageing