DCU Article: Chatbot Design: 7 Tips for UX Teams

Tip 1. Don’t pretend your bot is a human

Ever get the creepy feeling that the human you think you’re interacting with online is actually a robot? This is the uncanny valley effect and consumers really do not like it. A 2019 study which measured the heart rate, sweat glands and frown lines of 31 participants found they experienced an increased Uncanny Valley Effect in the more human-like chatbot – which was a complex, animated avatar – rather than the simpler text chatbot⁵.

This study also found that the more “weird” the avatar chatbot seemed for the participant, the less competent it was judged to be. So chatbots “should not be designed to pretend to be human — at least not if they do so ineptly” [Ciechanowskia, 2019].

Okay, another study found that a chatbot having human-like traits increases trust and familiarity – but only as long as you make it clear that the chatbot is not human, by using an illustration of a person instead of a photo of an actual person⁶. To avoid the uncanny valley effect, tech giant Salesforce recommends that businesses “identify the bot as a bot, not a human, and let your customers know how the bot can help”⁷.

Does this mean less is more when designing your chatbot? Not if you want to create something to compete with Microsoft’s ‘empathetic social chatbot’ XiaoIce⁸. Targeting the Chinese community on microblogging site Weibo, XiaoIce has 660 million registered users⁹, at least some of whom are men who claim to be madly in love with her¹⁰. Romantic declarations aside, a success metric for social chatbots is conversation turns per session [CPS] [Hornigold, 2019]. While Siri and Google Home have around 3 apiece, XiaoIce boasts 23 back-and-forths across all users [Hornigold, 2019]. Her researchers claim that this means XiaoIce is more engaging to talk to than the average human [Hornigold, 2019]. So, that’s the competition. No pressure.

Tip 2. Sync chatbot “personality” with brand

While your chatbot shouldn’t pretend to be human, it should still have a “personality” to facilitate engagement with users. The personality and tone of your bot – whether casual and friendly or more formal and reserved – will depend on your business. For example, most of us probably appreciate some friendly encouragement in the kitchen, and in a comparison test between two different chatbot personalities, the majority chose the friendlier bot for a meal planning app¹¹. Not that the homely personality is the favourite every time. When two virtual interviewers were created for a recruiting event, users were found to confide more in the chatbot with a serious, reserved personality over the warm, cheery one¹². A chatbot is a representative of a brand – so the tone should “match” that brand. It should also align with customer expectations of brand “voice”. Chatbots aren’t human but they can still influence the way consumers feel about a company, the same way any member of staff can impact on a consumer’s impression of a brand [Araujo, 2018] First chatbot impressions last.

They may be perceived as less stigmatising than formal health care support³² but their “limited capacity to recreate human interactions and to offer tailored treatment, combined with the current lack of access to mental health services in real time, raises the question of whether chatbots could do harm to users. These harms would go largely unseen unless specifically tracked.” [Kretzschmar, 2019.] At least there are some ethical guidelines here: health chatbots should respect users’ privacy, be evidence-based and ensure users’ safety. They should also be as transparent as possible about what they can offer [Kretzschmar, 2019].

No wonder Tesla and SpaceX boss Elon Musk called AI more dangerous than nuclear warheads and called for a regulatory body to oversee super intelligence³³.

Algorithms are increasingly trusted with operations, decisions and choices³⁴ despite being fundamentally biased³⁵.

An AI tool that has revolutionised the ability of computers to interpret everyday language has been shown to exhibit striking gender and racial biases³⁶. Yes, machines are, sadly, not neutral, but riddled with the human bias in human data³⁷.

“A lot of people are saying this is showing that AI is prejudiced. No. This is showing we’re prejudiced and that AI is learning it,” says Joanna Bryson, a computer scientist at the University of Bath, United Kingdom.” [Devlin, 2017]. Beware of this ethical minefield, UX designers.

Tip 3. Be engaging

XiaoIce is not the only bot designed to be as engaging as possible. The “primary goal” of one of the earliest chatbots – a robot called Chatterbox – was to “build a conversational agent that would answer questions”¹³. That primary goal hasn’t changed much. Chatbot technology uses Natural Language Processing [NLP] to understand user questions and bots are programmed to learn from every interaction, eventually ‘thinking’ and responding like a human might. But most question-answering systems today can only answer relatively simple queries. These questions should be “no more than one to three sentences long, have a lucid expression of language, should be answerable by searching for text mentions or by formulating structured queries, and most importantly, involve minimal reasoning”¹⁴.

But there is another user interface in town. CI, or Conversational Interfaces, are giving NLP a run for its money¹⁵. While NLP focuses on understanding what the users say, CI interacts through chat, voice, buttons, images, menus and videos [Pan, 2018].

Which method is better? Whichever works best for you – and your customer.

Tip 4. Gather the best data

How does your chatbot predict your questions and reply to them? By feeding on your data, that’s how. So when designing a chatbot, consider the data you want to collect from users and how ¹⁶.

Chatbots collect data in the background of user activity or when they ask users questions directly ¹⁷.

But how to get the best data? Know the questions to ask ¹⁸ and steer the chatbot’s conversation accordingly ¹⁹. Data gathered by chatbots must be obtained legally, fairly and ethically, protecting user privacy and security at all times ²⁰. This data has many functions and can be used to answer user questions, provide customer service or information, push relevant information or product sales and train ML [Machine Learning] chatbots ²¹. Best practice explains how the collected data is used and allows the user to opt out of certain data sharing. Warning: Even if a user opts in to sharing their data by agreeing to terms and conditions, if their data is presented in a new, unexpected context this can create surprise and mistrust and can give the impression that their data isn’t private. By explaining where the data was sourced when providing a notification to the user and allowing the user to manage the type of data they are willing to share, the user feels more in control of their data and trust is maintained [pair.withgoogle.com, 2020].

Tip 5. Let the conversation flow

How do you “script” your chatbot’s potential Q&As? Using a conversation diagram can help you map out everything your chatbot might say in response to the most commonly asked questions for your bot ²². However, you can’t script for every possible permutation of what a user may ask, especially if they are ‘playing’ with a new piece of tech, testing its limits by asking nonsensical questions. If users are messing with it, your bot should communicate that it does not understand – then prompt the user with alternative ways it can help [pair.withgoogle.com, 2020].

Your bot can also put the ball back in the user’s court, asking for feedback or clarification. This way you can gather data about a user-need that the system may not be serving. [pair.withgoogle.com, 2020] If your bot cannot ‘hint’ the user back to its arena of knowledge it may end up repeating the same statements over and over verbatim ²³. This is not how most (enjoyable) human conversations go and the repetition will increase the uncanny valley effect. The repetition will increase the uncanny valley effect. The repetition will increase the uncanny valley effect.

Giving your chatbot at least a few variations of its statements and queries will help make the conversation feel more natural. Also, make sure the content your bot is dealing with is ordered in such a way that it can be accessed using different phrases ²⁴.

For example, if it is a chatbot for movie recommendations, some people may search by genre, others by lead actor and yet more by director. The bot should anticipate a user searching in any of these ways and display its results. But it should also present information so the user can change the way they are searching. In other words, the bot should be able to handle these different queries without requiring the user to cancel out and start a fresh chat. The conversational flow should then feel seamless, providing a more natural back-and-forth and a better user experience ²⁵.

Tip 6. Don’t be sexist

In general chatbot gender does not have an effect on users’ overall satisfaction and gender-stereotypical perception ²⁶. But only if you are in pretty gender-neutral surrounds of, say, an online bank. Users are more likely to apply gender stereotypes when a chatbot system operates within a gender-stereotypical subject domain, such as mechanics, and when the chatbot gender does not conform to gender stereotypes [McDonnell & Baxter, 2019].

So if you are in the auto repair business you will need to be very ‘woke’ to make your chatbot female when you know male bots elicit more customer trust. What should you do? Exploit the situation, make a big, butch bot called Brad and perpetuate the bias and stereotyping? Or become part of a vanguard breaking the gender bias mould? That’s entirely up to you – and your client.

Just remember UNESCO have published guidelines around avoiding gender bias in your chatbot, the first of which is to end the practice of making digital assistants female by default ²⁷. The majority of AI assistants – like Google Home, Alexa, Siri and Cortana – have been designed with a female voice and personality ²⁸. UNESCO wants this practice to stop because, among other reasons, it “makes women the face of glitches and errors that result from the limitations of hardware and software designed by men” [unesco.org, 2019].

According to Michelle Habell-Pallan, Gender Studies professor at the University of Washington, the cultural projection of the woman’s role as “assistant” is reflected in the technological field ²⁹ where the proportion of female employees remains a risible 18% ³⁰.

Tip 7. Be Ethical

It sounds like something straight out of Black Mirror – because it is. The episode ‘Be Right Back’ explored the morally dubious idea of technology facilitating “conversations” with our dead loved ones.

Guess what? It’s already here.

In 2016, Russian AI entrepreneur Eugenia Kuyda built a chatbot that she trained on over 8,000 lines of text messages and social media posts from her deceased best friend Roman Mazurenko ³¹. Using predictive analytics her bot mines Roman’s extensive dataset to predict what he might “say” in a chatbot conversation. His loved ones who have conversed with bot Roman found it “eerily convincing” [Elder, 2020].

No doubt bots like this could represent a lucrative market, but does that make designing them right?

It is a moral minefield and UX principles must evolve to address such potential dilemmas.

Other well-meaning automated bots designed to help people with mental illness have raised troublesome questions.

They may be perceived as less stigmatising than formal health care support ³² but their “limited capacity to recreate human interactions and to offer tailored treatment, combined with the current lack of access to mental health services in real time, raises the question of whether chatbots could do harm to users. These harms would go largely unseen unless specifically tracked.” [Kretzschmar, 2019.] At least there are some ethical guidelines here: health chatbots should respect users’ privacy, be evidence-based and ensure users’ safety. They should also be as transparent as possible about what they can offer [Kretzschmar, 2019].

No wonder Tesla and SpaceX boss Elon Musk called AI more dangerous than nuclear warheads and called for a regulatory body to oversee super intelligence ³³.

Algorithms are increasingly trusted with operations, decisions and choices ³⁴ despite being fundamentally biased ³⁵.

An AI tool that has revolutionised the ability of computers to interpret everyday language has been shown to exhibit striking gender and racial biases ³⁶. Yes, machines are, sadly, not neutral, but riddled with the human bias in human data ³⁷.

“A lot of people are saying this is showing that AI is prejudiced. No. This is showing we’re prejudiced and that AI is learning it,” says Joanna Bryson, a computer scientist at the University of Bath, United Kingdom.” [Devlin, 2017]. Beware of this ethical minefield, UX designers.

PS Tip 8. Try and save the world!

We mentioned coronavirus at the top – and with the number of cases rising, UX design might have a role to play. Certainly there is room for innovation. The Wall Street Journal reported that coronavirus has revealed the limits of AI health tools, with makers of ‘symptom checker’ chatbots struggling to update algorithms to respond to the epidemic³⁸. Several chatbots were deployed to fight disinformation³⁹. An interactive agent that calms users in a crisis by providing rational, helpful information? There’s a fine example of effective communication by chatbots.