Behind the paper · AIS Transactions on HCI · 2019
Designing virtual in-vehicle assistants for a convincing user experience
A car is a hard place for a voice assistant. These guidelines say what to get right, and quietly introduce the idea of a companion.
“Companionship first surfaced here, in the interviews for this study, long before it became the centre of my research.”
Read the story behind the paper
In brief
A car is the hardest place for a voice assistant.
Every word competes with the road, mistakes carry consequences, and the general advice for voice interfaces did not carry over.
Fifteen requirements, four blocks of guidelines.
Derived from the literature and six automotive and UX experts, refined after HICSS, then applied in a virtual reality prototype with eight driving scenarios.
Easy to use, comforting, and sometimes too eager.
Nineteen participants rated the assistant clearly positive, praised its proactivity, and told us that sometimes less is more.
The companion was born here.
The fourth guideline block defined the virtual companion, and the following years of research grew out of it.
01 / The problem
The driver is already busy.
By 2019, Siri, Alexa, and Google Assistant had made voice interaction ordinary. Cars had thirty years of assistance systems behind them, from anti-lock brakes to adaptive cruise control, but no systematic guidance for a conversational assistant that talks to the driver. The general advice for designing voice interfaces did not carry over, because the car adds constraints that phones and homes do not have. Every word the assistant says competes with the road. Errors are not merely annoying.
The paper frames the challenge as two design activities. Representational design is about style and character: who the assistant is, how it speaks, whether it has a face. Interaction design is about how the work is divided between person and machine and how the exchange unfolds. Both shape whether people find the assistant easy, enjoyable, and worth using.
02 / The approach
Literature, experts, community, prototype.
The work ran through three design cycles. The first drew assistant requirements from the literature on user experience and voice interfaces, then put them to five experts: a psychologist specialising in human–machine interaction in vehicles, an automotive engineer with a doctorate, a project manager developing speech assistants, a UX designer for in-vehicle assistants, and an independent speech-recognition researcher. Their interviews were coded into 40 codes in 15 categories and turned into concrete guidelines, which a sixth expert, a UX designer at a large German carmaker, then evaluated.
The second cycle refined the guidelines after presenting them at HICSS 2019. The community’s feedback added trust as a topic, added the companionship concept, added the range of automotive contexts, and raised the possibility that a user might take the assistant into another car. The third cycle built a prototype to show that the guidelines could be applied.
03 / The guidelines
Fifteen requirements, four blocks.
Each requirement carries several guidelines, with the ones practitioners should prioritise marked in the paper. Two requirements stand out for the car in particular: reducing cognitive load, and finding the right moment to be proactive. A timely alternative route before the traffic jam, or a warning when vehicle data shows something needs attention, are the kind of proactivity the experts had in mind.
Representational design
- Give the assistant a persona with a name, background, and personality, and use it as a style guide
- Use familiar conversational norms and casual language that still fits the brand
- Prefer an abstract visualisation over a human-like avatar
- Never pretend to be human; be transparent about limits
Intuitive conversation
- Learn from how people signal whom they are addressing, instead of relying only on wake words
- Support undo, repeat, help, and stop
- Understand short commands and natural phrasing alike
- Design flexible dialogues the user can enter at any point
Simple and transparent operation
- Minimise cognitive load: do as much as possible for the driver, keep messages crisp, break processes into steps
- Calibrate feedback to confidence, to how critical a mistake would be, and to the user’s familiarity
- Explain why data is needed and what value it creates
- Handle errors gracefully; hand over to a human after repeated failure or in emergencies
Companionship
- Recognise and respond to the driver’s emotions, within limits
- Be proactive where it eases the drive or improves safety, and learn when proactivity annoys
- Be aware of context: route, weather, time, passengers
- Remember the user, but not things a human would have forgotten
04 / The prototype
A car cockpit in virtual reality.
Testing an assistant in real traffic was neither feasible nor safe, so the team built a virtual car cockpit with Figma and a VR add-on, placed the assistant where the radio would be, and scripted eight scenarios in Google Dialogflow: a quiet motorway drive to Berlin, cold weather, poor visibility, waiting at a light on the way to work, the same on the way home, a frozen road, several passengers, and a tyre change. In the poor-visibility scenario the assistant recognises the conditions and switches on the headlights itself. On the motorway it opens the conversation, offers some information about the destination, and remembers that the driver looked for a restaurant the day before.
Nineteen university students wore VR glasses while a researcher triggered scenarios and the assistant spoke through a speaker. Afterwards they rated the experience on seven-point scales and answered open questions.
| Statement | Mean, 1 to 7 |
|---|---|
| Using the assistant during the ride increased my comfort | 5.16 |
| Dealing with the assistant is clear and understandable | 5.53 |
| Overall, I find the virtual passenger easy to use | 5.68 |
| I have the feeling that the assistant understands me | 5.05 |
| Using the assistant made me feel entertained | 4.47 |
Most participants could imagine using the assistant in everyday driving. They praised the proactivity, the courteous communication, and the actions that went beyond the car’s own functions. They also found the voice too computer-like and some interventions unnecessary, the headlights among them. One participant’s verdict became a design lesson in itself: sometimes less is more. Several suggested longer conversations on night drives to fight fatigue, and sensors to read the driver’s mood. Without being asked, they were describing a companion.
05 / Where it led
The seed of virtual companionship.
The fourth guideline block defines the virtual companion as a conversational, personalised, helpful, learning, social, emotional, cognitive, and collaborative agent that interacts proactively and autonomously to build a long-term relationship. The prototype itself was conceived on a virtual companion canvas. Within a few years, that definition became a research programme: a design theory for virtual companionship, learning companions, and a study of what companionship does to the people who use it.
The paper also flagged what it could not settle. Trust and privacy got a single guideline and deserved more. And autonomous driving, it noted, may make the driver’s assistant obsolete and turn it into something that assists passengers instead.
Behind the paper
The idea, and
the path to it.
A paper presents the polished result. This is the part that usually stays unwritten: where the question came from and how the work actually unfolded.
Companionship first surfaced here, in the interviews for this study, long before it became the centre of my research.
The project was a collaboration with one of the world’s largest car manufacturers. A very interdisciplinary project team set out to understand how virtual assistants would work inside a car, and that gave us something researchers rarely get: real access to practice. We could hold expert conversations, run interviews, and analyse what the people building these systems actually thought.
The road to the journal ran through Hawaii. The first version was a HICSS 2019 paper, and presenting it there shaped the article decisively. The discussion at the conference brought trust, context, and the companionship concept into the guidelines, and it earned the paper a fast track into AIS Transactions on Human–Computer Interaction. The fourth guideline block is where my research went next.