Chatting with an AI Agent While Driving

Collecting and Analyzing Speech Data to Optimize the Conversational Experience for Tmap, South Korea's #1 Navigation Platform

💫 Released

📰 Press

At this time, it was a pivotal time for research on user needs for AI in real-life scenarios, and during this time, I developed an AI conversation guideline specifically for driving situations.
Starting with experimental design, I created an environment that simulated both driving situations and the actual use of the AI agent to gather user needs.

By analyzing the collected sentences, I identified user needs and established guidelines based on them.

The resulting output was delivered to the business team of Tmap, South Korea's #1 navigation application with 20 million monthly active users, implemented in actual services, and has been in operation since 2024.

May - Aug. 2017

Team project
7 researchers

My Role
- Designed experiments
- Conducted experiments and in-depth interviews
- Analyzed speech data
- Write user needs part in voice user interaction guideline

Need to design a virtual driving test environment to learn users’ needs and improve in-car AI assistant.

As of 2017, there was little in the way of a standardized method for collecting the needs of AI assistants while driving.
‍

Need to establish an experiment environment simulating real driving for a safe survey.
‍

To build an appropriate and safe experimental environment, adapted a Wizard of Oz approach. To identify real user needs, captured various driving situations through photos. Through pilot tests, collected natural utterances for the prepared photos and established refined dialog sets based on them.
‍

Progress in designing an experiment

2. Main Test

In the main test, collected 2,721 pieces of speech data and extracted 5 use cases through conversation analysis.

Progess 01:
Collected 2721 speech data through OZ dialogConducted a test that provides a feeling of conversation with a real in-car agent using two rooms to 34 drivers with 2+ years of experience. After test, implemented in-depth interviews to determine their impressions of AI assistant.

Progess 02:
Analysis of speech data through thematic analysisTogether with three other researchers, I established criteria for analyzing sentences based on object, needs, and source through sample coding. We initially focused on these key components for the AI assistant’s understanding, and later refined the criteria by analyzing 2721 speech data via thematic analysis.
‍

Extracted 5 use cases and also listed the detailed needs corresponding to each use case.

I identified 5 use cases and 27 specific needs. Beyond basic navigation, the use cases highlighted the need for integrating with existing databases, creating new ones, and connecting with vehicles or smartphones, especially for inquiries about surroundings or casual conversation during driving.

Moreover, due to the unique context of driving, I recognized the necessity of a conversation design that can quickly and accurately understand the driver's intent.

Iterated design through user test of conversation and the navigation app.
The integration of an AI assistant into the existing navigation app necessitated UI design user testing to ensure harmonious coexistence of both functionalities within a single application. User testing was conducted with two design alternatives (A and B) across 10 distinct tasks with 8 drivers with their own cars.

Example of testValidated the most suitable approach for text representation of navigation voice during driving, whether to utilize the entire screen or only half.

The comprehensive Voice User Interaction (VUI) Guidelines for navigation were established through the synthesis of data collected from pilot testing, main testing, and user testing phases. I primarily contributed to the detailed analysis of user needs. The guidelines encompassed systematic elements including personas, design principles, dialogue structures, and database components.

1. AI conversation design principles

2. Dialog Structure

Added two new systems to reduce errors with the goal of achieving the desired final result within 1-2 turns while driving: Slot Auto-fill, Best Guess

Takeaways

‍
In 2017, integrating AI into an existing app was a new challenge for me. Going beyond conventional qualitative research, I carried out the entire process from systematic experimental design to implementation and delivery of actual results. The fact that these results have been applied to commercial services for seven years until now (as of 2024) demonstrates the effectiveness of the research.

Beyond my undergraduate experience with sentence symbolization in linguistic philosophy, applying the practical methodology of thematic analysis to conduct large-scale sentence analysis and establish criteria became a significant turning point in my career.

I am confident that the sentence analysis experience gained through this project will become a core competency for future AI projects, and I look forward to participating in more projects in this field.