Kakao Voice AI
A project brief received from Kakao ai to conceptualize an AI service using speech recognition and object recognition technology in the mobile environment.
Duration: 1 week
Concept designs for an ai cooking assistant that finds recipes, gives step by step instructions, saves recipes to the user's cookbook, and calculates unit conversions along with general tasks such as playing music or sending text messages.
I love inviting over my friends, and cooking them a meal. And I'll often find a new recipe to try on my phone. But it's always such a hassle to access the recipe while both my hands are wet and full of ingredients. I need to look at the recipe on my phone, search online for unit conversion, and read texts from my friends, but the screen often turns off.
After getting a voice ai assistant in my house, I've found that it's much easier to look for unit conversions and set timers. But there still isn't a way for me to easily follow recipes on my phone while cooking.
This pain point was the inspiration for an ai cooking assistant.
User Testing and Role Play
In order to better understand the accessibility of our smart mobile devices while cooking, I conducted a diary study where I recorded myself cooking from a recipe. I counted the # of time when I used the mobile device while cooking and noted the reasons why I needed my device.
During the 20 minutes I was cooking using a recipe, I needed to look at the phone 10 times for these reasons:
View the ingredients list multiple times
Read the next steps, but the phone kept turning off
Convert unit measurements
Look at the photos in the recipe for reference
Current Experience Diagram
Studying Voice AI on Mobile Devices
Since I didn't own a kakao voice assistant, I took a trip to the kakao store to understand it's personality and how it works. I wanted to know its loading time to answer commands, its ability to answer general and specific questions as well as it's tone of voice and language.
In my interaction, I also learned that Kakao could not answer my command "헤이 카카오, 파스타 레시피 찾아줘" (Hey Kakao, find me a recipe for cooking pasta).
I also reviewed various voice assistants that I could get my hands on from Google, Siri, Clover by Naver, KakaoMap on my mobile device, to carefully observe their interactions.
High level call flow
Dialogue Flow of new voice AI Experience
Dialogue diagram (click to zoom)
Voice interaction states
Finding recipes using kakao voice ai
Retrieving recipes and getting recipe directions through kakao voice ai
Prototype the above voice interaction with another person or through a voice recording
Refine the language to achieve most natural human voice interaction
Go through all the possible ways a user might state the command
Cover multiple use cases for voice ai interaction while cooking
Design for those multiple use cases
Further refinement for accessibility
Potential consideration for localization to english