Kakao Voice AI
A project brief received from Kakao ai to conceptualize an AI service using speech recognition and object recognition technology in the mobile environment.
Duration: 1 week
Role
Solo Project
UX Design
UI Design
Research
Concept designs for an ai cooking assistant that finds recipes, gives step by step instructions, saves recipes to the user's cookbook, and calculates unit conversions along with general tasks such as playing music or sending text messages.
Inspiration
I love inviting over my friends, and cooking them a meal. And I'll often find a new recipe to try on my phone. But it's always such a hassle to access the recipe while both my hands are wet and full of ingredients. I need to look at the recipe on my phone, search online for unit conversion, and read texts from my friends, but the screen often turns off.
After getting a voice ai assistant in my house, I've found that it's much easier to look for unit conversions and set timers. But there still isn't a way for me to easily follow recipes on my phone while cooking.
This pain point was the inspiration for an ai cooking assistant.
User Testing and Role Play
In order to better understand the accessibility of our smart mobile devices while cooking, I conducted a diary study where I recorded myself cooking from a recipe. I counted the # of time when I used the mobile device while cooking and noted the reasons why I needed my device.
During the 20 minutes I was cooking using a recipe, I needed to look at the phone 10 times for these reasons:
-
View the ingredients list multiple times
-
Read the next steps, but the phone kept turning off
-
Convert unit measurements
-
Look at the photos in the recipe for reference
Current Experience Diagram
Studying Voice AI on Mobile Devices
Since I didn't own a kakao voice assistant, I took a trip to the kakao store to understand it's personality and how it works. I wanted to know its loading time to answer commands, its ability to answer general and specific questions as well as it's tone of voice and language.
In my interaction, I also learned that Kakao could not answer my command "헤이 카카오, 파스타 레시피 찾아줘" (Hey Kakao, find me a recipe for cooking pasta).
I also reviewed various voice assistants that I could get my hands on from Google, Siri, Clover by Naver, KakaoMap on my mobile device, to carefully observe their interactions.
Sketch Ideation
High level call flow
Dialogue Flow of new voice AI Experience
Dialogue diagram (click to zoom)
Explorations
Final Designs
Voice interaction states
Motion prototype
Finding recipes using kakao voice ai
Retrieving recipes and getting recipe directions through kakao voice ai
Next Steps
-
Prototype the above voice interaction with another person or through a voice recording
-
Refine the language to achieve most natural human voice interaction
-
Go through all the possible ways a user might state the command
-
Cover multiple use cases for voice ai interaction while cooking
-
Design for those multiple use cases
-
Further refinement for accessibility
-
Potential consideration for localization to english
