Kakao Voice AI
A project brief received from Kakao ai to conceptualize an AI service using speech recognition and object recognition technology in the mobile environment.

Duration: 1 week

Role

Solo Project

UX Design

UI Design

Research

kakao-vui-cover2.jpg

Concept designs for an ai cooking assistant that finds recipes, gives step by step instructions, saves recipes to the user's cookbook, and calculates unit conversions along with general tasks such as playing music or sending text messages.

Inspiration

I love inviting over my friends, and cooking them a meal. And I'll often find a new recipe to try on my phone. But it's always such a hassle to access the recipe while both my hands are wet and full of ingredients. I need to look at the recipe on my phone, search online for unit conversion, and read texts from my friends, but the screen often turns off.

After getting a voice ai assistant in my house, I've found that it's much easier to look for unit conversions and set timers. But there still isn't a way for me to easily follow recipes on my phone while cooking.

This pain point was the inspiration for an ai cooking assistant.

User Testing and Role Play

In order to better understand the accessibility of our smart mobile devices while cooking, I conducted a diary study where I recorded myself cooking from a recipe. I counted the # of time when I used the mobile device while cooking and noted the reasons why I needed my  device. 

During the 20 minutes I was cooking using a recipe, I needed to look at the phone 10 times for these reasons:

  • View the ingredients list multiple times

  • Read the next steps, but the phone kept turning off

  • Convert unit measurements

  • Look at the photos in the recipe for reference

Current Experience Diagram

Studying Voice AI on Mobile Devices

Since I didn't own a kakao voice assistant, I took a trip to the kakao store to understand it's personality and how it works. I wanted to know its loading time to answer commands, its ability to answer general and specific questions as well as it's tone of voice and language.

 

In my interaction, I also learned that Kakao could not answer my command "헤이 카카오, 파스타 레시피 찾아줘" (Hey Kakao, find me a recipe for cooking pasta).

I also reviewed various voice assistants that I could get my hands on from Google, Siri, Clover by Naver, KakaoMap on my mobile device, to carefully observe their interactions.

Sketch Ideation

High level call flow

Dialogue Flow of new voice AI Experience

Dialogue diagram.png

Dialogue diagram (click to zoom)

Explorations

 

Final Designs

Voice interaction states

Motion prototype

Finding recipes using kakao voice ai

Retrieving recipes and getting recipe directions through kakao voice ai

Next Steps

  • Prototype the above voice interaction with another person or through a voice recording

  • Refine the language to achieve most natural human voice interaction

  • Go through all the possible ways a user might state the command

  • Cover multiple use cases for voice ai interaction while cooking

  • Design for those multiple use cases

  • Further refinement for accessibility

  • Potential consideration for localization to english