Mandarin learning VR APP
Sep 2018 - Sep 2018
UX Designer / UX Prototyper / Developer
Made with Unity, Google Cardboard
INTRO
The biggest benefit of using virtual reality to learn a foreign language is that it allows people to step into an immersive and authentic foreign location to learn a new language without stepping out of their homes. So when they travel to these places in real life, they will be well prepared to communicate correctly. It creates a great virtual environment for learners to grasp language skills in daily communication. This application allows language learners to learn, practice and review the foreign languages in real-life scenarios.
Process: Information Architecture, User Flow, Sketch, Physical Prototype, Digital Prototype.
DESIGN PROCESS
Main Features
More immersive and direct way to learn a foreign language via VR technology
​
- Learning. Learners are allowed to select different real-life settings like shopping in a virtual shopping mall or ordering a lunch in a foreign restaurant to learn the language in a more immersive & realistic scenario. Those realistic daily-life stories will vividly teach users about how to use this language.
​
- Practicing. By chatting with the avatar of language teachers or matched study pals, language learners can improve their language skills with vivid virtual friends in a more realistic chatting environment, such as a virtual cafe.
​
- Reviewing. Users are allowed to save the content that they are unfamiliar with or want to review later in their virtual notebooks, in the form of video, text or audio. Or just go to the real settings to review what they learned.
​​
Information Architecture
The purpose of information architecture for designers is to help us visualize the abstract features on a figurative map that indicates the location of those information and the connections between them. A reasonable information map enables users to quickly navigate between different information and then accomplish their tasks.
Abstract Features → Figurative Information Map
A user-friendly VR application should naturally penetrate the architecture of this virtual world to users by setting logical locations for different features. In this case, the main task that we need help users to achieve is to learn Mandarin in various real-world settings, which brings the key requirements for the information architecture that it should allow users to easily select and navigate between different learning scenarios without feeling lost in this world. Thus, in this design, I set up a start room before users can dive into learning scenarios. It is like a home page of the website, where users are able to review what they learned as well as make plans and decisions about where to learn. In the next stage, I'll create an interactive component -- Anywhere door -- to assist users to conveniently and efficiently navigate between scenes. I visualized the main features by showing the corresponding information with locations and the connections among them.
User Flow
By extending the detailed step-by-step actions of each feature
we can fully consider what will happen in the process of each activity that this application allows users to do.
​
At the beginning of writing the story for this app, we should think about what device we allow users to use in this VR application. It’s because there are too many possible methods to input in a VR application like using different types of controllers, hand gestures, voice input or just one button, which will bring totally different design considerations even for the identical task of an APP. As shown in the following figure, the column “media” describes the supported interactions in this application, what matters for the next stage of design.
In the following diagram, I used five “W” way — where, who, do what, why and how to help me depict the whole story about how those features work in this app.
Interaction Design Consideration
Navigation and reviewing are a little bit tricky to design
Difficulties
There are two difficulties I encountered during the design. One is how to navigate in such located-based VR application without breaking the immersion of the application. Another one is how users review the learned content in a VR application which matters to an effective learning process.
​
Possible Approaches
For the navigation part, the current market of VR language learning products tends to use a round menu for users to navigate between the scenes as shown in the figure on the left side (Mondly). For the review part, none of them has this features but I think it is a necessary feature for a language learner.
​
My solution
Anywhere door and notebook on users’ virtual hands. I will explain two interaction components in the following article.
Sketch
Sketch is the first step of the process that we transform our design considerations to the figurative application.
Even though the appearance of a VR application is 3D
the 2D flat sketch can still work as a great visualized foundation for the latter design.
The Difficulties:
Sketch the 3D environment on a 2D paper could be a little bit time-consuming because we should keep perspective in mind; except for the information in front of users, the left and right and back side of space should be taken into account.
My solution:
My approach to efficiently visualize my design idea is extracting the most critical interaction scenarios to draw. The sketch might not be a well-perspective 3D space, but every sketch tells the story about the essential part of a scenarios corresponding to one main feature.
Features - Collect important learning contents into your notebook. When users open their hands, there is an icon of the notebook in the center of their palms. Users can tap on it and then the big notebook will appear on their hands.
Features: Chat with a NPC.
Two sphere buttons are floating on the right side of this virtual waitress that allow users to open the dialog box and save the sentences they feel important. In the dialog box, users can view the subtitles and re-play the animations.
Features: Navigate between different scenes.
Anywhere door, which will show up when users want to go to someplace. There are three buttons that require users to select the mode: learning by watching NPCs conversation or practicing by chatting with NPCs or real-users conversation. Then there is a map on top of the three buttons for location set-up. For example, if users selected the learning mode and restaurant, then the door opened, the scene will switch to the NPCs-chatting-in-the-restaurant scene.
Prototype
Rapidly iterating our ideas through prototypes for a VR application becomes more complicated
because we need to build a 3D world for users to interact with
and the interactions happen here changed a lot compared with a 2D flat-screen application.
I selected two main tasks of this application to do the prototype. One is that users practice their speaking skills by chatting with NPCs in a restaurant and another one is that users chat with real users to apply what they learned in the real life.
Low-Fidelity Prototype
Goal: Tell the story of those two tasks and test how our audience perceives it.
Method: Physical Prototype. The process of building 3D prototypes with Play-Doh help us to capture some details of the interactions that we probably missed in the 2D sketch like how to place those characters and models before time-consumingly diving into a 3D development tool like Unity.
​
High-Fidelity Prototype
Goal: View the design in the real VR device
Method: Unity and Cardboard
Result:​​​​
-
The transition between the scenes is extremely important for users to understand the relationships between those scenarios.
-
Avoid strong lighting changing.
Development Process
Similar Works​
 VR 3D MODELINGHTC VIVE + Unity 3D |  Gazed-controlled Fetching RobotARKit + Ardunio |  VR ResumeA-frame |
|---|---|---|
 VR Mandarin Learning APPPhysical Prototype + Unity 3D |  AR Grocery ShoppingA-Frame |  AR Inspiration CollectorA-Frame + Space AR |
 Future WorkspacePersonal Research Project |  AR DrawingKeynote |  AR ReadingKeynote |
 Power Strip RedesignRhino + Keyshot |  AR Picture BookDesigner + Prototyper |