When a Gesture Becomes a Feeling

Problem: In immersive environments like VR and mixed reality, users often rely on controllers that limit how naturally they can convey emotion. Avatars lack facial expressions, subtle cues, and the human signals that make communication feel authentic. As a result, users struggle to express emotions or reactions in real time, creating a communication gap that reduces presence, trust, and overall immersion. We needed a fast, intuitive way for people to express emotion without breaking flow, overloading cognitive load, or generating false positives.

Solution: I led the interaction design for a gesture driven emoji expression system that enabled users to quickly trigger emotions through hand based shortcuts. By combining ergonomic controller exploration with gesture design and input device prototyping, we created a system where expressive icons could be invoked naturally through the user’s hands, without interrupting the experience. This work included studying hand ergonomics, minimizing gesture ambiguity, and evaluating new controller shapes that improved comfort while reducing accidental activations. The result was an expressive communication layer that allowed avatars in AR/MR environments to feel more human, responsive, and emotionally present bridging the gap between physical and virtual interaction.