Google Nebula
Nebula was created during a Google-sponsored studio at ArtCenter College of Design — a collaboration with product designer Sara Ferris and Jiacheng Yang. The brief: design creative tools for the year 2025 using AR and VR. Our answer was a system that turns any physical object into a controller for a digital environment, built around how children between 4 and 8 already naturally play.
My Role: UI design for the phone-to-screen interaction system, conductive mat interaction design, coordinating and running user testing, and physical-digital connection architecture.
Year
2018
Client
Google at ArtCenter College of Design
Role
Ux designer and researcher
The Insight
Nebula allows any physical toy or object to act as a controller for things in a digital space.
Kids between the ages of 4 and 8 naturally use common objects to imagine and create stories. Nebula builds on this natural trait to create a new digital experience utilizing physical objects.
Kids between the ages of 4 and 8 naturally use common objects to imagine and create stories. Nebula builds on this natural trait to create a new digital experience utilizing physical objects.
Nebula allows any physical toy or object to act as a controller for things in a digital space.
Kids between the ages of 4 and 8 naturally use common objects to imagine and create stories. Nebula builds on this natural trait to create a new digital experience utilizing physical objects.
Kids between the ages of 4 and 8 naturally use common objects to imagine and create stories. Nebula builds on this natural trait to create a new digital experience utilizing physical objects.
How It Works
A conductive fabric mat detects any physical object placed on it and maps it to a corresponding element in a digital scene on screen.
Kids build worlds by grabbing whatever is nearby — no special toys required, no learning curve. The phone serves as the bridge between the physical mat and the television screen, translating object detection into real-time digital response.
User Flow
SETTING UP A SCENE AND A TOY
Users can
- Create a new world and populate it with physical objects
- Navigate and edit their digital scene using voice commands
- Record and share their stories through a curated Nebula YouTube experience where public content created on the mat lives
- Navigate and edit their digital scene using voice commands
- Record and share their stories through a curated Nebula YouTube experience where public content created on the mat lives
User Flow
SETTING UP A SCENE AND A TOY
Process
Experimentation
Process
Experimentation
Process
User testing
We conducted two rounds of user testing with children aged 8 and 12. The sessions revealed two critical design tensions that shaped the final product:
Voice commands vs. processing limits
When discovering ability to edit scene with voice, kids tend to ask for outrageous commands. Our system had interpret the command in a way that doesn't over load the processor but also satisfies the users' interest
Mat durability vs. tactile invitation
Kids immediately started grabbing everything around them to test on the mat — and they weren't gentle about it. The mat had to be structurally strong enough to withstand aggressive swiping while remaining soft enough to invite that kind of fearless physical interaction. Hard and durable felt wrong. Soft and inviting had to also mean resilient.Both insights led to design decisions that made the experience feel effortless precisely because of the constraints working behind the scenes.
