Umbrella Ghosts

Year: 2017
Role: ARVR Designer/Developer
Exhibition: NYFW '17


Umbrella Ghosts is a collection of interactive fashion garments combining textile design with augmented and virtual reality technology. Narrated through cultural fears and superstitions, the collection constructs an artistic digital atmosphere for viewers to engage in dialogues with the garments.

The project is supported by LIFEWTR, and debuted at New York Fashion Week ’17. Check out Women’s Wear Daily’s coverage on NYFW ’17 about the project.

“Scanning the AR triggers on the prints of each garment in the collection will unlock virtual world in which ghosts can seem lively and fun, and we see our world anew.”

Council of Fashion Designers of America


Working with fashion designers from ItchyItchy Studio, I defined the look and feel of the ARVR experience by translating textile design into the digital space. Following a standard iteration process, I designed and developed four ARVR experiences through collaborative brainstorming, interviewing, wireframing, prototyping, user testing, and feedback iterations.


By reimagining the intersection between fashion and technology, we set out to redefine how runway design collections are viewed outside of traditional platforms. Using cutting-edge augmented and virtual reality technologies as tools for digital storytelling, these fashion garments formed a collective playground where viewers can navigate between physical and virtual worlds to engage with the garments.


In order to translate textile design into 3D assets, I first sketched out concept arts that pinpoint what and where different objects will be placed in a 360 environment. Through discussions with the fashion designers, these sketches were later converted to Unity scenes based on the feedback.

Textile prints designed by Tiffany Huang.

Textile Print (Cactus)

ARVR Perspective Concept Art (Cactus)

VR Environment (Cactus)

Textile Print (Eyeball)

ARVR Perspective Concept Art (Eyeball)

VR Environment (Eyeball)

Textile Print (Rain)

ARVR Perspective Concept Art (Rain)

VR Environment (Rain)

Textile Print (Heart)

ARVR Perspective Concept Art (Heart)

VR Environment (Heart)

ARVR Design

From AR to VR

One of the challenges designing Umbrella Ghosts is creating a seamless experience for mixed reality ARVR environment. I referenced Vuforia’s guide on Best practices for mixed reality AR/VR experiences to design the user experience in both AR and VR.

AR Mode

In AR Mode, the user sees 3D digital content overlaid on top of each garment through their smartphones. The digital content blends in with reality and becomes an extension of the fashion garment.

VR Mode

In VR Mode, the user enters a fully immersive environment where the garment’s design extends into a virtual world, embodying the user. In VR, the user is able to interact with 3D objects using gaze trigger and timer.


Umbrella Ghosts utilizes Vuforia Unity API (AR) and Google VR SDK for Unity (VR) technology to create an immersive mixed reality experience for Google Cardboard. Building on top of SDK, we implemented gaze detection on 3d objects to trigger transitions between the AR and VR modes.


Rethinking AR Triggers For Better UX

Version 1 of Umbrella Ghosts uses a reticle timer and a 2D image as the AR Trigger that leads user from AR Mode to VR Mode. When the user gazes at the 2D image, a circular timer pops up and start running. The user needs to keep gazing at the image until the timer stops. From there, the user is transitioned via a fade to black effect to the VR Mode.

We user tested version 1 with fashion design students from Parsons and creative technologists within our network. The overall feedback indicated that the experience felt visually cluttered, and the 2D elements were distracting users from being immersed in the experience. User testers reflected that the transition from AR and VR felt unnatural to them because the 2D and 3D elements didn’t blend well together, and ended up fighting for attention.


To address the feedback that we received, I redesigned the AR to VR experience by adopting a more natural interaction that leads users from AR to VR.

First, I removed the 2D reticle timer and 2D images that were used for gaze trigger. Since the AR Mode already has a layer of 3D elements on top of the garments, having additional 2D information overlaying the 3D elements makes the scene chaotic, and hard for users to process.

After minimizing the visuals in the screen, I added a glow effect to 3D elements for highlighting user interaction. When user gazes at the 3D object, the object’s exposure doubles, indicating that the user is currently interacting with the object. When users continue to gaze at the object, the screen starts to fade to white. When the screen fades entirely to white, the scene transitions from AR to VR, and brings the user to the VR world via a fade in.

Using the combination of highlights and fade-to-white effect replaces the need of using the traditional 2D reticle timer to inform users about scene transition. The resulting design shows a cleaner, friendlier interface, and facilitates a deeper ARVR engagement with users.

NYFW ’17

During New York Fashion Week ’17, Umbrella Ghosts was unveiled at the CFDA x LIFEWTR Spring/Summer 2018 presentation at Pier 59 Studios. The event generated media attention from Business Insider, Women’s Wear Daily, and Fashion Week Daily.

LIFEWTR Thirst Inspiration Commercial

The LIFEWTR Thirst Inspiration campaign features work-in-progress videos of the process working with fashion designer Tiffany Huang to combine textile design with augmented and virtual reality technology.