STEAM POWERED UI: Empathy Machines

This project created a galley exhibit that featured the work of young digital artists using a unique interface. Students from Gameheads Oakland came together to make traditional art that was then animated by our team. A gallery-goer could then cycle through the both their work and a second screen that shows information about the artist. The whole experience was controlled by buttons reverse engineered from a Guitar Hero controller.

My Role: UI Design, Project Management, Community Outreach

Team: Elsie Huang, John Tait, Niloo Khodabakhsh

Design Challenge: How might we engage young makers in STEAM (Science Technology Engineering Arts & Mathematics) powered projects?

Solution: Actively work with students to demonstrate how their traditional art work can be transformed into digital, interactive content.

Technology Used: Processing 3.0, MakeyMakey, Adobe CS


We started by identifying a target age group and various organizations that were open to working with us on a collaborative multimedia project. Luckily, Damon Packwood is an alum of our graduate program and the founder and director of Gameheads Oakland. His organization teaches kids in under served communities to design and code games. We created a pitch deck for him and then a simple questionnaire for his students to gauge interest and schedule a creation session. We found four awesome participants who were interested in learning more about translating their illustration skills to the screen.


We wanted to reveal two screens simultaneously operated by one button each. We first used java-based programming to take the digitized artwork and animate it. We then developed a second screen that showed a picture, a bio and the original illustration.


We developed several working prototypes that connect two MakeyMakeys, each wired to the same set of button presses. One controlled a Mac Mini to project the visualizations. A second computer cycled through the bios. We used a Guitar Hero controller to stay on theme with game play and game design.



The Toys They Carried is series of data visualizations based on the percentages of child refugees in the Middle East. Our team created a JSON file that aggregates data based on the the most recent UN data of children seeking asylum from Middle Eastern countries. We then used a ragged and singed doll as the interface.

My Role: UI Design, Co-developer

Team: Niloo Khodabakhsh

Design Challenge: How might we present data in a form that triggers a deeper empathy in individual users?

Solution: Use an object as interface to better connect a user to numbers on a screen.

Technology Used: Arduino, Firmata, Processing 3.0, JSON data






We developed a generative visualization convention to show the number of refugees displaced from a given country and then compared that to the percentage who were children under the age of 18.

We then embedded a button in the doll to call the visualizations. Typically when you squeeze a doll it says “Mama”, but in this case it shows the startling statistics of war’s youngest victims.

Based on how the data is retrieved from the JSON file we wrote, there is potential to add new stats from different countries or build on what information is compared.



Road to Nibanna uses an RFID system to call generative visualizations and original music. When tokens are placed on the alter, an animation and musical sting initiates that corresponds to one of the seven chakras. The RFID chips were hidden in the objects and the reader embedded in the alter to evoke a calming, mystical experience.

My Role: Sole Creative Technologist

Design Challenge: Create an interactive experience using Java that does not use a keyboard or a mouse.

Solution: Embed RFID chips in objects to call multimedia content.

Technology Used: Processing 3.0, RFID Chip reader, Garage Band