Hullabaloo (喧囂) – Tangible UI

A little over 12 years ago I was working on a number of projects that explored tangible and ambient interfaces. As my feeling at the time that websites and software were so ephemeral, it was an area that was very interesting to me, especially with their potential application to products for children. Physical and environmental UI have the potential for greater permanence. My daughter at the time was a great inspiration as she would use all kinds of household objects to create music. Since the basic affordance of these objects lent to their ease of use, and they lacked intimidation, I started to wonder what would happen if we attached sensors to the objects that could allow for basic software control. We could create all kinds of different noises!

This project was the first step towards more full featured experiences and utilized custom sensors that could only send on/off signals to the computer. As they were embedded in everyday objects, like pillows, this simplicity was largely hidden from the user. This operational simplicity restricted exactly what kind of experience we could create and challenged our imagination (the end result was much like Simon, from my youth). Additionally, this allowed for the focus to be on game development and music, not the interface. Later projects we created sensors that were much more sophisticated.

Hullaballoo utilized custom made switches, wireless controller, pillows, IKEA chairs, and software to make for an entertaining musical game for kids.

I lead the concept, with brainstorming help from the team, interface design, research, sound design, and arranged and mastered 9 pieces of music to be used in the games. 施香蘭 helped with the exhibition and created the beautiful pillows. 陳建泰 was lead engineer.

Overview and Background

The name Hullabaloo means great noise or excitement and seemed like an appropriate term to describe the device I set out to create.

This implementation of Hullabaloo involved embedding a simple constructed switch into the seat of a wooden chair. This version aimed to allow children to have the ability to control a voice from simple melodies in an overall composition. It worked well but testing found that chairs themselves lead to surprise, not attraction, and did not lead to the kind of engagement we were looking for. In order to attract children to the device and make interaction more comfortable, a change in form was needed. As well, since the interaction was to be quite simple, we decided to abandon the interacting with a musical composition idea and to create a musical game, and make the interface as portable as possible. With that direction we explored additional objects to add to the chairs, and eventually found that pillows would prove to be a great choice (this process was very much trial and error, we didn’t have the opportunity for lengthy a/b testing at this point). Pillows are an extremely common object in many peoples environment and represent an opportunity to allow for a great deal of aesthetic expression. They are approachable and may either elicit a sense of play or comfort.

Hiroshi Ishii’s Music Bottles were a great early influence on much of this work. In his paper “Bottles as a Minimal Interface to Access Digital Information” with Ali Mazalek, and Jay Lee he writes about the need to “explore the transparency of an interface that weaves itself into the fabric of everyday life”, an idea that meshes well with a number of the projects we were working on. In selecting pillows as an interface, and this applies to all the interfaces I created, there were a number of important considerations. First, when we add new digital meanings and functionality to inert physical objects, we need to maintain coherency of the conceptual model in both the physical and digital worlds. This requires seamlessly extending the metaphor and built-in physical affordances of objects to the digital domain. Secondly, when designing the pillows interface it was necessary to identify a fundamental set of interactions that were both appropriate to the task and also compatible with the available sensor technology.

The basic affordance of pillows is to provide support and comfort. In addition to this there are some basic interactions such as squeezing and throwing. Due to sensor limitations, unlike the Music Bottles I was not yet able to create more sophisticated gestures. The lack of gestures allowed us to keep the interaction model as simple as possible, creating a minimal design that allowed for pillows most basic affordance, comfort.

This affordance lead us to fix the pillows to a chair and not as an independent object.

The Game

The inspiration to the design of the game for Hullabaloo was university music listening classes and Simon, a game from my youth. In university it was common practice to walk into class and have a professor put a LP on the player, simply drop the needle anywhere, and then require you to identify the piece of music and the location within that work. This game was much less intimidating and primarily designed to test children’s listening and memory skills. A desired secondary effect is the introduction of various pieces of music in general.

The game is played as a group and requires some initial coaching, but kids showed that they could learn the interaction very quickly.

The game scenerio is as follows:

    Using the hullabaloo software GUI, the coach or teacher chooses a piece of music of a varying level of difficulty then plays the song in its entirety. The song can be then played again or it can be decided to continue on to the challenge.

    Upon deciding to move on, the software then randomly divides the song into four pieces and each piece is assigned to one chair.

    With help from the class, the child is then required to sit on the chairs to hear which piece of music has been assigned to which chair and try to put them in the proper order. The child is rewarded with a success signal if they get the correct piece and a error signal if they are wrong. They are then required to remember the order of the chairs to sit in them in succession in order to play the song in the proper order.

Quick technical sketch of an early version of the sensor placement


The initial design for Hullabaloo called for 4 chairs with matching pillows. The pillows themselves were fixed by fabric to the chairs so as to ensure that they stayed in place. Activating the interface is fairly simple, the user must simply sit on the chair, an action which activates a switch. The switch itself was hidden inside each pillow, one switch per pillow. Upon activation the chairs communicate with a software client running on an iMac and produces sound through a sound system. The switch itself was fashioned with standard off the shelf hardware and a simple switch to form the circuit. The game itself was run using a color user interface that could be used by users with any level of experience.

Testing results

User tests proved the concept, but there were some shortcomings. One of the most frequent comments was the lack of any kind visual reinforcement in the interface, we relied primarily on aural feedback. Examples provided included, when a child sits on the chair a light could change colour, and a correct answer produces a green light while an incorrect answer a red light. The goal of audio signals was to see if it is possible for children to enjoy interacting with a device with only audio reinforcement. In a world dominated by the visual I wanted have children focus with their ears. In testing this proved to be more difficult than imagined.

Another frequent criticism was the selection of music. As this was an early prototype I didnt take the time to arrange and mix more than two pieces of music. From this criticism came a suggestion which was to be incorporated into the next version. It was suggested that I include the ability to select different skill levels and corresponding pieces of music. As I had planned on creating a large library of music, it seemed that creating a easy, medium, and hard categorization would be a worthwhile addition to the software client.

An early version being tested:

Hullaballoo being tested at Hsinchu International school. The results were promising enough to create a proposal to allow for a more advanced prototype.

Custom pillows and smaller sensors: