Aug 18, 2016

Sound is the way we primarily experience music, but there is also something visceral about sitting in a concert hall as an orchestra plays at full tilt.

A London-based company who specializes in wearable technology has recently unveiled a project that looks to be able to turn up the volume on the way we feel that concert so that a hearing-impaired audience might be able to experience music in a new way.

The Sound Shirt is a wearable tech product, produced by the English firm CuteCircuit, that has been more than a decade in the making. Inside the shirt are small actuators or motors that vibrate in specific places and patterns based on commands given to them by software. CuteCircuit partnered with the Junge Symphoniker, an orchestra in Hamburg, Germany, to create a new concert experience for a deaf audience. Watch the shirt in action below:

Ryan Genz is the co-founder of CuteCircuit along with Francesca Rosella. The company has been combining fashion and technology — including recently producing designs for singer Katy Perry — since 2004. The Sound Shirt with the Junge Symphoniker uses microphones located at different positions onstage that translate that information to vibrations at different parts of the body. Genz explains that he wanted the music to be mapped to the body in an intuitive way.

“Say you're at a loud concert venue of electronic music. When you feel the bass, you always feel it in your chest and belly and kidneys. But higher notes you feel higher in your extremities because that's where you get the resonance,” said Genz. “We sort of mapped them that way. You have the deeper notes from the cellos down in the lower back, and then the higher notes are up in the shoulder blades and the collar bone.”

Genz also said that he wanted the person wearing the shirt to be able to quickly recognize the patterns between what they saw onstage and what they felt.

“They can see the violins playing with their eyes and understand that when the violins make a note, they feel it around the clavicle of the shirt. Within a few minutes, you start to be able to intuitively parse the code,” he said.

For Genz, the Sound Shirt is one of hopefully several applications for a technology that he and Rosella have been working on since the early 2000s. The design duo produced their first prototype for this kind of wearable tech in 2002, but the purpose of the tech was as a kind of touch telecommunications device. The Hug Shirt measured the strength, location and duration of touch as well as everything from the heartbeat rate and skin warmth of the wearer. The shirt would then allow the sensation of a hug from a distant loved one to be transmitted and recreated.

“We worked on it very slowly, diligently over the years to be able to produce better and better versions,” Genz said of the Hug Shirt. “We're at the point now where it's actually a stable system. We're not ready to mass produce it yet, but we're very close. The prototypes we're producing are much more durable and wearable and light weight. They feel just like normal clothes.”

As they started showing the shirt to different people, Genz said they were met with a variety of ideas for what their design might be used for. Members of the deaf community saw possible applications, one of which was realized with the Sound Shirt, while some high school students who saw the shirt imagined a secret communication system they could use while in school.

“It’s great because we always intended to design something with the Hug Shirt that is an open system. A computer doesn't do one task. It's an interface that is capable of doing a lot of different tasks,” Genz said.

Informally, the “hug” in Hug Shirt has become an acronym within CuteCircuit, referring to a Haptic User-Interface Garment. The Sound Shirt used by Jungen Symphoniker is a Hug Shirt, using an identical system, but it runs on different software.

Since the orchestra collaboration launched, Genz said they’ve been contacted by a handful of other symphonies, including two more orchestras in Germany who want to offer the system to their patrons. Closer to CuteCircuit’s London home, they’ve also been contacted by the UK’s National Deaf Children’s Society.

Because of the scale needed to bring a product to market, Genz said they are looking into what other uses might be found for their technology, and music might offer some more avenues. For example, he said it could be possible to stream a concert and have a user feel the concert in real time. For audiences with hearing, it can also add a new dimension to music. Genz points to the experience of attending a show by the electronic music group Kraftwerk.

“I know I've been to shows … and I'm not really sure which musician is making what sound,” he said. “But if you sort of encoded that information into a Sound Shirt, that could make it even more dimensional when you realize one of the band members is always making sounds that you feel on your left side.”

Ultimately, these other uses could help make the Sound Shirt commercially viable and available to the deaf community all over the world who want to experience music in the same way as what is currently offered at Jungen Symphoniker.

“Making it specifically for that need probably wouldn't meet that critical mass for manufacturability and would keep the price prohibitively high,” said Genz. “But if we can make a Sound Shirt product that also meets interesting needs for other communities, then we could have a product that is manufactured at a decent price point, and I think that's the real goal.”