Touch to Talk

A Wearable Representing Social Media Metaphors


Overview

Problem

Bringing social media metaphors from the digital world to the real, physical world

Solution

We have proposed and prototyped a wearable that represents two social media metaphors: poke and labeling people as friends, acquaintances, and strangers. We aim to bring these metaphors into the physical world and understand the interactions that follow.

Project Duration

This was undertaken as a course project from September - December 2017.

My Role

We were a team of 4. My primary role was of ideation, design and development.

Academic paper

Here is a link to the CHI 2018 late breaking work paper.

Design Process

Introduction

We introduce a wearable prototype that highlights the wearer’s interactions with friends, acquaintances, and strangers in a social gathering through a playful and exploratory manner. This work aims to bring social media metaphors of ‘poke’ and ‘categorizing the people one knows’ into the physical world. Facebook allows unique features based on the level of intimacy one shares with the people on one’s list of friends. For example, Facebook prioritizes news feeds of people that one marks as friends over those marked as acquaintances. Moreover, it does not allow one to view certain elements of information shared by people not on one’s friend’s list. Similarly, we investigate how the same interaction pattern between one’s friends, acquaintances and strangers can be represented in the physical world through elements that are playful in nature. We also include the ‘poke’ metaphor of Facebook in the design.

Aim

hope to provide a platform that encourages people to understand and interact with social media metaphors in the physical world, rather than completely relying on the virtual world. We also hope that the design encourages people to interact with a wider audience.

Research Questions

Through our work we aim to answer the following research questions: How do people interact socially while wearing the design? Does the design encourage social interactions? Do people feel they have interacted differently than without the wearable? What do people feel about the social media metaphor that has been brought into the real world? What is the reaction of passerby’s? How do people interact with their own wearable and others’ wearables? Do they perceive the person differently by making sense of what the wearable represents? What mental model of the interaction do people form?

System Design

Our design consists of a T-shirt with 3 patches and LEDs arranged in a grid-like manner. Each patch represents a level of social interaction - friends, acquaintances, and strangers. As the person wearing the T-shirt interacts with different people, they press the patch corresponding to their interaction. This results in some of the LEDs being lit up and as more people press the patches, more LEDs light up with a dynamic brightness and color which are based upon the number of different types of interaction that the wearer has had. The image below shows the front and back of this wearable



The brightness and color of the LEDs at any given point of time is a fusion of all the interactions till that point. The LEDs light up to work their way towards forming a pattern. When 49 LEDS light up, the entire pattern is revealed. This design has been intended to be used in casual social settings such as graduate mixer parties.

Implementation

Our current design consists of a vest which has 300 LEDs that are connected to an Arduino-compatible microcontroller called Flora and are all stuck on to the vest in a grid-like manner. The three conductive fabric squares that comprise of the patches have been stuck on to a long rectangular cloth. The Flora senses a touch on the conductive cloth through capacitive touch sensing. The cloth containing the patches is attached to the Flora through alligator clips (that are tucked inside the cloth), making it adjustable in length and flexible in positioning. The flora and LEDs are charged through a portable battery pack that is kept in the pocket of the wearer. People can wear their own T-shirts/tops over this system. The image below shows the vest and the patches



Interaction

Each patch is mapped to the number of LEDs that light up, the brightness of the LEDs as well as the color added to the existing color palette. We have decided on having 1 LED light up every time the friends patch is pressed, 2 for acquaintances, and 3 for strangers. We have set the color for a friend to be blue, acquaintance to be green, and stranger to be red. The brightness of the LEDs is maximum for a friend’s touch, intermediate for an acquaintance's touch and low for a stranger’s touch. The brightness and color of the LEDs at any given point of time is a fusion of all the interactions till that point. For instance, if the friend patch is pressed first, an LED brightly lights up in green. Then, if the stranger patch is pressed, three more LEDs are lit up. The overall brightness and color of the four LEDs is now the average of the above-mentioned interactions. The color of the 4 LEDS is a mix of 50 % red and 50% green due to the 2 interactions. Similarly, the brightness is a mix of 50% high intensity and 50% low intensity settings. At the end, when the user lights up the threshold number of 49 LEDS, the pattern revealed is personalized based on the color and brightness of all the interactions. The other 251 LEDs light up in contrast to this color, hence making the pattern stand out. The image below shows the LEDs lit up in different patterns and colors.



The Concept behind the Interaction

This LED settings were selected to model interactions on Facebook. We feel that generally people on Facebook, would interact more with their friends than with someone who is not on their friends list. We made the interaction such that a person would need to put in more effort to complete the pattern if they interacted dominantly with friends. A person interacting more with strangers would complete the pattern faster as we feel that interacting with unknown people is relatively tougher and rarer and hence the wearer is compensated for the same. Hence, we light only 1 LED for an interaction with a friend, 2 for an acquaintance. and 3 for a stranger. Also, a person’s feed on Facebook is usually dominated with updates from their friends and less so from people that they do not know. We model this using the brightness of the LEDs where the brightness achieved when the friends patch is pressed is higher than the stranger patch, giving more importance to friends. The pressing of the patches is inspired from Facebook’s ‘poke’ metaphor. When a friend touches the ‘friend’ patch, colored LEDs at the bottom of the T-shirt light up and reveal the exact number of interactions that have happened with friends, acquaintances, and strangers. These LEDs are lit only till the friend’s touch lasts. Through this interaction we mimic how Facebook allows more access of a user’s information to their friends as compared to strangers.



Evaluation

We created one prototype of the t-shirt. One of the team members will wore the t-shirt for a few hours and went about their daily lives. After that, another researcher interviewed the user based on their interactions, observations, and differences in social behavior if any. Based on this informal deployment, we noted the following observations.

Interacting with the vest: Many people interacting with the wearer were excited to touch the patches. One person touched every patch to decipher what was happening with the LEDs. Another person hit the stranger patch multiple times to reveal the pattern and see the color that it would reflect. People were intrigued by the novelty of the wearable and enjoyed interacting with the LED lights. They were especially engaged when the pattern would reveal itself. One of the interactors also took a video of the wearable.

Thoughts on wearing the vest: People were asked if they would wear this system and many showed enthusiasm to wear the vest but only if more people around them were wearing it as well. People wanted to wear it in parties and social gatherings but were reluctant to be the only one’s wearing it. One person mentioned that if in future wearing LEDs would become a fashion trend, he would like to wear this vest. Another person noted and liked the flexibility of wearing their own t-shirt on top of the vest. A few girls were hesitant about people touch them and they were keen on making the patches completely detachable. Currently, our design allows the wearer to place the patches on different areas of the back and arm, we will further explore how these patches can be made completely detachable. One person wanted the LEDs to be on an accessory such as a hat rather than a t-shirt. Another person commented that they were not comfortable in showing the interactions that they had to everyone. However, we feel that the ambiguity that has been introduced in the way the LEDs light up should solve this concern.

Applications: People were asked on how they would use this vest. One person mentioned that the vest could be used for encouraging middle school children to mingle as she felt that children in that age have a hard time approaching others and interacting. She also mentioned that it could be used for freshmen and graduate mixers, similar to what we propose. People also gave applications such as speed dating and concerts.

Deeper questions: A few questions that people asked the wearer were intriguing and worth mentioning. One person mentioned that she would be a stranger, the first time she met the wearer, but after that she would be an acquaintance and hence forth maybe a friend. Hence, if she met her multiple times on the first day of their interaction, what patches would she press and what sense would that interaction make? Another person wondered how is interactions and perception of a person would change if he saw their vest lit up with only red LEDs (stranger LEDs) and if that would make someone more approachable. Another suggestion given was to make the patterns customizable so that people could feed their own patterns into the system.

The wearer’s point of view: The researcher wearing the vest found the vest comfortable and nothing different from her usual wear. She mentioned that she did not feel any electronic component on her body. She liked what she was wearing as the LEDs caught the attention of many people. She was open to people interacting with her and touching the patches. However, she noticed that many people who looked at the t-shirt were hesitant to approach her, and she approached them instead. A few people did come to interact with her, and people also joined in further while she was explaining the system’s working.


Reflection

While brainstorming design ideas, we derived inspiration from the work on Patches [4] to explore the different interaction levels of a person with friends, acquaintances, and strangers through the added poke metaphor. We wished to build a wearable that could be used in a social gathering setting and could serve as a conduit to spark new and interesting conversations. We felt that in such a setting, the wearable would have to be attention grabbing to encourage people to come up and inquire about it and hence decided on using LEDs to achieve this. The question was then to decide how best to represent the different levels of interactions using the LEDs - do we have separate representations, or attempt to merge them? To bring in ambiguity in the representations, we decided to fuse the brightness and colors of the different interactions so that the proportion of each level of interaction would not be apparent. We also light up the LEDs in an incremental manner such that the absolute number of interactions is very hard to decipher. Furthermore, we felt that having the LEDs form a specific meaningful pattern would be an incentive for people to use the wearable. It would also spark interest in others to try and figure out what pattern the lights represent. The pattern reveals incrementally reveal a ‘poke’ symbol, with the index finger pointing towards the patches. This entire interaction has been designed to be reflective rather than performance based as the colors, brightness and LEDs do not represent absolute information but rather give a playful and meaningful interface [9]. The colors and brightness also give the option of some personalization as the users can opt to interact in a manner that lights up the LEDs in the color that they want. With our current design, people can wear the system under their own T-shirts. Moreover, the length of the cloth with the patches can be adjusted over a person’s arm and shoulder or can be attached to their back. We feel this can help people personalize the t-shirt by wearing whatever they want, and it can also help to overcome the uneasiness associated with being touched.

Future Work

We aim to make 8-10 more of these prototypes and deploy them in a party or large social gathering event. We will then investigate our initial research questions on how people interact with each other when they wear the vest, how they perceive and interact with people based on the LEDs lighting up, and how people feel about the social media metaphor being mapped into the physical world. We also aim to make the vest more robust by using multiple strands of conductive thread rather than alligator clips, use light lithium batteries as opposed to battery packs, and make the patches detachable for the vest. We will also investigate how people can make their own patterns and integrate it with the vest.