A tangible system for storing and interacting with memories
The aim of this project was to introduce a tangible user interface for interacting with memories in a fun way.
Journaling events, emotions and memories in a written narration is a common practice among the ‘personal diary’ keepers. It is also usual to record memorable moments on camera, capturing these memories in digital devices to be viewed later via a graphical user interface. We designed a system that lets the users record their memories as well as their narrations in an audio-visual form and associate these memories to tangible objects - introducing TUI in a GUI dominant activity. The aim was to let the users record and preserve these intricately woven webs of thoughts, feelings and memories and interact with it in a tangible way while introducing an element of added fun into the process.
This project was undertaken from January to March 2017. This was a class project under the course 'Advanced Ubiquitous Computing' at UCI.
We were a team of 2 people who worked on this project end to end.
Here is a link to the academic paper for this project.
Our objective was to design a system that could allow users to record memories in a more meaningful way. We aimed to minimize the dependence on mobile phones and laptops as the only tools to capture and revisit memorable incidents. We wanted users to manipulate the physical environment around them to make the interaction more intuitive and engaging. We also aimed to mitigate the dependence of screens (mobile phone or computers) from one experience (video recording / diary writing) of our lives. We have built a preliminary prototype that conveys our idea and we have also designed a diegetic prototype to convey the ideal solution for our task.
Our work draws its inspiration from the Disney movie ‘Inside Out’. We wanted to create a tangible system like the one portrayed inside Riley’s mind for making and storing memories. Each memory has an emotion attached to it and is stored in a ball where the color of the ball signifies the emotion. The balls can be picked up and the memory associated with it is displayed. Important memories are called core memories as they shape the personality of the person. Similarly, in our system, each memory would be associated with a colored ball. The memory can be played by picking up the ball and placing it on the system. The more times a user interacts with a certain memory, the more important it becomes and thus the system gives a visualization indicating the same.
The video below shows the working of the prototype. The user sits in front of the recording system and presses a button signifying the memory's emotion. The ball that matches the color associated to that emotion drops into the recording slot. Angry maps to a red colored ball, sad maps to blue and happy maps to green. The user then narrates his / her memory. Once the user is done recording, he / she can place the ball in the storage space. A light goes on at the storage space signifying the importance of that memory. The user can view the memory by placing it back on the recording slot. The more a memory is accessed, the brighter the light at it's storage spot gets, signifying it to be turning into an important (core) memory.
The prototype consists of a switch board where the user inputs his/her mood. A color sensor identifies a ball which has a color coherent to the mood. The slides controlled by motors, drop the ball to the record/view slot which has an RFID reader. The balls contain RFID tags that the RFID reader discovers. The user starts recording as soon as the ball drops and stops recording by lifting the ball.
Shown below is a box for storing memories. The user can view an old memory by picking up the ball and placing it in the record/view slot. The more a particular memory is viewed, the more it's storage space lights up, signifying it is an important (core) memory.
The image below shows the entire setup
The images below show a step by step working of the prototype. The user inputs his/her mood (angry). The system finds a red ball (red maps to angry). The color sensor first reads a blue ball and discards it. It then reads a red ball, the slide turns and drops the red ball into the record/view slot. The webcam turns on immediately and the user starts recording. To stop recording, the user simply lifts the ball. The user can place the memory in the storage system where a light turns on. The memory can be viewed again by placing the ball in the record/view slot
The system was tested out with 7 users where they were asked to record and access their memories. These users were graduate student at UCI and the test was performed inside a lab setting. The system was introduced as a tangible way of accessing memories inspired by the movie Inside Out. After giving a quick overview of the components, users were asked to record and play their memories. They were given additional time at the end to explore the system more. They were asked about the overall experience with the system, their suggestions and how they would use the system if it was kept at home. We received a positive response as the users enjoyed interacting with the system. A few users struggled to take out an old memory from the storage due to the design of the box. For the next prototype, the box will be lower in height for easier accessibility and the LED will be outside the box for better visibility. One of the users thought that a more robust system would enhance the experience. We also observed that a few people were hesitant to sit in front of a camera and record. We have proposed a solution to tackle this problem in our ideal version. Many users were intrigued by the motors and color sensing mechanism. One of the users tried to test out the color sensor as she said, “So, it can recognize the blue ball even if I mess up the sequence. What if I put two blue balls? Oh, this works.” Users also played around with the balls and gave us more ideas for the dynamic color detection. One of the users wanted to record a narrative on top of her memory. This too has been taken care of in the ideal system design. While a few users understood how to record and view the memory quite fast, some users were still trying to access the laptop to control the video. In hindsight, we should have made only the screen of the laptop visible to the users. In the end, users were asked if they would use the system in any other way. Most of the users found the idea of using it as a personal video diary interesting. One suggested, that it could be used in houses where the family members don’t see each other much due to their busy lifestyles. It could be used to leave messages for each other and the color coding would help identify which message is for whom. They also came up with ideas such as dream recording, child therapy and making to-do lists.
The video below describes the working of the ideal system that we designed for this prototype after receiving user input. This is a diegetic prototype based on technologies that may be available in the future.
The ideal system was designed after receiving user feedback. In the near future, we believe that intelligent glasses will have the functionality of recording what is being seen by the user and discarding the content intermittently. In the ideal version of the design, users can record what they see through their glasses and send it back to the system at home. Users can also record while they sit in front of the system, using it as a personal diary. Given below is an image of the system. It has a storage, a dais for recording and viewing memories, buttons that signify emotions and another space for storing old memories.
Once the user commands the intelligent glasses to send back a recording, a new ball in the system comes out. It rolls on to the dais where the memory is stored in the ball. The same can be done by the user if he / she sits in front of the system and presses a button signifying an emotion. The ball changed into the color of the emotion once it is lifted on the dais. The color / emotion of the ball can be changed anytime, even of an old memory, by pressing the corresponding button. Given below is an image explaining the same.
A place to store all old memories has also been created. By rotating the knobs attached to the storage, the balls rotate synchronously and thumbnails of the memories associated with the balls are displayed on the screen. The user can view any old memory by placing the ball on the dais. The memory associated with it is played on the screen.
The user can also add a narrative to a memory by placing a smaller audio ball in the audio slot as shown in the image below. The user then continues to narrate as the memory plays. Once done, the user simply has to place the audio ball inside the bigger memory ball. Thus, whenever the bigger memory ball is accessed, the narrative on top of it is played along as well. The image below explains the same.