Shiva’s Rangoli: Tangible and Interactive Storytelling in Ambient Environments

A storytelling experience rooted in Indian mythology and Culture


Overview

Problem

To create an interactive storytelling experience that can engage readers without giving them control over the story's plot

Solution

Created and Evaluated a tangible interactive storytelling experience that gives readers control over the emotional and aesthetic context of the story, without changing the story's plot. Readers can change the ambience of their space (lights, sound, and music) to set an emotional tone. The installation has been inspired from Indian mythology and culture

Project Duration

This was my year long Master's thesis at UCI. The project started in April 2017 and ended on June 2018.

My Role

This is an individual project that I am worked on under the guidance of my advisor Dr. Josh Tanenbaum at UCI

Academic paper

Here is a link to my thesis.

One of our publications has been accepted for TEI (March, 2019)

Design Process

Theory and Inspiration

How is Interactive Storytelling generally achieved?
Many systems term ‘interactivity’ or ‘agency’ of the reader as freedom of choice over the narrative, giving readers the ability to change the story’s plot by taking different choices and branching out in different pathways. Although such experiences can be pleasurable, they favor the reader’s interaction over the author’s core story. Freedom over the story plot can also force the author to create branching pathways of the story, leading to combinatory explosions, that can weaken the narrative even further. These claims have been grounded by literature - Tanenbaum and Tanenbaum (2009), Tanenbaum (2015), Nay and Zagal (2017). These works advocate for giving readers the ability to interact meaningfully with the narrative rather than giving freedom of choice over the plot.

Interactivity in Shiva's Rangoli
I aim to explore ways that encourage readers to interact meaningfully without giving them control over the story’s plot. I explore how readers interact with a narrative where they have agency over the emotional and aesthetic context of the story that gives them the ability to change the mood and tone of the narrative only.

Why Tangible Storytelling?
Tangible interfaces are interfaces that use physical objects to interact with technology. I believe that such interfaces when used in interactive narratives can break the boundary between the reader and the fictional world. They open a realm of possibilities for connecting readers with the story such as diegetic objects. Diegetic in narratology means something that is inside the story. ‘Diegetic’ objects in tangible narratives act as objects that exist in the reader’s world alongside existing in the story world. In my thesis, I have used a tangible diegetic interface . Furthermore, I transform the physical space of the reader to mimic the environment of the fictional world.

Goals
Through Shiva’s Rangoli, I attempt to find balance between the author’s story plot and the reader’s interactions by giving readers agency over the mood and tone of the story, but not the plot. I also aim to connect readers with the story world through Diegetic objects and Ambiences. I explore a solution that aims to connect readers with the characters and makes them feel a part of the story by using diegetic interfaces and by giving readers narrative responsibility, without compromising the story content.

Installation Experience

Shiva’s Rangoli enables readers to shape the ambience (lights, sound, music, and video) of their environment, through a tangible interface. By doing so, they not only set the mood and tone of their physical space but, also set an emotional context for the story by impacting the mood of the characters.

Readers hear two interleaved audio narratives within the context of this configurable ambience. The TUI represents a traditional Indian artform called Rangoli Making and the narratives have been inspired by Indian Mythology. Rangolis are elaborate colorful patterns that are made during festivals in India. The TUI also acts as a diegetic interface as it is a part of the physical world of the reader as well as the fictional world of the stories. The Rangolis created by the reader represent the rangolis created by the protagonist (Shiva) in the narratives. The environment of the fictional world transforms to match the ambience of the physical space created by the reader. The image below shows the interaction of the reader


The Tangible User Interface (TUI)
When readers enter the installation space, they are first introduced to the concept of Rangolis. Rangolis are colorful patterns that people make on the floors of their homes for celebrating various festivities. Traditionally, they are created with repetitive patterns that represent geometric shapes, deity impressions, and nature representations.


The installation is a space enclosed by a tent as shown below. Inside the tent lies a table, twelve artistically patterned Rangoli rings and two center pieces. 3 of these rings and a center piece are placed on the table top in a concentric manner to create a Rangoli.


The Rangoli helps to set the ambience of the space. The inner rings set the music for the space, the middle rings set sound effects, and the outer rings set the lighting inside the tent. The two center pieces represent two different yet inter-related narratives. There are four sets of three rings each. Each set of rings represents a particular mood .The yellow ring represents happy, green represents calm, blue represents sad, and red represents angry. The colors and emotion mappings have been grounded in work by Russel’s Circumplex model of affect and by work in eMoto.

Once the reader assembles the entire Rangoli, a chapter from one of the stories, is narrated through audio. The chapter adapts to the emotional tone set by the reader as mood based textual elements in the narrative change accordingly. The yellow – happy rings create a warm, upbeat, and chirpy ambience. The green - calm rings create a soothing and soft one. The blue – sad rings create a low and dull environment. The red - angry rings create a stormy, thundering ambience. These effects were decided after iterative user tests.

Below are images of the rings and the ambiences.




The Narratives
I have authored the stories. In the stories, Shiva is a powerful God who creates Rangolis to express his emotions. His emotions trickle down into the human world as well. For example, when Shiva is angry, the world’s skies become bloody with dark with heavy rains and raging thunderstorms. The Rangolis that the readers create represent the Rangolis made by Shiva. They shape the ambience of the fictional world as well as the space in the tent, Hence both, the Rangoli and the dynamic ambiences in the tent, are designed to serve as diegetic. Each chapter has two versions: a positive version and a negative version. For simplicity, I have grouped happy and calm as positive and sad and angry as negative. The system reads the overall mood value of the Rangoli assembled by the reader and triggers one of the two versions. If you use different colored rings in the Rangolis, the system picks up the dominant mood. In both versions, the story plot remains the same, but the tone and emotional context of the story changes. Alongside the text, the narrator’s tone, volume and pitch are different between both versions as well. In both stories, Shiva’s Rangolis are acknowledged frequently to help readers confirm their understanding of how the colors map to the moods and how the interface connects with the story

Create Story Endings
After readers were done with their experience, they were asked if they were willing to contribute by recording their version of an ending of a story. Readers recorded their narratives on center pieces, set different ambiences, and decorated their pieces. An image of these center pieces is shown below. People could also hear what others had recorded by placing the center piece on the table.


Design Iterations

Here are some earlier solutions that I had proposed. A user would interact with certain objects to control the lights, sound and music of the room, thus shaping an ambience. This ambience would be connected to a genre. The story would then take the shape of the particular genre. In the end the user would be asked to shape an ambience and create an alternate end to the story, in congruence with the genre that their ambience portrays.


However, this solution did not fully engage the readers and neither did it capture the essence of Indian culture.

In the second storyboard, I decided to proceed with the designed TUI. This was close to the final design, yet the Rangoli and ambiences were not diegetic in nature. I created paper prototypes to test this solution with 4 acquaintances in an informal setting. Users responded well to the TUI. However they were not engaged enough in the process. Thus, I moved onto my final solution by making the Rangoli impact the character's mood. The final solution was tested with 4 people through cardboard prototypes

Hardware Iterations

The images on the top shows the cardboard prototype that I created to test out the system. I then moved on to build the actual hardware. Pieces were made out of wood, cut with the help of the laser cutter. The electronic hardware consisted of a Makey-Makey and an RFID reader and tags. The circuits was completed through snap switches shown in the image below. Each rangoli piece locks into a certain place with the help of magnets, thus snapping down a certain switch. The rings also differ in heights so that they are easy to pull out and replace without disturbing the other rings.


I then scaled the solution up. The image on the left represents the base of the structure on top of which the rangolis are placed. The image in the middle shows a close up where the switches and magnets have been attached. The right picture shows the back of a ring.


The rings were laser cut and hand painted. Below are images of the making and a few assembled pieces


The images below show me making a rangoli by locking in the rings to the table top.


These are images of the hardware - the Makey Makey, RFID Reader, and an RGB individually addressable light. The tent was created by suspending a metal frame from the ceiling of our lab and draping it with fabric and curtains.


Research Questions

RQ1: How do people interpret and interact with a narrative where they can craft the ambience of their space to impact the emotional and aesthetic context of the story, without changing the plot?
RQ2: How do people engage with diegetic interfaces in an interactive narrative?

Methods

I conducted a study and participants were recruited through postings on Facebook, flyers in UCI campus, and through word of mouth. 25 study sessions were conducted in which 3 were pilot tests. The majority of participants were graduate and undergraduate students at UC Irvine. Participants were first introduced to the concept of Rangolis. They were asked to explore the demo version of the system where they did not hear the narratives but only explored the ambient effects. Once the readers were done, the narratives would begin to play. This experience lasted for 30-40 minutes. Participants were then interviewed based on their experience. I conducted an open qualitative analysis and coded the interview transcripts to analyze the data. I then categorized the codes into various themes. Below are images of a participant assembling Rangolis during the study.



Findings

These are a few findings from the thesis. I observed readers interact with different styles of agency. I observed the role of ambiences, roles people took in the experience, and the mismatch between the ambience and plot that people observed. Given below is an image of the coding diagram I made to visualize my findings.


Different styles of Agency
The participants interacted with the piece dominantly through 3 interaction styles based on their perception of the agency they had over the story. These categories were not mutually exclusive as people took on different interaction styles as the experience progressed Story supporters felt they had no agency over the story, and acted in ways to support the story ; story controllers felt that they asserted some amount of control over the plot of the story at key points; and meaning makers – readers who felt agency over the mood and tone but not the plot. Given below are quotes by readers describing the same.

P02: “It’s almost like a play where I am the setting or the backdrops or stage hand. You just foretold that there is going to be an argument. Now I need to change the mood because I need to support the story since that is where the story is going.”


P16: “Every time it (story) acknowledged me, I felt like I had accomplished something or changed the narrative.”


P13: “I felt because I had influenced the mood of the character, even though it didn’t influence the progression, I felt like I was able to tell my own story of his emotional ups and downs and that felt powerful to me, I very much enjoyed that specific section”


Ambience
Ambience as Diegetic: Readers felt as if they were positioned inside the story world because of the environment created inside the tent. Ambiences helped readers to connect with how the characters were feeling.

P06: “I was able to picture everything and having those things in the background, almost it was like I was there also, like I was in the background just watching it all play out. Watching the stories.”


Ambiences help set focused space : Readers felt that ambiences helped them focus more on the stories. The environment acted as an aid to the auditory narrative and improved story coherence.

Derive different meanings: The ambiences helped readers see different perspectives of the story, perspectives that they would have not seen had they heard the stories outside this context.

P17: “The music and visuals were very nice, but the ending was pretty tragic (for ‘the river’). Hopeful for a good beginning.”


Perceived as continuous entities: Readers perceived the ambience to give incremental and gradual changes in the mood and tone setting. They tried to represent a mix of emotions and express varying intensities of emotions.

P02: “I could influence how angry someone was, I didn’t want to put all red up there, I did not want to see what would happen. I thought the system might show me if I did.”


Roles
Many readers connected to Shiva in different ways. A few readers transformed into Shiva himself. Some felt as if they were a part of Shiva, like a part of his personality or a person in his mind. Some felt that they were a part of the story but disassociated form Shiva, like a God controlling the other Gods, or like an observer in that world A few felt as passive listeners. My goal was to make readers feel as if they were a part of the narrative in some way.

P05: “I felt like I was the almighty (Shiva), I was determining life and death. I had the responsibility not only to destroy but also overlook Vishnu and Brahma’s jobs. I felt very powerful, I had this huge responsibility and I had to act more mature and more calm in order to live up to that role.”


P01: “(I felt) Like a person in that era or a person in that story. Not like I am Shiva. I felt like I was there closely watching what was unfolding in front of me. Like someone in that world.”


Mismatches
At times there would be a mismatch between the plot and the ambience created, especially when a happy ambience was created for a sad event. Even though a few textual elements changed to accommodate this, at times it resulted in a jarring experience. However these mismatches also had positive impacts on the reader’s experience. It helped them understand the agency they had over the narratives. It brought more attention and focus. It helped them learn how the system worked and get rid of false assumptions, such as the experience may have branching narratives or they may have control over the plot. The mismatch in ambience and story plot broke immersion, which encouraged people to change the Rangoli rings after every chapter, as they wished to create a meaningful experience.

Design Implications

Here are a few design implications from the thesis

Styles of Interaction
Each participant had a unique experience and a different style of interaction. It is important to note that 20 out of the 22 participants found a meaningful way to interact with the system. This was possible only because the system gave them enough room to interact as they desired without forcing them to act a particular way, Hence, such systems should not emphasize a right or wrong way of interaction. People should not be penalized by breaking story coherence, or halting and disrupting the experience. Essentially, the show must go on. In the study I observed 3 interaction styles. I feel researchers can use these levels as a framework to guide the type of agency they wish to give to readers. In my thesis I have also proposed implications for designing to support these different levels of agency

Diegetic Interfaces
I observed that readers enjoyed assembling pieces of their own Rangolis. Hence, I believe that readers should have control over the way they craft the interface. These interactions with the diegetic interface can then impact the narrative in different ways based on how the interface is crafted, providing readers some form of agency over the story. In the study, the acknowledgment of the Rangoli helped readers connect with the narrative. I propose that it is important to acknowledge the reader’s interactions with the diegetic interface to help them understand how the interface connects with the story. Through my findings I feel it is worth connecting the diegetic interface to an element that can gives the reader some responsibility in the narrative, as it encourages them to act meaningfully. In the study the Rangoli was in the foreground and the ambience in the background. It is useful if the diegetic object in the background influences the story in a way that supports the impact of the foreground interface.

Future Work

For the scope of my master’s thesis, I concentrated my study more on ambiences, interaction styles, and diegetic interfaces. In future studies I will elaborate my findings on the cultural aspect of the experience. I would also like to conduct a quantitative analysis of the data to further analyze the cause and effects of the interaction styles, to what extent the demo helped in building literacy, and whether the cultural background of the person impacted their experience. I would also like to extend this design to accommodate multiple readers. I hope to take this piece into performative spaces such as classical music, dance, and theater. I am also interested in studying how interactive ambient settings can be used in broader contexts such as public spaces, homes, and educational settings.


Back to Work