Final Project Documentation | Reflections

Description

The user finds themselves inside of a small, rectangular town. Pink buildings encircle a clear blue lake, with no visible exit from the road which follows its edge. In the lake, the reflection of the town can be seen. Nearer to the player, a bridge stretches across the lake, and on the opposite end of the lake, a pier stretches into it. People are scattered about the town, walking around, sitting, or standing, going about their lives. Approaching and speaking with the residents reveals the simple society and economy of the town. Its everyday people are generally distrustful of difference and find their purpose in diving and digging for stones to give to the rulers in exchange for property rights. After talking with the king about stones, a mysterious light appears at the end of the pier.

Upon interacting with the light, most light disappears. A subtle mirroring of building positions suggest this is not the same place, though the same people are scattered about. Faint lights illuminate the road, and a bright spotlight falls on the most cheerful peasant from the first town. Approaching and interacting with the residents, the user finds that they residents no longer recognize the player. Instead, as the user moves from scene to scene, the underlying emotional and political complexities of the town unveil themselves. An overly cheerful peasant lives with depression. A busy wife cheats on her yearning husband. Virtuous rulers exploit the labor of the people for valuables without proper compensation or explanation of their worth. An angry witch sees all but does nothing, for the wizard keeps everyone under illusion. A charismatic and responsible king submits to the dutiful queen’s insistence on greed despite his guilty conscience, as well as the waning powers and reasoned council of his trusted wizard A loyal servant to the queen kills a merchant who refuses to explain how they arrived in the town, driven mad by her suspicions of the rulers.

Process

This project was created in collaboration with Yeji and Ganjina.

At the beginning of the process, we had each voted for different topics, so we brainstormed different compelling narratives and environments for each topic. We narrowed down that list to two. The first fell into the apocalypse theme. The user would find themselves in an apocalyptic environment. When they interacted with the environment, they would be sent into a memory, during which they could choose to alter the timeline and repair the piece of the environment with which they had interacted. The second idea involved expanding upon the city of Valdrada from Invisible Cities. According to Calvino, this city was “built. . . on the shores of a lake, with houses all verandas one above the other,” such that “the traveler . . . sees two cities: one erect above the lake, and the other reflected, upside down” (Calvino 45). Calvino goes on to describe it as “twin cities” that “are not equal,” for “every face and gesture is answered, from the mirror, by a face and gesture inverted, point by point” (45-46). In this way, they “live for each other . . . but there is no love between them” (46). Feedback from the class was inconclusive, so we expanded both ideas a bit more, searching for compelling stories, interactions, and environments which might take place.

• 

• 

• 

At our next meeting, we felt more compelled by the ideas we had created from Valdrada and from reflection as a guiding concept. Two ideas prevailed, one out of my creative process and one out of Yeji’s.

My idea put the user on a tropical-island environment. Guided by the question, “What would it be like to be a child in a city where everyone knows their reflections are a different, mirror world?” the narrative followed the relationship between two people throughout their lives. In one world, they would become best friends. In the reflection, they would become mortal enemies. Moreover, in each scene, the two worlds would interact through the reflections, and the adults would anticipate these interactions and respond to them nonchalantly. For instance, if one of the main characters tackled the other through wall in the enemy world, then that hole in the wall would exist in both worlds. In both worlds, the parents would fix the wall without much comment.

• 

• 

• 

• 

Yeji’s idea emerged from the environment of Amsterdam and revolved around the question, “What would be reflected?” This idea focused more on a commentary on how people tend to hide their true thoughts and feelings in order to get along, yet suppressing these feelings can lead to outbursts when one feels safe or reckless enough to let them out. In this narrative, the story would emerge from the user interacting with the characters to understand some big event that connected everyone in the city. The user would start in a bright city where people glossed over this big event to go about their daily lives and would transition to the second city through an environmental interaction. In the second city, the environment would be darker to match the darker responses to the big event, responses which disrupted the functioning of the city. The user would also encounter these responses as a voyeur rather than a participant, indicating the private, and perhaps truer nature of them.

The clarity of the interaction and story, as well as the feasibility given the remote working environment ultimately pushed us to develop upon Yeji’s idea. In my idea, the primary interaction would be the user’s choice of how to experience the story, and the story would be delivered via scenes which occurred around the user as a ghost. Given the need to deliver two world narratives in parallel, our best solution was an environmentally embedded panel present at the start of each scene, where the user would choose good or evil. They would then be brought into the scene to watch it. In Yeji’a idea, the primary interactions would be with the characters and with a single transition into the reflection. The user’s agency and participation would also be manipulated to convey story rather than held constant to make room for it. Furthermore, my idea would rely upon complex and compelling animations across many different scenes while Yeji’s would still be compelling with simpler animations and fewer scenes. Hence, across the board, Yeji’s idea seemed more conceptually appealing, explorative of the capabilities of VR, and feasible to execute.

One problem still remained: how would we voice the characters? Voiceovers did not make sense without proper voice actors because of the number of characters we were using. After a bit of deliberation, we decided to go with a text-based dialogue system, similar to those of 2D text-based adventure games, like the early Pokémon games.

With this general outline for the experience, we sketched out the environment as the user would experience it for the paper prototyping session

• 

• 

• 

• 

• 

Feedback from this session solidified an already-developing idea to use the dialogue and interactions with characters to guide the user to the scene-transition interaction. Another option was to start the user right by the pier, but our desire to have the story emerge from the characters suggested a focus on encouraging exploration rather than completion in the first scene. This session also pushed us to move away form having the private scenes occur inside of the houses. We realized that the narrative would still be clear, and we could instead constrain the user’s agency in order to achieve the feeling of privacy. With a relatively clear idea of the world, an outline for a story, we divided up the creative work. Yeji would focus on the story, figuring out the overarching big event and scripting dialogue. Ganjina would start to build the environment based upon our sketches. I would start figuring out how to achieve a functioning dialogue system in virtual reality.

An extensive outline of the design process for the dialogue system is available in my development journal, but I will summarize and expand upon some points here.

To start off, we wanted the system to feel comfortable in VR. The system would also be the object which would carry our story to the user, so it had exist at the intersection of user and environment. As a result, we decided to deliver the system via a HUD panel rather than multiple panels attached to each NPC. Mike Alger’s recommendations on comfortable relative locations for workspace UI were used to determine the dimensions of the panel, its distances from the main camera, its relative angle, and its curvature. Limitations of the assets we were working with meant it could not curve around the optimal spherical shape, but only around a cylindrical shape.

• 

• 

The remainder of the visual design choices occurred on an as-needed basis as more parts of the project were integrated and tested with play testers in a series of demoes that can be found in my development journal.

Visual design choices:

• The dialogue itself is curved along the same curve as the panel’s edges.
• A next button and stop button indicate when there is more text to scroll through and when there is less text to scroll through.
• The text is black when it is bright and white when it is dark to optimize viewing.
• The user’s reticle is a bright blue that does not blend in with the colors in any of the scenes. It expands into a polygon instead of a circle to match the low-poly aesthetic.
• The reticle expands when an interaction is possible because this was clearer than a simple color change that might make the reticle disappear into the background.
• The translucent blue of the panel provides enough contrast without disrupting the visual environment.
• The panel disappears when deactivated. Initially, it changed form translucent to opaque, but feedback from play testers suggested this was too disruptive the visual experience.

In addition to the visual design of the panel, we also had to develop functions which did not detract from the experience and were doable with the restrictions of the Google Cardboard. Hence, everything requires one-click or click-and-hold. Clicks under specific conditions create specific responses from the environment. These conditions were set to be as intuitive as possible. Finally, these conditions were determined and coded in a master script called dialogueManager. The script is around 500 to 600 lines, so posting it here would not be conducive to understanding it. Many of the functional design choices emerged from a gradual process of integrating the dialogueManger script with the pacing of the dialogue and the animations of each scene.

Functional design choices:

• When the dialogue panel is active, it must be deactivated before another dialogue can be opened. This is reinforced by changing nothing if the user clicks on another NPC when the dialogue for one NPC is active.
• The dialogue panel can be reset and deactivated by clicking on anything which is not the panel or another NPC. Hence, the user can close out any dialogue at any point in the dialogue. This adds to their agency as an explorer of the narrative.
• Clicking on the panel scrolls through the dialogue and ends the dialogue when there is no more of it. The brevity of our dialogue meant that scrolling backwards through the dialogue was unnecessary. Scrolling backwards would not have made sense in the night scene either, when the user is witnessing rather than conversing.
• In the second scene, the user cannot click through the dialogue until certain animations have played. This occurs during the cheating scene, with its delayed arrival of the peasant husband. The reticle still indicates a potential interaction, however, to indicate that the user will be able to move forward in the dialogue.

Before the environment and dialogue were completely flushed out, I also worked on the mechanics of the interaction between the user and the NPCs, since these determined much of the dialogue system’s functionality. We did not want to have any interactions triggered from an unintuitive distance, so I had to develop in-range detection systems for the NPCs. For reasons described in my development journal, I used a pseudo-cyclindrical, box collider, trigger method to detect when the player was close enough to interact and set the GoogleVR reticle to detect objects from a similar distance.

During this time, I also figured out a simple script to have the NPC look at the player upon interaction. This script became more complicated later, when we had to have two NPCs stop their pacing, turn toward the player, and then return to their pacing routine when the dialogue interaction was completed.

Finally, I also figured out the auto-walking mechanics so that the user could move by clicking and holding.

As the dialogue and then the environment were completed, our focus shifted to animating the characters in the space. We used a back-and-forth workflow, and during my part I integrated the dialogue system into the animations.

Dialogue System & Animation Design:

• Day Scene:
  • NPCs will turn toward the user when in the user interacts with them. This acknowledges the user as a participant and gives them the agency of control over the start of the interaction, a feature of public encounters between people.
  • NPCs return to their original positions or routines when the interaction ends. This maintains a sense of immersion, or that the user is in a place which exists independently of them.
  • The user can continue to interact with NPCs until they desire to move on to the next scene. This encourages exploration and understanding of the different characters and reinforces the user’s role.
• Night Scene:
  • NPCs do not turn toward the player when the user interacts. This implies that the user is a voyeur, and that the NPCs are not necessarily aware of their presence.
  • Each dialogue-animation set is relative to the scene playing out rather than the NPC activated. This further reinforces the idea that the user is a witness of an otherwise private interaction.
  • The user cannot interact with NPCs whenever they want, but must do so in the prescribed, indicated order. This ensure that the story unfolds progressively, such that the pieces build on each other and organize the bits provided in the day scene.
  • With the exception of the arrival of the peasant husband allowing progression in the dialogue, the dialogue activates the animations of the characters. This was accomplished through edits to dialogueManager.
  • In the cases of the death animations, animation is both a condition and an effect. The deathTrigger script checks if the attacker of the NPC to which it is attached is attacking and then plays the death animation of the attached NPC.

I also figured out how to do a root motion pacing animation cycle that could be interrupted. This allowed us to add some motion into the otherwise static environment of the day scene, giving more of a feeling of lively town and further distributing the interactions around the space. The pacing script attaches to an object with two box collider triggers. These are placed at the limits of the NPCs pacing area. It detects them and changes their animations on trigger.

With the animations worked out, the last components were the sound design, the scene change interaction, and the lighting for the second scene.

Since the order of scene encounter matters in the night scene, there needed to be an indication of which scene could be interacted with beyond the reticle changing. Due to the dark environment of the night scene that helped establish the mood of a darker narrative, we decided to use theater-esque lighting to indicate which characters the user should go and interact with. The sceneLightsManager script was developed, and it either checks the animation or the state of the dialogue for a given scene, or both, on an as-needed basis. Hence, as the user completes each scene, they are visually guided to the next one they can interact with.

The visual cue for the scene change had to be coherent with the visual cues we used elsewhere. Hence, we used an orb of light as a button. This was visually coherent with our use of light in the second scene and our use of single-click interactions. A similar distance-sensing method to the NPCs was used to ensure the user could only interact once they were within a relatively intuitive distance. Finally, the scene change only appears in certain conditions.

In the day scene, it appears once the user has spoken with the king, and the king has encouraged the user to go acquire stones. When combined with the dialogue of the rogue and the swimming man in the water, it seemed coherent with the story that the king’s dialogue would push the user to try and find stones, and that going to the edge of the pier might allow them to do so.

In the night scene, the orb only appears once the user finishes the last scene. This fits with the intention to require the user to witness all of the truth of the town before being able to return to its daytime. Using the same orb in the same position suggests that it will do the same thing it did before, teleport the user to a new scene.

As for the sound design, these were done mostly by Ganjina and Yeji. I helped integrate them into the animations of the night scene using similar methods from when I ordered the specific pieces of dialogue and animations.

Finally, we had to think about the ending. We were choosing between a fade-to-black or a return to the day scene. In the case of the return to day scene, we would remove the murdered. We also were choosing between maintaining the interactions with the characters or eliminating them, disabling the dialogue system completely. Returning the day scene without allowing interaction seemed to be the strongest indicator that the user’s role had fundamentally changed as a result of what they had witnessed. They now occupied the voice in all of the characters heads which caused the suspicion and conflict, so it made more sense that the town would no longer want to speak to the user.

Reflection

Overall, this piece was successful in delivering the story we wanted. It conveys the commentary on the political and emotional undercurrents of our public interactions. It embeds the story within the characters distributed throughout the environment. The affordances of the interactions with the dialogue system and the scene-change are clear, and their conditions support the delivery and pacing of the story. Furthermore, the experience takes advantage of the unique ability of VR to manipulate the agency and participation of the user, and this manipulation helps convey the meaning of the story.

However, the experience is not perfect. Sometimes the NPCs do not turn toward the user. Sometimes the dialogue panel is triggered by pointing at the pseudo-cylindrical colliders on the ground. There is some asymmetrical pacing in the delivery of the cheating scene due to the dependencies between the animations and dialogue pieces. However, these bugs have been reduced as much as possible and rebutted against by the logic of the system and the story. As a result, they do not detract significantly from the overall delivery of the piece.