Neyva Hernandez – Alternate Realities

Paths of Perception: AR Final Project

Project Description

Building off of Japanese folklore, Paths of Perception is an interactive story-telling experience in which the user is both thrust into a wonderland, but challenged to escape, lest they be stuck there forever. Originally intended for Google Cardboard, the experience was designed for Mac and PC

Drawing inspiration from Japanese Kitsune, which are fox spirits that can be both malevolent or benevolent, the user finds themselves in a Shinto shrine/graveyard. The user can see in 360° around where they are standing, but they are not meant to move and explore past their visible boundaries. After observing the peaceful scene— cicadas chirping in the summer, the sun high in the sky— the user is suddenly transported to a more eerie, and otherworldly place. It is still the the shrine, but it now seems ominous and something seems wrong; the sky turns red and the music becomes chilling. This is when 2 Kitsune appear, blocking the only path of the cemetary.

The user is then met with the challenge, find the 3 objects that changed in order to break the evil Kitsune’s — known as nogitsune— curse that he placed on you. The 3 small details in the environment changed from the transition between the day, peaceful scene, to the ominous surreal location, and require a keen eye to identify.

If the user is stuck, the 2 Kitsune offer riddles in order to help identify the objects. But again, one of the Kitsune is trying to trick the user and spouts lies, while the other one is trying to help you escape. It is up to the user to infer and identify which Kitsune is the one trying to help, else, they are stuck in this other world, possessed by the evil Kitsune with no chance to escape.

Process and Implementation

Brainstorming & Implementation

When Mari and I began brainstorming, we were working off of the words “Escape room” and “Wonderland.” We wanted to create the wonder and curiosity that comes within a mystical wonderland, but we knew from the beginning that we wanted to user to only have a limited scope of exploration within the wonderland, lest they become too distracted in the open world and stray from the story we wanted to tell. Within the idea of the escape room, we wanted the user to have some kind of puzzle, or challenge. The wonderland is not merely meant to be gawked at, but to pay close attention to in order to solve a puzzle. We were inspired in part by Alice in Wonderland, specifically when she must drink a potion to fit through a door and search for a key. We thought that was a good execution of Escape Room + Wonderland, as in that scene, Alice has to look for details in her environment to help her solve a problem and escape.

As we were originally building for the Google Cardboard, we designed in mind with the “point and click” interface. This would be the main interaction in furthering the narrative, and so we settled on clicking and selecting objects within the environment we created.

Mari and I are both interested in Japanese folklore, so we began by looking at various Yokai, or Japanese spirits. From there, we found the idea of the Kitsune, and knew that we wanted to incorporate them.

Implementation

Environment

The environment’s design is one of the key components of our project, as the challenge lies within the details of the environment. Originally, we wanted our user to stand in the middle of a path going in 2 directions, with a Kitsune fox on either side. (Shown below)

Our initial storyboard, showing how key this road was in the design originally

Kyoto, Japan. A Shinto graveyard on the mountainside at the shrine ... — We wanted the area to have lots of details and clutter, such as in this example image.

After user testing, we found that this was not the most effective way to lay out our environment. Users were simply disappointed that they could not further explore down the paths, and felt very limited by not being able to walk around and explore. The layout also had a plethora of objects in it to the point that it was a bit too difficult to see what had changed or not. There were too many gravestones, too many Tori gates (the red gates), too many statues, candles, lanterns, plants, fences etc. It was just too overwhelming in this design.

A lot of the items felt/seemed very far away

Too much clutter also made it hard to pay attention to the different details in the environment

At first users just arbitrarily started looking at this side of the road

After the user testing, we found that a layout that was more conductive to our idea was to create a closed-off circular layout. In this, there is simply one path for the user to look to, as they are standing in a dead-end. This small area had a limit on how many items we could place around and change, thus accomplishing our goal of not overwhelming the user with the details.

Top view of our new scene design: all the objects are placed in relation to the user’s location in the center

The foreground and background items are now easily distinguishable

Interaction

Finally finished with our new environment design, we began to focus on the interaction of the project. The original plan was for the user to simply look around and observe their environment before anything changed, and then point and click to select the objects that changed. In order to facilitate animations, we made the user light a candle once they were done selecting their 3 objects that changed. It also allowed for the user to look directly at the foxes after their selection, as the foxes next actions were indicative of whether or not the correct selections had been made.

When they need to select the 3 objects that changed within the environment, once the pointer on the screen lands on it, the object will change color. This was a way to make object selection intuitive for the user, as there were many objects. We also saw it as a way to facilitate the actual selection of the objects, as clicking between 5 objects made it easier than choosing between all of the objects within the environment. We also decided to make one of the objects very obvious (a gray stone fence becoming a bright red) in order to help the user decipher which fox was helping them. Both foxes had riddles appear above them, which hint at which objects changed. But only one of the riddles is completely correct, as one of the foxes is trying to trick the user to select the wrong objects and stay in this alternate world.

If the user selects the apporporiate 3 items, they are then transported back to the normal human world, with the good fox still standing in the path. The words “Welcome Back” great the user.

If the user selects the wrong items, then the evil fox lunges at the user and the screen goes black. This is meant to represent the evil fox finally possessing the user, and the fact that at this point, there is no longer an escape from the surreal reality.

Hovering over an object changed its color, showing it’s selectable

Clicking over it changed its hue to blue, indicating it’s selected.

Reflection

Overall, I am very happy with how the project turned out. With some minor tweaking, we should be able to still make it for Google Cardboard. (We really want to show our friends!) While we had some minor issues with the selection color and text not displaying properly/at the right times, we were able to create the detailed, and beautiful environment and interaction that we wanted. Mari and I felt from the very beginning that the scope of our project was realistic, and we accomplished our beginning goals within the project.

Additional photos:

VIDEOS

Good Ending

Bad Ending

Project 3 Development Journal

Brainstorming

In this final project, Mari and I wanted to work with the themes of wonderland and escape room. Wonderland is defined as:

but we only wanted to work loosely off of this definition, as we knew we wanted some kind of “darker” side to the wonderland. Within our brainstorming for the wonderland setting, we looked at Alice in wonderland’s infamous door know scene found here:

We looked at this scene in particular due to the fact that Alice is indeed in a wonderland, but faces troubles and fears within it due to a fear of the unknown. In this example of a wonderland, there isn’t something inherently bad, but the main character is frightened in the uncertainty that shrouds the strange place. We knew we really wanted to try our best to incorporate this into our assignment; the idea that a wonderland inherently carries a sense of the unknown which can make it frightening, although the wonderland may not have any conventionally frightening things.

Within the idea of causing anxiety to our user within the environment, we briefly looked into the SAW horror films as an example of a (literal) escape room type situation.

An image from the Official Saw Escape room found in Las Vegas (site found here: https://sawescaperoom.com/)

This was quickly dropped, as we decided we wanted our wonderland to follow more of a “fear of the unknown” rather than a blatant horror style.

An escape room, according to wikipedia, is defined as the following:

Again we wanted to loosely interpret this, and so our idea of an escape room is meant to represent the goal of the user to look for a kind of relief within the wonderland. This fit in nicely with our idea of the fear of the unknown, as so we had a general basis to work with from there.

Concretization of the Idea

At this point we knew what we generally wanted, and so we had to pick a concrete idea to work with within our wonderland environment. We liked the idea of creating something from folklore, and after some discussion settled on Shintō Folklore from Japan. Specifically within Shintō folklore, we looked at the Yōkai. We liked the idea of focusing on Yōkai, due to the fact that they are not necessarily associated with “good or bad” and playing with the gray area they fall into.

A brief definition of Yōkai from Wikipedia

Narrowing it down further, we really liked the myths of the Kitsune. This is a type of fox yōkai, or spirit, that has various abilities such as being able to can shape-shift into a human. According to Shintō folklore, the Kitsune have multiple tails and can reach up to 9 tails. The tails represent their age, power, and intelligence, so a Kitsune with 9 tails carries the implication of being of the highest caliber in these regards, and is a very rare sight.

There are also various types of Kitsune. The two that are key figures in our game are the following:

Nogitsune: Nogitsune are wild foxes that do not serve as messengers for the gods. They are known to torment, trick, and even possess humans, sometimes using their shape-shifting powers to do so.
Zenko: Also known as benevolent foxes, the zenko are mostly associated with the deity of rice, Inari. These kitsune are white in color and are also known to be able to ward off evil, also at times serving as guardian spirits. They also help protect humans against nogitsune.

In Japan, there are many Kitsune statues found within Shintō shrines. Therefore, we decided to use a Shintō shrine as our setting for the project.

Along with making the environment be a shrine, we were also influenced aesthetically by the Nintendo game Muramasa Demon Blade . The game is 2D, but we really liked some of the visual cues used in the game in order to articulate more specifics in the setting.

An example of visual cues: the shadows of the pagodas in the background help add another layer to the scene, as it is not just a forest the main character is running through, but a dense forest right next to people. As you continue moving as this character, the pagodas in the background become less, showing that you are progressing away from civilization.

A Shintō shrine (shrines can be cemeteries as well.) A great visual cue here is the green, frightening hue that environment takes on in order to create a sense of otherworldliness.

Storyline so far . . .

And so finally for our story, we decided that we would like it to be based off of the zenko and the nogitsune foxes. The current layout of the narrative would be as follows.

The user finds themselves alone in the middle of a shrine/cemetery during sunset
As the sun sets, the environment starts looking more hostile/surreal (haze, colored skybox, creepy background sound, based off of some of the aesthetics shown above in Muramasa Demon Blade)
Once the environment is fully “surreal”, two foxes appear in front of the user. Both have 9 tails and look similar. (one is an Inari fox, the other is a wild fox that has disguised its appearance)
The user is prompted to “make a choice” and pick one of the two foxes.
If the user chooses the Inari fox, or the “good” fox, the environment goes back to how it normally was (we are still considering different options on how to make this outcome more interesting/exciting)
If the user chooses the wild (bad) fox (which is disguised as a good Kitsune), they stay trapped in the surreal space.

Finally, a helpful summary of some of the feedback we heard in class:

Ending: does it end due to a user’s option? Or just naturally? Or does the user just take the Google Cardboard off ?
How do we hint at the choice that the user has to make? → we could possibly have the kitsunes be on different path and then the user chooses between them → does this mean that they move somewhere else after following the path? The user appears in another part of the shrine?
How do we create a satisfying ending for the good fox? (right now the “bad ending” seems more interesting)

Invisible Cities – Response

To begin, I want to state the fact that I read the book in English first, and then read the majority (but not all of it) in Italian. I wanted to see how good the translation was and honestly, the descriptions were exactly the same. I did not find one clear discrepancy in which I thought “this was translated totally wrong.”

The city I chose to focus on was the city of Zirma on page 16. The reason this city stood out to me was due to its memorable description of the people present: “the blind black man” , “a girl walking with a puma on a leash”, “a fat woman fanning herself” , “a tattoo artist arranging his needles and inks and pierced patterns on his bench.” Many of the cities had descriptions of the environment in detail, but I found the characters described above to be very memorable and easy to picture. So much so, I was inspired to draw this image based off of the city.

For me, the city of Zirma drew me in because I wanted to know the back stories of all the characters, and I was drawn in to learn more about them.

The city really reminded me of when I visited Istanbul in Turkey. I remember seeing so many memorable people in which I wanted to know how they ended up in Istanbul; be they tourist, or people residing in the city. I understand that this is completely subjective, as one can find “interesting” looking people in any city if they look hard enough, but . Istanbul was the first time I saw a mix of hipster fashion next to traditional clothing on every street. I was drawn into many of the people I saw, and wondered where they were from, if they lived in Turkey, or what brought them there.

Boxing with Ares: Project 2 Documentation

Boxing With Ares Final

Project Description

Group Members: Nhi, Vince, Neyva

“Boxing with Ares” gets its name from the Greek god of war Ares, who represents violence. Ares was chosen over the Roman god Mars, due to the fact that Mars represents valor in war instead of the violent aspect we wanted shown in our project.

Within our project, we really wanted to use the motion of “punching” in terms of boxing. While originally this was going to be done with VR controllers which gave us feedback, we needed to scale it back to a mouse click within Unity. There was a loss in translation, but we were able to achieve the boxing animation that we wanted.

The boxing motion takes place in a dark, and bleak environment. Red clouds fill a back sky, and the character stands in a light fog. Small punching bags just beyond reach are spread throughout the sky, filling the space above the user’s head. An ominous soundtrack (which can be found here: https://www.youtube.com/watch?v=Qm-El3qztgw ) plays on loop in the background. The hellish landscape contains nothing the user can interact with except for the one large punching bag situated in the middle of an open area.

The most important part of the project is the fact that once the user punches, white doves appear out of the bag. Since the project had to have an unexpected action occur from an everyday motion, we found the idea of doves emerging from a punching bag to be one of the least expected actions to occur. The dove is meant to represent the fleeting beacon of hope within the dark and hostile environment. Furthermore, the action of hope is juxtaposed by the violent action that caused the doves to spawn from the punching bag.

The thought was to create an alternate universe in which to situate this punching bag. We did not want to create a conventional area, such as a gym or forest in which this would take place, but created our own imaginary scene.

Process and Implementation

When we first began with the project, we thought to put the punching bag in a theater space.

Large punching bag situated on a theater stage, with an audience of chairs.

Our thought process was that any kind of prop can be placed on a theater stage and it will look natural, and so we wanted to plop the large punching bag on the stage instead of using something like a gym environment. In the case of the gym environment, the emergence of the doves would then seem more nonsensical than not.

After talking to Sarah, we reached the idea of creating our own unique environment. An influence from this was the matrix scene (shown below) in which there is nothing to interact with in the space besides the necessary objects.

Matrix scene after Neo first takes the red pill to see the truth

Building from this idea, we then began to look into imaginary dreamscapes that did not fit into a conventional kind of environment.

These were some of our earlier ideas, but finally we found one image which was a huge influence in our project.

We then took elements from this image, such as the lightbulbs being replaced by the smaller punching bags, and the fog. There is also heavy black and gray which we followed as well in our landscape.

Luckily, we were able to find a punching bag/boxing asset in the Unity store for free. It came with a large and small punching bag, plus a character wearing boxing gloves. This was the perfect fit for our project.

Here you can see the 200 punching bags we placed in the sky in order to recreate the lightbulb feeling from the image above.

We then added multiple clouds that were dark red and gray in the sky.

Then within the floor, we created 2 particle systems to create the fog effect around the user. (This tutorial was the one followed: https://www.youtube.com/watch?v=DvKRGwCImJ4)

We used a sheet of textures in order to create the smooth transition within the fog, shown below.

The second particle system was a lot simpler, as it was just floating dust motes. This was done simply using the Unity Particle pack.

Lastly, as part of the main interaction, we took the character from the boxing package and mounted the camera on him. This would allow the user to feel as if it was first person. The boxing package came with the scripts and sound effects to be punched, and so from there we built a script to spawn the white doves whenever there was a detected collision between the user punching and the punching bag.

Reflection

I believe we were able to achieve an alternate reality, due to the fact that the setting was very ominous and was meant to seem like it came from a hellish dream. That, coupled with the music playing on loop and the only interaction being to punch a punching bag definitely allows the user to be in a completely new environment. I believe it was a good choice to follow the matrix scene example in which the only object the user can interact with are necessary items. The large punching bag being the only touchable object and placed all alone is a very clear indicator for the user to come and punch it, and also the hands of the player being boxing gloves is another clear affordance to go and punch the bag and see what happens.

Overall, we were able to create the project that we had envisioned originally, except for the feedback controller aspect. We thought it would have been very fun to punch in VR, and we felt the mouse click from our computer did not match this motion we had originally wanted. BUT, other than that, we were able to create the scene we had originally envisioned.

On Agency…

Originally, through the use of the punching motion through the VIVE controllers, the user was meant to feel satisfaction as seeing the punching bag move and react to their touches, along with satisfaction at seeing birds spawn from their actions. This could have been improved in our project with the mouse click being held in order to “charge a punch” and for that to have a kind of effect when punching. I also spoke above of the clear affordances in our project, as the punching bag was the only thing to interact with within the environment.

Response as a Medium; Reading 2

In Krueger’s Responsive Environments he describes his development of creating responsive environments and installations in order to procure a certain response from his audience. He gives some guidelines on what he learned after his project GLOWFLOW, and I want to break these down. (423)

He sets these as a precedent/series of goals in which he hopes to establish an interactive or “responsive” environment. While I believe that some of these still hold true to the test of time, and can be applicable to people within the interactive media/multimedia field, I think our advancements in technology have allowed for an expansion within certain rules. I believe that number 3 no longer holds true, as we can see example with large groups of people participating within interactive art. (Ex: Teamlab borderless comes to mind. Link https://borderless.teamlab.art/) I also believe that number 6 belongs to preference, as these aspects are important within creating an entire environment. I find it hard to cultivate an environment of interaction when key elements of the art piece are ignored.

Response as a medium should take into account the constant input of it’s users and generating new output within an environment that lends itself to the constant cycle.

Project 1 Documentation: The Land of Weilgalia

Project Description

For my initial project, I wanted to focus on creating a space in the sky. I love the idea of standing on a floating island and looking into the expanse of the sky. I wanted to keep it close to what the Earth’s sky looks like (blue with clouds) and during the day instead of at night.

So I created The Land of Weilgalia (the name was taken as inspiration from the 2003 JRPG game Tales of Symphonia, in which there is a floating city in the sky filled with angels called Welgaia.) This project was completed using Unity in order to make it 360 VR. Furthermore, using an Android Build, it is possible for users to experience The Land of Weilgalia through Google Cardboard. The user should see islands around them, with only a small cropping of land beneath their feet that is filled with green grass. The islands around them are in the distance, and there is no visible way to get from island to island; only to look at them from afar.

Inspiration

This VR project was heavily inspired by the Gamecube Game Tales of Symphonia. Throughout the game, you use magical flying transportation to fly around the map of two interconnected worlds. These flying machines allow you to visit places above the worlds and in the sky, and as such, there are multiple towns you can land in and visit. I focused on their city in the game called Exire.

(Although the name was inspired from the city in the game called Welgaia which is the city of angels, but it had a very different, scifi and futuristic aesthetic which I was not looking for.)

Welcoming animation into the City of Exire

Exire’s aesthetic for the islands is what I was envisioning for my VR project.

The Scifi, futuristic land of Welgaia in the game.

In this game full of amazing, imaginary landscapes, I chose the floating islands due to the calm I think the environment can create. While the area can in theory look frightening, as you are alone very high up in the sky with no reprieve, it can also be very peaceful. With the correct lighting, sound, and some composition decisions, I wanted the user to feel transported into another world where they could explore the serenity found in floating islands in the sky.

I was also further inspired by other games and films with floating islands in the sky such as the feature film Avatar.

This was not an ideal example to follow though, as I found the close proximity of all the islands to give me a sense of claustrophobia. This was the opposite of what I wanted to accomplish, as I wanted the user to have more open space in the expanse of the sky they were viewing. I did like the amount of fog and cloudiness within these clouds, and wanted to take that element into my project. While it can add onto the claustrophobia, I wanted to camouflage the islands a bit within the cover of the clouds.

Another inspiration was from the game Zelda Skyward Sword. The user can fly between islands in the sky in order to go on different adventures. The entirety of the game takes place in the sky, and there is minimal landing onto the surface of the Earth.

The islands in Zelda gave me a good idea of what I wanted the shapes of the islands to look like, but I did not like the ever-present cloud cover under the islands. Again, this is why I defaulted to the Tales of Syphonia game in the city of Exire, as I found it to be a perfect example of what I wanted to create.

Process and Implementation

Since the beginning of the project, I knew that I wanted to avoid using lowpoly assets and use a more realistic or artistic interpretation of the environment. From there, I began to sketch out my idea from the Tales of Symphonia game.

From there, I began to explore the different asset packages I could to begin to create the environment I wanted.

***Apologies in advance, most of my screenshots are not maximized as I did not know how to maximize on unity at the time***

Originally my terrain was flat, but I had to change that in order to make it into an island shape.
A zoomed out picture to show the distance between my islands.
I experimented with the terrain tool in order to create my floating islands.
I had to decrease some of the hills on the terrain in order for the sky around it to be visible.
I inserted a song that I thought was a perfect fit for the scenery.

My audio theme: The city of Exire (as shown above.)

One of the later examples of what my terrain looked like.

After a suggestion from Sarah, I added in different colored lighting to make it less dull in color.

When creating my environment, there were specific things that I knew I had to avoid in order to make it feel serene.

Firstly, the viewer could not be standing on the edge of the floating island they were on. Although this would give me more room to work with in the visuals below the island, I knew that having the viewer on the edge would cause a slight sense of anxiety. Because of this, I situated my user straight in the middle of a medium sized floating island.

Secondly, I knew I needed to make the sky daytime in order to make all my details visible. I considered making the islands at night, but I thought this might give my user a sense of anxiety. I believe I would feel anxious if I were in the dark and I had huge, floating, and unidentified objects around me. I also thought that creating the space at night wouldn’t allow for the wide expanse of space to be appreciated; I wanted the user to see floating islands in the distance, and I thought that would’ve been harder to accomplish in the night.

Thirdly and lastly, I chose to use a semi-realistic landscape in order to connect to something the user is already familiar with. I wanted to take a patch of green grass which everyone is familiar with but place it in an environment that is hard to envision. But this small sense of familiarity would help connect the user into the environment by giving them something small to tether onto.

Reflections

Overall, I did face some difficulties in executing my project as I first intended. Originally, I wanted there to be cities in the distance on the floating islands. This was harder for me to implement, as no matter where I tried to position different buildings, I could not get them to the quality and size that I wanted them to be. This I think I can improve on in time, but within this first project, I really struggled with getting the buildings to look the way I wanted them too. (I also had a lot of trouble with some of the asset packages I would download for buildings. A lot of the scripts would begin to interfere as soon as I downloaded them and I was unable to debug the problems with some of the packages that best fit the aesthetic I was looking for.)

The bottom of the floating islands were also just an enlarged rock prefab from a certain package, and I found this to be very limiting when making the floating islands. In the future, in order to improve on my islands, I would need to find a more viable alternative on how to create the textured bottom of the islands instead of the one rock asset.

Development Journal: The Fantasy Land of Weilgailia

For my first VR project, I really wanted to create a fantasy landscape. I was heavily inspired by video games/rpgs such as Tales of Symphonia, The Legend of Zelda Skyward Sword and Legend of Zelda Twilight Princess. (Pictured below) I took particular interest in their scenes and puzzles in the sky.

Tales of Symphonia; entrance to the floating city in the sky known as Exire

Tales of Symphonia; walking on the floating city of Exire

Legend of Zelda Skyward Sword the city of Skyloft

These served as the inspiration for the fantasy islands I wanted to create, but I also want to include the aesthtics and architecture from Avatar the Last Airbender, and Avatar the feature film (with the blue people.)

An air temple from Avatar the Last Airbender; an example of architecture I want to follow

A temple from Avatar the Last Airbender that is upside down on a piece of land

The floating Islands in the feature film Avatar

Lastly, here is a sketch of my desired design.

I think an important aspect of my project is that I do not want the islands to be seen in full clarity, but to be seen in the distance with only vague shapes of their architecture. The user will be standing on a floating island with no connection to other islands, and will be looking into the distance at the other mythical places.

I hope to put a small amount of motion in the islands floating, as they move a little up and down, and I also hope to use a song, again from the game of Tales of Symphonia. (Possible song videos below)

Hamlet on the Holodeck

In reading Hamlet on the Holodeck, I found it hard to categorize which of the four principal properties of digital environments best suited VR. I think there are two audiences here for VR; the creators and the participants. I make this distinction because I think some elements of digital environments simply apply more to the fabricators’ experiences than to the participants and vice versa.

What do I mean by this? Well, I want to start by saying that I thought procedural was the least fitting for VR. While there is a sense of “procedural” within VR, as there are rules and structures that guide the worlds inside VR, I do not think they apply as much to the audience as the do to the creators. The creators can see the rules and algorithms in the making of VR, but I think in most cases they reach a level of complexity that it becomes hard for the user to understand these patterns. In the reading example of Eliza, her pattern was basic enough for the users to understand and even become frustrated at due to unintentional humor. I do not believe the audience of an open world in VR would begin to understand the series of rules unless the creators themselves wish for the audience to understand.

The most fitting aspect of the four digital rules I found was participatory and spatial. I say this because from personal experience, I found VR games/worlds with the utmost basic layout to still have effective narratives through the user participation. And in the case of VR being focused on more spatial than participatory, it simply becomes a complex type of “film” in which the audience are merely viewers. I think combining these two aspects are what bring the success and wonder of VR.