Category: Additive to Expressive

In the chapter “From Additive to Expressive Form: Beyond ‘Multimedia’,” Janet Murray discussed the ideas of additive forms and expressive forms, stating that “additive formulations are a sign that the medium is in an early stage of development and is still depending on formats derived from earlier technologies instead of exploiting its own expressive power” (p.83). Indeed, additive forms, such as narrative films, are merely dependent on the available technology without any further exploration into possible changes of physical properties. Meanwhile, expressive forms are the result after a long aggressive process of discoveries, inventions, and adoptions, extending the horizon of the current digital world. 

From my perspective, without much prior experience with virtual reality (VR), I believe that our today’s idea of VR must also start as a simple additive form, and has been developed throughout history to achieve the current much more expressive state. VR falls in between additive and expressive forms, as it is really challenging to define VR exclusively belonging to one. It is more of a process where VR is heading towards more innovations in technology and bringing about a better expressive virtual environment for the users. Just a few decades ago, the concept of 3D or VR was still new to most of the people. Now, VR is hitting the mainstream – hundreds of companies are working in creating and improving VR technology, adding on devices and features. VR is existent everywhere, in games and films, and it is what makes the journey of VR development even more appealing. There has been a long way since its start, and I firmly believe that VR is advancing towards a more expressive form.

As is mentioned in Hamlet on the Holodeck, the expressive forms have their own affordances and norms to be regarded as a new media format. From my point of view, VR falls in the expressive form given its unique nature of immersive and interactive experience. One can argue that VR seems to be the first-person form of narrative that could be categorized as a combination of film/game and 3D imaging technology. Nevertheless, the gap between 2D expression and VR experience is so large that we could count it as a brand-new field. In the field of VR, how different types of sensations are simulated undoubtedly replies to certain rules and expectations. To my knowledge, there’s no existing media form that focuses on giving people a realistic feeling of exploring the world with one vision and one sound.

In the chapter “From Additive to Expressive Form: Beyond ‘Multimedia'”, Murray claims that multimedia, or whatever people are referring to with the word, remain to be an additive form and have not yet been exploited of its expressive power, while the 2016 update illustrates how a lot of progress has been made in the industry ever since, in terms of the four affordances of digital environments: “procedural,” “participatory,” “spatial” and “encyclopedic.” In explanation for the terms “additive” and “expressive,” Murray says, “… additive formulations like ‘photo-play’ or the contemporary catchall ‘multimedia’ are a sign that the medium is in an early stage of development and is still depending on formats derived from earlier technologies instead of exploitig its own expressive power” (p. 83). Superficially, “additive formulations” refer to seemingly new media that present no essential breakthrough from their traditional predecessors, while “expressive” media are the ones that are actually innovative and significantly different from their predecessors. It seems to me that the two terms are more like two ends of a spectrum than they are definitive. There exist no standards with which we can define a medium as completely additive but not expressive, or vice versa. All new media, including VR, are to some extent more expressive compared to old ones; it is more a matter of degree of how much expressive power a medium has been exploited of. VR specifically, seems to be towards the more additive end in my opinion, given that it is still on an early stage of development and yet to be very accessible.

It appears interesting to me how Murray mentions the word “convention” a few times in different contexts. She attributes the success of Zork partly to how its designers used “literary and gaming conventions to constrain the players’ behaviors” (p. 96), while also pointing out that “hypertext fiction is still awaiting the development of formal conventions of organization that will allow the reader/interactor to explore an encyclopedic medium without being overwhelmed” (p. 105). Convention seems to be a constraining factor for new media to be more expressive; nonetheless, it is also important that new media break out of old conventions and build new ones. For instance, Murray categorizes e-book as an additive medium, but Kindle, the most famous e-book, channels some conventions of book and as a result, succeeds commercially. Would Kindle still be successful if it displayed text, rather than on separate pages like a book, on a continuous page where readers have to keep scrolling down? The question is, how do we build a new medium upon old conventions but not get contrained by them?

In Chapter 3 of Hamlet on the Holodeck, Janet H. Murray elaborates on the concept of additive and expressive forms. Additive forms, including narrative films (initially), eBooks, and even web soaps, are those that depend and even piggyback on existing media formats instead of taking advantage of the new affordances and forms of expression they offer. Expressive forms, on the other hand, are capable of maximizing their “own affordances that can be used for creating new forms of narrative” (113). For instance, social media platforms, with their own particular rules, norms, and expectations, allow for new modes of expression online. Unless in Twitter, for example, people would not necessarily limit their what they say or express to 280 characters.

–

In my opinion, the additive or expressive nature of VR cannot be fully generalized, and instead seems to change according to its different cases and applications. For instance, it could be argued that 360 films in VR lean more towards the additive spectrum, as it utilizes virtual reality as another, arguably more immersive, movie theater. However, when using 360 VR films in a way that leverages on the capabilities of a VR headset, either through spatial audio or through the capacity of moving inside the film, this medium veers away from its predecessors. For instance, once I experienced a VR film where the user was physically placed in the middle of 4 different locations (an art gallery, a dark alley, an apartment room, and a hallway), all having scenes happening simultaneously. Depending on which scene one views, one could hear and understand what was happening in that one pocket of the story. As the narrative progressed, it became clear that the 4 locations are heavily related to one another and to the overall story, with one character eventually going through all of them. Being an omniscient viewer that could literally see all 4 scenes representing the same moment in time was something I had never experienced before, and which I consider a positive push for VR towards becoming a more expressive form. This same argument can also be applied to VR games, which could be considered the additive form of 3D digital games. When leveraging on the unique affordances of Virtual Reality, such as by providing more immersion through more intuitive controllers whose functionality fits with their use in VR, the medium definitely veers more towards being an expressive form.

Where do you think VR falls in terms of being an additive or an expressive form (or somewhere in between)?

Virtual Reality as a simulated experience seems like the sum of a body sensor and various existed mediums; however, instead of reckoning it as an additive form, we can hardly deny its expressive essence that differs it from other older traditions. VR is more than one plus one equals two, firstly because of the innovative form of 3D visuals. Even though 3D visuals can also be found in 3D films, only with the VR equipment can we have different perspectives when we move around our head or change position. The more you can explore by moving around, the more spacious the VR world feels. Since it also enables 360 degree presentation, it can more easily create an immersive sense, which adds to the possibility of the created world being more imaginative and unrealistic. Besides, rather than only using fingers to trigger some feedback in the case of computer, VR can receive a set of movements of human body which can be designed creatively. As VR tries to build a world around us, it can set rules directly to us instead of the character we control on the screen and thus the restriction and effect on the person’s behavior can also be a new sort of feedback we can get. Moreover, due to the illusion that VR can bring sometimes, the moment when we exit VR can cause a lot of human emotions as we wake up from a dream, which is also very special of VR but different form other mediums. In short, the expressive attribute of VR lies in the way the inside VR world can be explored, the way the interaction between human’s body and the environment happens, the way the person can act in a world that may have completely different rules and perspectives, the way to exit and the way human’s senses can be allured. 

I found it interesting to consider VR as a conceptual descendant of sorts to ELIZA and other archaic technologies that were not as visually intensive. Although one might not immediately see the nontrivial similarities between a chatbot and an audiovisual framework, both technologies are designed to fool the end user’s perception of reality. In ELIZA’s case, this meant fooling the user into believing ELIZA was human; in the case of VR, this means fooling the user into believing that the virtual world that they are seeing and hearing is real. What made ELIZA so impressive was the fact that despite its eery humanity, its implementation was quite small and simple — even current graphing calculators are able to run the full implementation flawlessly. In a way, this was yet another proof that the human senses are easily fooled.

Likewise, VR headsets are portable and compact; although they are still rather computationally intensive, they can run reasonably well on most consumer-grade computers. Without prior exposure to VR headsets, one might not expect these glorified goggles to already simulate reality so well — and yet they do. In fact, there exist many videos online of people who, while watching a VR roller-coaster ride, have toppled over due to the disorienting incongruence between their “real” and “virtual” spatial perception. Thus, it might not be so far-fetched to call VR an ELIZA for the modern age.