The History and Science of Virtual Reality

This article describes the design and implementation of an experimental undergraduate course, team-taught by a cognitive scientist and a scholar of Italian studies and focused on the history, science, and technology of 3D vision, from the Renaissance invention of linear perspective to contemporary concepts of ‘virtual reality’. Analysing such devices as the panorama and the stereoscope and the immersive experiences they enable, we ask: What is the information in a 2D image that enables us to see, and interact with, a three-dimensional environment? What are the goals and the cultural implications of the immersive experiences produced by our optical devices? By reviewing project-oriented pedagogical strategies aimed to combine experimental and critical methods, the article suggests a possible way of crossing the divide between scientific and humanistic curricula, toward a holistic understanding of 3D vision and ‘virtual reality’.SOMMARIOQuesto articolo descrive la progettazione e l’attuazione di un corso universitario sperimentale, insegnato da uno scienziato che studia lo sviluppo cognitivo e uno studioso di italianistica; si concentra sull’aspetto storico, scientifico e tecnologico della visione 3D, dall’invenzione rinascimentale della prospettiva lineare al concetto di ‘realtà virtuale’. Attraverso l’analisi di dispositivi quali il panorama e lo stereoscopio e le esperienze di tipo immersivo che gli stessi permettono, ci si chiede: quali sono le informazioni contenute in un’immagine bidimensionale che ci permettono di vedere ed interagire in un ambiente tridimensionale? Quali sono gli obiettivi e le implicazioni culturali delle esperienze immersive prodotte dai nostri apparati ottici? Attraverso la revisione di strategie pedagogiche basate su progetti realizzati dagli studenti combinando metodi critici e sperimentali, l’articolo suggerisce un percorso per oltrepassare la separazione tra curricula scientifici ed umanistici e giungere a una comprensione olistica della visione 3D e della ‘realtà virtuale’.KEYWORDS: virtual realityaugmented realityextended realitypedagogycognitive science3D visionPAROLE CHIAVE: realtà virtualerealtà aumentatarealtà estesapedagogiascienze cognitivevisione 3D Notes1 The Collaborative Humanities initiative is described at . All online sources were verified on 20 October 2022.2 For the whole description, see .3 The entire list can be accessed at .4 Massimo Riva, Shadow Plays: Virtual Realities in an Analog World (Redwood City, CA: Stanford University Press, 2022). Accessible at .6 The quotation can be read in its context at . For a discussion of virtual Italy in the age of the Grand Tour, see Stephanie Malia Hom, The Beautiful Country: Tourism and the Impossible State of Destination Italy (Toronto: University of Toronto Press, 2018) .7 C. P. Snow, The Two Cultures and the Scientific Revolution (New York: Cambridge University Press, 1961).8 Carlo Alberto Zotti Minici, Dispositivi ottici alle origini del cinema: Immaginario scientifico e spettacolo nel XVII e XVIII secolo (Bologna: CLUEB, 1998), pp. 67–8.9 The playful nature of scientific discovery and the fundamental role of play, and games, in the evolution of the modern mind have been explained by Steven Johnson, Wonderland: How Play Made the Modern World (New York: Riverhead Books, 2016).10 The syllabus is accessible at .11 Samuel Edgerton, The Mirror, the Window and the Telescope: How Renaissance Linear Perspective Changed Our Vision of the Universe (Ithaca, NY: Cornell University Press, 2009); Anne Friedberg, The Virtual Window: From Alberti to Microsoft (Cambridge, MA: MIT Press, 2009); David Hockney and Charles M. Falco, Secret Knowledge: Rediscovering the Lost Techniques of the Old Masters, new and expanded edition (New York: Viking Studio, 2006).12 Martin Kemp, The Science of Art: Optical Themes in Western Art from Brunelleschi to Seurat (New Haven, CT: Yale University Press, 1992).13 Lorenzo Selva, Sei dialoghi ottico-pratici dedicati all’eccellentissimo senato da Lorenzo Selva ottico pubblico stipendiato (Venice: Simone Occhi, 1787), p. 170.14 Francesco Algarotti, An Essay on Painting Written in Italian, English translation (London: printed for L. Davis and C. Reymers, 1764), pp. 74–5; and Il Newtonianismo per le dame, ovvero, dialoghi sopra la luce e i colori, 1737. I quote from p. 80 of the 1739 edition (Naples: Giambattista Pasquale, 1739): ‘La camera oscura è l’interiore del nostro occhio, ch’è della figura a un dipresso d’una palla; il foro della finestra è la pupilla, che è nella parte anteriore dell’occhio, e che apparisce in tutti, come un foro nero ora più grande, ed ora più picciolo ... ’. English translation by Elizabeth Carter, Sir Isaac Newton’s philosophy explain’d for the use of the ladies: In six dialogues on light and colours (London: printed for E. Cave, at St.John’s-Gate, 1739), p. 136. See this passage from the latter work by Algarotti: ‘But what will you imagine if I say to you when we are in it [the camera obscura], suppose yourself to be placed in one of your Eyes, and to see everything that passes there?’15 Among general introductory readings, in addition to Shadow Plays and handouts collecting scientific primers and primary sources: Jonathan Crary, Techniques of the Observer: On Vision and Modernity in the Nineteenth Century (Cambridge, MA: MIT Press, 1992); Oliver Grau, Virtual Art: From Illusion to Immersion (Cambridge, MA: MIT Press, 2004); Stephen Ottermann, The Panorama: History of a Mass Medium (New York: Zone Books, 1997).16 Wendell Holmes’s theories and the experience of stereoscopic travels are discussed in chapter 6 of Shadow Plays, ‘Sun Sculptures and Fossilized Shadows: Italy Through the Stereoscope’.17 Stereoscopic kits from the Brown library are available to students both in class labs and on reserve, along with headsets and 360° videos.18 Dhanraj Vishwanath, ‘Toward a New Theory of Stereopsis’, Psychological Review, 121.2 (2014), 151–78 (p. 151); Dhanraj Vishwanath and Paul B. Hibbard, ‘Seeing in 3-D with Just One Eye: Stereopsis Without Binocular Vision’, Psychological Science, 24.9 (2013), 1673–85.19 Jeanine K. Stefanucci, Morgan Saxon, and Mirinda Whitaker, ‘Embodied Perception and Action in Real and Virtual Environments’, in Handbook of Embodied Psychology, ed. by Michael D. Robinson and Laura E. Thomas (Cham: Springer, 2021) .20 Roderick Coover ; Brooke Belisle .21 David J. Chalmers, Reality+: Virtual Worlds and the Problems of Philosophy (New York: Norton, 2022); Matthew Ball, The Metaverse: And How it Will Revolutionize Everything (New York: Liveright, 2022).22 Jean Baudrillard, Simulacra and Simulation, English translation by Paul Foss, Paul Batton, and Philip Beitchman (Ann Arbour: University of Michigan Press, 1983) (original French edition: Simulacres et simulation, 1981)23 While simulation is a developing field of theoretical and pedagogical foundation, and a strategy used in Education studies in order to assess and evaluate outcomes, or in experiential learning, as it especially applies to schools of Nursing and Social Work, we adopt it as a broader critical process. See e.g., Shalini Rahul Tiwari, Lubna Nafees, and Omkumar Krishnan, ‘Simulation as a Pedagogical Tool: Measurement of Impact on Perceived Effective Learning’, International Journal of Management Education, 12.3 (2014), 260–70. For a critical view: Gwen Erlam, Liz Smythe, and Valerie Wright-St Clair, ‘Simulation Is Not a Pedagogy’, Open Journal of Nursing, 7 (2017), 779–87. DOI: 10.4236/ojn.2017.7705.24 Jay David Bolter and Richard Grusin, Remediation: Understanding New Media (Cambridge, MA: MIT Press, 1998).25 A discussion of the course, in the form of an interview conducted by the director of the Cogut Institute, Amanda Anderson, can be found in a podcast (4 December 2020) at . On the same page a short video illustrates some of the projects realized by students in the spring of 2020

Art · Art history · Human–computer interaction · Virtual reality · 3D Surveying and Cultural Heritage · Computer Science · Educational and Social Studies · Virtual Reality Applications and Impacts