Modalities of VR: From 360 images to 3D worlds

VR experiences fall on a spectrum of complexity in terms of what they provide and the requirements for development. Klippel et al. (2019) proposed a taxonomy for virtual field trips (VFT) that includes three levels: basic, plus, and advanced.

Basic iVFT: Provides a virtual replication of a traditional experience. Technical expertise required for production is limited.

Plus iVFT: All features of basic, plus opportunities for new spatial perspectives (e.g., bird's eye view, site-by-site comparison of spatially distant locales, or change of scale). Technical expertise for production is limited.

Advanced iVFT: Allows for additional simulation features (for example, the ability to interact with a rock outcrop over the course of its existence, observing changes over time or collaborating with others). Technical expertise for production is significant.

The taxonomy is helpful in terms of highlighting that a wide range of virtual experiences exist and can be used to support learning. Likewise, there are two end-members to virtual reality technologies that relate to this taxonomy: 360-imagery and 3D worlds.

• Photospheres and 360-degree video - At the most accessible and easily producible end of the VR spectrum is 360-imagery. Given that this technology can readily be used by anyone, 360-degree imagery is ideal for allowing students to visit a real-world field site and development resources are limited. Experiences with 360-degree imagery can be made with still photos or video collected with a 360-degree camera, which are widely available today. Photospheres can even be created from multiple photos taken with any smartphone using Google's Streetv View app (Android; iOS). Many platforms are available for displaying this content and making it accessible for students, such as via Google Maps.
• Interactive 360-degree imagery - Various platforms exist that enable one to embed a variety of interactive content in 360-degree photos or video. It is also possible to link together multiple images to enable students to partake in a virtual tour of one or more field locations, with unique content shared at each stop. Further details are given under the discussion of software.
• Immersive 3D Worlds - At the upper end of the VR spectrum are immersive worlds that students can explore and interact with. These worlds can be fully computer generated or built from real-world sites using topographic data or 3D computer models obtained from laser scanning or photogrammetry (structure from motion or SfM). Experiences can range from simply exploring a terrain (as in Google Earth VR) to providing students with complex interactions by developing applications using game engines (such as Unity3D or Unreal Engine). There are also options for the technologies used for enabling user interaction with the 3D world, ranging from room scale motion with hand controllers to eye and body tracking.

Equipment for Supporting VR Experiences

VR Headsets

The hardware needed to create and experience VR is rapidly evolving, so this overview represents a view on technologies available at one particular moment in time (spring 2020). While the best commercial platforms available today can provide an incredible VR experience, it is important to realize that great educational experiences can also be created and shared without any specialized electronics other than a smartphone. Given the fact that almost all students own smartphones, this may be the most accessible technology for widespread adoption when teaching in classrooms and for off-campus students. The figure below shows a range of hardware that can be used to allow students to engage in VR field experiences.

Smartphone-Based VR: These VR viewers allow students to place their smartphone in a low-cost set of lenses, like those shown in Figures a and b, to view VR content. The most well known example is Google Cardboard (b), which includes one button to allow for input and interaction. The low cost of such viewers (<$20) makes them highly accessible for use in a wide range of teaching environments. Higher-end platforms for smartphones, such as Samsung's Gear VR and Google Daydream (c), expanded these experiences to include a controller as an input device, but have both been recently discontinued and are therefore not recommended.

Pros:

• Low cost
• Accessible for almost all students owning a smartphone
• Works well for experiences utilizing 360-based imagery (such as virtual field trips)

Cons:

• Limited ability to interact with environment (typically only a one-button input)
• Quality is lower than dedicated VR platforms
• No spatial tracking of headset

Standalone VR: There is a trend toward standalone VR headsets that do not require any additional computing or display hardware. The Lenovo Mirage Solo (d) and Oculus Go (e) are examples of low cost devices that provide an experience similar to that obtained with smartphone-based VR, but with improved controllers and motion than available on a smartphone. At a slightly higher price-point, new standalone technologies are emerging that provide experiences similar to computer-based VR platforms. For example, the Oculus Quest (Figure f) allows for full motion tracking for room-scale VR experiences with hand controllers (or direct hand tracking) for enabling sophisticated interactions with the environment.

Pros:

• No additional equipment, wiring, or external sensors required
• Potential for immersive and interactive experiences
• May allow for room-scale VR (with Oculus Quest)

Cons:

• Moderate cost (i.e., likely not accessible for every student)
• Lower-quality experiences than computer-based VR

Computer-Based VR: The best-commercially available VR experiences require VR headsets that are tethered to computers with high-end graphics cards. As such, these systems can provide extremely high-quality experiences that require large memory and computational abilities, such as rendering realistic terrain. The Oculus Rift and HTC Vive are the leading commercial systems.

Pros:

• Best VR experiences possible
• Highly immersive and interactive experiences
• Room-scale VR is readily supported
• Most capable of representing realistic, large-scale terrain

Cons:

• Costs are very high (including an external computer), thus inaccessible to students
• A large amount of space is required to utilize room-scale VR
• Not likely to be used in classroom settings (though some campuses may have labs)

Software for Creating VR Experiences

Software for VR experiences can be classified based on whether it is a resource that can be used directly in an activity versus if it is a tool for creating new experiences.

Using Existing VR Resources

With some creative thinking there are some software resources available today that can be adapted for use in teaching without any development of new content required. Google Earth VR is a good example of a fully immersive tool around which lesson plans could be developed that would allow students to explore 3D terrains around the world. Tailored toward geoscience, the Virtual Field Trip group at the Arizona State University has a very nice collection of interactive field trips using 360-degree imagery and high resolution photos (gigapans), though only some allow for an immersive VR experience (as opposed to being viewed on a computer screen). Likewise, the University of Worcester has developed a set of field experiences focused on VR Glaciers and Glaciated Landscapes that relies on 360-degree imagery. The eRock project at the University of Aberdeen has compiled a set of 3D geologic models and has integrated a few of these into field trip activities. The University of Leeds has developed a variety of virtual landscapes that can be explored via computer, but these do not currently offer a fully immersive VR experience.

Overall, there are relatively few examples of immersive virtual reality experiences designed specifically for the geosciences that are ready for deployment in a classroom. Over time, we hope that this will change and that experiences will be added to the collection of resources available through this website at SERC!

Developing Your Own VR Experiences

There are two main approaches to developing immersive VR content for a classroom. The first approach utilizes editors to enhance 360-degree photos and videos using links to multimedia. The second utilizes game engines to produce arbitrarily complex VR experiences. A quick overview is provided below for context, but more detailed examples will be provided in the Guides and Tutorials section of this website.

360-Image Editors:

The easiest way to get started with creating VR experiences is using an editor such as Google Tour Creator, ThingLink, or WondaVR. There are a wide range of other editors for creating tours with 360-images, but these three provide examples of web-based editors with representative capabilities. In most cases, the final product is accessible to students via the web or through a free proprietary app.

Google Tour Creator allows you to add limited multimedia content into a 360-photo and link multiple photos together. The two main advantages of Tour Creator are that (i) students in the same physical setting, i.e., a classroom, can join the same tour and be guided by an instructor using Google Expeditions (there is no remote capability to do this), and (ii) 360-photos can be pulled directly from Google Maps to create a VR experience. Click here for an example.

ThingLink is an editor that provides a great deal of flexibility in the content that can be embedded within a 360-image or video. Photos, videos, audio, or essentially anything from the web can be embedded, including forms, 3D models, or interactive maps. Experiences can be accessed directly from the web and used with Google Cardboard and a smartphone to provide an immersive experience, though interactions are limited to button clicks so students may only be able to interact with some embedded material via a computer screen and mouse. Click here for example.

WondaVR Spaces allows for the ability to mix 3D models with 360-photos and video to support interactive virtual tours on a desktop computer or VR headset, such as the Oculus Go. A key feature of WondaVR is that an instructor can guide and interact with remote students in the virtual environment. WondaVR Spaces only supports embedding of images, video, audio, and text, however, it does have an integrated quiz system that allows students to answer automatically scored multiple choice questions. Click here for an example(this example was created using the older WondaVR Studio).

Game Engines:

To create game-like interactions in three-dimensional environments, it is generally necessary to develop experiences using a game engine, such as Unity or Unreal Engine. Creating experiences within these systems is typically a time intensive task and it is recommended that this work is completed in coordination with a developer. Both of the above engines, however, have excellent tutorials for the brave of heart!

Asset Repositories:

Creating 360-imagery and 3D models is easy, but It is not always possible to collect your own imagery or 3D models. There are some great places to obtain content that can be licensed through Creative Commons or other agreements. For example, 360-images can be obtained from general commercial sites like 360cities or platform-specific content repositories (e.g., ThingLink, Google Maps). Three-dimensional models can be obtained from sites like sketchfab or geology specific efforts, like eRock.