Penn State Bolsters Experiential Learning Through Immersive VR Technology And Simulations

Written by Kevin Sliman — Communications Manager, Institutes of Energy and the Environment

For many, seeing is believing. That is what Alex Klippel and his team have been investing in since the creation of Penn State's Center for Immersive Experiences . It is also what some energy and environment researchers have been exploring to better communicate their science.

“Immersive technologies extend opportunities to not only see changes in the environment but also to experience them,” said Klippel, director of the Center for Immersive Experiences. “We can advance understanding and our ability to communicate scientific facts about the environment to a different level by creating embodied place-based experiences of remote locations, of places in the past or future.”

The mission of the Center for Immersive Experiences is to provide cutting-edge immersive technologies, knowledge, and skills to the Penn State community. Its goal is to create leadership in immersive experiences by advancing the underlying science and integrating it into the academic spectrum through research, education, and outreach. The center is a University-wide initiative that is supported by 12 colleges, the University Libraries, and the Institute for Computational and Data Sciences.

According to Klippel, simulations play a key role in many academic fields. With today’s technology, there is the opportunity to simulate any experience the imagination or scientific data can produce.

“It’s not just what physical reality allows us to experience,” said Klippel. “It is at scales and perspectives usually not accessible to humans. We can crawl like an ant or look at the world from above like a giraffe or drone. We can make the unseen visible, and we can experience the consequences of our actions.”

One of the researchers using the center is Erica Smithwick. She and her team are working on a National Science Foundation-funded project called “ Visualizing Forest Futures .”

Visualizing Forest Futures allows users to interact with a forest, exploring the impacts of climate change on various tree species.

“We were looking for a medium in which we could engage communities more deeply in conversations about climate change and forest management,” said Smithwick, associate director of the Institutes of Energy and the Environment (IEE). “The immersive environment of virtual reality (VR) allows us to create an environment in which stakeholders can feel immersed in a forest that they can actually visit today.”

Smithwick said traditional landscape modeling that is used to simulate the effects of climate change on forests generally produces landscape maps, which might only be useful for land managers or scientists.

In Visualizing Forest Futures, a user can visit a forest simulation under current conditions as well as a possible future under climate change.

The model used in the VR simulation is driven by the team’s understanding of forest management practices of the Menominee Nation of Northern Wisconsin and by the National Forest Service. Both groups are part of the project’s research team. In addition to the forests of today, the simulation also offers climate change scenarios of what future forests could look like.

“Through that experience, the goal of the project is to find out what they value about forests generally, said Smithwick, “and whether or not these future forests that are being simulated represent those values fully, or are things missing?”

To make the simulated forest as real as possible, Smithwick and her colleagues worked extensively with the Center for Immersive Experiences. They have correlated forest data in the model and transferred it to a VR environment, a process that took many iterations.

“It's actually quite complex to move from the model output into a virtual experience that looks realistic and that actually models the forests in a way that looks right, feels right, and sounds right,” Smithwick said.

Jessica Myrick, an IEE researcher and associate professor of media studies in the Donald P. Bellisario College of Communications, worked on a project involving a virtual reality game.

The VR game allows the user to walk through the Critical Zone Observatory and interact with the surrounding ecosystem in dynamic ways.

In the game, the player is a journalist sent to examine how events such as rain and new development projects might affect the nearby ecosystem.

“Alex’s team designed the experience to look like the Shale Hills Critical Zone (CZ) Observatory ,” she said. “If you were to literally walk around Shale Hills, you would not necessarily see the interaction of all the different Earth systems involved.”

Through virtual reality, Myrick said the experience shows people how water flows through soil and how water levels affect plants, air, and humans in different ways at different times.

The CZO simulation allows users to see the impact of rain above and below ground.

“VR allows us to literally lift out sections of the ground and give people a view of what, during our normal lives, is all invisible below our feet,” Myrick said. “If seeing is believing, then VR allows us to help people not only learn about the Critical Zone but also more fully grasp its importance to our life.”

Upon completing the simulation of the Shale Hills Critical Zone Observatory, the team then connected their digital model to an HTC VIVE Pro PC-VR kit. The advanced image resolution and high-definition audio of the VIVE Pro help to bring students the full experience of nature’s ecosystem, allowing for a more enriching learning opportunity.

Myrick said her team chose to use the Center for Immersive Experiences because it can be difficult to understand the dynamics of the CZ for those who have not dedicated their lives to studying it.

“We thought using a VR game would help people more easily approach the complex, interwoven dynamics of CZ science in a way that is easier to visualize and understand,” she said.

For Myrick’s role in the project, she is exploring how playing a VR game related to the earth sciences might make people more curious about earth science and the environment.

“If we can generate curiosity, then we don’t have to depend on feeding people a steady diet of VR games for them to learn,” she said. “Instead, we know that curiosity motivates people to continue learning after the initial curiosity-evoking experience, and hopefully being curious can help generate additional knowledge gain and policy interest in the future.”

Jennifer Baka used the center to educate her students on energy.

The field trip allows users to visit different energy sites around Pennsylvania.

“It’s so hard to coordinate an actual field trip to an energy site,” said Baka, an IEE researcher and assistant professor of geography in the College of Earth and Mineral Sciences. “They tend to be far from State College, off the beaten path, and, in the case of fracking and solar, on private lands.”

Baka said VR tours are a great way to immerse students in energy landscapes to overcome these challenges.

“Going forward, I would like to use VR technology in my work on energy governance and public participation,” she said. “I’d like to have different stakeholders — government, civil society, industry — get together, do the tours and then sit down and talk about differences.”

Baka said she believes this has a lot of potential, including overcoming polarization in energy debates or making energy decision making more democratic.

“It’s been really great collaborating with Alex and Tom [Murphy, Baka’s project colleague]. We each bring different, complementary skills to the table, which has made for good interactions,” Baka said. “The tours also add some great diversity to my teaching and bring classes to a whole other level.”