Learning through Play: The Augmented Reality Sandbox

“Augmented reality can also help students learn by placing course content in rich contextual settings that more closely mirror real-world situations in which new knowledge can be applied” (Horizon Report, 2016).

Augmented reality (AR) offers a new way of seeing and interacting with the learner’s natural environment. Augmented reality describes the addition of a computer assisted contextual layer of information to the physical world, thereby creating an enhanced experience. One common application is the visualization of large datasets. Instead of exploring and manipulating the data via a computer interface, learners can control and interact in a real space, by moving material with their finger, hand, arm, or body.

Augmented reality used to require specialized equipment, none of which was widely accessible or easily portable. Today’s applications and mobile devices allow digital information to be overlaid anywhere, anytime, at low cost. This opens the door for creative educational scenarios. While most augmented reality applications target older students and adult learners, informal learning spaces such as museums have broadened the audience to various age groups, including younger children, even in the pre-K sphere.

It is my pleasure to talk to the implementors of such an application: NC State Researchers Dr. Robert Reed and Josh Mathis from the Center for Applied Aquatic Ecology constructed an Augmented Reality Sandbox with funding from the Burroughs Wellcome Fund. Oliver Kreylos, a computer scientist studying 3D scientific visualizations and computational geosciences at UC Davis designed and programmed the AR Sandbox software, supported by a National Science Foundation grant. The Augmented Reality Sandbox allows learners to ‘move mountains’ in the sandbox and with the wave of a hand create rainfall and see how these interventions affect the resulting water flow, thereby fostering the understanding of watersheds and subsequently, our role in protecting water quality.

Please tell us a little about the concept behind the AR sandbox. How does it technically work?

The AR Sandbox uses a computer projector and a motion sensing input device (a Kinect 3D Camera) mounted above a box of sand.  As a visitor interacts with the sand in the box, the Kinect detects the distance to the sand below, and a visualization of an elevation model with contour lines and a color map assigned by elevation is cast from an overhead projector onto the surface of the sand.  As visitors move the sand, the Kinect perceives changes in the distance to the sand surface, and the projected colors and contour lines change accordingly.

Rainfall occurs when an object like a hand is sensed at a particular height above the surface of the sand.  The virtual rain appears as a blue, shimmering visualization on the surface below.  The water appears to flow down the slopes to the lower surfaces.  Real models of fluid dynamics have been used to base the depiction of water flow.

This design is based on ‘tangible computing’ which is using objects in the physical world that can be manipulated to alter or operate a computer program, in this case, a visualization of a landscape.  The camera in the bonnet of the exhibit is a Microsoft Kinect camera, the same camera used in video games.  It uses an infrared projector, camera and special microchip to track the movement of objects in 3D.

Are watersheds a particularly fitting topic for an augmented reality environment?

NC State University’s Center for Applied Aquatic Ecology is committed to reaching out to the community to increase awareness of water quality as well as empowering members to help care for water resources.  Understanding the watershed you live in is an integral part of this.  We have a Floating Classroom Program aboard a research/education ship, RV Humphries located on the Neuse River in New Bern, NC, which has provided hands-on education to students and their teachers about the importance of protecting the quality of the Neuse River watershed.

We also conduct summer camps teaching the many aspects of water quality and other scientific topics thanks to the generous funding of the Burroughs Welcome Fund.  We have found that the more hands on and engaged we can keep our audience, the more learning is accomplished.  Once we found the idea of the Augmented Reality Sandbox, we knew it would be instrumental in communicating the important aspects of watershed protection.  By displaying an infinite number of landscape scenarios as dynamic as nature itself, the AR Sandbox engages all who dig their hands in the sand to fully appreciate the rain, water flow, creation and destruction of landforms and the interaction between them.

Was the sandbox always intended as a learning space or do you apply the sandbox in your research lab?

The AR Sandbox has only been used as an education outreach tool at this point.  The research at our Center is primarily analytical as we focus on monitoring many reservoirs and estuaries for specific water quality parameters.  We are using the AR Sandbox as a tool to communicate the importance of safeguarding these water resources.

How do different age groups vary in their interaction with the AR sand box?

The AR sandbox’s strongest appeal is the fact that it entices young and old to get involved.  While the concepts and scenarios are kept simple for the younger kids, one can still discuss contour lines, mountains, piedmonts, valleys, damns, watersheds and basic water flow.  When you are engaging older students or adults, the scenarios and the concepts can become more complex.  The learning experience can now include discussions of landforms, elevation, and best land management practices such as retention ponds and swales.  Processes such as erosion, tectonics, and glaciation can even be visualized.  These elements are all principles of geomorphology, hydrology, earth science and environmental studies.

Were you surprised by the reception of the exhibit by children and adults at the children’s museum?

We were very pleased to see the reaction by both the adults and children.  An interesting observation was the interaction between parent and child while playing in the sandbox together.  Both were having fun and communicating between one another what they were doing and what was happening.   There was a definite connection between them while exploring the different attributes of the sandbox.

We must say that we expected a strong interest and were not disappointed.  Our favorite quote that a Kidzu employee shared with us was the child, with hand on hip, telling his father who was deep in play,  that he had to “leave the sandbox as they were already late for lunch.”

How does the learning experience compare to a more traditional presentation of the material?

As mentioned earlier, we have noticed in all of our education outreach adventures, the more engaged and hands-on your activity is the higher level of learning occurs with any audience.  This is regardless of whether the audience is a group of preschoolers or college level hydrology students.  This is just one more tool to engage them in that learning process.

Based on your experiences, is the development effort justified by learning gains?

Most definitely, yes!  Even though our researchers, Dr. Robert Reed and Josh Mathis were working off of recommendations from UC Davis, there were still many aspects to our specific design that needed to be engineered.  We knew we needed it to be easily transported and that set-up and break down should be accomplished in a relatively short amount of time.  Josh Mathis was able to come up with an excellent version of the sandbox that is working very well for our goals.  It took the better part of a 3 month venture for the finished product but we are certain that many will benefit from the educational experience that the AR sandbox will bring for many years to come.

What are next steps for your organization? Are you planning on developing more AR learning tools? What is the future of the sand box?

We are committed to continuing our education outreach on many levels.  Even though we are not planning on developing any more AR learning tools at this point, we do want to expand our overall outreach efforts as well as the AR sandbox at local museums, area schools, and NC State Classes.  We will be featuring it at the Water Resources Research Institute of the UNC System (WRRI) Annual Conference on March 17-18, 2016.  This two day conference features oral presentations, poster presentations, themed panel discussions, ample networking opportunities, and hands-on interactive sessions for in-depth discussions and problem solving related to North Carolina’s water resources.

Further Information

“Shaping Watersheds” Augmented Reality Sandbox Facilitator’s Guide, lead author Dr. Sarah Reed, science and technology educator at the Lawrence Hall of Science, University of California, Berkeley.