Facilitating Spatial Thinking with Augmented Reality Sandboxes

KATHERINE RYKER (kryker@emich.edu), assistant professor of geography and geology, Eastern Michigan University, Ypsilanti, Michigan; KAREN MCNEAL (ksmcneal@ncsu.edu), associate professor of marine, Earth and atmospheric sciences, North Carolina State University, Raleigh, North Carolina; RACHEL ATKINS (ratkins@ncsu.edu), PhD student in marine, Earth and atmospheric sciences, North Carolina State University, Raleigh, North Carolina; NICOLE LADUE (nladue@niu.edu), assistant professor in geology and environmental geosciences, Northern Illinois University, DeKalb, Illinois; and CHRISTINE CLARK (cclark7@emich.edu), professor of geography and geology, Eastern Michigan University, Ypsilanti, Michigan

Augmented Reality (AR), a technology that enhances a realistic digital display of the physical world with additional digital elements, is increasingly being explored to teach and communicate difficult-to-grasp topics to learners. AR occupies the center in a continuum between the real and virtual world (Milgram and Kishino, 1994), with the real world having no digital augmentation and the virtual world having no physical context (Greeno, Moore and Smith, 1993). It straddles the middle path by registering digital information onto real-world objects or locations.

The AR Sandbox is an innovative and powerful tool available to those who teach map-reading skills. Noting the possibilities for collaboration on exploring the sandbox as a pedagogical tool and as an educational research project, we formed an inter-institutional partnership. Here, we share our version of an AR Sandbox, advice on building your own, some of the potential learning benefits and challenges, and research questions we see arising as we move forward in an era of AR technology.

The AR Sandbox is a physical box of sand that can be manipulated by students to represent mountains, volcanoes, river channels, or glacial deposits. Microsoft's Xbox Kinect camera and a digital data projector project contour lines on the sand surface in real time to fit students' landforms, enabling them to see the topographic map as it is created and modified in concert with their sandbox landforms. Students can also make it virtually "rain" over the display to see where water will flow and collect. The idea behind this tool was initially developed by Piper et al. (2002), but was made much more easily and economically buildable by the LakeViz3D project (Reed et al., 2016), whose team has provided the user community with a variety of technological support, including open access Linux code. Instructors can download the software and see hardware recommendations on Kreylos's (2016) website. The original Sandscape cost over $40,000 to build, but advancing technologies have dropped the cost to under $3,000 (see Table 1). Additionally, there is a community of users ready to help build and troubleshoot these tools: more than 150 institutions around the world are using them for everything from informal education at museums to upper-level structural geology courses (Reed et al., 2016). We built three AR Sandboxes using Kreylos's code and plans for the physical sandbox developed by Giorgis and Mahlen (pers. comm., 2015). These are being shared by various departments at Eastern Michigan University, North Carolina State University, and Northern Illinois University.

One potential learning outcome of using the AR Sandbox is enhanced spatial skills, needed in geoscience topics from hydrology to atmospheric dynamics. It can be difficult to help those new to geology, for instance, to build the skills needed to integrate the geologic history of a region based on localized rock textures and structures in the field (Manduca and Kastens, 2012). When a novice first looks at a topographic or geologic map, there can be an initial feeling of being overwhelmed by "a jumbled array of randomly converging lines, circling and curving in haphazard directions across a map's surface [where relationships] are not immediately apparent" (Rapp et al., 2007; p. 1). While these feelings diminish with hours spent pouring over maps in labs, in lectures, and out in the field, it would be more efficient and effective if that initial experience were less abstract to students. The use of an AR Sandbox could enable them, sooner and with greater accuracy, to use maps to interpret information about streamflow, delineate drainage basins, and describe how the Earth's surface changes through processes like erosion and deposition, among other tasks.

The AR Sandbox may improve learning through other mechanisms as well. For students in introductory general education classes, for example, being able to interact with a topographic map and see it change in real time could raise interest and motivation. Since topographic maps are a known challenge for introductory students, enabling them to compare features on the sandbox to those seen in traditional paper-based labs could create a bridge between the tangible and the abstract. Even after they learn the basic skills of topographic map reading (e.g., contour intervals, hills, Rule of V's), the AR Sandbox could help them relate topographic maps to more complex structures. For example, instructors could add geologic structure models to the sandbox to demonstrate the topographic features of folds, faults, or geomorphological structures.

As always, introducing technology can present challenges during implementation and interaction with the new concepts (Edelson, 2001). Among them are these three things.

The instructor needs to be familiar with the Linux-based software in order to troubleshoot calibration issues that arise when setting up the sandbox. Once the sandbox has been set up to manipulate attributes of the display (rain, color-shaded relief, etc.), however, students should be able to use a list of provided commands to complete assigned tasks.

Though each of our sandboxes was set up to be moveable (either on a utility cart or on wheels), we found the time needed to take down and set up the sandbox again to be a limiting factor. We suggest having a semi-permanent location where students can come to the sandbox. Moving the sandbox for recruitment-type events on campus is still possible but, generally, it should stay in place.

Finally, it is possible that the novelty of the tool itself may be a distraction to learning. Because this technology is so new, we have found some students are hesitant when given the opportunity to freely play with the sand. This suggests the need for a scaffolded activity or the presence of an instructor to ask students questions about what they are seeing.

We have introduced the sandbox into geoscience labs at our institutions by modifying existing content or adding an additional lab. Students use it as an exploratory learning tool to play with a topographic map in 3-D in conjunction with the 2-D versions traditionally used in labs. They are not provided with formal questions to answer. Instead, lab instructors encourage "free play" by giving an initial prompt for timid students (e.g., "Try building an island"), followed by questions that promote reflection (e.g., "Have you seen that pattern anywhere else?"). A more structured model could involve giving students scenarios to model in the sandbox and requiring them to submit their answers for scoring.

Questions To Be Researched

As with any new tool or technology, it is usually the practitioners who first recognize the need to determine its efficacy and effectiveness and do the initial trials. Testing helps in the development of best practices by empirically determining what works and what doesn't work to accomplish desired learning outcomes. The evidence that has been collected about the effects of AR on learning are "shallow" (Wu et al., 2013), and limited research has tested its efficacy. Professional presentations on the AR Sandbox specifically (e.g., Clucas and Wirth, 2015; Jenkins et al., 2014; Giorgis et al., 2015; Woods et al., 2015) have focused on the fact that students respond positively to the technology, but these claims are anecdotal and not focused on learning gains.

Our institutions are working collaboratively to address a variety of research questions including: Is free play or structured questioning more effective for learning about topographic maps when students use the sandbox? How are students' spatial skills impacted by interacting with the sandbox? What are the characteristics of students who learn the most while using the sandbox? How can we utilize such AR experiences in online platforms? These are just a few examples of potential research directions that should be addressed as the AR Sandbox becomes more ubiquitous. There is a unique opportunity in our community to conduct cross-institutional experimental studies testing such research questions.

REFERENCES

Clucas, T., and Wirth, G.S., 2015, Seeing the world in a grain of sand: Abstract ED51B-0813 presented at the American Geophysical Union Fall Meeting, San Francisco, California, 14 December.

Edelson, D.C., 2001, Learning-for-use: a framework for the design of technology-supported inquiry activities, Journal of Research in Science Teaching, v. 38(3), p. 355-385.

Giorgis, S., and Mahlen, N., 2015, personal interview and e-mail.

Giorgis, S., Mahlen, N., and Anne, K., 2015, Visualizing topography and the Rule of V's using an Augmented Reality Sandbox, Geological Society of America Abstracts with Programs, v. 47(7), p. 109.

Greeno, J. G., Moore, J. L., and Smith, D. R., 1993, Transfer of situated learning, in Transfer on Trial: Intelligence, Cognition, and Instruction: Westport, Connecticut, Ablex Publishing, 296 p.

Jenkins, H.S., Gant, R., and Hopkins, D., 2014, Shifting sands and turning tides: Using 3D visualization technology to shape the environment for undergraduate students: Abstract ED53B-3489, American Geophysical Union Fall Meeting, San Francisco, California, 12 December.

Kreylos, O., 2016, Augmented Reality Sandbox, http://idav.ucdavis.edu/~okreylos/ResDev/SARndbox/, accessed June 8, 2016.

Manduca, C.A., and Kastens, K.A., 2012, Geoscience and geoscientists: Uniquely equipped to study Earth, in Kastens, K.A., and Manduca C.A., eds., Earth and Mind II: A synthesis of research on thinking and learning in the geosciences: Geological Society of America Special Paper 486, p. 1-12.

Milgram, P., and Kishino, F., 1994, A taxonomy of mixed reality visual displays, IEICE transactions on information and systems, v. 77(12), p. 1321-1329.

Piper, B., Ratti, C., and Ishii, H., 2002, Illuminating clay: A 3-D tangible interface for landscape analysis, in Proceedings of the CHI 2002 conference on human factors in computing systems, Minneapolis, Minnesota, April 2002, p. 355-362.

Rapp, D.N., Culpepper, S.A., Kirkby, K., and Morin, P., 2007, Fostering students' comprehension of topographic maps: Journal of Geoscience Education, v. 55(1), p. 5-16.

Reed, S., Hsi S., Kreylos, O., Yikilmaz, M.B., Kellogg, L.H., Schladow, S.G., Segale, H., and Chan, L., 2016, Augmented reality turns a sandbox into a geoscience lesson, Eos, 97, doi:10.1029/2016EO056135.

Woods, T.L., Woods, J.A., and Woods, M.R., 2015, Using the Kreylos Augmented Reality Sandbox to teach topographic maps and surficial processes in introductory geology lab at East Carolina University. Geological Society of America Abstracts with Programs, v. 47 (7), p. 111.

Wu, H.S., Lee, H.W., Chang, H.Y., and Liang, J.C., 2013, Current status, opportunities and challenges of Augmented Reality in education, Computers & Education, v. 62, p. 41-49.