Geospatial Data Visualization at West Point
This team of West Point researchers is at the cutting edge of augmented and virtual reality applied research.
Background
The Department of Geography and Environmental Engineering at the United States Military Academy – West Point (USMA) has set up an advanced visualization workspace that facilitates the immersive evaluation and exploitation of three-dimensional (3D) geospatial data. Currently, the workspace provides both an Augmented Reality Sandtable (ARES) and a Geospatial Virtually Immersive Evaluation Workstation (GeoVIEW 3D).
ARES combines an actual table of sand with an infrared Microsoft® Xbox Kinect sensor, and digital light processing (DLP) projector. The Kinect sensor continuously measures the height of the sand and the projector displays topographic contours that dynamically change when the sand is physically manipulated. ARES was developed at the U.S. Army Natick Soldier Research, Development and Engineering Command Simulation and Training Technology Center (STTC) in Orlando, Florida (https://simulation.arl.army.mil/ares/) and is similar to other augmented reality (AR) sandbox hardware configurations (e.g., https://arsandbox.ucdavis.edu/).
The GeoVIEW 3D system supports immersive 3D visualization and is based on a hardware and software configuration developed by UC Davis Associate Researcher, Dr. Oliver Kreylos (http://idav.ucdavis.edu/~okreylos/ResDev/LowCostVR/). GeoVIEW 3D consists of a 75-inch active 3D LED television coupled with an Optitrack® infrared tracking array, Nintendo® Wii remote for user interaction, and open source virtual reality software (VRUI).
The immediate goal of the Geospatial Visualization Lab at USMA is to provide students and researchers with innovative approaches for geospatial data interaction and analysis. ARES and GeoVIEW 3D have proven to be valuable, both in the classroom and in support of research projects. The ability of modern sensors to capture the world around us in high-resolution 3D warrants advanced tools and techniques that promote intuitive interaction and analysis, and strive to harness the full utility of these data.
Applied Research
The Geospatial Visualization Lab was recently utilized for a multidisciplinary project that set out to create an AR experience for the World War II D-Day landing site located at Pointe du Hoc on the northern coast of France. The project team included cadets and faculty from the Departments of History, Geography and Environmental Engineering, Systems Engineering, and Electrical Engineering and Computer Science.
The project resulted in an iOS based mobile device application that allows a user to intuitively explore a high-resolution 3D model of modern-day Pointe du Hoc using a mobile device directed toward a printed copy of the historic map used by the U.S. Army Rangers to plan the assault (Fig. 1). The resulting AR application provides an interactive educational experience that is used in the USMA ‘History of the Military Art’ course and is available to the public through the Apple App Store.
Figure 1: Augmented reality visualization of Pointe du Hoc 3D terrain model on a mobile device (left) and historic U.S. Army Ranger planning map for Pointe du Hoc invasion (right).
The Pointe du Hoc terrain model used in the AR application was created by Second Lieutenant (2LT) Jacob Marchillo – a 2018 USMA graduate who majored in Geospatial Information Science through the Department of Geography and Environmental Engineering. 2LT Marchillo analyzed over 10,000 aerial images of Pointe du Hoc acquired in 2016 by CyArk with a small unmanned aircraft system (sUAS).
Utilizing the Pix4DMapper software, approximately 3,000 of the sUAS images were used to generate a dense 3D point cloud and subsequent 3D surface mesh. Under direction of his project advisor, Captain (CPT) W. Gabe Powell, 2LT Marchillo removed all unnecessary images (e.g. poorly focused or excessively oblique) from the processing workflow, generated numerous 3D reconstructions to assess the effect of resolution and texture manipulation on the resulting 3D mesh, and performed extensive point cloud editing to remove artifacts from the final mesh construction.
GeoVIEW 3D and ARES were both used to assess and explore the resulting Point du Hoc 3D geospatial data. The GeoVIEW 3D system facilitated a visual qualitative assessment of the Point du Hoc 3D reconstruction and provided invaluable insight into the nature of consumer sUAS-based photogrammetric point clouds (Fig. 2).
Using ARES and the resulting 3D terrain model of Pointe du Hoc, CPT Powell and 2LT Marchillo in collaboration with STTC, developed a unique augmented visualization experience that relies on co-registration of the terrain model with the surface of the sandtable. With generalized topography for Pointe du Hoc formed in the sandtable and the addition of a HTC Vive virtual reality (VR) system (for the purpose of the included hand controller and integrated tracking), a user is able to intuitively explore the projected 3D historic sandtable map with a Vive hand controller.
Serving as a virtual camera, the Vive hand control manipulates the point-of-view of the 3D terrain model displayed on the accompanied television (Fig. 3). It should be noted that the Vive Headset is not actually worn for this technique, but serves as part of the tracking system. The seamless interaction between the historic map and the 3D terrain model combined with one’s ability to reach into the virtual environment and manipulate the point-of-view makes for an enriching, insightful experience. Especially, with regard to the incredible feat of U.S. Army Rangers scaling the 100 foot cliffs in order to assault the Nazi fortifications.
Figure 2: Pointe du Hoc Photogrammetric point cloud data displayed in the GeoVIEW 3D system.
Figure 3: Generalized terrain of Pointe du Hoc formed in the sandtable (left) and the integrated visualization of the historic map and 3D terrain model using the ARES system (right).
Further information about 2LT Marchillo’s project, and the visualization systems can be found at the following links:
Pointe du Hoc Project and demonstrations of ARES and GeoVIEW 3D: https://www.youtube.com/watch?v=xdoY-VSCKXk
ARES (Augmented Reality Sandtable): https://simulation.arl.army.mil/ares/About.
GeoVIEW 3D (Geospatial Virtually Immersive Workstation): https://www.researchgate.net/publication/304011852_Setup_and_Calibration_of_GeoMat_VR_A_3D_TV_based_Immersive_Virtual_Reality_Visualization_System
####
Matt O’Banion, Ph.D. and CPT W. Gabe Powell
United States Military Academy – West Point
Geospatial Information Science Program, Department of Geography and Environmental Engineering