(Editor’s Note: Mad Scientist Laboratory is pleased to present the second guest blog post by Dr. Richard Nabors, Associate Director for Strategic Planning and Deputy Director, Operations Division, U.S. Army Research, Development and Engineering Command (RDECOM) Communications-Electronics Research, Development and Engineering Center (CERDEC), addressing how Augmented and Mixed Reality are the critical elements required for integrated sensor systems to become truly operational and support Soldiers’ needs in complex environments.
Dr. Nabors’ previous guest post addressed how the proliferation of sensors, integrated via the Internet of Battlefield Things [IoBT], will provide Future Soldiers with the requisite situational awareness to fight and win in increasingly complex and advanced battlespaces.)
Speed has always been and will be a critical component in assuring military dominance. Historically, the military has sought to increase the speed of its jets, ships, tanks, and missiles. However, one of the greatest leaps that has yet to come and is coming is the ability to significantly increase the speed of the decision-making process of the individual at the small unit level.
AR and MR function in real-time, bringing the elements of the digital world into a Soldier’s perceived real world, resulting in optimal, timely, and relevant decisions and actions. AR and MR allow for the overlay of information and sensor data into the physical space in a way that is intuitive, serves the point of need, and requires minimal training to interpret. AR and MR will enable the U.S. military to survive in complex environments by decentralizing decision-making from mission command and placing substantial capabilities in Soldiers’ hands in a manner that does not overwhelm them with information.
In the near-term, the Army Research and Development (R&D) community is investing in the following areas:
• Reliable position tracking devices that self-calibrate for head orientation of head-worn sensors.
• Ultralight, ultrabright, ultra-transparent display eyewear with wide field of view.
• Three-dimensional viewers with battlefield terrain visualization, incorporating real-time data from unmanned aerial vehicles, etc.
In the mid-term, R&D activities are focusing on:
• Manned vehicles with sensors and processing capabilities for moving autonomously, tasked for Soldier protection.
• Robotic assets, tele-operated, semi-autonomous, or autonomous and imbued with intelligence, with limbs that can keep pace with Soldiers and act as teammates.
• Enhanced reconnaissance through deep-penetration mapping of building layouts, cyber activity, and subterranean infrastructure.
Once AR and MR prototypes and systems have seen widespread use, the far term focus will be on automation that could track and react to a Soldier’s changing situation by tailoring the augmentation the Soldier receives and by coordinating across the unit.
In addition, AR and MR will revolutionize training, empowering Soldiers to train as they fight. Soldiers will be able to use real-time sensor data from unmanned aerial vehicles to visualize battlefield terrain with geographic awareness of roads, buildings, and other structures before conducting their missions. They will be able to rehearse courses of action and analyze them before execution to improve situational awareness. AR and MR are increasingly valuable aids to tactical training in preparation for combat in complex and congested environments.
AR and MR are the critical elements required for integrated sensor systems to become truly operational and support Soldiers’ needs in complex environments. Solving the challenge of how and where to use AR and MR will enable the military to get full value from its investments in complex integrated sensor systems.
For more information on how the convergence of technologies will enhance Soldiers on future battlefields, see:
– The discussion on advanced decision-making in An Advanced Engagement Battlespace: Tactical, Operational and Strategic Implications for the Future Operational Environment, published by our colleagues at Small Wars Journal.
– Dr. James Canton’s presentation from the Mad Scientist Robotics, Artificial Intelligence, & Autonomy Conference at Georgia Tech Research Institute last March.
– Dr. Rob Smith’s Mad Scientist Speaker Series presentation on Operationalizing Big Data, where he addresses the applicability of AR to sports and games training as an analogy to combat training (noting “Serious sport is war minus the shooting” — George Orwell).
Dr. Richard Nabors is Associate Director for Strategic Planning, US Army CERDEC Night Vision and Electronic Sensors Directorate.
(Editor’s Note: Mad Scientist successfully facilitated the Bio Convergence and Soldier 2050 Conference on 8-9 March 2018 with our co-sponsor, SRI International, at their Silicon Valley campus in Menlo Park, California. With over 400 live and virtual participants, our first West Coast conference brought together World class expertise in biology and the tech convergences that will have a significant impact on the changing character of future conflict.)
This will be accomplished through improved cognitive and physical skills, as well as maintaining the critical role of human judgement with the 
1) 
offer unprecedented advances in medicine and the neurosciences, but also major vulnerabilities to Soldiers and the homeland. With advanced biological technology evolution comes a host of moral challenges, security vulnerabilities, and new threat vectors. In the future, protecting one’s genomic information will require safeguards similar to how we currently protect our digital identities. We will be more vulnerable to advanced bioweapons and information warfare available to states and non-state organizations. 
3) Customization: Advances in biology offer much greater customization in medicine which could improve how quickly our Soldiers learn and how they handle stress and anxiety associated with combat zones. Human 2.0 will have direct Warfighter applications, providing Soldiers with sensory enhancements, human-machine teaming, brains plugged into the Internet of Battle Things (IoBT), and uploadable / downloadable memories. Customization of battlefield medical care will be enabled by advanced diagnostics worn by Soldiers (uniforms and equipment) and eventually embedded. 
In other countries, we can expect to see the customization of humans with genome editing children to increase height, improve intelligence, and expand creativity. 
We will compete with states, non-state groups, and 
5)