Mad Scientist Laboratory

[Editor’s Note:  In today’s post, returning guest bloggers Mr. Joseph DeFranco, CAPT (USN – Ret.) L. R. Bremseth , and Dr. James Giordano examine how China is leveraging neuroscience and technology (neuroS/T) as a soft weapon to assume hegemonic advantage in their ascendancy to superpower status.  The stage is set — the U.S. and the West must decide if and how they will compete with China in the overall S/T arena!]

Brain trust (n): Group of official or unofficial advisers concerned especially with planning and strategy. — Merriam-Webster Dictionary

Significant developments in neuroscience and technology (neuroS/T) are employable in warfare, intelligence, and national security (WINS) operations.¹  As has been shown, these tools and methods are certainly viable for use in kinetic warfare;²  however, we believe that it is far more feasible, facile – and therefore of greater value – to consider and pursue the brain sciences for producing “mass disruption” effects in non-kinetic engagements.³  Weaponry (i.e., means of contending against others) can be generally categorized as “Hard” and “Soft.” “Hard” weaponized applications of neuroS/T include pharmacological agents, microbes, organic toxins, and devices (i.e., “drugs, bugs, toxins, and tech”). Research, development, and use of these weapons are regulated by current international conventions and treaties, at least to some extent;⁴  and the scope and limitations of these treaties remain a focus of international discussion, contention, and debate. But it is equally important to acknowledge the capability that can be leveraged by employing forms of neuroS/T as “soft” weapons, to influence multinational, if not global economic, social, and political stability as well as balances of power.

Moreover, with the growth of non-western countries’ (e.g., China’s) interests, investments, and activities in neuroS/T, it is important to note that differing cultural (and political) needs, values, philosophies, norms, and mores can and often do affect the ethical codes that guide and govern the conduct of scientific research. In some cases, these differing ethical standards may create opportunistic windows to expedite neuroS/T research, and advance outcomes and products to ultimately effect global markets. China has recognized the technical, social, medical, military, and political value of neuroS/T, prompting the fortification of current programs, and initiation of new programs in brain sciences that are aimed at broad translational use(s).⁵  Toward such ends, China has both stated intent and capability to use precision disruptive methods to target competitors’ vulnerabilities to incur multi-dimensional ripple effects to influence various spheres of economic, social and geo-political power.⁶

As we have stated in a previous blog and elsewhere,⁷  China operates (1) on longer, more protracted timetables (i.e., Five-Year Plans [FYPs]), and (2) with adept synergy via cooperative engagement of government, academia, and commercial sectors (i.e., the “triple helix”) that allows for the centralization and coordination of resources and personnel on agenda, and projects of prime national interest. These aspects of their regime enable coordination and productivity of politically motivated intent, research, and outcomes for a range of applications (including WINS operations).⁸  In essence, China has established a reciprocally supportive brain trust focusing upon advancing enterprises in brain science. We have identified three ways that China looks to non-kinetically leverage neuroS/T as a soft weapon on the world stage:  (1) research tourism; (2) control of intellectual property; and (3) medical tourism. In these ways, China seeks to advance neuroS/T, as constituent to broader S/T, and economic initiatives to assume a hegemonic advantage and become a (if not the) global superpower.

Research Tourism

Research tourism is a strategy to attract both distinguished, experienced scientists (in most cases from Western countries) and younger scientists to contribute to and advance productivity, innovation, and prestige of China’s S/T (and other academic) enterprises. These efforts are evidenced in initiatives such as the Thousand Talents Program (launched in 2008) and other programs (e.g., Hundred Person Program, Spring Light Program, Youth Thousand Talents Program, etc.) that aim draw foreign researchers, incubate domestic talent, and incentivize the return and retention of Chinese scientists who have studied and/or worked abroad.^{9, 10}  Such programs are supported and enthused by broad intramural cooperation of leading neurobiological research universities, and initiatives to formally (and financially) conjoin Chinese institutes to leading programs worldwide. For example, the IDG/McGovern Institute for Brain Research at Tsinghua University in Beijing works closely with Massachusetts Institute of Technology’s brain research institute.¹¹  Additionally, in 2019, Hong Kong University of Science and Technology joined with Harvard Medical School, Stanford University School of Medicine, and University College London to engage in cooperative and collective neurodegenerative research and intend to open an institute in Hong Kong.¹²  These partnerships foster international collaboration to further therapeutics and other neuroS/T advances; however, it also augments China’s ability to even the proverbial “playing field” of the brain sciences for WINS applications.

Intellectual Property Control

China has also leveraged intellectual property (IP) policy and law to advance (and veil) neuroS/T and other biotechnologies through: (1) exploiting their own patent process; (2) enabling compulsory licensing under their IP and patent laws; and (3) internationally enforcing their patent and IP rights.  China’s system creates a “patent thicket” which, unlike the United States, emphasizes the end-utility of a concept rather than innovation in ideas. This in turn produces an abundance of patents, which are based on parts of other concepts or previous, completed patents. These IP laws allow China’s commercial entities to copy, or in some case, usurp foreign patent information, applications, and products. Furthermore, Chinese patent laws allow for complete seizure of international research and development (R&D) under certain conditions, which often remain vague (e.g., “for the benefit of public health”, “a major technological advancement”, etc.).¹³  A 2017 report by the Commission on the Theft of American Intellectual Property stated that China accounts for 87% of counterfeit goods stolen from United States companies.¹⁴  Taken together, such practices allow the Chinese government to acquire S/T for any political, economic, or WINS application.

The recent global expansion of China’s commercial, academic, and economic institutions has established significant stakes in myriad international enterprises that are capable of realizing rapid and broad advances in China’s S/T R&D. Such use of law (i.e., what is referred to as “lawfare”) empowers Chinese academic and corporate endeavors via economic and legal support provided by the PRC to align S/T R&D with explicit national directives and agendas to exercise international effect and dominance.^{15, 16} This can create market saturation of key products that are essential to multiple functions and domains of other countries’ infrastructure, economy, stability, and standing. As well, such products can be used for intelligence purposes to track particular domains and activities of individuals, groups, and communities. The proliferate infiltration of such products also establishes dependence upon China’s supply and maintenance of resources and renders these technologies (and the activities they serve) vulnerable to manipulation and control.

Medical Tourism

The capacity to advance and/or develop areas of biomedicine / biotechnology in ways that are not feasible elsewhere synergizes both a strong translational (“bench to bedside”) capability, and explicit as well as tacit attraction and solicitation of international individuals seeking interventions that are only available in China. These, at present, could range from the relatively sublime (e.g., using deep brain stimulation to treat drug addiction¹⁷) to the seemingly science fictional (e.g., the proposed body to head transplant to be conducted at Harbin Medical University in collaboration with Italian neurosurgeon Sergio Canavero¹⁸). Such enterprise may be fortified by research efforts that seek to expedite development of treatments for diseases for high global import and effect. More provocatively, research could also be directed toward producing agents that incur global effect, which could only be ameliorated through the use of specific interventions that are “made in China.”¹⁹  Initiatives to foster medical tourism are synergized by programs to promulgate China’s S/T in world markets, thereby strengthening multinational dependence on Chinese market influence. This enables reliance on products and services “made/delivered in China” in addition for those “made by China” for ubiquitous use elsewhere.²⁰

Prompting the Questions…

Due to these tactics of economic infiltration and saturation, China can create power hierarchies that induce ‘bio-political’ strategically latent effects that influence real and perceived positional dominance and affect world order. Hence, the United States and its allies must:

(1) Recognize the reality – and gain insight to processes/mechanisms – of China’s ascendant S/T capability;

(2) Evaluate what current and near-term trends in S/T portend for global position, influence, and power; and

(3) Decide which option – and paths-to-effect – to accept and assume.

In sum, it prompts the questions:

– Should the U.S. and its allies be content to let China continue to rise in S/T capability, and become a co-superpower, thereby rivaling if not surpassing the global influence of the U.S. and the West?

Or…

– Should China’s current and planned activities in S/T prompt the U.S. and its allies to adopt renewed, new – or perhaps China’s own – strategies to increase investment, innovation, and enterprise to maintain sole superpower status?

If you enjoyed this post, please also see:

BrAIn Gain > BrAIn Drain: Strategic Competition for Intellect

Designer Genes: Made in China? by Mr. Joseph DeFranco and Dr. James Giordano

The Importance of Integrative Science/Technology Intelligence (InS/TINT) to the Prediction of Future Vistas of Emerging Threats by Dr. James Giordano,  CAPT (USN – Ret.) L. R. Bremseth, and Joseph DeFranco

China’s Drive for Innovation Dominance

Authors:

Joseph DeFranco is J5 Donovan Group Fellow in Biowarfare and Biosecurity, at U.S. Special Operations Command (USSOCOM). He is currently studying neuroscience in the College of Arts and Sciences, and biodefense at the Schar School of Policy and Government of George Mason University, VA, and formerly served on the staff of Congressman Donald S. Beyer (VA-08). His current research focuses upon the possible use of novel microbiological agents and big data as force-multiplying elements in non-kinetic, hybrid, and kinetic engagements, and the role of global agencies in biosecurity.

L.R. Bremseth (CAPT, USN SEAL [Ret]) serves as the Senior Special Operations Advisor for CSCI, a strategic support organization in Springfield, VA. He previously served as the Deputy Senior Director of the Integration Support Directorate (ISD) for the Department of the Navy (DON). As such, he was a key advisor to the Secretary, Under Secretary and Deputy Under Secretary of the Navy for sensitive activities. CAPT Bremseth was appointed to the Defense Intelligence Senior Level, and Director, Operations and Executive Director prior to his appointment as Deputy Senior Director, ISD. He retired from the Navy in 2006 with 29 years of service, during which he commanded SEAL Team EIGHT (1996-1998) and served a major command tour at Naval Special Warfare Group THREE (2003-2005).

 Mad Scientist James Giordano, PhD, is Professor of Neurology and Biochemistry, Chief of the Neuroethics Studies Program, and Co-Director of the O’Neill-Pellegrino Program in Brain Science and Global Law and Policy at Georgetown University Medical Center. He currently serves as J5 Donovan Group Senior Fellow, Biowarfare and Biosecurity, U.S. Special Operations Command (USSOCOM), and as an appointed member of the Neuroethics, Legal, and Social Issues (NELSI) Advisory Panel of the Defense Advanced Research Projects Agency (DARPA). Previously, Dr. Giordano served as Senior Science Advisory Fellow of the Strategic Multilayer Assessment Group of the Joint Staff of the Pentagon; and was Senior Research Fellow and Task Leader for the EU Human Brain Project Subproject on Dual Use Brain Science.

Disclaimer:  The views expressed in this blog are those of the authors, and do not necessarily reflect those of the Department of Defense, U.S. Special Operations Command, the Defense Advanced Research Projects Agency (DARPA), Department of the Army, Army Futures Command (AFC), or Training and Doctrine Command (TRADOC).

Acknowledgments:
This blog was adapted from the authors’ forthcoming work appearing in the Health Security journal, Strategic Studies Quarterly, and the Cambridge Quarterly of Healthcare Ethics. JG’s work is supported in part by funding from CSCI and Leadership Initiatives.

References:

---

¹ For additional resources, please see:

Moreno J.D. (2006). Mind wars: Brain research and national defense.

Flower, R., Dando, M., Hay, A., Iverson, S., Robbins, T., Robinson, J. P., Rose, S., Stirling, A., Tracey, I., & Wessely, S. (2012). Brain Waves Module 3: Neuroscience, conflict and security.

Giordano J, Forsythe C, Olds J. Neuroscience, neurotechnology and national security: The need for preparedness and an ethics of responsible action. AJOB-Neuroscience 1(2): 1-3 (2010).

Forsythe C, Giordano J. On the need for neurotechnology in the national intelligence and defense agenda: Scope and trajectory. Synesis: A Journal of Science, Technology, Ethics and Policy 2(1): T5-8 (2011).

² Giordano J. (ed.) Neurotechnology in National Security and Defense: Practical Considerations Neuroethical Concerns. Boca Raton: CRC Press (2015).

³ DeFranco J, DiEuliis D, Bremseth LR, Snow JJ, Giordano J (2019). Emerging Technologies for Disruptive Effects in Non-Kinetic Engagements. Journal of the Homeland Defense & Security Information Analysis Center, 6(2).

⁴ Gerstein, D., & Giordano, J. (2017). Rethinking the Biological and Toxin Weapons Convention?. Health security, 15(6), 638-641.

⁵ DeFranco J, Bremseth LR, Giordano J. The Importance of Integrative Science/Technology Intelligence (InS/TINT) to the Prediction of Future Vistas of Emerging Threats. Mad Scientist Laboratory Post #125, 13. March 2019. Available online at: https://madsciblog.tradoc.army.mil/125-the-importance-of-integrative-science-technology-intelligence-ins-tint-to-the-prediction-of-future-vistas-of-emerging-threats/

⁶ For example, please see: Bremseth L.R. & Giordano, J. (2019, July 4). The undeclared war America is losing. Retrieved from https://www.washingtontimes.com/news/2019/jul/4/why-fentanyl-must-be-designated-a-weapon-of-mass-d/

⁷ Chen C, Andriola J, Giordano J. (2018). Biotechnology, commercial veiling and implications for strategic latency: The exemplar of neuroscience and neurotechnology research and development in China. In: Davis ZD, Nacht M. (eds.) Strategic Latency Red, White and Blue: Managing the National and International Security Consequences of Disruptive Technologies. Livermore, CA: Lawrence Livermore Press, pp. 12-32.

⁸ DeFranco, J, Bremseth LR, Giordano J (2019). Dual- and Non-kinetic Use of Chinese Brain Science: Current Activities and Future Implications. In: Peterson N (ed.) The Future of Global Competition and Conflict. A Strategic Multi-Layer (SMA) Periodic Publication. Washington, DC: Strategic Multilayer Assessment Office, Office of the Secretary of Defense.

⁹ Engel, R. & Werner, K. (2019, July 14). China’s rising tech scene threatens U.S. brain drain as ‘sea turtles’ return home. Retrieved from https://www.nbcnews.com/tech/tech-news/china-s-rising-tech-scene-threatens-u-s-brain-drain-n1029256

¹⁰ Ibid, ref. 7.

¹¹ For information on the IDG/McGovern Institute for Brain Research at Tsinghua University, please see: http://mcgovern.med.tsinghua.edu.cn/

¹² Yan. (2019, January 16). HK to set up neuroscience research center with world’s top universities to tackle aging population. Retrieved from http://www.xinhuanet.com/english/2019-01/16/c_137749495.htm

¹³ Ibid, ref. 7.

¹⁴ For the complete 2017 update of the IP Commission report on the Theft of American Intellectual Property, please see: http://ipcommission.org/report/IP_Commission_Report_Update_2017.pdf

¹⁵ Ibid, ref. 8.

¹⁶ McFate S. (2019). The New Rules of War: Victory in the Age of Durable Disorder. NY: Harper-Collins.

¹⁷ ABC News. (2019, May 8). China conducting world-first trial of brain implants to treat drug addiction. Retrieved from https://www.abc.net.au/news/2019-05-08/china-trials-brain-implants-to-treat-drug-addiction/11090936

¹⁸ Jacobson, R. (2018, May 15). Two surgeons in China developing a method to transplant a human head. Retrieved from https://www.cnbc.com/2018/05/15/two-surgeons-in-china-developing-a-method-to-transplant-a-human-head.html

¹⁹ DeFranco J, Snow JJ, Giordano J. Dead Deer, and Mad Cows, and Humans (?) … Oh My! U.S. Army Training and Doctrine Command Mad Scientist Laboratory Post #143, 13. May 2019. Available online at: https://madsciblog.tradoc.army.mil/143-dead-deer-and-mad-cows-and-humans-oh-my/

²⁰ For the United States Chamber of Commerce report on Made in China 2025, please see: https://www.uschamber.com/sites/default/files/final_made_in_china_2025_report_full.pdf

[Editor’s Note:  Returning guest bloggers Mr. Joseph DeFranco and Dr. James Giordano explore the emerging geo-political ramifications and associated ethical implications of recent advances in genetic engineering.  Read on to learn how China is currently positioned to drive the evolution of this brave new world!]

Gene editing (i.e., directly modifying an organism’s genetic material to achieve a desired effect and outcome) has been used as a viable technique for several decades. It has been intended and employed for treating a variety of conditions, including immunodeficiency and blood disorders, and certain types of cancers. Recently, gene editing methods were augmented by the discovery of Clustered Regularly Interspaced Short Palindromic Repeats (CRISPR) and its associated Cas9 nuclease. CRISPR techniques enable a more simplified and rapid approach to altering cellular DNA, and thus modifying genetic information. Despite the relative ease afforded by CRISPR/Cas9, the method is not without problems, including the production of mutations by incurring “off-target” genetic edits. Yet, such issues and limitations appear to be little more than proverbial “speed bumps” in the paths to broadening capabilities afforded by what is likely a game-changing tool in and for molecular biology – and perhaps the production of bioweapons and instruments of non-kinetic disruption, as well.¹

In November 2018, Chinese Scientist Dr. He Jiankui claimed to have used CRISPR/Cas9 to edit the CCR5 gene of human embryos to generate an inherited resistance to HIV, smallpox, and cholera.²  Although it appears that this genetic modification will create a resistance to certain types of infections, altering CCR5 expression may have other phenotypic effects, as well. For example, studies by Joy and colleagues showed that the deletion of the CCR5 gene may affect memory; and therefore this specific gene edit may have modified the cognitive functions of these “CRISPR babies.”³  Whether this mutation was intended or not, off-target mutations should be expected until certain extant gaps in knowledge of genomics, proteomics, and neuroscience are understood and bridged. A report published in Nature Medicine suggests that this gene edit could detrimentally affect the normal lifespan of the gene edited babies,⁴  in part by making individuals more susceptible to other viral infections (e.g., influenza virus⁵  and West Nile virus⁶ ).

Of course, a number of questions arise from He’s work. First is whether embryos should be edited, and if so, is enough known about the novel CRISPR-Cas9 method to employ it in approaches to modifying human genetics. A complete discussion of these ethico-legal questions is beyond the scope of this blog but are addressed elsewhere in some detail.⁷  Second, what opportunistic circumstances led to, or allowed for, the practice of what many are deeming to be “ethically fuzzy” science? To be sure, international standards for research (e.g., The Nuremberg Code⁸ ; etc.⁹ ) have been recognized by the scientific community. But the needs, values, and precepts that contribute to and/or constitute ethical principles are not necessarily homogeneous across cultures. Thus, as cutting-edged bioscientific research becomes an ever-more international enterprise, distinctions in cultural norms, mores, and practices can – and increasingly will – be placed in stark relief.

There may have been some attempt to gauge the international reaction to the use of CRISPR/Cas9 technologies to modify human embryos. After the initial pushback from several nations, the Chinese government declared that Dr. He was a “rogue” scientist. This we find difficult to believe. China’s seamless triple helix of government, academia, and industry enable purposive synchronization and synergy of initiatives,¹⁰  which allows rapid resource and personnel deployment for national directives. The confirmation of a second Chinese CRISPR pregnancy in early 2019¹¹  only adds to our disbelief – and non-acceptance – that such efforts are the work of rogue scientists who are operating without the full knowledge and endorsement of the Chinese government.

Given that CRISPR/Cas9, along with other gene editing tools, can be used for disruptive purposes (e.g., development of “designer pathogens” not currently listed/governed by international bioweapon treaties and conventions; alteration of human structure and functions; modification of agricultural systems, etc.), we posit that such advancements can pose defined risks and threats to global security.¹²  In March 2019, the World Health Organization’s newly formed advisory committee for international governance on human genome editing determined that such modifications to human germlines are “irresponsible”.¹³  The committee proposed the need for a central registry of human genome editing research in order to facilitate more detailed insight to – and stringent and oversight of – risks and hazards.

The recognition and acknowledgment that CRISPR methods could be used to generate novel biological weapons have prompted studies of reversing CRISPR-induced effects. A recent article in the MIT Technology Review identified laboratories that are working to find “anti-CRISPR” molecules: proteins in nature that can “turn off” CRISPR-induced gene edits.¹⁴  Such efforts reflect steps to control open source research and counter the potential use of gene-editing to produce biological weapons and/or affect global health, and this is noteworthy. However, such regulation may not be encompassing or sufficient. Of growing concern in this light are clandestine enterprises (i.e., biohacking) and research activities of nation-states and/or non-state actors that blatantly disregard international standards and guidelines. In June 2019, a Russian scientist declared plans to implant gene-edited embryos into women.¹⁵  Clearly, this announcement comes after numerous appeals for an international moratorium on germline editing;¹⁶  thereby reinforcing the reality that CRISPR-based methods are relatively easy to develop and use, but not necessarily easy to regulate, govern, and control.

While the use of gene editing techniques to create lethal and/or destructive agents is certainly possible, we believe that it is far more feasible – and perhaps more effective and efficient – to pursue the use of gene-edited agents in “mass disruptive” non-kinetic engagements. Such precision disruption can exploit a society’s vulnerabilities, and generate ripple effects to unsettle various domains and dimensions of geo-political stability and power. For example, allowing human use of gene editing techniques could foster attraction of both researchers seeking to advance potential translational applications of genetic methods (i.e., “research tourism”), and patients seeking newly developed gene-editing interventions (i.e., “medical tourism”) for the treatment of certain diseases as well as genetic modification for “wellness” and/or enhancement. Moreover, the use of gene editing techniques to modify crops, livestock, and to change environmental flora and fauna can incur equally disruptive effects on global ecologies, markets, and economics.

Without doubt, the economic aspects and advantages of global hegemony of bioscience, biotechnology, and biomedicine are focal to China’s ongoing strategic plan.¹⁷  The Wall Street Journal reported that China spent the equivalent of approximately $410 billion (USD) in research and development (R&D) in 2016.¹⁸  This is in contrast to the United States’ R&D investment, which allocated only 1.7% ($66.5 billion USD) of the federal budget in 2017. Additionally, China has increased educational programs in order to realize the intent to become the global leader in science, technology, engineering, and medicine (STEM) graduates by 2030. According to the World Economic Forum, China graduated almost five million STEM students from universities in 2016, which dwarfs the United States graduates in that year (around 1.8 million).¹⁹  As China takes more of an interest, and makes increasingly greater investments in the life sciences, it is apparent that this will be one of the new frontiers of economic and diplomatic engagement.²⁰

Given China’s stated intent to become a – if not the – global superpower within the next 30 years, a vital question is what kind(s) of engagement(s) will this prompt or demand. Any meaningful answer must appreciate the hegemonic views and objectives of China (and other strategically competitive nations) as well as those of the United States and its allies. Will the U.S. be content to become “a superpower among superpowers,” and if so, what does this portend for activities of preparation, cooperation, and collaboration? Or, will there be a durable struggle for global power supremacy (in and among the many domains), and what does this infer or establish for competition, combativeness, or (some sort of) capitulation? These decisions are crucial as we determine and articulate programs and policies of scientific, technological, and socio-political investment and activity for the decade(s) to come. Quoting the late Lee Iacocca, we ask:  who will “…lead, follow, or get out of the way”²¹  and what will such postures entail and obtain?

If you enjoyed reading today’s post, please also see the following blog posts:

Dead Deer, and Mad Cows, and Humans (?) … Oh My! by LtCol Jennifer Snow, Dr. James Giordano, and Joseph DeFranco

The Importance of Integrative Science/Technology Intelligence (InS/TINT) to the Prediction of Future Vistas of Emerging Threats by Dr. James Giordano,  CAPT (USN – Ret.) L. R. Bremseth, and Joseph DeFranco

Benefits, Vulnerabilities, and the Ethics of Soldier Enhancement

China’s Drive for Innovation Dominance

… as well as Hank Greely‘s presentation on Future Legal and Ethical Implications of Bio Technology from the Mad Scientist Bio Convergence and Soldier 2050 Conference, 8-9 March 2019 in Menlo Park, CA

Authors:

Joseph DeFranco was J5 Donovan Group Fellow in Biowarfare and Biosecurity, at U.S. Special Operations Command (USSOCOM). He is currently studying neuroscience in the College of Arts and Sciences, and biodefense at the Schar School of Policy and Government of George Mason University, VA, and formerly served on the staff of Congressman Donald S. Beyer (VA-08). His current research focuses upon the possible use of novel microbiological agents and big data as force-multiplying elements in non-kinetic, hybrid, and kinetic engagements, and the role of global agencies in biosecurity.

Mad Scientist James Giordano, PhD, is Professor of Neurology and Biochemistry, Chief of the Neuroethics Studies Program, and Co-Director of the O’Neill-Pellegrino Program in Brain Science and Global Law and Policy at Georgetown University Medical Center. He currently serves as an appointed member of the Neuroethics, Legal, and Social Issues (NELSI) Advisory Panel of the Defense Advanced Research Projects Agency (DARPA). Dr. Giordano previously served as J5 Donovan Group Senior Fellow, Biowarfare and Biosecurity, U.S. Special Operations Command (USSOCOM); Senior Science Advisory Fellow of the Strategic Multilayer Assessment Group of the Joint Staff of the Pentagon; and Senior Research Fellow and Task Leader for the EU Human Brain Project Subproject on Dual Use Brain Science.

Disclaimer:  The views expressed in this blog are those of the authors, and do not necessarily reflect those of the Department of Defense, U.S. Special Operations Command, the Defense Advanced Research Projects Agency (DARPA), Department of the Army, Army Futures Command (AFC), or Training and Doctrine Command (TRADOC).

Acknowledgments:  This blog was adapted from the authors’ forthcoming work appearing in the PRISM journal and an interview (of JG) conducted by Stew Magnussen for National Defense Magazine (https://www.nationaldefensemagazine.org/articles/2019/3/26/editors-notes-national-security-implications-of-gene-editing). JG’s work is supported in part by funding from CSCI, Leadership Initiatives, and federal funds UL1TR001409 from the National Center for Advancing Translational Sciences (NCATS), National Institutes of Health, through the Clinical and Translational Science Awards Program (CTSA), a trademark of the Department of Health and Human Services, part of the Roadmap Initiative, “Re-Engineering the Clinical Research Enterprise.”

References:

---

¹ DeFranco J, DiEuliis D, Bremseth LR, Snow JJ, Giordano J (2019). Emerging Technologies for Disruptive Effects in Non-Kinetic Engagements. Journal of the Homeland Defense & Security Information Analysis Center, 6(2).

² Regalado, A. (2018, November 26). Chinese scientists are creating CRISPR babies. Retrieved from https://www.technologyreview.com/s/612458/exclusive-chinese-scientists-are-creating-crispr-babies/
Kolata, G., Wee, S. L., & Belluck, P. (2018, November 26). Chinese Scientist Claims to Use Crispr to Make First Genetically Edited Babies. Retrieved from https://www.nytimes.com/2018/11/26/health/gene-editing-babies-china.html

³ Joy, M. T., Assayag, E. B., Shabashov-Stone, D., Liraz-Zaltsman, S., Mazzitelli, J., Arenas, M., Abduljawad, N., Kliper, E., Korczyn, A. D., Thareja, N. S., Kesner, E. L., Zhou, M., Huang, S., Silva, T. K., Katz, N., Bornstein, N. M., Silva, A. J., Shohami, E., & Carmichael, S. T. (2019). CCR5 is a therapeutic target for recovery after stroke and traumatic brain injury. Cell, 176(5), 1143-1157.
Regalado, A. (2019, February 21). China’s CRISPR twins might have had their brain inadvertently enhanced. Retrieved from https://www.technologyreview.com/s/612997/the-crispr-twins-had-their-brains-altered/

⁴ Wei, X., & Nielsen, R. (2019). CCR5-∆ 32 is deleterious in the homozygous state in humans. Nature medicine, 1.

⁵ Falcon, A., Cuevas, M. T., Rodriguez-Frandsen, A., Reyes, N., Pozo, F., Moreno, S., Ledesma, J., Martínez-Alarcón, J., Nieto, A., & Casas, I. (2015). CCR5 deficiency predisposes to fatal outcome in influenza virus infection. Journal of General Virology, 96(8), 2074-2078.

⁶ Glass, W. G., McDermott, D. H., Lim, J. K., Lekhong, S., Yu, S. F., Frank, W. A., Pape, J., Cheshier, R. C., & Murphy, P. M. (2006). CCR5 deficiency increases risk of symptomatic West Nile virus infection. Journal of Experimental Medicine, 203(1), 35-40.

⁷ For sources which discuss the ethico-legal questions of CRISPR and gene editing, please see: Brokowski, C., & Adli, M. (2019). CRISPR ethics: moral considerations for applications of a powerful tool. Journal of molecular biology, 431(1), 88-101.
Reyes, A. P., & Lanner, F. (2017). Towards a CRISPR view of early human development: applications, limitations and ethical concerns of genome editing in human embryos. Development, 144(1), 3-7.
Caplan, A. L., Parent, B., Shen, M., & Plunkett, C. (2015). No time to waste—the ethical challenges created by CRISPR. EMBO reports, 16(11), 1421-1426.

⁸ For more information on the Nuremberg Code, please see: https://history.nih.gov/research/downloads/nuremberg.pdf
& https://www.loc.gov/rr/frd/Military_Law/pdf/NT_war-criminals_Vol-II.pdf

⁹ For more examples of international standards for research, please see:
The Belmont Report: https://www.hhs.gov/ohrp/regulations-and-policy/belmont-report/index.html
United States Department of Health & Human Services, Responsible Conduct of Research: https://ori.hhs.gov/ori-introduction-responsible-conduct-research
World Health Organization, Ethical standards and procedures for research with human beings: https://www.who.int/ethics/research/en/

¹⁰ Ibid. ref. 1.

¹¹ O’Callaghan, J. (2019, January 24). Chinese officials circle as a second CRISPR pregnancy is confirmed. Retrieved from https://www.wired.co.uk/article/second-crispr-baby

¹² DeFranco J, Bremseth LR, Giordano J. The Importance of Integrative Science/Technology Intelligence (InS/TINT) to the Prediction of Future Vistas of Emerging Threats. Mad Scientist Laboratory Post #125, 13. March 2019. Available online at: https://madsciblog.tradoc.army.mil/125-the-importance-of-integrative-science-technology-intelligence-ins-tint-to-the-prediction-of-future-vistas-of-emerging-threats/

¹³ WHO expert panel paves way for strong international governance on human genome editing (2019, March 19). Retrieved from https://www.who.int/news-room/detail/19-03-2019-who-expert-panel-paves-way-for-strong-international-governance-on-human-genome-editing

¹⁴ Regalado, A. (2019, May 2). The search for the kryptonite that can stop CRISPR. Retrieved from https://www.technologyreview.com/s/613309/the-search-for-the-kryptonite-that-can-stop-crispr/

¹⁵ Cyranoski, D. (2019). Russian biologist plans more CRISPR-edited babies. Nature, 570(7760), 145.

¹⁶ Wolinetz, C. D., & Collins, F. S. (2019). NIH supports call for moratorium on clinical uses of germline gene editing.

¹⁷ Chen C, Andriola J, Giordano J. (2018). Biotechnology, commercial veiling and implications for strategic latency: The exemplar of neuroscience and neurotechnology research and development in China. In: Davis ZD, Nacht M. (eds.) Strategic Latency Red, White and Blue: Managing the National and International Security Consequences of Disruptive Technologies. Livermore, CA: Lawrence Livermore Press, pp. 12-32.

¹⁸ Emanuel, E., Gadsden, A., & Moore, S. (2019, April 19) How the U.S. Surrendered to China on Scientific Research. Retrieved from https://www.wsj.com/amp/articles/how-the-u-s-surrendered-to-china-on-scientific-research-11555666200

¹⁹ The World Economic Forum’s 2016 Human Capital Report: http://reports.weforum.org/human-capital-report-2016/measuring-human-capital/?doing_wp_cron=1486038808.8636078834533691406250

²⁰ Gehrke, J. (2019, July 2). Genetic information is the newest front in US-China great power competition. Retrieved from https://www.washingtonexaminer.com/policy/defense-national-security/genetic-information-is-the-newest-front-in-u-s-china-great-power-competition

²¹ Woodruff, D. (1992, August 24). Mr. Iacocca, Please Go Graciously. Retrieved from https://www.bloomberg.com/news/articles/1992-08-23/mr-dot-iacocca-please-go-graciously

[Editor’s Note: In today’s post, proclaimed Mad Scientist and returning guest blogger Ms. Marie Murphy addresses the implications of weaponized Artificial Intelligence (AI) when employed in information operations.  Truth and trust are the first causalities of this perception-altering capability!]

Artificial Intelligence (AI) is a tool that effectively enables machine-speed actions, posing new threats never before seen over the course of history. This disruptive capability can both erase existing data and create entirely false realities. If nefarious actors are able to effectively harness AI, the Army may potentially operate in a manufactured information environment with altered data guiding its decision-making. Using AI in this way is a competition-phase tactic that affects the will of people and deteriorates their trust, both within and about the Army.

Historical Precedence

The concept of altering or removing information from public memory has been around since the time of Ancient Rome. Damnatio memoriae is the contemporary term for the condemning of the memory of an emperor by the Senate posthumously.¹  The idea is “to erase a person as if he (or she) never existed.”²  The practice died out with the emerging prominence of the Catholic Church but resurfaced in Northern Italy during the Renaissance.³  A “damnation of memory” was ordered when a new government took power in Florence, purging the powerful citizens and families of the old regime from societal recollection.⁴  In the 20th century, at the height of the Soviet Union, Josef Stalin worked to remove any mention or figure of his political enemies from public view, to the point where they were edited out of textbooks and pictures.⁵

Today, history is repeating itself. China is attempting to remove all reference to the events that occurred in Tiananmen Square on 3-4 June 1989, and they have an advantage those in the past did not: AI. The absence of any public information about the events surrounding Tiananmen Square in contemporary China stems from the Chinese government’s efforts to censor any mention or insinuation of the incident online and in reality. AI works at machine-speed to filter internet search results and social media posts to assist human censors. ⁶ A recent study proved the effectiveness of this campaign, showing that a meager 15% of Beijing University students could accurately identify the “tank man” photograph, one of the internationally recognized symbols of the protest.⁷ While most people appear to be aware of the incident by the time they’re well into their 20s, there is a general apathy toward, or even sympathy with the government amongst the current generation regarding the 1989 protests. ⁸

Future Threat

AI can be weaponized in information and cyber warfare, which characterize the interactions between strategic competitors in the competition phase. However, it’s often discussed in the context of lethal autonomy, man-machine teaming, or practical battlefield applications. But what happens when AI begins to psychologically alter the operational environment by changing the perceptions of both military and civilian personnel alike? Through the deletion or alteration of the past or the creation of a fabricated present, AI is a tool which aids in and speeds up the process of identifying, changing, and generating information. False narratives based on AI-manipulated or -generated information and media can originate from anyone, anywhere; developing and spreading rapidly with a detrimental effect on trust. This deterioration of trust moves in two directions: The Soldier not trusting the information they are given which is crucial for decision-making and combat performance, and civilians and other militaries not trusting the actions of the U.S. Army because their perceptions have been affected by AI-influenced content.

If AI, employed by a nefarious actor, is able to manipulate the information available about the past and deliver false information about the present, there is likely to be a strong impact on Soldier and Commander trust of any information received. Altering history to the point that it is impossible to discern what the truth is, and potentially basing decisions on inaccurate information about the past, could have unintended and devastating consequences. AI manipulations can also affect trust because of their applicability in enabling small-scale, personalized warfare. A particular Soldier could be targeted with a computer-generated fake message from home that someone is very ill. This false information could cause a breakdown of trust and a rise in skepticism in the individual Soldier to the point that they begin to question every piece of information they’re given, wasting valuable time and energy.

The Army is also facing the challenge of AI manipulating the perceptions of military and civilian personnel support elements.  AI and those who program it can accomplish this by doctoring or deleting the records or social perceptions of prior Army actions. To create a false reality that corrupts the information environment, AI could be used to spread malicious rumors influencing international will and public opinion. One such rumor might be a manufactured atrocity. It would be incredibly hard to deny the alleged incident if there are audio and video recordings depicting it (even though that evidence is fabricated with the help of AI); the programmer could even use AI to create computer-generated people “on camera” giving false eye-witness accounts. If the public and other militaries whom the U.S. aligns with do not trust the Army, then operations will become strategically and tactically more challenging.

While AI may present the Army with many strategic and tactical benefits by providing machine-speed analysis and decision-making, it can also work counter to the Army’s goals and initiatives. With other entities such as China developing AI at a rapid pace and willing to deploy it against its own citizens, the time of AI-enabled damnatio memoriae has arrived. Whether it be altering the past or creating a false present, AI-generated information may greatly impact future operability and trust of warfighters.

If you enjoyed this post, please also see:

 Influence at Machine Speed: The Coming of AI-Powered Propaganda by MAJ Chris Telley

The Death of Authenticity: A New Era of Information Warfare

China’s Drive for Innovation Dominance

Proclaimed Mad Scientist Marie Murphy is a rising senior at The College of William and Mary in Virginia, studying International Relations and Arabic. She is a regular contributor to the Mad Scientist Laboratory, interned at Headquarters, U.S. Army Training and Doctrine Command (TRADOC) with the Mad Scientist Initiative last summer, and has returned as a consultant this summer.  She was a Research Fellow for William and Mary’s Project on International Peace and Security.

Disclaimer:  The views expressed in this article do not imply endorsement by the U.S. Army Training and Doctrine Command, the Army Futures Command, the U.S. Army, the Department of Defense, or the U.S. Government.  This piece is meant to be thought-provoking and does not reflect the current position of the U.S. Army.

---

¹ “Damnatio Memoriae.” Livius.org, updated June 23, 2019. https://www.livius.org/articles/concept/damnatio-memoriae/

² Petersen, Lauren Hackworth. “The Presence of ‘Damnatio Memoriae in Roman Art.” Notes in the History of Art, vol. 30, no. 2, 2011, p.1. JSTOR. www.jstor.org/stable/23208566

³ “Damnatio Memoriae.” Livius.org, updated June 23, 2019. https://www.livius.org/articles/concept/damnatio-memoriae/

⁴ Petersen, Lauren Hackworth. “The Presence of ‘Damnatio Memoriae in Roman Art.” Notes in the History of Art, vol. 30, no. 2, 2011, p.1. JSTOR. www.jstor.org/stable/23208566

⁵ Bond, Sarah. “How Do You Purge The Memory Of An Emperor.” Forbes.com, April 11, 2017. https://www.forbes.com/sites/drsarahbond/2017/04/11/how-do-you-damn-the-memory-of-a-roman-emperor/#49228baf49b2

⁶ Gilbert, David. “How China Is Wiping Memories of Tiananmen Square Off The Internet.” Vice News, June 4, 2019. https://news.vice.com/en_us/article/7xge3b/chinese-dissidents-are-running-out-of-ways-to-remember-tiananmen-square

⁷ Keng Kuek Ser, Kuang. “How China has censored words relating to the Tiananmen Square anniversary.” Pri.org, Hune 4, 2016. https://www.pri.org/stories/2016-06-03/how-china-has-censored-words-relating-tiananmen-square-anniversary

⁸ Fish, Eric. “Tiananmen Shaped China’s History. But Chinese Millennials Have Mixed Views About Its Legacy.” Time, June 3, 2019. https://time.com/5599060/china-millennials-tiananmen-anniversary/

[Editor’s Note: Today’s post by returning guest blogger Mr. Ian Sullivan examines the Battle of Agincourt and turns the heretofore orthodoxy of thought regarding its implications on its head.  Agincourt is traditionally viewed as an historical milestone in the democratization of warfare, with English and Welsh longbowmen triumphing over France’s numerically superior force of heavily armored mounted nobility.  Mr. Sullivan re-examines the battle across the arc of history and identifies three important lessons for the U.S. Army to consider as it seeks to modernize to prevail in Competition and win decisively in armed Conflict against peers or near peers.]

There is nothing more powerful than words that are well-written or well-uttered. They seep into our very consciousness, forming emotions, thoughts, and even images and concepts that are difficult to dispel. But sometimes, words mislead. Take William Shakespeare’s epic work on the famous Battle of Agincourt, found in Henry V. His language was masterful. Who has not felt lump in their throat and their eyes well up during the St. Crispin’s Day speech? “We few, we happy few, we band of brothers” resonates to this day as a motivational speech that neither George Patton nor Vince Lombardi could surpass. Who cannot fail to imagine the grizzled line of English longbowmen moving “once more unto the breach?” The Bard’s words have taken the story of Agincourt, where an outnumbered and desperate English force stood against the armed might of France and won, and immortalized their epic victory.

The popular understanding of Agincourt holds that a revolution in military affairs occurred that wet October day in 1415, when well-trained English yeomen archers armed with their formidable longbows—capable of ranging some 250 yards with rates of fire between 10-20 shots a minute—decimated the heavily armored mounted French knights who heretofore had dominated medieval battlefields. After a day of brutal fighting, the English force, estimated to be around 6,000-8,000 strong, outfought and essentially slaughtered the much larger French force, which was estimated around 25,000-35,000 men, including a large contingent of armored, mounted knights. The losses that day were astounding; the French suffered 7,000-10,000 dead, with another 1,500 nobles taken captive. The English suffered between 100 and 500 dead. The yeomen archer and his deadly longbow won a decisive victory.

Military theorists and historians point to Agincourt as a decisive moment in history that forever ended the dominance of the mounted knight on the battlefield. The poor yeomen archers proved ascendant, and the day of the armored knight was over. Agincourt, or an Agincourt moment, is held to this day as a symbol of the failure to adapt, for a failure to perceive change, and as a failure to accept that technology can quickly surpass the status quo. As recently as this April, writing for the New York Times, Bret Stephens published an opinion piece entitled “The US Military: Like the French at Agincourt?” making this claim.

In demonstrating the relevance of Agincourt today, the subtitle of his piece notes “America risks a catastrophic defeat if it doesn’t radically change the way it thinks about war,” and proclaims “the logic here is the same one that decided the Battle of Agincourt, where the humble and effective English longbow made short work of the expensive and vulnerable French cavalry.” His logic holds that the tragedy of Agincourt is compounded because the French should have known better.

And he is correct in that assessment. The longbow was not a new revelation at Agincourt, but instead was the ultimate pink flamingo; indeed, a previous generation of French knights were similarly tormented by English longbowmen in the first part of the Hundred Years’ War at the Battles of Crecy (1346) and Poitiers (1356). Stephens argues that the contemporary U.S. military, like the French before them, is pouring money into exquisite systems that will soon be made obsolete by other low-tech, simpler systems. He references $13 billion aircraft carriers and $90 million fighter aircraft—the French knights of our day—as vulnerable to disposable swarms of intelligent machines, today’s longbow.

This is a powerful image when linked to Agincourt and definitely is a train of thought that needs to be accounted for in terms of defense planning and in terms of designing a future joint force. However, this logic, which is widespread among many futurists, should have an asterisk by it, as it misses a key point, which is perhaps understandable in light of the powerful mythology of Agincourt fostered by the transcendent words of Shakespeare; that in spite of what happened at Agincourt, the FRENCH WON THE WAR.

The English victory at Agincourt, it turns out, was not really much of a victory. Henry V’s army was so weakened—by poor logistics and disease—that it actually was forced to withdraw from the field after winning such a major victory. By the end of the month, Henry had withdrawn to Calais, his invasion of Normandy halted. He was able to turn his success at Agincourt into the Treaty of Troyes, which named him as the heir to the French throne, but this proved to be nothing more than a temporary truce. The English failed to capitalize on the advantage the longbow gave them and instead squandered an opportunity to conclude the war successfully.

So how did the French prevail? I think there are three important lessons here that should be considered, particularly as the United States ponders the implications of a return to great power competition and conflict against peers or near peers.

The first is that there is a continuum between competition, conflict, and return to competition that is an essential aspect of great power competition. Few great power conflicts end in the total defeat of one or the other. The Hundred Years’ War is likely more emblematic of the types of conflicts that will be waged in the near future, in the Era of Accelerated Human Progress and the Era of Contested Equality. The Hundred Years War was really a combination of several separate wars linked together by the inability of either side to compel a decisive victory; a period of competition, conflict, and return to competition that lasted for 116 years.

Although the English won many battles in the first phase of the war, their most important achievement was not the tactical success of the longbow, but instead was their ability to use other elements of national power in the competition phase to surround the French with enemies on three sides (England, Aquitaine, and Burgundy). For its part, the French had cultivated Scotland as an ally, posing a real and imagined threat to the English, but to secure final victory, the French would need to split England’s allies on the continent.

This was only made possible through a series of military victories. The Treaty of Troyes only remained in effect for six years, when the French again resumed the fight. The English responded with a renewed invasion, and in 1428 they besieged the French city of Orleans.

At this point, the second essential element of the French victory revealed itself; better leadership. Although the English possessed a number of fine battle captains, it was the emergence of the most dynamic leader of the war, Joan of Arc, who made a significant difference. Her greatest feat did not lie in military organization, strategy, or tactics, but instead in an ability to inspire and unite the French. She arrived at Orleans just as the city was about to fall to the English and broke the siege. She then led a highly effective campaign in the Loire Valley which beat the English in several key fights, including at Patay, in June 1429.

Patay, an often forgotten battle, is relevant because it was the antithesis to Agincourt. At Patay, a force of armored French knights, not hampered by rain and mud as they were at Agincourt, charged into the ranks of the helpless English bowmen. They relied on better battlefield awareness, better intelligence, and well concealed movement to maneuver to a point where their knights could effectively charge. Some 2,500 English—mostly longbowmen—were killed for the loss of only 100 French. With this victory, the French demonstrated that strong leadership and an appreciation of their enemy’s capabilities could in fact overcome the tactical strengths wrought by the longbow.

The French successes in the Loire Valley led to the strengthening of the overall French position. In spite of Joan’s capture and execution, Charles VII was crowned king in 1429, and he was able to completely reverse the diplomatic situation. His most important success arguably came not on the battlefield, but with the 1435 Treaty of Arras, where he was able to cut a diplomatic deal with Burgundy, who accepted Charles’ claim to the throne and abandoned the English. This allowed the French to embark on a final campaign, which in turn revealed the third and final factor in France’s victory; an effective military modernization.

The French appeared to understand that the character of the war they were fighting was changing. It would be neither armed knights nor the English longbowmen that would determine final victory, but instead it would be a more professional, sophisticated, combined arms force equipped with effective artillery, that would finally prevail. The French reorganized their forces into Compagnies d’Ordonnance, combined arms units of heavy cavalry, archers, men-at-arms, and supporting units that consisted of about 600 men. By 1445, the French Army had 15 of these new formations, which gave them a more professional, well-organized force. Additionally, the French also quickly mastered and integrated the production of new and more sophisticated artillery into their force and were able to mass hundreds of guns to undertake sieges, the type of fight that would dominate the final phase of the war.

These organizational and technological innovations, matched by sound leadership and effective diplomacy, led to the final French victory. First, they captured Rouen in 1449, and then Formigny. By 1451, the French added Bordeaux and Bayonne to their victories, and finally, in the last great battle of the war, utterly defeated the English at Castillon in 1453. In this fight, it was the English, who launched an all-out assault against a French fortified camp equipped with cannon and archers, who suffered slaughter, losing 4,000 in the process.

The English hold over France was forever broken and by 1453 its only possession on the continent was the port of Calais. To protect its hard-earned victories, the French again mastered the return to the competition phase by using its diplomatic and economic power to help foster instability in England, which quickly descended into its own dynastic troubles with the Wars of the Roses. Although it suffered dramatically, the Hundred Years’ War solidified French rule and in many ways, was the springboard that launched France as a nation.

So what does this all mean for the U.S. Army of today? The most important lesson of Agincourt is that nothing is as simple as it seems. Yes, the English longbow was transcendent that day, as it was at Crecy and Poitiers 70 years prior. However, the era of the knight did not end that day. Armored knights remained relevant—if not at times dominant—for the rest of the war. In fact, armored cavalrymen would be part of a combined arms force that remained relevant throughout the Wars of the Roses, and really for another 100 years.

Agincourt, on its face, seems to offer an important lesson; the idea the expensive, exquisite force can be highly vulnerable to cheaper systems, particularly at range. We clearly do not wish to be the French at Agincourt; a force that ignored a brazen pink flamingo in its midst and was so stubborn and set in its ways that it underestimated its enemy and smashed its exquisite force against ranks of English archers on multiple occasions. They ignored their enemy’s stand-off capability, which is in some ways akin to the dilemma posed by the anti-access/area denial challenge to the U.S. Joint force today and in the future. The first step is accepting and understanding the problem, so that we do not ignore the pink flamingo in our midst; adversaries who can challenge us in multiple domains and along multiple axes at stand-off ranges.

However, the more subtle lesson of Agincourt is that while we must try to avoid the arrogance and stubbornness of the French, we equally do not wish to emulate the English, who failed to parley their tactical advantage into any kind of real success, namely because they failed to understand the character of the war they were fighting or to effectively shape the transition between conflict and return to competition. This too is an important lesson, particularly as the U.S. Army is on the precipice of adding a variety of new high-tech systems to its ranks in the near future. The English teach us that technology and tactical advantage can create challenges for an adversary, but they do not, unto themselves, guarantee victory.

As the French finally demonstrated—although belatedly and after much suffering—tactical and technological advantages can be overcome. The French improved and modernized their own force with smart innovation in terms of technology (artillery) and organizational structures, and benefited from capable and inspired leadership. Finally, and arguably most importantly, they recognized that great power conflict requires more than a military solution. I agree with Bret Stephens that the true risk the U.S. military faces is that we do not wish to recognize that our exquisite force is vulnerable to cheaper technology. So our task is to understand that key point and in turn make the Joint force capable of not only protecting itself against today’s longbows but also to effectively and creatively think about how to use our unique—and at times exquisite—capabilities to present dilemmas to our adversaries that translate into success.

In a future where capabilities between peer and near-peers will be roughly equivalent, with specific advantages and characteristics held by each of the main competitors, but an overall rough equivalency balancing at the macro level, the weapons, equipment, and technology possessed by a nation will be important, but likely will not confer a war-winning advantage. Instead, leadership, innovation of ideas, sophisticated training, and a whole-of-government approach to competition and conflict matched by the smart and effective introduction of technology and organizational change will, as they did in 15th Century France, remain essential elements of success in a future conflict.

If you enjoyed this post, please see the following previous posts by Ian Sullivan:

– Making the Future More Personal: The Oft-Forgotten Human Driver in Future’s Analysis

– Lessons Learned in Assessing the Operational Environment

… as well as:

– The Changing Dynamics of Innovation

– Setting the Army for the Future (Parts I, II, and III)

– Winning Future Wars through Developing the Intellectual Component of Fighting Power: The Australian Army’s Approach to Professional Military Education by LTCOL Greg Colton, Australian Army

Ian Sullivan is the Assistant G-2, ISR and Futures, at Headquarters, TRADOC.

[Editor’s Note:  In Part I of this series, Dr. Nick Marsella addressed the duty we have to examine our assumptions about emergent warfighting technologies / capabilities and their associated implications to identify potential second / third order and “evil” effects.  In today’s post, he prescribes five actions we can embrace both individually and as organizations to avoid confirmation bias and falling into cognitive thinking traps when confronting the ramifications of emergent technologies — Enjoy!]

In part I of my blog post, I advised those advocating for the development and/or fielding of technology to be mindful of the second/third order and possible “evil” effects. I defined “evil” as an unexpected and profound negative implication(s) of the adoption/adaption of a technology and related policy. I also recommended heeding the admonition of Tim Cook, Apple’s CEO, to take responsibility for our technological creations and have the courage to think things through their potential implications – both for the good and the bad.

But even if we as individuals are willing to challenge our assumptions and think broadly and imaginatively, we can still come up short.  Why?  The answer is due in part to the challenges of prediction and the fact we are human with both great cognitive abilities, but also weaknesses.¹

THE CHALLENGES

First, common sense and many theorists remind us that the future is unknowable.²  The futurist Arthur C. Clark noted in 1962, “It is impossible to predict the future, and all attempts to do in any detail appear ludicrous within a very few years.”³  Within this fixed condition of unknowing, the U.S. Departments of Defense and the Army must make choices to determine:

– what capabilities we desire and its related technology;

– where to invest in research and development; and

– what to purchase and field.

As Michael E. Raynor pointed out in his book, The Strategy Paradox, making choices and developing strategies, which include adopting new technology, where the future is unknowable and “deeply unpredictable” either produces monumental successes or monumental failures.⁴  For a company this can spell financial ruin, while for the Army, it can spell disaster and the loss of Soldiers’ lives and national treasure.  Yet, we must also choose a “way ahead” – often by considering “boundaries” – in a sense the left and right limits – of potential implications.

Second, our ability to predict when technology will become available is often in error nor can we accurately determine its implications. For example, in a New York Times editorial on December 8, 1903, it noted:  “A man carrying airplane will eventually be built, but only if mathematicians and engineers work steadily for the next ten million years.”⁵  The Wright Brothers first flight took place the following week, but it took decades for commercial and military aviation to evolve.

Each of the services has faced force modernization challenges due to the adoption of technologies that weren’t mature enough; whose capabilities were over-promised; or did not support operations in the envisioned Operational Environment.  For the U.S. Army, Future Combat Systems (FCS) was the most obvious and recent example of this challenge – costing billions of dollars and, just as importantly, lost time and confidence.⁶

Third, in terms of future technology, predictions vary widely. For example, predictions by MIT’s Rodney Brooks and futurist/inventor Ray Kurzweil vary widely to the question – “What year do you think human-level Artificial Intelligence [AI, i.e., a true thinking machine] might be achieved with a 50% probability?” Kurzweil predicts AI (as defined) will be available in 2029, while Brooks is a bit more conservative – the year 2200 – a difference of over 170 years.⁷

While one can understand the difficulty of predicting when “general AI” will be available, other technologies offer similar difficulties. The vision of the “self-driving” autonomous vehicle was first written about and experimented with in the 1920s, yet technological limitations, legal concerns, customer fears, and other barriers make implementing this capability challenging.

Lastly, as Rodney Brooks has highlighted, Amara’s Law maintains:

“we tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.”⁸

 

 

SECOND, THIRD ORDER AND EVIL EFFECTS

I contend that we either:

– intentionally focus on the positive effects of adopting a technology;

– ignore the experts and those who identify the potential downside of technology; and

– fail to think thorough the short, long, and potentially dangerous effects or dependencies – imaginatively.

We do this because we are human, and more often because we fall into the confirmation bias or other cognitive thinking traps that might affect our program or pet technology. As noted by RAND in Truth Decay:

“Cognitive biases and the ways in which human beings process information and make decisions cause people to look for information, opinion, and analyses that confirm preexisting beliefs, to weigh experience more heavily than data and facts, and then to rely on mental shortcuts and the beliefs of those in the same social networks when forming opinions and making decisions.”⁹

Yet, the warning signs or signals for many unanticipated effects can be found simply by conducting a literature review of government, think tank, academic, and popular reports (to perhaps include science fiction). This may deter some, given the sheer volume of literature is often overwhelming, but contained within these reports are many of the challenges associated with future technology, such as:  the potential for deep fakes, loss of privacy, hacking, accidents, deception, and loss of industries/jobs.

At the national level, both the current and previous administrations published reports on AI, as has the Department of Defense – some more balanced in addressing of issues of fairness, safety, governance, and ethics.¹⁰

For example, using photo recognition to help law enforcement and others seems like a great idea, yet its use is increasingly being questioned. This past May, San Francisco became the first city to ban its use, and others may follow suit.¹¹  While some question this decision to ban vice impose a moratorium until the technology produces less errors, the question remains – did the developers and policy makers consider if there would be push back to its introduction given its current performance? What sources did they consult?

Another challenge is language. You’ve seen the commercials – “there is much we can do with AI” – in farming, business, and even beer making.  While “AI” is a clever tag line, it may invoke unnecessary fear and misunderstanding.  For example, I believe for most people equate “robotics” and “AI” with the “Terminator” franchise, raising the fear of uncontrolled autonomous military force overwhelming humanity.  Even worse – under the moniker of “AI,” we will buy into “snake-oil” propositions.

We are at least decades away from “general AI” where a “system exhibits apparently intelligence behavior at least as advanced as a person” across the “full range of cognitive tasks,” but we continue to flaunt the use of “AI” without elaboration.¹²

SOME ACTIONS TO TAKE

Embrace a learning organization. Popularized by Peter Senge’s The Fifth Discipline: The Art and Practice of the Learning Organization (1990), a learning organization is a place “where people continually expand their capacity to create the results they truly desire, where new and expansive patterns of thinking are nurtured, where collective aspiration is set free, and where people are continually learning how to learn together.”¹³  While some question whether we can truly make a learning organization, Senge’s thinking may help solve our problem of identifying 2nd/3rd order and evil effects.

First, we must embrace new and expansive patterns of thinking.  Force developers use the acronym – “DOTMPLF-P” as shorthand for Doctrine, Organization, Training, Materiel, Personnel, Leadership, Facilities, and Policy implications – yet other factors might be as important.

For example, trust in automated or robotics systems must be instilled and measured.  In one poll, 73% of respondents noted they would be afraid of riding in a fully autonomous vehicle – up 10% points from the previous year.¹⁴  What if we develop a military autonomous vehicle or a leader-follower system and one or more of our host nation allies prohibits its use? What happens if an accident occurs and the host population protests, arguing that the Americans don’t care for their safety?

Technologists and concept developers must move from being stove-piped in labs and single-focused to consider the broader view of technological implications across functions.

Embrace Humility. Given that we cannot predict the future and its potential consequences, developers must seek out diversified opinions, and more importantly, recognize divergent opinions.

Embrace the Data.  In New York City during World War II, a small organization of some of the most brilliant minds in America worked in the office of the “Applied Mathematics Panel.”  In Europe, the 8th U.S. Air Force suffered from devastating bomber losses.  Leaders wondered where best to reinforce the bombers with additional armor without significantly increasing their weight to improve survivability. Reports indicated returning bombers suffered gunfire hits over the wings and fuselage, but not in the tail or cockpits, so to the casual observer it made sense to reinforce the wings and fuselage where the damage was clearly visible.

Abraham Wald, a mathematician, worked on the problem. He noted that reinforcing the spots where damage was evident in returning aircraft wasn’t necessarily the best course of action, given that hits to the cockpit and tail were probably causing aircraft (and aircrew) losses during the mission.  In a series of memorandums, he developed a “method of estimating plane vulnerability based on damage of survivors.”¹⁵  As Sir Arthur Conan Doyle’s Sherlock Holmes noted more than 100 years ago,  “It is a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts.”¹⁶

Additionally, we need to reinforce our practice of performing quality literature reviews to aid in our identification of these issues.

Employ Skepticism and a Devil’s Advocate Approach.  In 1587, the Roman Catholic Church created the position of Promoter of the Faith – commonly referred to as the Devil’s Advocate. To insure a candidate for sainthood met the qualification of being canonized, the role of the Devil’s Advocate was to be the skeptic (loyal to the institution), looking for reasons why not to canonize the individual.  Similarly, Leaders must embrace the concept of employing a devil’s advocate approach to identifying 2nd/3rd order and evil effects of technology – if not designating a devil’s advocate with the right expertise to identify assumptions, risks, and effects. But creating any devil’s advocate or “risk identification” positions is only useful if their input is seriously considered.

Embrace the Suck.  Lastly, if we are serious in identifying the 2nd/3rd order and evil effects, to include identifying new dependencies, ethical issues, or other issues which challenge the adaption or development of a new technology or policy – then we must be honest, resilient, and tough. There will be setbacks, delays, and frustration.

FINAL THOUGHT

Technology holds the power to improve and change our personal and professional lives.  For the military, technology changes the ways wars are fought (i.e., the character of war), but it may also change the nature of war by potentially reducing the “human dimension” in war.  If robotics, AI, and other technological developments eliminate “danger, physical exertion, intelligence and friction,”¹⁷ is war easier to wage?  Will technology increase its potential frequency?  Will technology create a further separation between the military and the population by decreasing the number of personnel needed? These may be some of the 2nd/3rd order and “evil” effects we need to consider.

If you enjoyed this post, please see Dr. Marsella’s previous post “Second/Third Order, and Evil Effects” – The Dark Side of Technology (Part I)

… as well as the following posts:

– Man-Machine Rules, by Dr. Nir Buras

– An Appropriate Level of Trust

Dr. Nick Marsella is a retired Army Colonel and is currently a Department of the Army civilian serving as the Devil’s Advocate/Red Team for the U.S. Army’s Training and Doctrine Command.

Disclaimer:  The views expressed in this article do not imply endorsement by the U.S. Army Training and Doctrine Command, the U.S. Army, the Department of Defense, or the U.S. Government.  This piece is meant to be thought-provoking and does not reflect the current position of the U.S. Army.

---

¹ Some readers will object to the use of predicting which implies a certain degree of fidelity, but I use the word as defined more loosely as defined by Merriam-Webster dictionary as “to declare or indicate in advance” with synonyms including to foretell, forecast, prognosticate. Retrieved from: https://www.merriam-webster.com/dictionary/predict#synonyms

² See: Williamson, Murray. (2017). America and the Future of War: The Past as Prologue.   Stanford, CA: Hoover Institute Press, p. 177; Gray, Colin S. (2005, April). Transformation and Strategic Surprise. Carlisle Barracks, PA: Strategic Studies Institute; Gray, Colin S. (2017). What Should the U.S. Army Learn from History? Recovery from a Strategy Deficit. Carlisle Barracks, PA: Strategic Studies Institute

³ Clark, Arthur C. (1962). Profiles of the Future. London, England: Pan Books., p.13

⁴ Raynor, Michael E. (2007). The Strategy Paradox. New York, New York: Doubleday.

⁵ Eppler, Mark. (2004). The Wright Way: 7 Problem-Solving Principles from the Wright Brother. New York, New York: American Management Association, 2004. Quote extracted from insert.

⁶ Department of the Army. (2011). Final Report of the 2010 Army Acquisition Review. Retrieved from: https://www.army.mil/article/62019/army_acquisition_review. See also RAND Report, Lessons from the Army’s Future Combat Systems Program (2012).

⁷ Rodney Brooks Post – AGI Has Been Delayed, May 17, 2019. Retrieved from https://rodneybrooks.com/agi-has-been-delayed/

⁸ Brooks, Rodney. (2017, Nov/Dec). The Seven Deadly Sins of AI Prediction. MIT Technology Review, 120(6), p. 79

⁹ Kavanagh, Jennifer & Rich, Michael D. (2018). Truth Decay: An Initial Exploration of the Diminishing Role of Facts and Analysis in American Public Life. Santa Monica, CA: RAND Corporation (RR 2314).

¹⁰ A sample of reports and studies just on AI include: Allen, Greg and Chan, Taniel. (July 2017) Artificial Intelligence and National Security.   Harvard Kennedy School – Belfer Center for Science and International Affairs; The White House Office of Science and Technology Policy. (2018). Summary of the 2018 White House Summit on Artificial Intelligence for American Industry; Executive Office of the President National Science and Technology Council Committee on Technology. (October 2016). Preparing for the Future of Artificial Intelligence; Department of Defense. (2018). Summary of the 2018 Department of Defense Artificial Intelligence Strategy: Harnessing AI to Advance Our Security and Prosperity; GAO. (2018, June 26). AI: Emerging Opportunities Challenges and Implications for Policy and Research: Statement of Timothy M. Persons, Chief Scientist Applied Research and Methods; Scharre, Paul & Horowitz, Michael C. (June 2018). Artificial Intelligence: What Every Policymaker Needs to Know. Center for New American Security. A simple Google search today using the term “artificial intelligence” produces 55 million results – many of dubious quality.

¹¹ Van Sant, Shannon and Gonzales, Richard. *2019, May 14). San Francisco Approves Ban on Government’s Use of Facial Recognition Technology. NPR. Retrieved from: https://www.npr.org/2019/05/14/723193785/san-francisco-considers-ban-on-governments-use-of-facial-recognition-technology

¹² Executive Office of the President National Science and Technology Council Committee on Technology. (October 2016). Preparing for the Future of Artificial Intelligence, p. 7.

¹³ Garvin, David A. (1993, July August). Building a Learning Organization. Harvard Business Review. Retrieved from: https://hbr.org/1993/07/building-a-learning-organization

¹⁴ Garner, Greg. (2018, May 22). “Even Millennials are losing confidence in autonomous cars, surveys reveal. Forbes. Retrieved from https://www.forbes.com/sites/greggardner/2018/05/22/millennials-self-driving-cars/#62b5aedd2628

¹⁵ Syed, M. (2015). Black Box Thinking: Why Most People Never Learn from their Mistakes – But Some Do. New York, New York: Penguin, p. 35-36. Some sources state Wald’s story is embellished. See Bill Casselman’s post the conflict in Wald’s often told story on the American Mathematical Society website. Retrieved from http://www.ams.org/publicoutreach/feature-column/fc-2016-06. Wald’s paper can be retrieved from: https://apps.dtic.mil/docs/citations/ADA091073

¹⁶ Doyle, Sir. Arthur. (1891). A Scandal in Bohemia.

¹⁷ Von Clausewitz, Carl. On War. Translated Michael Howard and Peter Paret. (Princeton, NJ, Princeton University Press, p. 88. See https://www.benning.army.mil/MSSP/Nature%20and%20Character/