[Editor’s Note: In today’s post, proclaimed Mad Scientist and returning guest blogger Ms. Marie Murphy addresses the implications of weaponized Artificial Intelligence (AI) when employed in information operations. Truth and trust are the first causalities of this perception-altering capability!]
Artificial Intelligence (AI) is a tool that effectively enables machine-speed actions, posing new threats never before seen over the course of history. This disruptive capability can both erase existing data and create entirely false realities. If nefarious actors are able to effectively harness AI, the Army may potentially operate in a manufactured information environment with altered data guiding its decision-making. Using AI in this way is a competition-phase tactic that affects the will of people and deteriorates their trust, both within and about the Army.
Historical Precedence
The concept of altering or removing information from public memory has been around since the time of Ancient Rome. Damnatio memoriae is the contemporary term for the condemning of the memory of an emperor by the Senate posthumously.1 The idea is “to erase a person as if he (or she) never existed.”2 The practice died out with the emerging prominence of the Catholic Church but resurfaced in Northern Italy during the Renaissance.3 A “damnation of memory” was ordered when a new government took power in Florence, purging the powerful citizens and families of the old regime from societal recollection.4 In the 20th century, at the height of the Soviet Union, Josef Stalin worked to remove any mention or figure of his political enemies from public view, to the point where they were edited out of textbooks and pictures.5
Today, history is repeating itself. China is attempting to remove all reference to the events that occurred in Tiananmen Square on 3-4 June 1989, and they have an advantage those in the past did not: AI. The absence of any public information about the events surrounding Tiananmen Square in contemporary China stems from the Chinese government’s efforts to censor any mention or insinuation of the incident online and in reality. AI works at machine-speed to filter internet search results and social media posts to assist human censors. 6 A recent study proved the effectiveness of this campaign, showing that a meager 15% of Beijing University students could accurately identify the “tank man” photograph, one of the internationally recognized symbols of the protest.7 While most people appear to be aware of the incident by the time they’re well into their 20s, there is a general apathy toward, or even sympathy with the government amongst the current generation regarding the 1989 protests. 8
Future Threat
AI can be weaponized in information and cyber warfare, which characterize the interactions between strategic competitors in the competition phase. However, it’s often discussed in the context of lethal autonomy, man-machine teaming, or practical battlefield applications. But what happens when AI begins to psychologically alter the operational environment by changing the perceptions of both military and civilian personnel alike? Through the deletion or alteration of the past or the creation of a fabricated present, AI is a tool which aids in and speeds up the process of identifying, changing, and 
generating information. False narratives based on AI-manipulated or -generated information and media can originate from anyone, anywhere; developing and spreading rapidly with a detrimental effect on trust. This deterioration of trust moves in two directions: The Soldier not trusting the information they are given which is crucial for decision-making and combat performance, and civilians and other militaries not trusting the actions of the U.S. Army because their perceptions have been affected by AI-influenced content.
If AI, employed by a nefarious actor, is able to manipulate the information available about the past and deliver false information about the present, there is likely to be a strong impact on Soldier and Commander trust of any information received. Altering history to the point that it is impossible to discern what the truth is, and potentially basing decisions on inaccurate information about the past, could have unintended and devastating consequences. 
AI manipulations can also affect trust because of their applicability in enabling small-scale, personalized warfare. A particular Soldier could be targeted with a computer-generated fake message from home that someone is very ill. This false information could cause a breakdown of trust and a rise in skepticism in the individual Soldier to the point that they begin to question every piece of information they’re given, wasting valuable time and energy.
The Army is also facing the challenge of AI manipulating the perceptions of military and civilian personnel support elements. AI and those who program it can accomplish this by doctoring or deleting the records or social perceptions of prior Army actions. To create a false reality that corrupts the information environment, AI could be used to spread malicious rumors influencing international will and public opinion. One such rumor might be a manufactured atrocity. It would be incredibly hard to deny the alleged incident if there are audio and video recordings depicting it (even though that evidence is fabricated with the help of AI); the programmer could even use AI to create computer-generated people “on camera” giving false eye-witness accounts. If the public and other militaries whom the U.S. aligns with do not trust the Army, then operations will become strategically and tactically more challenging.
While AI may present the Army with many strategic and tactical benefits by providing machine-speed analysis and decision-making, it can also work counter to the Army’s goals and initiatives. With other entities such as China developing AI at a rapid pace and willing to deploy it against its own citizens, the time of AI-enabled damnatio memoriae has arrived. Whether it be altering the past or creating a false present, AI-generated information may greatly impact future operability and trust of warfighters.
If you enjoyed this post, please also see:
Influence at Machine Speed: The Coming of AI-Powered Propaganda by MAJ Chris Telley
The Death of Authenticity: A New Era of Information Warfare
China’s Drive for Innovation Dominance
Proclaimed Mad Scientist Marie Murphy is a rising senior at The College of William and Mary in Virginia, studying International Relations and Arabic. She is a regular contributor to the Mad Scientist Laboratory, interned at Headquarters, U.S. Army Training and Doctrine Command (TRADOC) with the Mad Scientist Initiative last summer, and has returned as a consultant this summer. She was a Research Fellow for William and Mary’s Project on International Peace and Security.
Disclaimer: The views expressed in this article do not imply endorsement by the U.S. Army Training and Doctrine Command, the Army Futures Command, the U.S. Army, the Department of Defense, or the U.S. Government. This piece is meant to be thought-provoking and does not reflect the current position of the U.S. Army.
1 “Damnatio Memoriae.” Livius.org, updated June 23, 2019. https://www.livius.org/articles/concept/damnatio-memoriae/
2 Petersen, Lauren Hackworth. “The Presence of ‘Damnatio Memoriae in Roman Art.” Notes in the History of Art, vol. 30, no. 2, 2011, p.1. JSTOR. www.jstor.org/stable/23208566
3 “Damnatio Memoriae.” Livius.org, updated June 23, 2019. https://www.livius.org/articles/concept/damnatio-memoriae/
4 Petersen, Lauren Hackworth. “The Presence of ‘Damnatio Memoriae in Roman Art.” Notes in the History of Art, vol. 30, no. 2, 2011, p.1. JSTOR. www.jstor.org/stable/23208566
5 Bond, Sarah. “How Do You Purge The Memory Of An Emperor.” Forbes.com, April 11, 2017. https://www.forbes.com/sites/drsarahbond/2017/04/11/how-do-you-damn-the-memory-of-a-roman-emperor/#49228baf49b2
6 Gilbert, David. “How China Is Wiping Memories of Tiananmen Square Off The Internet.” Vice News, June 4, 2019. https://news.vice.com/en_us/article/7xge3b/chinese-dissidents-are-running-out-of-ways-to-remember-tiananmen-square
7 Keng Kuek Ser, Kuang. “How China has censored words relating to the Tiananmen Square anniversary.” Pri.org, Hune 4, 2016. https://www.pri.org/stories/2016-06-03/how-china-has-censored-words-relating-tiananmen-square-anniversary
8 Fish, Eric. “Tiananmen Shaped China’s History. But Chinese Millennials Have Mixed Views About Its Legacy.” Time, June 3, 2019. https://time.com/5599060/china-millennials-tiananmen-anniversary/
So this is and will be a permanent huge issue for not just the nation but the species. When anything one downloads or sees can be extremely believable but completely unreal, what is the mechanism for assuring credibility? Who determines, or stamps, or approves, what we will see in the future as “true” information? Which US agency will be charged with “information credibility assurance”? Or at least giving a rating to sources and products? And quis custodiet ipsos custodes? When entities to include national governments do not use science and reality as the basis for its screening mechanisms, the default is dogma or the opinions of those currently in power. Which entity will be wholly apolitical, wholly empirical and wholly committed to trying to achieve the utmost veracity in the information seen by citizens in and out of uniform? Where does the intersection of free speech and information warfare (both foreign and domestic) lead to? Does the DoD take the lead, now that adversary IO and cyber attacks now happen in our homeland? Does another entity such as the IC? Or a brand new entity? These questions will have to be answered, and soon, or we will continue to fall further behind our adversaries as they conduct these IO and false information campaigns.
Mr. Kramer,
You bring up some excellent questions that will have to be addressed with the proliferation of advanced digital manipulation technologies. It is a whole policy area in and of itself, but policing information is an endeavor that democratic governments tend to find great issue with. It would likely require a “whole of government” approach to verify information in this operational environment because this issue, as you correctly point out, touches the DoD, the IC, and other homeland entities. Hopefully international responses to these kinds of scenarios are proactive rather than reactive.