Fighting in the Metaverse: a virtual world that is already available to our Army experts

Between February and May 2023, from the Mediterranean to the large manoeuvre camps in Champagne, via the port of Sète, France will be transformed into a vast “shared hybrid operational theater” (SHOT) for the purposes of Operation Orion. This unprecedented exercise will train our armies in high-intensity warfare on a scale not seen since the end of the Cold War. Up to 12,000 troops will be deployed during the most intense phase.

Many others will be present virtually through the tactical consoles of mobilized PCs. This will be the case of an American division, whose participation will be entirely digitized in an Army training center. In all “environments” (land, air, sea, space) and “fields” (informational, electronic, cyber), the watchword given by the Army Staff to Orion “players” is: “Tactical freedom, realism, innovation”.

Orion’s SHOT, which will articulate layers of “virtual, augmented, and hybrid” reality, is a life-size illustration of these military “multiverses” pioneered by our armies, along with their American alter egos, explained Lt. Col Raphaël Briant, an expert at the Ministry of the Army’s Directorate General for International Relations and Strategy (DGRIS) at a conference on the metaverse organized in early December by the Army’s Command Doctrine and Education Center (CDEC) at the École Militaire in Paris.

Thanks to the simulators present both in PCs and on board combat vehicles, and the associated virtual training programs, for which the French company Masa Group is the leader for staffs, the military have become accustomed to “mixing virtuality and reality”. This allows them to optimize their peacetime constraints while increasing realism in training.

The new deal is that the real war has managed to invite itself into these hybrid spaces that are still difficult to define. These are “territories” where “reality and virtuality merge to allow people to interact, work, create, play and share”, summarizes consultant Matthieu Flaig of Sqorus. A place where it is possible to “make intelligence and influence behaviors in order to transform an informational capital into an operational capital”, adds Colonel Samir Yaker, the CDEC specialist.

The new underlying technologies have opened up a whole new field of possibilities. “In the web and its first extension, as in social media, we know where the users have been. Yet, these new universes are recording, for the first time, behaviors and emotions triggered by the range of simulations encountered during the immersive experience,” says Alexandre Bouchet, president of the “Laval Virtual” trade show and of the AFXR association, which brings together players in the “extended reality” field.

What follows are the challenges of capturing and analyzing this new data. Until now, it was a matter of sifting through connection data as well as texts and images deposited on online servers. In virtual, augmented and hybrid worlds, it is necessary to screen the behavior of bodies and the intonation of voices, and then to have sufficiently large pipes to share and exploit this data.

This “new world” is still largely unknown. Its geography consists of a puzzle of scattered bricks, more or less interconnected. Decentraland is an example of these small virtual communities of a few thousand individuals that are emerging. Created in 2020 by two Argentinians, this augmented reality platform embodies a virtual world divided into 90,000 parcels totaling 23 km2, which can be acquired with virtual currency. Thanks to 5G, augmented reality gurus are now dreaming of perfect virtual twin worlds in the real world.

Still, their vision runs into the wall of technical and financial challenges. In 2019, the Californian start-up Magic Leap, which has raised funds from the Americans Google and Qualcomm as well as from the Chinese Alibaba, announced that in 2021 it would deliver “Tomorrowland”, a “mirror city” freed from the drawbacks of its real model, which anyone could access by equipping themselves with the in-house augmented reality headset.

Three years later, the product’s failure forced the company’s executives to lay off a thousand employees and review its strategy. Last September, Magic Leap announced that it would finally produce a headset for professionals: a smaller and lighter model, with an extended field of vision, therefore more expensive.

Microsoft, which has been awarded a $12 billion contract to supply its HoloLens headset to the U.S. military, was asked by the Pentagon to perfect its product: its intensive use was causing nausea among soldiers. According to Timothée Sylvestre, director of YSpot, the open innovation center of CEA Tech in Grenoble, the metaverse obeys Amara’s law: “We tend to overestimate its short-term impact and underestimate it in the long term.”

While waiting for the progress of technology and networks, we can see, according to him, the potential future of the metaverse: a “mirror world” that will interconnect humans, robots, and avatars to, ultimately, allow the remote control of more or less complex operations, in the service of peace or war, just as with each leap in science.

In the meantime, Pierre Paperon, head of the France Méta association, predicts the swift advent of “metavers as a service” consumption by organizations and citizens, in other words, on a subscription basis.  He is already suggesting that the military equip themselves with search engines capable of indexing all existing and future virtual universes on the web. It is urgent to anticipate the appearance of bubbles of counter-narratives, and their possible instrumentalization by competitors – whether state-owned or not – who deploy power strategies.

In the metaverse, the “cultural war” is already raging, says David Nahon, director of innovation for the immersive experience at Dassault Systèmes, a global specialist in digital twins. As the main promoters of the technologies that give substance to virtual worlds, “the major American digital players are industrializing Hollywood’s fantasies while enacting and imposing their laws, their sanctions, and their virtual currencies.”

Who will own the data recorded in augmented reality headsets in the future, and will it be anonymous? Will there be borders in these worlds, and who will police them? While waiting for clear answers, our military wants to be able to better understand these new worlds so that, by 2025, it can “maneuver” in them. This is what the Chief of Staff of the Armed Forces has asked of his troops, in light of the latest developments in warfare.

The Ukrainian conflict shows how much high-intensity warfare has indeed taken over the digital world. For General Pierre-Joseph Givre, director of the CDEC, the Army must first accelerate the hybridization of its digital ecosystems with those of the civilian world. Without Starlink, the private internet and satellite communication provider, he explains, it would have been much more difficult for the Ukrainians to lead and win the game in the digital or cyber field.

At the same time, the institution must acquire the “tools to capture, link, exploit and transform the data accumulated in the metaverse, into strategic and tactical targeting.” At a time when 80% of intelligence material is collected in open source, one must be quickly “capable of dipping into the new ocean of data that is opening up; in a week at the beginning of the conflict, in an hour now, the Ukrainians are developing new informational layers in cyberspace according to their needs”.  “If the battle for technology is largely lost, resumes General Givre, the Army can play a role in the competition that has begun to set the uses in the metaverse.”

Getting information, mapping 3D areas in real time, influencing and deceiving at the pace of operations, but also recruiting and training: this new project has started. For Black Friday, the Army designed a striking digital poster using this concept to target young people aged 17 to 25 who are addicted to social media. “We have to go get them where they are,” pleads Lieutenant Colonel Hubert de Quièvrecourt, the leader of this battle to capture the “workforce”. Experts are imagining the next stage: for example, offering pre-selected candidates an experience of skiing in a military simulator.

The most “addicted” would be entitled to extra sessions before going on to practice. This sport is becoming less and less popular among young people and causes more discouraging injuries to beginners than in the past. For their part, those in charge of training tank and helicopter crews or special forces operators are thinking about an optimized simulation, by providing, through the metaverse, models of behaviors and emotions,” says Colonel Jean-Gabriel Herminet, in charge of digital technology at the French General Staff.

Of course, these new “traces” are also within the reach of the enemy, and this is a loophole that other specialists are working to turn to the institution’s advantage. “Rather than pretending to prohibit our soldiers from living in the digital world, which would be unrealistic, we are looking to blend their traces into the ambient noise. We have succeeded in simulating the presence of a unit in its garrison even though it had returned to its theater of operation,” says Lieutenant-Colonel Fabien Simon.