Keywords

1 Introduction

Dismounted soldiers are confronted with complex and physically demanding tasks. The expectation is that future soldiers will be equipped with advanced technologies aimed to support the execution of their tasks. The large amount of equipment will not only influence physical performance but may also impact cognitive performance as these new technologies are not always attuned to military operations. Physical load experienced by dismounted soldiers has already been studied extensively and is often measured using a military obstacle course. However, to investigate and understand the effects of new technologies (e.g., battlefield systems, augmented technology) on cognitive task performance in the military domain, no such framework exist. Current cognitive test batteries for soldiers particularly aim to predict future performance [1] or aim to identify individuals with Traumatic Brain Injury [2]. Moreover, only a few studies have focused on the cognitive load of soldiers during military operations in the field as compared to research among pilots [3, 4]. Nevertheless, studying cognitive load among dismounted soldiers is important to identify how new (information) technology facilitate or impede cognitive performance.

In the current project, the aim was to gain insight into the tasks and cognitive load of a troop commander during a troop hasty attack with the purpose to implement a mixed reality platform that will be used to understand the impact of new technology on soldier’s cognitive performance. The performance during a hasty attack was studied as such an attack involves the fundamental basics of infantry actions in conventional warfare and is also known to be challenging due to various uncertainties (e.g., unknown threat location, limited terrain knowledge) caused by a limited preparation time.

2 Method

2.1 Participants

Six recruits in the marines’ officer’s training program POTOM (Praktische Opleiding tot Officier der Mariniers) participated in this study (age between 24 and 30 years, on average 27 years). Two participants were already employed at the Royal Netherlands Marine Corps (RNLMC) in an operational and supervisory function. This research was approved by the ethical committee of TNO.

2.2 Setting

This study was performed by four researchers during a live fire training of the troop hasty attack by the Royal Netherlands Marine Corps (RNLMC) in the hilly Sennybridge Training Area in Wales (UK) in May 2017. Within four days, six runs of hasty attacks were studied, four in the morning and two in the afternoon. Each participant held the position of troop commander during one run. A run consisted of one or more hasty attacks. Each run was studied by a pair of researchers.

2.3 Procedure

Upon arrival at Sennybridge, all participants were briefed on the purpose of the research. A day before the participant would perform the hasty attack run, the participant signed the informed consent form and was instructed to fill out the questionnaires regarding sleep duration and quality the next morning directly upon waking. Immediately before the hasty attack run took place, the participant also indicated his level of sleepiness. During the run, two researchers closely followed the troop commander. On average, a run lasted two hours. Directly after completing the run, the participant attended the after-action review held by the instructors, followed by filling out the cognitive and physical load questionnaires and an interview under the guidance of the two researchers. This interview took place in a private room to ensure the privacy of the participants. Finally, participants were thanked for their time and effort. On the same day as the hasty attack the POTOM lead instructor filled out a questionnaire rating the performance of the recruit.

2.4 Measures

In this study, a cognitive task analysis (CTA [5]) was used to examine the cognitive load of a troop commander during a hasty attack. The CTA consisted of three parts, namely observation of behavior during the hasty attack runs, an interview directly after the hasty attack run, and various questionnaires before and after the hasty attack run. As preparation, before the CTA took place, a hierarchical task analysis (HTA [6]) was conducted to obtain an understanding of the tasks and subtasks of a troop commander during a hasty attack. Doctrines, interviews with Subject Matter Experts and a field observation were used to gather this knowledge.

Observation.

Two researchers observed whether and how the troop commander performed the tasks required during a hasty attack using an observation form based on the HTA. The researchers observed the troop commander from a short distance (~10 m) and were listening to either the troop radio net or the squadron radio net. After the hasty attack, the observers also attended the after-action review.

Interview.

Directly after the after-action review, an interview was conducted with the participant using the Critical Decision Method (CDM [5, 7]) one of the most commonly used methods of implementing a CTA [8]. This method gives the interviewer the opportunity to return to a critical moment together with the interviewee and to identify why and based on what information, decisions were made [7]. The interview was structured using four phases. In the first phase a specific event was identified that caused a high level of cognitive workload for the participant. In the second phase, a timeline was constructed to obtain a clear overview of this event. In the third phase, the event was discussed in more detail to better understand how the situation and the cognitive load was experienced by the participant (e.g., what information was available, how did they make the decision). In the last phase, ‘what if’ questions were asked to invite the participant to speculate on how his decisions might have differed when having, for example, different technology available during the attack. Interviews lasted an hour. When there was time left after discussing one event, another event with a high cognitive load was identified and analyzed. The audio during the interviews was recorded.

Questionnaires.

Participants filled out questionnaires regarding sleep since earlier research conducted with a previous POTOM class showed an accumulation of sleep debt [9]. Duration of sleep and subjective sleep quality of the previous night were measured with questions from the Pittsburgh Sleep Diary [10] and Karolinska Sleep Diary [11]. In addition, a question was asked about the average sleep duration during the whole training period in Sennybridge that started two weeks before the start of this study. At the beginning of the hasty attack run, participants filled out the Stanford Sleepiness Scale [12] to measure their level of sleepiness.

To gain insight into the experienced cognitive load during the hasty attack, participants completed the NASA TLX [13] and the Rating Scale Mental Effort (RSME [14]). To assess experienced physical load, participants filled outthe rate of perceived exertion (RPE) questionnaire [15].

Additionally, the POTOM lead instructor filled out a short questionnaire concerning his view on the recruit’s performance. Five-point scales were used to indicate general functioning, stress resistance, learning ability and potential as troop commander. A one indicated ‘very bad’ and a five ‘very good’. Furthermore, a grade was given between 1 and 10 to indicate how well the hasty attack run was executed by the recruit.

2.5 Data Analysis

Questionnaires.

The results of the questionnaires filled out by the participants and the instructors were visualized in graphs and globally reported such that a participant’s individual scores are no longer recognizable.

Observation and Interview.

The transcripts of the interviews and notes from the observations were analyzed by the four involved researchers. Each researcher analyzed each hasty attack run (six in total) by answering the following seven questions:

  • Which phase(s) in the hasty attack caused the highest cognitive workload?

  • Which elements were contributing to this cognitive workload?

  • How was the behavior of the participant (based on observation)?

  • What were the operational consequences of the cognitive workload?

  • What were the mitigation strategies applied by the participant?

  • How did the participant expect that a more experienced troop commander would respond?

  • What technologies could have supported the participant during the hasty attack?

Afterwards, these insights were discussed by the four researchers to create a common understanding of the experience of the participant (see Results section).

3 Results

3.1 Questionnaires

Sleep Duration and Sleep Quality.

Table 1 shows the results of the questionnaires concerning sleep duration, sleep experience and the Stanford sleepiness. Results show that participants did have an acceptable sleep duration and experience.

Table 1. Results of sleep duration, sleep experience and Stanford Sleepiness scale
Full size table

Subjectively Experienced Load.

Both physical and cognitive load were relatively high. The physical load (RPE) was on average 14.3 indicating ‘relatively heavy’ (SD = 1.5). The cognitive load calculated by averaging over the six NASA-TLX questions and participants was 13.7 (SD = 1.22) on a scale of 0 (none) to 20 (maximal). The average mental effort (RSME) score was 67.6 (SD = 19.3) on a scale of 0 to 150. Four participants indicated ‘considerable effort’, one participant ‘great effort’ and one participant ‘some effort’.

Performance Evaluation by Instructor.

The participants were rated by the POTOM lead instructor on several scales, see Table 2. The highest scores on all scales belonged to the participants with previous military experience.

Table 2. Performance evaluation by instructor
Full size table

3.2 Observation and Interview

For a better understanding of the results in this section, Table 3 gives an overview of the phases of a hasty attack. The next subsections will address the cognitive load during these different phases.

Table 3. Overview of tasks of the troop commander during a hasty attack
Full size table

Cognitive Demanding Phases During the Hasty Attack.

Table 4 shows the phases during the hasty attack that were highlighted as most cognitively demanding by the participants during the interviews. The number of hasty attacks a participant performed during a run varied per participant. When multiple enemies were encountered, the later encounters seemed to be more cognitively demanding than the first. During the later encounters, the troop was often more scattered over the terrain due to previous attacks and the evacuation of casualties.

Table 4. Most cognitive demanding phases during the runs as indicated by the participants.
Full size table

Four phases were indicated as most demanding by the participants:

  • 2a Initial reaction: This phase was indicated as most demanding in the situation of a sudden new enemy contact, while the reorganization after previous enemy contact(s) was not yet completed. During these high demanding initial reactions, the troop was relatively dispersed over the terrain due to multiple attacks or casualties. Maintaining command and control was reported as more difficult.

  • 2b Quick estimate: During this phase the highest cognitive demand was during planning of the hasty attack. This demand was high for several reasons; difficulties to decide how to approach the enemy, e.g., via the left or the right flank, the multitude of events occurring simultaneously, e.g. casualties, or a lack of basic skills needed to communicate the plan according to protocol to the squadron commander.

  • 3b Approach: Two different high demanding situations were reported during the approach phase. First, during two hasty attacks, the troop commander misinterpreted the terrain in terms of opportunities to hide for the enemy, resulting in a situation where their own troops were visible for the enemy. Therefore, a new plan was made and communicated. During another hasty attack, the troop did not approach the enemy from the correct angle, causing a potentially dangerous situation increasing the risk of friendly fire.

  • 4 Reorganization: Participants had difficulties with setting up a good strategic position in the terrain (i.e., a position with a clear 360° overview of the surrounding area and hidden from the enemy). In addition, it was difficult to keep the priorities in the right order, e.g., prioritizing safety of the troop above dealing with casualties.

Cognitive Workload Factors.

The different elements contributing to the cognitive load were categorized in different cognitive load factors. Table 5 gives a description of these factors and in how many events these factors contributed to the cognitive load. The factors are ordered by the number of events.

Table 5. Overview of factors that increased the cognitive workload during the events mentioned as cognitive demanding.
Full size table

Figure 1 gives an overview of the different cognitive load factors that occur within the different phases of the hasty attack. The most cognitive load factors were experienced during the quick estimate phase; nine out of ten workload factors were contributing to the workload in this phase. A factor contributing to all phases were the casualties.

Fig. 1.
figure 1

Occurrences of cognitive load factors for each phase in the hasty attack

Full size image

Mistakes During the Hasty Attack and Consequences.

Below an overview of mistakes of the recruits that we identified from the observations, interviews, and after-action reviews ordered by cognitive load factor.

Command and Control.

Mistakes resulting from a lack of command and control typically result in reduced safety, reduced sustainability and loss of momentum of the attack. Typically mistakes were: a lack of awareness of the position of all sections, positioning themselves near the front-line when confronted with an armored threat, forgot to allocate the weapon systems during the reorganization and forgot to clearly instruct the attack plan to a section.

Terrain.

Amistake that was made by multiple recruits was choosing the wrong route for the flanking maneuver due to inaccurate estimation of terrain characteristics. In one occasion, this reduced sustainability and momentum as the recruit should create a new plan. In another case, the recruit continued the operation through open terrain and therefore increasing the risk being detected by the enemy.

Threat.

In several runs, recruits forgot their own cover in the field (e.g., lifting their heads too high or being alone in the field without a section for protection). This is very risky as it is important for the mission and troop that the troop commander remains safe.

Basic Skills.

One recruit gave a wrong coordinate of the enemy position. This mistake was corrected by the instructor, but otherwise it could have increased the risk of friendly fire and not being able to defeat the enemy during the attack.

Supporting Technologies.

During the interviews, the recruits were asked which technologies would support the execution of the hasty attack. Several technologies were mentioned. To increase situational awareness, a tool that tracks your own position, the position of the other sections, or the enemy’s position was mentioned. This information can be used during the quick estimate phase. Also, eyes in the sky (i.e., drones) could help them to find the enemy location or to explore terrain characteristics (e.g., finding locations to cover for the enemy), while the troop commander can stay safely in the back of the terrain. Augmented reality was mentioned as technology that allow them to see through the eyes of the fire section (this section is often closer to the enemy) resulting in an increase in situational awareness. In addition, one recruit suggested that inclinations of the terrain should be visible on a map. Altitude lines on a map can be difficult to interpret and can make it harder to estimate distances. Another recruit suggested a digital foldable map allowing them to zoom in and out or see the terrain from another perspective. Smaller radios were also suggested, as the radio currently carried by the troop commander is large and limits their mobility. Overall findings showed that the most frequently mentioned technologies were those that increase situational awareness.

4 Discussion

In the current study, the CTA methodology was applied to a troop hasty attack aiming to better understand the cognitive load of a troop commander during such an attack. Results showed a relatively high level of cognitive load and different phases of the attack contributing to this load. The phases that contributed most to the experienced load were the initial reaction, quick estimate, approach and reorganization. For each phase, the cognitive load factors contributing to the cognitive load varied. The number of cognitive load factors was the largest during the quick estimate, where a variety of cognitive tasks were performed by the troop commander. During the quick estimate, four crucial cognitive tasks occurred simultaneously, namely acquiring situational awareness, deciding the best route for approaching the enemy, and communicating and planning the attack. The results of the interview showed that during this phase also several mistakes were made that could have been the result of the high cognitive load experienced by the recruits. Typical mistakes were: choosing the wrong route, unsafe positioning during information collection (i.e., visible for the enemy or too close to contact), unclear orders during the communication of the plan and inaccurate awareness of the position of the sections in the field. In the current study, the assault phase was not mentioned as a high demanding phase. However, during an attack in real life, the experienced cognitive load will probably be larger, due to higher levels of experienced stress that is difficult or even impossible to simulate during a training.

The factor most frequently mentioned to contribute to the cognitive load was Command and Control (C2), which is also an important task for a troop commander during a hasty attack. Two other factors, casualties and threat, were also mentioned frequently. These factors were demanding for the recruits since both factors happen suddenly and most of the time the original plan needed to be redefined. For example, in a threatening situation, the threat should be eliminated as quickly as possible while taking into account the safety of the troop. Finally, also the terrain was mentioned as a demanding factor. Perceiving and understanding the characteristics of the terrain is an important prerequisite to ensure a safe and efficient elimination of the threat. Many recruits had difficulties with the correct interpretation of the terrain, finding the optimal route to attack the enemy and to position themselves having a clear overview of the battle space without being seen by the enemy.

The current study also has some limitations. The first limitation is that during the training program special attention was paid to C2 and the recruits knew they were credited based on their performance, that could have overestimated the contribution of C2 on cognitive load. In addition, the presence of instructors and the fact that the performance of the recruits was judged might have changed their behavior and decisions. Another factor is the number of updates asked by the squadron commander. This was more than during a regular training or a real life hasty attack. Additionally, the recruits that were assigned to be one of the section leaders also tried to show the best of themselves and gave more information on the radio than in a more realistic scenario. This might have increased cognitive load, since the troop commander had more information available to process.

5 Mixed Reality Platform

The different insights gained in the current study are used for the development of a mixed reality platform; a platform that combines new technological innovations with a virtual reality environment in which users will interact with their physical equipment while walking on a treadmill that is coupled to a virtual world displayed on a large screen to increase immersion (see Fig. 2). Such a platform offers the opportunity to investigate the effects of new technological innovations on cognitive performance under conditions mimicking real-life situations, while controlling for potential confounding variables. In this section, the important insights are discussed that are prerequisites for the development of such a platform.

Fig. 2.
figure 2

Example of the mixed reality platform

Full size image

5.1 Cognitive Load

From the four most cognitive demanding phases the quick estimate and approach are the most interesting to implement in the mixed reality platform. During the quick estimate the troop commander was applying all the steps in the situational awareness (SA) cycle as defined by [16]; perception, comprehension and projection. According to [16] and in line with our observations, the SA the commander has of the battle field (e.g., where is the threat, where are my sections, characteristics of the terrain) determines the decisions the troop commander is making (e.g., what is the best route to follow, how to position my sections, what weapons systems do I need). During the approach phase the troop commander was often confronted with unexpected events increasing cognitive load (e.g., the terrain was more open with less elements to hide, another enemy appeared, casualties occurred). Therefore, the virtual environment and the scenario should be realistic. For example, the terrain must provide possibilities to hide and overlook the battle field. In addition, for a realistic situation, relevant stressors that impact cognitive performance such as environmental noise, time pressure, uncertainty, fatigue, and unexpected events must be implemented in the scenario.

Both during the quick estimate and approach, the troop commander performed many different tasks and frequently switched between them (e.g., inform higher command, dealing with casualties, estimate the (new) enemy position, preparing the attack, change the initial plan based on new information). Therefore, the scenario in the mixed reality platform should include multiple tasks that are performed simultaneously.

Communication with the section and squadron commander is important to gain situational awareness and control the troop. When an order is communicated to the sections, the execution of that order should be monitored carefully to ensure the plan is correctly executed, which increases cognitive load. In addition, when the plan is not well understood it will increase the likelihood that the troop commander must intervene, which costs additional cognitive resources and time. Furthermore, environmental noise, such as gun shots, will increase the difficulty to understand messages, resulting in an increase of cognitive load. Therefore, within the mixed reality platform the commander should be able to communicate with at least two section commanders (e.g., one section that is under fire and another section that approaches the enemy). Hence, communication devices will be coupled to allow the troop commander to communicate with other sections and the squadron commander.

5.2 Technologies

Numerous technologies are currently on the market that claim to increase operational performance. The CTA showed that technologies that support the commander to build up SA have large potential. Examples are systems with blue force tracking or systems with cameras to inspect the terrain. Integrating these into the mixed reality platform allows us to study how and when information can be presented most effectively, which information displays are most effective (e.g., hand held displays, augmented reality) or what modalities can be used in this context (e.g., haptic, visual, auditive).

5.3 Physical Exertion

The troop commander experienced a high level of physical load before the quick estimate phase and during the approach phase, since the troop commander followed the section that approached the enemy. This stresses the importance to study cognitive performance while being physically active or induce physical fatigue before cognitive demanding tasks. Therefore, a treadmill will be coupled to navigate within the virtual environment.

6 Conclusion

The results of the current study successfully contributed to the current development of a mixed reality platform. In the future, this platform allows to examine the effects of new technological innovations on cognitive performance under conditions mimicking real-life situations, while controlling for potential confounding variables. In the future, it would be interesting to perform a cognitive task analysis with more experienced military and include a larger set of military tasks. This might result in additional requirements that can be used to optimize the mixed reality platform. In addition, the mixed reality platform will be developed and tested involving experienced military users to ensure operational relevancy.