Keywords

1 Introduction

With the rapid development of aviation technology and computer technology, a fighter avionics system becomes more and more complex and data information generated by the system rises explosively so that a pilot has to receive much more information in a short period of time. HMDs interface is an augmented reality (AR) interface. In comparison with the traditional avionics system, whose display interface is featured by single and limited data sources, fixed interaction modes and low-dimensional information visualization, the information sources of HMDs are diversified; its information is real-time and dynamic; its information presentation is characterized by overlapping, multi-dimensionality and spatiality. For different flight missions, the pilot needs to acquire and focus on the information importance. If alarm information during different mission phases is scientifically designed and effectively hinted, the situational awareness ability of the pilot will be significantly improved to avoid accidents.

Many scholars at home and abroad have carried out relevant researches on the interface layout and information presentation of avionics systems such as HMDs. For the development of optical principles and the visual expression of interfaces of HUD and HMDs, Collinson et al. made comprehensive researches to ensure a pilot can perform missions safely and efficiently [1]. Rolland et al. conducted holographic waveguide HMD experimental researches on the improvement of imaging quality of HMDs [2]. Zhang et al. performed experimental researches on depth perception of HMDs, analyzed and summarized factors affecting the optimal retro-reflection screen [3]. For an unfixed-wing fighter in extreme environments such as fog, dust and darkness, Doehler et al. researched the interface navigation design of HMDs [4]. By right of ways of highlighting including brightness and flashlight, Van Orden et al. experimented symbol shapes and colors and probed into their influences on searching time [5]. For the design and development of graphical user interfaces, Hackos and Redish (1998) put forward effective user demand analysis methods and user models easily applied by software developers [6]. Also, some researchers, with regard to the specific designs of graphic user interface elements such as window layout, graphic design, pointer design, menu form, color and symbol, etc., proposed instructional design principles and empirical methods. Based on experimental observation, Fleetwood and Byrne found out the factors affecting user’s visual searches, namely, the first factor is the number of icons, the second one is the target region boundary and the last one indicates the quality and definition of icons [7]. Wu et al. brought forward a video image processing method to detect the azimuth and pitch angle of HMDs relative to the fighter and optimize the current angle of HMDs tracking head [8]. Peinecke adopted a multi-sensor information fusion to enhance the visual display effect of HMDs [9]. In order to raise the situational awareness of a helicopter pilot under low visibility situations, Knabl et al. developed a symbolic system suitable for demonstrations, covering obstacles, route information and threat areas [10]. During the HMPP (Human Measures and Performance Project) study, NASA especially focused on the color security and availability design displayed in the various complex graphic interfaces [11]. Yeh and Wickens experimentally studied how to better present information about combats in a chaotic environment [12]. Montgomery and Sorkin made some experimental researches on impacts of brightness on the interface information identification [13]. Tullis and Schum researched the identification efficiency of digital display and graphic display information coding [14, 15]. Monnier made a comparative experiment on colors and positions by the experimental paradigm of digital delay search mission [16].

In accordance with Chinese military standards, navigation standards and USA military standards, etc., HMDs interface is divided into several major areas in this paper. As shown in Fig. 1, extending out from the center, the core area is regarded as an aiming display area, which displays no symbols and icons in principle to avoid sheltering any aiming target; the upper part of this interface is classified as heading indicating area, which is relatively intuitive, so that the pilot can conclude the current heading and target heading information when he or she observes the interface directly; the left side of this interface is classified as speed indicating area; the left side is a height indicating area; the lower part of this interface is an attitude indicating information area. For the division of icons, HMDs interface information layout is divided into 5 parts according to guiding principles for the layout specified in Chinese military standards.

Fig. 1.
figure 1

Typical schematic diagram of HMDs interface

Full size image

Where, area 1 indicates localizer deviation area, area 2 indicates speed indicating area, area 3 indicates attitude and guidance area, area 4 indicates height indicating area and area 5 shows aiming display area. The upper edge and lower center of this interface are white reserve areas (the flight mode announcement area is placed on the upper part of the localizer deviation area, which lies on the upper edge of the whole interface, the lower white reserve area is an attitude indicator presentation area, as shown in Fig. 2).

Fig. 2.
figure 2

Division diagram of HMDs interface layout

Full size image

2 Materials and Methods

2.1 Participants

Twenty subjects (11 males and 9 females) were present undergraduates (n = 6), postgraduate (n = 8) and doctoral candidates (n = 6) from China University of Mining and Technology. They ranged in age from 20 to 35 years, with a mean age of 24 years. They had no color blindness, with the corrected visual acuity over 1.0. They were required to practice and train to know the experimental procedure and operation requirements. Each participant put on electrode cap and sat in a comfortable chair in a soft light and soundproofed room, and eyes gazed at the center of the screen. A 17-in. CRT monitor with a 1024 × 768 pixel resolution was used in the experiment. The distance between participant eyes and the screen was approximately 60 cm, while the horizontal and vertical picture viewing angle was within 2.3°.

2.2 Tasks and Procedures

HMDs interface is divided into 4 areas, including area A, B, C and D, the division basis is as shown in Fig. 3, so as to statistically analyze and number the experimental data.

Fig. 3.
figure 3

Subdivision diagram of HMDs alarm information presentation areas

Full size image

Four areas mentioned above are subdivided, area A is subdivided into 5 parts, which focuses on the interface aiming area; area B is subdivided into 6 parts, which aims at the fighter heading reminder area and secondary center area; area C is subdivided into 4 parts, which mainly considers the presentation position of speed instrument and height meter; area D is subdivided into 4 parts, which mainly considers the division of edge area and secondary center area in the upper visual field displayed by HMDs. Accordingly, there are 19 subareas in total, so as to statistically analyze and number the experimental data.

The experiment is the simulation of four flight missions of the aircraft, which is the experimental factor one. Each state has A/B/C/D large regional variables, which is the experimental factor two. In each group of experiments, warning information was presented randomly in the regional variables. In order to test whether there was any difference among different factors, the factorial experimental design was used. After the start of each group of experiments, the observation tasks at each stage were given to the subjects, such as the take-off stage, which required them to constantly observe the speed, height, load and other information degrees, as shown in Fig. 4. Then the alarm information is presented suddenly at an uncertain time, and the position is random. The subjects are required to accurately find the alarm information and press the response key to enter the detection interface. The subjects need to match the alarm information observed just now, press the A key if it is correct, and press the L key when it’s wrong. There were four sets of experiments, each with 19 small areas, each of which was repeated three times, for a total of 228 times. Through the random arrangement of the order, keep the attention of the subjects. Based on the typical flight state and mission of the aircraft, the experiment is divided into four parts, which are the four typical flight stages of the aircraft, and are carried out in sequence. In each part of the experiment, the subjects read the instructions and started the experiment with any key on the keyboard. Firstly, the center of the screen showed the gaze point “+” 500 ms, and then randomly presented the alarm-free information interface. The subjects were observed according to the task requirements. The alarm information interface will be presented randomly, and the subjects will react, record the reaction time, and enter the matching judgment interface after a delay of 300 ms. After the judgment, the subjects will press “A” and “L”, and the statistical accuracy rate will be correct. After the completion of each part of the experiment, there was a rest time of 2 min. It took about 0.5 h for each person to complete the whole experiment.

Fig. 4.
figure 4

Descriptions of experimental tasks

Full size image

3 Results

The data of accuracy rate and reaction time of experiment on test subjects are statistically analyzed to eliminate extreme data. As for the accuracy rate and reaction time of test subject regarding alarm information for different areas, refer to Figs. 5 and 6.

Fig. 5.
figure 5

Statistics of accuracy rate of test subjects

Full size image
Fig. 6.
figure 6

Statistics of reaction time of test subjects

Full size image

A variance analysis of this accuracy rate (F shows significant difference level and P indicates test level) indicates that alarm information presents insignificant main effects of different areas (F = 0.722, P = 0.203 > 0.05).

As shown in Fig. 6, the variance analysis of reaction time shows that alarm information presents significant main effects of different areas (F = 18.857, P = 0.001 < 0.05). Under the missions for four flight phases of the fighter, obviously, different presentation areas of alarm information show significant effects on the cognitive speed of test subjects; but no significant effects on the visual cognitive capacity and accuracy of the test subjects.

As for the test and analysis of multiple comparisons of the minimum significant difference method for main effects of different alarm information presentation positions on reaction time of test subjects, the result is as shown in Table 1.

Table 1. Test on multiple comparisons of the minimum significant difference method
Full size table

The reaction time of both area A and B has no significant difference; that of area C and D has no significant difference; there are significant differences among area A, B and C; significant differences are available among area B, C and D. The reaction time relationship of four areas is D > C > B > A; the accuracy rate relationship is A > B > C > D. As a result, During the HMDs interface alarm information presentation design, area A more easily attracts the attention of test subjects, while area D uneasily causes the reactions of test subjects. In different flight missions, HMDs interface information that the pilot requires paying attention to is different, some areas, as key areas, may be unfit for presenting alarm information, deep analyses are thereby conducted in this paper according to information presentation requirements for specific flight missions.

3.1 Analysis of Experimental Result for the Takeoff Phase

The statistics of reaction time and accuracy rate of test subjects during the takeoff phase is shown in Fig. 7.

Fig. 7.
figure 7

Statistics of accuracy rate and reaction time of test subjects during the take-off phase

Full size image

As shown in Fig. 7, the accuracy rate relationship of four areas is A > C > B > D; the reaction time relationship is D > C > B > A. During the takeoff phase, the pilot must focus on the speed and height changes. In order to ensure the smooth takeoff of the fighter, the pilot needs to retract the landing gear and flaps at the specified height, adjust the pitch angle and pay attention to the fighter load. It is required to eliminate these necessary information presentation areas and main presentable areas of alarm information of the fighter during the takeoff phase are referred to Fig. 8 (Area No.: A-1, A-2, A-3, A-4, A-5, A-6, B-1, B-2, B-5, B-6).

Fig. 8.
figure 8

Presentable areas of alarm information during the takeoff phase

Full size image

A statistical analysis is conducted for reaction time of test subjects against such 9 subdivision areas, as shown in Fig. 9.

Fig. 9.
figure 9

Reaction time analysis of test subjects against presentable areas during the takeoff phase

Full size image

In the flight mission of fighter during the takeoff phase, according the statistical analysis in Fig. 9, the reaction time of test subjects is the shortest when alarm information is presented in area A-5 while it is the longest when presented in area B-6. As a whole, reaction time of test subjects presenting alarm information in the interface center is generally shorter, which indicates that visual cognitive reactions of test subjects are the most sensitive to the central visual area but comparatively weaker to the upper visual area. During the takeoff phase, the priority ranking of presentation of alarm information for different position areas is as follows: area A-5, area A-1, area A-2, area B-1, area B-2, area A-3, area A- 4. Area B-5, area B-6.

In summary, as shown in Fig. 10, test subjects prefer to choose a priority visualization presentation of information for different position areas during the takeoff phase. The preferred area is expressed by red system, the secondary area is expressed by orange system and the alternative area by yellow system. Apparently, the information presentation position area easily found by test subjects tends to the upper center of HMDs interface during the takeoff phase. As for the selection of important alarm information presentation position during takeoff phase of the fighter, it is necessary to give priority to the upper middle position of this interface, especially area A-5.

Fig. 10.
figure 10

Priority division of presentation areas of alarm information during the takeoff phase (Color figure online)

Full size image

3.2 Analysis of Experimental Result for the Cruise Phase

As for the reaction time and accuracy rate of test subjects during the cruise phase, refer to Fig. 11.

Fig. 11.
figure 11

Statistics of accuracy rate and reaction time of test subjects during the cruise phase

Full size image

As shown in Fig. 11, the accuracy rate relationship of four areas during the cruise phase is A > B > C > D; the reaction time relationship is D > C > A > B. In accordance with the universal display information classification and typical flight mission analysis of fighter, the pilot, during the cruise phase, needs to not only focus on the flight speed, height and attitude of fighter but also adjust information such as fighter heading and height according to specified information so as to reach the specified cruise destination. Therefore, it is required to eliminate these necessary information presentation areas and main presentable areas of alarm information of the fighter during the cruise phase are shown in Fig. 12 (Area No.: A-1, A-2, A-3, A-4, A-5, B-1, B-2, C-4).

Fig. 12.
figure 12

Presentable areas of alarm information during the cruise phase

Full size image

A statistical analysis is conducted for reaction time of test subjects against such 8 subareas, as shown in Fig. 13.

Fig. 13.
figure 13

Analysis of reaction time of test subjects for presentable areas during the cruise phase

Full size image

During the cruise flight mission of the fighter, in accordance with the statistical analysis in Fig. 13, the reaction time of test subjects is the shortest when alarm information is presented in area B-1 while it is the longest when presented in area C-4. As a whole, the reaction time of test subjects presenting alarm information in the upper middle area and the upper left middle area of this interface is generally shorter, which indicates that visual cognitive reactions of test subjects are the most sensitive to the upper center of this interface but comparatively weaker to the lower area. The priority ranking of alarm information presentations of different position areas during the cruise phase is as follows: area B-1, area B-2, area A-1, area A-2, area A-5, area A-3, area A-4, area C-4.

In summary, as shown in Fig. 14, test subjects prefer to choose the priority visualization presentation of information for different position areas during the cruise phase. The preferred area is expressed by red system, the secondary area is expressed by orange system and the alternative area by yellow system. Obviously, the information presentation position area easily found by test subjects tends to the upper left of HMDs interface during the cruise phase. As for the selection of important alarm information presentation position areas during the cruise phase, therefore, it is necessary to give priority to the upper center of this interface, especially area B-1.

Fig. 14.
figure 14

Priority division of presentation areas of alarm information for the cruise phase (Color figure online)

Full size image

3.3 Analysis of Experimental Result for the Combat Phase

As for the statistics of reaction time and accuracy rate of test subjects during the combat phase, refer to Fig. 15.

Fig. 15.
figure 15

Statistics of accuracy rate and reaction time of test subjects during the combat phase

Full size image

As shown in Fig. 15, the accuracy rate relationship of four areas during the combat phase is A > B > C > D; the reaction time relationship is D > C > B > A. In accordance with the universal display information classification and typical flight mission analysis of fighter, HMDs for the combat phase is slightly different from that for other flight phases. During the combat phase, how to find out a hostile plane as early as possible and accurately predict its intention of action for ultimate destruction is an important mission of the pilot, so HMDs interface during this phase mainly displays such information as combat command, weapon status, target information and fighter information. However, the pilot needs real-time control over basic parameters, including flight speed, height and attitude. Therefore, it is required to eliminate these necessary information presentation areas and main presentable areas of alarm information are shown in Fig. 16 (Area No.: B-3, D-1, D-2, D-3, C-4).

Fig. 16.
figure 16

Presentable areas of alarm information for the combat phase

Full size image

As for the statistical analysis of reaction time of test subjects against such five subareas, refer to Fig. 17.

Fig. 17.
figure 17

Analysis of reaction time of test subjects for presentable areas during the combat phase

Full size image

During the combat flight mission of the fighter, according to the statistical analysis in Fig. 17, the reaction time of test subjects is the shortest when alarm information is presented in area B-3 while it is the longest when presented in area D-3. As a whole, the reaction time of test subjects presenting alarm information in the left center and lower left center of this interface is generally shorter, which indicates that the visual cognitive reactions of test subjects are the most sensitive to these areas and comparatively weaker to the edge areas. The priority ranking of different alarm information presentation position areas during the combat phase is as follows: area B-3, area C-4, area D-1, area D-2, and area D-3 (The remaining position areas are not recommended).

In summary, as shown in Fig. 18, test subjects prefer to choose priority visualization presentation of information for different position areas during the combat phase. The preferred area is expressed by red system, the secondary area is expressed by orange system and the alternative area by yellow system. Obviously, considering the pilot needs to aiming at and observing weapon information, the information presentation position area easily found by the test subjects tends to the lower left of HMDs interface during the combat phase. As for the selection of important alarm information presentation positions, therefore, it is necessary to give priority to the lower center of this interface, especially area B-3.

Fig. 18.
figure 18

Priority division of alarm information presentation areas during the combat phase (Color figure online)

Full size image

3.4 Analysis of Experimental Result for the Landing Phase

As for the statistics of reaction time and accuracy rate of test subjects during the landing phase, refer to Fig. 19.

Fig. 19.
figure 19

Statistics of accuracy rate and reaction time of test subjects during the landing phase

Full size image

As shown in Fig. 19, the accuracy rate relationship of four areas during the landing phase is A > C > B > D; the reaction time relationship is D > C > B > A. During the landing phase, the main mission of the pilot is to safely land in the specified time, when the runway benchmark must be displayed except the flight speed, height and attitude. According to rules in Display Symbology for Head up Display of Aircraft, the runway benchmark is used for mobilization and landing modes, which are displayed during use of an instrument landing system and indicate the aiming point and earthing point of the runway. The length, width and angle of symbol form a perspective relationship corresponding to the actual runway. Therefore, it is required to eliminate these necessary information presentation areas and main presentable areas of alarm information of the fighter are shown in Fig. 20 (Area No.: B-1, B-2, B-3, C-3, C-4).

Fig. 20.
figure 20

Presentable areas of alarm information for the landing phase

Full size image

A statistical analysis is conducted for reaction time of test subjects against such five subareas, as shown in Fig. 21.

Fig. 21.
figure 21

Analysis of reaction time of test subjects for the presentable areas during the landing phase

Full size image

During the landing flight mission of the fighter, according to the statistical analysis in Fig. 21, the reaction time of test subjects is the shortest when alarm information is presented in area B-1 while it is the longest when presented in area C-4. As a whole, the reaction time of test subjects presenting alarm information in the left center and upper center of this interface is generally shorter, which indicates that visual cognitive reactions of test subjects are the most sensitive to these areas and comparatively weaker to the left and right edge areas. The priority ranking of different alarm information presentation areas during the landing phase is as follows: area B-1, area B-2, area B-3, area C-3 and area C-4 (other position areas are not recommended).

In conclusion, as shown in Fig. 22, test subjects prefer to choose priority visualization presentation of information for different position areas during the landing phase. The preferred area is expressed by red system, the secondary area is expressed by orange system and the alternative area by yellow system. Obviously, the information presentation position area easily found by test subjects tends to the upper left of HMDs interface during the landing phase. For the selection of important alarm information presentation area during the landing phase, therefore, it is necessary to give priority to the upper left of this interface, especially area B-1.

Fig. 22.
figure 22

Priority division of alarm information presentation areas during the landing phase (Color figure online)

Full size image

4 Discussion

Based on the statistical analysis of the experimental data in three stages of the fighter, the visual perception priority ranking of the specific subdivision region is summarized, and the information presentation requirements in different flight stages are discussed. The preferred, sub-preferred and alternative regions of the alarm information in the three stages of takeoff, cruise and landing are summarized. The experimental results are shown in Table 2.

Table 2. Statistical table of experimental conclusions
Full size table

5 Conclusion

This paper classifies and discusses the general display information of aircraft, summarizes the priority of information presentation of avionics system, and summarizes the information presentation requirements of typical mission phases such as takeoff, cruise and landing. The HMDs information layout interface is divided into 4 large areas and 19 subdivisions according to the experimental requirements. Based on the research basis of flight missions and visual perception cognition, combined with the division of HMDs information layout, the experiment of HMDs interface alarm information distribution was carried out. The experimental data are analyzed in detail, and the preferred, sub-preferred and alternative regions of alarm information in takeoff, cruise and landing phases are summarized. The research results of this paper provide experimental basis for HMDs interface information layout coding, and will effectively improve the efficiency of the system.