Effects of Military Training Based on the Virtual Reality of Army Using AHP Method

In order to enhance the effectiveness of military training, military education authorities are very interested in the use of virtual reality (VR), which is the core technology of the 4th industrial revolution. The purpose of this paper is to analyze effects of the army’s VR-based military training. For this purpose, we draw evaluation factors of AHP through related literature and Delphi technique and divide them into two levels: equipment effects and learning effects. The subfactors of each factors consist that equipment effects are reality, safety, and availability, and learning effects are interest, immersion, and understanding. The questionnaire for evaluation factors was composed of pairwise comparison on 9-point scales. The 14 experts who had experience of teaching the VR-based training participated in a pairwise comparison survey. As a result of this study, the top six factors determined are as follows: interest, reality, immersion, understanding, availability, and safety. The VR-based training showed higher values in term of Interest (6.3), reality (5.2), immersion (3.5), and understanding (3.1) than existing training method of video contents (VC)-based training. On the other hand, availability (0.9) and safety (0.9) were lower. It is expected that the result of this study will be used as the basic data for the military’s VRbased training policy in the future.


Introduction
The actual trainings of Army use to be limit according to the training conditions such as training location, budget, weather, available time, and civil complaints. Therefore, the importance of the scientific training system that enables situations in which actual training is impossible is gradually increasing. For this, military education authorities are very interested in the use of virtual reality (VR) which is the core technology of the 4th industrial revolution to enhance the effectiveness of military training [1].
The VR-based training in the military is to pre-train by making virtual environments similar to real battlefields before soldiers are put into operations. The Army has been conducting VR-based training wearing HMD (Head Mounted Display) since 2019 to train shooting, disassembly, assembly, and manipulation of expensive weapons with conducting in parallel with video contents (VC)-based training that has been applied since 2014. Especially, the VR-based training in the military has been applied to weapon training with high-risk and high-cost. In this study, we analyze how effective VR-based training is compared to video contents-based training, and if so, in what factors. For this, we use the AHP (Analytic Hierarchy Process) method to draw the relative importance of the evaluation factors through pairwise comparison.

Literature Review 2.1. AHP (Analytical Hierarchy Process) Method
The Analytic Hierarchy Process (AHP) is a method to solve by making a complexed hierarchical structure decision problems, using math and psychology. It was developed by Thomas L. Saaty in the 1970s and has been refined since then. AHP contains three parts: the ultimate goal or problem you're trying to solve, all of the possible solutions, called alternatives, and the criteria you will judge the alternatives on. AHP provides a rational framework for a needed decision by quantifying factors and alternative options through pairwise comparisons by the experts, and for relating those elements to the overall goal [2].
Consistency of experts' responses is judged by the Consistency Ratio (CR), and the consistency ratio represents the reliability of responses. If the consistency ratio is lower than 0.1, it can be judged that a good response was given. This quantifying capability distinguishes AHP from other decision-making techniques [3].
In the final step of the process, numerical priorities are calculated for each of the alternative options. These numbers represent the most desired solutions, based on all users' values. All the steps of AHP is shown in Figure 1 [4].

Figure 1. The steps of AHP
As a study related to effect measurement using AHP, Lee [5] drew the major factors for improving the class effect by examining the relative weight of the factors on the class effect of students of business schools in the Korea and America. Kim [6] analyzed the effects of convened education and distance education of the Army through AHP. Kim [7] measured the effectiveness of the improved training method for Army recruits using AHP. Kim [8] verified the effects of Havruta learning method which was known as how to enhance inner motives, creativity, and service values of Army new recruit training. Lee [9] analyzed priorities of the factors affecting the introduction of smart factory using AHP.

VR (Virtual Reality)-based and VC (Video Contents)-based training
Virtual reality is a simulated experience that can be similar to or completely different from real world. Therefore, the application of VR must be included entertainment and educational purposes. The types of virtual reality are divided into immersive virtual reality, simulation-type virtual reality, and display-type virtual reality as shown in Figure 2.
<Immersive> <Simulation-type> <Display-type> Figure 2. The types of VR Immersive virtual reality is a way for users to be actively engaged in virtual reality through special equipment such as HMD (Head Mounted Display). Display-type virtual reality is a type of technology that allows you to experience a virtual world through a display such as a screen or monitor. It is a way to experience virtual reality by using a keyboard, mouse, joystick, or gloves or devices with sensors attached. Simulation-type virtual reality is a form developed by adding virtual reality technology to the simulator, and is used for the purposes of games, as well as specialized fields such as aviation and military fields, and medical care [1].
The earliest and the foremost vital application of VR technology is the military training and is also one among the fields where VR technologies is used at a large extent [10]. Additionally, U.S. Defense department states VR technology is the technology that would bring a change in the ideas and ways to military utility. The use of VR in military training consist of many fields such as virtual training, virtual weapon manufacturing and designing, virtual battleground exercise, maritime application, warfare and also engineering designing [11]. In case of Korea Army, VR-based training in the army is an immersive virtual reality type, and is a training method designed to practice the operation and disassembly of weapons by wearing a HMD. It has been applied since 2019 and is undergoing to use in training and will be applied to various types of military training and gradually expanded.
Video contents (VC)-based training means a training that learns while watching a video on a computer screen for practice such as equipment operation, operation, and maintenance procedures. It is also referred to as a kind of computer-based training (CBT), and it has advantages in terms of space, time, and economy because many people can learn at the same time at low cost, but it has a disadvantage that the learning effect is not high [12]. The Army has been conducting immersive VR-based training with HMDs since 2019 in parallel with VCbased training for expensive weapon training. The main training contents are disassembly and assembly, shooting preparation, and shooting execution. Figure 3 shows the VR-based training and VC-based training.

<VR-based training>
<VC-based training> Figure 3. VR-based and VC-based training As a study on the learning effect using virtual reality, Ahn [13] analyzed the educational effects (understanding, interest, and fun) of VR educational contents for middle school students. He emphasized the realism for fun and interaction for understanding. Jung [14] conducted a comparative study on the effects of virtual reality education and video education through questionnaires. The group who conducted virtual reality education also declared higher educational interests than those who conducted video training. Noh [15] analyzed the effects of virtual reality-based lessons for 6th grade elementary school students on academic achievement, learning interest, and immersion. As a result, it showed virtual reality-based lessons were more effective than general lessons.

Figure 4. The evaluation factors of AHP
The evaluation factors of AHP applied in this study are as shown in Figure 4. Anderson [16] classified factors for measuring educational effectiveness into three factors: cognitive domain (related to "knowing"), psychomotor domain (related to "physical ability"), and affective domain (related to "values") as factors for measuring educational effectiveness. In level 1, learning and equipment effects were chosen. This is because the cognitive domain (learning effects) and psychomotor domain (equipment effects) are the domains related to equipment such as weapons. The lower factors of the learning effects consist of the interest, immersion and understanding, and the lower factors of the equipment effects consist of reality, safety, and availability.
The meaning of the six evaluation factors is as follows. The interest refers to the degree to which fun or concern is induced. The immersion refers to the degree to which you concentrate deeply when learning. The understanding is the degree to which a problem is understood and solved. The reality refers to the degree to which training reflecting the actual situation is possible. The safety refers to the degree of freedom from risks or accidents caused by fear or carelessness when using equipment. The availability refers to the degree of freedom from equipment damage and aging [17,18].
As for the lower level of learning and equipment effects, nine evaluation factors were derived from the aforementioned virtual reality-related literature, and the final six factors were selected through the Delphi technique with expert instructors based on the derived nine evaluation elements. For AHP analysis, SPSS 26.0 and I Make It program were used.

General characteristics of respondents (n=14)
The 14 respondents are professional instructors who have experienced both training methods (VR-based and VC-based training). Respondents are all male, and consist of 5 people in their 30s, 6 people in their 40s, and 3 people in their 50s. The work experiences consist of 4 people under 5 years, 8 people from 6 to 9 years, and 2 people over 10 years, and the ranks consist of 3 officers, 7 warrant officers, and 4 NCOs. Therefore, the composition of the respondents was judged to be reasonably chosen. The details are shown in Table 1.

The weights (priority) of level 1 and level 2
The weights (priority) of level 1 and level 2 are as shown in Table 2. Table 2: The weights (priority) of level 1 and level 2 As for the importance at level 1 of the training effects evaluation factors, it was analyzed that the equipment effects (0.554) is more important than the learning effects (0.446). In the level 2 of the training effects evaluation factors, first, looking at the importance of the learning effects sub-factors, interest (0.423) showed the highest value, followed by immersion (0.316), understanding (0.261). And the consistency ratio (CR) was 0.043. As for the importance of the equipment effects sub-factors, reality (0.575) was the highest value, followed by safety (0.334), availability (0.091). And the consistency ratio (CR) was 0.035.

Relative importance and effects of VR-based and VC-based training
In order to compare the effects of VR-based and VC-based training, first, the relative importance for evaluation factors of level 2 was derived as shown in Table 3  Looking first at the importance of the level 2 evaluation factors for VC-based training, safety (0.514) was the highest, followed by availability (0.502), understanding (0.240), immersion (0.221) and reality (0.160). Interest (0.137) was the lowest. The importance of the level 2 evaluation factors of VR-based training showed that interest (0.863) was the highest, followed by reality (0.840), immersion (0.779), understanding (0.760), and availability (0.498). Safety (0.486) was the lowest.
To calculate each effect index of VR-based and VC-based training, first, the weights of the level 1 and level 2 evaluation factors (Table 2) and the relative importance of the level 2 evaluation factors (Table 3) are multiplied. Assuming that the effect of VC-based training is '1', the relative ratio of how effective VR-based training is as shown in Table 4. The most effective VR-based training compared to VC-based training was the interest (6.3), followed by reality (5.2), immersion (3.5), and understanding (3.1). On the other hand, safety (0.9) and availability (0.9) showed relatively low values.

Conclusion
In this study, we conducted a literature review and examined the effects of VR-based training. Based on this, we drew two major factors and six sub-factors to be applied for the AHP hierarchy questionnaire. Also, the questionnaire results were analyzed by the AHP analysis program (I Make It) to quantify the relative importance of level 1 and level 2 and determine the priority. As a result, equipment effects and learning effects were chosen in level 1 of the AHP. In level 2, the sub-factors of learning effects appeared in the order of interest, immersion, and understanding, and the sub-factors of equipment effects was shown in the order of reality, safety, and availability. Finally, in order to analyze the effects of VR-based and VC-based training, each relative importance was derived, and the effect index was calculated by multiplying it by the weights of the level 1 and level 2 evaluation factors, and then the effect degree through the relative ratio was calculated. The VR-based training was shown effective in terms of interest, reality, immersion, and understanding.
This study is the first study to analyze the effectiveness of virtual reality applied to military training in the army, and it is believed that the basis for applying virtual reality to various military training in the future has been prepared through the research results. In the future study, we need to analyze the difference in satisfaction and academic achievement between virtual reality-based and video contents-based training method for learners.