Development of a high-fidelity fighter 6DoF simulator for immersive education and training

Abstract

Fighter jets play a crucial role in modern combat. However, pilot training on real aircraft is costly and involves considerable risk. Regarding this issue, high-fidelity simulators have attracted increasing attention in recent years and have been widely recognized as an effective approach for reducing training expenditure and enhancing safety. Based on this background, this study develops an F-16 simulator with a particular emphasis on simulation fidelity, which is mainly achieved through two digital twin subsystems proposed in this work. For the first subsystem, a rigorous nonlinear modeling approach and publicly available NASA data are used to construct an accurate F-16 flight dynamics model. For the second subsystem, Unreal Engine (UE) and Cesium are employed to provide a highly realistic visualization of the aircraft and the virtual world. A general framework is further proposed to efficiently integrate both subsystems and several essential interfaces, including hardware joysticks and virtual reality (VR) devices, thereby providing a more comprehensive flight experience. In addition, a dedicated logging module is incorporated to enable seamless integration with the well-known analysis tool Tacview for post-mission review and debriefing. Finally, a simplified cannon-shooting training scenario is implemented, and the results indicate that the proposed simulator achieves a high level of realism and immersion.

Keywords

Fighter simulator 6DoF flight dynamics flight control virtual reality Unreal Engine

1. Introduction

The presence of fighter jets is reshaping our conventional understanding of military doctrines.¹ These powerful aircrafts, known for their efficiency and flexibility in combat, often serve as the primary offensive force in tactical planning. Consequently, their value in defense applications is widely recognized. However, the associated costs are extremely high. Among these, the most well-known is the expense of pilot training, which can sometimes even exceed the hardware cost of the aircraft itself. At the same time, as related development technologies continue to evolve, fighter jets now carry an increasing number of advanced onboard systems to meet the diverse requirements of modern combat. These advanced hardware and software systems make flight operations more complex and further increase the difficulty of pilot training. In addition, flight training also places a strong emphasis on the pilot’s response to dangerous situations, which often requires a great deal of practice to ensure that the pilot can react correctly in real-life emergencies. However, due to safety concerns, the relevant training will be adapted or simplified in the real world, which will reduce the pilots’ response sensitivity. Because of the physical limits of both aircraft and pilots, it is also difficult to repeatedly conduct real-world flight training under specific conditions, such as flying through clouds, performing instrument flight in severe weather, chasing moving targets, or evading missile lock. In summary, although live flight training for pilots is essential and cannot be fully replaced, it is also associated with high costs and significant risks, and its training efficiency is not always ideal. Therefore, there is a clear need for an auxiliary training method that helps reduce these costs while improving overall training efficiency. Among the various auxiliary training approaches, high-fidelity flight simulators have received growing attention in recent years. For example, in modern defense training architectures such as Live, Virtual, and Constructive (LVC), these realistic simulators serve as core components and play a key role in the overall training process.^2,3 Moreover, many studies^4–7 have revealed that introducing simulators into the training process can provide substantial benefits in reducing costs and risks and ensure training effectiveness at the same time.

In recent years, simulators for training purposes have developed rapidly. To provide a higher level of immersion, simulators are often integrated with virtual reality (VR) devices, and related research and practical applications have attracted considerable attention.^8–11 In flight-training scenarios, such systems can offer users a more comprehensive view and experience, which can help improve training effectiveness.^4,12 Furthermore, when mixed reality (MR) technology is integrated into simulators, it can help reduce cybersickness and improve the quality of interaction between users and physical hardware in the real world.^13,14 Such immersive simulators can also be used for tactical and medical training,^15–18 providing concrete and practical support in many other domains.^19–21 In fact, simulators are not limited to training purposes. Taking reinforcement learning, a field that has become very popular in recent years, as an example, one key step is to use appropriate high-precision modeling methods to construct a detailed model of how an agent interacts with its environment. In this way, we can build a virtual environment that supports a large number of repeated trials and training, which can essentially be regarded as a type of simulator.^22–24 Such a simulator helps avoid the low efficiency and high cost that may arise when reinforcement learning algorithms are developed and tested directly on real systems.

Regarding this study’s focus on supporting fighter pilot training in defense applications, our main concern is how realistic the simulator can be. If the level of realism is insufficient, the gap between the simulator and actual flight becomes larger, which may reduce training effectiveness. Actually, the simulator’s realism is dominated by two main aspects. The first is the establishment and validation of a high-precision flight dynamics model, so that the aircraft’s response can be accurately reproduced. The second is the presentation of a highly realistic virtual environment. In this study, these two aspects are directly reflected in the two proposed subsystems based on the digital twin concept.^25–28

The first digital twin subsystem aims to accurately establish the mapping relationship between command inputs and the fighter aircraft’s dynamic responses. In this study, a high-precision nonlinear modeling method,^29,30 combined with parameters given in the NASA technical report, is used to construct an accurate dynamic model of the F-16 fighter and its corresponding angular velocity controller in MATLAB. The responses of this model are then examined through multiple simulated flight maneuvers to confirm its validity. In this way, the model can more accurately capture the aircraft’s behavior under different flight conditions than oversimplified models. As for the second digital twin subsystem, it focuses on reconstructing a highly realistic environment. Unreal Engine (UE), which is known for its high-fidelity rendering capability, is adopted as the core of the proposed simulator, and Cesium is also integrated to provide a high-resolution global map in real time, thereby creating a virtual environment that closely matches real flight scenarios. Furthermore, this study proposes a general framework based on C++, deeply integrating these two digital twin subsystems in a modular way. MATLAB Coder is used to convert the first digital twin subsystem into C++ code, which is then encapsulated as a DLL file and seamlessly integrated into the proposed framework as the dynamics module. Within this framework, both hardware joystick input and VR display are also well organized. Joystick signals are sent to the dynamics module described above for real-time flight state computation, and the rendered views are streamed to VR devices, providing users with a panoramic view for a more realistic flight experience. In addition, this study implements a logging module within the proposed framework to record flight states, enabling the simulator to integrate with the well-known flight analysis tool Tacview for post-mission review and debriefing. Thanks to the modular design of the proposed framework, a high-fidelity simulator for a new fighter jet or other vehicle can be built rapidly by just defining the corresponding dynamic model and visualization components within it, which further demonstrates the framework’s portability.

The remainder of this article is organized as follows: Section 2 describes the F-16 flight dynamics modeling and the corresponding control law design. Section 3 presents the validation results of the proposed model. Section 4 introduces the high-fidelity virtual environment built upon UE and the overall integration framework. Finally, Section 5 presents the conclusions and future work.

2. Flight dynamics modeling and control design

To achieve a high-fidelity flight experience, one key design task is to develop a complete mathematical dynamic model of the fighter aircraft, which serves as the basis of the first digital twin subsystem in this work. In this section, the primary objective is to build an angular velocity control system for the fighter. The control system consists of two main parts: system modeling and controller design. First, system modeling constructs a mathematical model that accurately describes the behavior of the aircraft, which provides the basis for controller design and strongly affects the realism of the flight experience. Second, based on this aircraft model, a nonlinear angular velocity controller is designed to achieve the angular velocity targets specified by the pilot’s joystick input.

Figure 1 shows the structure of the angular velocity control framework. The angular velocity command $ω_{c}$ is given by the pilot or trainee through the joystick. The angular velocity controller then generates the desired torques $τ_{d}$ . For the F-16 6DoF fighter, the control inputs to the aircraft model include three control surfaces (aileron, elevator, and rudder) and one throttle. The deflections of these control surfaces are used to compute the aerodynamic forces and moments based on the aerodynamic model. To generate the desired moment, the system applies the control allocation law to compute the required control surface deflections $u$ . To further simulate the actuator behavior of a real F-16, the computed control surface deflections are passed through a first-order actuator model with saturation. The actuator bandwidth is selected based on actual F-16 behavior, so that both the physical limits and the transient characteristics of the control surfaces can be represented. This design ensures that the closed-loop simulation considers both the desired control surface command $u$ and the actual feasible control surface command $u_{feasible}$ , allowing the resulting 6DoF motion of the F-16 fighter to more closely reflect real operational behavior.

Figure 1.

F-16 angular velocity control framework.

To model the aircraft precisely, the F-16’s aerodynamic model, engine model, and atmosphere model provided in NASA Technical Report TP-1538³¹ are used in this study to ensure the accuracy of the simulation. Fighter F-16 was chosen as an example model because the data are sufficient and publicly available, allowing this study to focus on the design of a framework for the development of a high-fidelity simulator platform that is highly scalable and can be easily applied to other aircraft types in the future.

The rigid-body dynamics model encompasses both translational and rotational motions, which will be described by Newton’s second law and Euler’s equations, respectively. Moreover, unlike typical ground vehicles, fighter jets often execute aggressive maneuvers in the air. Therefore, aerodynamic forces and moments must be considered in the model to accurately represent the vehicle’s motion.^32,33 For fighter aircraft, force equations are typically formulated in the wind-axis coordinate system, and this modeling choice is especially important when flying at a high angle of attack while simultaneously executing rolling maneuvers.^34,35

2.1. Fighter aircraft equations of motion

To exhibit the behavior of fighters in the simulator, the aircraft states must be obtained by the dynamics model. The aircraft states include position $P$ , airspeed $V_{T}$ , angle of attack $α$ , sideslip angle $β$ , heading angle $χ$ , flight path angle $γ$ , and velocity roll angle $μ$ . The governing equations can be divided into translational and rotational motions.^32–35 The equations of translational motion include (a) the position state equation and (b) the airspeed state equation. The equations of rotational motion include (c) the heading angle state equation, (d) the flight path angle state equation, and (e) the velocity roll angle state equation. For details about the theoretical derivations of the equations of motion, please refer to Appendix 1.

2.1.1. Position state equation

The position vector $P$ can be obtained by

\overset{\cdot}{P} = C_{G / W} V^{W}

(1)

where $C_{G / W}$ is the rotation matrix of the wind-axis rotating to the global frame, and the rotation sequence is as follows $(χ \to γ \to μ)$ ; $V^{W}$ is velocity vectors in the wind axis, where $V^{W} = {[\begin{matrix} V_{T} & 0 & 0 \end{matrix}]}^{T}$ .

2.1.2. Airspeed state equation

The airspeed $V_{T}$ can be obtained by

{\overset{\cdot}{V}}_{T} = \frac{1}{m} (- mg \sin γ - D + T_{x} \cos α \cos β)

(2)

where $m$ is the aircraft mass; $g$ is gravitational acceleration; $D$ is the air drag force aligned with the direction $- x^{W}$ ; and $T_{x}$ is the thrust aligned with the direction $x^{B}$ . The engine thrust is calculated from the engine model whose input is the throttle position. The Mach number can be computed from the airspeed state using the following equation:

Mach = \frac{V_{T}}{\sqrt{1.4 \times 1716.3 \times T}}

(3)

where temperature $T$ (in Kelvin) can be computed from the altitude $h$ as follows:

T = {\begin{matrix} 390 & h \geq 35000 ft \\ [1 - (0.703 \times 10^{- 5} \times h)] \times 519 & h < 35000 ft \end{matrix}

(4)

2.1.3. Heading angle state equation

The heading angle $χ$ can be obtained by

\begin{matrix} \overset{\cdot}{χ} = \frac{(- C \cos μ + L \sin μ)}{m V_{T} \cos γ} \\ + \frac{T_{x} (\sin α \sin μ - \cos α \cos μ \sin β)}{m V_{T} \cos γ} \end{matrix}

(5)

where $L$ is the lift force aligned with the direction $- z^{W}$ ; $C$ is the side force aligned with the direction $- y^{W}$ . $D$ , $L$ , and $C$ can be calculated by the aerodynamic model whose inputs are the control surface deflection angles.

2.1.4. Flight path angle state equation

The flight path angle $γ$ can be obtained by

\begin{matrix} \overset{\cdot}{γ} = \frac{(- mg \cos γ + L \cos μ + C \sin μ)}{m V_{T}} \\ + \frac{T_{x} (\cos μ \sin α + \cos α \sin β \sin μ)}{m V_{T}} \end{matrix}

(6)

The above state equations for airspeed, heading, and flight path angle can be proved by applying the force equations in the wind-axis frame. As for the state equations for velocity roll, angle of attack, and sideslip angle, they can be derived using the angular velocity relationship between the global and body frames. All of these derivations are provided in detail in Appendix 1.

2.1.5. Velocity roll angle state equation

The velocity roll angle $μ$ can be obtained by

\begin{matrix} \overset{\cdot}{μ} = \frac{(ω_{x} \cos α + ω_{z} \sin α + \overset{\cdot}{χ} \cos β \sin γ)}{\cos β} \\ + \frac{(\overset{\cdot}{γ} \cos μ \sin β + \overset{\cdot}{χ} \cos γ \sin β \sin μ)}{\cos β} \end{matrix}

(7)

where $ω_{x}$ is angular velocity aligned with the direction $x^{B}$ ; $ω_{z}$ is angular velocity aligned with the direction $z^{B}$ . $\overset{\cdot}{χ}$ and $\overset{\cdot}{γ}$ are the time derivatives of the heading angle and the flight path angle, respectively, as described in Equations (5) and (6).

2.1.6. Angle of attack state equation

The angle of attack $α$ can be obtained by

\begin{matrix} \overset{\cdot}{α} = - \frac{\overset{\cdot}{γ} \cos μ - ω_{y} \cos β + \overset{\cdot}{χ} \cos γ \sin μ}{\cos γ} \\ - \frac{ω_{x} \cos α \sin β + ω_{z} \sin α \sin β}{\cos γ} \end{matrix}

(8)

where $ω_{y}$ is the angular velocity aligned with the direction $y^{B}$ .

2.1.7. Sideslip angle state equation

The sideslip angle $β$ can be obtained by

\overset{\cdot}{β} = ω_{x} \sin (α) - \overset{\cdot}{γ} \sin (μ) - ω_{z} \cos (α) + \overset{\cdot}{χ} \cos (γ) \cos (μ)

(9)

The inputs to the state equations above include the control surface deflection angles and the throttle setting. The corresponding aerodynamic and engine models are then used to compute the aerodynamic forces and engine thrust. Using Equations (1)–(9), the position of the aircraft in the global frame can be obtained.

2.2. Attitude state equations

The aircraft attitude will be represented with the quaternion $Q = {[\begin{matrix} q_{0} & q \end{matrix}]}^{T}$ . Quaternion is a method of describing attitude that helps to overcome problems that can occur when describing attitude using Euler angles, such as gimbal lock. A detailed description can be found in the work by Šenk and Cheze.³⁶ The aircraft attitude in the global frame can be obtained from the attitude state equations given below. Proofs of these equations are available in the works by Stevens et al.³³ and Dam et al.³⁷

2.2.1. Quaternion state equation

The quaternion $Q$ can be obtained by

\overset{\cdot}{Q} = [\begin{matrix} - \frac{1}{2} q^{T} ω \\ \frac{1}{2} (q_{0} I_{3} + q^{\times}) ω \end{matrix}]

(10)

where $I_{3}$ is the $3 \times 3$ identity matrix; $q^{\times}$ is the cross-product matrix whose general formula is defined as:

S (q) = q^{\times} = [\begin{matrix} 0 & - q_{3} & q_{2} \\ q_{3} & 0 & - q_{1} \\ - q_{2} & q_{1} & 0 \end{matrix}]

(11)

2.2.2. Angular velocity state equation

The angular velocity vector $ω$ can be obtained by

\overset{\cdot}{ω} = J^{- 1} (τ - (ω^{\times} J ω))

(12)

where $ω^{\times}$ is the cross-product matrix with the same form shown in Equation (11); $τ$ is the moment acting on the body frame, computed using the aerodynamic model with the control surface deflection angles as inputs; $J$ denotes the moment of inertia matrix.

2.3. Design of angular velocity controller

It is known that the commands from the pilot’s joystick are angular velocities, and in order for the aircraft to track these angular velocity commands, it is necessary to build a closed-loop control system and the corresponding angular velocity controller, as shown in Figure 1. With this controller, the aircraft can achieve stable and accurate angular velocity tracking, enabling it to execute the planned flight maneuvers.

First, establish the error equation for angular velocity:

e_{1} = ω - ω_{c}

(13)

where $ω_{c}$ is the desired angular velocity, and $ω$ is the angular velocity from the feedback state. The controller is designed to drive $e_{1}$ to zero based on the feedback linearization and feed-forward control methods. First, taking the time derivative of Equation (13) and substituting $\overset{\cdot}{ω}$ from Equation (12) into the above equation yields:

{\overset{\cdot}{e}}_{1} = \overset{\cdot}{ω} - {\overset{\cdot}{ω}}_{c} = J^{- 1} (τ - (ω^{\times} J ω)) - {\overset{\cdot}{ω}}_{c}

(14)

The above equation is an error state equation, which can be transformed into a linear form by designing an appropriate control law. Once it is converted into a linear form, the system analysis and controller design become much simpler. Therefore, in this study, the control law is designed as follows:

τ_{c} = (ω^{\times} J ω) + J ({\overset{\cdot}{ω}}_{c} - K_{p} e_{1} - K_{i} \int_{0}^{t} e_{1} (ε) d ε)

(15)

This control law includes the nonlinear feedback compensation, command feed-forward, and the proportional–integral (PI) controller, where $K_{p}$ and $K_{i}$ are positive control gain matrices, and their diagonal entries can be tuned separately along each axis. Substituting the control law (15) into Equation (14) simplifies the error state equation to a linear equation. Furthermore, an extended virtual state variable $e_{0} = \int_{0}^{t} e_{1} (ε) d ε$ is defined, which yields the following closed-loop error dynamics:

{\overset{\cdot}{e}}_{c} = [\begin{matrix} {\overset{\cdot}{e}}_{0} \\ {\overset{\cdot}{e}}_{1} \end{matrix}] = [\begin{matrix} 0_{3 \times 3} & I_{3 \times 3} \\ - K_{i} & - K_{p} \end{matrix}] [\begin{matrix} e_{0} \\ e_{1} \end{matrix}] = A_{c} e_{c}

(16)

After designing the corresponding control law, the closed-loop stability must be analyzed. In this study, the stability is ensured by designing the control gain $A_{c}$ as a Hurwitz matrix. Since $A_{c}$ is a Hurwitz matrix, there exist matrices $P^{T} = P > 0$ and $Q^{T} = Q > 0$ such that $A_{c}^{T} P + P A_{c} = - Q$ is satisfied. The Lyapunov function is selected as $V_{c} = e_{c}^{T} P e_{c}$ , and differentiating $V_{c}$ yields:

\begin{matrix} {\overset{\cdot}{V}}_{c} = {\overset{\cdot}{e}}_{c}^{T} P e_{c} + e_{c}^{T} P {\overset{\cdot}{e}}_{c} = e_{c}^{T} (A_{c}^{T} P + P A_{c}) e_{c} \\ = - e_{c}^{T} Q e_{c} \leq 0 for \forall e_{c} \notin 0 \end{matrix}

(17)

From the above equation, the closed-loop system is guaranteed to be asymptotically stable as long as $A_{c}$ is designed to be a Hurwitz matrix.³⁸

2.4. Fighter control surface allocation

Based on the angular velocity controller design described above, the controller computes the moments required to achieve the desired angular velocity. To realize the desired moments, the aircraft must actuate its control surfaces. Therefore, this study introduces a control allocation law that maps the desired moments to the deflection angle commands of each control surface. In addition, actuator saturation is taken into account, as shown in Figure 1, so that the generated deflection commands are more consistent with real flight conditions.

The aerodynamic moments in the direction $x^{B}$ and $z^{B}$ can be expressed as the following equation:

\hat{τ} = [\begin{matrix} τ_{x} \\ τ_{z} \end{matrix}] = f (x, δ_{e}) + g (x) u

(18)

where $u = [\begin{matrix} δ_{a} & δ_{r} \end{matrix}]$ denotes the deflection angle commands of the aileron and rudder, respectively. Moreover, $f (x, δ_{e})$ and $g (x)$ are all made up of complex aerodynamic parameters which can be obtained by looking up the aerodynamic parameter table according to the current state and elevator deflection angle. Note that based on the properties of the aircraft, the aerodynamic moment in the $y^{B}$ direction cannot be expressed by an affined system. Therefore, the non-affined one is expressed by:

τ_{y} = \tilde{f} (x, δ_{e})

(19)

where $\tilde{f} (x, δ_{e})$ is also composed of multiple aerodynamic parameters which can be obtained by looking up the aerodynamic parameter table according to the current state and elevator deflection angle, similar to $f (x, δ_{e})$ and $g (x)$ .

To solve for the elevator deflection angle command independently, the bisection method is used. For Equation (19), define an error function:

e_{m} (δ_{e}) = τ_{y} (δ_{e}) - τ_{yd}

(20)

where $τ_{yd}$ represents the desired moment generated by the controller shown in Equation (15). Then, the elevator deflection angle can be obtained by numerical superposition, so that the value of the error in Equation (20) can be suppressed less than a small value $ε$ :

| e_{m} (δ_{e}) | < ε

(21)

After obtaining the elevator deflection angle command, substitute it into the following equation to solve for the deflection angle commands of aileron and rudder.

u = g^{- 1} (x) ({\hat{τ}}_{d} - f (x, δ_{e}))

(22)

Once the control surface deflection angles are all obtained from the control allocation law, they are sent to the corresponding actuator models with saturation. The resulting constrained outputs are then fed into the fighter’s 6DoF dynamic model to generate realistic simulated flight states.

3. Simulation results and evaluation

Following the controller design described above, the simulation results presented below focus on examining whether the relationships among the desired command inputs, the angular velocity control responses, and the corresponding control surface deflections are reasonable. As mentioned above, to make the simulation more realistic, this study further incorporates the actuator dynamics of the control surfaces,³¹ as shown in the following equation.

δ_{transient} = \frac{20.202}{s + 20.202} δ

(23)

where $δ_{transient}$ represents the actual deflection angle of the control surface, and $δ$ denotes the commanded deflection angle obtained from the control allocation algorithm.

Since fighter aircraft often executes high-bandwidth, aggressive maneuvers, an insufficient sampling rate may distort the model response and degrade model fidelity. To avoid this, the simulated states are propagated at 1 kHz using a classical fourth-order Runge–Kutta (RK4) scheme.

3.1. F-16 pure rolling maneuver simulation

Rolling maneuvers are an important part of high-agility flight. In this simulation test, a 180 deg/s roll rate command is applied as an extreme test condition, while the pitch and yaw rate commands are kept at zero. Figure 2(a) shows the overall flight simulation, where the blue arrows indicate the velocity direction of the aircraft. In Figure 2(b), the red curve denotes the commanded roll rate, and the blue curve shows the actual roll rate response of the aircraft. Figure 2(c) presents the responses of the control surfaces, in which the red curves represent the desired deflection angles, and the blue curves indicate the actual deflection angles after considering the actuator dynamics. From these results, it can be clearly seen that the desired roll rate in Figure 2(b) is well tracked, which confirms the effectiveness of the controller design in this study. Because the actuator dynamics are included, the transient response of the control surfaces prevents the desired moments from being realized immediately, leading to small oscillations in the control results. Nevertheless, the corresponding tracking errors remain small and within an acceptable range.

Figure 2.

Simulation results for a pure rolling maneuver. (a) Flight trajectory responses. (b) Angular velocity responses. (c) Control surface responses.

3.2. F-16 pure pitching maneuver simulation

Following the pure rolling maneuver simulation, a pure pitching maneuver is then considered. To examine whether the model response remains reasonable under extreme conditions, this test applies a fairly aggressive 20 deg/s pitch rate command, while the other commands are kept at zero. Figure 3(a)–(c) shows the corresponding simulation results. As seen in Figure 3(b), the model tracks the pitch rate command closely, and some oscillations appear in the responses as well, arising from the same cause as in the previous case.

Figure 3.

Simulation results for a pure pitching maneuver. (a) Flight trajectory responses. (b) Angular velocity responses. (c) Control surface responses.

In an actual pure pitching maneuver, a stall can be triggered quite easily. To observe this behavior more clearly, this test, additionally presents the time responses of several key state variables, including airspeed, angle of attack, and sideslip angle, as shown in Figure 4(a) and (b). From these figures, it can be observed that the angle of attack becomes very large and the airspeed drops sharply at around 10 s. This rapid loss of speed corresponds to a large increase in drag and is a typical sign of stall. Based on the above analysis, the results also reveal that the proposed F-16 dynamic model can accurately capture the aircraft’s response under extreme flight conditions, including near-stall behavior.

Figure 4.

Responses of airspeed, angle of attack, and sideslip angle for the pure pitching maneuver test. (a) Airspeed responses. (b) Angle of attack and sideslip angle responses.

3.3. F-16 pure yawing maneuver simulation

In this test, a pure yawing maneuver is simulated by applying a 3 deg/s yaw rate command, with the other commands kept at zero. The corresponding simulation results are shown in Figure 5(a)–(c). As shown in Figure 5(b), the aircraft is able to track the yaw rate command, while small tracking errors still remain. At the same time, Figure 5(c) shows that the rudder deflection is saturated between about 15 and 30 s. Overall, these results clearly indicate that a 3 deg/s pure yaw rate is the limit that this aircraft can achieve. It is also worth noting that, to generate a positive yaw rate response, the rudder deflection should be negative. However, the resulting yawing motion also induces an additional positive rolling moment. To maintain a pure yawing motion, the flight controller, therefore, applies a positive aileron deflection to cancel this accompanying effect. These behaviors are consistent with the simulation results shown in Figure 5(c) and further demonstrate that the control allocation algorithm used in this study is accurate and reliable.

Figure 5.

Simulation results for a pure yawing maneuver. (a) Flight trajectory responses. (b) Angular velocity responses. (c) Control surface responses.

3.4. F-16 figure-eight flight maneuver simulation

In actual flight, most high-agility maneuvers are achieved by applying roll rate and pitch rate commands simultaneously. Following the previous single-axis tests, both roll rate and pitch rate commands are set as shown in Figure 6(b), which corresponds to a figure-eight flight trajectory. The simulation results are shown in Figure 6(a)–(c). It can be seen that the aircraft tracks the commands quite well, and the tracking errors remain within an acceptable range. In addition, Figure 6(c) shows that no saturation occurs in the control surface deflections. These observations show that the proposed model has good tracking performance under a scenario that matches typical fighter operations.

Figure 6.

Simulation results for a figure-eight flight maneuver. (a) Flight trajectory responses. (b) Angular velocity responses. (c) Control surface responses.

In this section, the proposed F-16 fighter dynamic model is validated through several tests. The results confirm that the model can accurately capture various physical properties of the real aircraft and can, therefore, serve as a solid foundation for the high-fidelity simulator emphasized in this study. With this, the first digital twin subsystem, which is responsible for describing the aircraft’s dynamic behavior, has been completed. The next section will focus on how the second digital twin subsystem and the proposed framework handle high-fidelity visual presentation and the modular integration of these two subsystems.

4. High-fidelity visual rendering using Unreal Engine

This section focuses on the second digital twin subsystem and the overall integration framework. A key issue here is that the choice of base platform strongly affects how the development workflow is organized and the final simulator’s performance. Therefore, before selecting a suitable platform, this study first identifies the relevant requirements, which are summarized as follows.

High-Efficiency Execution: The flight dynamics model requires strict real-time performance, and an insufficient sampling rate may degrade the accuracy of the aircraft response. Therefore, the target platform is expected to support C-based languages, which offer higher execution efficiency.

Hardware Joystick Interface: The platform must support a hardware joystick so that operator commands can be smoothly passed to the F-16 dynamic model.

VR Devices Support: In real missions, fighter pilots regularly scan the surroundings to obtain a panoramic view, which cannot be achieved with a conventional 2D display. Therefore, the platform should support VR devices to provide such an experience.

High-Fidelity Visual Rendering: To ensure the realism and immersion of the simulator, the platform should be equipped with realistic visual rendering capabilities.

Finally, this study adopts UE, a game engine based on C++, as the primary development platform, which aligns well with the above specifications. UE is well-known for its powerful rendering capability,^39,40 which is highly consistent with the high-fidelity objective pursued in this study. It has been widely used in simulators that demand highly realistic visual effects, such as the well-known AirSim⁴¹ and CARLA⁴² simulators, which are commonly used in autonomous driving research, as shown in Figure 7. In summary, by leveraging UE’s powerful capabilities, the quality of the visual presentation and the subsequent integration process can be strongly guaranteed, thereby further enhancing the overall realism and immersion of the resulting simulator. Following the above discussion on UE, the next section will present the overall framework design, which integrates the two digital twin subsystems and all required interfaces.

Figure 7.

Well-known simulators based on UE. (a) AirSim simulator. (b) CARLA simulator.

4.1. Simulator framework overview

Figure 8 illustrates the overall simulator framework used in this study. The framework integrates the two proposed digital twin subsystems. The first subsystem focuses on the highly nonlinear dynamics of the F-16, as discussed in the previous section, while the second subsystem is responsible for establishing and rendering highly realistic scenes. Within this framework, the dynamics model is first constructed and validated in MATLAB and then converted into C++ using MATLAB Coder. This conversion facilitates seamless integration with UE as the dynamics module and ensures reliable real-time execution. To follow a modular design, the generated C++ dynamics functions are further encapsulated into a DLL file, which greatly improves maintainability and extensibility. Using the Windows RawInput plugin, hardware joystick inputs (angular velocity commands and throttle setting) are captured and sent to the dynamics module for high-precision simulation of aircraft pose and control surface deflections. Based on these simulated states, the UStaticMeshComponent associated with the fighter APawn instance is updated to animate the corresponding aircraft motion, while the motion of the in-cockpit control set (stick, throttle, and rudder pedals) is rendered directly from the raw joystick signals. In addition, UCameraComponent instances are mounted both inside the cockpit and outside the aircraft to provide first-person and third-person perspectives. These views are then rendered and streamed to the VR headset via Meta Quest Link, while global terrain is supplied in real time using the Cesium for Unreal plugin, resulting in a highly immersive flight experience. A dedicated flight logging module is also included in the framework to export flight data in a specific CSV format, enabling seamless integration with the well-known analysis tool Tacview for post-mission review and debriefing. Owing to the modular design of the proposed framework, the system achieves strong portability and extensibility, which will be demonstrated through the validation results in the following section. Finally, although flight evaluation is not the main focus of this study, a custom scoring module can be incorporated in the future due to the framework’s strong extensibility. As indicated by the dashed box in Figure 8, this scoring module is an optional, scenario-dependent component and is shown only at the conceptual level to complete the overall framework illustration.

Figure 8.

The proposed development framework.

4.2. High-fidelity scene construction

The second digital twin subsystem is dedicated to constructing a realistic scene that includes the F-16 fighter jet and the virtual environment. Within the UE development workflow, the visualization of the virtual fighter is implemented as an APawn instance. To animate the aircraft’s motion, we first define the hierarchical relationships among the static mesh components in this APawn to establish the correct transformation chain between meshes. These meshes are organized and edited through Blueprints, as shown in Figure 9(a). When the dynamics module outputs the latest aircraft states, including the pose and control surface deflection angles, the position and orientation of the fighter APawn are updated accordingly, and the associated control surface meshes are animated to reflect the corresponding deflections, as illustrated in Figure 9(b)–(d).

Figure 9.

Control surface deflections of the virtual F-16 fighter jet. (a) Editing the control surface meshes via Blueprints. (b) Elevator deflection. (c) Aileron deflection. (d) Rudder deflection.

Inspired by flyscript,⁴³ the UChildActorComponent class is employed to enhance the maintainability of the system. The use of UChildActorComponent enables the complex visualization components to be developed and validated independently. Following this design principle, both the cockpit and nozzle visualization components are constructed and attached to the fighter APawn. In the virtual cockpit, the corresponding joystick is animated according to the raw command inputs from the hardware joystick, as shown in Figure 10, using the same mechanism as the control surface animation described above. For the nozzle visualization component, the implementation is adapted from flyscript.⁴³ The nozzle expands and contracts according to the throttle setting and triggers the afterburner flame effect with shock diamonds when the throttle exceeds 0.77, as shown in Figure 11.

Figure 10.

Virtual F-16 cockpit (F-16A block 15 cockpit mesh is taken from CGTrader⁴⁴).

Figure 11.

Animation of the virtual nozzle. (a) Before activating the afterburner. (b) After activating the afterburner.

In addition to the F-16 fighter jet, constructing realistic surroundings is a key element of the second digital twin subsystem. In this study, Cesium is adopted to provide a high-resolution global map, which can be easily integrated into the UE project by installing the Cesium for Unreal plugin. It is worth noting that Cesium supports various realistic lighting conditions and flexible initial position configuration, as illustrated in Figures 12 and 13, respectively. These features are highly beneficial for creating diverse mission scenarios for pilot training.

Figure 12.

Different lighting conditions generated in Cesium, with the camera located at 585 m above the Department of Aeronautics and Astronautics, National Cheng Kung University, Tainan, Taiwan. (a) 7:00 AM. (b) 12:00 PM. (c) 6:40 PM. (d) 7:00 PM.

Figure 13.

Different initial aircraft position settings in Cesium. (a) Tainan, Taiwan. (b) Taipei, Taiwan.

This study, in addition, uses the volumetric cloud system provided by UE to render various types of clouds, as shown in Figure 14(b) and (c). In this way, typical scenarios involving a fighter flying above the clouds can be realistically simulated, as illustrated in Figure 14(a).

Figure 14.

Cloud creation and configuration using the volumetric cloud system. (a) Virtual fighter jet flying above the clouds in the proposed simulator. (b) Thin clouds. (c) Dense clouds.

To further verify the potential of the proposed simulator to support various customized training tasks, this study, in addition, develops a simplified virtual cannon-shooting scenario for training pilots in basic aircraft control and gun-aiming skills. In this scenario, virtual cannon bullets are created using the A Actor instances, and mesh-overlap detection is employed to determine whether each bullet collides with any object. Once a collision is detected, the simulator triggers the corresponding impact effect, as illustrated in Figure 15(a). Multiple targets are randomly placed, and each target is programmed to explode and be removed from the scene after it has received 30 bullet hits. Figure 15(b) illustrates the operator performing the resulting virtual training scenario with a joystick and a VR device, further confirming the simulator’s flexibility in supporting customized training tasks. This demonstration also clearly shows that the two proposed digital twin subsystems and the integration framework can indeed create a highly realistic flight-training environment.

Figure 15.

Constructing a simplified virtual cannon-shooting scenario in the proposed simulator to verify its flexibility in creating customized training tasks. (a) Bullet impact effects triggered by enabling mesh-overlap detection. (b) Virtual training in a cannon-shooting scenario using a joystick and a VR device.

Besides, a practical simulator should record flight states for subsequent analysis and debriefing, aligning with real training practice. Accordingly, our framework incorporates a dedicated logging module, as shown in Figure 8. This module exports flight data in a specific CSV format, thereby enabling seamless integration with Tacview, a well-known flight analysis tool. With Tacview, we can replay full missions, inspect trajectories from multiple viewpoints, and pause at any time for detailed review, as illustrated in Figure 16.

Figure 16.

Mission replay and debriefing via Tacview integration. (a) Flight Trial 1. (b) Flight Trial 2. (c) Flight Trial 3. (d) Flight Trial 4.

In summary, the above results show how we use UE to build the second digital twin subsystem, which focuses on creating highly realistic environments and user interaction. These results also indicate that the proposed simulator achieves a high level of realism and can provide users with an immersive and high-quality flight-training experience. These high-fidelity capabilities can be largely attributed to the two digital twin subsystems and their efficient integration through the proposed framework, as illustrated in Figure 8. In addition, based on the modular design of the proposed framework, another simulator with comparable fidelity can be developed simply by replacing the dynamics model and its corresponding visualization components. As illustrated in Figure 17, the same framework is used to construct a realistic drone simulator, which further demonstrates the strong portability of the proposed framework.

Figure 17.

Construction of a drone simulator using the proposed framework. (The drone visual 3D model is taken from freakering⁴⁵).

5. Conclusion

Using simulators to support fighter training clearly reduces both cost and risk. Building on this advantage, this study presents a high-fidelity simulator composed of two proposed digital twin subsystems. The first subsystem is an accurate F-16 flight dynamics model based on rigorous nonlinear modeling theory and publicly available NASA data. The model is first implemented in MATLAB and then validated using multiple flight maneuvers to confirm that its responses are reasonable and realistic. The second subsystem is responsible for rendering highly realistic scenes. In this study, UE and Cesium are adopted to provide a realistic visualization of the virtual aircraft and diverse global environments, making the simulated scenarios closer to real-world conditions. This study further proposes a general simulator framework in C++ that integrates these two subsystems and all required interfaces, including the hardware joysticks and VR devices. Within this framework, the F-16 flight dynamics model is converted to C++ using MATLAB Coder, allowing it to be integrated into the UE workflow while facilitating real-time performance. The resulting dynamics module receives command inputs from the hardware joystick and generates high-fidelity responses, while the corresponding panoramic views are rendered and streamed to VR devices, further providing a realistic flight experience for the users. A dedicated flight data logging module is also constructed and included in this framework. It records flight states in specific CSV files, enabling integration with the well-known flight analysis tool Tacview for post-mission review and debriefing. To demonstrate the overall performance of the proposed simulator and its potential for supporting customized training tasks, a simplified cannon-shooting scenario is presented. The corresponding results indicate that the simulator can efficiently provide an immersive training environment. Moreover, following the modular design of the proposed framework, switching the main vehicle or adding a custom evaluation metric system can be easily achieved without extensive programming. In future work, this framework will be extended to support online multi-aircraft training, and more diverse evaluation metrics will also be incorporated. The goal is to make the simulator more comprehensive and more consistent with real defense training requirements, while keeping the design practical, extensible, and easy to maintain.

Supplemental Material

sj-pdf-1-dms-10.1177_15485129261433592 – Supplemental material for Development of a high-fidelity fighter 6DoF simulator for immersive education and training

Supplemental material, sj-pdf-1-dms-10.1177_15485129261433592 for Development of a high-fidelity fighter 6DoF simulator for immersive education and training by Rong He and Chao-Chung Peng in The Journal of Defense Modeling and Simulation

Footnotes

Appendix 1 Acknowledgements

The authors would like to thank Mr K. T. Chen and Mr C. H. Chih for their valuable discussions.

ORCID iDs

Rong He

Chao-Chung Peng

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Moreover, the authors would like to express their deepest gratitude to the National Cheng Kung University (NCKU) and National Science and Technology Council (NSTC) under the project number NSTC 111-2221-E-006-170, NSTC 112-2221-E-006-104-MY3 for their financial support in carrying out this research project.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Supplemental material

Supplemental material for this article is available online.

Author biographies

Rong He was born in Tainan, Taiwan, in 1996. He received the BS and MS degrees from the Department of Aeronautics and Astronautics (DAA), National Cheng Kung University (NCKU), Tainan, Taiwan, in 2019 and 2021, respectively. He is currently pursuing the PhD degree in the same department. His research interests include state estimation and localization, with a particular focus on SLAM-related technologies such as Visual-Inertial Odometry (VIO) and LiDAR-Inertial Odometry (LIO). He also develops Unreal Engine–based digital twin systems and high-fidelity simulators, with extensive experience integrating C/C++, Python, and MATLAB.

Chao-Chung Peng (Member, IEEE) was born in Kaohsiung, Taiwan, in 1980. He received the BS degree and the PhD degree in Aeronautics and Astronautics from National Cheng Kung University (NCKU), Tainan, Taiwan, in 2003 and 2009, respectively. From 2008 to 2009, he was a research assistant in the Department of Engineering, University of Leicester, UK. He subsequently served as a postdoctoral fellow in the Department of Mechanical Engineering at NCKU from 2010 to 2012. In 2012, he joined ADLINK Technology Inc., Taipei, Taiwan, as a senior engineer in the Embedded System Development Section, Measurement and Automation Department. From 2014 to 2016, he was with the Automation and Instrumentation System Development Section of the Iron & Steel Research and Development Department at China Steel Corporation (CSC), Kaohsiung, Taiwan. Since 2016, he has been with the Department of Aeronautics and Astronautics at NCKU, where he was appointed Assistant Professor in 2016, promoted to Associate Professor in 2020, and to Full Professor in 2023. In 2024, he was appointed the position of Director of the Advanced Intelligent Unmanned Aerial Vehicle Research and Development Center at NCKU. His research interests include high-performance motion control, unmanned vehicle system design, advanced flight control, autonomous robotics, intelligent simultaneous localization and mapping (SLAM), system modeling, and real-time diagnostic technologies. He was elected to the Phi Tau Phi Scholastic Honor Society in 2009. He received the Excellent Young Engineering Professor Award from the Chinese Society of Mechanical Engineers (CSME), in 2019, and the Rising Star Award from the College of Engineering, NCKU, in 2023.

References

Bongers

Torres

JL.

Technological change in U.S. jet fighter aircraft. Res Policy 2014; 43: 1570–1581.

Aronsson

Artman

Mitchell

, et al. A live mindset in Live Virtual Constructive simulations: a spin-up for future LVC air combat training. J Def Model Simul 2023; 20: 447–465.

Thijssen

van Pelt

van Benthem

, et al. Exploring the integration of live, virtual, and constructive environments in fighter pilot training through Link-16: lessons learned regarding training aspects. J Def Model Simul 2025; 22: 529–542.

Saastamoinen

Maunula

Usefulness of flight simulator as a part of military pilots training—case study: Grob G 115E. Procedia Comput Sci 2021; 192: 1670–1676.

Orlansky

String

Cost-effectiveness of flight simulators for military training. Vol. 1. Use and effectiveness of flight simulators. Alexandria, VA: Institute for Defense Analyses, 1977.

Saber

Zakaria

Kamal

AM.

Integration of JSBSim and unreal engine for flight simulator development: a case study on the Cessna T-37. Int Conf Aerosp Sci Aviat Technol 2025; 21: 1–14.

Somerville

Joiner

Lynar

, et al. Application of augmented reality for aviation equipment inspection and maintenance training. In: 2022 8th international conference on applied system innovation (ICASI), Nantou, 22–23 April 2022, pp. 58–63. New York: IEEE.

Peng

Chang

Chu

YL.

Application of augmented reality for aviation equipment inspection and maintenance training. In: 2022 8th international conference on applied system innovation (ICASI), Nantou, 22–23 April 2022, pp. 58–63. New York: IEEE.

Xie

Liu

Alghofaili

, et al. A review on virtual reality skill training applications. Front Virtual Real 2021; 2: 645153.

10.

Cross

Boag-Hodgson

CC.

A collaborative virtual reality flight simulator: efficacy, challenges, and potential. IEEE Trans Learn Technol 2025; 18: 119–129.

11.

Cross

Ryley

Assessing evidence-based training in a collaborative virtual reality flight simulator. Aeronaut J 2025; 129: 261–281.

12.

Duruaku

Nguyen

Sonnenfeld

, et al. Unveiling virtual reality overheads and their potential impact on flightcrew training. Proc Hum Factors Ergon Soc Annu Meet 2024; 68: 1047–1053.

13.

Lindberg

CKK

Dahlkvist

Ramberg

, et al. Cybersickness between a virtual and mixed reality flight simulator–a comparative study. In: MODSIM World 2024, Norfolk, VA, 20–22 May 2024, pp. 1–11.

14.

Ernst

Laudien

Schmerwitz

Implementation of a mixed-reality flight simulator: blending real and virtual with a video-see-through head-mounted display. Proc SPIE Artif Intell Mach Learn Multi-Domain Oper Appl V 2023; 12538: 181–190.

15.

Hiremath

Saravanan

A comprehensive study of virtual reality (VR) in defence combat training. In: 2024 8th international conference on computational system and information technology for sustainable solutions (CSITSS), Bengaluru, India, 7–9 November 2024, pp. 1–6. New York: IEEE.

16.

Jiang

Hang

, et al. Methodological innovation in evaluating the cost-effectiveness of simulation training combining transfer effectiveness and change-point analysis. J Med Educ Curric Dev 2025; 12: 23821205251368247.

17.

Pedram

Kennedy

Sanzone

Assessing the validity of VR as a training tool for medical students. Virtual Real 2024; 28: 15.

18.

Javvaji

Reddy

Vagha

, et al. Immersive innovations: exploring the diverse applications of virtual reality (VR) in healthcare. Cureus 2024; 16: e56137.

19.

Arjun

Sanjay

Enhancing military training through VR applications. Int Sci J Eng Manag 2024; 3: 1–9.

20.

Johansson

Roupé

Real-world applications of BIM and immersive VR in construction. Autom Constr 2024; 158: 105233.

21.

Akindele

Taiwo

Sarvari

, et al. A state-of-the-art analysis of virtual reality applications in construction health and safety. Results Eng 2024; 23: 102382.

22.

Chen

Chiang

Peng

CC.

Physics-informed reinforcement learning for air-to-air F16 target aiming. Adv Eng Inform 2026; 69: 103988.

23.

Peng

Chiang

Chen

YH.

Autonomous gain tuning for differential drive robots targeting control using soft actor-critic. In: 2024 IEEE conference on artificial intelligence (CAI), Singapore, 25–27 June 2024, pp. 255–260. New York: IEEE.

24.

Kuo

Chen

Lin

, et al. Deep reinforcement learning–based collision avoidance strategy for multiple unmanned aerial vehicles. Eng Appl Artif Intell 2025; 160: 111862.

25.

Liu

Fang

Shu

, et al. Dynamic production capacity assessment of aircraft overhaul shop based on digital twin. Comput Ind Eng 2024; 194: 110360.

26.

Peng

Chen

YH.

Fixed-wing unmanned aerial vehicle rotary engine anomaly detection via online digital twin methods. IEEE Trans Aerosp Electron Syst 2024; 60: 741–758.

27.

Peng

Chen

YH.

A hybrid neural ordinary differential equation based digital twin modeling and online diagnosis for an industrial cooling fan. Future Internet 2023; 15: 302.

28.

Peng

Chen

YH.

Digital twins-based online monitoring of TFE-731 turbofan engine using fast orthogonal search. IEEE Syst J 2022; 16: 3060–3071.

29.

Chih

Peng

CC.

Dynamics modeling and nonlinear attitude controller design for a rocket-type unmanned aerial vehicle. ISA Trans 2024; 152: 15–27.

30.

Zhang

Peng

CC.

Loop shaping-based attitude controller design and flight validation for a fixed-wing UAV. Drones 2025; 9: 697.

31.

Nguyen

LT.

Simulator study of stall/post-stall characteristics of a fighter airplane with relaxed longitudinal static stability. Washington, DC: National Aeronautics and Space Administration, 1979.

32.

Cook

MV.

Flight dynamics principles: a linear systems approach to aircraft stability and control. Oxford: Butterworth-Heinemann, 2012.

33.

Stevens

Lewis

Johnson

EN.

Aircraft control and simulation: dynamics, controls design, and autonomous systems. Hoboken, NJ: John Wiley & Sons, 2015.

34.

Miele

Flight mechanics: theory of flight paths. Garden City, NY: Courier Dover Publications, 2016.

35.

Singh

Steinberg

Page

. Nonlinear adaptive and sliding mode flight path control of F/A-18 model. IEEE Trans Aerosp Electron Syst 2003; 39: 1250–1262.

36.

Šenk

Cheze

Rotation sequence as an important factor in shoulder kinematics. Clin Biomech 2006; 21: S3–S8.

37.

Dam

Koch

Lillholm

. Quaternions, interpolation and animation. Vol. 2. Copenhagen, Denmark: Datalogisk Institut, Københavns Universitet, 1998.

38.

Peng

Tsai

MC.

Analysis of an electronic heating device: system modeling, identification and control. IEEE Trans Ind Electron 2024; 71: 16463–16472.

39.

Nop

Manissaward

Oattarapon

Development of character design frameworks using game engine: unreal engine. In: 2019 joint international conference on digital arts, media and technology with ECTI northern section conference on electrical, electronics, computer and telecommunications engineering (ECTI DAMT-NCON), Nan, Thailand, 30 January–2 February 2019, pp. 54–59. New York: IEEE.

40.

Jiang

Dhamankar

Liu

, et al. Co-simulation of the Unreal Engine and MATLAB/Simulink for automated grain offloading. IFAC-PapersOnLine 2022; 55: 379–384.

41.

Shah

Dey

Lovett

, et al. AirSim: high-fidelity visual and physical simulation for autonomous vehicles. In: Hutter

Siegwart

(eds) Field and service robotics: results of the 11th international conference. Cham: Springer, 2018, pp. 621–635.

42.

Dosovitskiy

Ros

Codevilla

, et al. CARLA: an open urban driving simulator. In: Conference on robot learning, Mountain View, CA, 13–15 November 2017, pp. 1–16. PMLR.

43.

flyscript. Angels-high, https://github.com/flyscript/Angels-High (accessed 11 September 2023).

44.

CGTrader. F-16A block 15 cockpit free 3D model, https://www.cgtrader.com/free-3d-models/aircraft/jet/f-16a-block-15-cockpit (accessed 15 September 2023).

45.

freakering. Inside drone, https://sketchfab.com/3d-models/inside-drone-d38af0afb72343f69eed6da6457df11b (accessed 13 November 2025).

46.

Diebel

Representing attitude: Euler angles, unit quaternions, and rotation vectors. Matrix 2006; 58: 1–35.

47.

Kibble

Berkshire

FH.

Classical mechanics. Singapore: World Scientific Publishing Company, 2004.