Machine Learning Revolutionizes Hypersonic Flow Modeling

Breakthrough in Non-Equilibrium Gas Dynamics

BLUF (Bottom Line Up Front)

Researchers at RWTH Aachen University have developed R13-ML, a machine learning framework that achieves Direct Simulation Monte Carlo (DSMC)-level accuracy for hypersonic flow predictions at a fraction of the computational cost. The model successfully simulates shock waves up to Mach 9 and generalizes to unsteady transient flows—critical capabilities for designing next-generation hypersonic vehicles. By learning complex gas dynamics from training data (Mach 1.2-8.0 shock structures) and embedding neural networks within physics-based solvers, R13-ML overcomes the failure of conventional computational fluid dynamics in rarefied, non-equilibrium regimes while maintaining conservation laws and physical consistency. Available artifacts include: complete training datasets, pre-trained neural network models, numerical solver code (Trixi.jl integration), and data processing scripts—all publicly accessible on GitHub.

---

Available Research Artifacts

The research team has made their complete computational framework publicly available, providing unprecedented transparency and enabling rapid adoption by the aerospace community:

1. Training Datasets

• Complete DSMC simulation data for one-dimensional argon shock waves spanning Mach 1.2 to 8.0 (18 cases in 0.4 increments)
• Each case includes 800 spatial grid points with full flow variables
• High-order moments sampled through molecular thermal velocity distributions
• Collision integral data extracted via steady-state flux derivatives
• Augmented datasets with polynomial interpolation and flow reversal transformations

2. Pre-trained Neural Network Models

• Four independently trained fully connected neural networks (FCNNs) for:
  • High-order moment mxxx (third-order moment tensor component)
  • High-order moment Rxx (fourth-order moment tensor component)
  • Collision integral Qxx (stress production term)
  • Collision integral Qx (heat flux production term)
• Architecture: Input layer → 6 hidden layers (128→64→64→64→64→64 neurons) → Output layer
• Softplus activation functions throughout hidden layers
• GPU-optimized training implementations with CPU deployment versions

3. Complete Numerical Solver

• Integration with Trixi.jl framework: Discontinuous Galerkin Spectral Element Method (DGSEM) solver for moment transfer equations
• Physics-based discretization preserving conservation laws
• Real-time closure updates via embedded neural networks during simulation
• Compatible with both steady and unsteady flow scenarios

4. Data Processing and Normalization Code

• Scaling algorithms using density-dependent mean free path and temperature-dependent sound speed
• Normalization procedures ensuring stable training across equilibrium and non-equilibrium regimes
• Collision integral extraction methodology from steady-state DSMC data

5. Validation Test Cases

• Shock structure simulations at Mach 5, 7, and 9
• Two-shock interaction scenarios (transient flows)
• Shock-high temperature region impingement cases
• Complete comparison data against DSMC, NSF, and standard R13 solutions

Access Information

All artifacts are available at: https://github.com/songhangRGD/R13-ML

The repository includes comprehensive documentation for:

• Installing dependencies and setting up the computational environment
• Running pre-trained models on new flow conditions
• Retraining networks with custom datasets
• Integrating R13-ML closures into existing moment equation solvers

This open-source release is particularly significant for the hypersonic research community, as high-fidelity rarefied flow simulation tools have traditionally been proprietary or limited to specialized research groups. The availability of both training data and trained models enables researchers to either apply the framework directly to new problems or use the methodology as a template for developing domain-specific closures for other gas species, temperature ranges, or flow geometries.

---

The Hypersonic Challenge

When spacecraft reenter Earth's atmosphere or next-generation vehicles travel at speeds exceeding Mach 5, they encounter conditions that defy conventional fluid dynamics. At these extreme velocities, air molecules don't have time to reach thermal equilibrium through collisions, creating what physicists call "rarefied flow" or "non-equilibrium conditions." Traditional computational fluid dynamics (CFD) tools, built on Navier-Stokes equations, simply break down in this regime.

"The degree of rarefaction, characterized by the Knudsen number, leads to significant deviations from equilibrium at large values due to reduced collisionality," explain the researchers in their paper. The Knudsen number compares the mean free path of gas molecules (the average distance they travel between collisions) to the characteristic size of the object moving through them.

The Computational Dilemma

Engineers have long faced a difficult trade-off. Direct Simulation Monte Carlo (DSMC) methods can accurately capture the molecular-scale physics of hypersonic flows by simulating individual particle collisions, but the computational cost becomes prohibitive—particularly in the "slip" and "transition" flow regimes that characterize many practical hypersonic scenarios.

Moment methods offer a promising middle ground, bridging kinetic theory and continuum mechanics. However, classical approaches like the Grad 13-moment equations (G13) or regularized 13-moment equations (R13) fail under strong non-equilibrium conditions. "These methods' inability to accurately represent high-order moments and capture complex collision dynamics, especially in the nonlinear regime at high Mach numbers," has limited their application to hypersonic flows, the researchers note.

The R13-ML Breakthrough

The team—comprising Hang Song, Satyvir Singh, Manuel Torrilhon from Applied and Computational Mathematics, and Semih Cayci from Mathematics of Machine Learning—introduced what they call the R13-ML model. This innovative approach combines physics-based moment equations with machine learning-derived closure relations.

Their framework addresses three critical requirements for accurate rarefied flow simulation: inclusion of conservation laws and cross-coupling of intermediate moments, physically consistent closures for high-order moments, and mathematically precise formulations of collision integrals.

The key innovation lies in using neural networks to learn the complex, nonlinear relationships between flow variables directly from high-fidelity DSMC data, rather than relying on analytical approximations that break down at high Mach numbers.

Training on Shock Waves

The researchers constructed their training dataset from one-dimensional shock wave simulations spanning Mach numbers from 1.2 to 8.0. Shock waves—the supersonic-to-subsonic transitions that form ahead of hypersonic vehicles—serve as ideal test cases because they concentrate extreme non-equilibrium effects into narrow regions.

One particularly clever aspect of their methodology involves extracting collision integrals from the DSMC data. "Leveraging the steady-flow condition, this study determines the source terms of moment equations via the steady-state relation," they explain. By using spatial derivatives of flow variables, they could compute collision effects that are otherwise analytically intractable.

To ensure the neural network learned physics rather than data artifacts, the team implemented careful normalization using temperature-dependent sound speeds and density-dependent mean free paths. "This preserves rarefaction effects and ensures stable machine-learning training," they note.

Hypersonic Performance

The R13-ML model's capabilities for hypersonic applications proved exceptional. In tests on shock structures at Mach 5 and 7—conditions within the training range—the model achieved excellent agreement with DSMC reference data. More impressively, it extrapolated successfully to Mach 9, well beyond its training regime.

For hypersonic vehicle designers, the most significant result may be the model's performance on unsteady, transient flows. The team tested two challenging scenarios: the collision of two approximately Mach 4 shock waves, and the impingement of a hypersonic stream on a high-temperature region. In both cases, R13-ML demonstrated "markedly superior accuracy compared to both NSF and standard R13 equations."

In the shock interaction case, the model achieved "near-perfect agreement with DSMC results in density and stress," with relative errors below 5% even in the most challenging flow regions. Traditional Navier-Stokes-Fourier (NSF) equations and standard R13 methods showed "substantial errors across all flow variables within the bilateral wavefront regions."

Implications for Aerospace Engineering

The R13-ML framework offers several critical advantages for hypersonic vehicle development:

Computational Efficiency: By embedding neural networks within a discontinuous Galerkin spectral element solver (Trixi.jl), the method achieves DSMC-level accuracy at a fraction of the computational cost. The offline training on GPUs produces models that run efficiently on CPUs during production simulations.

Physical Consistency: Unlike purely data-driven approaches, R13-ML preserves conservation laws and physical structure by solving the underlying moment equations with learned closures rather than replacing the physics entirely.

Generalization: The model's ability to extrapolate to higher Mach numbers and handle unsteady flows suggests it could predict conditions not directly represented in training data—essential for exploring novel vehicle designs.

Aerothermal Predictions: Accurate modeling of heat flux and stress distributions is critical for thermal protection system design. The R13-ML model's precise capture of these quantities in shock regions directly addresses key challenges in hypersonic vehicle development.

Broader Context in ML-Enhanced Fluid Dynamics

This work represents a maturation of machine learning applications in computational fluid dynamics. While recent years have seen numerous ML approaches to turbulence modeling and fluid simulation, the hypersonic rarefied flow regime has remained particularly challenging.

The success of R13-ML demonstrates what researchers call "physics-informed machine learning"—using neural networks not to replace physical models entirely, but to learn the complex constitutive relations that connect physical quantities in regimes where analytical approximations fail. This philosophy aligns with growing recognition that the most effective ML applications in engineering augment rather than supplant domain knowledge.

Future Directions

The researchers acknowledge that their current implementation addresses one-dimensional flows. "In the future, R13-ML will be extended to multiple space dimensions using appropriate data sets," they note. This extension will be crucial for modeling three-dimensional hypersonic vehicle geometries with realistic shock interactions, boundary layers, and flow separation.

The team has made their training data, pre-trained models, and numerical solver publicly available on GitHub, facilitating further research and applications by the broader aerospace community.

Conclusion

As nations and private companies race to develop hypersonic vehicles for military, commercial, and space applications, the R13-ML framework offers a pathway to more accurate and efficient simulations of the extreme flow conditions these vehicles encounter. By successfully bridging machine learning with continuum mechanics while preserving fundamental physics, this work establishes "a new paradigm for ML-enhanced kinetic-fluid modeling."

For aerospace engineers designing the next generation of hypersonic aircraft, reentry vehicles, and spacecraft, the ability to rapidly and accurately predict aerothermal loads under non-equilibrium conditions could accelerate development cycles and improve vehicle performance—potentially bringing hypersonic flight from the realm of specialized military applications into broader commercial use.

Sidebar: The Fundamental Breakdown of Conventional CFD

Conventional computational fluid dynamics (CFD) based on the Navier-Stokes-Fourier (NSF) equations fails in hypersonic rarefied flow conditions due to several interconnected physical and mathematical breakdowns:

1. Violation of the Continuum Hypothesis

The Navier-Stokes equations are derived under the continuum assumption—that gas can be treated as a continuous medium rather than discrete molecules. This assumption requires that:

• The mean free path (λ) of molecules is much smaller than the characteristic length scale (L) of the flow
• The Knudsen number (Kn = λ/L) remains small (typically Kn < 0.01)

In hypersonic rarefied flows, the Knudsen number increases dramatically because:

• High velocities create shock waves with extremely thin transition regions (small L)
• Low atmospheric densities at high altitudes increase the mean free path (large λ)
• At the shock front itself, Kn can exceed 0.1 or even approach 1.0, firmly in the "transition regime"

As the researchers note, "at large Kn due to reduced collisionality, whenever the mean free path λ reaches similar magnitudes as the macroscopic length scale L," the continuum description becomes invalid.

2. Failure of Constitutive Relations

The NSF equations close the system through constitutive relations that assume:

• Stress is linearly proportional to velocity gradients (Newton's law of viscosity): σ = -μ(∇v + ∇v^T)
• Heat flux is linearly proportional to temperature gradients (Fourier's law): q = -κ∇T

These linear relationships are derived from near-equilibrium assumptions using the Chapman-Enskog expansion, which breaks down when:

• Molecular collisions are insufficient to maintain local thermodynamic equilibrium
• The distribution function deviates significantly from the Maxwell-Boltzmann distribution
• Non-linear effects dominate the transport processes

The paper explicitly states: "NSF becomes invalid due to the breakdown of the constitutive relations in rarefied flow (Kn ≳ 0.01-0.1)."

3. Missing Physics: High-Order Moment Coupling

In strongly non-equilibrium conditions, the flow physics cannot be captured by the five conserved variables (density, three velocity components, temperature) alone. Higher-order moments of the velocity distribution function become dynamically significant:

• Stress tensor components (second-order moments) are no longer determined by local velocity gradients
• Heat flux (third-order moments) depends on the history of the flow, not just local temperature gradients
• Fourth and higher-order moments become non-negligible and couple back to affect the lower-order moments

The researchers explain that conventional NSF theory lacks "stress tensor and heat flux as independent variables to be solved as part of the fluid-dynamic system."

4. Inability to Capture Shock Structure

Shock waves in hypersonic flow present a particularly severe challenge:

• Shock thickness approaches molecular length scales where the continuum assumption fails
• Extreme gradients within the shock violate the small-perturbation assumptions underlying NSF
• Strong thermodynamic non-equilibrium means different internal energy modes (translational, rotational, vibrational) equilibrate at different rates
• Collision integrals become highly nonlinear and cannot be approximated by simple relaxation models

The paper's results dramatically illustrate this: "NSF severely underestimates the wave thickness" and shows "substantial errors across all flow variables within the bilateral wavefront regions."

5. Time Scale Separation Breakdown

NSF equations assume a clear separation of time scales:

• Fast microscopic collisions (collision time τ_collision) establish local equilibrium
• Slow macroscopic evolution (flow time τ_flow) allows the system to remain near equilibrium

This requires τ_collision << τ_flow. In rarefied hypersonic flows:

• Reduced density increases collision times
• Sharp gradients decrease characteristic flow times
• The ratio τ_collision/τ_flow becomes O(1), violating the time scale separation

6. Nonlinear Transport at High Mach Numbers

At high Mach numbers (Ma > 5), the research shows that even extended moment methods with linear closures fail:

• The classical R13 closure equations (shown in the paper as equations 1 and 2) assume linear relationships between high-order moments and gradients of lower-order moments
• These "fail under strong non-equilibrium conditions" because "nonlinear moment relations and collision integrals become analytically intractable"
• The paper demonstrates that "as the Mach number increases, the discrepancies in high-order moments between the DSMC and the linear-theory-based R13 baseline model progressively intensify"

Sidebar: Comparative Computational Costs: DSMC vs. R13-ML

While the paper doesn't provide explicit timing comparisons or FLOP counts, we can analyze the computational economics based on the methodology and established scaling behavior of these approaches:

DSMC Computational Cost Structure

DSMC simulation cost scales as:

Cost_DSMC ∝ N_particles × N_timesteps × N_collisions

Where:

• N_particles: Number of simulated particles (typically 10^6 to 10^9 for realistic 3D problems)
• N_timesteps: Must resolve the collision time τ_collision (extremely small in hypersonic flows)
• N_collisions: Number of collision pairs evaluated per timestep (scales as N_particles^2 in naive implementations, N_particles × log(N_particles) with spatial sorting)

Critical cost drivers in hypersonic DSMC:

1. Statistical noise requires large particle counts for accurate moment sampling
2. Small timesteps (Δt < τ_collision) are mandatory for collision accuracy
3. Long physical times needed for steady-state convergence in transient problems
4. Collision detection becomes expensive in dense regions (shock fronts)

The paper notes: "DSMC naturally resolves molecular collisions and rarefaction effects, providing reliable high-order moments and collision terms, though at high computational cost."

R13-ML Computational Cost Structure

The R13-ML approach has two distinct cost phases:

Offline Training Cost (one-time):

• Dataset generation: DSMC simulations of 18 shock cases (Mach 1.2-8.0)
• Neural network training: GPU-accelerated, converged in hours to days (standard for FCNNs of this size)
• This cost is amortized across all future simulations

Online Simulation Cost (per problem): Cost_R13ML ∝ N_grid × N_timesteps × (Cost_DG + Cost_NN)

Where:

• N_grid: Number of grid points (typically 10^2-10^4 for 1D/2D problems, far fewer than DSMC particles)
• N_timesteps: Can be much larger than DSMC (Δt can exceed collision time)
• Cost_DG: Discontinuous Galerkin spatial discretization (sparse matrix operations)
• Cost_NN: Four neural network evaluations per grid point per timestep
  • Each FCNN: 10 input → 128 → 64 → 64 → 64 → 64 → 64 → 1 output
  • Total operations: ~30,000 multiply-adds per network evaluation
  • Four networks: ~120,000 operations per grid point

Key Computational Advantages of R13-ML

1. Dramatically Fewer Degrees of Freedom:

   • DSMC: ~10^6-10^9 particles
   • R13-ML: ~10^2-10^4 grid points
   • Reduction factor: 10^4 to 10^5

2. Larger Stable Timesteps:

   • DSMC: Δt ≤ τ_collision (microseconds in typical conditions)
   • R13-ML: Δt limited by CFL condition for hyperbolic system (~10-100× larger)
   • Speedup factor: 10-100×

3. Deterministic vs. Statistical:

   • DSMC requires ensemble averaging to reduce statistical noise
   • R13-ML produces deterministic solutions (no noise)
   • No need for multiple realizations

4. Memory Efficiency:

   • DSMC stores particle positions, velocities, internal states
   • R13-ML stores 13 moment variables per grid point
   • Memory reduction: ~100-1000×

5. CPU-Only Deployment:

   • The paper notes: "The trained neural network is subsequently deployed on CPU architectures for seamless integration"
   • No GPU required for production runs (though DSMC can also run on CPUs)

Estimated Speedup

Based on typical performance characteristics:

• For 1D steady shocks: R13-ML likely achieves 100-1000× speedup over DSMC

  • Grid-based methods scale favorably in low dimensions
  • No statistical sampling noise
  • Larger timesteps permitted

• For 1D unsteady problems: R13-ML likely achieves 50-500× speedup

  • Time evolution still required
  • Must resolve wave propagation
  • But deterministic convergence

• Expected for 2D/3D (future work): Speedup may be 10-100×

  • Higher dimensional grids increase cost
  • But still massively fewer degrees of freedom than particle methods
  • Neural network cost becomes more significant relative to grid operations

Accuracy-Cost Trade-off

The paper's results show that R13-ML achieves "DSMC-level accuracy" with these dramatic cost reductions:

• Mach 5, 7 shocks: "Excellent agreement" with DSMC
• Mach 9 shock (extrapolation): Strong agreement despite being outside training range
• Unsteady two-shock interaction: "Near-perfect agreement with DSMC results in density and stress, while heat flux exhibits relative errors below 5%"
• Shock-temperature discontinuity: "Near-perfect agreement with DSMC results in density and stress"

In contrast, classical R13 with linear closures shows progressively increasing errors at high Mach numbers, and NSF "severely underestimates the wave thickness."

The Cost of Getting It Wrong

An important but often overlooked consideration is the cost of inaccurate predictions:

• NSF predictions can be orders of magnitude wrong in heat flux and stress at shock fronts

  • This leads to catastrophic underestimation of thermal loads
  • Could result in thermal protection system failure
  • The computational "savings" of NSF are worthless if predictions are unreliable

• Standard R13 improves on NSF but still shows "substantial errors" at high Mach numbers

  • May be acceptable for preliminary design but not certification

• R13-ML provides the accuracy needed for high-fidelity design at intermediate cost

  • Fills the critical gap between fast-but-inaccurate continuum methods and slow-but-accurate particle methods

When Is Each Method Appropriate?

The paper's results suggest a three-tier strategy:

1. Preliminary design (Kn < 0.01, Ma < 3): NSF sufficient

   • Cost: Lowest
   • Accuracy: Adequate for near-equilibrium flows

2. Detailed design (0.01 < Kn < 0.1, Ma = 3-10): R13-ML optimal

   • Cost: Intermediate (100-1000× faster than DSMC)
   • Accuracy: DSMC-level for moments and bulk properties

3. Validation and edge cases (Kn > 0.1, complex chemistry, 3D geometries): DSMC necessary

   • Cost: Highest
   • Accuracy: First-principles resolution of kinetic effects

The Machine Learning Cost-Benefit Analysis

The R13-ML approach exemplifies transfer learning in computational physics:

• Invest heavily once in generating high-fidelity training data (DSMC simulations)
• Extract generalizable closures that encode the physics of non-equilibrium transport
• Apply cheaply to new problems within the learned regime
• Extend to new regimes by fine-tuning or augmenting training data

This paradigm is particularly powerful when:

• The same physics appears across many problems (shock waves in hypersonic flows)
• High-fidelity simulations are expensive but possible for canonical cases
• Rapid turnaround is needed for design iterations
• Slight accuracy reduction (from ideal first-principles) is acceptable for large speedup

The researchers' decision to make all artifacts publicly available amplifies this benefit: the community can leverage their expensive DSMC training data without regenerating it.

Future Scalability Considerations

The extension to 2D/3D will present challenges:

• Neural network input dimension increases with spatial derivatives (more gradient components)
• Training data requirements scale unfavorably with dimension (curse of dimensionality)
• Grid costs increase polynomially (N^2 for 2D, N^3 for 3D)

However, physics-informed approaches like R13-ML fare better than purely data-driven methods:

• The underlying moment equations remain the same structure
• Only the closure relations need dimensional extension
• Physical symmetries and invariances reduce the effective dimension

Conclusion

Conventional CFD fails in hypersonic rarefied flows due to the fundamental breakdown of the continuum hypothesis, linear constitutive relations, and time-scale separation assumptions. The Navier-Stokes equations simply do not contain the physics of strongly non-equilibrium molecular transport.

The R13-ML framework bridges the accuracy gap between conventional CFD and DSMC at a computational cost that is 100-1000× lower than DSMC while achieving comparable accuracy. This positions it as the method of choice for high-fidelity hypersonic design workflows, where the cost of DSMC is prohibitive but the inaccuracy of NSF is unacceptable.

By combining physics-based moment equations with machine-learned closures, R13-ML preserves conservation laws and physical structure while capturing the complex nonlinear dependencies that elude analytical treatment—a paradigm that may well define the future of computational fluid dynamics in extreme regimes.

 

---

Sources

1. Song, H., Singh, S., Torrilhon, M., & Cayci, S. (2025). Extraction of Moment Closures for Strongly Non-Equilibrium Flows via Machine Learning. arXiv preprint arXiv:2511.00545v1 [physics.flu-dyn]. https://arxiv.org/abs/2511.00545

2. Anderson, J. D., Jr. (2006). Hypersonic and High Temperature Gas Dynamics. McGraw-Hill, New York, NY.

3. Bird, G. A. (1994). Molecular Gas Dynamics and the Direct Simulation of Gas Flows. Oxford University Press, Oxford, UK.

4. Torrilhon, M. (2016). Modeling nonequilibrium gas flow based on moment equations. Annual Review of Fluid Mechanics, 48, 429-458. https://doi.org/10.1146/annurev-fluid-122414-034259

5. Brunton, S. L., Noack, B. R., & Koumoutsakos, P. (2020). Machine learning for fluid mechanics. Annual Review of Fluid Mechanics, 52, 477-508. https://doi.org/10.1146/annurev-fluid-010719-060214

6. Han, J., Ma, C., Ma, Z., & E, W. (2019). Uniformly accurate machine learning-based hydrodynamic models for kinetic equations. Proceedings of the National Academy of Sciences, 116(44), 21983-21991. https://doi.org/10.1073/pnas.1909854116

7. Schlottke-Lakemper, M., Winters, A. R., Ranocha, H., & Gassner, G. J. (2021). A purely hyperbolic discontinuous Galerkin approach for self-gravitating gas dynamics. Journal of Computational Physics, 442, 110467. https://doi.org/10.1016/j.jcp.2021.110467

8. Song, H., Singh, S., Torrilhon, M., & Cayci, S. (2024). Extraction of Moment Closures for Strongly Non-Equilibrium Flows via Machine Learning: Code Implementation. GitHub. https://github.com/songhangRGD/R13-ML

 

Hypersonic flow research story with sources