TAES Special Section on Deep Learning for Radar Applications

Guest Editorial for the TAES Special Section on Deep Learning for Radar Applications | IEEE Journals & Magazine | IEEE Xplore: 

• Sevgi Z. Gurbuz Department of Electrical and Computer Engineering, The University of Alabama, Tuscaloosa, AL, USA
• Igal BilikSchool of Electrical and Computer Engineering, Ben Gurion University of the Negev, Beersheba, Israel
• Luke RosenbergSTELaRLab, Lockheed Martin, University of Adelaide, Adelaide, SA, Australia

 Date of Publication: 08 February 2024

---

It has been roughly a decade since the first papers using deep neural networks (DNNs) for radar applications were published. Deep learning has revolutionized almost every technical area, from computer vision and natural language processing to health, finance, and biology—any field where data can be analyzed to provide insight. However, in radar applications, deep learning faces unique challenges due to the phenomenology of radio frequency (RF) propagation that creates essential differences in the data itself and impacts the design of DNNs for radar signal analysis [1], [2], [3].

Unlike computer vision, radar data are not inherently an image. The received RF signal is a time series of complex samples, which, after radar signal processing, can be converted into various 2D, 3D, or even 4D tensors: range–Doppler (RD) maps or range–angle maps as a function of time, micro-Doppler signatures, RD–angle cubes as a function of time, RD–azimuth–elevation tensors, synthetic aperture radar (SAR) images, high-resolution range profiles, and others. These representations have been utilized in the literature as inputs to DNNs for classification [4], [5]. However, optimizing the best input representation is challenging when many require varying degrees of computational complexity, memory requirements, and computation time (latency). A wide range of input representations appear in the articles included in this special section.

Closely related to input representation is the development of complex neural networks, especially as raw RF data are complex, and therefore, complex values should be preserved in computed high-dimensional representations at the DNN input. Complex representations of RF data have recently garnered increased attention as they capture more information via the representation of signal phase and amplitude. One way to address complex data is to separate the real and imaginary parts of the signal into two separate channels, which are then processed by separate networks but jointly exploited using feature-level or decision-level fusion. While this approach is easily implementable using existing tools, it does not completely capture the relationship between the real and imaginary components of the RF signal. More recently, however, complex neural networks that utilize complex convolutions, max pooling, and activation functions have been proposed to learn from complex data [6], [7] directly. Naturally, these architectures are of great interest to the radar community, which is finding new and innovative ways of exploiting complex DNNs for radar data analysis. Thus, the authors in [A1] utilize a complex convolutional neural network (CNN) when processing radar data.

Of course, a challenge faced in all deep learning applications, but perhaps most acutely in the radar domain, is that posed by training under limited data availability. Most DNNs, such as CNNs, are randomly initialized prior to model training. However, because the objective functions of DNNs are highly convex, containing many local minima, random initialization and training with gradient-based optimization may not result in convergence to global minima. Consequently, randomly initialized networks typically require considerable training sample support. The two main approaches to minimize training data requirements are unsupervised pretraining and transfer learning. In unsupervised pretraining, an autoencoder (AE) or convolutional AE (CAE) is greedily pretrained to learn nonlinear input transformations such that identity mapping is learned. When the decoder is removed, the encoder weights, trained without labels, provide a much better initialization for network weights than random initialization, thereby minimizing supervised training data requirements. Alternatively, in transfer learning, data from other sources are utilized to initialize the network—this could be data from different sensors or synthetic RF data simulated from physics-based models or using generative learning. For example, transfer learning techniques are exploited in [A2] and [A3] and AEs in [A4], whereas radar simulation methods are advanced in [A5] and [A6].

While initial works involving deep learning for radar applications have focused almost exclusively on automatic target recognition and classification, recently, there has been an increase in work exploiting deep learning for more fundamental radar signal processing tasks and other radar functions. The papers in this special section contribute new techniques for the application of deep learning to target recognition [A2], [A5] and detection [A4], [A7], [A13], interference mitigation [A8], direction-of-arrival (DOA) estimation [A9], beamforming [A10], radar imaging [A1], [A6], [A11], human pose estimation [A12], and cognitive radar [A3], [A13]. The synergy between signal processing with deep learning [8] enables the advancement of novel algorithms that can consider environment-specific variables. Cognitive radar takes this one step further by considering adaptive processing and adaptive transmission—also called fully adaptive radar—to optimize performance in dynamic environments [9], [10]. Integral to the design of cognitive radars is learning and metacognition—the process of thinking and learning—which are essential cognitive processes embodied by all intelligent agents. How deep learning can be exploited toward these aims represents an essential challenge whose solution will pave the way for next-generation radar systems.

SECTION II.

Main Contributions of the Articles in This Special Section

In [A1], the challenge of achieving high bandwidth for high-resolution 3D synthetic aperture imaging is addressed through the design of a hybrid, dual-domain complex-valued CNN to fuse multiband signals and infer the missing samples in the frequency gaps between subbands. In particular, results are shown for the fusion of two millimeter-wave radars, each with a bandwidth of 4 GHz operating at 60 GHz and 77 GHz, to achieve an effective bandwidth of 21 GHz by robustly estimating the full-band signal. The proposed DNN, kR-Net, provides high-fidelity images of concealed weapons, such as a knife with a serrated edge or pistol, when placed in a cardboard box.

In [A2], transfer learning is utilized to enable the recognition of ships in inverse SAR (ISAR) imagery in an aspect angle agnostic fashion. To overcome the challenge of limited data and better generalize across angles, four different DNNs (VGG16, VGG19, Inception-v3, and Xception) are pretrained on optical imagery in the ImageNet database [11]. The results show that transfer learning can enable consistently high classification accuracy even when the data exhibits variability in appearance due to the ship's motion and viewing geometry.

In [A3], transfer learning is utilized within a reinforcement learning framework to enable cognitive radar to adapt to dynamic changes in the environment. A Monte Carlo tree search (MCTS) is aided by a DNN while transfer learning accelerates the transfer of the policy learned by training the DNN based on MCTS on an initial parameter distribution (environment) to the policy required for a new distribution. In this way, the authors show that transfer learning can be used to enable the cognitive radar to adapt to both rapid and gradual changes in the environment while producing near-optimal performance with 20-fold less time and data.

In [A4], small radar cross-section targets, namely drones, are detected using contractive AEs—an AE variant that adds a penalty term consisting of the Frobenius norm of the Jacobian matrix of encoder activations with respect to the input. It has been shown [12] that this penalty enables the AE to be robust to slight variations in the input by better capturing the local directions of variation represented in the data, which correspond to a lower-dimensional non-linear manifold while being invariant to directions orthogonal to the manifold. Thus, contractive AEs are more robust to noise with results showing improved target detection probability as a result.

Drone detection and identification is an important, emergent area of research due to the widespread availability of drones and potential misuse for nefarious purposes. Complementing the aforementioned work on drone detection, in [A5], mechanical control information is employed to improve UAV classification accuracy. Each drone has its own set of unique RD signatures that depend on the drone's movement mechanism. While model-based simulation is a common technique to pretrain DNNs for radar signature classification, the failure of simulations to capture the variations in the drone signature due to its motion is a limitation on the efficacy of such pretraining. Typically, drones have eight main unique movements: the drone is able to throttle up, throttle down, pitch forward, pitch backward, roll left, roll right, yaw left, and yaw right. This work utilizes full-wave electromagnetic models combined with the kinematic model of the drone movements to improve the recognition accuracy of two quadcopters, a hexacopter, and an unmanned helicopter.

In [A6], a knowledge-aided approach to improve deep model training. In particular, the modeling of geometric priors and constraints in optimization pipelines is proposed to train DNNs that are more robust under limited data, with an improved capability to solve ill-posed inverse problems, such as 3D object reconstruction with SAR. The proposed approach combines methods from 3D computer graphics with neural rendering and is the first work (to the best of our knowledge) to develop differentiable rendering for SAR domain imagery.

In [A7], a different radar data representation is explored for application to maritime target recognition, namely, a 3D-ISAR technique is developed to generate a 3D point cloud representation of the target. Features are extracted from the 3D-ISAR-generated point cloud of the target from different perspectives (side, top, and front views) to form three point density images, which are then utilized by a CNN to classify the targets. The results showed that recognition accuracy increased with knowledge of aspect angle but that even when the angle was unknown, simulations could be used to train the CNN to achieve high accuracy on real datasets.

In [A8], the DNN-based approach was introduced to suppress the interference in the RD map representation of the passive bistatic radar, which exploits frequency modulation radio transmitters as illuminators. The AE and the synthetic RD map generation approach were introduced. The superiority of the proposed approach over other methods was demonstrated via simulation and using recorded signals in practical environments.

In [A9], the problem of DOA estimation in the presence of non-Gaussian, heavy-tailed, and spatially-colored interference was addressed. The challenge of the maximum-likelihood-based estimators in the presence of non-Gaussian and spatially colored interference was addressed by the DNN-based approach. By considering a scenario with an unknown number of sources, the proposed approach utilized a single NN instance for simultaneous source enumeration and DOA estimation.

In [A10], considering a sparse array for the DOA estimation, the supervised neural-network-based beamformer that maximizes the signal-to-interference ratio was introduced. Formulating the problem as a classification task, it was proposed to generate class labels using an efficient sparse beamformer spectral analysis approach, which uses explicit information from the unknown narrowband interference environment for the DNN training. The resulting DNN effectively approximated the unknown mapping from the input received data spatial correlations to the output of sparse configuration with effective interference mitigation capability.

In [A11], a passive radar interferometric inversion imaging algorithm using a spectral estimation method with a priori was proposed using a DNN-based algorithm. The DNN approach uses a priori information for modeling and improving sample efficiency of the interferometric inversion methods. The proposed approach uses the CNNs as a denoiser for passive SAR with the performance evaluated via numerical simulations.

In [A12], a motion tracking system using WiFi micro-Doppler signatures is proposed. The MDPose framework for human skeletal motion reconstruction represents human activity by reconstructing a skeleton model with 17 key points. The MDPose contains: 1) the denoising and weak Doppler measurements enhancement; 2) CNN-recurrent neural network architecture for temporal–spatial dependency learning; and 3) a pose optimization mechanism for the initial skeleton state estimation. The performance of the MDPose framework was evaluated using recordings from various environments and multiple daily activities.

In [A13], a metacognitive approach for radar target detection within various clutter distributions is presented. In contrast to the generalized likelihood ratio test detectors that assume the clutter can be modeled according to a consistent probability distribution, the proposed approach maintains constant-false-alarm-ratelike behavior in randomly selected data from a range of clutter distribution models (Gaussian, K, and Pareto). This makes the proposed detection approach especially attractive for practical airborne and naval applications, where the assumption of consistent clutter distributions typically does not hold.

The exponentially growing number and variety of DNN architectures, mainly in computer vision, multimedia, and speech processing, opens opportunities for innovative radar signal processing approaches, as shown in this special section. This special section provides some guidance on possible directions of DNN-based radar signal processing that have considerable potential to advance future radar applications.

Sevgi Z. Gurbuz, Lead Guest Editor Department of Electrical and Computer Engineering The University of Alabama Tuscaloosa, AL 35487 USA e-mail: szgurbuz@eng.ua.edu Igal Bilik, Guest Editor School of Electrical and Computer Engineering Ben Gurion University of the Negev Beer-Sheva 8410501, Israel e-mail: bilik@bgu.ac.il Luke Rosenberg, Guest Editor STELaRLab Lockheed Martin Australia Adelaide, SA 5000, Australia e-mail: luke.rosenberg@ieee.org

ACKNOWLEDGMENT

The guest editors sincerely thank all the authors and reviewers for their precious contribution, and they especially thank the Editor-in-Chief and staff members of IEEE Transactions on Aerospace and Electronic Systems for their constant support over the entire process from proposal to final publication.

1. J. W. Smith and M. Torlak, “Deep-learning-based multiband signal fusion for 3-D SAR superresolution,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 1, Feb. 2024, pp. 8–24.

2. L. Rosenberg, W. Zhao, A. Heng, L. Hamey, S. T. Nguyen, and M. A. Orgun, “Classification of ISAR ship imagery using transfer learning,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 1, Feb. 2024, pp. 25–36

3. S. Akbar, R. S. Adve, Z. Ding, and P. W. Moo, “Transfer-based DRL for task scheduling in dynamic environments for cognitive radar,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 1, Feb. 2024, pp. 37–50.

4. S. Wagner, W. Johannes, D. Qosja, and S. Brüggenwirth, “Small target detection in a radar surveillance system using contractive autoencoders,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 1, Feb. 2024, pp. 51–67

5. A. N. Sayed, O. M. Ramahi, and G. Shaker, “Machine learning for UAV classification employing mechanical control information,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 1, Feb. 2024, pp. 68–81.

6. M. C. Wilmanski and J. I. Tamir, “Differentiable rendering for synthetic aperture radar imagery,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 1, Feb. 2024, pp. 82–93.

7. C. Y. Pui, B. Ng, L. Rosenberg, and T.-T. Cao, “Target classification for 3D-ISAR using CNNs,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 1, Feb. 2024, pp. 94–105.

8. D.-H. Park, G.-H. Park, J.-H. Park, J.-H. Bang, D. Kim, and H.-N. Kim, “Interference suppression for an FM-radio-based passive radar via deep convolutional autoencoder,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 1, Feb. 2024, pp. 106–118.

9. S. Feintuch, J. Tabrikian, I. Bilik, and H. Permuter, “Neural-network-based DOA estimation in the presence of non-Gaussian interference,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 1, Feb. 2024, pp. 119–132.

10. S. A. Hamza and M. G. Amin, “Sparse array design for optimum beamfoming using deep learning,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 1, Feb. 2024, pp. 133–144.

11. S. Kazemi, B. Yonel, and B. Yazici, “Interferometric passive radar imaging with deep denoising priors,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 1, Feb. 2024, pp. 145–156.

12. C. Tang, W. Li, S. Vishwakarma, F. Shi, S. Julier, and K. Chetty, “MDPose: Human skeletal motion reconstruction using WiFi micro-Doppler signatures,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 1, Feb. 2024, pp. 157–167.

13. A. Stringer, G. Dolinger, D. Hogue, L. Schley, and J. G. Metcalf, “A meta-cognitive approach to adaptive radar detection,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 1, Feb. 2024, pp. 168–185.