It has been roughly a decade since the first papers
using deep neural networks (DNNs) for radar applications were published.
Deep learning has revolutionized almost every technical area, from
computer vision and natural language processing to health, finance, and
biology—any field where data can be analyzed to provide insight.
However, in radar applications, deep learning faces unique challenges
due to the phenomenology of radio frequency (RF) propagation that
creates essential differences in the data itself and impacts the design
of DNNs for radar signal analysis [1], [2], [3].
Unlike
computer vision, radar data are not inherently an image. The received
RF signal is a time series of complex samples, which, after radar signal
processing, can be converted into various 2D, 3D, or even 4D tensors:
range–Doppler (RD) maps or range–angle maps as a function of time,
micro-Doppler signatures, RD–angle cubes as a function of time,
RD–azimuth–elevation tensors, synthetic aperture radar (SAR) images,
high-resolution range profiles, and others. These representations have
been utilized in the literature as inputs to DNNs for classification [4], [5].
However, optimizing the best input representation is challenging when
many require varying degrees of computational complexity, memory
requirements, and computation time (latency). A wide range of input
representations appear in the articles included in this special section.
Closely
related to input representation is the development of complex neural
networks, especially as raw RF data are complex, and therefore, complex
values should be preserved in computed high-dimensional representations
at the DNN input. Complex representations of RF data have recently
garnered increased attention as they capture more information via the
representation of signal phase and amplitude. One way to address complex
data is to separate the real and imaginary parts of the signal into two
separate channels, which are then processed by separate networks but
jointly exploited using feature-level or decision-level fusion. While
this approach is easily implementable using existing tools, it does not
completely capture the relationship between the real and imaginary
components of the RF signal. More recently, however, complex neural
networks that utilize complex convolutions, max pooling, and activation
functions have been proposed to learn from complex data [6], [7]
directly. Naturally, these architectures are of great interest to the
radar community, which is finding new and innovative ways of exploiting
complex DNNs for radar data analysis. Thus, the authors in [A1] utilize a
complex convolutional neural network (CNN) when processing radar data.
Of
course, a challenge faced in all deep learning applications, but
perhaps most acutely in the radar domain, is that posed by training
under limited data availability. Most DNNs, such as CNNs, are randomly
initialized prior to model training. However, because the objective
functions of DNNs are highly convex, containing many local minima,
random initialization and training with gradient-based optimization may
not result in convergence to global minima. Consequently, randomly
initialized networks typically require considerable training sample
support. The two main approaches to minimize training data requirements
are unsupervised pretraining and transfer learning. In unsupervised
pretraining, an autoencoder (AE) or convolutional AE (CAE) is greedily
pretrained to learn nonlinear input transformations such that identity
mapping is learned. When the decoder is removed, the encoder weights,
trained without labels, provide a much better initialization for network
weights than random initialization, thereby minimizing supervised
training data requirements. Alternatively, in transfer learning, data
from other sources are utilized to initialize the network—this could be
data from different sensors or synthetic RF data simulated from
physics-based models or using generative learning. For example, transfer
learning techniques are exploited in [A2] and [A3] and AEs in [A4],
whereas radar simulation methods are advanced in [A5] and [A6].
While
initial works involving deep learning for radar applications have
focused almost exclusively on automatic target recognition and
classification, recently, there has been an increase in work exploiting
deep learning for more fundamental radar signal processing tasks and
other radar functions. The papers in this special section contribute new
techniques for the application of deep learning to target recognition
[A2], [A5] and detection [A4], [A7], [A13], interference mitigation
[A8], direction-of-arrival (DOA) estimation [A9], beamforming [A10],
radar imaging [A1], [A6], [A11], human pose estimation [A12], and
cognitive radar [A3], [A13]. The synergy between signal processing with
deep learning [8] enables
the advancement of novel algorithms that can consider
environment-specific variables. Cognitive radar takes this one step
further by considering adaptive processing and adaptive
transmission—also called fully adaptive radar—to optimize performance in
dynamic environments [9], [10]. Integral to the design of cognitive radars is learning and metacognition—the
process of thinking and learning—which are essential cognitive
processes embodied by all intelligent agents. How deep learning can be
exploited toward these aims represents an essential challenge whose
solution will pave the way for next-generation radar systems.
SECTION II.
Main Contributions of the Articles in This Special Section
In
[A1], the challenge of achieving high bandwidth for high-resolution 3D
synthetic aperture imaging is addressed through the design of a hybrid,
dual-domain complex-valued CNN to fuse multiband signals and infer the
missing samples in the frequency gaps between subbands. In particular,
results are shown for the fusion of two millimeter-wave radars, each
with a bandwidth of 4 GHz operating at 60 GHz and 77 GHz, to achieve an
effective bandwidth of 21 GHz by robustly estimating the full-band
signal. The proposed DNN, kR-Net, provides high-fidelity images of
concealed weapons, such as a knife with a serrated edge or pistol, when
placed in a cardboard box.
In [A2], transfer learning is utilized
to enable the recognition of ships in inverse SAR (ISAR) imagery in an
aspect angle agnostic fashion. To overcome the challenge of limited data
and better generalize across angles, four different DNNs (VGG16, VGG19,
Inception-v3, and Xception) are pretrained on optical imagery in the
ImageNet database [11].
The results show that transfer learning can enable consistently high
classification accuracy even when the data exhibits variability in
appearance due to the ship's motion and viewing geometry.
In [A3],
transfer learning is utilized within a reinforcement learning framework
to enable cognitive radar to adapt to dynamic changes in the
environment. A Monte Carlo tree search (MCTS) is aided by a DNN while
transfer learning accelerates the transfer of the policy learned by
training the DNN based on MCTS on an initial parameter distribution
(environment) to the policy required for a new distribution. In this
way, the authors show that transfer learning can be used to enable the
cognitive radar to adapt to both rapid and gradual changes in the
environment while producing near-optimal performance with 20-fold less
time and data.
In [A4], small radar cross-section targets, namely
drones, are detected using contractive AEs—an AE variant that adds a
penalty term consisting of the Frobenius norm of the Jacobian matrix of
encoder activations with respect to the input. It has been shown [12]
that this penalty enables the AE to be robust to slight variations in
the input by better capturing the local directions of variation
represented in the data, which correspond to a lower-dimensional
non-linear manifold while being invariant to directions orthogonal to
the manifold. Thus, contractive AEs are more robust to noise with
results showing improved target detection probability as a result.
Drone
detection and identification is an important, emergent area of research
due to the widespread availability of drones and potential misuse for
nefarious purposes. Complementing the aforementioned work on drone
detection, in [A5], mechanical control information is employed to
improve UAV classification accuracy. Each drone has its own set of
unique RD signatures that depend on the drone's movement mechanism.
While model-based simulation is a common technique to pretrain DNNs for
radar signature classification, the failure of simulations to capture
the variations in the drone signature due to its motion is a limitation
on the efficacy of such pretraining. Typically, drones have eight main
unique movements: the drone is able to throttle up, throttle down, pitch
forward, pitch backward, roll left, roll right, yaw left, and yaw
right. This work utilizes full-wave electromagnetic models combined with
the kinematic model of the drone movements to improve the recognition
accuracy of two quadcopters, a hexacopter, and an unmanned helicopter.
In
[A6], a knowledge-aided approach to improve deep model training. In
particular, the modeling of geometric priors and constraints in
optimization pipelines is proposed to train DNNs that are more robust
under limited data, with an improved capability to solve ill-posed
inverse problems, such as 3D object reconstruction with SAR. The
proposed approach combines methods from 3D computer graphics with neural
rendering and is the first work (to the best of our knowledge) to
develop differentiable rendering for SAR domain imagery.
In [A7], a
different radar data representation is explored for application to
maritime target recognition, namely, a 3D-ISAR technique is developed to
generate a 3D point cloud representation of the target. Features are
extracted from the 3D-ISAR-generated point cloud of the target from
different perspectives (side, top, and front views) to form three point
density images, which are then utilized by a CNN to classify the
targets. The results showed that recognition accuracy increased with
knowledge of aspect angle but that even when the angle was unknown,
simulations could be used to train the CNN to achieve high accuracy on
real datasets.
In [A8], the DNN-based approach was introduced to
suppress the interference in the RD map representation of the passive
bistatic radar, which exploits frequency modulation radio transmitters
as illuminators. The AE and the synthetic RD map generation approach
were introduced. The superiority of the proposed approach over other
methods was demonstrated via simulation and using recorded signals in
practical environments.
In [A9], the problem of DOA estimation in
the presence of non-Gaussian, heavy-tailed, and spatially-colored
interference was addressed. The challenge of the
maximum-likelihood-based estimators in the presence of non-Gaussian and
spatially colored interference was addressed by the DNN-based approach.
By considering a scenario with an unknown number of sources, the
proposed approach utilized a single NN instance for simultaneous source
enumeration and DOA estimation.
In [A10], considering a sparse
array for the DOA estimation, the supervised neural-network-based
beamformer that maximizes the signal-to-interference ratio was
introduced. Formulating the problem as a classification task, it was
proposed to generate class labels using an efficient sparse beamformer
spectral analysis approach, which uses explicit information from the
unknown narrowband interference environment for the DNN training. The
resulting DNN effectively approximated the unknown mapping from the
input received data spatial correlations to the output of sparse
configuration with effective interference mitigation capability.
In
[A11], a passive radar interferometric inversion imaging algorithm
using a spectral estimation method with a priori was proposed using a
DNN-based algorithm. The DNN approach uses a priori information for
modeling and improving sample efficiency of the interferometric
inversion methods. The proposed approach uses the CNNs as a denoiser for
passive SAR with the performance evaluated via numerical simulations.
In
[A12], a motion tracking system using WiFi micro-Doppler signatures is
proposed. The MDPose framework for human skeletal motion reconstruction
represents human activity by reconstructing a skeleton model with 17 key
points. The MDPose contains: 1) the denoising and weak Doppler
measurements enhancement; 2) CNN-recurrent neural network architecture
for temporal–spatial dependency learning; and 3) a pose optimization
mechanism for the initial skeleton state estimation. The performance of
the MDPose framework was evaluated using recordings from various
environments and multiple daily activities.
In [A13], a
metacognitive approach for radar target detection within various clutter
distributions is presented. In contrast to the generalized likelihood
ratio test detectors that assume the clutter can be modeled according to
a consistent probability distribution, the proposed approach maintains
constant-false-alarm-ratelike behavior in randomly selected data from a
range of clutter distribution models (Gaussian, K, and Pareto). This
makes the proposed detection approach especially attractive for
practical airborne and naval applications, where the assumption of
consistent clutter distributions typically does not hold.
The exponentially growing number and variety of DNN
architectures, mainly in computer vision, multimedia, and speech
processing, opens opportunities for innovative radar signal processing
approaches, as shown in this special section. This special section
provides some guidance on possible directions of DNN-based radar signal
processing that have considerable potential to advance future radar
applications.
Sevgi Z. Gurbuz, Lead Guest Editor
Department of Electrical and Computer Engineering
The University of Alabama
Tuscaloosa, AL 35487 USA
e-mail: szgurbuz@eng.ua.edu
Igal Bilik, Guest Editor
School of Electrical and Computer Engineering
Ben Gurion University of the Negev
Beer-Sheva 8410501, Israel
e-mail: bilik@bgu.ac.il
Luke Rosenberg, Guest Editor
STELaRLab
Lockheed Martin Australia
Adelaide, SA 5000, Australia
e-mail: luke.rosenberg@ieee.org
ACKNOWLEDGMENT
The guest editors sincerely thank all the
authors and reviewers for their precious contribution, and they
especially thank the Editor-in-Chief and staff members of IEEE Transactions on Aerospace and Electronic Systems for their constant support over the entire process from proposal to final publication.
J. W. Smith and M. Torlak, “Deep-learning-based multiband signal fusion for 3-D SAR superresolution,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 1, Feb. 2024, pp. 8–24.
L.
Rosenberg, W. Zhao, A. Heng, L. Hamey, S. T. Nguyen, and M. A. Orgun,
“Classification of ISAR ship imagery using transfer learning,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 1, Feb. 2024, pp. 25–36
S.
Akbar, R. S. Adve, Z. Ding, and P. W. Moo, “Transfer-based DRL for task
scheduling in dynamic environments for cognitive radar,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 1, Feb. 2024, pp. 37–50.
S.
Wagner, W. Johannes, D. Qosja, and S. Brüggenwirth, “Small target
detection in a radar surveillance system using contractive
autoencoders,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 1, Feb. 2024, pp. 51–67
A. N. Sayed, O. M. Ramahi, and G. Shaker, “Machine learning for UAV classification employing mechanical control information,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 1, Feb. 2024, pp. 68–81.
M. C. Wilmanski and J. I. Tamir, “Differentiable rendering for synthetic aperture radar imagery,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 1, Feb. 2024, pp. 82–93.
C. Y. Pui, B. Ng, L. Rosenberg, and T.-T. Cao, “Target classification for 3D-ISAR using CNNs,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 1, Feb. 2024, pp. 94–105.
D.-H.
Park, G.-H. Park, J.-H. Park, J.-H. Bang, D. Kim, and H.-N. Kim,
“Interference suppression for an FM-radio-based passive radar via deep
convolutional autoencoder,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 1, Feb. 2024, pp. 106–118.
S.
Feintuch, J. Tabrikian, I. Bilik, and H. Permuter,
“Neural-network-based DOA estimation in the presence of non-Gaussian
interference,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 1, Feb. 2024, pp. 119–132.
S. A. Hamza and M. G. Amin, “Sparse array design for optimum beamfoming using deep learning,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 1, Feb. 2024, pp. 133–144.
S. Kazemi, B. Yonel, and B. Yazici, “Interferometric passive radar imaging with deep denoising priors,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 1, Feb. 2024, pp. 145–156.
C.
Tang, W. Li, S. Vishwakarma, F. Shi, S. Julier, and K. Chetty, “MDPose:
Human skeletal motion reconstruction using WiFi micro-Doppler
signatures,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 1, Feb. 2024, pp. 157–167.
A. Stringer, G. Dolinger, D. Hogue, L. Schley, and J. G. Metcalf, “A meta-cognitive approach to adaptive radar detection,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 1, Feb. 2024, pp. 168–185.
/ @brettintech Share This Video
