A Geometric Auto-Calibration Method for Multiview UAV-Borne FMCW SAR Images | IEEE Journals & Magazine | IEEE Xplore

Schematic of geometric auto-calibration based on multiview spaceborne SAR images.

A Geometric Auto-Calibration Method for Multiview UAV-Borne FMCW SAR Images | IEEE Journals & Magazine | IEEE Xplore

Publisher: IEEE

PDF

Yitong Luo; Xiaolan Qiu; Yao Cheng

---

Abstract

Document Sections

• I. Introduction
• II. Basic Theory and Model for SAR Geometric Auto-Calibration
• III. Methodology
• IV. Experiment
• V. Conclusion

Abstract:

In recent years, the geometric calibration for synthetic aperture radar (SAR) has been developing toward automation and lack of ground control points (GCPs). In this case, the calibration equations become ill-conditioned, making the results unstable. The slant range error of unmanned aerial vehicle (UAV)-borne SAR, whose transmitting signal is a frequency modulation continuous wave, is usually spatially variable, which makes this problem more complicated. In this letter, a geometric auto-calibration method for UAV-borne SAR is proposed. First, a robust auto-calibration model is proposed to calibrate the space-varying slant range error. Second, a weighting strategy based on the quality of tie points (TPs) is proposed to further improve the calibration accuracy of this method. Finally, an iterative method based on correcting characteristic values is utilized to find the least squares solution robustly. Simulation results confirm about 70% performance enhancement of the proposed method over the traditional approach. In real-data experiments, the auto-calibration yielded a positioning accuracy improvement to 11.5 cm.

Published in: IEEE Geoscience and Remote Sensing Letters ( Volume: 21)

Article Sequence Number: 4001805

Date of Publication: 08 January 2024

ISSN Information:

DOI: 10.1109/LGRS.2024.3350903

Publisher: IEEE

Funding Agency:

SECTION I. Introduction

Unmanned aerial vehicle (UAV)-borne synthetic aperture radar (SAR) has high commercial value and great application potential, because of its advantages of miniaturization, high mobility, low cost, and easy modification. At present, UAV-borne SAR has been successfully applied in terrain mapping, urban 3-D reconstruction, and other fields, and has achieved remarkable results [1], [2].

Similar to spaceborne SAR, geometric calibration is also utilized to improve the geometric accuracy of UAV-borne SAR, which directly affects the mapping accuracy. In addition, 3-D control points made by stereo SAR positioning can also be utilized to determine the absolute elevation for interferometric SAR (InSAR) and to calibrate phase errors for tomographic SAR (TomoSAR) [3], [4]. Therefore, geometric calibration also has an important impact on InSAR and TomoSAR.

The common geometric calibration methods are divided into three categories: field-based calibration [5], [6], cross-calibration [7], and auto-calibration [8]. Field-based calibration is a reliable method and has high accuracy, but depends on a large number of ground control points (GCPs). Cross-calibration utilizes a reference image that has been calibrated to correct another image that has a similar incidence angle. On the contrary, auto-calibration does not require any GCPs and reference images. It estimates the calibration parameters only by multiview overlapping images [8], [9], [10]. Therefore, auto-calibration is an effective and feasible method when GCPs are lacking.

Geometric auto-calibration is an ill-conditioned problem, and most of the current research is based on spaceborne SAR. Zhang et al. [8] utilized multiview spaceborne SAR images to carry out an auto-calibration experiment without GCPs, achieving a planar accuracy of 3.83 m. In this experiment, many overlapping images were utilized to deal with the ill-conditioned normal equations. At the same time, multiview overlapping images can also reduce the disturbance of orbit and velocity errors to a certain extent. Xu et al. [9] utilized those SAR images constrained by symmetric geometry to calculate high-accuracy 3-D coordinates of tie points (TPs), and thus estimated calibration constants. However, their method is demanding for SAR images and even requires a high-accuracy digital elevation model (DEM). Therefore, it can only be regarded as semi-auto-calibration in essence. In 2023, Yin et al. [10] proposed a twofold calibration method. The key of their method is to decouple the solution of the 3-D coordinates of the TPs and the solution of the calibration constants, and thus improve the stability of the method.

However, the differences between UAV-borne SAR and spaceborne SAR necessitate modifications to the auto-calibration method, presenting both reasonable simplifications and intricate challenges. First, due to the low speed of the UAV, the azimuth positioning error caused by azimuth time shift (less than 1 ms) is usually less than 1 cm. Hence the slant range error is dominant in the UAV-borne SAR. Second, in comparison to satellites, UAVs operate at lower flight altitudes, making atmospheric propagation delay negligible. Last but not least, since the payload of the UAV is limited, the SAR mounted on it needs to be miniaturized. Therefore, the transmitting signals of UAV-borne SAR are mostly frequency-modulated continuous waves (FMCWs), rather than pulse-modulated signals utilized in spaceborne SAR. In this case, the dispersion effect on the FMCW system loop can cause a space-varying slant range error [2]. Specifically, slant range errors may vary linearly with range coordinates, which is more complex than in the spaceborne SAR case, where such errors are almost constant. In this case, the method proposed in [10] is unable to be directly applied to UAV-borne FMCW SAR, because the subtraction of the range equations of two different incidence angles cannot offset the slant range error. The model proposed in [8] also needs to be improved for the space-varying slant range errors. In this letter, we propose a geometric auto-calibration method for multiview UAV-borne FMCW SAR. The proposed method comprehensively improves the stability of auto-calibration from three aspects: improved model, weighting strategy for TPs, and robust solution algorithm.

The rest of this letter is organized as follows. Section II recaps the basic theory and model of geometric auto-calibration. Section III describes the proposed auto-calibration method in detail. The experimental data set and results are presented in Section IV. Our conclusions are drawn in Section V.

SECTION II. Basic Theory and Model for SAR Geometric Auto-Calibration

Traditional spaceborne SAR geometric auto-calibration detects slant range errors according to the differences of the spatial intersection points of multiple stereo image pairs, so it does not need a calibration field and reference image [8], [11].

As shown in Fig. 1, the ground target point P is photographed three times by an SAR satellite from different incidence angles. The positions of this satellite are S1 , S2 , and S3 . The corresponding slant ranges are R1 , R2 , and R3 . When there is no slant range error, the spatial intersection of these three images should be exactly at P . However, when there is a slant range error RS in these images, the intersection points between the two of these three images will appear at different positions, assuming that they are points A , B , and C , respectively. This intersection difference is called the intersection residual. Thus, the slant range error can be detected by minimizing the intersection residual. Assuming there are N multiview SAR images with M TPs on the overlapping area, the traditional geometric auto-calibration model is as follows:

minRS,Pj,j=1,2,…,ML=∑j=1M⎧⎩⎨⎪⎪⎪⎪∑i=1N⎧⎩⎨⎪⎪⎪⎪[∥Pj−Sij∥−(Rij+RS+Ratmo,i)]2+[2Vij⋅(Pj−Sij)λ⋅(Rij+RS+Ratmo,i)−fDij]2⎫⎭⎬⎪⎪⎪⎪⎫⎭⎬⎪⎪⎪⎪(1)

View Source where Pj(Xj , Yj , Zj ) is the 3-D coordinate of the j th TP, Sij (XSij , YSij , ZSij ), Vij (VXij , VYij , VZij ), fDij , and Rij are the satellite position, satellite velocity, Doppler center frequency, and slant range of the j th TP corresponding to the i th SAR image, Ratmo,i is the atmospheric propagation delay of the i th SAR image, and RS is the slant range error to be calibrated. The auto-calibration treats both RS and Pj as unknowns and solves them simultaneously.

Fig. 1. Schematic of geometric auto-calibration based on multiview spaceborne SAR images.

Show All

In general, we can solve (1) by the Gauss-Newton iteration method. First, we linearize (1) to get the following system of linear equations:

A⋅x=b(2)

View Source where A is the Jacobian matrix, b is the residual vector, and x is the vector to be solved at each iteration. The vector x contains the 3-D coordinates of the TPs and the calibration parameters. The least squares solution of (2) is thus,

x=(ATA)−1⋅ATb.(3)

View Source

The 3-D coordinates of the TPs and the calibration parameters are then updated. The next iteration continues until it converges.

SECTION III. Methodology

A. Improved Auto-Calibration Model

Considering the differences between UAV-borne FMCW SAR and spaceborne SAR, we first propose an improved auto-calibration model based on (1), as shown below

minRS0,RS1Pj,j=1,…,MfEi,i=1,…,NL=∑j=1M⋅⎧⎩⎨⎪⎪⎪⎪∑i=1N⎧⎩⎨⎪⎪⎪⎪[∥Pj−Sij∥−(Rij+RS0+RS1⋅xij)]2+[2Vij⋅(Pj−Sij)λ⋅(Rij+RS0+RS1⋅xij)−(fDij+fEi)]2⎫⎭⎬⎪⎪⎪⎪⎫⎭⎬⎪⎪⎪⎪(4)

View Source where RS1 and RS0 are the coefficients of the primary and constant terms of the slant range error, xij is the range coordinate of the j th TP on the i th image, fEi is the equivalent Doppler center frequency error of the i th image, and the other parameters have the same meaning as in (1).

As stated in Section I, due to the low flight speed and altitude of the UAV, the effects of azimuth time shift and atmospheric propagation delay [Ratmo,i in (1)] can be ignored. In our proposed model, the calibration parameters include the primary term and the constant term coefficients to accommodate the slant range errors which vary linearly with the range coordinates. In addition, the traditional model ignores the residual errors of the trajectory position and velocity, which can be propagated to the slant range error. This is also one of the reasons for the instability of calibration results. Therefore, the term fEi is also added to the Doppler equation to absorb the equivalent error of the trajectory position and velocity errors. The experiments in Section IV will show that this parameter works indeed.

B. Weighting Strategy for TPs

In the geometric auto-calibration, we will select several objects as TPs in the surveyed area. Ideally, these TPs should meet two requirements: on the one hand, these TPs should be isolated point-like targets as much as possible to ensure that their scattering centers are the same at different incidence angles. On the other hand, the distribution conditions of these TPs should be good. This means that these TPs cover a large area and are uniformly distributed. For example, street lamps arranged regularly along a road meet the above requirements well. Based on the above requirements, we design a weighting strategy for TPs to further improve the stability and accuracy of the proposed method.

We introduce the peak sidelobe ratio (PSLR) as the first factor to evaluate the quality of a TP. PSLR is an important indicator to evaluate the imaging quality of a point target. From another perspective, it can also be utilized to evaluate whether the coordinates of the pricked point-like target are accurate. In the case of a street lamp, the backscattering of its base resembles a point-like target. However, some appendages on the lamp pole may disrupt the acquisition of the coordinates of the base. The PSLR then can be utilized to weaken the weights of those TPs whose coordinates may be inaccurate.

Second, in practice, the distribution of TPs is not ideally wide and uniform. Therefore, we design a second factor, the distribution condition factor (DCF), to evaluate the distribution condition of a TP. The DCF is the product of the covering factor (CF) and the uniform factor (UF)

 DCFj=CFj=CFj⋅ UFjΓ(∑k=1,k≠jMdis(pj,pk)), UFj=1Nnbh,j(5)

View Source where pj represents the j th TP, dis(⋅ ) is a function of the Euclidean distance between two TPs, Γ(⋅) is a normalized function, and Nnbh,j is the number of TPs in the neighborhood of the j th TP. According to the optimization theory, the CF can strengthen the weights of TPs that lie on the edge of a group of TPs, which determines the coverage of this group of TPs. The UF can weaken the weights of TPs clustered in a small neighborhood. Finally, the weight of the j th TP on the i th image is calculated by the following formula:

wij=|PSLRij⋅ DCFj|(6)

View Source where PSLRij represents the PSLR of the j th TP on the i th image. The weight wij is calculated by PSLRij and DCF_j to evaluate the image quality and distribution conditions of a TP respectively.

C. Least Squares Solution Based on the IMCCV

As mentioned above, the geometric auto-calibration without GCPs is an ill-conditioned problem. Specifically, the condition number for the normal equation ATA in (3) is large. In order to reduce the condition number and unbiasedly estimate x . We utilize the iteration method by correcting characteristic value (IMCCV) to find the least squares solution of (4). First, (3) is equivalently transformed to

(ATA+I)⋅x=ATb+x(7)

View Source where I is an identity matrix.

Subsequently, we solve for x iteratively. The iterative formula is as follows:

x(k)=(ATA+I)−1⋅(ATb+x(k−1))(8)

View Source where x(k) is the solution of (7) for the k th iteration, and x(0) = 0. Thus, we only need to calculate the inverse of AT A + I, instead of the inverse of ATA .

We assume that λ is the eigenvalue of ATA , and thus the eigenvalue of AT A + I is λ + 1. We assume that the maximum and minimum eigenvalues of ATA are λ max and λ min, respectively, then the condition numbers for ATA and AT A + I are

 cond(ATA)=λmaxλmin, cond(ATA+I)=λmax+1λmin+1.(9)

View Source

Since ATA is ill-conditioned, that means λmin≪λmax , then

 cond(ATA+I)≪ cond(ATA).(10)

View Source

Therefore, IMCCV can reduce the condition numbers, thus making the solution more robust. Moreover, according to [12], when x(0) = 0, IMCCV is unbiased and convergent.

SECTION IV. Experiment

In order to verify the effectiveness and superiority of the proposed method, we conducted two sets of experiments based on simulated data and real data respectively. Both experiments were carried out based on microwave vision 3-D SAR (MV3DSAR). MV3DSAR is a small UAV-borne array InSAR system developed by Qiu et al. [2] in 2021. This system is equipped with four antennas and operates in the Ku-band. Its transmitting signal is FMCW, so we need to calibrate a space-varying slant range error. Detailed information about the MV3DSAR is shown in Table I.

TABLE I Description of the MV3DSAR: POS Means Position and Orientation System

In order to evaluate the calibration accuracy of the slant range, we define its relative estimation accuracy of the primary term and constant term as

εRS1=R^S1−R∗S1R∗S1,εRS0=R^S0−R∗S0R∗S0(11)

View Source where R^S1 and R^S0 are the estimated values of the primary and constant terms respectively. R∗S1 and R∗S0 are the true values of the primary and constant terms respectively.

A. Experiments Based on Simulated Data

In the simulation experiment, the parameters of the SAR, UAV, and POS were simulated based on Table I. The errors that affect the positioning accuracy of SAR were added to the simulation system, including the errors in trajectory position, velocity, and slant range [7]. The slant range error was set to vary linearly with the range coordinates. The error sources are shown in Table II.

TABLE II Error Sources Introduced in the Simulation Experiment

Then, we randomly generated 10 TPs and different flight tracks in a simulated scene. As shown in Fig. 2, there are six groups of experiments, each of which contains 3–8 randomly generated flight tracks. The distribution of the simulated TPs is also shown in Fig. 2. Finally, three methods of the traditional model, improved model, and weighted improved model were utilized to calibrate the slant range errors. At the same time, the 3-D coordinates of the TPs were also solved. We repeated the experiment 20 times for each group. The results shown in Table III are the root mean square error (RMSE) of 20 repeated experiments.

TABLE III Simulation Experiment Result: the Traditional Model Refers to (1), But Removes Ratmo,i and Changes Rs to Linear Form. The Improved Model Refers to (4). The Weighted Improved Model Applies the Proposed Weighting Strategy Based on (4)

Fig. 2.

Simulated flight tracks and the distribution of TPs. The color of a TP represents the value of DCF. The color bar shows the DCF values for different colors. The numbers in the upper left represent the number of flight tracks.

Show All

Because these simulated TPs are ideal point targets, their PSLRs are the same. The weights calculated in the simulation experiment only reflect the DCFs. The colors of the TPs in Fig. 2 represent the values of DCFs. We utilized CFs to strengthen the weight of TPs that determine the coverage of these TPs, that is, the TPs on the edge. At the same time, we utilized UFs to reduce the weight of those TPs that are too close together. The weights shown in Fig. 2 are exactly what we would expect.

According to Table III, the space-varying slant range error makes the traditional method unstable. Especially when the number of observations is insufficient, the errors of the trajectory position and velocity propagated to the slant range, resulting in low calibration accuracy. Our improved model utilized fEi to absorb some of these errors, resulting in a significant improvement in accuracy. Moreover, we assigned weights to different TPs based on the distribution conditions of each TP, which further improved the calibration accuracy. In general, this simulation experiment proves that the improved model and the weighting strategy are effective.

B. Experiments Based on Real Data

We conducted an experiment in Tianjin based on MV3DSAR to verify the effectiveness of the proposed auto-calibration method. First, we set up eight calibrators as check points (CPs) in the surveyed area. Their GPS survey results are taken as true 3-D coordinates. The UAV flew along the routes shown in Fig. 3(b), acquiring SAR images from eight perspectives. We pricked ten artificial targets as TPs on the overlapping area. These TPs were street lamps. Fig. 3 shows the distribution of TPs and CPs.

Fig. 3. (a) Optical images of the surveyed area and a street lamp (CPs marked in red and TPs marked in yellow). (b) Flight tracks and the distribution of TPs and CPs. The colors of TPs represent the weights calculated by DCFs and PSLRs. The color bar shows the weight values for different colors.

Show All

In fact, these street lamps are not ideal point targets. Their scattering centers are usually the bases of the lamp pole, but the appendages on the lamp pole sometimes seriously affect pricking points. Therefore, the coordinates of a TP on different images may not be the same point. Fig. 4 shows SAR images of a street lamp taken from two incidence angles.

Fig. 4. (a) SAR images of a street lamp taken from two incidence angles and their azimuthal PSLRs. (b) Up-sampled images of (a). (c) Azimuthal impulse responses of (a).

Show All

The scattering center shown in the first row of Fig. 4 is the base of a lamp, which is similar to an ideal point target and has a small PSLR. The scattering center in the second row is affected by the appendages, so the coordinates of the pricked point may not be accurate, resulting in a large PSLR. Therefore, our proposed weighting strategy weakened the weights of TPs with large PSLR. At the same time, DCF assigned different weights to different TPs based on their distribution conditions. The colors of those TPs in Fig. 3 represent their weights. Finally, the least squares solution was robustly solved based on IMCCV. Then, we also performed a field-based calibration for comparison based on those CPs. The result of field-based calibration was regarded as a reference value.

Fig. 5 and Table IV show a comparison of different calibration methods. The purple points in Fig. 5, representing slant range errors calculated by CPs, clearly demonstrate a linear variation with the range coordinates. The trend of the error curve estimated by the traditional model is similar to the reference curve. However, it does not take into account the residual error of trajectory position and velocity, resulting in a deviation from the reference curve. Our improved model adds fEi to absorb some of these errors, thus improving its performance. Further, the proposed weighting strategy evaluated different TPs based on the DCFs and PSLRs, and finally obtained high-precision calibration parameters. Table V shows the 3-D positioning accuracy of CPs with different calibration methods. The traditional auto-calibration method is better than the non-calibration, but it still has a positioning error of 150.5 cm. Our improved model can improve the positioning accuracy to 22.5 cm. After assigning weights to different TPs, the positioning accuracy is further improved to 11.5 cm, which is close to the accuracy of the field-based calibration. In summary, the improved model and the proposed weighting strategy can effectively improve the calibration accuracy.

TABLE IV Calibration Result of the Slant Range Errors: The Meaning of Each Model is the Same as in Table III

TABLE V Three-Dimensional Positioning Accuracy of CPs After Geometric Calibration by Different Methods: The Meaning of Each Model in the Auto-Calibration is the Same as in Table III

Fig. 5. Slant range error curves estimated by different methods.

Show All

SECTION V. Conclusion

In this letter, we proposed a geometric auto-calibration method based on multiview UAV-borne FMCW SAR images. This method comprehensively improves the accuracy and stability of the auto-calibration from three aspects: improved model, weighting strategy, and robust solution algorithm. In experiments based on simulated and real data, we show the space-varying slant range error of the UAV-borne FMCW SAR. Our method can calibrate such errors robustly and obtain more accurate calibration results. Theoretically, the proposed method also has the potential to be applied to spaceborne SAR.