Previous Abstract
Return to Session D6
Next Abstract
### Session D6: Navigation Using Environmental Features

Previous Abstract Return to Session D6 Next Abstract

**Navigation Augmentation for Landing on Vertipads Utilizing Optical Detection of Standard ICAO Circular Markings**

*Finn Hübner, Robert Haupt, Ulf Bestmann, Peter Hecker, TU Braunschweig*

**Date/Time:** Friday, Sep. 20, 2:58 p.m.

This paper reviews the applicability of a navigation calculation based on circular landing pad detection for the landing of manned VTOLs on standard helipads/vertipads. The landing of VTOLs in an urban environment poses multiple challenges and requirements regarding its positioning. Supplementing a GNSS/INS pose by exploiting the feature rich visual environment has been shown to make a substantial contribution towards resilience and integrity of a navigation system. In an approach using only existing markings which are specified in ICAO Annex 14 Vol. II or PTS-VPT-DSN there is a significant prevalence of circular markings for which a concept for detection and relative pose estimation is studied in this paper. First, the landing pad design for VTOLs and helicopters is characterized. Algorithms for the detection of circular markings are developed and their performance is evaluated. Geometric relations in the projective geometry of oblique elliptical cones are exploited to generate a pose estimate with a remaining one degree of freedom. Solutions to fully determine the position are discussed. Based on simulated approach data and full scale flight test measurement data together with precise reference positions the performance of the solution in terms of availability and accuracy is analyzed. Finally, the potential for supplementing an INS/GNSS navigation solution with respect to the application for the landing of manned VTOLs is evaluated.

We are able to show the availability of a detected circular marking for the majority of the final descent while the selected filter criteria lead to low rates of false detections. Sub-meter accuracy for slant-range to the circle and relative height is achieved. The absolute heading can be estimated by evaluating the identification marking („V“ or „H“) inside the circular marking although with much higher uncertainty and not at all distances. Position Confidence is not evenly distributed by a confidence ellipsoid, but the solution can still be used by a flight controller and integrity checking for a GNSS/INS solution can be conducted.

Significance of the study

Landing operations of VTOLs and Helicopters in urban environments pose advanced challenges and high requirements on any navigation solution. GNSS systems are exposed to the effect of signal shading, multipath and interference issues but at the same time require sufficient integrity levels and error detection capabilities. Observation of the feature rich visual environment is a major source of information human pilots use for navigation and it has been shown that relative optical positioning can augment GNSS/INS navigation systems as an independent solution for landing, in example during the C2Land project.

While it has been shown that special optical markers such as ArUco markers have benefits in reliable identification [1] it appears favourable to constrain to what is already marked and specified to be on vertipads and helipads. No change of the specification is required and no additional prerequisites are required for such a solution. In the specification for visual aids on helipads defined in ICAO Annex 14 Vol. II and the prototype specifications for vertipads PTS-VPT-DSN both can be of circular or rectangular shape. Although the statistical distribution between circular and rectangular helipads is not known, it can be expected to see both in considerable count. The outer and most prominent markings are the touchdown and lift-off area (TLOF) which follows the vertipad/helipad shape and the touch down positioning marking (TDPM) which is a circular marking. As a vertipad identification marking a filled blue circle with an “V” inside is proposed. Position estimation by 2D to 3D correspondences from detected corner points of rectangular markings can be achieved by solving the well studied perspective-n-point problem with bundle adjustment. Circular markings do not feature direct corner points for such a calculation. On the other hand circular markings have positive characteristics, such as tolerance against occlusion and motion blur and are expected to be detectable even from greater distances [2].

Studies exist on the detection of circular markings for relative navigation and vertical landing of drones. However, very few study the characteristics and requirements of full scale manned operations. Additionally mainly specific preconditions or markings are exploited. We conduct simulations and full scale flight tests with accurate ground truth INS/GNSS and radar-altimeter data available.

Objectives, main contributions and key innovative steps

The objective for this paper is to increase resilience and integrity of a navigation solution for the landing of VTOLs by developing and evaluating a concept for pose estimation by detection of circular markings. Circular marker detection and pose estimation is one building block among several computer vision algorithms to develop a navigation system that enables the automatic landing of helicopters and VTOLs.

Our first contribution is an analysis of the specification of markings for helipads and vertipads followed by the investigation of existing computer vision algorithms for circular marking detection. Based on real scale flight tests data and augmented or simulated approaches the performance of these algorithms for detection is evaluated and an own concept for detection and filtering of the markings is derived.

Three algorithms for ellipse detection have been selected for testing from literature. The algorithms were required to be real-time capable in existing applications, and to be able to detect ellipses independent of their size and aspect ratio.

One algorithm follows the steps outlined by Lu et al. [3] and represents a general ellipse detection algorithm based on creating line segments from edges. The line segments are grouped into arcs to which an ellipse is fitted. The distance from ellipse to the line segments evaluates the ellipse. The second algorithm evaluated on datasets is based on the algorithm proposed by Jia et al. [4]. This algorithm is also a general ellipse detector. The edges of an image are split into arcs corresponding to the quadrants of a circle. The arcs are grouped into candidate ellipses based on multiple geometric conditions and evaluated against the edges. The third algorithm by Keipour et al. [5] fits ellipses to all contours and compares both representations to each other. This method is already used in robotic and drone applications to identify landing sites in images. A modification of this algorithm is introduced to split branching contours in order to resolve elliptical contours independently.

Vertipad markings can be identified from the detected ellipses using multiple methods. One method uses the concentricity of the circular markings to filter the ellipses into groups. The groups are generated only allowing a maximum centre distance defined as a fraction of the ellipse size. Another method uses the color of the vertipad markings to filter the image and identify rings. The center of the ring is calculated and compared against the centres of the ellipses. The allowed distance to assign an ellipse to a vertipad candidate from ellipse to ring centre is again limited to a fraction of their sizes. Using the known diameter ration between markings a region of interest can be generated from the filtering step, which includes the complete vertipad. When using the colour-filter as a necessary confirmation step, ellipses only need to be detected inside the region of interest, which reduces computation time and false-positives.

Our own proposed detector and filter exploits the fact, that the TLOF-, FATO- and TDPM-markings consist of two concentric circles with a defined diameter ratio. Therefore when detecting nested contours they can be filtered twofold. First each contour is checked to be of general elliptical shape. For this we consider only contours bigger than a minimal size, filter contour lengths greater than the circumscribing bounding box and check for the convexity and complexity of the contour. The filtered ellipse candidates are then checked to have a second inner ellipse contour and the area ratio and color then classify the candidates.

Second, we contribute two concepts and compact implementations for deriving pose and position information mathematically by exploiting the projective geometry. One straightforward estimation of slant range and height can be calculated if the assumption holds true that the distance to the circle is large compared to the circle diameter and that the image plane is orthogonal to the direction to the circle. Then the simplifying conditions of the parallel projection can be assumed and the center of the ellipse corresponds exactly with the projected center of the circle on the image plane. Likewise, the major semi-axis of the ellipse is parallel to the ground plane and the eccentricity of the projected ellipse depends entirely on the angle ? between the viewing direction and the normal vector of the ground plane. If these simplifications are adopted to the central projection, the length of the major semi-axis is directly proportional to the distance to the center of the circle. From this, the height can be determined using trigonometric relationships. The appropriateness and validity of the simplifying assumptions is limited for the application under consideration and the exact limits will be determined in an experimental analysis.

Without these simplifications the observation of a circle by a camera lead to the the more complex projective geometric relations of an oblique elliptical cone. However, with this one can derive the pose with one remaining degree of freedom. First, from the detected ellipse contour on the image plane the matrix representation of the general ellipse quadric is formed. Using the known intrinsic parameters of the camera their effects on the representation can be eliminated and the resulting matrix represents the oblique cone of which the observed ellipse results from intersecting with the image plane. Second, the eigenvalues of this matrix can be determined by singular value decomposition. From this, the direction and distance to the center of the circular marking can be calculated and hence the altitude and orientation deduced. The direction from which the circle is approached is estimated from the identification marking “H” or “V” by creating a birdseye view of these markings and unrotating it.

Third, the implemented detection and pose estimation concepts are tested with recordings of landings on real sized vertipads for which accurate ground-truth position information is available. The concept is analyzed in respect to availability and performance of detection algorithms and positional accuracy of the pose estimation.

To our knowledge, the utilization of optical information for the navigation of large scale, potentially manned eVTOLs has only received limited research attention but can contribute substantially to reliable, resilient and accurate automatic landing solutions. Especially under the impression of increased occurrences of jamming and spoofing of GNSS there is the increased necessity for augmenting navigation systems with independent and board autonomous position estimation. The first innovative step is the specific adaptation and development of the detection algorithm to existing defined landing site markings. Fine-tuned filters are specifically designed to extract only the desired circles/ellipses. In addition, the use of circular markers instead of feature points is a step that significantly extends the range of application. Then using real sized vertipads and flying realistic scenarios in terms of distance and trajectories to generate authentic test data one can evaluate the detection and position estimation with high validity. Evaluation in the positional domain takes direct account of the requirements of an automatic landing system.

(Anticipated) Results

With multiple prerecorded datasets the developed concept is evaluated. In a first step, approaches recorded by a drone augmented by a vertipad with aligned lighting and scene color are used, to test the detection algorithms. This has the benefits to exactly know the position of the ellipses in the image and due to the precise trajectory the landing spot is continuously visible. This dataset is enhanced with images from the approach to real painted vertipads in multiple locations. In a second step, the performance of the pose estimation concept is tested using a simple projective simulation model to gather a large range of 3D positions. The results are verified by the extracts from the real recordings.

First results show a reliable detection of the circular markings below a certain distance to the vertipad until the markings get larger than the field of view. This maximum distance appears to be larger than what can be achieved by the detection of feature points. Our implementation of the algorithm by Lu et al. produced no plausible results and was excluded from the further analysis. The other ellipse detectors seem to have a big success rate, but at the cost of considerable false detections. This is further reinforced by using the modification of the algorithm by Keipour et al. While the evaluation of the specifically crafted detector for ICAO-markings is still remaining, the detector is expected to minimize false detections due to the implemented filter steps to identify a vertipad and only slightly reducing the availability.

In simulation the position error of the slant distance to the vertipad is below one meter as far as 60 meters above the vertipad for both estimation methods and linearly decreases with lower heights. However, below 30 meters the error of the “simple” method rises non-linearly up to 1.5 m until the ellipse get out of the view while the estimation based on deconstructing the oblique elliptical cone remains functional. These results indicate the suitability of estimating distance and height above the vertipad for automatic landing. Regarding the relative orientation we anticipate the same appropriateness and a full analysis will be included in the full paper.

Conclusions

In general, the localization utilizing the optical detection of circular landing site markings appears beneficial to augment a navigation system for the automatic landing of a VTOL. Circular marking detection is a significant contribution to exploit all defined markings on vertipads/helipads. Our proposed algorithms for detecting circular markings appear to detect the markings from longer distances and more reliably than point features, and can compete with the algorithms found in the literature. The developed filters effectively prevent many false positives. The simple height and distance algorithm produces acceptable results for very distant observations, but generally underestimates the distance to the ground. The more sophisticated approach, using the geometry of an oblique elliptical cone, calculates height and distance to the vertipad to sub-meter accuracy, with performance increasing as the vertipad is approached. Full orientation estimation seems possible. This includes estimating heading, but so far with less accuracy. Fusion or monitoring of GNSS/INS system appears feasible, although fusion needs more investigations due to the non-uniform distribution of error distributions and will be the subject of future research.

References

[1] F. Hübner, S. Wolkow, A. Dekiert, M. Angermann und U. Bestmann, „Assessment of Optical Markers for On-Board Autonomous Localization of eVTOLs during Landing,“ in ICRAT 2020, 2020.

[2] M. Nitsche, T. Krajník, P. ?ížek, M. Mejail und T. Duckett, „WhyCon: An Efficent, Marker-based Localization System,“ in IROS Workshop on Open Source Aerial Robotics, 2015.

[3] C. Lu, S. Xia, W. Huang, M. Shao und Y. Fu, „Circle detection by arc-support line segments,“ in 2017 IEEE International Conference on Image Processing, 2017.

[4] Q. Jia, X. Fan, Z. Luo, L. Song und T. Qiu, „A Fast Ellipse Detector Using Projective Invariant Pruning,“ IEEE Transactions on Image Processing, Nr. 26, 2017.

[5] A. Keipour, G. Pereira und S. Scherer, „Real-Time Ellipse Detection for Robotics Applications,“ IEEE Robotics and Automation Letters, Nr. 6, 2021.

Previous Abstract Return to Session D6 Next Abstract

For Attendees * *Technical Program * *Registration * *CGSIC * *Hotel * *Travel and Visas * *Smartphone Decimeter Challenge * *Exhibits * *Submit Kepler Nomination For Authors and Chairs * *Abstract Management * *Author Resource Center * *Session Chair Resources * *Panel Moderator Resources * *Student Paper Awards * *Editorial Review Policies * *Publication Ethics Policies For Exhibitors * *Exhibitor Resource Center * *Marketing Resources Other Years * *Future Meetings * *Past Meetings