Real-Time Anomaly Detection in Underwater Gliders: Experimental Evaluation

Table of Links

Abstract and Intro

Anomaly Detection Algorithm

Experimental Evaluation

Conclusion and References

III. EXPERIMENTAL EVALUATION

We apply the anomaly detection algorithm to four glider deployments across the coastal ocean of Florida and Georgia, USA. For evaluation, the anomaly detected by the algorithm is cross-validated by high-resolution glider DBD data and pilot notes. In particular, we simulate the online detection process on SBD data and compare the result with that detected from DBD data. For reference, the designed parameters are listed in TABLE I.

*A. Experimental Setup
*

The deployment details of four glider deployments are shown in TABLE II along with the Google Earth trajectories in Fig. 2. It is worth mentioning that USF-Sam is piloted under the support of GENIoS Python [26] in real time, and USF-Sam is simulated by the online detection algorithm to report any potential anomaly.

B. Large-scale Experiments

The large-scale experiments apply hindcast anomaly detection to full resolution DBD files downloaded from the glider on the shore. For verification, the algorithm-detected anomaly is compared with that directly seen from post-mission DBD data with the highest possible resolution and pilot logs.

1) Franklin: From October 12 to October 13, 2022, Franklin experienced two aborts and delays of up to 40 minutes of subsequent surfacings. The glider pilot time believed that Franklin had attracted remoras or had encountered an obstruction on his port wing, resulting in a roll change shown in Fig. 3a. Its climb to the ocean surface is unexpectedly slow even though flying with climb ballast near the upper limits of extended buoyancy pump, as shown in Fig. 3b.

Applied to the DBD data downloaded from the glider, the detection algorithm guarantees convergence of the estimated trajectory to the true trajectory as shown in Fig. 4. There are four basis functions (four green circles) covering the glider trajectory in the whole flow fields, which is an essential condition for parameter estimation to converge. As shown in Fig. 5, the maximum CLLE is small enough as 2.5m, considering the glider moves hundreds of kilometers in the entire deployment, so we can also conclude the convergence of CLLE.

The CLLE convergence guarantees the convergence of both the glider speed estimate and the flow speed estimate. For precise comparison, the flow is divided into its West-East (W-E or zonal) component, denoted as u, and its North-South (N-S or meridional) component, denoted as v. The graphs in Fig. 6a demonstrate that the algorithm-estimated W-E flow is close to the corresponding glider estimate, indicating minimal error in the u flow estimation. A similar comparison can be observed for the N-S flow, as shown in Fig. 6b. This comparative analysis provides reliability of the anomaly detection when it is triggered. If the estimated glider speed drops out of the normal speed range, the anomaly should have occurred. As shown in Fig. 7, the estimated glider speed drops out of the normal speed range (green dot line) at around October 13, 2022, 15:00 UTC. The timestamp when the anomaly is detected by the algorithm corresponds to the

timestamp detected from the glider team’s report and the post-recovery DBD data. Therefore, the algorithm is verified by successfully detecting the anomaly.

2) USF-Sam: During the mission, glider pilots suggest that the remora attachment should occur between March 11 and March 12, 2023 UTC when USF-Sam has a couple of roll and pitch changes shown in Fig. 8a and Fig. 8b. This suggestion is reinforced by USF-Sam’s prolonged period of being stuck at a certain depth.

Based on the DBD data, the detection algorithm generates the estimated trajectory, which is close to the true trajectory as shown in Fig. 9. From quantitative analysis in Fig. 10, the maximum CLLE 45m is small enough to conclude the CLLE convergence. We follow the same process of evaluating flow estimation error in Section III-B.1. As shown in Fig. 11, the small flow estimation error suggests that the detection result can be trusted. As shown in Fig. 12, the estimated glider speed drops out of the normal speed range (green dot line) at around March 11, 2023, 20:00 UTC. The timestamp when

the anomaly is detected by the algorithm corresponds to the timestamp detected from the glider team’s report and the DBD dataset.

The simulated online experiment implements anomaly detection using subsetted real-time SBD files transmitted from the glider to the dockserver during the mission. For example, the SBD file may contain fewer than 30 variables at 18-1800 s intervals, and is often subsampled to every 3rd or 4th yo (or down-up cycle), compared to the approximately 3000 variables stored at approximately 1 s interval on every yo in the DBD file processed on shore. The algorithm fetches new SBD files from the dockserver, parses SBD data from the SBD files, and applies the detection algorithm to the SBD data in an online mode. The online detection holds unique significance from the perspective that the detection results could help pilots monitor glider conditions in real time, thus

circumventing any further loss and DBD data anomaly is only available post recovery.

Instead of waiting for the DBD data after the entire mission, the online detection is capable of utilizing the SBD data in real time. As shown in Fig. 13, the detection algorithm can also achieve trajectory convergence similar to using the DBD data. The maximum CLLE 5m in Fig. 14 is sufficiently small given the glider moving range in the ocean. Therefore, we can conclude the CLLE convergence. We follow the same process of evaluating flow estimation error in Section IIIB.1. As shown in Fig. 15, the small flow estimation error suggests that the online detection result is trustworthy. As shown in Fig. 16, the estimated glider speed drops out of the normal speed range (green dot line) at around March 12, 2023, 03:00 UTC. This result matches reasonably well the above result from the DBD data, which justifies that we can trust the online anomaly detection applied to real-time SBD data.

3) USF-Gansett: At November 12, 2021, 22:32 UTC, the glider USF-Gansett sharply rolled to starboard 47° and pitches to 54°, settling back by 22:36 UTC to a roll of 11° − 15° and normal pitches as shown in Fig. 17a and

Fig. 17b. Heading changes during this time also varied by over 100° as shown in Fig. 17c. This abnormal roll persisted even though pitch and heading returns to normal afterwards.

Upon recovery, gouges resembling teeth marks are evident on the aft hull and science bay as shown in Fig. 18a. The arc of the marks span approximately 9 inches. The chord between the top and bottom ends of the aft hull markings is approximately 7.5 inches. The netting on the hull was cut in numerous areas, suggesting a serious shark strike. It is highly hypothesized that the bent starboard wing in Fig. 18b is caused by the shark strike.

Based on the DBD data, the estimated trajectory is close to the true trajectory as shown in Fig. 19. From quantitative analysis in Fig. 20, the maximum CLLE 1.1m is small enough to conclude the CLLE convergence. We follow the same process of evaluating flow estimation error in Section III-B.1. As shown in Fig. 21, the small flow estimation error suggests that the detection result can be trusted. As shown in Fig. 22, the estimated glider speed dropped out of the normal speed range (green dot line) at around November 12, 2021, 22:00 UTC, followed by radical speed changes that match the persistent roll change in the glider team’s report. The timestamp when the anomaly is detected by the

4) USF-Stella: After performing hindcast analysis of the ground truth data as shown in Fig. 23, the glider team was certain that USF-Stella takes several hits during the deployment. At some point, the strike was serious enough that one of the wing support rails are broken, as shown in Fig. 24.

Based on the DBD data, the algorithm-estimated trajectory is close to the true trajectory, as shown in Fig. 25. From quantitative analysis in Fig. 26, the maximum CLLE 45m is small enough to conclude the CLLE convergence. We follow the same process of evaluating flow estimation error in Section III-B.1. As shown in Fig. 27, the small flow estimation error suggests that the detection result can be trusted. As shown in Fig. 28, the estimated glider speed dropped out of the normal speed range (green dot line) at

around April 02, 2023, 15:00 UTC. The anomaly timestamp detected by the algorithm corresponds to the timestamp in the glider team’s report.