资源共享

	EnglishVersion

本站首页

小组动态

Deviation Relationship Learning for Remote Sensing Anomaly Detection

(Detecting Any Image)

Jingtao Li, Xinyu Wang, Hengwei Zhao, Shaoyu Wang, Yanfei Zhong

Abstract

Remote sensing anomaly detector can find the objects deviating from the background as potential targets for Earth monitoring. Given the diversity in earth anomaly types, designing a transferring model with cross-scene and cross-modality detection ability should be cost-effective and flexible to new earth observation sources and anomaly types. However, the current anomaly detectors aim to learn the certain background distribution, the trained model cannot be transferred to unseen images.
Inspired by the fact that the deviation relationship between anomaly and background is consistent and independent from the image distribution, our work exploits the learning target conversion from the varying background distribution to the consistent deviation relationship. Following this idea, we have published three related works, each with a different focus, to build the anomaly detection system based on deviation relationship learning as in Fig. 1.

Deviation Relationship Learning

Traditional models focus on learning the certain image distribution \(P(\mathbf{\mathrm{X}})\) first and then use some existing deviating metric \(S\) to rank the anomaly score. In different scenes and modalities, since the distributions of background and anomaly are varying, the prior learned model cannot be transferred to unseen image distribution. Inspired by the fact that the deviating metric \(S\) is independent and consistent for all the images, our work aims to bypass the image distribution learning and learn the deviating metric directly, achieving the cross-scene and cross-modality detection ability. Fig. 1 shows the principle difference intuitively.

Fig. 1. Principle of deviation relationship learning, which supports the related three works.

1.Hyperspectral Anomaly Detection based on Deviation Relationship Learning (Research 1)

In this work, we propose a one-step paradigm for hyperspectral anomaly detection, which can output the detection map directly and infer the unseen hyperspectral image directly as in Fig.2. The instantiated model is trained using simulated anomaly samples with spectral deviation relationship.

● Research paper: Link
● Code: Link

Fig. 2. (a) Proposed one-step detecting paradigm. (b) Qualitative comparison experiments, where the instantiated TDD model infers the testing images directly while tradition models are tained and tested on each image.

2.Cross-Modality Anomaly Detection based on Deviation Relationship Learning (Research 2)

In this work, we built a cross-modal and cross-scene anomaly detector based on research 1. We firstly proved that satisfying the large margin condition in the simulated samples is the key for the transferring ability of learned deviation metric \(S\) (Theorem 1 in the paper). Then, the detector is trained on both simulated spatial and spectral anomalies, optimized with large-margin deviation ranking at both feature level and pixel level. Fig.3 shows the training framework and Fig. 4 reports the qualitative comparison experiments on five modalities.

● Research paper: Link
● Code: Link

Fig. 3. Training framework of built cross-modal and cross-scene anomaly detector.

Fig. 4. Qualitative comparison experiments on five modalities, where the instantiated model infers all the modalities directly while tradition models are tained and tested on each modality.

3.Instance-level Anomaly Detection based on Deviation Relationship Learning (Research 3)

In this work, we built the first instance-level anomaly segmentation detector, which can reduce the false alarms and support the anomaly counting ability compared to traditional pixel-level detector. Since the anomaly objects contain a variety of categories and always unseen objects, the current state-of-the-art (SOTA) query-based models designed for certain categories perform unsatisfactorily when applied to the anomaly instances. To bridge this gap, we propose general adaptations guided by the pixel-level deviation map for any query-based model, which adapts the model from learning certain category representation to learning anomaly-aware representation in different categories. Fig. 5 shows the main framework and Fig. 6 reports the qualitative comparison experiments.

● Research paper: Link
● Code: Link

Fig. 5. Proposed adaptation for any query-based model. To make the pixel embeddings and query anomaly-aware, we first build a sperate branch to output the pixel-level anomaly map, which is further used to refine the embeddings and queries.

Fig. 6. Qualitative comparison experiments, where we apply the general adaptations on both SOTA Mask2Former and Mask DINO to visualize the promotion.

5.Copyright

The copyright belongs to Intelligent Data Extraction, Analysis and Applications of Remote Sensing(RSIDEA) academic research group, State Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing (LIESMARS), Wuhan University, China. The research works can only be used for academic purposes and need to cite the following paper, and any commercial use is prohibited. Otherwise, RSIDEA of Wuhan University reserves the right to pursue legal responsibility.

6.Contact

If you have any the problem or feedback in using our data and code, please contact:
Mr. Jingtao Li: jingtaoli@whu.edu.cn Home Page
Dr. Xinyu Wang: wangxinyu@whu.edu.cn
Prof. Yanfei Zhong: zhongyanfei@whu.edu.cn

Reference:
[1] Jingtao Li, Xinyu Wang, Hengwei Zhao, Yanfei Zhong. Learning a Cross-modality Anomaly Detector for Remote Sensing Imagery[J]. IEEE Transactions on Image Processing, 2024, 33: 6607-6621.
[2] Jingtao Li, Xinyu Wang, Shaoyu Wang, Hengwei Zhao, Yanfei Zhong. One Step Detection Paradigm for Hyperspectral Anomaly Detection via Spectral Deviation Relationship Learning[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 1-15.
[3] Jingtao Li, Yanfei Zhong, Hengwei Zhao, Zhi Gao, Xinyu Wang. Segmenting Remote Sensing Anomalies at Instance-level via Anomaly Map Guided Adaptation[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 1-16.