Extraction of salt-marsh vegetation "fairy circles" from UAV images by the combination of SAM visual segmentation model and random forest machine learning algorithm

Zhou Ruotong; Tan Kai; Yang Jianru; Han Jiangtao; Zhang Weiguo

doi:10.12284/hyxb2024048

Article Contents

Article Navigation > Haiyang Xuebao > 2024 > Accepted Manuscript

Zhou Ruotong，Tan Kai，Yang Jianru, et al. Extraction of salt-marsh vegetation 'fairy circles' from UAV images by the combination of SAM visual segmentation model and random forest machine learning algorithm[J]. Haiyang Xuebao，2024, 46(x)：1–11 doi: 10.12284/hyxb2024048

Citation:

Zhou Ruotong，Tan Kai，Yang Jianru, et al. Extraction of salt-marsh vegetation "fairy circles" from UAV images by the combination of SAM visual segmentation model and random forest machine learning algorithm[J]. Haiyang Xuebao，2024, 46(x)：1–11 doi: 10.12284/hyxb2024048

Citation:

PDF( 4961 KB)

Extraction of salt-marsh vegetation "fairy circles" from UAV images by the combination of SAM visual segmentation model and random forest machine learning algorithm

doi: 10.12284/hyxb2024048

State Key Laboratory of Estuarine and Coastal Research, East China Normal University, Shanghai 200241, China

Received Date: 2023-09-29
Accepted Date: 2024-03-01
Rev Recd Date: 2023-12-28

Available Online: 2024-03-11

Abstract

Abstract

The “fairy circle” represents a unique form of spatial self-organization found within coastal salt marsh ecosystems, profoundly influencing the productivity, stability, and resilience of these wetlands. Unmanned Aerial Vehicle (UAV) imagery plays a pivotal role in precisely pinpointing the “fairy circle” locations and deciphering their temporal and spatial development trends. However, identifying “fairy circle” pixels within two-dimensional images poses a considerable technical challenge due to the subtle differences in color and shape characteristics between these pixels and their surroundings. Therefore, intelligently and accurately identify “fairy circle” pixels from two-dimensional images and form individual “fairy circle” for the identified pixels are the current technical difficulties. This paper introduces an innovative approach to extract “fairy circle” from UAV images by integrating the SAM (Segment Anything Model) visual segmentation model with random forest machine learning. This novel method accomplishes the recognition and extraction of individual “fairy circle” through a two-step process: segmentation followed by classification. Initially, we establish Dice (Sørensen-Dice coefficient) and IOU (Intersection Over Union) evaluation metrics, and optimize SAM’s pre-trained model parameters, which produces segmentation mask devoid of attribute information by fully automated image segmentation. Subsequently, we align the segmentation mask with the original image, and utilizes RGB (red, green, and blue) color channels and spatial coordinates to construct a feature index for the segmentation mask. These features undergo analysis and selection based on Out-of-Bag (OOB) error reduction and feature distribution patterns. Ultimately, the refined features are employed to train a random forest model, enabling the automatic identification and classification of “fairy circle” vegetation, common vegetation, and bare flat areas. The experimental results show that the average correct extraction rate of "fairy circle" is 96.1%, and the average wrong extraction rate is 9.5%, which provides methodological and technological support for the accurate depiction of the spatial and temporal pattern of "fairy circle" as well as the processing of coastal remote sensing images by UAVs.
- salt marsh vegetation,
- fairy circle,
- segment anything model (SAM),
- unmanned Aerial Vehicle images,
- machine learning

FullText(HTML)

References(28)

References

[1]	韩倩倩, 牛振国, 吴孟泉, 等. 基于潮位校正的中国潮间带遥感监测及变化[J]. 科学通报, 2019, 64(4): 456−473. doi: 10.1360/N972018-00723 Han Qianqian, Niu Zhenguo, Wu Mengquan, et al. Remote-sensing monitoring and analysis of China intertidal zone changes based on tidal correction[J]. Chinese Science Bulletin, 2019, 64(4): 456−473. doi: 10.1360/N972018-00723
[2]	陈一宁, 陈鹭真. 滨海蓝碳生态系统的碳库间相互作用研究进展及展望[J]. 海洋学研究, 2023, 41(1): 3−13. Chen Yining, Chen Luzhen. Interactions between vegetation and sediment carbon pools within coastal blue carbon ecosystems: a review and perspective[J]. Journal of Marine Sciences, 2023, 41(1): 3−13.
[3]	Zhao Lixia, Zhang Kang, Siteur K, et al. Fairy circles reveal the resilience of self-organized salt marshes[J]. Science Advances, 2021, 7(6): eabe1100. doi: 10.1126/sciadv.abe1100
[4]	Rietkerk M, Bastiaansen R, Banerjee S, et al. Evasion of tipping in complex systems through spatial pattern formation[J]. Science, 2021, 374(6564): eabj0359. doi: 10.1126/science.abj0359
[5]	Ruiz-Reynés D, Gomila D, Sintes T, et al. Fairy circle landscapes under the sea[J]. Science Advances, 2017, 3(8): e1603262. doi: 10.1126/sciadv.1603262
[6]	de Paoli H, van der Heide T, van den Berg A, et al. Behavioral self-organization underlies the resilience of a coastal ecosystem[J]. Proceedings of the National Academy of Sciences of the United States of America, 2017, 114(30): 8035−8040.
[7]	Tao Pengjie, Tan Kai, Ke Tao, et al. Recognition of ecological vegetation fairy circles in intertidal salt marshes from UAV LiDAR point clouds[J]. International Journal of Applied Earth Observation and Geoinformation, 2022, 114: 103029. doi: 10.1016/j.jag.2022.103029
[8]	Liu Qingsheng, Huang Chong, Liu Gaohuan, et al. Comparison of CBERS-04, GF-1, and GF-2 satellite panchromatic images for mapping quasi-circular vegetation patches in the Yellow River Delta, China[J]. Sensors, 2018, 18(8): 2733. doi: 10.3390/s18082733
[9]	Shi Lei, Liu Qingsheng, Huang Chong, et al. Mapping quasi-circular vegetation patch dynamics in the Yellow River Delta, China, between 1994 and 2016[J]. Ecological Indicators, 2021, 126: 107656. doi: 10.1016/j.ecolind.2021.107656
[10]	Zhang Yunjie, Liu Qingsheng, Liu Gaohuan, et al. Mapping of circular or elliptical vegetation community patches: a comparative use of SPOT-5, ALOS and ZY-3 imagery[C]//Proceedings of the 8th International Congress on Image and Signal Processing. Shenyang: IEEE, 2015.
[11]	Zhang Xianlong, Zhang Fei, Qi Yaxiao, et al. New research methods for vegetation information extraction based on visible light remote sensing images from an unmanned aerial vehicle (UAV)[J]. International Journal of Applied Earth Observation and Geoinformation, 2019, 78: 215−226. doi: 10.1016/j.jag.2019.01.001
[12]	Kirillov A , Mintun E , Ravi N , et al. Segment Anything[C]//2023 IEEE/CVF International Conference on Computer Vision (ICCV). 0[2024-02-29]. DOI: 10.1109/ICCV51070.2023.00371.
[13]	Ning Guochen, Liang Hanyin, Jiang Zhongliang, et al. The potential of 'Segment Anything' (SAM) for universal intelligent ultrasound image guidance[J]. Bioscience Trends, 2023, 17(3): 230−233. doi: 10.5582/bst.2023.01119
[14]	Chen Fang, Chen Lingyu, Han Haojie, et al. The ability of Segmenting Anything Model (SAM) to segment ultrasound images[J]. Bioscience Trends, 2023, 17(3): 211−218. doi: 10.5582/bst.2023.01128
[15]	Shi Peilun, Qiu Jianing, Abaxi S M D, et al. Generalist vision foundation models for medical imaging: a case study of segment anything model on zero-shot medical segmentation[J]. Diagnostics, 2023, 13(11): 1947. doi: 10.3390/diagnostics13111947
[16]	Maxwell A E, Warner T A, Fang Fang. Implementation of machine-learning classification in remote sensing: an applied review[J]. International Journal of Remote Sensing, 2018, 39(9): 2784−2817. doi: 10.1080/01431161.2018.1433343
[17]	Pádua L, Adão T, Hruška J, et al. Vineyard classification using machine learning techniques applied to RGB-UAV imagery[C]//Proceedings of 2020 IEEE International Geoscience and Remote Sensing Symposium. Waikoloa: IEEE, 2020.
[18]	Juel A, Groom G B, Svenning J C, et al. Spatial application of Random Forest models for fine-scale coastal vegetation classification using object based analysis of aerial orthophoto and DEM data[J]. International Journal of Applied Earth Observation and Geoinformation, 2015, 42: 106−114. doi: 10.1016/j.jag.2015.05.008
[19]	周小成, 郑磊, 黄洪宇. 基于多特征优选的无人机可见光遥感林分类型分类[J]. 林业科学, 2021, 57(6): 24−36. Zhou Xiaocheng, Zheng Lei, Huang Hongyu. Classification of forest stand based on multi-feature optimization of UAV visible light remote sensing[J]. Scientia Silvae Sinicae, 2021, 57(6): 24−36.
[20]	Yang Shuting, Gu Lingjia, Li Xiaofeng, et al. Crop classification method based on optimal feature selection and hybrid CNN-RF networks for multi-temporal remote sensing imagery[J]. Remote Sensing, 2020, 12(19): 3119. doi: 10.3390/rs12193119
[21]	Fu Bolin, Liu Man, He Hongchang, et al. Comparison of optimized object-based RF-DT algorithm and SegNet algorithm for classifying Karst wetland vegetation communities using ultra-high spatial resolution UAV data[J]. International Journal of Applied Earth Observation and Geoinformation, 2021, 104: 102553. doi: 10.1016/j.jag.2021.102553
[22]	Han Kai, Wang Yunhe, Chen Hanting, et al. A survey on vision transformer[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(1): 87−110. doi: 10.1109/TPAMI.2022.3152247
[23]	Liaw A, Wiener M. Classification and regression by randomForest[J]. R News, 2002, 2(3): 18−22.
[24]	Breiman L. Random forests[J]. Machine Learning, 2001, 45(1): 5−32. doi: 10.1023/A:1010933404324
[25]	Rodriguez-Galiano V F, Ghimire B, Rogan J, et al. An assessment of the effectiveness of a random forest classifier for land-cover classification[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2012, 67: 93−104. doi: 10.1016/j.isprsjprs.2011.11.002
[26]	Nguyen M H, de la Torre F. Optimal feature selection for support vector machines[J]. Pattern Recognition, 2010, 43(3): 584−591. doi: 10.1016/j.patcog.2009.09.003
[27]	Morgan G R, Wang Cuizhen, Morris J T. RGB indices and canopy height modelling for mapping tidal marsh biomass from a small unmanned aerial system[J]. Remote Sensing, 2021, 13(17): 3406. doi: 10.3390/rs13173406
[28]	Congalton R G, Green K. Assessing the Accuracy of Remotely Sensed Data: Principles and Practices[M]. 3rd ed. Boca Raton: CRC Press, 2019.