734 / 2024-09-19 10:48:33

Synthesize Large-scale in situ Images for Training Plankton Detection Algorithms

Image Synthesis,Generative Models,Plankton Detection

The content distribution of full-frame raw images captured by in situ plankton imaging instruments is highly complex and variable, easily affected by multiple factors such as the properties of seawater, the quantity, taxonomy, posture, and morphology of underwater objects, as well as changes in camera focusing status and working settings. Therefore, most plankton observation methods based on in situ imaging have divided the recognition task into two stages: the first is to locate the objects from the raw full-frame images and crop out the region of interest (ROI) vignettes containing only one target; the second is to classify the ROI images to achieve the ultimate automatic plankton recognition. However, the raw image complexity can significantly impact the performance of the two-stage plankton target cropping and recognition algorithms, and may even cause serious errors for the observation of interested plankton. Although existing in situ imaging instruments have already collected massive full-frame image data under different sea conditions, their content variety poses great challenges for manual annotation, making it almost impossible to obtain large-scale high-quality datasets to train usable end-to-end deep CNN models for plankton detection. We note that it is possible to utilize deep generative models to perform large-scale raw image synthesis using a limited amount of full-frame images and annotated ROI data. Based on the images collected by our dark-field underwater imager IPP (Imaging Plankton Probe) across different sea regions in China and Australia, we propose a method that can quickly build a massive full-frame image dataset IsPlanktonGC with definite object identity ground truth. The preliminary experimental results show that the imitated images have very realistic visual similarity to the true in-situ collected ones. This strategy is expected to be very helpful in providing big data foundations for developing robust downstream object detection algorithms to better observe marine plankton and monitor HAB events.