Each year, underwater remotely operated vehicles (ROVs) collect thousands of hours of video of unexplored ocean habitats revealing a plethora of information regarding biodiversity on Earth. However, fully utilizing this information remains a challenge as proper annotations and analysis require trained scientists’ time, which is both limited and costly. To this end, we present a Dataset for Underwater Substrate and Invertebrate Analysis (DUSIA), a benchmark suite and growing large-scale dataset to train, validate, and test methods for temporally localizing four underwater substrates as well as temporally and spatially localizing 59 underwater invertebrate species. DUSIA currently includes over ten hours of footage across 25 videos captured in 1080p at 30 fps by an ROV following pre-planned transects across the ocean floor near the Channel Islands of California. Each video includes annotations indicating the start and end times of substrates across the video in addition to counts of species of interest. Some frames are annotated with precise bounding box locations for invertebrate species of interest, as seen in Figure 1. To our knowledge, DUSIA is the first dataset of its kind for deep sea exploration, with video from a moving camera, that includes substrate annotations and invertebrate species that are present at significant depths where sunlight does not penetrate. Additionally, we present the novel context-driven object detector (CDD) where we use explicit substrate classification to influence an object detection network to simultaneously predict a substrate and species class influenced by that substrate. We also present a method for improving training on partially annotated bounding box frames. Finally, we offer a baseline method for automating the counting of invertebrate species of interest.
In order to download the videos of our Dataset for Underwater Invertebrate Species Analysis (DUSIA), please visit BisQue instance at https://bisque2.ece.ucsb.edu/, and follow the steps below.
From the drop down menu select "Download Manager".
Near the bottom left, uncheck "Include annotations".
Near the bottom left, select the arrow to the left of "Download". Select "as TARball". Other archive formats may not be supported for DUSIA at this time.
Choose the download destination.
BisQue has been used to manage and analyze 23.3 hours (884GB) of high definition video from dives in Bering Sea submarine canyons to evaluate the density of fishes, structure-forming corals and sponges and to document and describe fishing damage. Non-overlapping frames were extracted from each video transect at a constant frequency of 1 frame per 30s. An image processing algorithm developed in Matlab was used to detect laser dots projected onto the seafloor as a scale reference. BisQue's module system allows to wrap this Matlab code into an analysis module that can be parallelized across a compute cluster. In addition, each frame was manually annotated with objects of interest (e.g., fishes, sponges, substrates) and these annotations and other image metadata (e.g., pixel resolution, GPS location) was stored in BisQue's flexible metadata store. The annotations were then used to compute the average density of species and co-habitation behavior in different regions of the canyons, resulting in new insights into this ecosystem.