Spectral-spatial features for material based object tracking in hyperspectral video

by   Fengchao Xiong, et al.

Traditional color images only depict color intensities in red, green and blue channels, often making object trackers fail when a target shares similar color or texture as its surrounding environment. Alternatively, material information of targets contained in a large amount of bands of hyperspectral images (HSI) is more robust to these challenging conditions. In this paper, we conduct a comprehensive study on how HSIs can be utilized to boost object tracking from three aspects: benchmark dataset, material feature representation and material based tracking. In terms of benchmark, we construct a dataset of fully-annotated videos which contain both hyperspectral and color sequences of the same scene. We extract two types of material features from these videos. We first introduce a novel 3D spectral-spatial histogram of gradient to describe the local spectral-spatial structure in an HSI. Then an HSI is decomposed into the detailed constituent materials and associate abundances, i.e., proportions of materials at each location, to encode the underlying information on material distribution. These two types of features are embedded into correlation filters, yielding material based tracking. Experimental results on the collected benchmark dataset show the potentials and advantages of material based object tracking.


page 1

page 3

page 4

page 7

page 9

page 10


Spectral-spatial features for material based object tracking in hyperspectral videos

Traditional color images only depict color intensities in red, green and...

Object Tracking in Hyperspectral Videos with Convolutional Features and Kernelized Correlation Filter

Target tracking in hyperspectral videos is a new research topic. In this...

HPFormer: Hyperspectral image prompt object tracking

Hyperspectral imagery contains abundant spectral information beyond the ...

MatSpectNet: Material Segmentation Network with Domain-Aware and Physically-Constrained Hyperspectral Reconstruction

Achieving accurate material segmentation for 3-channel RGB images is cha...

Aerial Vehicle Tracking by Adaptive Fusion of Hyperspectral Likelihood Maps

Hyperspectral cameras can provide unique spectral signatures for consist...

Neural Network Learning of Chemical Bond Representations in Spectral Indices and Features

In this paper we investigate neural networks for classification in hyper...

Materialistic: Selecting Similar Materials in Images

Separating an image into meaningful underlying components is a crucial f...

Please sign up or login with your details

Forgot password? Click here to reset