Filter array-based spectral imaging - design choices and practical realization

International audience; This talk discusses the practical difficulties encountered in the practical implementation of MSFAs imaging systems in 2014. We provide hints and solutions on several problems. We provide means of evaluation and criticise them. Although, we present recent results obtain at the University of Bourgogne.

Demultiplexing Visible and Near-Infrared Information in Single-Sensor Multispectral Imaging

In this paper, we study a single-sensor imaging system that uses a multispectral filter array to spectrally sample the scene. Our system captures information in both visible and near-infrared bands of the electromagnetic spectrum. Due to manufacturing limitations, the visible filters in this system also transmit the NIR radiation. Similarly, visible light is transmitted by the NIR filter, leading to inaccurate mixed spectral measurements. We present an algorithm that resolves this issue by separating NIR and visible information. Our method achieves this goal by exploiting the correlation of multispectral images in both spatial and spectral domains. Simulation results show that the mean squa…

THE INFLUENCE OF CHROMATIC ABERRATION ON DEMOSAICKING

International audience; The wide deployment of colour imaging devices owes much to the use of colour filter array (CFA). A CFA produces a mosaic image, and normally a subsequent CFA demosaick-ing algorithm interpolates the mosaic image and estimates the full-resolution colour image. Among various types of optical aberrations from which a mosaic image may suffer, chromatic aberration (CA) influences the spatial and spectral correlation through the artefacts such as blur and mis-registration, which demosaicking also relies on. In this paper we propose a simulation framework aimed at an investigation of the influence of CA on demosaicking. Results show that CA benefits de-mosaicking to some ex…

Image registration for quality assessment of projection displays

International audience; In the full reference metric based image quality assessment of projection displays, it is critical to achieve accurate and fully automatic image registration between the captured projection and its reference image in order to establish a subpixel level mapping. The preservation of geometrical order as well as the intensity and chromaticity relationships between two consecutive pixels must be maximized. The existing camera based image registration methods do not meet this requirement well. In this paper, we propose a markerless and view independent method to use an un-calibrated camera to perform the task. The proposed method including three main components: feature e…

Spectral Characterization of a Prototype SFA Camera for Joint Visible and NIR Acquisition

International audience; Multispectral acquisition improves machine vision since it permits capturing more information on object surface properties than color imaging. The concept of spectral filter arrays has been developed recently and allows multispectral single shot acquisition with a compact camera design. Due to filter manufacturing difficulties, there was, up to recently, no system available for a large span of spectrum, i.e., visible and Near Infra-Red acquisition. This article presents the achievement of a prototype of camera that captures seven visible and one near infra-red bands on the same sensor chip. A calibration is proposed to characterize the sensor, and images are captured…

Evaluation of the Colorimetric Performance of Single-Sensor Image Acquisition Systems Employing Colour and Multispectral Filter Array

International audience; Single-sensor colour imaging systems mostly employ a colour filter array (CFA). This enables the acquisition of a colour image by a single sensor at one exposure at the cost of reduced spatial resolution. The idea of CFA fit itself well with multispectral purposes by incorporating more than three types of filters into the array which results in multispectral filter array (MSFA). In comparison with a CFA, an MSFA trades spatial resolution for spectral resolution. A simulation was performed to evaluate the colorimetric performance of such CFA/MSFA imaging systems and investigate the trade-off between spatial resolution and spectral resolution by comparing CFA and MSFA …

Cross-Media Color Reproduction and Display Characterization

International audience; In this chapter, we present the problem of cross-media color reproduction, that is, how to achieve consistent reproduction of images in different media with different technologies. Of particular relevance for the color image processing community is displays, whose color properties have not been extensively covered in previous literature. Therefore, we go more in depth concerning how to model displays in order to achieve colorimetric consistency. The structure of this chapter is as follows: After a short introduction, we introduce the field of cross-media color reproduction, including a brief description of current standards for color management, the concept of colori…

Sensors based on MultiSpectral Filter Arrays

International audience; Spectral Filter arrays (SFA) is an emerging technology for multispectral image acquisition. SFAs make possible the use of a compact multispectral sensor to acquire still images or video. One part of this talk will present and compare typical multispectral and color image acquisition systems in order to clearly identify differences underlying SFA technologies. The other part of this talk will show the difficulties and advances that we have made at the University of Bourgogne in the building of such sensors from the design of filters to data processing, such as demosaicing.

Multispectral filter arrays: Recent advances and practical implementation

Thanks to some technical progress in interferencefilter design based on different technologies, we can finally successfully implement the concept of multispectral filter array-based sensors. This article provides the relevant state-of-the-art for multispectral imaging systems and presents the characteristics of the elements of our multispectral sensor as a case study. The spectral characteristics are based on two different spatial arrangements that distribute eight different bandpass filters in the visible and near-infrared area of the spectrum. We demonstrate that the system is viable and evaluate its performance through sensor spectral simulation. Multispectral filter arrays: Recent advan…

Quality Assessment of Reconstruction and Relighting from RTI Images: Application to Manufactured Surfaces

In this paper, we propose to evaluate the quality of the reconstruction and relighting from images acquired by a Reflectance Transformation Imaging (RTI) device. Three relighting models, namely the PTM, HSH and DMD, are evaluated using PSNR and SSIM. A visual assessment of how the reconstructed surfaces are perceived is also carried out through a sensory experiment. This study allows to estimate the relevance of these models to reproduce the appearance of the manufactured surfaces. It also shows that DMD reproduces the most accurate reconstruction/relighting to an acquired measurement and that a higher sampling density don't mean necessarily a higher perceptual quality.

Energy balance in single exposure multispectral sensors

International audience; Recent simulations of multispectral sensors are based on a simple Gaussian model, which includes filters transmittance and substrate absorption. In this paper we want to make the distinction between these two layers. We discuss the balance of energy by channel in multispectral solid state sensors and propose an updated simple Gaussian model to simulate multispectral sensors. Results are based on simulation of typical sensor configurations.

Background subtraction with multispectral video sequences

International audience; Motion analysis of moving targets is an important issue in several applications such as video surveillance or robotics. Background subtraction is one of the simplest and widely used techniques for moving target detection in video sequences. In this paper, we investigate the advantages of using a multispectral video acquisition system of more than three bands for background subtraction over the use of trichromatic or monochromatic video sequences. To this end, we have established a dataset of multispectral videos with a manual annotation of moving objects. To the best of our knowledge, this is the first publicly available dataset of multispectral video sequences. Expe…

On the uniform sampling of CIELAB color space and the number of discernible colors

This paper presents a useful algorithmic strategy to sample uniformly the CIELAB color space based on close packed hexagonal grid. This sampling scheme has been used successfully in different research works from computational color science to color image processing. The main objective of this paper is to demonstrate the relevance and the accuracy of the hexagonal grid sampling method applied to the CIELAB color space. The second objective of this paper is to show that the number of color samples computed depends on the application and on the color gamut boundary considered. As demonstration, we use this sampling to support a discussion on the number of discernible colors related to a JND.

The PLVC display color characterization model revisited

This work proposes a study of the Piecewise Linear assuming Variation in Chromaticity (PLVC) dis- play color characterization model. This model has not been widely used as the improved accuracy compared with the more common PLCC (Piecewise Linear assuming Chromaticity Constancy) model is not significant for CRT (Cathode Ray Tube) display technology, and it requires more computing power than this model. With today's computers, computational complexity is less of a problem, and today's display technologies show a different colori- metric behavior than CRTs. The main contribution of this work is to generalize the PLVC model to multiprimary displays and to provide extensive experimental results…

HDR Imaging Pipeline for Spectral Filter Array Cameras

Multispectral single shot imaging systems can benefit computer vision applications in needs of a compact and affordable imaging system. Spectral filter arrays technology meets the requirement, but can lead to artifacts due to inhomogeneous intensity levels between spectral channels due to filter manufacturing constraints, illumination and object properties. One solution to solve this problem is to use high dynamic range imaging techniques on these sensors. We define a spectral imaging pipeline that incorporates high dynamic range, demosaicing and color image visualization. Qualitative evaluation is based on real images captured with a prototype of spectral filter array sensor in the visible…

An Adaptive Combination of Dark and Bright Channel Priors for Single Image Dehazing

Dehazing methods based on prior assumptions derived from statistical image properties fail when these properties do not hold. This is most likely to happen when the scene contains large bright areas, such as snow and sky, due to the ambiguity between the airlight and the depth information. This is the case for the popular dehazing method Dark Channel Prior. In order to improve its performance, the authors propose to combine it with the recent multiscale STRESS, which serves to estimate Bright Channel Prior. Visual and quantitative evaluations show that this method outperforms Dark Channel Prior and competes with the most robust dehazing methods, since it separates bright and dark areas and …

A spectral hazy image database

We introduce a new database to promote visibility enhancement techniques intended for spectral image dehazing. SHIA (Spectral Hazy Image database for Assessment) is composed of two real indoor scenes M1 and M2 of 10 levels of fog each and their corresponding fog-free (ground-truth) images, taken in the visible and the near infrared ranges every 10 nm starting from 450 to 1000 nm. The number of images that form SHIA is 1540 with a size of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1312\,\times \,1082$$\end{d…

Multispectral imaging: narrow or wide band filters?

This is an Open Access article. This is the publisher’s PDF originally published in Journal of the International Colour Association: http://aic-colour-journal.org/index.php/JAIC/article/view/149 In every aspect, spectral characteristics of filters play an important role in an image acquisition system. For a colorimetric system, traditionally, it is believed that narrow-band filters give rise to higher accuracy of colour reproduction, whereas wide-band filters, such as complementary colour filters, have the advantage of higher sensitivity. In the context of multispectral image capture, the objective is very often to retrieve an estimation of the spectral reflectance of the captured objects. …

Colorimetric Characterization of a Positive Film Scanner Using an Extremely Reduced Training Data Set

International audience; In this work, we address the problem of having an accurate colorimetric characterization of a scanner for traditional posi- tive film in order to guarantee the accuracy of the color informa- tion during the digitization of a movie. The scanning of a posi- tive film is not an usual task, however it can happen for cultural heritage purpose. Art-movies, are often created and stored as positive-film in museums. One of the problems one can face for a colorimetric characterization is to have a reasonable number of measurements from an item. In this work we succeeded in having a reasonable accuracy with just a few number of measurement (typically 4 to 7 ∆Ea∗b units with 2 t…

High-end colorimetric display characterization using an adaptive training set

A new, accurate, and technology-independent display color-characterization model is introduced. It is based on polyharmonic spline interpolation and on an optimized adaptive training data set. The establishment of this model is fully automatic and requires only a few minutes, making it efficient in a practical situation. The experimental results are very good for both the forward and inverse models. Typically, the proposed model yields an average model prediction error of about 1 ∆Eab* unit or below for several displays. The maximum error is shown to be low as well. freedom given to the model considering the choice of a tar- get color space and of the kernel and smoothing factor for the int…

Calibration et Rectification

Discrete wavelet transform based multispectral filter array demosaicking

International audience; The idea of colour filter array may be adapted to multi-spectral image acquisition by integrating more filter types into the array, and developing associated demosaicking algorithms. Several methods employing discrete wavelet transform (DWT) have been proposed for CFA demosaicking. In this work, we put forward an extended use of DWT for mul-tispectral filter array demosaicking. The extension seemed straightforward, however we observed striking results. This work contributes to better understanding of the issue by demonstrating that spectral correlation and spatial resolution of the images exerts a crucial influence on the performance of DWT based demosaicking.

Multispectral imaging for computer vision

A Gamut Preserving Color Image Quantization

International audience; We propose a new approach for color image quantization which preserves the shape of the color gamut of the studied image. Quantization consists to find a set of color representative of the color distribution of the image. We are looking here for an optimal LUT (look up table) which contains information on the image's gamut and on the color distribution of this image. The main motivation of this work is to control the reproduction of color images on different output devices in order to have the same color feeling, coupling intrinsic informations on the image gamut and output device calibration. We have developped a color quantization algorithm based on an image depend…

A Color Image Database for Haze Model and Dehazing Methods Evaluation

International audience; One of the major issues related to dehazing methods (single or multiple image based) evaluation is the absence of the haze-free image (ground-truth). This is also a problem when it concerns the validation of Koschmieder model or its subsequent dehazing methods. To overcome this problem, we created a database called CHIC (Color Hazy Image for Comparison), consisting of two scenes in controlled environment. In addition to the haze-free image, we provide 9 images of different fog densities. Moreover, for each scene, we provide a number of parameters such as local scene depth, distance from the camera of known objects such as Macbeth Color Checkers, their radiance, and t…

High Dynamic Range Spectral Imaging Pipeline For Multispectral Filter Array Cameras

Spectral filter arrays imaging exhibits a strong similarity with color filter arrays. This permits us to embed this technology in practical vision systems with little adaptation of the existing solutions. In this communication, we define an imaging pipeline that permits high dynamic range (HDR)-spectral imaging, which is extended from color filter arrays. We propose an implementation of this pipeline on a prototype sensor and evaluate the quality of our implementation results on real data with objective metrics and visual examples. We demonstrate that we reduce noise, and, in particular we solve the problem of noise generated by the lack of energy balance. Data are provided to the community…

A Study on the Impact of Spectral Characteristics of Filters on Multispectral Image Acquisition

International audience; In every aspect, filter design plays an important role in an image acquisition system based on a single image sensor and a colour filter array (CFA) mounted onto the sensor. Complementary CFAs are used by some colour cameras in the interest of higher sensitivity, which motivated us to employ filters of wide pass bands in the effort to adapt CFA for multispectral image acquisition. In this context, filter design has an effect on the accuracy of spectrum reconstruction in addition to other aspects. The results show that wider bandwidths in general result in more faithful spectrum reconstruction and higher signal-to-noise performance.

Deep learning for dehazing: Benchmark and analysis

International audience; We compare a recent dehazing method based on deep learning , Dehazenet, with traditional state-of-the-art approach, on benchmark data with reference. Dehazenet estimates the depth map from a single color image, which is used to inverse the Koschmieder model of imaging in the presence of haze. In this sense, the solution is still attached to the Koschmieder model. We demonstrate that this method exhibits the same limitation than other inversions of this imaging model.

Seam-Based Edge Blending for Multi-Projection Systems

Perceptual seamlessness of large-scale tiled displays is still a challenge. One way to avoid Bezel effects from contiguous displays is to blend superimposed parts of the image over the edges. This work proposes a new approach for edge blending. It is based on intensity edge blending adapted on the seam description of the image content. The main advantage of this method is to reduce visual artifacts thanks to context adaptation and smooth transitions. We evaluate the quality of the method with a perceptual experiment where it is compared with state-of-the-art methods. The new method shows most improvement in low frequency areas compared to the other techniques. This method can be inserted in…

A geometrical approach for inverting display color-characterization models

— Some display color-characterization models are not easily inverted. This work proposes ways to build geometrical inverse models given any forward color-characterization model. The main contribution is to propose and analyze several methods to optimize the 3-D geometrical structure of an inverse color-characterization model directly based on the forward model. Both the amount of data and their distribution in color space is especially focused on. Several optimization criteria, related either to an evaluation data set or to the geometrical structure itself, are considered. A practical case with several display devices, combining the different methods proposed in the article, are considered …

A Database of Spectral Filter Array Images that Combine Visible and NIR

International audience; Spectral filter array emerges as a multispectral imaging technology, which could benefit several applications. Although several instantiations are prototyped and commercialized, there are yet only a few raw data available that could serve research and help to evaluate and design adequate related image processing and algorithms. This document presents a freely available spectral filter array database of images that combine visible and near infra-red information.