An FPGA-based design for real-time Super Resolution Reconstruction

Since several decades, the camera spatial resolution is gradually increasing with the CMOS technology evolution. The image sensors provide more and more pixels, generating new constraints for the suitable optics. As an alternative, promising solutions propose Super Resolution (SR) image reconstruction to extend the image size without modifying the sensor architecture. Convincing state-of art studies demonstrate that these methods could even be implemented in real-time. Nevertheless, artifacts can be observed in highly textured areas of the image. In this paper, we propose a Local Adaptive Spatial Super Resolution (LASSR) method to fix this limitation. A real-time texture analysis is include…

Efficient smart-camera accelerator: A configurable motion estimator dedicated to video codec

Smart cameras are used in a large range of applications. Usually the smart cameras transmit the video or/and extracted information from the video scene, frequently on compressed format to fit with the application requirements. An efficient hardware accelerator that can be adapted and provide the required coding performances according to the events detected in the video, the available network bandwidth or user requirements, is therefore a key element for smart camera solutions. We propose in this paper to focus on a key part of the compression system: motion estimation. We have developed a flexible hardware implementation of the motion estimator based on FPGA component, fully compatible with…

Efficient smart-camera accelerator: an configurable motion estimator dedicated to video codec

International audience; Smart cameras are used in a large range of applications. Usually the smart cameras transmit the video or/and extracted information from the video scene, frequently on compressed format to fit with the application requirements. An efficient hardware accelerator that can be adapted and provide the required coding performances according to the events detected in the video, the available network bandwidth or user requirements, is therefore a key element for smart camera solutions. We propose in this paper to focus on a key part of the compression system: motion estimation. We have developed a flexible hardware implementation of the motion estimator based on FPGA componen…

Smart camera design for intensive embedded computing

Computer-assisted vision plays an important role in our society, in various fields such as personal and goods safety, industrial production, telecommunications, robotics, etc. However, technical developments are still rare and slowed down by various factors linked to sensor cost, lack of system flexibility, difficulty of rapidly developing complex and robust applications, and lack of interaction among these systems themselves, or with their environment. This paper describes our proposal for a smart camera with real-time video processing capabilities. A CMOS sensor, processor and, reconfigurable unit associated in the same chip will allow scalability, flexibility, and high performance.

Real-time High Dynamic Range based on Multiple Non Destructive ReadOut during a Single Exposure

This paper presents a new method based on Non Destructive Readout (NDRO) to improve multi-exposure High Dynamic Range (HDR) Imaging. A sequence of Low-Dynamic Range (LDR) images can then be acquired during a single exposure. The concept enables the latency between LDR images to be removed as well as the intrinsic ghost artifacts observed using state-of-art HDR systems based on multi-exposures. The method has been applied to improve the performances of HDR sensor based on logarithmic pixels. Using the NDRO method, a Short Wave InfraRed (SWIR) camera has been designed to produce HDR IR videos. A real-time HDR video stream generation is achieved based on GPU implantation.

A 1.3 megapixel FPGA-based smart camera for high dynamic range real time video

International audience; A camera is able to capture only a part of a high dynamic range scene information. The same scene can be fully perceived by the human visual system. This is true especially for real scenes where the difference in light intensity between the dark areas and bright areas is high. The imaging technique which can overcome this problem is called HDR (High Dynamic Range). It produces images from a set of multiple LDR images (Low Dynamic Range), captured with different exposure times. This technique appears as one of the most appropriate and a cheap solution to enhance the dynamic range of captured environments. We developed an FPGA-based smart camera that produces a HDR liv…

An affordable contactless security system access for restricted area

International audience; We present in this paper a security system based on identity verification process and a low-cost smart camera , intended to avoid unauthorized access to restricted area. The Le2i laboratory has a longstanding experience in smart cameras implementation and design [1], for example in the case of real-time classical face detection [2] or human fall detection [3]. The principle of the system, fully thought and designed in our laboratory, is as follows: the allowed user presents a RFID card to the reader based on Odalid system [4]. The card ID, time and date of authorized access are checked using connection to an online server. In the same time, multi-modality identity ve…

High Dynamic Range Real-time Vision System for Robotic Applications

International audience; Robotics applications often requires vision systems capable of capturing a large amount of information related to a scene. With many camera sensors, the perception of information is limited in areas with strong contrasts. The High Dynamic Range (HDR) vision system can deal with these limitations. This paper describes the HDR-ARtiSt hardware platform (High Dynamic Range Advanced Real-time imaging System), a FPGA-based architecture that can produce a real- time high dynamic range video from successive image acquisition.

HDR-ARtiSt: High Dynamic Range Advanced Real-Time Imaging System

International audience; This paper describes the HDR-ARtiSt hardware platform, a FPGA-based architecture that can produce a real- time high dynamic range video from successive image acquisition. The hardware platform is built around a standard low dynamic range (LDR) CMOS sensor and a Virtex 5 FPGA board. The CMOS sensor is a EV76C560 provided by e2v. This 1.3 Megapixel device offers novel pixel integration/readout modes and em- bedded image pre-processing capabilities including multiframe acquisition with various exposure times. Our approach consists of a hardware architecture with different algorithms: double exposure control during image capture, building of an HDR image by combining the…

HDR-ARtiSt : une caméra intelligente dédiée à la vidéo à grande dynamique en temps réel

International audience; Une caméra vidéo standard n'est capable de capturer qu'une partie de l'information contenue dans une scène de gamme dynamique élevée. La même scène peut être presque entièrement perçue par le système visuel humain. Cela est particulièrement vrai pour les scènes réelles où la différence d'intensité lumineuse entre les zones sombres et les zones claires est élevée. La technique d'imagerie qui peut surmonter ce problème est appelé HDRi (High Dynamic Range imaging). Elle produit des images à partir d'un ensemble de plusieurs images LDRs (Low Dynamic Range), capturées avec différents temps d'exposition. Cette technique apparaît comme l'une des solutions la plus appropriée…

Fast prototyping of a SoC-based smart-camera: a real-time fall detection case study

International audience; Smart camera, i.e. cameras that are able to acquire and process images in real-time, is a typical example of the new embedded computer vision systems. A key example of application is automatic fall detection, which can be useful for helping elderly people in daily life. In this paper, we propose a methodology for development and fast-prototyping of a fall detection system based on such a smart camera, which allows to reduce the development time compared to standard approaches. Founded on a supervised classification approach, we propose a HW/SW implementation to detect falls in a home environment using a single camera and an optimized descriptor adapted to real-time t…

WiseEye: A Platform to Manage and Experiment on Smart Camera Networks

International audience; Embedded vision is probably at the edge of phenomenal expansion. The smart cameras are embedding some processing units which are more and more powerful. Last decade, high-speed image processing can be implemented on specifically designed architectures [1] nevertheless the designing time of such systems was quite high and time to market therefore as well. Since, powerful chips (i.e System On Chip) and quick prototyping methodologies are contently emerging [2],[3],[4] and enable more complex algorithms to be implemented faster. Moreover, smart cameras which are embedding flexible and powerful multi-core processors or Graphic Processors Unit (GPU) are now available and …

Robust spatio-temporal descriptors for real-time SVM-based fall detection

Overview of ghost correction for HDR video stream generation

International audience; Most digital cameras use low dynamic range image sensors, these LDR sensors can capture only a limited luminance dynamic range of the scene[1], to about two orders of magnitude (about 256 to 1024 levels). However, the dynamic range of real-world scenes varies over several orders of magnitude (10.000 levels). To overcome this limitation, several methods exist for creating high dynamic range (HDR) image (expensive method uses dedicated HDR image sensor and low-cost solutions using a conventional LDR image sensor). Large number of low-cost solutions applies a temporal exposure bracketing. The HDR image may be constructed with a HDR standard method (an additional step ca…

Scene-based noise reduction on a smart camera

International audience; Raw output data from CMOS image sensors tends to exhibit significant noise called Fixed-Pattern Noise (FPN) due to on-die variations between pixel photodetectors. FPN is often corrected by subtracting its value, estimated through calibration, from the sensor's raw signal. This paper introduces an on-line scene-based technique for an improved FPN compensation which does not rely on calibration, and hence is more robust to the dynamic changes in the FPN which may occur slowly over time. Development has been done with a special emphasis on real-time hardware implementation on a FPGA-based smart camera. Experimental results on different scenes are depicted showing that t…

HDR-ARtiSt: A 1280x1024-pixel Adaptive Real-time Smart camera for High Dynamic Range video

International audience; Standard cameras capture only a fraction of the information that is visible to the human visual system. This is specifically true for natural scenes including areas of low and high illumination due to transitions between sunlit and shaded areas. When capturing such a scene, many cameras are unable to store the full Dynamic Range (DR) resulting in low quality video where details are concealed in shadows or washed out by sunlight.The imaging technique that can overcome this problem is called HDR (High Dynamic Range) imaging. This paper describes a complete smart camera built around a standard off-the-shelf LDR (Low Dynamic Range) sensor and a Virtex 6 FPGA board. This …

A smart camera for High Dynamic Range imaging

International audience; A camera or a video camera is able to capture only a part of a high dynamic range scene information. The same scene can be almost totally perceived by the human visual system. This is true especially for real scenes where the difference in light intensity between the dark areas and bright areas is high. The imaging technique which can overcome this problem is called HDR (High Dynamic Range). It produces images from a set of multiple LDR images (Low Dynamic Range), captured with different exposure times. This technique appears as one of the most appropriate and a cheap solution to enhance the dynamic range of captured environments. We developed an FPGA-based smart cam…

Smart camera design for realtime High Dynamic Range imaging

International audience; Many camera sensors suffer from limited dynamic range. The result is that there is a lack of clear details in displayed images and videos. This paper describes our approach to generate high dynamic range (HDR) from an image sequence while modifying exposure times for each new frame. For this purpose, we propose an FPGA-based architecture that can produce a real-time high dynamic range video from successive image acquisition. Our hardware platform is build around a standard low dynamic range CMOS sensor and a Virtex 5 FPGA board. The CMOS sensor is a EV76C560 provided by e2v. This 1.3 Megapixel device offers novel pixel integration/readout modes and embedded image pre…