Visual-auditory substitution device for indoor navigation based on fast visual marker detection

FPGA-based smart camera : industrial applications

International audience; For the last two decades and still today, smart cameras offer innovative solutions for industrial vision applications. This kind of system associates a flexible image acquisition with high-speed processing possibilities. Many smart camera designs are based on FPGA components to obtain these two features. Indeed, the FPGA enables the CMOS sensor to be controlled and therefore to propose a configurable acquisition according to the application constraints (i.e. dynamic windowing). The configurable structure of a FPGA represents a key advantage for modifying the embedded processing (even on-the-fly using dynamic reconfiguration). Meanwhile, FPGA components offer a large …

An FPGA-based design for real-time Super Resolution Reconstruction

Since several decades, the camera spatial resolution is gradually increasing with the CMOS technology evolution. The image sensors provide more and more pixels, generating new constraints for the suitable optics. As an alternative, promising solutions propose Super Resolution (SR) image reconstruction to extend the image size without modifying the sensor architecture. Convincing state-of art studies demonstrate that these methods could even be implemented in real-time. Nevertheless, artifacts can be observed in highly textured areas of the image. In this paper, we propose a Local Adaptive Spatial Super Resolution (LASSR) method to fix this limitation. A real-time texture analysis is include…

Efficient smart-camera accelerator: A configurable motion estimator dedicated to video codec

Smart cameras are used in a large range of applications. Usually the smart cameras transmit the video or/and extracted information from the video scene, frequently on compressed format to fit with the application requirements. An efficient hardware accelerator that can be adapted and provide the required coding performances according to the events detected in the video, the available network bandwidth or user requirements, is therefore a key element for smart camera solutions. We propose in this paper to focus on a key part of the compression system: motion estimation. We have developed a flexible hardware implementation of the motion estimator based on FPGA component, fully compatible with…

Remote Photoplethysmography Based on Implicit Living Skin Tissue Segmentation

International audience; Region of interest selection is an essential part for remote photoplethysmography (rPPG) algorithms. Most of the time, face detection provided by a supervised learning of physical appearance features coupled with skin detection is used for region of interest selection. However, both methods have several limitations and we propose to implicitly select living skin tissue via their particular pulsatility feature. The input video stream is decomposed into several temporal superpixels from which pulse signals are extracted. Pulsatility measure for each temporal superpixel is then used to merge pulse traces and estimate the photoplethysmogram signal. This allows to select …

An Efficient Hardware implementation of MQ Decoder of JPEG2000

International audience; JPEG2000 is an international standard for still images intended to overcome the shortcomings of the existing JPEG standard. Compared to JPEG image compression techniques, JPEG2000 standard has not only better not only has better compression ratios, but it also offers some exciting features. As it's hard to meet the real-time requirement of image compression systems by software, it is necessary to implement compression system by hardware. The MQ decoder of the JPEG2000 standard is an important bottleneck for real-time applications. In order to meet the real-time requirement we propose in this paper a novel architecture for a MQ decoder with high throughput which is co…

Characterisation of centrifugal spreaders : from collection trays to high speed stereovision

AgrosupEAGEAPSICT1; Presenting Author Jürgen Vangeyte - ILVO, Agricultural Engineering, Belgium Description Precision fertilization requires accurate techniques for determining the spread pattern of centrifugal fertilizer spreaders. Traditionally, the spread pattern was determined by measuring the fertilizer distribution on the ground. Because such measurements are time consuming and complex, a high speed stereovision setup and dedicated image processing algorithms were developed for position and motion estimation of fertilizer grains leaving a centrifugal spreader. The measurements are combined with a ballistic flight model to predict the landing points of the individual grains and the act…

Embedded System Study for Real Time Boosting Based Face Detection

This paper describes a study for a real time embedded face detection system. Recently, the boosting based face detection algorithms proposed by [(Viola, P and Jone, M, 2001); (Lienhart, R, et al., 2003)] have gained a lot of attention and are considered as the fastest accurate face detection algorithms today. However, the embedded implementation of such algorithms into hardware is still a challenge, since these algorithms are heavily based on memory access. A sequential implementation model is built showing its lack of regularity in time consuming and speed of detection. We propose a parallel implementation that exploits the parallelism and the pipelining in these algorithms. This implement…

Efficient smart-camera accelerator: an configurable motion estimator dedicated to video codec

International audience; Smart cameras are used in a large range of applications. Usually the smart cameras transmit the video or/and extracted information from the video scene, frequently on compressed format to fit with the application requirements. An efficient hardware accelerator that can be adapted and provide the required coding performances according to the events detected in the video, the available network bandwidth or user requirements, is therefore a key element for smart camera solutions. We propose in this paper to focus on a key part of the compression system: motion estimation. We have developed a flexible hardware implementation of the motion estimator based on FPGA componen…

Definition and Performance Evaluation of a Robust SVM Based Fall Detection Solution

We propose an automatic approach to detect falls in home environment. A Support Vector Machine based classifier is fed by a set of selected features extracted from human body silhouette tracking. The classifier is followed by filtering operations taking into account the temporal nature of a video. The features are based on height and width of human body bounding box, the user's trajectory with her/his orientation, Projection Histograms and moments of order 0, 1 and 2. We study several combinations of usual transformations of the features (Fourier Transform, Wavelet transform, first and second derivatives), and we show experimentally that it is possible to achieve high performance using a si…

Real-Time 3D Face Acquisition Using Reconfigurable Hybrid Architecture

Acquiring 3D data of human face is a general problem which can be applied in face recognition, virtual reality, and many other applications. It can be solved using stereovision. This technique consists in acquiring data in three dimensions from two cameras. The aim is to implement an algorithmic chain which makes it possible to obtain a three-dimensional space from two two-dimensional spaces: two images coming from the two cameras. Several implementations have already been considered. We propose a new simple real-time implementation based on a hybrid architecture (FPGA-DSP), allowing to consider an embedded and reconfigurable processing. Then we show our method which provides depth map of …

Real-time High Dynamic Range based on Multiple Non Destructive ReadOut during a Single Exposure

This paper presents a new method based on Non Destructive Readout (NDRO) to improve multi-exposure High Dynamic Range (HDR) Imaging. A sequence of Low-Dynamic Range (LDR) images can then be acquired during a single exposure. The concept enables the latency between LDR images to be removed as well as the intrinsic ghost artifacts observed using state-of-art HDR systems based on multi-exposures. The method has been applied to improve the performances of HDR sensor based on logarithmic pixels. Using the NDRO method, a Short Wave InfraRed (SWIR) camera has been designed to produce HDR IR videos. A real-time HDR video stream generation is achieved based on GPU implantation.

An efficient hardware implementation of MQ decoder of the JPEG2000

Abstract JPEG2000 is an international standard for still images intended to overcome the shortcomings of the existing JPEG standard. Compared to JPEG image compression techniques, JPEG2000 standard has not only better not only has better compression ratios, but it also offers some exciting features. As it’s hard to meet the real-time requirement of image compression systems by software, it is necessary to implement compression system by hardware. The MQ decoder of the JPEG2000 standard is an important bottleneck for real-time applications. In order to meet the real-time requirement we propose in this paper a novel architecture for a MQ decoder with high throughput which is comparable to tha…

Real-time flaw detection on a complex object: comparison of results using classification with a support vector machine, boosting, and hyperrectangle-based method

We present a classification work performed on industrial parts using artificial vision, a support vector machine (SVM), boost- ing, and a combination of classifiers. The object to be controlled is a coated heater used in television sets. Our project consists of detect- ing anomalies under manufacturer production, as well as in classi- fying the anomalies among 20 listed categories. Manufacturer speci- fications require a minimum of ten inspections per second without a decrease in the quality of the produced parts. This problem is ad- dressed by using a classification system relying on real-time ma- chine vision. To fulfill both real-time and quality constraints, three classification algorit…

Machine Learning VS Transfer Learning - Smart Camera Implementation for Face Authentication

The aim of this paper is to highlight differences between classical machine learning and transfer learning applied to low cost real-time face authentication. Furthermore, in an access control context, the size of biometric data should be minimized so it can be stored on a remote personal media. These constraints have led us to compare only lightest versions of these algorithms. Transfer learning applied on Mobilenet v1 raises to 85% of accuracy, for a 457Ko model, with 3680s and 1.43s for training and prediction tasks. In comparison, the fastest integrated method (Random Forest) shows accuracy up to 90% for a 7,9Ko model, with a fifth of a second to be trained and a hundred of microseconds …

A platform for the development and the validation of HW IP components starting from reference software specifications

Abstract Signal processing algorithms become more and more efficient as a result of the developments of new standards. It is particularly true in the field video compression. However, at each improvement in efficiency and functionality, the complexity of the algorithms is also increasing. Textual specifications, that in the past were the original form of specifications, have been substituted by reference software which became the starting point of any design flow leading to implementation. Therefore, designing an embedded application has become equivalent to port a generic software on a, possibly heterogeneous, embedded platform. Such operation is getting more and more difficult because of …

Smart camera based on an Embedded HW/SW Co-Processor

Abstract This paper describes an image acquisition and a processing system based on a new coprocessor architecture designed for CMOS sensor imaging. The system exploits the full potential CMOS selective access imaging technology because the coprocessor unit is integrated into the image acquisition loop. The acquisition and coprocessing architecture are compatible with the majority of CMOS sensors. It enables the dynamic selection of a wide variety of acquisition modes as well as the reconfiguration and implementation of high-performance image preprocessing algorithms (calibration, filtering, denoising, binarization, pattern recognition). Furthermore, the processing and data transfer, from t…

An affordable contactless security system access for restricted area

International audience; We present in this paper a security system based on identity verification process and a low-cost smart camera , intended to avoid unauthorized access to restricted area. The Le2i laboratory has a longstanding experience in smart cameras implementation and design [1], for example in the case of real-time classical face detection [2] or human fall detection [3]. The principle of the system, fully thought and designed in our laboratory, is as follows: the allowed user presents a RFID card to the reader based on Odalid system [4]. The card ID, time and date of authorized access are checked using connection to an online server. In the same time, multi-modality identity ve…

Hardware Implementation of a Configurable Motion Estimator for Adjusting the Video Coding Performances

International audience; Despite the diversity of video compression standard, the motion estimation still remains a key process which is used in most of them. Moreover, the required coding performances (bit-rate, PSNR, image spatial resolution, etc.) depend obviously of the application, the environment and the network communication. The motion estimation can therefore be adapted to fit with these performances. Meanwhile, the real time encoding is required in many applications. In order to reach this goal, we propose in this paper a hardware implementation of the motion estimator which enables the integer motion search algorithms to be modified and the fractional search and variable block siz…

An Efficient Hardware Implementation of Diamond Search Motion Estimation Based on CAL Dataflow Language

International audience

Design of low noise and low power photo-receptor for CMOS vision sensor

International audience

WiseNET: smart camera network combined with ontological reasoning for smart building management

International audience; Visual sensor networks (VSN) have become a part of our daily life [1] [2]. Based on our experience we have identified two main problems on VSN. Firstly, the problem of selecting relevant information from the huge amount of data given by the network. Secondly, the problem of integrating the information coming from the different nodes of the network, i.e., linking the different informations together in order to take a decision. These problems can be overcome by including smart cameras in charge of extracting the significant information from the scene and by adding contextual semantic information, i.e., semantic information of what the camera observes, building informat…

High Efficiency Architecture of Half-Pel Motion Estimation for H.264 Video Coding

International audience

«Motion Estimation Accelerator with User Search Strategy in an RVC Context»

Motion estimation represents a key module in video compression. The RVC context requires proposing a flexible solution for motion estimation. According to the nature of the application, a full search is sometimes not suitable, hence, alternative fast/reduced solutions should be considered. This paper proposes a model and implementation of a flexible motion estimation engine, which can be configured to support any user-defined search strategy. Typically, the computational requirements of the search strategy can be traded with the RD-performance of the obtained video encoder. A CAL dataflow description of the accelerator is proposed so that it can be easily handled in the RVC context. An auto…

Fast prototyping of a SoC-based smart-camera: a real-time fall detection case study

International audience; Smart camera, i.e. cameras that are able to acquire and process images in real-time, is a typical example of the new embedded computer vision systems. A key example of application is automatic fall detection, which can be useful for helping elderly people in daily life. In this paper, we propose a methodology for development and fast-prototyping of a fall detection system based on such a smart camera, which allows to reduce the development time compared to standard approaches. Founded on a supervised classification approach, we propose a HW/SW implementation to detect falls in a home environment using a single camera and an optimized descriptor adapted to real-time t…

Smart Vision systems: adequacy architecture algorithm, design and hardware implementations

WiseEye: A Platform to Manage and Experiment on Smart Camera Networks

International audience; Embedded vision is probably at the edge of phenomenal expansion. The smart cameras are embedding some processing units which are more and more powerful. Last decade, high-speed image processing can be implemented on specifically designed architectures [1] nevertheless the designing time of such systems was quite high and time to market therefore as well. Since, powerful chips (i.e System On Chip) and quick prototyping methodologies are contently emerging [2],[3],[4] and enable more complex algorithms to be implemented faster. Moreover, smart cameras which are embedding flexible and powerful multi-core processors or Graphic Processors Unit (GPU) are now available and …

Compression embarquée temps réel pour caméras rapides

Les caméras rapides sont de puissants outils pour étudier, par exemple, la dynamique des fluides ou le déplacement des pièces mécaniques lors d'un processus de fabrication. Nous décrivons dans ce papier, un nouveau type de caméra rapide possédant un fonctionnement original. En effet, outre le fait qu'elle utilise comme d'autres caméras, la grande flexibilité des capteurs CMOS en termes d'acquisition (ROI), elle est novatrice au niveau du transfert des données. Celles-ci pouvant être à la fois traitées et/ou compressées en temps réel au sein même de la caméra. Le transfert peut s'effectuer alors à l'aide d'une simple connection série de type USB 2.0. On réalise ainsi l'économie d'une mémoire…

Know Beyond Seeing: Combining Computer Vision with Semantic Reasoning

International audience; To date, computer vision systems are limited to extract the digital data of what the cameras "see". However, the meaning of what they observe could be greatly enhanced by considering the environment and common-sense knowledge. A new approach to combine computer vision with semantic modeling has been developed. This approach extracts the knowledge from images and uses it to perform real-time reasoning according to the contextual information, events of interest and logic rules. The reasoning with image knowledge allows protecting the privacy of the users, to overcome some problems of computer vision such as occlusion and missed detections and to offer services such as …

Periodic Variance Maximization using Generalized Eigenvalue Decomposition applied to Remote Photoplethysmography estimation

International audience; A generic periodic variance maximization algorithm to extract periodic or quasi-periodic signals of unknown periods embedded into multi-channel temporal signal recordings is described in this paper. The algorithm combines the notion of maximizing a periodicity metric combined with the global optimization scheme to estimate the source periodic signal of an unknown period. The periodicity maximization is performed using Generalized Eigenvalue Decomposition (GEVD) and the global optimization is performed using tabu search. A case study of remote photoplethysmography signal estimation has been utilized to assess the performance of the method using videos from public data…

Robust spatio-temporal descriptors for real-time SVM-based fall detection

Comparative study of deep learning and classical methods applied to face authentication in context of high constraints application

International audience

Two-step cross correlation-based algorithm for motion estimation applied to fertilizer granules' motion during centrifugal spreading

Imaging systems are progressing in both accuracy and ro- bustness, and their use in precision agriculture is increasing accordingly. One application of imaging systems is to understand and control the cen- trifugal fertilizing spreading process. Predicting the spreading pattern on the ground relies on an estimation of the trajectories and velocities of ejected granules. The algorithms proposed to date have shown low ac- curacy, with an error rate of a few pixels. But a more accurate estimation of the motion of the granules can be achieved. Our new two-step cross- correlation-based algorithm is based on the technique used in particle image velocimetry (PIV), which has yielded highly accurate…

Optimisation conjointe de la taille de stockage et des performances de modèles de classification pour l’authentification de visages

International audience

An efficient hardware implementation of Diamond Search motion estimation using CAL dataflow language

Motion estimation represents a key module in video compression. The Reconfigurable Video Coding context (RVC) requires proposing a flexible solution for motion estimation. The motion estimation performance should be modified to fit with the user or the environment's constraints. Depending on the required performances fixed by the application, a full search is sometimes not suitable, hence, alternative fast/reduced solutions should be considered. In this paper, an efficient Diamond Search motion estimation, described in RVC-CAL actor language, is introduced. Starting from a high level description based CAL language, an automatic translation of the proposed CAL module to HDL is performed. Thi…

An Optimised Indoor Deployment of Visual Sensor Networks

Optimized Parallel Implementation of Face Detection based on GPU component

Display Omitted An algorithm for face detection has been implemented on CPU.An acceleration of this algorithm on GPU migration.Performance of GPU implementation shows the effectiveness of this implementation.Another optimization method on GPU are operated. Face detection is an important aspect for various domains such as: biometrics, video surveillance and human computer interaction. Generally a generic face processing system includes a face detection, or recognition step, as well as tracking and rendering phase. In this paper, we develop a real-time and robust face detection implementation based on GPU component. Face detection is performed by adapting the Viola and Jones algorithm. We hav…

Overview of ghost correction for HDR video stream generation

International audience; Most digital cameras use low dynamic range image sensors, these LDR sensors can capture only a limited luminance dynamic range of the scene[1], to about two orders of magnitude (about 256 to 1024 levels). However, the dynamic range of real-world scenes varies over several orders of magnitude (10.000 levels). To overcome this limitation, several methods exist for creating high dynamic range (HDR) image (expensive method uses dedicated HDR image sensor and low-cost solutions using a conventional LDR image sensor). Large number of low-cost solutions applies a temporal exposure bracketing. The HDR image may be constructed with a HDR standard method (an additional step ca…

Optimized spatio-temporal descriptors for real-time fall detection: comparison of support vector machine and Adaboost-based classification

We propose a supervised approach to detect falls in a home environment using an optimized descriptor adapted to real-time tasks. We introduce a realistic dataset of 222 videos, a new metric allowing evaluation of fall detection performance in a video stream, and an automatically optimized set of spatio-temporal descriptors which fed a supervised classifier. We build the initial spatio-temporal descriptor named STHF using several combinations of transformations of geometrical features (height and width of human body bounding box, the user’s trajectory with her/his orientation, projection histograms, and moments of orders 0, 1, and 2). We study the combinations of usual transformations of the…

Real-Time Temporal Superpixels for Unsupervised Remote Photoplethysmography

International audience; Segmentation is a critical step for many computer vision applications. Among them, the remote photoplethys-mography technique is significantly impacted by the quality of region of interest segmentation. With the heart-rate estimation accuracy, the processing time is obviously a key issue for real-time monitoring. Recent face detection algorithms can perform real-time processing, however for unsupervised algorithms, i.e. without any subject detection based on supervised learning, existing methods are not able to achieve real-time on regular platform. In this paper, we propose a new method to perform real-time un-supervised remote photoplethysmograhy based on efficient…

A New Development Framework for Multi-Core Processor based Smart-Camera Implementations

International audience; The exponential evolution of the smart camera processing performances is directly linked to the improvements on hardware processing elements. Nowadays, high processing performances can be reached considering hardware targets which enables a high level of task parallelism to be implemented. Highly regular tasks are good candidate for a reconfigurable logic implementation and less regular parts of the algorithm could be described on the processor. Meanwhile the prototyping time is related to the selected target and the associated development methodology. The implementation on reconfigurable logic is highly efficient in exploiting the intrinsic task parallelism neverthe…

Development of a High Irradiance LED Configuration for Small Field of View Motion Estimation of Fertilizer Particles

International audience; Better characterization of the fertilizer spreading process, especially the fertilizer pattern distribution on the ground, requires an accurate measurement of individual particle properties and dynamics. Both 2D and 3D high speed imaging techniques have been developed for this purpose. To maximize the accuracy of the predictions, a specific illumination level is required. This paper describes the development of a high irradiance LED system for high speed motion estimation of fertilizer particles. A spectral sensitivity factor was used to select the optimal LED in relation to the used camera from a range of commercially available high power LEDs. A multiple objective …

SYSTEMES DE VISION INTELLIGENTS : ADEQUATION ARCHITECTURE ALGORITHME, CONCEPTION ET IMPLANTATION MATERIELLES

Predicting spread patterns of centrifugal fertiliser spreaders

International audience; Nowadays farmers recognize the importance of a correct and precise fertiliser application: non-uniform spread patterns cause extra pressure on the environment and might result in economic losses for the farmer. In Europe most spreading is done by centrifugal fertilizer spreaders but their spreading process is not easy to monitor and to control. To perform a precise fertilising farmers need proper tools to determine and evaluate the spread pattern at farm level. Therefore the Flemish Institute for Agricultural and Fisheries Research (ILVO) is exploring and developing a fast and accurate technique for measuring the spread pattern of conventional centrifugal spreaders. …

Real Time Robust Embedded Face Detection Using High Level Description

Face detection is a fundamental prerequisite step in the process of face recognition. It consists of automatically finding all the faces in an image despite the considerable variations of lighting, background, appearance of people, position/orientation of faces, and their sizes. This type of object detection has the distinction of having a very large intra-class, making it a particularly difficult problem to solve, especially when one wishes to achieve real time processing. A human being has a great ability to analyze images. He can extract the information about it and focus only on areas of interest (the phenomenon of attention). Thereafter he can detect faces in an extremely reliable way.…

WiseNET - smart camera network interacting with a semantic model

This paper presents an innovative concept for a distributed system that combines a smart camera network with semantic reasoning. The proposed system is context sensitive and combines the information extracted by the smart camera with logic rules and knowledge of what the camera observes, building information and events that may occurred. The proposed system is a justification for the use of smart cameras, and it can improve the classical visual sensor networks (VSN) and enhance the standard computer vision approach. The main application of our system is smart building management, where we specifically focus on increasing the services of the building users.

Generation of Hardware/Software systems based on CAL dataflow description

International audience; This paper presents a new development of rapid prototyping tools for system design based on data-flow specifications. In this context, the efficiency of tools for the automatic translation from the data-flow programs to C and/or HDL are assessed by means of two design cases. The paper also introduces the new concept of the automatic synthesis of interfaces. Such generic interfaces are implemented by using an embedded microprocessor, which can support a large variety of interfaces already available as native IP libraries in the case of FPGA. The two design cases described here have been developed, tested and validated on different implementation platforms. The results…

Low-noise and low power photoreceptor using split-length MSOFET

International audience

Spatio-temporal descriptor for SVM and Adaboost based fall detection

International audience

Real-Time Temporal Superpixels for Unsupervised remote photopletysmography

International audience