Support vector machines in engineering: an overview

2014

This paper provides an overview of the support vector machine SVM methodology and its applicability to real-world engineering problems. Specifically, the aim of this study is to review the current state of the SVM technique, and to show some of its latest successful results in real-world problems present in different engineering fields. The paper starts by reviewing the main basic concepts of SVMs and kernel methods. Kernel theory, SVMs, support vector regression SVR, and SVM in signal processing and hybridization of SVMs with meta-heuristics are fully described in the first part of this paper. The adoption of SVMs in engineering is nowadays a fact. As we illustrate in this paper, SVMs can …

Multi-Temporal Image Classification with Kernels

2009

A Unified SVM Framework for Signal Estimation

2013

This paper presents a unified framework to tackle estimation problems in Digital Signal Processing (DSP) using Support Vector Machines (SVMs). The use of SVMs in estimation problems has been traditionally limited to its mere use as a black-box model. Noting such limitations in the literature, we take advantage of several properties of Mercer's kernels and functional analysis to develop a family of SVM methods for estimation in DSP. Three types of signal model equations are analyzed. First, when a specific time-signal structure is assumed to model the underlying system that generated the data, the linear signal model (so called Primal Signal Model formulation) is first stated and analyzed. T…

Robust γ-filter using support vector machines

2009

This Letter presents a new approach to time-series modelling using the support vector machines (SVM). Although the g-filter can provide stability in several time-series models, the SVM is proposed here to provide robustness in the estimation of the g-filter coefficients. Examples in chaotic time-series prediction and channel equalization show the advantages of the joint SVM g-filter. Teoría de la Señal y Comunicaciones

An Introduction to Kernel Methods

2009

Machine learning has experienced a great advance in the eighties and nineties due to the active research in artificial neural networks and adaptive systems. These tools have demonstrated good results in many real applications, since neither a priori knowledge about the distribution of the available data nor the relationships among the independent variables should be necessarily assumed. Overfitting due to reduced training data sets is controlled by means of a regularized functional which minimizes the complexity of the machine. Working with high dimensional input spaces is no longer a problem thanks to the use of kernel methods. Such methods also provide us with new ways to interpret the cl…

From Signal Processing to Machine Learning

2018

This chapter reviews the main landmarks of signal processing in the 20th century from the perspective of algorithmic developments. It focuses on cross‐fertilization with the field of statistical (machine) learning in the last decades. In the 21st century, model and data assumptions as well as algorithmic constraints are no longer valid, and the field of machine‐learning signal processing has erupted, with many successful stories to tell. The chapter also focuses on digital signal processing (DSP), which deals with the analysis of digitized and discrete sampled signals. Machine learning is a branch of computer science and artificial intelligence that enables computers to learn from data. Mac…

Explicit Recursive and Adaptive Filtering in Reproducing Kernel Hilbert Spaces

2014

This brief presents a methodology to develop recursive filters in reproducing kernel Hilbert spaces. Unlike previous approaches that exploit the kernel trick on filtered and then mapped samples, we explicitly define the model recursivity in the Hilbert space. For that, we exploit some properties of functional analysis and recursive computation of dot products without the need of preimaging or a training dataset. We illustrate the feasibility of the methodology in the particular case of the $\gamma$ -filter, which is an infinite impulse response filter with controlled stability and memory depth. Different algorithmic formulations emerge from the signal model. Experiments in chaotic and elect…

Non-linear System Identification with Composite Relevance Vector Machines

2007

Nonlinear system identification based on relevance vector machines (RVMs) has been traditionally addressed by stacking the input and/or output regressors and then performing standard RVM regression. This letter introduces a full family of composite kernels in order to integrate the input and output information in the mapping function efficiently and hence generalize the standard approach. An improved trade-off between accuracy and sparsity is obtained in several benchmark problems. Also, the RVM yields confidence intervals for the predictions, and it is less sensitive to free parameter selection. Teoría de la Señal y Comunicaciones

Explicit recursivity into reproducing kernel Hilbert spaces

2011

This paper presents a methodology to develop recursive filters in reproducing kernel Hilbert spaces (RKHS). Unlike previous approaches that exploit the kernel trick on filtered and then mapped samples, we explicitly define model recursivity in the Hilbert space. The method exploits some properties of functional analysis and recursive computation of dot products without the need of pre-imaging. We illustrate the feasibility of the methodology in the particular case of the gamma-filter, an infinite impulse response (IIR) filter with controlled stability and memory depth. Different algorithmic formulations emerge from the signal model. Experiments in chaotic and electroencephalographic time se…

Learning non-linear time-scales with kernel -filters

2009

A family of kernel methods, based on the @c-filter structure, is presented for non-linear system identification and time series prediction. The kernel trick allows us to develop the natural non-linear extension of the (linear) support vector machine (SVM) @c-filter [G. Camps-Valls, M. Martinez-Ramon, J.L. Rojo-Alvarez, E. Soria-Olivas, Robust @c-filter using support vector machines, Neurocomput. J. 62(12) (2004) 493-499.], but this approach yields a rigid system model without non-linear cross relation between time-scales. Several functional analysis properties allow us to develop a full, principled family of kernel @c-filters. The improved performance in several application examples suggest…

A Support Vector Machine Signal Estimation Framework

2018

Support vector machine (SVM) were originally conceived as efficient methods for pattern recognition and classification, and the SVR was subsequently proposed as the SVM implementation for regression and function approximation. Nowadays, the SVR and other kernel‐based regression methods have become a mature and recognized tool in digital signal processing (DSP). This chapter starts to pave the way to treat all the problems within the field of kernel machines, and presents the fundamentals for a simple, framework for tackling estimation problems in DSP using support vector machine SVM. It outlines the particular models and approximations defined within the framework. The chapter concludes wit…

Introduction to Digital Signal Processing

2018

Signal processing deals with the representation, transformation, and manipulation of signals and the information they contain. Typical examples include extracting the pure signals from a mixture observation (a field commonly known as deconvolution) or particular signal (frequency) components from noisy observations (generally known as filtering). This chapter outlines the basics of signal processing and then introduces the more advanced concepts of time‐frequency and time‐scale representations, as well as emerging fields of compressed sensing and multidimensional signal processing. When moving to multidimensional signal processing, a modern approach is taken from the point of view of statis…

Support Vector Machines Framework for Linear Signal Processing

2005

This paper presents a support vector machines (SVM) framework to deal with linear signal processing (LSP) problems. The approach relies on three basic steps for model building: (1) identifying the suitable base of the Hilbert signal space in the model, (2) using a robust cost function, and (3) minimizing a constrained, regularized functional by means of the method of Lagrange multipliers. Recently, autoregressive moving average (ARMA) system identification and non-parametric spectral analysis have been formulated under this framework. The generalized, yet simple, formulation of SVM LSP problems is particularized here for three different issues: parametric spectral estimation, stability of I…

Support Vector Machine and Kernel Classification Algorithms

2018

This chapter introduces the basics of support vector machine (SVM) and other kernel classifiers for pattern recognition and detection. It also introduces the main elements and concept underlying the successful binary SVM. The chapter starts by introducing the main elements and concept underlying the successful binary SVM. Next, it introduces more advanced topics in SVM for classification, including large margin filtering (LMF), SSL, active learning, and large‐scale classification using SVMs. The LMF method performs both signal filtering and classification simultaneously by learning the most appropriate filters. SSL with SVMs exploits the information contained in both labeled and unlabeled e…

Adaptive Kernel Learning for Signal Processing

2018

Adaptive filtering is a central topic in digital signal processing (DSP). By applying linear adaptive filtering principles in the kernel feature space, powerful nonlinear adaptive filtering algorithms can be obtained. This chapter introduces the wide topic of adaptive signal processing, and explores the emerging field of kernel adaptive filtering (KAF). In many signal processing applications, the problem of signal estimation is addressed. Probabilistic models have proven to be very useful in this context. The chapter discusses two families of kernel adaptive filters, namely kernel least mean squares (KLMS) and kernel recursive least‐squares (KRLS) algorithms. In order to design a practical …

Applications of Kernel Methods

2009

In this chapter, we give a survey of applications of the kernel methods introduced in the previous chapter. We focus on different application domains that are particularly active in both direct application of well-known kernel methods, and in new algorithmic developments suited to a particular problem. In particular, we consider the following application fields: biomedical engineering (comprising both biological signal processing and bioinformatics), communications, signal, speech and image processing.

Kernel-Based Framework for Multitemporal and Multisource Remote Sensing Data Classification and Change Detection

2008

The multitemporal classification of remote sensing images is a challenging problem, in which the efficient combination of different sources of information (e.g., temporal, contextual, or multisensor) can improve the results. In this paper, we present a general framework based on kernel methods for the integration of heterogeneous sources of information. Using the theoretical principles in this framework, three main contributions are presented. First, a novel family of kernel-based methods for multitemporal classification of remote sensing images is presented. The second contribution is the development of nonlinear kernel classifiers for the well-known difference and ratioing change detectio…

Sparse Deconvolution Using Support Vector Machines

2008

Sparse deconvolution is a classical subject in digital signal processing, having many practical applications. Support vector machine (SVM) algorithms show a series of characteristics, such as sparse solutions and implicit regularization, which make them attractive for solving sparse deconvolution problems. Here, a sparse deconvolution algorithm based on the SVM framework for signal processing is presented and analyzed, including comparative evaluations of its performance from the points of view of estimation and detection capabilities, and of robustness with respect to non-Gaussian additive noise. Publicado

Discrete Time Signal Processing Framework with Support Vector Machines

2007

Digital signal processing (DSP) of time series using SVM has been addressed in the literature with a straightforward application of the SVM kernel regression, but the assumption of independently distributed samples in regression models is not fulfilled by a time-series problem. Therefore, a new branch of SVM algorithms has to be developed for the advantageous application of SVM concepts when we process data with underlying time-series structure. In this chapter, we summarize our past, present, and future proposal for the SVM-DSP frame-work, which consists of several principles for creating linear and nonlinear SVM algorithms devoted to DSP problems. First, the statement of linear signal mod…

Upport vector machines for nonlinear kernel ARMA system identification.

2006

Nonlinear system identification based on support vector machines (SVM) has been usually addressed by means of the standard SVM regression (SVR), which can be seen as an implicit nonlinear autoregressive and moving average (ARMA) model in some reproducing kernel Hilbert space (RKHS). The proposal of this letter is twofold. First, the explicit consideration of an ARMA model in an RKHS (SVM-ARMA 2k) is proposed. We show that stating the ARMA equations in an RKHS leads to solving the regularized normal equations in that RKHS, in terms of the autocorrelation and cross correlation of the (nonlinearly) transformed input and output discrete time processes. Second, a general class of SVM-based syste…

Robust g-filter using support vector method

2004

This Letter presents a new approach to time series modelling using the support vector machines (SVM). Although the g filter can provide stability in several time series models, the SVM is proposed here to provide robustness in the estimation of the g filter coefficients. Examples in chaotic time series prediction and channel equalization show the advantages of the joint SVM g filter. Publicado