Support vector machines in engineering: an overview

This paper provides an overview of the support vector machine SVM methodology and its applicability to real-world engineering problems. Specifically, the aim of this study is to review the current state of the SVM technique, and to show some of its latest successful results in real-world problems present in different engineering fields. The paper starts by reviewing the main basic concepts of SVMs and kernel methods. Kernel theory, SVMs, support vector regression SVR, and SVM in signal processing and hybridization of SVMs with meta-heuristics are fully described in the first part of this paper. The adoption of SVMs in engineering is nowadays a fact. As we illustrate in this paper, SVMs can …

Multi-Temporal Image Classification with Kernels

Assessing the Impact of Temporary Retail Price Discounts Intervals Using SVM Semiparametric Regression

Although the marketing literature has found that temporary retail price discounts cause a significant sales increase, little is known about the specific characteristics of deals that influence the magnitude of the sales spike. In this paper, we analyse the impact of the length of temporary retail price discounts periods on the sales increase using scanner-store daily-sales data for two frequently purchased product categories: ground coffee (a storable category) and yogurt (a perishable category).Wedevelop a robust semiparametric regression model based on support vector statistical theory with several previously proposed predictors along with a daily time description. This model also makes i…

Using Daily Store-level Data to Understand Price Promotion Effects in a Semiparametric Regression Model

Though it has been widely reported in the marketing literature that temporary price discounts generate substantial short-term sales increase, the shape of the deal effect curve constitutes a key research topic, for which there are still limited empirical results. To address this issue, a semiparametric regression approach is used to model the complex nature of this phenomenon. Our model is developed at the brand level using daily store-level scanner-data, which allows the study of several nonreported promotional effects, such as the influence of the day of the week both in promotional and nonpromotional periods. The results show that the weekend is the most effective in increasing promotion…

Evaluating temporary retail price discounts using semiparametric regression

PurposeTo analyze the impact of temporary retail price discount on a consumer goods product category using semiparametric regression and considering different promotional price discount characteristics as well as brand characteristics.Design/methodology/approachA semiparametric regression model using Support Vector Machines, which aim to evaluate retailers' decisions about temporary price discounts, has been developed. The model is derived from the analysis of historical sales data, which provide precise evaluation of previous temporary price discounts periods. The model is also consistent with ample empirical evidence showing that historical retail sales data can be used to evaluate the im…

A Unified SVM Framework for Signal Estimation

This paper presents a unified framework to tackle estimation problems in Digital Signal Processing (DSP) using Support Vector Machines (SVMs). The use of SVMs in estimation problems has been traditionally limited to its mere use as a black-box model. Noting such limitations in the literature, we take advantage of several properties of Mercer's kernels and functional analysis to develop a family of SVM methods for estimation in DSP. Three types of signal model equations are analyzed. First, when a specific time-signal structure is assumed to model the underlying system that generated the data, the linear signal model (so called Primal Signal Model formulation) is first stated and analyzed. T…

Robust γ-filter using support vector machines

This Letter presents a new approach to time-series modelling using the support vector machines (SVM). Although the g-filter can provide stability in several time-series models, the SVM is proposed here to provide robustness in the estimation of the g-filter coefficients. Examples in chaotic time-series prediction and channel equalization show the advantages of the joint SVM g-filter. Teoría de la Señal y Comunicaciones

An Introduction to Kernel Methods

Machine learning has experienced a great advance in the eighties and nineties due to the active research in artificial neural networks and adaptive systems. These tools have demonstrated good results in many real applications, since neither a priori knowledge about the distribution of the available data nor the relationships among the independent variables should be necessarily assumed. Overfitting due to reduced training data sets is controlled by means of a regularized functional which minimizes the complexity of the machine. Working with high dimensional input spaces is no longer a problem thanks to the use of kernel methods. Such methods also provide us with new ways to interpret the cl…

From Signal Processing to Machine Learning

This chapter reviews the main landmarks of signal processing in the 20th century from the perspective of algorithmic developments. It focuses on cross‐fertilization with the field of statistical (machine) learning in the last decades. In the 21st century, model and data assumptions as well as algorithmic constraints are no longer valid, and the field of machine‐learning signal processing has erupted, with many successful stories to tell. The chapter also focuses on digital signal processing (DSP), which deals with the analysis of digitized and discrete sampled signals. Machine learning is a branch of computer science and artificial intelligence that enables computers to learn from data. Mac…

A Review of Kernel Methods in ECG Signal Classification

Kernel methods have been shown to be effective in the analysis of electrocardiogram (ECG) signals. These techniques provide a consistent and well-founded theoretical framework for developing nonlinear algorithms. Kernel methods exhibit useful properties when applied to challenging design scenarios, such as: (1) when dealing with low number of (potentially high dimensional) training samples; (2) in the presence of heterogenous multimodalities; and (3) with different noise sources in the data. These characteristics are particularly appropriate for biomedical signal processing and analysis, and hence, the widespread of these techniques in biomedical signal processing in general, and in ECG dat…

Fuzzy sigmoid kernel for support vector classifiers

This Letter proposes the use of the fuzzy sigmoid function presented in (IEEE Trans. Neural Networks 14(6) (2003) 1576) as non-positive semi-definite kernel in the support vector machines framework. The fuzzy sigmoid kernel allows lower computational cost, and higher rate of positive eigenvalues of the kernel matrix, which alleviates current limitations of the sigmoid kernel.

Explicit Recursive and Adaptive Filtering in Reproducing Kernel Hilbert Spaces

This brief presents a methodology to develop recursive filters in reproducing kernel Hilbert spaces. Unlike previous approaches that exploit the kernel trick on filtered and then mapped samples, we explicitly define the model recursivity in the Hilbert space. For that, we exploit some properties of functional analysis and recursive computation of dot products without the need of preimaging or a training dataset. We illustrate the feasibility of the methodology in the particular case of the $\gamma$ -filter, which is an infinite impulse response filter with controlled stability and memory depth. Different algorithmic formulations emerge from the signal model. Experiments in chaotic and elect…

Non-linear System Identification with Composite Relevance Vector Machines

Nonlinear system identification based on relevance vector machines (RVMs) has been traditionally addressed by stacking the input and/or output regressors and then performing standard RVM regression. This letter introduces a full family of composite kernels in order to integrate the input and output information in the mapping function efficiently and hence generalize the standard approach. An improved trade-off between accuracy and sparsity is obtained in several benchmark problems. Also, the RVM yields confidence intervals for the predictions, and it is less sensitive to free parameter selection. Teoría de la Señal y Comunicaciones

Learning non-linear time-scales with kernel -filters

A family of kernel methods, based on the @c-filter structure, is presented for non-linear system identification and time series prediction. The kernel trick allows us to develop the natural non-linear extension of the (linear) support vector machine (SVM) @c-filter [G. Camps-Valls, M. Martinez-Ramon, J.L. Rojo-Alvarez, E. Soria-Olivas, Robust @c-filter using support vector machines, Neurocomput. J. 62(12) (2004) 493-499.], but this approach yields a rigid system model without non-linear cross relation between time-scales. Several functional analysis properties allow us to develop a full, principled family of kernel @c-filters. The improved performance in several application examples suggest…

A Support Vector Machine Signal Estimation Framework

Support vector machine (SVM) were originally conceived as efficient methods for pattern recognition and classification, and the SVR was subsequently proposed as the SVM implementation for regression and function approximation. Nowadays, the SVR and other kernel‐based regression methods have become a mature and recognized tool in digital signal processing (DSP). This chapter starts to pave the way to treat all the problems within the field of kernel machines, and presents the fundamentals for a simple, framework for tackling estimation problems in DSP using support vector machine SVM. It outlines the particular models and approximations defined within the framework. The chapter concludes wit…

Introduction to Digital Signal Processing

Signal processing deals with the representation, transformation, and manipulation of signals and the information they contain. Typical examples include extracting the pure signals from a mixture observation (a field commonly known as deconvolution) or particular signal (frequency) components from noisy observations (generally known as filtering). This chapter outlines the basics of signal processing and then introduces the more advanced concepts of time‐frequency and time‐scale representations, as well as emerging fields of compressed sensing and multidimensional signal processing. When moving to multidimensional signal processing, a modern approach is taken from the point of view of statis…

Support Vector Machines Framework for Linear Signal Processing

This paper presents a support vector machines (SVM) framework to deal with linear signal processing (LSP) problems. The approach relies on three basic steps for model building: (1) identifying the suitable base of the Hilbert signal space in the model, (2) using a robust cost function, and (3) minimizing a constrained, regularized functional by means of the method of Lagrange multipliers. Recently, autoregressive moving average (ARMA) system identification and non-parametric spectral analysis have been formulated under this framework. The generalized, yet simple, formulation of SVM LSP problems is particularized here for three different issues: parametric spectral estimation, stability of I…

Support Vector Machine and Kernel Classification Algorithms

This chapter introduces the basics of support vector machine (SVM) and other kernel classifiers for pattern recognition and detection. It also introduces the main elements and concept underlying the successful binary SVM. The chapter starts by introducing the main elements and concept underlying the successful binary SVM. Next, it introduces more advanced topics in SVM for classification, including large margin filtering (LMF), SSL, active learning, and large‐scale classification using SVMs. The LMF method performs both signal filtering and classification simultaneously by learning the most appropriate filters. SSL with SVMs exploits the information contained in both labeled and unlabeled e…

Electrocardiographic Imaging for Atrial Fibrillation: A Perspective From Computer Models and Animal Experiments to Clinical Value

[EN] Electrocardiographic imaging (ECGI) is a technique to reconstruct non-invasively the electrical activity on the heart surface from body-surface potential recordings and geometric information of the torso and the heart. ECGI has shown scientific and clinical value when used to characterize and treat both atrial and ventricular arrhythmias. Regarding atrial fibrillation (AF), the characterization of the electrical propagation and the underlying substrate favoring AF is inherently more challenging than for ventricular arrhythmias, due to the progressive and heterogeneous nature of the disease and its manifestation, the small volume and wall thickness of the atria, and the relatively large…

Feature selection using support vector machines and bootstrap methods for ventricular fibrillation detection

Early detection of ventricular fibrillation (VF) is crucial for the success of the defibrillation therapy in automatic devices. A high number of detectors have been proposed based on temporal, spectral, and time-frequency parameters extracted from the surface electrocardiogram (ECG), showing always a limited performance. The combination ECG parameters on different domain (time, frequency, and time-frequency) using machine learning algorithms has been used to improve detection efficiency. However, the potential utilization of a wide number of parameters benefiting machine learning schemes has raised the need of efficient feature selection (FS) procedures. In this study, we propose a novel FS…

Adaptive Kernel Learning for Signal Processing

Adaptive filtering is a central topic in digital signal processing (DSP). By applying linear adaptive filtering principles in the kernel feature space, powerful nonlinear adaptive filtering algorithms can be obtained. This chapter introduces the wide topic of adaptive signal processing, and explores the emerging field of kernel adaptive filtering (KAF). In many signal processing applications, the problem of signal estimation is addressed. Probabilistic models have proven to be very useful in this context. The chapter discusses two families of kernel adaptive filters, namely kernel least mean squares (KLMS) and kernel recursive least‐squares (KRLS) algorithms. In order to design a practical …

Robust g-filter using support vector method

This Letter presents a new approach to time series modelling using the support vector machines (SVM). Although the g filter can provide stability in several time series models, the SVM is proposed here to provide robustness in the estimation of the g filter coefficients. Examples in chaotic time series prediction and channel equalization show the advantages of the joint SVM g filter. Publicado

Applications of Kernel Methods

In this chapter, we give a survey of applications of the kernel methods introduced in the previous chapter. We focus on different application domains that are particularly active in both direct application of well-known kernel methods, and in new algorithmic developments suited to a particular problem. In particular, we consider the following application fields: biomedical engineering (comprising both biological signal processing and bioinformatics), communications, signal, speech and image processing.

Kernel-Based Framework for Multitemporal and Multisource Remote Sensing Data Classification and Change Detection

The multitemporal classification of remote sensing images is a challenging problem, in which the efficient combination of different sources of information (e.g., temporal, contextual, or multisensor) can improve the results. In this paper, we present a general framework based on kernel methods for the integration of heterogeneous sources of information. Using the theoretical principles in this framework, three main contributions are presented. First, a novel family of kernel-based methods for multitemporal classification of remote sensing images is presented. The second contribution is the development of nonlinear kernel classifiers for the well-known difference and ratioing change detectio…

Sparse Deconvolution Using Support Vector Machines

Sparse deconvolution is a classical subject in digital signal processing, having many practical applications. Support vector machine (SVM) algorithms show a series of characteristics, such as sparse solutions and implicit regularization, which make them attractive for solving sparse deconvolution problems. Here, a sparse deconvolution algorithm based on the SVM framework for signal processing is presented and analyzed, including comparative evaluations of its performance from the points of view of estimation and detection capabilities, and of robustness with respect to non-Gaussian additive noise. Publicado

Discrete Time Signal Processing Framework with Support Vector Machines

Digital signal processing (DSP) of time series using SVM has been addressed in the literature with a straightforward application of the SVM kernel regression, but the assumption of independently distributed samples in regression models is not fulfilled by a time-series problem. Therefore, a new branch of SVM algorithms has to be developed for the advantageous application of SVM concepts when we process data with underlying time-series structure. In this chapter, we summarize our past, present, and future proposal for the SVM-DSP frame-work, which consists of several principles for creating linear and nonlinear SVM algorithms devoted to DSP problems. First, the statement of linear signal mod…

Using Support Vector Semiparametric Regression to estimate the effects of pricing on brand substitution

Upport vector machines for nonlinear kernel ARMA system identification.

Nonlinear system identification based on support vector machines (SVM) has been usually addressed by means of the standard SVM regression (SVR), which can be seen as an implicit nonlinear autoregressive and moving average (ARMA) model in some reproducing kernel Hilbert space (RKHS). The proposal of this letter is twofold. First, the explicit consideration of an ARMA model in an RKHS (SVM-ARMA 2k) is proposed. We show that stating the ARMA equations in an RKHS leads to solving the regularized normal equations in that RKHS, in terms of the autocorrelation and cross correlation of the (nonlinearly) transformed input and output discrete time processes. Second, a general class of SVM-based syste…