Search results for "Random subspace method"
showing 10 items of 11 documents
Bagging and Boosting with Dynamic Integration of Classifiers
2000
One approach in classification tasks is to use machine learning techniques to derive classifiers using learning instances. The co-operation of several base classifiers as a decision committee has succeeded to reduce classification error. The main current decision committee learning approaches boosting and bagging use resampling with the training set and they can be used with different machine learning techniques which derive base classifiers. Boosting uses a kind of weighted voting and bagging uses equal weight voting as a combining method. Both do not take into account the local aspects that the base classifiers may have inside the problem space. We have proposed a dynamic integration tech…
A local complexity based combination method for decision forests trained with high-dimensional data
2012
Accurate machine learning with high-dimensional data is affected by phenomena known as the “curse” of dimensionality. One of the main strategies explored in the last decade to deal with this problem is the use of multi-classifier systems. Several of such approaches are inspired by the Random Subspace Method for the construction of decision forests. Furthermore, other studies rely on estimations of the individual classifiers' competence, to enhance the combination in the multi-classifier and improve the accuracy. We propose a competence estimate which is based on local complexity measurements, to perform a weighted average combination of the decision forest. Experimental results show how thi…
A dynamic integration algorithm for an ensemble of classifiers
1999
Numerous data mining methods have recently been developed, and there is often a need to select the most appropriate data mining method or methods. The method selection can be done statically or dynamically. Dynamic selection takes into account characteristics of a new instance and usually results in higher classification accuracy. We discuss a dynamic integration algorithm for an ensemble of classifiers. Our algorithm is a new variation of the stacked generalization method and is based on the basic assumption that each basic classifier is best inside certain subareas of the application domain. The algorithm includes two main phases: a learning phase, which collects information about the qua…
Decision Committee Learning with Dynamic Integration of Classifiers
2000
Decision committee learning has demonstrated spectacular success in reducing classification error from learned classifiers. These techniques develop a classifier in the form of a committee of subsidiary classifiers. The combination of outputs is usually performed by majority vote. Voting, however, has a shortcoming. It is unable to take into account local expertise. When a new instance is difficult to classify, then the average classifier will give a wrong prediction, and the majority vote will more probably result in a wrong prediction. Instead of voting, dynamic integration of classifiers can be used, which is based on the assumption that each committee member is best inside certain subar…
Ranking of Brain Tumour Classifiers Using a Bayesian Approach
2009
This study presents a ranking for classifers using a Bayesian perspective. This ranking framework is able to evaluate the performance of the models to be compared when they are inferred from different sets of data. It also takes into account the performance obtained on samples not used during the training of the classifiers. Besides, this ranking assigns a prior to each model based on a measure of similarity of the training data to a test case. An evaluation consisting of ranking brain tumour classifiers is presented. These multilayer perceptron classifiers are trained with 1H magnetic resonance spectroscopy (MRS) signals following a multiproject multicenter evaluation approach. We demonstr…
Dynamic integration of classifiers in the space of principal components
2003
Recent research has shown the integration of multiple classifiers to be one of the most important directions in machine learning and data mining. It was shown that, for an ensemble to be successful, it should consist of accurate and diverse base classifiers. However, it is also important that the integration procedure in the ensemble should properly utilize the ensemble diversity. In this paper, we present an algorithm for the dynamic integration of classifiers in the space of extracted features (FEDIC). It is based on the technique of dynamic integration, in which local accuracy estimates are calculated for each base classifier of an ensemble, in the neighborhood of a new instance to be pr…
Feature Selection for Ensembles of Simple Bayesian Classifiers
2002
A popular method for creating an accurate classifier from a set of training data is to train several classifiers, and then to combine their predictions. The ensembles of simple Bayesian classifiers have traditionally not been a focus of research. However, the simple Bayesian classifier has much broader applicability than previously thought. Besides its high classification accuracy, it also has advantages in terms of simplicity, learning speed, classification speed, storage space, and incrementality. One way to generate an ensemble of simple Bayesian classifiers is to use different feature subsets as in the random subspace method. In this paper we present a technique for building ensembles o…
Ensemble Feature Selection Based on Contextual Merit and Correlation Heuristics
2001
Recent research has proven the benefits of using ensembles of classifiers for classification problems. Ensembles of diverse and accurate base classifiers are constructed by machine learning methods manipulating the training sets. One way to manipulate the training set is to use feature selection heuristics generating the base classifiers. In this paper we examine two of them: correlation-based and contextual merit -based heuristics. Both rely on quite similar assumptions concerning heterogeneous classification problems. Experiments are considered on several data sets from UCI Repository. We construct fixed number of base classifiers over selected feature subsets and refine the ensemble iter…
Ensemble feature selection with the simple Bayesian classification
2003
Abstract A popular method for creating an accurate classifier from a set of training data is to build several classifiers, and then to combine their predictions. The ensembles of simple Bayesian classifiers have traditionally not been a focus of research. One way to generate an ensemble of accurate and diverse simple Bayesian classifiers is to use different feature subsets generated with the random subspace method. In this case, the ensemble consists of multiple classifiers constructed by randomly selecting feature subsets, that is, classifiers constructed in randomly chosen subspaces. In this paper, we present an algorithm for building ensembles of simple Bayesian classifiers in random sub…
Evaluating Classifiers for Mobile-Masquerader Detection
2006
As a result of the impersonation of a user of a mobile terminal, sensitive information kept locally or accessible over the network can be abused. The means of masquerader detection are therefore needed to detect the cases of impersonation. In this paper, the problem of mobile-masquerader detection is considered as a problem of classifying the user behaviour as originating from the legitimate user or someone else. Different behavioural characteristics are analysed by designated one-class classifiers whose classifications are combined. The paper focuses on selecting the classifiers for mobile-masquerader detection. The selection process is conducted in two phases. First, the classification ac…