Statistical analysis of multilayer perceptrons performances

The paper is based on a series of studies on the learning capabilities of multilayered perceptrons (MLP). The complexity of these nonlinear systems can be varied, acting for instance on the number of hidden units, but we will be confronted with a choice dilemma, concerning the optimal complexity of the system for a given problem. By the mean of statistical methods, we have found that the effective number of hidden units is smaller than the potential size; some units have a "binary" activation level or a time constant activation. We also prove that weight initialization to small values is recommended and reduce the effective size of the hidden layer.

On Hagelbarger’s and Shannon’s matching pennies playing machines

Abstract In the 1950s, Hagelbarger’s Sequence Extrapolating Robot (SEER) and Shannon’s Mind-Reading Machine (MRM) were the state-of-the-art research results in playing the well-known “matching pennies” game. In our research we perform a software implementation for both machines in order to test the common statement that MRM, even simpler, beats SEER. Also, we propose a simple contextual predictor (SCP) and use it to compete with SEER and MRM. As expected, experimental results proves the claimed MRM superiority over SEER and even the SCP’s superiority over both SEER and MRM. At the end, we draw some conclusions and propose further research ideas, like the use of mixing models methods and the…

Improving Karhunen-Loeve based transform coding by using square isometries

We propose, for an image compression system based on the Karhunen-Loeve transform implemented by neural networks, to take into consideration the 8 square isometries of an image block. The proper isometry applied puts the 8*8 square image block in a standard position, before applying the image block as input to the neural network architecture. The standard position is defined based on the variance of its four 4*4 sub-blocks (quadro partitioned) and brings the sub-block having the greatest variance in a specific corner and in another specific adjoining corner the sub-block having the second variance (if this is not possible the third is considered). The use of this "preprocessing" phase was e…

DBSCAN Algorithm for Document Clustering

Abstract Document clustering is a problem of automatically grouping similar document into categories based on some similarity metrics. Almost all available data, usually on the web, are unclassified so we need powerful clustering algorithms that work with these types of data. All common search engines return a list of pages relevant to the user query. This list needs to be generated fast and as correct as possible. For this type of problems, because the web pages are unclassified, we need powerful clustering algorithms. In this paper we present a clustering algorithm called DBSCAN – Density-Based Spatial Clustering of Applications with Noise – and its limitations on documents (or web pages)…

Unknown order process emulation

Approaches the emulation problem using feedforward neural networks of single input single output (SISO) processes, applying a backpropagation method with a higher convergence rate. In this kind of application, difficult problems appear when the system's order is a priori unknown. A search through the SISO processes space is proposed, aiming to find a favorable neural emulator over the training examples set.

Weights Space Exploration Using Genetic Algorithms for Meta-classifier in Text Document Classification

Merging the transform step and the quantization step for Karhunen-Loeve transform based image compression

Transform coding is one of the most important methods for lossy image compression. The optimum linear transform - known as Karhunen-Loeve transform (KLT) - was difficult to implement in the classic way. Now, due to continuous improvements in neural network's performance, the KLT method becomes more topical then ever. We propose a new scheme where the quantization step is merged together with the transform step during the learning phase. The new method is tested for different levels of quantization and for different types of quantizers. Experimental results presented in the paper prove that the new proposed scheme always gives better results than the state-of-the-art solution.