A data driven approach to a multilayer extreme learning machine

Handwritten Digit Recognition is a fundamental aspect of Computer Vision. It is a useful application in our daily lives like scanning checks at the bank, addresses from packages at the post office, facial recognition, or autonomous vehicle configuration. My project explores two main neural network architectures, The Extreme Learning Machine and Convolutional Neural Network, that combine and complement each other in computational efficiency as well as simplicity as compared to well-known neural network architectures.

In this thesis, we explore neural network techniques for use in recognizing images, first handwritten digits and then results for whole colored images are also presented. This work will focus solely on numeral networks to solve our problem. Current industry algorithms of regression and classification techniques have been used to achieve a small testing error of 0.35%. Our goal is to create a model that will be able to recognize and determine the handwritten digits from its image by using the concepts of Convolution Neural Networks as well as Extreme Learning machines. We also discuss how the goal of digit recognition can be extended to letters and an individual’s handwriting as well as whole colored images.

We propose a data driven approach to optimizing the training process while still maintaining the desired high accuracy rates. We attempt this by using of artificial neural networks that make use of randomized weights and biases on feed forward multi-layer networks where the weights and biases are easily solved rather than being constantly updated by back-propagation.

We further study the training process by exploring the idea of reconstruction error rates as a metric. By use of a convolution neural network, we extract features that can then be trained to recognize whole images. This reconstruction metric can help us better understand what happens between the layers instead of the proverbial black box.

We will give a brief introduction into neural networks, their architectures and propose a one-shot method for recursive feature selection and deep networks. We will discuss the advantages of our proposed model based on shortcomings of popular industry techniques. Next, we focus on the Handwritten Digit Recognition problem as a tool in Computer Vision. We explore the various industry standard algorithms of solving these problems in Computer Vision and how model adjustments have been made. We will then give an in depth discussion on Convolution Neural Networks and Extreme Learning Machines.