Journal article
Two-Stage Orthogonal Least Squares Methods for Neural Network Construction
IEEE transaction on neural networks and learning systems, Vol.26(8), pp.1608-1621
08/2015
DOI: 10.1109/TNNLS.2014.2346399
PMID: 25222956
Abstract
A number of neural networks can be formulated as the linear-in-the-parameters models. Training such networks can be transformed to a model selection problem where a compact model is selected from all the candidates using subset selection algorithms. Forward selection methods are popular fast subset selection approaches. However, they may only produce suboptimal models and can be trapped into a local minimum. More recently, a two-stage fast recursive algorithm (TSFRA) combining forward selection and backward model refinement has been proposed to improve the compactness and generalization performance of the model. This paper proposes unified two-stage orthogonal least squares methods instead of the fast recursive-based methods. In contrast to the TSFRA, this paper derives a new simplified relationship between the forward and the backward stages to avoid repetitive computations using the inherent orthogonal properties of the least squares methods. Furthermore, a new term exchanging scheme for backward model refinement is introduced to reduce computational demand. Finally, given the error reduction ratio criterion, effective and efficient forward and backward subset selection procedures are proposed. Extensive examples are presented to demonstrate the improved model compactness constructed by the proposed technique in comparison with some popular methods.
Details
- Title: Subtitle
- Two-Stage Orthogonal Least Squares Methods for Neural Network Construction
- Creators
- Long Zhang - Queen's University BelfastKang Li - Queen's University BelfastEr-Wei Bai - University of IowaGeorge W Irwin - Queen's University Belfast
- Resource Type
- Journal article
- Publication Details
- IEEE transaction on neural networks and learning systems, Vol.26(8), pp.1608-1621
- Publisher
- IEEE
- DOI
- 10.1109/TNNLS.2014.2346399
- PMID
- 25222956
- ISSN
- 2162-237X
- eISSN
- 2162-2388
- Grant note
- EP/G042594/1; EP/L001063/1 / U.K. Research Councils Chinese Scholarship Council 11ZR1413100 / Science and Technology Commission of Shanghai Municipality (10.13039/501100003399) 51077022; 61271347; 61273040 / National Natural Science Foundation of China (10.13039/501100001809)
- Language
- English
- Date published
- 08/2015
- Academic Unit
- Electrical and Computer Engineering
- Record Identifier
- 9984197078902771
Metrics
13 Record Views