Poised Solutions Tech Library

Neural Networks

Neural Networks Author: Simon Haykin Publisher: Prentice Hall Published: 1994 Pages: 696

Neural Networks

Neural Networks Chapters

Neural Networks Chapters

1. Introduction
   • What is a Neural Network?
   • Structural Levels of Organization in the Brain
   • Models of a Neuron
   • Neural Networks Viewed as Directed Graphs
   • Feedback
   • Network Architectures
   • Knowledge Representation
   • Visualizing Process in Neural Networks
   • Artificial Intelligence and Neural Networks
   • Historical Notes
2. Learning Process
   • Introduction
   • Error-Correction Learning
   • Hebbian Learning
   • Competitive Learning
   • Boltzmann Learning
   • The Credit-Assignment Problem
   • Supervised Learning
   • Reinforcement Learning
   • Unsupervised Learning
   • Learning Tasks
   • Adaption and Learning
   • Statistical Nature of the Learning Process
   • Learning Theory
   • Discussion
3. Correlation Matrix Memory
   • Introduction
   • Distributed Memory Mapping
   • Correlation Matrix Memory
   • Error Correction Applied to a Correlation Matrix Memory
   • Discussion
4. The Perception
   • Introduction
   • Basic Considerations
   • The Perception Convergence Theorem
   • Performance Measure
   • Maximum-Likelihood Gaussian Classifier
   • Discussion
5. Least-Mean-Square Algorithm
   • Introduction
   • Wiener-Hopf Equations
   • Method of Steepest Descent
   • Least-Mean-Square Algorithm
   • Convergence Considerations of the LMS Algorithm
   • Learning Curve
   • Learning Rate Annealing Schedules
   • Adaline
   • Discussion
6. Multilayer Perceptions
   • Introduction
   • Some Preliminaries
   • Derivation of the Back-Propagation Algorithm
   • Summary of the Back-Propagation Algorithm
   • Initialization
   • The XOR Problem
   • Some Hints for Making the Back-Propagation Algorithm Perform Better
   • Output Representation and Decision Rule
   • Computer Experiment
   • Generalization
   • Cross-Validation
   • Approximations of Functions
   • Back-Propagation and Differentiation
   • Virtues and Limitations of Back Propagation Learning
   • Accelerated Convergence of Back Propagation Through Learning Rate Adaption
   • Fizzy Control of Back Propagation Learning
   • Network-Pruning Techniques
   • Supervised Learning Viewed as a Nonlinear Identification Problem
   • Supervised Learning as a Function Optimization Problem
   • Supervised Learning of Probability Distributions by Multilayer Perceptions
   • Discussion
   • Applications
7. Radial-Basis Function Networks
   • Introduction
   • Cover's Theorem on the Separability of Patterns
   • Interpolation Problem
   • Supervised Learning as an Ill-Posed Hypersurface Reconstruction Problem
   • Regularization Theory
   • Regularization Networks
   • General Radial Basis Function Networks
   • The XOR Problem (Revisited)
   • Comparison of RBF Networks and Multilayer Perceptrons
   • Mixture Models
   • Learning Strategies
   • Computer Experiment
   • Factorizable Radial-Basis Functions
   • Discussion
   • Applications
8. Recurrent Networks Rooted in Statistical Physics
   • Introduction
   • Dynamical Considerations
   • The Hopfield Network
   • Computer Experiment I
   • Energy Function
   • Error Performance of the Hopfield Network
   • Isomorphism Between a Hopfield Network and a SpinGlass Mdoel
   • Stochastic Neurons
   • Phase Diagrams of the Hopfield Network and Related Properties
   • Simulated Annealing
   • The Boltzmann Machine
   • A Markov Chain Model of the Boltzmann Machine
   • The Mean-Field-Theory Machine
   • Computer Experiments II
   • Discussion
9. Self-Organizing Systems I: Hebbian Learning
   • Introduction
   • Some Intuitive Principles of Self-Organization
   • Self-Organized Feature Analysis
   • Discussion
   • Principal Components Analysis
   • A Linear Neural Model as a Maximum Eigenfilter
   • Self-Organized Principal Components Analysis
   • Adaptive Principal Components Analysis Using Lateral Inhibition
   • Tow Classes of PCA Algorithms
   • How Useful is Principal Components Analysis?
10. Self Organizing Systems II: Competitive Learning
    • Introduction
    • Computational Maps in the Cerebral Cortex
    • Two Basic Feature Mapping Models
    • Modification of Stimulus by Lateral Feedback
    • Self-Organizing Feature-Mapping Algorithm
    • Properties of the SOFM Algorithm
    • Reformulation of the Topological Neighborhood
    • Adaptive Pattern Classification
    • Learning Vector Quantization
    • Applications
    • Discussion
11. Self Organizing Systems III: Information Theoretic Models
    • Introduction
    • Shannon's Information Theory
    • The Principle of Maximum Information Preservation
    • Generation of Topologically Ordered Maps
    • Discussion
    • Spatially Coherent Features
    • Another Information Theoretic Model of the Perceptual System
    • Concluding Remarks
12. Modular Networks
    • Introduction
    • Basic Notions of Modularity
    • Associative Gaussian Mixture Model
    • Stochastic-Gradient Learning Algorithm
    • Hierarchical Structure of Adaptive Expert Networks
    • Piecewise Control Using Modular Networks
    • Summary and Discussion
13. Temporal Processing
    • Introduction
    • Spatio-Temporal Models of a Neuron
    • FIR Multilayer Perceptron
    • Temporal Back-Propagation Learning
    • Temporal Back-Propagation with Adaptive Time Delays
    • Back-Propagation Through Time
    • Real-Time Recurrent Networks
    • Real-Time Nonlinear Adaptive Prediction of Nonstationary Signals
    • Partially Recurrent Network
    • Discussion
14. Neurodynamics
    • Introduction
    • Dynamical Systems
    • Stability of Equilibrium States
    • Attractors
    • Strange Attractors and Chaos
    • Neurodynamical Models
    • Manipulation of Attractors as a Recurrent Network Paradigm
    • Dynamics of Hopfield Models
    • The Cohen-Grossberg Theorem
    • The Hopfield Model as a Content-Addressable Memory
    • Brain-State-in-a-Box Model
    • Recurrent Back-Propagation
    • Discussion
15. VLSI Implementation of Neural Networks
    • Introduction
    • Major Design Considerations
    • Categories of VLSI Implementations
    • Neurocomputing Hardware
    • Concluding Remarks

Neural Networks Appendices

1. Pseudoinverse Matrix Memory
2. A General Tool for Convergence Analysis of Stochastic Approximation Algorithms
3. Fokker-Planck Equation
4. Bibliography
5. Index