Optimization of extreme learning machine parameters with metaheuristic algorithms

Abstract

Feedforward neural networks are a very popular and frequently used network architecture in many predictive applications. In these architectures, the information flow between neurons is from the input layer to the output layer. Extreme Learning Machine (ELM) is a feed forward neural network algorithm with a single hidden layer. Unlike conventional feedforward neural networks, the input weights (w) and weight bias (b) are randomly determined in ELM, and the weights of the neurons in the output layer are calculated analytically using the generalized matrix inversion method called Moore-Penrose. Instead of iteratively adjusting neural network parameters with gradient-based algorithms, it was observed that the learning speed and generalization performance increased with analytical computation in ELM. Although random selection of input weight and hidden bias parameters in ELM enables the learning process to happen very quickly, generalization performance does not give a good result in some cases. In addition, ELM may require more neurons in the hidden layer compared to gradient-based algorithms. For these reasons, there are studies to determine the optimum values of input weights and hidden bias values. In this study, meta-heuristic optimization algorithms were used to determine the optimum values of the ELM parameters. The output weights of ELM are calculated analytically by using the determined optimum input weights and hidden bias values. In addition to swarm intelligence algorithms such as Particle Swarm Optimization (PSO), Competitive Swarm Optimizer (CSO), which were previously applied in the optimization of ELM, a new ELM optimization model was developed with Harris’ Hawks Optimization (HHO). These hybrid models developed for the optimization of ELM were tested on open access data sets and the results obtained were compared with each other.