Efficient extreme learning machine via very sparse random projection

摘要

Extreme learning machine (ELM) is a kind of random projection-based neural networks, whose advantages are fast training speed?and high generalization. However, three issues can be improved in ELM: (1) the calculation of output weights takes $$Oleft( {L^{2}N} ight) $$ O L 2 N time (with N training samples and L hidden nodes), which is relatively slow to train a model for large N and L ; (2) the manual tuning of L is tedious, exhaustive and time-consuming; (3) the redundant or irrelevant information in the hidden layer may cause overfitting and may hinder high generalization. Inspired from compressive sensing theory, we propose an efficient ELM via very sparse random projection (VSRP) called VSRP-ELM for training with large N and L . The proposed VSRP-ELM adds a novel compression layer between the hidden layer and output layer, which compresses the dimension of the hidden layer from $$Nimes L$$ N × L to $$Nimes k ,(hbox {where } k