Machine learning is a powerful tool to learn patterns and make inferences in complex problems based on a large number of examples. It relies on models with tunable parameters, for example Neural Networks with interconnection weights between neurons tuned to perform a particular task. In supervised learning, these weights are trained using a large number of examples, which consist in pairs of input data with the desired outputs. Thanks to increasing computational power and a large amount of available data to process, they have achieved state-of-the-art performance on very diverse tasks, such as image recognition [29], Natural Language Processing, or recommender systems [30] to name a few. Today machine learning is a blooming field, and there is a number of other machine learning approaches beyond neural networks such as kernel methods and decision trees. Computational efficiency is a major research direction, as there is a strong need to scale down the heavy machine learning computations in smaller devices.

Recurrent Neural Networks are notoriously hard to train [31]. Recurrent connections are a challenge for error back- propagation, which is the method of choice to train neural networks with feed-forward connections such as Convolutional Neural Networks. As a possible solution to bypass this training issue, Echo-State Networks (ESNs) are Recurrent Neural Networks with randomly fixed internal weights. Only the output weights are trained for a particular task, reducing the training to a simple linear regression. The number of tunable parameters is thus smaller, but this does not necessarily mean that this Neural Network model is less expressive than fully-tunable Recurrent Neural Networks. It is very easy to increase the number of neurons and it has been proven that large networks can universally approximate any continuous function.