When there's too much information, reducing the amount of information - simplifying the data - can reveal more. We can use unsupervised learning techniques to do this. This is done by designing a neural network that imposes a bottleneck in the network which forces a compressed knowledge representation of the original input. This is called an autoencoder and this idea is foundational to understanding how new architectures are being used in natural language processing.
If the input features were each independent of one another, this compression and subsequent reconstruction would be a very difficult task. But, if some sort of structure exists in the data (correlations between input features), this can be learned and then leveraged when forcing the input through the network's bottleneck.
It's easier to understand this concept with an example.
Let’s say we are trying to predict whether someone has the flu from a checklist of three symptoms; cough, high temperature, aching joint...
- - - - -
The rest of this post is available for our Pro Members. Please login if you're a member. And, if you're not, subscribe now to get access to all of our Pro Member content!
Email us with any questions.