Neural Networks and Language Models
Overview
In this part, we outline neural network -related work leading to contemporary language models, starting with introducing neural networks and forming machine-understandable representations of data and text, followed by problems related to using neural networks with sequential data. Finally, the part introduces self-attention and transformers, which are the basis for many of the current large language models.
The chapters of this part are as follows.
- Neural Networks introduces neural networks and discusses how they can be used to form machine-understandable representations of data and text.
- Embeddings and Word Embeddings introduces embeddings and word embeddings, which are used to represent words in a machine-understandable way.
- Sequential Data and Neural Networks discusses the problems related to using neural networks with sequential data.
- Self-Attention and Transformers introduces self-attention and transformers, which are the basis for many of the current large language models.