Pytorch Embedding Model Part 1

Today we are building a word embedding model in PyTorch from scratch, mostly from memory. The goal is simple. Learn how embeddings work and use them for semantic similarity, like finding words that mean similar things. We started a model file, set up an Embedding layer with a vocab size and an embedding size like 256, and picked GELU as an activation. Along the way we cleared up what an embedding really is. It is basically a lookup table that grabs a row from a weight matrix instead of doing a full matrix multiply, which makes it fast. Next we made a plan: get data, normalize text, build a word to index dictionary with a PAD token at 0, then vectorize each sentence into token IDs. We also talked about how training will work, using ideas like n grams or continuous bag of words, plus a loss like negative log likelihood and optimizers like SGD, Adam, and AdamW. We hit a few small bugs in text cleaning and naming, but the dictionary and vectorization steps are now in place.