Writing
Notes from the bench
Explainers with working code: retrieval, transformers, embeddings, and the PyTorch underneath.
10 posts · 7 topics · 2024
Start here
- Topic Modeling for Research ArticlesMulti-Label Classification Using Hugging Face TransformerMulti-label topic classification of research articles with a fine-tuned BERT-base model: data exploration, the evaluation metric, and learning curves. · 5 min read
- Retrieval-Augmented GenerationRetrieval-Augmented Generation (RAG)How retrieval-augmented generation works: keyword search, vector search, and hybrid search, then a basic RAG implementation walked through in code. · 7 min read
All posts
2024, 10 posts
- Multi-Label Classification Using Hugging Face TransformerMulti-label topic classification of research articles with a fine-tuned BERT-base model: data exploration, the evaluation metric, and learning curves.Topic Modeling for Research Articles · May 28 · 5 minTopic Modeling for Research Articles
- Multi-Label Classification Using PyTorch TransformerEncoderLayerThe same multi-label research-article task, this time with a transformer encoder built from PyTorch's TransformerEncoderLayer instead of pretrained BERT.Topic Modeling for Research Articles · May 28 · 2 minTopic Modeling for Research Articles
- Search NeurIPS-2022 Similar PapersFinding similar NeurIPS 2022 papers with Sentence-BERT embeddings and an Annoy index, deployed as a Streamlit app.Document Retrieval · May 28 · 1 minDocument Retrieval
- PyTorch Functions You Should Know Before Building TransformerThe PyTorch tensor operations you need before implementing a transformer from scratch, each with a short example.Natural Language Processing · May 28 · 3 minNatural Language Processing
- Concepts You Should Know Before Getting into TransformerThe building blocks of a transformer, one at a time: input embeddings, positional encoding, scaled dot-product attention, residual connections, masking, and softmax.Natural Language Processing · May 27 · 3 minNatural Language Processing
- Convolution for NLP TaskHow a 1D convolution runs over word embeddings, computed by hand and checked against PyTorch's nn.Conv1d, then a 1D-CNN sentiment classifier on IMDb reviews.Natural Language Processing · May 27 · 4 minNatural Language Processing
- Understanding and Implementing a Recurrent Neural NetworkWhat a recurrent neural network computes at each step, and how to implement one in PyTorch.Recurrent Neural Network · May 27 · 2 minRecurrent Neural Network
- Sentiment Classification and Word EmbeddingsSentiment classification on IMDB movie reviews: data exploration, word embeddings, an LSTM classifier, and t-SNE views of what the embeddings learn.Sentiment Classification · May 27 · 4 minSentiment Classification
- Understanding Linear Regression in Supervised LearningLinear regression from first principles: the model, the cost function, and gradient descent, with an interactive plot and a from-scratch notebook.Linear Regression · May 24 · 6 minLinear Regression
- Retrieval-Augmented Generation (RAG)How retrieval-augmented generation works: keyword search, vector search, and hybrid search, then a basic RAG implementation walked through in code.Retrieval-Augmented Generation · Feb 2 · 7 minRetrieval-Augmented Generation