Also on Medium.
One running maximum, one running sum, and the correction that keeps them in sync.
Read post →Before we make attention fast, let's make sure its numbers behave: the attention shapes, why memory matters, and a numerically safe softmax.
Read post →Mathematical derivation of distillation from Hinton’s paper: teacher and student logits, soft targets, and the high-temperature limit.
Read on Medium ↗Types of word embeddings, the math behind them, and why they matter, starting from co-occurrence matrices.
Read on Medium ↗Containerizing a FastAPI app with Docker and deploying it on Hugging Face Spaces.
Read on Medium ↗Lesser-known PyTorch functions worth knowing.
Read on Medium ↗