Blog

Notes & write-ups

Also on Medium.

02 Nov 2026
✦
FlashAttention, Part 2: Softmax on the fly

One running maximum, one running sum, and the correction that keeps them in sync.

Read post →
01 Nov 2026
FlashAttention, Part 1: Making softmax safe

Before we make attention fast, let's make sure its numbers behave: the attention shapes, why memory matters, and a numerically safe softmax.

Read post →
12 Jul 2025
Knowledge Distillation

Mathematical derivation of distillation from Hinton’s paper: teacher and student logits, soft targets, and the high-temperature limit.

Read on Medium ↗
24 Jun 2025
Word Embeddings

Types of word embeddings, the math behind them, and why they matter, starting from co-occurrence matrices.

Read on Medium ↗
27 Nov 2023
Deploying Your FastAPI Applications on Huggingface Via Docker

Containerizing a FastAPI app with Docker and deploying it on Hugging Face Spaces.

Read on Medium ↗
17 Nov 2023
Top PyTorch functions you didn’t know about

Lesser-known PyTorch functions worth knowing.

Read on Medium ↗