Coding for MLintermediatemust-know5 min

Implement Linear Regression from Scratch

Building a complete Linear Regression model from scratch using NumPy vector operations and Gradient Descent.

Implement Linear Regression from Scratch demonstrates foundational machine learning coding fundamentals. The model fits a linear equation predicting continuous outputs by minimizing Mean Squared Error loss. The implementation covers weight and bias vector initialization, forward pass matrix multiplication, analytical prediction error calculation, and gradient descent backpropagation updates.

Read the deep dive →
Coding for MLadvancedmust-know5 min

Implement Self-Attention from Scratch

Building Scaled Dot Product Self Attention from scratch using Query, Key, and Value linear projections in PyTorch.

Implement Self-Attention from Scratch builds the core mechanism of Transformer models. Scaled Dot-Product Attention transforms input sequence vectors into Query, Key, and Value matrices. The algorithm calculates query-key similarity dot products, scales by the square root of key dimension, applies Softmax normalization, and computes weighted sums over Value vectors.

Read the deep dive →
Coding for MLbeginner5 min

NumPy Broadcasting & Vectorization

Writing high performance vectorized array operations using NumPy shape broadcasting rules.

NumPy Broadcasting and Vectorization enables fast array operations without explicit C or Python loops. Broadcasting defines rules for operating on arrays of different shapes by implicitly stretching smaller dimensions. Vectorized operations execute low level BLAS linear algebra instructions, achieving 100x speedup over Python loops.

Read the deep dive →
Coding for MLbeginner5 min

Pandas: GroupBy, Merge, Window

Mastering Pandas GroupBy, Merge, and Rolling Window operations for tabular feature engineering.

Pandas GroupBy, Merge, and Rolling Window functions form the primary toolkit for tabular feature engineering. GroupBy aggregates records across categorical partitions, Merge combines DataFrames using relational database join logic, and Rolling Window functions compute temporal moving averages. Writing clean Pandas operations enables rapid feature extraction for machine learning models.

Read the deep dive →
Coding for MLintermediate5 min

Implement Logistic Regression from Scratch

Building a complete binary classification Logistic Regression model using Sigmoid activation and Binary Cross Entropy loss.

Implement Logistic Regression from Scratch covers binary classification coding fundamentals. The model transforms linear combination logits into probability values between 0 and 1 using the Sigmoid activation function. The implementation calculates Binary Cross Entropy loss gradients and updates weights via gradient descent.

Read the deep dive →
Coding for MLintermediate5 min

Implement k-Means from Scratch

Building the unsupervised k-Means clustering algorithm from scratch using centroid initialization, distance assignment, and mean updates.

Implement k-Means from Scratch builds the classic unsupervised clustering algorithm using NumPy. The algorithm partitions unlabeled data into K distinct clusters by alternating between assigning points to the nearest centroid and recomputing centroids as class means until convergence. The implementation covers centroid initialization strategies, Euclidean distance matrix calculation, cluster assignment, and centroid mean updates.

Read the deep dive →
Coding for MLintermediate5 min

Implement k-NN Efficiently

Building an efficient k Nearest Neighbors classification and regression algorithm using vectorized distance matrix computation.

Implement k-NN Efficiently demonstrates lazy non-parametric model coding. k-Nearest Neighbors performs no explicit training phase, storing training data and computing pairwise Euclidean distances to query samples during inference. The efficient implementation uses matrix expansion tricks to compute pairwise distances in parallel, identifying top K nearest neighbor labels using fast partitioning.

Read the deep dive →
Coding for MLintermediate5 min

Implement Softmax + Cross-Entropy Stably

Building a numerically stable Softmax and Categorical Cross Entropy loss module using log sum exp stabilization.

Implement Softmax and Cross-Entropy Stably prevents numerical overflow and underflow in multi-class classification. Raw Softmax exponentiation of large logits produces floating point NaN errors. The stable implementation subtracts maximum logit values before exponentiation and combines Softmax with Cross Entropy loss into a unified Log-Sum-Exp derivative form.

Read the deep dive →
Coding for MLintermediate5 min

Implement Vector Similarity Search

Building exact vector nearest neighbor search from scratch using Cosine Similarity, L2 normalization, and matrix dot products.

Implement Vector Similarity Search builds exact dense vector retrieval engines in NumPy. Vector search ranks candidate document embeddings against query vectors using Cosine Similarity or Dot Product distance. The implementation normalizes vectors to unit length, performs parallel matrix multiplication, and extracts top K nearest neighbor indices.

Read the deep dive →
Coding for MLintermediate5 min

Implement ROC-AUC from Scratch

Building the Area Under Receiver Operating Characteristic Curve algorithm from scratch using rank ordering pairs.

Implement ROC-AUC from Scratch covers non-parametric threshold evaluation metrics. ROC-AUC measures a classifier's ability to rank positive instances higher than negative instances across all possible decision probability thresholds. The efficient implementation sorts prediction probability scores, calculates True Positive and False Positive rates across unique threshold cutoffs, and integrates trapezoidal curve areas.

Read the deep dive →
Coding for MLintermediate5 min

Implement Non-Max Suppression

Building object detection post processing Non Max Suppression from scratch using Intersection over Union box overlap filtering.

Implement Non-Max Suppression (NMS) builds the essential post-processing step for computer vision object detectors. Object detection models (like YOLO) output hundreds of overlapping candidate bounding boxes for a single object. Non-Max Suppression filters redundant detection boxes by sorting candidates by confidence score and discarding overlapping boxes that cross Intersection over Union (IoU) thresholds.

Read the deep dive →
Coding for MLintermediate5 min

Views, Copies & Memory in NumPy

Understanding contiguous array memory layouts, stride tricks, array views, and explicit copies to avoid silent data mutation bugs.

Views, Copies, and Memory Layout in NumPy govern how array data sits in system RAM memory. Array Views share underlying data memory buffers with original parent arrays, providing fast slicing without memory duplication. Modifying a View mutates original array values silently. Engineers use explicit copies, memory contiguous checks, and C vs Fortran order layouts to write high performance, bug free machine learning code.

Read the deep dive →
Coding for MLintermediate5 min

The PyTorch Training Loop

Writing the fundamental PyTorch model training loop: zero_grad, forward pass, loss calculation, backward, and step.

The PyTorch Training Loop represents the core execution pattern for training neural networks. Unlike high level abstractions, PyTorch training requires explicitly managing data batch iteration, zeroing gradient buffers, executing model forward passes, calculating loss, invoking backward autograd, and stepping optimizer parameters.

Read the deep dive →
Coding for MLintermediate5 min

Dataset & DataLoader Patterns

Building custom PyTorch Dataset classes and configuring multi process DataLoader batching pipelines.

PyTorch Dataset and DataLoader abstractions separate data storage logic from mini-batch iteration pipelines. The Dataset class defines how individual data samples and targets are loaded and transformed. The DataLoader wraps Datasets to handle multi-threaded parallel data loading, automatic mini-batch collating, data shuffling, and GPU memory pinning.

Read the deep dive →
Coding for MLintermediate5 min

Making Pandas Fast

Optimizing slow Pandas feature processing pipelines using vectorization, categorical datatypes, memory reduction, and Polars.

Making Pandas Fast optimizes data processing speed and RAM utilization for large datasets. Unoptimized Pandas pipelines suffer from slow Python row iteration and inefficient object datatypes. Engineers speed up Pandas operations by eliminating iterrows loops in favor of vectorization, downcasting numeric types, converting text to categorical types, and adopting modern multi-threaded engines like Polars.

Read the deep dive →
Coding for MLintermediate5 min

Complexity Questions in ML Coding

Analyzing Big-O time and space complexity for machine learning algorithms, matrix operations, and data structures.

Complexity Questions in ML Coding evaluates your ability to analyze Big-O time and space complexity for machine learning algorithms. Interviewers expect candidates to state time and space bounds for matrix operations, tree algorithms, nearest neighbor lookups, and attention mechanisms. Stating exact complexity bounds demonstrates engineering rigor when designing scalable systems.

Read the deep dive →
Coding for MLintermediate5 min

Your Loss Is NaN. Now What?

Systematically diagnosing and fixing exploding gradients, zero division, and numerical instability when model loss becomes NaN.

Debugging NaN Loss provides a systematic troubleshooting guide for training neural networks. A training loss returning NaN (Not a Number) indicates numerical instability in mathematical operations. Engineers fix NaN losses by detecting exploding gradients with gradient clipping, adding epsilon constants inside logarithms, checking input features for un-normalized nulls, and lowering high learning rates.

Read the deep dive →
Coding for MLintermediate5 min

Writing ML Code Someone Else Can Run

Writing modular, reproducible machine learning code bases with configuration management, clean functions, and unit test coverage.

Writing Testable ML Code ensures that machine learning code bases can be reproduced, maintained, and audited by teammates. Jupyter notebooks are great for quick exploratory analysis but suffer from out of order cell execution and global state pollution. Production ML code modularizes pipelines into pure functions, manages parameters using YAML configuration files, sets deterministic random seeds, and writes unit tests.

Read the deep dive →
Coding for MLadvanced5 min

Implement Multi-Head Attention

Building Multi Head Attention from scratch by splitting hidden dimension representations into parallel attention sub spaces.

Implement Multi-Head Attention builds the full attention module used in Transformer architectures. Instead of computing a single attention pass, Multi-Head Attention projects Query, Key, and Value vectors into H parallel sub spaces. This enables the model to jointly attend to information from different representation sub spaces and positions simultaneously, concatenating head outputs through a final linear projection.

Read the deep dive →
Coding for MLadvanced5 min

Implement Backprop for an MLP

Building full forward and backward pass backpropagation from scratch for a Multi Layer Perceptron using NumPy.

Implement Backprop for an MLP demonstrates how neural networks learn parameters using automatic chain rule differentiation. The implementation covers dense layer matrix multiplication, non-linear activation forward passes (ReLU/Sigmoid), loss calculation, and backward pass gradient propagation through chain rule matrix calculus.

Read the deep dive →
Coding for MLadvanced5 min

Implement BatchNorm Forward & Backward

Building Batch Normalization forward and backward passes from scratch using batch statistics and learnable gamma and beta scale parameters.

Implement BatchNorm Forward and Backward builds the landmark deep learning normalization layer introduced by Ioffe and Szegedy. Batch Normalization standardizes intermediate layer activations across mini-batches to zero mean and unit variance, applying learnable gamma scale and beta shift parameters. The implementation covers forward batch normalization, tracking exponential moving averages for inference, and calculating analytical backpropagation gradients.

Read the deep dive →
Coding for MLadvanced5 min

Implement Conv2D from Scratch

Building a 2D Spatial Convolution layer from scratch using sliding windows and im2col matrix multiplication.

Implement Conv2D from Scratch demonstrates the foundational spatial feature extraction layer of Computer Vision. A 2D Convolution layer slides learnable filter kernels across spatial image dimensions, computing element wise dot products to extract local visual patterns like edges, textures, and shapes. The efficient implementation transforms 2D spatial image patches into matrix columns using im2col for fast parallel GPU matrix multiplication.

Read the deep dive →
Coding for MLadvanced5 min

Implement a Decision Tree Split

Building a Decision Tree splitting algorithm from scratch using Gini Impurity and Information Gain.

Implement a Decision Tree Split demonstrates greedy recursive tree partitioning. A Decision Tree finds optimal feature thresholds by searching across all features to maximize Information Gain or reduce Gini Impurity. The implementation covers Gini Impurity calculation, best split threshold search over continuous features, recursive binary node splitting, and tree stopping criteria.

Read the deep dive →
Coding for MLadvanced5 min

Implement a Toy BPE Tokenizer

Building Byte Pair Encoding (BPE) tokenization from scratch by iteratively merging frequent character pairs.

Implement a Toy BPE Tokenizer builds subword tokenization algorithms used by GPT models. Byte Pair Encoding starts by representing text as individual characters. The algorithm iteratively counts adjacent character pairs across a text corpus, merging the single most frequent pair into a new combined subword token until target vocabulary size is achieved.

Read the deep dive →
Coding for MLadvanced5 min

Autograd Gotchas: detach, no_grad, retain_graph

Navigating common PyTorch autograd graph memory leaks, detach operations, no_grad contexts, and retain_graph errors.

Autograd Gotchas in PyTorch cause subtle GPU memory leaks and computational graph crashes. PyTorch dynamically constructs computational graphs during forward passes to compute automatic backpropagation gradients. Engineers use tensor.detach(), torch.no_grad(), retain_graph=True, and item() conversions correctly to avoid out of memory exceptions and graph execution bugs.

Read the deep dive →
SCROLL · SAVE · TAP TO GO DEEPER