Feature Stores

Dual Storage Architecture

                       DATA SOURCES (Kafka, Snowflake, S3)
                                       │
                                       ▼
                       [ FEATURE STORE ENGINE ]
           Centralized Definitions, Versioning & Access Control
                                       │
            ┌──────────────────────────┴──────────────────────────┐
            ▼                                                     ▼
 [ OFFLINE FEATURE STORE ]                               [ ONLINE FEATURE STORE ]
  - Storage: S3, Parquet, Snowflake, BigQuery             - Storage: Redis, DynamoDB, Cassandra
  - Latency: Minutes / Hours                              - Latency: Sub-10ms (< 5ms)
  - Access: Bulk Point-in-Time Joins (ASOF)               - Access: Single-entity Key-Value Lookups
  - Use: Model Training & Backtesting                     - Use: Real-time Online Model Inference

Core Problems Solved

1. Eliminates Training-Serving Skew: Ensures that the exact same feature transformation logic used during offline training is served at online inference time.
2. Prevents Point-in-Time Feature Leakage: Executes automated ASOF joins (feature_timestamp <= event_timestamp), ensuring training sets contain zero future information.
3. Feature Sharing & Reusability: Data scientists can discover and reuse pre-computed features (e.g., user_30d_spend) across multiple models, avoiding redundant engineering pipelines.

Feature Store Ingestion Patterns

• Batch Ingestion: Nightly ETL jobs compute rolling aggregations in Spark/SQL and load results into Offline and Online stores.
• Streaming Ingestion: Real-time event streams (Kafka/Flink) compute sliding window aggregations on-the-fly (e.g. user_click_count_5m) and write directly to the Online Store with < 1s lag.

Say this out loud

"A Feature Store centralizes ML feature management to eliminate training-serving skew and feature leakage. It uses a dual storage architecture: an Offline Store on S3/Snowflake for high-throughput historical training via point-in-time joins, and an Online Store on Redis/DynamoDB for sub-10ms real-time inference lookups."

Follow-ups to expect

• What open-source Feature Store frameworks are standard? Feast (lightweight, modular open-source) and Hopsworks (full-platform feature store with integrated model registry). Commercial solutions include Tecton and Databricks Feature Store.
• How does a Feature Store handle feature deprecation? Features are versioned (user_spend_v1 vs user_spend_v2). Deprecated features are marked with metadata warnings, maintaining backward compatibility for older deployed models while new models migrate to v2.