Federated Learning

Training machine learning models collaboratively across millions of edge devices without centralizing raw user data.

🔴 advanced5 min readprivacy

Federated Learning enables decentralized model training directly on edge devices. Traditional machine learning uploads raw user data to central cloud servers for training, creating privacy risks. Federated Learning sends global model weights to client devices (smartphones, laptops), trains local models on private device data, and aggregates model weight updates back to a central server using Federated Averaging.

What is Federated Learning?

Traditional machine learning requires centralizing raw user data:

$$\text{User Mobile Devices} \xrightarrow{\text{Upload Raw Private Data}} \text{Central Cloud GPU Cluster} \to \text{Train Model}$$

This centralized architecture creates severe privacy risks for sensitive data (smartphone typing histories, photo libraries, hospital medical records).

Federated Learning (McMahan et al., Google, 2017) reverses this workflow:

$$\text{Bring the Model to the Data, NOT the Data to the Model!}$$

                          [ CENTRAL AGGREGATION SERVER ]
                                 ▲              │
                 Send Weight     │              │ Send Global Model
                 Updates Only!   │              ▼ Weights
                        ┌────────┴──────────────┴────────┐
                        │                                │
             [ Client 1 (Phone) ]              [ Client 2 (Phone) ]
             (Trains on Local Data)            (Trains on Local Data)

The 4 Steps of the Federated Learning Loop

┌─────────────────────────────────────────────────────────────┐
│ 1. DOWNLOAD:   Client devices download global model weights.│
│ 2. LOCAL TRAIN:Clients train local models on private data.  │
│ 3. UPLOAD:     Clients send encrypted weight updates back.  │
│ 4. AGGREGATE:  Server averages updates via FedAvg algorithm.│
└─────────────────────────────────────────────────────────────┘

Federated Averaging (FedAvg)

The central server computes a weighted average of model parameter updates $\theta_k$ received from $K$ selected client devices:

$$\theta_{\text{global}} = \sum_{k=1}^K \frac{n_k}{N} \theta_k$$

Where $n_k$ is the number of local training samples on client $k$, and $N$ is the total samples across all selected clients.

Key Engineering Challenges

Non-IID Data Distributions: User data is not Independently and Identically Distributed. A user in Tokyo types completely different words than a user in London.
Communication Bottlenecks: Mobile wireless networks are slow and unreliable. Compress weight updates using gradient quantization and sparse updates.
System Heterogeneity: Client devices range from high end smartphones to old budget phones with weak battery and RAM capacity. Train only when devices are idle, plugged in, and connected to unmetered Wi-Fi.

Say this out loud

Federated Learning trains machine learning models across decentralized edge devices without centralizing raw user data. Global model weights are sent to client devices, local models train on private data, and encrypted parameter updates are aggregated using Federated Averaging. This protects user privacy while leveraging edge compute.

Followups to expect

How do you protect weight updates against malicious reverse-engineering? Combine Federated Learning with Secure Aggregation (cryptographic secret sharing) and Differential Privacy to ensure central servers cannot reconstruct raw text from weight updates.
What is Personalised Federated Learning? Adapting global federated models with additional local fine-tuning layers on individual client devices to personalize predictions per user.

Check yourself

Question 1 of 3

What primary data privacy advantage does Federated Learning offer over traditional cloud training?

More in Responsible AI & Behavioural

See all →

Telling Your ML Project Story5 min Bias & Fairness in ML5 min Explainability: SHAP & LIME4 min