The Full FedAvg Algorithm

Intuition

Repeat FedAvg rounds with a random subset of clients each time. Watch the global loss fall, using far fewer communication rounds than one-gradient-step-per-round SGD would need.

Precise explanation

w = w0
for t in 0..R-1:
    S_t = sample_clients(K, C, seed, t)                 # FL-08
    for k in S_t: w_k = local_sgd(w, data_k, lr, E, B, seed*1000 + t*K + k)   # FL-06
    w = Σ_{k∈S_t} n_k w_k / Σ_{k∈S_t} n_k               # FL-02 over participants only
    record global_loss(w)

Global loss = size-weighted mean of client MSEs = MSE on the pooled data. On IID data FedAvg converges quickly; on label/feature-skewed data local models drift apart and progress slows or oscillates (McMahan et al.; FL survey), motivating FedProx (FL-14).

FedAvg loss curves IIDnon-IID round

Figure (original, schematic): typical global loss curves.

Worked example

3 clients holding noise-free data from y = 2x₀ − x₁ converge to w ≈ [2, −1].

Common mistakes

  • Averaging over all clients instead of the sampled ones.
  • Evaluating loss only on participants.
  • Reusing one seed for every client and round.

Why this matters

This is the end-to-end algorithm deployed in production FL systems (with many engineering additions).

References