Federated Averaging (FedAvg), introduced by McMahan et al. in 2017, is the algorithm that made federated learning practical. Its idea is disarmingly simple: instead of having clients send gradients after every single step (which would require constant communication), let each client run several local training steps and send the resulting model. The server then averages the client models, weighted by how much data each client used.
Formally, if client k holds n_k training examples and produces model w_k after local training, the server computes the new global model as w = sum_k (n_k / n) * w_k, where n is the total number of examples across participating clients. Clients with more data pull the average more strongly.
Why does this matter so much? Communication is the bottleneck in cross-device FL — uploading from a phone is slow and expensive. FedAvg reduces communication by one to two orders of magnitude compared to naive federated SGD, because many local updates are compressed into a single model upload per round.
FedAvg is also the baseline of the field: essentially every published FL method is evaluated against it. Understanding exactly when and why it fails — which we will do on Day 4 — is the entry ticket to modern FL research.