AI 棱镜站

主题

联邦学习与隐私计算参考资料 ​

联邦学习基础 ​

  • 开创论文: "Communication-Efficient Learning of Deep Networks from Decentralized Data" (McMahan et al., 2017)
  • FedAvg: 联邦平均算法 — 客户端本地训练 + 服务器聚合
  • FedSGD: 联邦随机梯度下降

算法演进 ​

  • FedProx: "Federated Optimization in Heterogeneous Networks" (Li et al., 2020) — 近端项约束
  • FedNova: "Tackling the Objective Inconsistency" (Wang et al., 2020) — 归一化平均
  • SCAFFOLD: "Scaffolding" (Karimireddy et al., 2020) — 方差缩减
  • FedDyn: "Federated Learning with Dynamical Regularization" (Acar et al., 2021)
  • FedMA: "Federated Learning with Matched Averaging" (Yurochkin et al., 2020)
  • FedPer/FedRep: 个性化联邦学习 — 表示+头部解耦

差分隐私 ​

  • DP-SGD: "Learning with Differential Privacy" (Abadi et al., 2016)
    • 梯度裁剪 + 噪声添加
    • (ε, δ)-差分隐私保证
  • RDP (Rényi Differential Privacy): 更紧的隐私界
  • zCDP: 零集中差分隐私
  • Privacy Amplification by Subsampling: 采样放大
  • Moments Accountant: 矩会计方法
  • Opacus: Facebook 差分隐私库 — https://github.com/pytorch/opacus

安全聚合 ​

  • SecAgg: Bonawitz et al., 2017 — 服务器不可见客户端更新
  • SecAgg+: 改进版, 更好的容错
  • FLUTE: Microsoft 联邦学习平台

通信优化 ​

  • 梯度压缩: Top-k, Random-k, 有损压缩
  • 量化: QSGD, ATOM, 1-bit SGD
  • 稀疏化: Gradient Dropping, Deep Gradient Compression
  • 知识蒸馏: 联邦蒸馏 (FD), 联邦自蒸馏 (C2FD)

嵌套学习 (Nested Learning) ​

  • 概念: 多层学习架构, 外层选择内层模型/策略
  • Hyperparameter Optimization: 联邦超参优化
  • Meta-Learning: 联邦元学习 (MAML 变体)
  • Neural Architecture Search: 联邦 NAS
  • Curriculum Learning: 课程学习嵌入联邦框架

框架 ​

垂直联邦学习 ​

  • Split Learning: 模型切分, 中间激活交换
  • VFL (Vertical Federated Learning): 特征维度切分
  • SecureBoost: 垂直联邦树模型
  • FedBCD: 块坐标下降垂直联邦

关键论文 ​

  • "Advances and Open Problems in Federated Learning" (Li et al., 2020)
  • "Differentially Private Federated Learning: A Client-Level Perspective" (Zhu et al., 2021)
  • "Federated Learning on Non-IID Data Silos: An Experimental Study" (Li et al., 2022)
最近更新