AI 棱镜站

主题

前沿论文参考资料 ​

Anthropic Mythos 架构 ​

  • 来源: Anthropic 技术博客与研究论文
  • 核心概念:
    • Constitutional AI (CAI): 自我改进对齐
    • 机制可解释性 (Mechanistic Interpretability): 因果追踪, 特征超位置
    • 涌现结构: 规模涌现能力
    • 模型权重分析: SVD, 特征几何
  • 关键论文:
    • "Constitutional AI: Harmlessness from AI Feedback" (Bai et al., 2022)
    • "Towards Monosemanticity: Decomposing Language Models With Dictionary Learning" (Cunningham et al., 2023)
    • "Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet" (Templeton et al., 2024)

GPT-5.6-sol 架构 ​

  • 来源: OpenAI 技术报告与社区分析
  • 核心概念:
    • 推理能力: Chain-of-Thought, Tree-of-Thought
    • 多模态融合: 原生视觉+音频+文本
    • 智能体能力: 工具使用, 自主规划
    • 规模定律: Chinchilla 最优, 计算最优分配
  • 关键论文:
    • "GPT-4 Technical Report" (OpenAI, 2023)
    • "Scaling Laws for Neural Language Models" (Kaplan et al., 2020)
    • "Let's Verify Step by Step" (Lightman et al., 2023)

架构对比分析 ​

  • Mythos vs GPT-5.6-sol 核心差异:
    • 对齐方法: CAI vs RLHF/DPO
    • 可解释性: 机制可解释性 vs 黑盒评估
    • 安全哲学: 主动安全 vs 能力对齐
    • 规模策略: 质量优先 vs 规模优先
    • 多模态: 后融合 vs 原生融合

何恺明多模态研究 ​

  • Masked Autoencoders (MAE): "Masked Autoencoders Are Scalable Vision Learners" (He et al., 2022)
  • MAE for Video: 视频掩码自编码器
  • Multi-Modal MAE: 多模态掩码预训练
  • 最新方向: 自监督多模态学习, 掩码策略创新
  • 关键论文:
    • "Masked Autoencoders Are Scalable Vision Learners" (He et al., CVPR 2022)
    • "Empirical Study of Optimizer and Batch Size in Self-Supervised Learning" (He et al., 2023)

Google 多模态研究 ​

  • Gemini 系列: "Gemini: A Family of Highly Capable Multimodal Models" (Google, 2024)
  • Gemini 1.5 Pro: 百万级上下文, MoE 架构
  • Gemini 2.0: 原生多模态, 智能体能力
  • Veo: 视频生成模型
  • Imagen 3: 图像生成模型
  • 关键论文:
    • "Gemini: A Family of Highly Capable Multimodal Models" (Google DeepMind, 2024)
    • "Gemini 1.5: Scaling to Millions of Tokens" (Google DeepMind, 2024)

规模定律 ​

  • Chinchilla: "Training Compute-Optimal Large Language Models" (Hoffmann et al., 2022)
  • Scaling Laws: Kaplan et al., 2020
  • Emergent Abilities: Wei et al., 2022
  • Inverse Scaling: McKenzie et al., 2023

其他前沿 ​

  • State Space Models (SSM): Mamba (Gu & Dao, 2023), S4 (Gu et al., 2022)
  • RetNet: "Retentive Network: A Successor to Transformer" (Sun et al., 2023)
  • RWKV: 线性 RNN + 注意力混合
  • Diffusion Models: EDM, Consistency Models, Flow Matching
  • AI Safety: 机制可解释性, 对齐税, 评估基准

重要会议与期刊 ​

  • NeurIPS: Neural Information Processing Systems
  • ICML: International Conference on Machine Learning
  • ICLR: International Conference on Learning Representations
  • CVPR: Computer Vision and Pattern Recognition
  • ACL: Association for Computational Linguistics
  • arXiv: 预印本服务器 — https://arxiv.org/
最近更新