Notes / NLP & LLMs / Mixture of Experts Mixture of Experts Learning Mixture of Experts 1. Introduction to Mixture of Experts WIP Experts, routing, multi-device parallelism, and MoE architectures