Notes / NLP & LLMs / Mixture of Experts Mixture of Experts Mixture of Experts architectures 1. Introduction to Mixture of Experts WIP Experts, routing, multi-device parallelism, and MoE architectures