StepLR
\[lr_{epoch} = \begin{cases}
\gamma \cdot lr_{epoch-1}, & \text{if epoch \% step\_size = 0} \\
lr_{epoch-1}, & \text{otherwise}
\end{cases}\]
MultiStepLR
\[lr_{epoch} = \begin{cases}
\gamma \cdot lr_{epoch-1}, & \text{if epoch in milestones} \\
lr_{epoch-1}, & \text{otherwise}
\end{cases}\]
ExponentialLR
\[lr_{epoch} = \gamma \cdot lr_{epoch-1}\]