Nesterov’s Accelerated Gradient Descent (AGD) method is a variant of gradient descent that incorporates acceleration- and momentum-based concepts to overcome the limitations found in its classical counterpart. In this tutorial paper, we will provide an intution introduction to the classical GD and momentum methods, see how they are incorporated into AGD, and compare how they perform against AGD on some toy problems. This is a tutorial paper that I wrote in STAT3007 in 2024 Semester 1.

• Tutorial paper