Basic aspects of Gradient Descent and an implementation in Go

This article provides a follow-up of my series about basic nonlinear optimization techniques with sample implementations. Again, we don’t aim to cover all deep aspects, but focus on giving an overview and introduction.

This account introduces another very popular, if not the most popular, algorithm for finding minima of real-valued functions in several variables — the Gradient Descent.

Our main task can be formulated as: