Newton's Method vs Gradient Descent
Problem statement
Compare Newton's method and gradient descent for optimization (or root-finding): what are the advantages and disadvantages of each?
Follow-up: near a simple root, how fast does Newton's method converge, and why?
Show hint
What local model of the function does the Newton update rely on?
Relevant roles: ML Engineer, Deep Learning Engineer, Research Scientist and Applied Scientist.