Newton's Method vs Gradient Descent

Reported in interviews at Akuna Capital

Problem statement

Compare Newton's method and gradient descent for optimization (or root-finding): what are the advantages and disadvantages of each?

Follow-up: near a simple root, how fast does Newton's method converge, and why?

Show hint

What local model of the function does the Newton update rely on?

Relevant roles: ML Engineer, Deep Learning Engineer, Research Scientist and Applied Scientist.

Related free problems