Newton–Raphson Method

Equivalent forms

\Delta x = -f/f'

x_{n+1} = \tfrac12\left(x_n + \frac{a}{x_n}\right)\ (\text{for } \sqrt a)

Quadratic convergence: near the root the number of correct digits roughly doubles every iteration — a handful of steps reach machine precision.

Symbol	Name	SI unit	Dimension	Range
$x_0$	Initial guess Starting point of the iteration	1	1	0.5 – 4
$a$	Target constant Solve x² = a (find √a)	1	1	1 – 9
$x^*$	Rootoutput Converged solution √a	1	1	0 – 3
$N$	Iterationsoutput Steps to reach tolerance	1	1	0 – 10

Unit systems

Symbol	SI	CGS	Imperial
$x_0$	1	1	1

Where it holds

Converges quadratically near a simple root if f is smooth and

f' \neq 0

there. Simple roots only; multiple roots slow it to linear convergence.

Dimensional analysis

x_{n+1} (dim x)

x_n - f/f'

f has some dimension [f];

f' = df/dx

has [f]/[x], so f/f' has dimension [x], matching x_n.

\checkmark

Discovery

Newton & Raphson · 1690

Newton described the method for polynomials around 1669; Joseph Raphson simplified it into the iterative form in 1690; Thomas Simpson generalized it to calculus in 1740. It remains the default root-finder inside optimizers, calculators, and machine-learning training loops.

Try this

You need to solve f(x)=0 but there's no formula. Newton's trick: pretend the curve is its tangent line, jump to where the line hits zero, repeat. It doubles the correct digits every step.

Watch the tangent line at your guess skate down to the axis, hand you a better guess, and converge on the root in a handful of leaps.

Research status: stable

Real-world applications

Hardware square-root and division units
Solving nonlinear equations in engineering
Backbone of optimization (Newton/quasi-Newton steps)
Implicit solvers in physics simulation

Common misconceptions

Newton always converges — it's only locally convergent; a bad guess can diverge or cycle.
It needs the actual root nearby — basins of attraction can be fractal (Newton fractals).
It works the same for multiple roots — convergence drops to linear there.

Experimental verification

Every hardware/software sqrt, division, and reciprocal uses Newton iteration; the famous fast inverse-square-root and GPU math units rely on one or two Newton steps and hit the correct answer to machine precision as predicted.

Derivation

Taylor-expand:

0 = f(x*) \approx f(x_n) + f'(x_n)(x* - x_n)

.

Solve for x*:

x* \approx x_n - f(x_n)/f'(x_n)

, which defines x_{n+1}.

The neglected quadratic term makes the error obey

e_{n+1} \approx (f''/2f')e_n^{2}

— quadratic convergence.

Limiting cases

f'(x_n) = 0

⟶ step divergesA flat tangent shoots the next guess to infinity — the method fails at critical points.

good initial guess⟶ quadratic convergenceError squares each step:

0.1 \to 0.01 \to 0.0001

…

poor guess / inflection⟶ may cycle or divergeNewton is only locally convergent; global behavior can be chaotic.

What if…

What if you don't have f'(x)?

Use the secant method (finite-difference slope) — slightly slower (order $\approx 1.618)$ but derivative-free.

What if there are many variables?

f' becomes the Jacobian matrix and the step solves a linear system $J\Delta x = -f$ — multidimensional Newton.

1

Compute √2 from x_0 = 2

Given ·

a:: 2
x 0:: 2

Find · two iterations

Steps

$x_{1} = \tfrac{1}{2}(2 + 2/2) = 1.5$
$x_{2} = \tfrac{1}{2}(1.5 + 2/1.5) = 1.41667$
$x_{3} = 1.41422$ — already 5 digits

Result ·

\approx 1.41421

2

Why the step is f/f'

Given ·

f:: $x^{2}$ -a

Find · iteration

Steps

$f' = 2x$
$x - (x^{2}$ -a)/(2x)
$= \tfrac{1}{2}(x$ + a/x) — the Babylonian rule

Result ·

x_{n+1} = \tfrac{1}{2}(x_n

+ a/x_n)

bisection methodTaylor Expansion→Mean Value Theorem→euler method numerical