The Kalman Filter: Two Bad Guesses, One Good Estimate

Picture a drone somewhere over a field, and two ways of knowing where it is. The first is a physics model: you knew its velocity a moment ago, so you extrapolate — a fraction of a second later, it should be roughly where velocity times time says it should be. The catch is that this never accounts for the gust that just nudged it, the motor that's spinning a hair slower than commanded, the hundred small forces no model captures. Left uncorrected, that guess drifts, and drift compounds. Ten seconds of pure extrapolation and the model can be meters away from reality and have no idea.

The second is GPS. It has no memory and doesn't care about your model — it just reports a position, fresh, every tick. The catch: consumer GPS is noisy, typically off by a few meters in any direction, and it doesn't get less noisy the longer you wait. Trust it blindly and you get a jittery, zigzagging trace that never lies twice in the same direction but sometimes lies by a lot.

Neither is usable alone. The model is smooth but wanders off, unmoored from the truth. The sensor is anchored to the truth but jumpy, occasionally badly wrong. The idea that fixes this runs underneath almost everything that estimates a moving quantity from imperfect information: blend the two, weighted by how much you trust each one *right now*. Get the blend right and the result beats both inputs individually — smoother than the sensor, tighter to reality than the model ever manages alone. By the end of this you'll know the two equations that do that blending — one that runs every time new information should shift what you believe, and one that runs every time it shouldn't yet — the same pair that guided Apollo to the Moon and now runs, silently, in your pocket.

Start smaller than a moving target: fuse two noisy measurements of one static quantity. Two rangefinders both aim at the same wall. One reads $x_1 = 4.8$ m with known noise $\sigma_1 = 0.5$ m; the other reads $x_2 = 5.4$ m with noise $\sigma_2 = 1.0$ m. Both instruments are unbiased — average infinitely many readings from either one and you'd land on the true distance — they just differ in how much any single reading wobbles around that truth. What's the best single estimate of the wall's true distance, given both readings at once?

Averaging them straight — $(4.8+5.4)/2 = 5.1$ — throws information away: it treats the noisier gauge as equally credible as the precise one. The better answer weights each reading by how much you trust it, and "how much you trust it" is exactly what the variance measures. This weighting isn't a heuristic — it's provably the estimate with the smallest possible variance among every way of combining the two readings:

Notice the swap: $x_1$'s weight uses $\sigma_2^2$, not $\sigma_1^2$. The precise instrument (small $\sigma_1$) earns a *big* weight because it's the *other* sensor's variance sitting in its numerator — the formula rewards confidence by inverting it. Plug in the numbers and the fused estimate lands around 4.92 m: pulled toward the more trustworthy rangefinder, but not fully snapped to it, because the noisier one still carries some information. Make one sensor infinitely noisy ($\sigma_2 \to \infty$) and its weight vanishes entirely — the formula degrades gracefully to "just trust the good one," exactly as it should.

There's a cleaner way to see the same result, and it's the one the Kalman filter actually uses internally. Instead of variance ($\sigma^2$), think in precision ($1/\sigma^2$) — how sure you are, where bigger means more confident. Fusing two independent estimates simply adds their precisions:

This is the exact shape of resistors combining in parallel — total conductance sums, and the combined resistance is always lower than either resistor alone. Same rule here: combined *precision* is always higher than either input's precision, which means the fused variance $\sigma^2$ is always smaller than both $\sigma_1^2$ and $\sigma_2^2$. Two okay estimates, correctly combined, produce a genuinely better one — not an average of the uncertainty, a reduction of it, no matter how mediocre either sensor is on its own.

That single idea — weigh by confidence, and confidence only grows when you combine sources — is the entire content of the Kalman filter. The only thing left to add is time: instead of fusing two static readings once, fuse a moving model's prediction against a fresh sensor reading, every tick, forever. Everything from here is that same idea running on a clock.

The static-fusion idea repeats every tick of the clock, but now the thing being estimated is moving — position and velocity both — tracked through a matrix version of exactly the same math. Two steps, back to back, forever, and the whole filter is just those two steps on a loop.

Predict. Before any new measurement arrives, project the last estimate forward using the motion model alone:

$F$ *is* the model — for a constant-velocity target it just carries position forward by velocity times the time step. $P$ is the covariance, the multi-dimensional stand-in for $\sigma^2$: how uncertain the estimate is, position and velocity together. Predicting always adds uncertainty (the $+Q$ term) — extrapolating without new evidence can only ever make you less sure, never more.

Update. When a measurement $z_k$ arrives, fuse it with the predicted state exactly the way the two rangefinders were fused above:

$H$ just says which parts of the state the sensor can actually see — for a GPS reading position but not velocity, $H$ picks out position and ignores the rest. $R$ is the sensor's own variance — how much you distrust the measurement. $K_k$, the Kalman gain, is the matrix version of the weight in the resistor formula: recomputed fresh every tick from $P^-_k$ and $R$, it decides how much of the *innovation* $(z_k - H\hat{x}^-_k)$ — the gap between what was predicted and what was actually measured — gets folded into the new estimate. Fold in the innovation, shrink $P$, and you're back at the top of the loop, ready to predict again. Grow, shrink, grow, shrink — that alternation is the filter's entire life cycle, and it never stops running as long as the system does.

Stare at the update equation and $K_k$ is doing all the interesting work. When $K_k$ is close to 1, nearly the whole innovation gets folded in — the filter is trusting the new measurement far more than its own prediction. When $K_k$ is close to 0, the innovation gets almost ignored — the filter trusts its model and shrugs off the new reading as probably noise. Every value in between is a genuine blend, not a switch between two extremes.

The part that surprises people the first time: nobody sets $K$. It isn't a dial you tune by hand the way you'd hand-tune a PID controller's gains. It falls straight out of $P^-_k$ and $R$ every single tick, computed from the actual uncertainties in play, not chosen. A confident model (small $P$) against a noisy sensor (large $R$) produces a small $K$ automatically. A shaky model against a precise sensor produces a large $K$ automatically. The trust dial turns itself, tick after tick, without anyone touching it — which is exactly why the filter can start out unsure and grow more decisive as its own uncertainty shrinks, all from the same unchanging equations.

What you *do* set is $Q$, the process noise — your stated humility about your own model. $Q$ says: even between measurements, I don't fully believe my own extrapolation; leave room for the world to surprise me. Set $Q$ too small and the filter grows overconfident in its model, discounts real deviations as sensor noise, and lags behind a target that's actually maneuvering. Set $Q$ to zero outright and the filter goes deaf: $P$ stops growing during predict, $K$ collapses toward zero for good, and the filter clings to its original model no matter what the sensor keeps reporting. Set $Q$ too large, and the opposite failure appears — the filter barely trusts its own model at all, tracking every sensor jitter as if it were real motion, which defeats the point of having a model in the first place. Humility about the model isn't optional, and it isn't free either — it's the knob that keeps the filter listening, tuned to match how much the real system actually wanders.

The filter's proof of concept was as demanding as it gets. The Apollo Guidance Computer ran a Kalman filter — invented less than a decade earlier by Rudolf Kálmán — to fuse an inertial platform's drifting model of position against periodic star-sighting measurements, on a computer with less memory than a modern light bulb. That debut set the pattern every later use follows: a model that drifts, a sensor that's sparse or noisy, fused into something better than either alone.

Your phone runs the same idea dozens of times a second: GPS is noisy and slow to update, so the phone fuses it with the accelerometer and gyroscope's dead-reckoning model to produce the smooth blue dot you actually see on the map. Drones and rockets fuse an inertial-measurement-unit's model against GPS or barometric altitude the same way. A robot doing SLAM — simultaneous localization and mapping — runs a Kalman filter, or one of its extended or unscented cousins, to fuse wheel-odometry drift against landmark sightings. Quant finance runs the identical state-space machinery to fuse a drifting model of some latent signal — an interest rate, a volatility level — against noisy market observations. Even a phone or laptop battery's "time remaining" estimate is usually a Kalman filter fusing a discharge model against noisy voltage readings, which is why that number gets steadier, not jumpier, the longer you watch it.