Drift Is Not a Discipline Problem

Host paths fanning away from a baseline line, with three sampling points marked

A machine is built in March against a hardening baseline, checked, signed off, and put into service. In September it fails four of the checks it passed at build time. Nobody logged in and disabled anything. No change ticket mentions those settings. The usual conclusion is that somebody was sloppy, and the usual remedy is a reminder about following the standard.

Both are wrong, and the diagnosis matters more than the remedy, because drift is not a lapse in behaviour. It is the ordinary output of a machine that is being used. A baseline describes a state; a running host is a process. The gap between those two things opens on its own.

Where the divergence actually comes from

Six mechanisms account for most of it, and only one of them involves a person being careless.

Package upgrades reassert vendor files. Most packaging systems distinguish between configuration files they own and files the administrator owns. Edit the wrong one and the next upgrade restores the shipped version, silently and correctly, because that is the documented contract. The setting was not removed by anyone. It was returned to its default by a process doing exactly what it promised.

Software installation adds surface. Installing a package to solve a real problem enables a service, creates an account, opens a socket, or writes a file with permissions chosen by whoever built the package rather than by you. The change is legitimate, approved and invisible to a baseline written before that software existed.

Runtime settings were never made persistent. A parameter tuned live to resolve a problem, and never written to the file that would restore it, holds until the next reboot. This is drift by reversion: the machine has not moved away from anything, it has simply forgotten a change that only ever existed in memory. It is the hardest class to reason about, because the host is compliant right up to a restart and non-compliant afterwards with no event in between.

Emergency changes outlive the emergency. A permission is widened at two in the morning to restore a service. It works. Nobody reverts it, not from laziness but because the person who could authorise the reversion cannot prove that the outage will not return, and the asymmetry of that bet is obvious to everyone involved.

Hosts were never identical to begin with. Machines built six months apart from different images with different package versions do not drift apart; they start apart. This gets counted as drift because the audit cannot tell the difference between a host that changed and a host that was never converged.

The baseline itself moves. A new revision of a benchmark adds checks, retires others, tightens a threshold. Nothing on the machine changed at all and its score fell. Treating that as a regression on the host is a category error, and it is remarkably common in reporting.

Only the emergency change involves human choice, and even that one is a rational response to an incentive rather than a failure of rigour.

Why a compliance percentage is nearly content-free

If drift accrues continuously and audits happen periodically, the number an audit produces is a sample, not a measurement. A quarterly scan reports the state of the estate on the day it ran and says almost nothing about the other eighty-nine days. Two estates can return the same figure while behaving completely differently: one drifts slowly and is never corrected, the other drifts fast and is corrected within hours. The second is in far better shape and scores identically.

The useful quantities are rates, not levels. How quickly does a freshly built host diverge from the state it was built in? How long does a known deviation survive before it is corrected? What proportion of deviations were introduced by an approved change rather than an unapproved one? Those three questions describe a system’s dynamics. A percentage describes one photograph of it.

There is a second reason the percentage misleads. Checks are not interchangeable, and a baseline typically contains a great many low-consequence items alongside a handful that genuinely change what an intruder can do on the box. Averaging them produces a number that improves fastest when you fix the things that matter least, because those are the cheap ones. If the metric is going to be a single figure, it should be weighted by consequence, and if nobody is willing to do that weighting then the figure should not be the metric.

Enforcement moves the problem rather than removing it

The structural answer to drift is convergence: a configuration system that reapplies the desired state on a schedule, idempotently, so that any divergence has a bounded lifetime. This genuinely works, and it changes the character of the problem in ways worth stating plainly.

Drift stops being a finding and becomes a background process, which is the point. But three new failure modes arrive with it.

The first is uniform error. A mistake in the desired state now propagates to every host that subscribes to it, at machine speed, with no natural friction to slow it down. The blast radius of a bad configuration change is exactly the inverse of the estate’s manual toil.

The second is silent reversion. Someone makes a change by hand for a reason that is real but undocumented, and the next convergence run removes it. The thing they were fixing breaks again, and from their vantage point the cause is invisible. Do this two or three times and people start disabling the agent on the hosts they care about — which produces precisely the population of unmanaged, important machines that enforcement was meant to eliminate.

The third is the exception list. Every real estate contains hosts that cannot take the standard state: an application that requires a weaker setting, a box under a support contract that forbids modification, a device whose vendor image cannot be altered. Those exceptions have to live somewhere machine-readable, with an owner and a reason, or they become tribal knowledge and then they become nothing at all. The exception list is not an administrative annoyance. In most estates it is the single most honest description of where the risk actually is, because it enumerates the machines the standard does not reach.

Drift, then, is a signal about the system rather than about the people operating it. An estate showing no drift at all is telling you one of two things: that convergence is running and working, or that the machines in question are not doing anything. It is worth knowing which.