FreeIPA, Kerberos and SSSD: our identity stack on Linux servers

Where a central identity provider ends, the question "and how do the Linux servers themselves authenticate?" begins. In larger environments, a directory service, Kerberos-based authentication, SSSD on the hosts, PAM as the glue, a second factor for sensitive logins, and the sync against an enterprise directory form a stack that has to be understood as one system. This post is the anonymized report of what such a stack looks like in daily work and what we learned about ordering and testability while operating it.

The building blocks, briefly

Each block has one job. The directory service holds identities and their group memberships, and it is the source of truth the other components agree on. Kerberos provides the ticket-based authentication that makes credentials never travel to the service being used. SSSD sits on every host, mediates between system services and the directory, caches what it can so a network hiccup does not lock a machine out, and hands results to PAM, which is where the login decision is actually made. A second-factor solution covers the logins where possession matters, and the sync layer keeps the enterprise directory and the environment in agreement about who exists at all.

None of these pieces is new, and that is the point: the stack works because each part is boring, old, and well understood. The complexity lives in their interactions, which is where the operations lessons come from.

The lesson about ordering

The single most valuable procedure this stack taught us is sequencing: reproduce the login behavior on one test host first, then roll out. Every change, from a directory schema adjustment to a PAM stack edit, goes to one disposable host that mirrors production configuration. The test host answers three questions before anything widescale happens: does login work, does login fail correctly, and does failure look the same for a service account as for an interactive user?

"Fail correctly" is the case always underestimated. A stack this deep has several points where a partial failure can leave a host in a state where root can log in but the directory services are stuck, or opposite: nobody can log in while the machine happily keeps running its services. Both states are recoverable, but only if you know in advance which local escape hatch applies, and testing is where that knowledge is bought.

Kerberos in practice

Kerberos earns its reputation for both directions at once: powerful and picky. Time synchronization is not optional politeness but a functional requirement, DNS correctness decides whether tickets resolve, and keytab management is housekeeping that rewards a scripted routine. What we stopped doing after one incident is manual keytab edits on individual hosts; the keytab now belongs to configuration management like any other state, with rotation run by the same automation that deploys the rest.

The reward side deserves equal mention: single sign-on across admin interfaces, service-to-service authentication that does not involve shared passwords, and credential forwarding that behaves predictably across hosts. Once a team is used to it, going back to per-service passwords feels like carrying keys as loose change.

SSSD, caching and the failure modes

SSSD is the component we now treat as the system's shock absorber, and we size and test it like one. Offline caching decides whether a host keeps its login path during a network partition, and the cache invalidation behavior after an outage is the part worth a scripted test: take the directory away for an hour, watch a cached host behave, bring it back, then watch that behavior too. The failure modes are mostly known and documented; the value is having exercised them when they do not matter.

The sync layer and what it simplifies

Keeping a plain identity agreement between environments sounds mundane and removes an entire category of incidents: users who exist in one system and not another, groups whose meaning drifts, deactivations that never propagate. The sync does not need to be clever, only complete and observable. When it breaks, monitoring should say so before the second Wednesday in a row where one helpdesk ticket per missing group membership arrives.

What we would tell someone starting this

Start from the directory and treat the host-level stack as a product of it; keep the test host non-negotiable; automate keytabs; test offline behavior; and write down the escape hatch per machine class before you need it. One more line belongs on the planning sheet: budget a maintenance day per quarter for the stack itself. Certificates, caches, and the sync layer age quietly, and the day that finds the aging first is the cheapest one. The earlier post on a central identity provider covers the web-facing side of identity, and the account of running Kubernetes in the small shows the same test-first discipline in another corner of our operations. If you are planning a Linux identity stack, the support pages describe how we accompany such projects, and the contact page is the direct way to reach us.