Designing Secure Authentication Systems

security software engineering
security authentication oauth oidc jwt passkeys architecture

Designing Secure Authentication Systems: Architecture, Threat Models, and Operational Lessons

Most authentication systems are secure—until they meet production reality.

They pass audits, follow OAuth/OIDC correctly, use JWTs, enforce TLS. Then they get exposed to:

  • credential stuffing at scale
  • mobile clients being reverse engineered
  • tokens leaking through logs or memory
  • sessions persisting longer than intended
  • recovery flows bypassing stronger controls

At that point, security stops being about protocol correctness and becomes about system behavior under pressure.

This article focuses on what actually holds up: where systems fail, which components absorb those failures, and how to design authentication as a controllable, observable system rather than a one-time verification step.

If you are applying this model to hostile native clients, the mobile companion piece is Modern Mobile Hardening. For the runtime and transport attack mechanics that make those assumptions necessary, see Friday Frida Hacking without the Why and Man-in-the-Middle.

Where this article includes “Field Notes” sections, those observations come from my own experience across enterprise, product, and contract software work, in addition to the standards and reference material that inform the rest of the article.

Authentication Is a System, Not a Feature

A typical flow crosses multiple boundaries:

  • mobile/web clients (untrusted)
  • API gateway (first enforcement)
  • authentication control plane (identity, tokens, sessions)
  • backend services (authorization)
  • monitoring and incident response

Each layer introduces failure modes. Security emerges from how these layers constrain, observe, and revoke trust.

Secure authentication depends as much on architecture and operations as on cryptographic protocols.

Most authentication systems are designed to verify identity once.
Secure systems are designed to survive ongoing compromise.

Threat Model (What Actually Happens)

Assume all of the following will occur:

  • credential theft (phishing, reuse)
  • token theft (logs, memory, compromised devices)
  • token replay
  • session hijacking
  • automated abuse (credential stuffing, bots)
  • targeted account takeover (admins, high-value users)

Design assumption:

Some credentials and sessions will be compromised. The system must limit damage, detect it, and contain it quickly.

That same posture applies to engineering tooling itself. If your teams are using AI across delivery, the supply-chain analogue is Threat Modeling AI as an Engineering Coprocessor.

Reference Architecture (Control Planes and Boundaries)

flowchart LR Client["Client (Mobile/Web)"] Edge["API Gateway / WAF"] Auth["Auth Control Plane"] IdP["Identity Provider"] Token["Token Service"] Session["Session Service"] Risk["Risk Engine"] API["Backend Services"] Policy["Authorization / Policy Engine"] Telemetry["Event Stream"] SIEM["SIEM / Detection"] Client --> Edge Edge --> Auth Auth --> IdP Auth --> Token Auth --> Session Auth --> Risk Edge --> API API --> Policy Auth --> Telemetry Edge --> Telemetry API --> Telemetry Telemetry --> SIEM

A hardened system separates responsibilities into planes:

  • Edge Enforcement: CDN/WAF, API gateway, rate limits
  • Authentication Control Plane: identity, MFA/passkeys, tokens, sessions, risk
  • Application Plane: business APIs + authorization
  • Security Plane: telemetry, detection, response

Trust boundaries:

  • Client: untrusted (programmable by the attacker)
  • Edge: first enforcement boundary
  • Control Plane: source of trust artifacts
  • Application: consumes verified identity
  • Security: detects and contains failures

Key idea:

Trust is minted in the control plane and continuously evaluated at runtime.

Components (What Actually Exists)

Client Layer (Untrusted)

Treat the client as an instrumented adversarial environment.

The client is not a security boundary.
It is a signal generator at best, and an attacker-controlled environment at worst.

Auth Flow Controller

  • PKCE verifier lifecycle
  • redirect handling
  • token exchange
  • refresh orchestration (single-flight)

Failure modes:

  • verifier leakage (logs/memory)
  • redirect hijacking
  • parallel refresh races

Token Storage

  • memory-only (best)
  • secure storage (acceptable)
  • filesystem (avoid)

Reality: secure storage reduces risk but does not eliminate it on compromised devices.

Anything stored on a mobile device should be assumed extractable under the right conditions.

Secure Hardware Interface

  • Secure Enclave / StrongBox / platform authenticators
  • non-exportable keys, user-presence gating

Network Layer

  • attaches tokens, handles refresh/retries
  • avoid token leakage in logs; guard against retry storms

Device Signals & Attestation

  • device/app signals, attestation (App Attest / Play Integrity)
  • use as probabilistic inputs, not hard trust

The client may help prove identity, but must never enforce authorization.

Edge / Gateway

  • validate tokens (signature/expiry)
  • check session/revocation state
  • rate limit (IP/account/device)
  • bot detection, geo controls

Most volumetric attacks should die here.

Authentication Control Plane

Auth Orchestrator

  • login, MFA/passkeys, recovery, step-up

Identity Provider

  • users, credentials, passkeys, account state

Token Service

  • access/refresh tokens, signing, expiry

Session Service (backbone)

  • session inventory, device mapping
  • token families, revocation state

Risk Engine

  • anomaly detection, risk scoring, step-up triggers

Modern authentication systems increasingly rely on risk scoring systems that aggregate signals across sessions, devices, and behavior.

At scale, this becomes a fraud detection problem rather than a pure authentication problem.

Device Signals (First-Class Inputs)

  • device fingerprint (coarse, privacy-aware)
  • OS/app version, patch level
  • attestation (App Attest / Play Integrity / SafetyNet legacy)
  • hardware-backed key presence

How they’re used

  • contribute to risk score (not hard allow/deny)
  • detect new/unknown devices
  • influence step-up decisions and session trust

Device signals are not a trust boundary. They are probabilistic evidence that improves detection and response.

Application Plane

  • backend services enforce authorization
  • policy engine (RBAC/ABAC/context)

Security Infrastructure

  • KMS/HSM (signing keys, rotation)
  • revocation cache (fast enforcement)
  • event stream (auth/session telemetry)
  • SIEM (detection/alerting)
  • audit logs (forensics)

Token Architecture (What Determines Blast Radius)

Token Types

Access tokens

  • short-lived (5–15 minutes)
  • bearer or PoP

Refresh tokens

  • long-lived
  • must rotate

Session records

  • authoritative state (user ↔ device ↔ token family)

PKCE (What It Solves—and Doesn’t)

sequenceDiagram participant C as Client participant A as Auth Server C->>A: Auth Request + Code Challenge A-->>C: Authorization Code C->>A: Code + Verifier A-->>C: Access + Refresh Tokens

PKCE prevents authorization code interception in public clients.

Limitations:

  • does not prevent phishing
  • does not protect tokens post-issuance

Refresh Token Rotation (and Detection)

If you are not rotating refresh tokens, you are issuing long-lived bearer credentials.

sequenceDiagram participant C as Client participant T as Token Service C->>T: Refresh RT1 T-->>C: RT2 + AT Note over T: RT1 invalidated C->>T: Refresh RT1 (reused) T-->>C: Reject Note over T: Trigger compromise response

If RT1 is reused after rotation → strong compromise signal.

Required response:

  • revoke session family
  • emit security event

Field Notes (What Breaks in Practice)

  • Systems omit refresh tokens and rely on user-triggered failures (401 → re-login). This removes server control and visibility over session lifecycle.
  • Clients attempt to generate encryption keys locally (e.g., for token protection) using weak or reproducible entropy. This collapses the protection model.
  • Datastores occasionally contain plaintext or reversibly encrypted credentials. This is a catastrophic failure regardless of upstream controls.

Practical guidance

  • Use refresh tokens with rotation + reuse detection; drive refresh from API responses, not UI failures.
  • Never rely on client-generated keys for long-term secrecy; prefer hardware-backed keys or server-side controls.
  • Store credentials using strong, adaptive hashing (e.g., Argon2/bcrypt with proper parameters) and treat the database as eventually exposed.

If your credential store is compromised, every upstream control is irrelevant.
Hashing is not optional—it is the last line of defense.

Proof of Possession (PoP)

Bearer tokens are replayable.

sequenceDiagram participant C as Client participant API C->>API: Request + Token + Signature API->>API: Verify token + verify signature

PoP binds tokens to a key (e.g., DPoP, mTLS).

Trade-offs:

  • higher complexity
  • strong replay resistance

Practical “Strong” Token Model

  • short-lived access tokens
  • rotating refresh tokens with reuse detection
  • server-side session tracking
  • optional PoP for high-risk APIs

Biometrics Done Correctly

Correct model:

biometric → unlock hardware key → sign challenge → server verifies

Anti-pattern:

biometric → unlock stored token

Properties:

  • biometric never leaves device
  • server trusts cryptographic proof
  • credentials are device-bound

Biometrics do not authenticate users to your system.
They unlock local cryptographic material. Confusing the two creates false security.

Field Notes (Common Failure Modes)

  • Storing encrypted passwords locally and unlocking them with biometrics. On jailbroken/rooted devices this is often extractable or bypassable.
  • Using biometrics as a UI gate only (e.g., a prompt with no hardware-backed key). This is trivial to bypass with instrumentation.
  • Fragmentation across devices/OS versions creates inconsistent guarantees and UX. Security properties vary more than teams expect.

Practical guidance

  • Tie biometrics to hardware-backed keys, not stored secrets.
  • Treat device compromise as possible; rely on server-side controls (sessions, step-up) to contain damage.
  • Keep the UX consistent, but design assuming heterogeneous security capabilities.

Sessions: The Hidden Backbone

Stateless-only systems lack control.

Stateful sessions provide:

  • revocation
  • visibility
  • incident response

Recommended hybrid:

  • stateless access tokens
  • stateful session + refresh tracking

Stateless authentication scales well—until you need to revoke trust.

Authentication verifies identity; sessions verify ongoing trust.

Attack → Failure → Control (Compressed Map)

AttackFailureControl
phishingweak auth factorspasskeys / phishing-resistant MFA
code interceptionredirect flowPKCE
access token theftclient storage/loggingshort TTL / PoP
refresh token theftlifecycle designrotation + reuse detection
session hijackno session controlsession state + revocation
replaybearer modelPoP
mobile tamperingtrusting clientbackend enforcement
credential stuffingno throttlingrate limits + bot defense
recovery abuseweak recoveryhardened recovery flows

Defense in Depth (What Actually Helps)

Layers:

  1. identity verification
  2. token issuance
  3. session tracking
  4. behavioral analysis
  5. operational response

Controls:

  • rate limiting and throttling
  • anomaly detection (location, device, behavior)
  • MFA and step-up for sensitive actions
  • device trust signals
  • comprehensive telemetry

Static controls (rate limits, MFA) are necessary but insufficient at scale.

Mature systems incorporate adaptive authentication, where access decisions are continuously adjusted based on behavioral signals and evolving risk scores.

Operational Security (Where Systems Win or Lose)

Required capabilities:

  • immediate session revocation
  • forced logout (user/all devices)
  • credential reset workflows
  • real-time monitoring

Key signals:

  • refresh token reuse
  • impossible travel
  • new device anomalies
  • abnormal API patterns

Without visibility and response, attacks are silent.

Many large-scale systems incorporate machine learning models to detect subtle anomalies (e.g., session drift, behavioral deviation).

These systems extend detection beyond static rules, but do not replace strong architectural controls.

Field Notes (Mobile Networking Realities)

  • Certificate pinning is effective but brittle: app updates lag certificate rotations, leading to outages or forced disables.
  • Static pinning strategies create operational risk during cert rollover or CA changes.
  • Dynamic pin distribution can reduce some rollover risk, but it adds material implementation and operational complexity of its own.

Practical guidance

  • Prefer pinning to public keys (SPKI) with backup pins.
  • Implement graceful rotation (overlapping pins) and remote configuration where feasible.
  • Follow OWASP’s nuance here: general guidance discourages pinning by default because the cost of failure is high, while mobile guidance treats it as a higher-assurance control only when you control the service, client update path, and pin rotation process.
  • Treat pinning as a defense-in-depth layer, not a sole control; assume bypass is possible on compromised devices.

Certificate pinning increases security—but also increases the cost of failure.
Plan for rotation, not just validation.

Supply Chain and Dependency Risk

Authentication systems increasingly depend on external components:

  • mobile SDKs
  • identity providers
  • cryptographic libraries
  • fraud detection services

These introduce supply chain risk:

  • outdated or unpatched dependencies
  • unclear ownership of critical libraries
  • inconsistent rollout across platforms/teams
  • hidden transitive dependencies affecting security posture

A secure authentication design can be undermined by an untracked or outdated dependency.

Practical guidance

  • maintain a software bill of materials (SBOM) for auth-related components
  • enforce version visibility and upgrade policies
  • ensure critical SDKs are observable in production (version, health)
  • treat auth dependencies as high-risk assets, not just libraries

Applying Secure Design Principles (CSSLP Perspective)

Authentication systems benefit from applying structured secure design principles:

Least Privilege

  • scope tokens narrowly
  • restrict session capabilities
  • limit blast radius of compromise

Defense in Depth

  • layer controls (client, edge, control plane, session, risk)
  • assume any single layer can fail

Secure by Default

  • short-lived tokens
  • rotation enabled
  • strong hashing enforced

Fail Securely

  • deny on ambiguity (invalid tokens, missing context)
  • avoid fallback paths that weaken authentication

Complete Mediation

  • validate authentication and authorization on every request
  • do not trust cached or client-provided state

Separation of Concerns

  • separate authentication, session, and authorization logic
  • isolate key management and token signing

Economy of Mechanism

  • avoid overly complex auth flows
  • complexity increases bypass risk (especially client-side)

Open Design

  • rely on proven protocols and primitives
  • avoid security through obscurity

Psychological Acceptability

  • consistent UX across auth flows
  • reduce user-driven bypass (e.g., MFA fatigue)

Many authentication failures are not due to missing controls, but due to violations of these foundational principles.

Common Architectural Failures

  • long-lived tokens
  • no revocation mechanism
  • trusting mobile/client enforcement
  • weak recovery flows
  • lack of telemetry
  • no session inventory

These are architectural issues, not protocol issues.

What Hardened Systems Do Differently

  • device-bound credentials (passkeys)
  • short-lived tokens with rotation
  • session state with visibility and control
  • continuous risk evaluation
  • step-up authentication for sensitive actions
  • fast revocation paths
  • integrated telemetry and detection

They treat authentication as a continuous trust system, not a one-time event.

Authentication Becomes Fraud Detection at Scale

As systems grow, authentication shifts from a one-time decision to a continuous evaluation:

identity (who are you?) → session (are you still you?) → behavior (are you acting like you?)

Characteristics:

  • trust becomes probabilistic and time-varying
  • decisions become contextual (device, location, behavior)
  • controls become adaptive (step-up, throttling, containment)

Device signals are a key input here, but not decisive on their own. They improve signal quality when combined with:

  • historical behavior
  • session continuity
  • network and geo context

Field Notes (Supply Chain Blind Spots)

In practice, fraud and adaptive systems are only as strong as their weakest integration point.

A recurring failure mode is when critical client-side or SDK components:

  • are not tracked as part of the supply chain
  • are obscured or renamed, making ownership unclear
  • become outdated and difficult to upgrade across teams

This creates a hidden risk surface:

  • inconsistent signal quality
  • broken or stale attestation
  • degraded fraud detection accuracy

Worse, these issues are often discovered during incidents, not proactively.

Practical guidance

  • Treat authentication and fraud-related SDKs as first-class supply chain dependencies
  • Maintain clear ownership and version visibility
  • Enforce upgrade paths and deprecation policies
  • Ensure signals are observable and verifiable in production

Architecture provides guarantees; fraud/risk systems provide detection and adaptation. You need both.

Final Mental Model

Identity → Tokens → Sessions → Behavior → Trust

Each stage must:

  • validate
  • constrain
  • observe
  • revoke

Authentication is not about proving identity.
It is about maintaining trust in a system that is actively being attacked.

Closing

Authentication is not about proving identity once.

It is about maintaining trust under continuous adversarial pressure—across devices, tokens, sessions, and behavior.

Systems that succeed are not those that implement the right protocol, but those that:

  • limit the impact of inevitable compromise
  • detect abnormal behavior quickly
  • and revoke trust without friction when needed

Everything else is just implementation detail.

References

Author Background

  • LinkedIn: Ryan Jennings for the professional background behind the field notes and operational commentary in this article.

Authentication System Design Foundations

Internal Reading

Core Identity and Protocol Standards

Token, Session, and Replay Security

Passwords, MFA, and Passkeys

Mobile and Platform Security

Operational Security and Supply Chain

References by Section