Using SPIFFE/SPIRE Identity

Give every agent a cryptographic workload identity on Red Hat OpenShift AI.

View as Markdown

Every AI agent running on OpenShift needs a verifiable identity to authenticate to other services, sign traces, and prove its provenance. SPIFFE (Secure Production Identity Framework for Everyone) provides a universal identity control plane, and SPIRE is its production-ready implementation — issuing and rotating X.509 and JWT identity documents scoped to each agent's workload.

With kagenti, SPIRE integration is automatic. Adding labels to your deployment triggers webhook injection of a spiffe-helper sidecar that writes SVID files into the pod. Your agent code reads these files at startup and uses the identity for AuthBridge token exchange, MLflow trace tagging, and cross-service authentication — no static credentials required.

SPIFFE/SPIRE Concepts

Identity format

SPIFFE identities are URIs that uniquely identify a workload. On Kubernetes, the identity encodes the trust domain, namespace, and service account:

SPIFFE identity format
# SPIFFE identity format
spiffe://{trust-domain}/ns/{namespace}/sa/{service-account}

# Examples
spiffe://localtest.me/ns/team1/sa/weather-service
spiffe://localtest.me/ns/team1/sa/slack-researcher
spiffe://apps.cluster.example.com/ns/team1/sa/bank-agent-backend

The trust domain is the SPIRE server's domain (e.g., localtest.me for local development, or your cluster's apps domain on OpenShift). The namespace and service account map directly to Kubernetes resources, giving each agent a unique, attestable identity.

SVID types

SPIRE issues SPIFFE Verifiable Identity Documents (SVIDs) in two formats:

SVID TypeFileUse Case
JWT SVIDjwt_svid.tokenHTTP API authentication, token exchange with Keycloak, identity claims for tracing
X.509 SVIDsvid.pem + svid_key.pemmTLS between services, certificate-based authentication

The spiffe-helper sidecar writes both to an emptyDir volume mounted at /spiffe. Files are automatically rotated before expiry.

JWT SVID payload
# JWT SVID payload (decoded)
{
  "sub": "spiffe://localtest.me/ns/team1/sa/weather-service",
  "aud": "kagenti",
  "exp": 1735689600,
  "iat": 1735686000,
  "iss": "https://spire-server.spire.svc.cluster.local:8443"
}

Agent Frameworks

LangGraph

LangGraph is the most common framework for building stateful, multi-actor agent applications. The SPIRE identity is loaded once at startup by reading the JWT SVID from the mounted volume, then attached as tags to every MLflow trace for provenance tracking.

This example is from the bank-voice-agent reference architecture, which runs a multi-agent banking assistant on OpenShift AI with full SPIRE workload identity.

SPIRE SVID loader + MLflow trace tags — Reads JWT/X.509 SVIDs and tags traces with identity claims
import os
import json

# ── SPIRE identity (read once at startup) ─────────────────────
_spire_identity: dict[str, str] = {}
_SVID_DIR = os.environ.get("SPIFFE_SVID_DIR", "/spiffe")
_JWT_SVID_PATH = os.path.join(_SVID_DIR, "jwt_svid.token")
_X509_SVID_PATH = os.path.join(_SVID_DIR, "svid.pem")

def _load_spire_identity() -> dict[str, str]:
    """Read SPIRE SVID files and extract identity claims."""
    identity: dict[str, str] = {}
    try:
        if os.path.isfile(_JWT_SVID_PATH):
            with open(_JWT_SVID_PATH) as f:
                jwt_token = f.read().strip()
            # Decode payload without verification (local file, trusted)
            parts = jwt_token.split(".")
            if len(parts) >= 2:
                import base64 as _b64
                padded = parts[1] + "=" * (4 - len(parts[1]) % 4)
                payload = json.loads(_b64.urlsafe_b64decode(padded))
                identity["spiffe.id"] = payload.get("sub", "")
                identity["spiffe.audience"] = str(payload.get("aud", ""))
                identity["spiffe.issuer"] = payload.get("iss", "")
                identity["spiffe.expiry"] = str(payload.get("exp", ""))
            identity["spiffe.jwt_svid"] = jwt_token[:80] + "..."
    except Exception as exc:
        print(f"[spire] Failed to read JWT SVID: {exc}", flush=True)
    try:
        if os.path.isfile(_X509_SVID_PATH):
            with open(_X509_SVID_PATH) as f:
                cert_pem = f.read().strip()
            identity["spiffe.x509_svid"] = cert_pem[:120] + "..."
    except Exception as exc:
        print(f"[spire] Failed to read X.509 SVID: {exc}", flush=True)
    return identity

_spire_identity = _load_spire_identity()
if _spire_identity:
    print(f"[spire] Identity loaded: "
          f"{_spire_identity.get('spiffe.id', 'unknown')}", flush=True)

The identity is loaded once at process start. Because LangGraph graph invocations run in asyncio.to_thread(), MLflow's thread-local trace context doesn't carry over — so the tagging function uses client.search_traces(max_results=1) to find the most recent trace.

CrewAI

CrewAI orchestrates role-based AI agents working together as a crew. Each crew member inherits the pod's SPIRE identity, which can be logged or forwarded to downstream services.

SPIRE identity in CrewAI — Loads workload identity at startup for zero-trust authentication
import os
import json
from crewai import Agent, Task, Crew, Process

# ── Load SPIRE identity ──────────────────────────────────────
def _load_spire_identity() -> dict[str, str]:
    svid_dir = os.environ.get("SPIFFE_SVID_DIR", "/spiffe")
    jwt_path = os.path.join(svid_dir, "jwt_svid.token")
    identity = {}
    if os.path.isfile(jwt_path):
        with open(jwt_path) as f:
            token = f.read().strip()
        parts = token.split(".")
        if len(parts) >= 2:
            import base64
            padded = parts[1] + "=" * (4 - len(parts[1]) % 4)
            payload = json.loads(base64.urlsafe_b64decode(padded))
            identity["spiffe.id"] = payload.get("sub", "")
            identity["spiffe.audience"] = str(payload.get("aud", ""))
    return identity

spire_identity = _load_spire_identity()
if spire_identity:
    print(f"[spire] CrewAI agent identity: {spire_identity.get('spiffe.id')}")

# ── Define agents and tasks ──────────────────────────────────
researcher = Agent(
    role="Researcher",
    goal="Find accurate information on the given topic",
    backstory="You are an expert research analyst.",
    verbose=True,
)

crew = Crew(
    agents=[researcher],
    tasks=[Task(
        description="Research {topic}",
        expected_output="A summary of findings",
        agent=researcher,
    )],
    process=Process.sequential,
)

result = crew.kickoff(inputs={"topic": "zero-trust agent identity"})

AutoGen

AutoGen enables multi-agent conversations where agents collaborate and solve problems together. SPIRE identity provides cryptographic proof of which workload is hosting the conversation.

SPIRE identity in AutoGen — Loads workload identity at startup for zero-trust authentication
import os
import json
from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.conditions import TextMentionTermination
from autogen_ext.models.openai import OpenAIChatCompletionClient

# ── Load SPIRE identity ──────────────────────────────────────
def _load_spire_identity() -> dict[str, str]:
    svid_dir = os.environ.get("SPIFFE_SVID_DIR", "/spiffe")
    jwt_path = os.path.join(svid_dir, "jwt_svid.token")
    identity = {}
    if os.path.isfile(jwt_path):
        with open(jwt_path) as f:
            token = f.read().strip()
        parts = token.split(".")
        if len(parts) >= 2:
            import base64
            padded = parts[1] + "=" * (4 - len(parts[1]) % 4)
            payload = json.loads(base64.urlsafe_b64decode(padded))
            identity["spiffe.id"] = payload.get("sub", "")
            identity["spiffe.audience"] = str(payload.get("aud", ""))
    return identity

spire_identity = _load_spire_identity()
if spire_identity:
    print(f"[spire] AutoGen agent identity: {spire_identity.get('spiffe.id')}")

# ── Define agents ─────────────────────────────────────────────
model_client = OpenAIChatCompletionClient(model="gpt-4o-mini")

planner = AssistantAgent(
    name="planner",
    model_client=model_client,
    system_message="You are a planning agent. Break tasks into steps.",
)

executor = AssistantAgent(
    name="executor",
    model_client=model_client,
    system_message="You execute the plan. Say TERMINATE when done.",
)

team = RoundRobinGroupChat(
    participants=[planner, executor],
    termination_condition=TextMentionTermination("TERMINATE"),
    max_turns=6,
)

import asyncio
result = asyncio.run(
    team.run(task="Explain SPIFFE workload identity for AI agents")
)

LlamaIndex

LlamaIndex specializes in RAG pipelines and data-connected agents. SPIRE identity ensures that only attested workloads can access the retrieval pipeline and its backing data sources.

SPIRE identity in LlamaIndex — Loads workload identity at startup for zero-trust authentication
import os
import json
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from llama_index.core import Settings
from llama_index.llms.openai import OpenAI

# ── Load SPIRE identity ──────────────────────────────────────
def _load_spire_identity() -> dict[str, str]:
    svid_dir = os.environ.get("SPIFFE_SVID_DIR", "/spiffe")
    jwt_path = os.path.join(svid_dir, "jwt_svid.token")
    identity = {}
    if os.path.isfile(jwt_path):
        with open(jwt_path) as f:
            token = f.read().strip()
        parts = token.split(".")
        if len(parts) >= 2:
            import base64
            padded = parts[1] + "=" * (4 - len(parts[1]) % 4)
            payload = json.loads(base64.urlsafe_b64decode(padded))
            identity["spiffe.id"] = payload.get("sub", "")
            identity["spiffe.audience"] = str(payload.get("aud", ""))
    return identity

spire_identity = _load_spire_identity()
if spire_identity:
    print(f"[spire] LlamaIndex agent identity: {spire_identity.get('spiffe.id')}")

# ── Build a RAG pipeline ─────────────────────────────────────
Settings.llm = OpenAI(model="gpt-4o-mini")

documents = SimpleDirectoryReader("./data").load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()

response = query_engine.query(
    "What are the key concepts in SPIFFE workload identity?"
)
print(response)

Google ADK

Google Agent Development Kit (ADK) builds agents using Gemini models with built-in tool use. The SPIRE identity can be exposed as an ADK tool, letting the agent inspect and report its own workload identity.

SPIRE identity as an ADK tool — Agent can inspect its own SPIFFE workload identity
import os
import json
from google.adk.agents import Agent
from google.adk.runners import Runner
from google.adk.sessions import InMemorySessionService
from google.genai import types

# ── Load SPIRE identity ──────────────────────────────────────
def _load_spire_identity() -> dict[str, str]:
    svid_dir = os.environ.get("SPIFFE_SVID_DIR", "/spiffe")
    jwt_path = os.path.join(svid_dir, "jwt_svid.token")
    identity = {}
    if os.path.isfile(jwt_path):
        with open(jwt_path) as f:
            token = f.read().strip()
        parts = token.split(".")
        if len(parts) >= 2:
            import base64
            padded = parts[1] + "=" * (4 - len(parts[1]) % 4)
            payload = json.loads(base64.urlsafe_b64decode(padded))
            identity["spiffe.id"] = payload.get("sub", "")
            identity["spiffe.audience"] = str(payload.get("aud", ""))
    return identity

spire_identity = _load_spire_identity()
if spire_identity:
    print(f"[spire] ADK agent identity: {spire_identity.get('spiffe.id')}")

# ── Define an ADK agent ──────────────────────────────────────
def check_identity() -> dict:
    """Return the agent's SPIFFE workload identity."""
    return spire_identity or {"error": "No SPIRE identity available"}

agent = Agent(
    name="identity_agent",
    model="gemini-2.0-flash",
    description="An agent that can inspect its own workload identity",
    instruction="Help users understand SPIFFE workload identity. "
                "Use the check_identity tool to show the agent's identity.",
    tools=[check_identity],
)

session_service = InMemorySessionService()
runner = Runner(agent=agent, app_name="identity_app",
                session_service=session_service)

session = session_service.create_session(
    app_name="identity_app", user_id="user1"
)

message = types.Content(
    role="user",
    parts=[types.Part(text="What is your workload identity?")],
)

import asyncio

async def run():
    async for event in runner.run_async(
        user_id="user1", session_id=session.id, new_message=message
    ):
        if event.is_final_response():
            print(event.content.parts[0].text)

asyncio.run(run())

OpenShift Deployment

Installing kagenti

The kagenti platform setup script installs the full stack on an OpenShift cluster: SPIRE, cert-manager, Keycloak, the kagenti operator and webhook, MCP Gateway, and optionally MLflow via RHOAI. It also configures Istio multi-mesh shared trust so ztunnel mTLS works across control planes.

Prerequisites: cluster-admin access (oc login), helm >= 3.18.0, and python3. The script auto-detects the cluster's trust domain from the DNS operator.

Install kagenti
# Clone the kagenti repo (if not already present)
git clone https://github.com/kagenti/kagenti.git ~/git/kagenti

# Run the platform setup (requires cluster-admin and helm >= 3.18.0)
cd ~/git/kagenti
./scripts/ocp/setup-kagenti.sh

The script installs three Helm releases: kagenti-deps (SPIRE, cert-manager, Istio, Keycloak operators and operands), kagenti (operator, webhook, UI), and mcp-gateway. It is idempotent — re-running upgrades existing releases.

Common options
# Use a local clone instead of auto-cloning from upstream
./scripts/ocp/setup-kagenti.sh --kagenti-repo ~/git/kagenti

# Custom Keycloak realm (default: kagenti)
./scripts/ocp/setup-kagenti.sh --realm nerc

# Skip MLflow integration
./scripts/ocp/setup-kagenti.sh --skip-mlflow

# Skip MCP Gateway installation
./scripts/ocp/setup-kagenti.sh --skip-mcp-gateway

# Skip the Kagenti UI and backend
./scripts/ocp/setup-kagenti.sh --skip-ui

# Dry run — show commands without executing
./scripts/ocp/setup-kagenti.sh --dry-run

Once the platform is running, agent namespaces listed in charts/kagenti/values.yaml under agentNamespaces (defaults: team1, team2) are configured for sidecar injection. Deploy your agent into one of these namespaces with the kagenti labels below to get automatic SPIRE identity.

Kagenti labels and annotations

When kagenti.enabled is set, the Helm chart adds labels and annotations to your deployment that trigger automatic sidecar injection. The kagenti webhook injects four containers into the pod:

ContainerPurpose
spiffe-helperObtains SVIDs from the SPIRE agent and writes them to /spiffe
envoy-proxyAuthBridge sidecar — validates inbound JWTs, exchanges outbound tokens
kagenti-client-registrationRegisters the workload's SPIFFE ID as a Keycloak client
proxy-initiptables init container for Envoy traffic interception
Deployment labels and annotations
# Deployment metadata labels
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-agent-backend
  labels:
    kagenti.io/type: "agent"
    kagenti.io/framework: "LangGraph"
    kagenti.io/workload-type: "deployment"
    protocol.kagenti.io/a2a: ""
spec:
  template:
    metadata:
      labels:
        # Enables webhook sidecar injection
        kagenti.io/inject: "enabled"
        kagenti.io/spire: "enabled"
        kagenti.io/type: "agent"
        kagenti.io/client-registration-inject: "true"
        protocol.kagenti.io/a2a: ""
        istio.io/dataplane-mode: "none"
      annotations:
        # Exclude WebSocket port from Envoy interception
        kagenti.io/inbound-ports-exclude: "8765"
Label / AnnotationPurpose
kagenti.io/inject: "enabled"Triggers AuthBridge sidecar injection
kagenti.io/spire: "enabled"Triggers SPIFFE helper sidecar injection
kagenti.io/type: "agent"Registers workload as an agent in the kagenti catalog
kagenti.io/client-registration-injectAuto-registers SPIFFE ID with Keycloak
kagenti.io/inbound-ports-excludePorts that bypass Envoy interception (e.g., WebSocket)

Helm chart configuration

Enable kagenti integration with a single flag. The chart creates a dedicated ServiceAccount for SPIRE identity binding, exposes the A2A port, mounts the SVID volume, and sets the KAGENTI_ENABLED environment variable.

values.yaml
kagenti:
  enabled: true
  a2aPort: 8080
  authbridge:
    # Target audience for outbound token exchange (catch-all route)
    targetAudience: "echo-service"
    # Token scopes for outbound token exchange
    tokenScopes: "openid"
ServiceAccount
# ServiceAccount for SPIRE workload identity
apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-agent-backend
  namespace: team1
SVID volume mount
# SVID volume mount in deployment spec
spec:
  containers:
    - name: backend
      env:
        - name: KAGENTI_ENABLED
          value: "true"
      volumeMounts:
        - name: svid-output
          mountPath: /spiffe
          readOnly: true
      ports:
        - name: a2a
          containerPort: 8080
          protocol: TCP
  # spiffe-helper writes SVIDs to this emptyDir
  volumes:
    - name: svid-output
      emptyDir: {}

Kubernetes manifests

For deployments without Helm, apply the kagenti labels directly to your Kubernetes manifests. Each agent needs a ServiceAccount, a Service with the A2A port, and a Deployment with the correct labels. See the kagenti agent examples for complete working manifests.

k8s.yaml
# Standalone Kubernetes manifest (without Helm)
apiVersion: v1
kind: ServiceAccount
metadata:
  name: weather-service
  namespace: team1
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/name: weather-service
    kagenti.io/type: agent
    protocol.kagenti.io/a2a: ""
  name: weather-service
  namespace: team1
spec:
  ports:
  - name: http
    port: 8080
    targetPort: 8000
  selector:
    app.kubernetes.io/name: weather-service
    kagenti.io/type: agent
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/name: weather-service
    kagenti.io/framework: LangGraph
    kagenti.io/type: agent
    kagenti.io/workload-type: deployment
    protocol.kagenti.io/a2a: ""
  name: weather-service
  namespace: team1
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: weather-service
      kagenti.io/type: agent
  template:
    metadata:
      labels:
        app.kubernetes.io/name: weather-service
        kagenti.io/framework: LangGraph
        kagenti.io/type: agent
        protocol.kagenti.io/a2a: ""
    spec:
      serviceAccountName: weather-service
      containers:
      - name: agent
        image: ghcr.io/kagenti/agent-examples/weather_service:latest
        ports:
        - containerPort: 8000
          name: http
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop: [ALL]
          runAsUser: 1000
      securityContext:
        runAsNonRoot: true
        seccompProfile:
          type: RuntimeDefault

AuthBridge

AuthBridge is the kagenti component that handles zero-trust authentication for agent-to-service communication. It runs as an Envoy sidecar that intercepts HTTP traffic in both directions:

  • Inbound — validates JWT signature, expiration, and issuer via JWKS; returns 401 for invalid tokens
  • Outbound — exchanges the agent's SPIRE SVID for a Keycloak-issued JWT scoped to the target service

Outbound route configuration

A ConfigMap controls which outbound requests get token exchange. Routes use first-match-wins with glob patterns. Internal services (Kubernetes API, LLM endpoints, MLflow) are set to passthrough; everything else gets a token exchange via Keycloak.

authproxy-routes.yaml
# AuthBridge outbound route configuration (ConfigMap)
apiVersion: v1
kind: ConfigMap
metadata:
  name: authproxy-routes
data:
  routes.yaml: |
    # First-match-wins routing
    # "*" does NOT match dots; use "**" for multi-segment hostnames

    # Cloud metadata service — passthrough
    - host: "169.254.169.254"
      passthrough: true

    # Kubernetes API server — passthrough
    - host: "kubernetes*"
      passthrough: true
    - host: "10.96.0.1"
      passthrough: true

    # Internal services that don't need AuthBridge tokens
    - host: "maas.**"
      passthrough: true
    - host: "**-predictor-**"
      passthrough: true

    # Catch-all: exchange tokens for all other destinations
    - host: "**"
      target_audience: "echo-service"
      token_scopes: "openid"

Token exchange

AuthBridge implements RFC 8693 OAuth2 Token Exchange. The agent's SPIRE SVID is used as the client credential, and the user's token is exchanged for a scoped token with the target service's audience. This provides least-privilege access — each tool receives only the permissions needed.

Token exchange (Python)
import requests
import jwt

def exchange_token_for_tool(user_token: str, tool_audience: str) -> str:
    """Exchange user token for tool-scoped token using SPIFFE identity."""

    # Read SPIFFE JWT SVID
    with open("/spiffe/jwt_svid.token", "r") as f:
        jwt_svid = f.read().strip()

    # Extract client ID from SVID
    payload = jwt.decode(jwt_svid, options={"verify_signature": False})
    client_id = payload["sub"]

    # RFC 8693 token exchange request
    response = requests.post(
        "http://keycloak.keycloak.svc.cluster.local:8080"
        "/realms/master/protocol/openid-connect/token",
        data={
            "grant_type": "urn:ietf:params:oauth:grant-type:token-exchange",
            "subject_token": user_token,
            "subject_token_type":
                "urn:ietf:params:oauth:token-type:access_token",
            "audience": tool_audience,
            "client_id": client_id,
        },
        headers={
            "Authorization": f"Bearer {jwt_svid}",
            "Content-Type": "application/x-www-form-urlencoded",
        },
    )

    if response.status_code != 200:
        raise Exception(f"Token exchange failed: {response.text}")

    return response.json()["access_token"]

# Usage: exchange user token for a tool-scoped token
user_token = request.headers.get("Authorization", "").replace("Bearer ", "")
tool_token = exchange_token_for_tool(user_token, "slack-tool")

tool_response = requests.post(
    "http://slack-tool.team1.svc.cluster.local:8000/mcp",
    headers={"Authorization": f"Bearer {tool_token}"},
    json={"method": "tools/list"},
)

Verifying Identity

You can verify that SPIRE identity is working by inspecting the SVID files on the pod, or by using a check_identity tool that calls an echo service to show the JWT claims that AuthBridge attaches to outbound requests.

Validate SVIDs on a running pod
# Validate SPIRE SVIDs on a running pod
kubectl exec -n team1 deployment/my-agent -- ls -la /spiffe/
# Expected: svid.pem  svid_key.pem  svid_bundle.pem  jwt_svid.token

# Decode JWT SVID to inspect claims
kubectl exec -n team1 deployment/my-agent -- \
  cat /spiffe/jwt_svid.token | cut -d'.' -f2 | base64 -d | jq .

# Check SPIRE workload registration
kubectl exec -n spire deployment/spire-server -- \
  /opt/spire/bin/spire-server entry show
check_identity tool
import os
import requests
from langchain_core.tools import tool

ECHO_SERVICE_URL = os.getenv("ECHO_SERVICE_URL", "")

@tool
def check_identity() -> dict:
    """Check the workload identity of this agent by calling the echo service.

    Returns the decoded JWT token claims that AuthBridge attaches to outbound
    requests, including sub, azp (authorized party), client_id, issuer, scope,
    and groups. This shows the zero-trust identity exchange in action.
    """
    if not ECHO_SERVICE_URL:
        return {"error": "ECHO_SERVICE_URL not configured"}
    try:
        resp = requests.get(f"{ECHO_SERVICE_URL}/identity", timeout=10)
        resp.raise_for_status()
        data = resp.json()
        token = data.get("token", {})
        if token.get("error"):
            return {"error": token["error"],
                    "detail": "AuthBridge may not be configured"}
        return {
            "azp": token.get("azp"),
            "client_id": token.get("client_id"),
            "sub": token.get("sub"),
            "iss": token.get("iss"),
            "scope": token.get("scope"),
            "groups": token.get("groups"),
            "preferred_username": token.get("preferred_username"),
        }
    except Exception as e:
        return {"error": str(e)}

The echo service decodes the Bearer token that AuthBridge attached to the outbound request and returns the claims — including sub (the user identity), azp (the agent's SPIFFE ID as authorized party), scope, and groups. This demonstrates the full zero-trust flow: SPIRE SVID to Keycloak token exchange to scoped access.