Secrets

Declare secrets in your spec, resolve from env / files / commands / Dashboard, inject into the sandbox container.

Secrets are how your spec gets credentials — LLM API keys, database passwords, third-party tokens — into the sandbox without ever committing them. Keystone takes a spec-as-source-of-truth approach: the spec lists every secret your scenario needs and where each value comes from. The SDK resolves locally, the server merges with Dashboard-stored values, the runtime injects them as env vars in the agent container.

Declaring a secret

secrets:
  - name: ANTHROPIC_API_KEY     # the env var name inside the sandbox
    source: env                  # where to pull the value from

Two top-level fields decide the value:

• source: — pull from the local environment, a file, a shell command, or the Dashboard. Resolved by the SDK on the caller's machine and forwarded to the server.
• from: — a spec-owned literal (from: "static://...") or a per-run generated value (from: generated). These bypass the source pipeline.

If neither is specified, source: env is the default.

Source types

env (default)

secrets:
  - name: OPENAI_API_KEY
    source: env                         # reads $OPENAI_API_KEY from the caller's env

Equivalent shorthand:

secrets:
  - name: OPENAI_API_KEY                # source defaults to env

The SDK reads process.env.OPENAI_API_KEY on the calling machine and forwards the value in the create request.

env:RENAMED — rename

secrets:
  - name: DB_PASSWORD                  # the sandbox sees DB_PASSWORD
    source: env:LOCAL_DB_PASS          # but pull from $LOCAL_DB_PASS locally

Useful when your local .env uses different names than the sandbox expects.

file: — read a file

secrets:
  - name: GCP_SERVICE_ACCOUNT
    source: "file:~/.config/gcloud/keystone-sa.json"

The SDK reads the file, strips trailing whitespace, forwards the bytes. ~/ expands to the user's home directory. Useful for credentials managed by external CLIs (gcloud, aws, vault) that write to disk.

command: — exec a shell command

secrets:
  - name: STRIPE_SECRET
    source: 'command:op read "op://Dev/Keystone/stripe-secret"'   # 1Password CLI
 
  - name: VAULT_TOKEN
    source: "command:vault token lookup -field=id"                # HashiCorp Vault
 
  - name: DOPPLER_TOKEN
    source: "command:doppler secrets get --plain MY_SECRET"        # Doppler
 
  - name: AWS_ACCESS_KEY
    source: "command:aws secretsmanager get-secret-value --secret-id prod/aws-key --query SecretString --output text"

The SDK runs the command via sh -c, captures stdout, strips whitespace, forwards. Failures (non-zero exit) skip the secret silently — the server may still resolve it from the Dashboard. If neither layer produces a value, sandbox boot fails with the missing name.

dashboard — server-side only

secrets:
  - name: STRIPE_LIVE_KEY
    source: dashboard                  # never overridable from a local machine

The SDK forwards nothing for this entry. The server pulls the value from the Dashboard Secrets tab (AES-256-GCM encrypted at rest, scoped to the billing owner).

Use source: dashboard when:

• A secret must be the same across the whole team, no exceptions.
• You don't trust local .env files to be authoritative for prod-critical keys.
• A teammate's stray STRIPE_LIVE_KEY=test_... should not leak into runs.

from: "static://..." — spec literal

secrets:
  - name: TEST_FIXTURE_TOKEN
    from: "static://fake-test-value"

A literal string committed alongside the spec. Always wins over every other source.

Never put real keys behind static://. This block is in your spec file, which is committed to git. Use source: env or source: dashboard for anything real. static:// is for deterministic test fixtures only.

from: generated — random per run

secrets:
  - name: GENERATED_DB_PASSWORD
    from: generated

The server generates a fresh 32-byte hex string per sandbox. Useful for ephemeral test passwords — your services.db block can reference {{ secrets.GENERATED_DB_PASSWORD }} and Postgres + your agent will both see the same generated value, but it changes every run.

Precedence

Highest priority wins:

1. Spec literal (from: static://... or from: generated) — never overridable.
2. SDK-forwarded source value (env, env:X, file:, command:) — resolved locally.
3. Dashboard secret — server-side fallback when source is dashboard or local resolution failed.

A secret that resolves to nothing at any layer fails the sandbox boot with the missing name in the error. No silent empties.

Auto-forwarding from a spec file

You don't have to manually pass secrets: { ... } to create(). If you give the SDK a path to your spec, it parses the secrets: block, resolves every entry's source, and forwards the resulting {name: value} map automatically.

const exp = await ks.experiments.create({
  name: "scenario-1",
  spec_id: "email-agent-01",
  specPath: "./specs/scenario-1.yaml",   // SDK resolves all sources
});
await ks.experiments.runAndWait(exp.id);

import { collectDeclaredSecretsFromFile } from "@polarityinc/polarity-keystone";
 
const secrets = collectDeclaredSecretsFromFile("./specs/scenario-1.yaml");
// → { ANTHROPIC_API_KEY: "sk-ant-...", DB_PASSWORD: "...", ... }

Scopes — env vs file

By default, secrets are injected as env vars inside the sandbox container. You can additionally template them into a config file:

secrets:
  - name: STRIPE_KEY
    from: "static://sk_test_xxx"
    scope:
      env: true
      file_template: "config/stripe.json"

file_template is templated into the named file at sandbox setup time. If the file already exists, secret values replace {{ STRIPE_KEY }} placeholders.

Simple scope shorthand

secrets:
  - name: GENERATED_PASS
    from: generated
    scope: env                # short for { env: true }

Interpolating secrets in services

Service env: values support {{ secrets.NAME }} substitution — the value comes from whatever source the secrets: block declares:

secrets:
  - name: DB_PASSWORD
    source: env
 
services:
  - name: db
    image: postgres:16
    env:
      POSTGRES_PASSWORD: "{{ secrets.DB_PASSWORD }}"      # substituted at boot
      POSTGRES_DB: northwind
    ports: [5432]

The substitution uses whatever value the resolver produced — your .env, the Dashboard, a from: generated random string, etc. This means you never bake real credentials into the spec, and rotating a key is just changing the source.

Dashboard Secrets tab

Go to app.paragon.run/app/keystone/settings → Secrets tab. Stored values:

• AES-256-GCM encrypted at rest; decrypted only in-process on the Keystone server.
• Scoped to the billing owner — every teammate on the same team shares the same secrets.
• Auto-inject into every sandbox when source: dashboard or the caller has no local value for a declared name.
• Show a warning next to keys declared by a spec but not set anywhere.

The Dashboard is the place to put team/prod baselines: things that should be the same everywhere and shouldn't depend on whoever ran the experiment having the right .env locally.

Secrets inside the sandbox

When the agent runs, declared secrets are present as env vars:

# inside the sandbox
$ env | grep -E '^(ANTHROPIC|DB_|STRIPE)'
ANTHROPIC_API_KEY=sk-ant-...
DB_PASSWORD=devpass
STRIPE_LIVE_KEY=sk_live_...

The sandbox-scoped Keystone token (KEYSTONE_API_KEY) is also injected automatically — but it's not your ks_live_ key. It's a per-sandbox ks_sb_<hex> token that's authorized only for this sandbox's resources. Safe to log; can't read other tenants.

Detection: secrets-in-logs

If you set:

forbidden:
  secrets_in_logs: deny

…Keystone scans the agent's stdout/stderr after the run and fails the scenario if any secret value appears verbatim. Use it as a defense against agents that print env vars during debugging.

Patterns

LLM key from local .env, prod key from Dashboard

secrets:
  # Dev: agent uses your personal Anthropic key from .env
  - name: ANTHROPIC_API_KEY
    source: env
 
  # Prod-critical webhook signing key — Dashboard only, never local override
  - name: WEBHOOK_SIGNING_SECRET
    source: dashboard

CI: inject from secret manager

secrets:
  - name: ANTHROPIC_API_KEY
    source: command:vault read -field=value secret/keystone/anthropic

In CI, set up Vault auth in the runner's prologue. The SDK then resolves on the runner.

Per-run randomness for fixtures

secrets:
  - name: TEST_API_TOKEN
    from: generated                   # fresh per run
 
services:
  - name: api-mock
    type: http_mock
    routes:
      - method: GET
        path: /me
        response: '{"token":"{{ secrets.TEST_API_TOKEN }}"}'

Each run uses a fresh token; the agent gets it via env, the mock returns it on /me. Use this to test agents that must not depend on a hardcoded token.