agent-policy-engine

A policy engine for Pi. It evaluates tool operations in this order:

1. Host prechecks
2. Deterministic policy rules
3. Decision cache
4. LLM review

allow permits an operation. deny blocks it. ask opens the host permission UI. Pi blocks ask when it has no UI.

Install

Pi

pi install /path/to/agent-policy-engine

For development:

bun install
pi -e ./src/pi/index.ts

Reviewer configuration

Pi reads global configuration from $PI_CODING_AGENT_DIR/policy-engine.json, then merges .pi/policy-engine.json from the current project. Project settings override global settings.

Use a Pi-configured model. This uses Pi's provider transport and authentication, including OAuth. It never reads OAuth credentials directly.

{
  "reviewer": {
    "kind": "pi",
    "provider": "openai-codex",
    "model": "gpt-5.6-luna",
    "reasoningEffort": "minimal",
    "promptCacheKey": "policy-engine-v1"
  }
}

Pi reviewers accept none, minimal, low, medium, high, xhigh, and max. The selected Pi model determines the supported levels. promptCacheKey is forwarded to Pi as its session ID, which Pi sends to OpenAI as prompt_cache_key for cache routing.

promptCacheKey improves cache routing for Pi models that support it. It does not guarantee a cache hit.

Disable LLM reviews. Requests that reach this stage ask for approval. Static rules, deterministic rules, and cached decisions still apply.

{
  "reviewer": {
    "kind": "none"
  }
}

Use local llama.cpp. This backend does not use authentication.

{
  "reviewer": {
    "kind": "llama.cpp",
    "baseUrl": "http://127.0.0.1:9931",
    "model": "reviewer",
    "enableThinking": true
  }
}

enableThinking defaults to false. llama.cpp always uses the deepseek reasoning format.

Jev classification through Pi

{
  "reviewer": {
    "kind": "classifier",
    "provider": "openrouter",
    "model": "typesafe/jev-1.13",
    "askThreshold": 0.5
  }
}

The threshold above passed the current live policy evaluation cases with Jev 1.13, but is not a general safety guarantee. Set an explicit finite threshold between 0 and 1 and evaluate it against your policy cases. Any risk probability at or above the threshold requires approval. Missing or invalid answers and provider errors also require approval.

This reviewer uses Pi's classifier API and existing provider credentials. Classifiers do not appear in Pi's chat model picker. typesafe/jev-router is a chat router, not the classifier. Jev answers seven policy risk questions in one call; reasons are fixed text, not generated explanations. Existing permission checks still run first. The default reviewer is unchanged.

Local classification through Pi

Pi's built-in llama.cpp extension also exposes chat models as classifiers. Connect to your router with /login llama.cpp, then use /llama to refresh its model catalog.

{
  "reviewer": {
    "kind": "classifier",
    "provider": "llama.cpp",
    "model": "unsloth/Qwen3.5-9B-GGUF:Q6_K",
    "askThreshold": 0.5
  }
}

For Qwen3.5, set ctx-checkpoints = 32 and checkpoint-min-step = 0 in the server preset to reuse the prompt between questions. Start with ctx-size = 8192 and parallel = 1. Pi includes the input state twice, so large tool inputs need more context. The threshold above is a starting point, not a validated setting for this model.

The standalone live evaluator uses ModelRuntime without loading Pi's built-in extensions, so it does not initialize the llama.cpp classifier provider. Evaluate this backend through a Pi session until the evaluator supports extension providers.

LLM and classifier review send the complete tool input to the configured provider. Do not use a cloud reviewer when this is not acceptable.

Pi permissions

Pi uses a root-level tools map and externalDirectories list. Any tool name can be configured. An omitted tool defaults to allow; set a tool to check for LLM review.

{
  "tools": {
    "bash": "check",
    "my_extension_tool": "check"
  },
  "externalDirectories": ["/tmp/pi"]
}

externalDirectories applies to Pi path tools. Each entry is a non-empty directory path: an external path is allowed only when it is the entry or nested below it by path components. Sibling paths with the same string prefix are not allowed. Its entries are also included in the LLM reviewer prompt with the same semantics.

Pi behavior

Pi keeps /pe-allow-bash, Pi static permissions, and Herdr herdr:blocked events around its confirmation dialog. /pe-skip-permissions skips LLM review for the current extension instance. Static permissions, deterministic rules, and cached decisions still apply; uncached requests allow. Use /pe-skip-permissions clear to disable it. The setting is not restored after an extension reload. Pi bypasses MCP tools at the Pi adapter boundary.

For Bash, Pi applies its session override and static permissions to the raw command. Pi and raw core calls then use the same tree-sitter parser and async deterministic evaluator. Policy checks evaluate each command independently using decoded arguments, assignments, and redirects. Commands joined by ;, &&, ||, pipes, or newlines use the same deterministic rules. Every command must pass for the full input to be allowed. Whole-root and whole-home searches with find, rg, or recursive grep require confirmation, including when the search root defaults to the request's working directory. Path checks and bounded, metadata-only glob expansion use the request's working directory; git -C is handled within its own command. Earlier commands do not change how later commands are checked. This does not model changes to the directory, environment, or filesystem, so it can miss sensitive paths reached through those changes. Unknown expansions and unsupported syntax still fall back to review of the full input. Parse errors cannot receive a deterministic allow. If parser assets are unavailable, Bash is blocked.

Bash cache keys preserve the exact source text and working directory. Cached Bash allows are neither read nor written, so an old filesystem snapshot cannot approve a later request. Cached asks and denies still apply.

PowerShell remains one LLM-reviewed operation.

Development

bun test
bunx eslint src test
bunx tsc --noEmit

Run the opt-in llama.cpp policy evaluation:

POLICY_EVAL_BASE_URL=http://127.0.0.1:9931 POLICY_EVAL_MODEL=reviewer bun test test/core/llm.test.ts

Run the same policy evaluation through Pi OAuth. Pi resolves the configured OAuth session; no API key is needed.

POLICY_EVAL_BACKEND=pi \
POLICY_EVAL_PROVIDER=openai-codex \
POLICY_EVAL_MODEL=gpt-5.6-luna \
POLICY_EVAL_REASONING_EFFORT=minimal \
bun test test/core/llm.test.ts

Pi evaluations using Pi's OpenAI Responses APIs use policy-engine-evaluation-v1 as the cache key; set POLICY_EVAL_PROMPT_CACHE_KEY to override it.

Run the same cases through the classifier reviewer with Pi's existing provider credentials:

POLICY_EVAL_BACKEND=classifier \
POLICY_EVAL_PROVIDER=openrouter \
POLICY_EVAL_MODEL=typesafe/jev-1.13 \
POLICY_EVAL_ASK_THRESHOLD=0.5 \
bun test test/core/llm.test.ts

POLICY_EVAL_ASK_THRESHOLD is required; 0.5 passed the current Jev 1.13 evaluation cases. Re-evaluate it when changing the model or questions. A threshold of 0.1 produced false positives on allowed operations. Classifier evaluations default to the openrouter provider. All backends receive the current working directory as policy context.

Tests redirect Pi state to a temporary directory, except opt-in Pi chat and classifier evaluations, which use the existing Pi credentials.