JEV Governance Test Lab

Board map — where to go and why

flowchart LR
S[0 START HERE] --> O[1 Overview]
S --> G[2 Governance]
S --> R[3 Real JEV]
S --> L[4 Local/Open]
S --> C[5 Control Plane]
S --> B[6 Benchmark]
S --> T[7 Tournament Lab]
S --> Q[8 Reconciliation]
S --> N[9 Next Runs]
S --> E[10 Sources]
S --> F[11 Train Your Own]
S --> W[12 Research Retrieval]
W --> W1[12.1 Photon Fast Search]
W --> W2[12.2 Research cascade]
W --> W3[12.3 Search vs Firecrawl]

Your first 30 minutes

flowchart TD
A[Open 0.1 Board Map] --> B{What do you need?}
B -- Understand --> C[1.2 Canonical Architecture]
B -- Real JEV --> D[3.2 Vercel → 3.3 OpenRouter]
B -- JEV at home --> E[4.6 Option Matrix → 4.1/4.2]
B -- Completion gate --> F[2.2 Completion + 5.1 Hooks]
B -- Benchmark --> G[6.1 Bake-off + 7.1 Tournament]
B -- Next action --> H[9.2 Dedicated Runs]
C --> I[0.4 Executive Status]
D --> I
E --> I
F --> I
G --> I
H --> I

Do not invent an AUTO threshold before the calibration run; do not choose a local model from project headline latency; do not read every source first.

Paths by objective

Executive status — what matters now

Executive summary

flowchart LR
A[Verbatim governance + OBL IDs] --> B[Exact gate]
B --> C{Exactly decidable?}
C -- yes --> D[Allow / deny + determining rule IDs]
C -- no --> E[Applicability / evidence pack]
E --> F[Fast semantic judge]
F --> G[Calibration + abstention]
G --> H{Risk band}
H -- low --> I[AUTO]
H -- recoverable --> J[REPLAN]
H -- ambiguous --> K[STRONG REVIEW]
K --> L{Resolved?}
L -- yes --> I
L -- no --> M[HUMAN]
D --> N[Append-only receipt]
I --> N
J --> N
M --> N

What is materially different from v05

Canonical architecture

flowchart LR
A[Verbatim governance + OBL IDs] --> B[Exact gate]
B --> C{Exactly decidable?}
C -- yes --> D[Allow / deny + determining rule IDs]
C -- no --> E[Applicability / evidence pack]
E --> F[Fast semantic judge]
F --> G[Calibration + abstention]
G --> H{Risk band}
H -- low --> I[AUTO]
H -- recoverable --> J[REPLAN]
H -- ambiguous --> K[STRONG REVIEW]
K --> L{Resolved?}
L -- yes --> I
L -- no --> M[HUMAN]
D --> N[Append-only receipt]
I --> N
J --> N
M --> N

Same architecture shown on Executive and Architecture tabs intentionally: this is the governing reference diagram.

Canonical findings

These are the conclusions v06 treats as current project knowledge. Status is evidence strength, not importance.

Decisions & invariants

Governance ontology

The evaluator does not receive “a pile of rules.” It receives addressable, source-grounded objects with explicit roles.

Minimum obligation record

{
  "id": "OBL-001",
  "source_id": "OUP-...",
  "source_span": {
    "start": 0,
    "end": 0,
    "quote": "verbatim"
  },
  "family": "KEF|KER|UI|Means|ADR|DEC|ERR|CON",
  "modality": "must|must_not|should|may|question|informational",
  "condition": null,
  "exception": null,
  "severity": "critical|high|medium|low",
  "lifecycle": "active|superseded|deprecated",
  "evidence_type": "trace|artifact|test|semantic|human",
  "dependencies": []
}

Completion contract

sequenceDiagram
autonumber
participant A as Main agent
participant H as Stop / TaskCompleted hook
participant R as Obligation registry
participant X as Exact evidence gate
participant J as Semantic judge
participant P as Policy
A->>H: Claim task complete
H->>R: Load all active obligations
R-->>H: OBL IDs + evidence requirements
H->>X: Check one disposition per OBL + files/tests/tool traces
alt Missing exact requirement
X-->>A: BLOCK completion + missing IDs
else Exact coverage complete
X->>J: Judge remaining semantic satisfaction
J-->>P: DONE/PARTIAL/MISSED/CONTRADICTED/UNEVIDENCED/ABSTAIN
alt All critical pass
P-->>A: ALLOW Stop
else Gaps recoverable
P-->>A: Continue work + exact IDs
else Critical ambiguity
P-->>A: Strong review / human escalation
end
end

Four independent coverage dimensions

Evidence & decision receipts

flowchart LR
S[Verbatim source span] --> O[OBL / KEF / KER / UI / Means ID]
O --> P[Plan/WBS item]
P --> A[Agent action]
A --> E[Evidence event]
E --> V[Evaluator verdict]
V --> D[Policy decision]
D --> R[Decision receipt]
R --> T[Test / metric record]
S --> R
O --> R
V --> R

Every semantic decision must be auditable back to source text and forward to the action/policy outcome.

{
  "decision_id": "DEC-JEV-000001",
  "state_hash": "sha256:...",
  "trace_id": "TRC-000042",
  "obligation_id": "OBL-017",
  "determining_rule_ids": [
    "KER-004",
    "ADR-012"
  ],
  "question_id": "Q-COMPLETE-001",
  "decision": "PASS|FAIL|PARTIAL|CONTRADICTED|UNEVIDENCED|INSUFFICIENT|ABSTAIN",
  "probabilities": {
    "PASS": 0.0
  },
  "calibration": {
    "method": "...",
    "calibration_set_id": "CAL-001",
    "as_of": "2026-09-23"
  },
  "evidence": [
    {
      "evidence_id": "EVD-000091",
      "kind": "tool_trace",
      "uri_or_hash": "sha256:..."
    }
  ],
  "model": {
    "provider": "local|typesafe|openrouter|anthropic",
    "model_id": "...",
    "resolved_version": "...",
    "quantization": "..."
  },
  "latency_ms": 0,
  "raw_response_hash": "sha256:..."
}

Exact vs semantic ownership

sequenceDiagram
autonumber
participant A as Main agent
participant H as Hook / interposer
participant X as Exact policy
participant S as Semantic judge
participant C as Calibration
participant P as Policy
participant U as Human
A->>H: Proposed tool + args + evidence
H->>X: Exact rules / protected paths / required approvals
alt Exact deny
X-->>A: BLOCK + determining IDs
else Needs semantics
X->>S: State + applicable OBL/ADR/DEC/UI questions
S-->>C: Raw distributions / scores / abstention
C-->>P: Calibrated risk
alt Low risk
P-->>A: ALLOW
else Recoverable
P-->>A: REPLAN + offending IDs
else Critical / unresolved
P->>U: HUMAN packet
U-->>A: approve / reject / clarify
end
end

Long-prompt / feedback pipeline

flowchart TD
A[30.6 KB verbatim user feedback] --> B[Deterministic labelled-block segmentation]
B --> C[Source spans + hashes]
C --> D[Candidate obligation extraction]
D --> E[Exception / condition / conflict pass]
E --> F[Persistent OBL registry]
F --> G[Coverage audit against every source block]
G --> H[Human / strong-model adjudication of residual ambiguity]
H --> I[Plan/WBS mapping]
I --> J[Execution evidence ledger]
J --> K[Completion matrix: mentioned / represented / satisfied / evidenced]

Required extraction passes

1. Deterministic segmentation of explicit labels/sections.
2. Quote-first candidate obligation extraction with offsets.
3. Separate conditions/exceptions/conflicts/supersession pass.
4. Persistent ID reconciliation; no destructive deduplication.
5. Coverage audit against every source block and residual unlabelled text.
6. Strong-model/human review only for unresolved boundaries.
7. Plan mapping, execution evidence, completion matrix.

TypeSafe direct

flowchart LR
A[Same canonical state + questions] --> T[TypeSafe direct]
A --> V[Vercel AI Gateway]
A --> O[OpenRouter Decisions]
T --> C[Route comparator]
V --> C
O --> C
C --> D[Resolved model + schema + latency + tokens + cost + data policy]
D --> E[Append-only benchmark record]

Vercel AI Gateway

flowchart LR
A[Same canonical state + questions] --> T[TypeSafe direct]
A --> V[Vercel AI Gateway]
A --> O[OpenRouter Decisions]
T --> C[Route comparator]
V --> C
O --> C
C --> D[Resolved model + schema + latency + tokens + cost + data policy]
D --> E[Append-only benchmark record]

OpenRouter

flowchart LR
A[Same canonical state + questions] --> T[TypeSafe direct]
A --> V[Vercel AI Gateway]
A --> O[OpenRouter Decisions]
T --> C[Route comparator]
V --> C
O --> C
C --> D[Resolved model + schema + latency + tokens + cost + data policy]
D --> E[Append-only benchmark record]

Jev integrations

Privacy & legal

Open / local System-One candidates

flowchart TD
A[Canonical bounded question] --> B{Local mechanism}
B --> C[Specialized System-One: Mapika / Kev / Von / Laya / Zefan]
B --> D[Direct logits: mini-Jev / open-jev / SemIf / PCD]
B --> E[Encoder/NLI: GLiNER / DeBERTa / SetFit / poorjev]
B --> F[AR structured judge: gpt-oss / Gemma / Nemotron / Phi / Qwen]
C --> G[Raw distribution]
D --> G
E --> G
F --> G
G --> H[Calibration + abstention]
H --> I[Common policy contract]

Direct logits / parallel scoring

sequenceDiagram
autonumber
participant S as Shared state
participant M as Frozen causal model
participant K as KV cache
participant O as Candidate options
participant C as Calibrator
S->>M: Single prefill
M-->>K: Reusable KV state
loop each bounded question / candidate set
K->>O: Fork suffix scoring
O->>O: Sequence log-likelihood / token logits
O-->>C: Raw candidate scores
end
C->>C: Length/token-bias controls + temperature/isotonic if justified
C-->>S: Calibrated choice distribution or abstain

Mandatory bias tests

• Candidate tokenization and multi-token length normalization.
• Option-order permutation.
• Prompt-template sensitivity.
• Quantization/version drift.
• Unconditional option-prior / PMI variants where justified.
• Held-out calibration and abstention; raw softmax is not correctness probability.

Encoders, NLI & few-shot classifiers

Autoregressive JEV emulation

flowchart TD
A[Canonical bounded question] --> B{Local mechanism}
B --> C[Specialized System-One: Mapika / Kev / Von / Laya / Zefan]
B --> D[Direct logits: mini-Jev / open-jev / SemIf / PCD]
B --> E[Encoder/NLI: GLiNER / DeBERTa / SetFit / poorjev]
B --> F[AR structured judge: gpt-oss / Gemma / Nemotron / Phi / Qwen]
C --> G[Raw distribution]
D --> G
E --> G
F --> G
G --> H[Calibration + abstention]
H --> I[Common policy contract]

Probability fidelity ladder

1. Best AR analogue: forced bounded choices + raw logprobs where genuinely exposed.
2. Second: repeated stochastic choice frequency under frozen sampling.
3. Lowest fidelity: generated p=0..1 self-report.

Windows / Intel / local runtime

Master option matrix

54 options; “Fit” is role fit, not an overall quality ranking.

Hooks & execution interposition

flowchart LR
A[Agent event] --> B[Harness hook / sandbox]
B --> C[OPA / Cedar / agent policy DSL]
C --> D{Hard decision?}
D -- deny --> E[Block + determining rule ID]
D -- allow --> F[Execute]
D -- semantic fact needed --> G[Bounded evaluator]
G --> C
F --> H[PostToolUse / trace]
H --> I[Evidence ledger]
I --> J[Completion gate]

Policy-as-code & formal enforcement

flowchart LR
A[Agent event] --> B[Harness hook / sandbox]
B --> C[OPA / Cedar / agent policy DSL]
C --> D{Hard decision?}
D -- deny --> E[Block + determining rule ID]
D -- allow --> F[Execute]
D -- semantic fact needed --> G[Bounded evaluator]
G --> C
F --> H[PostToolUse / trace]
H --> I[Evidence ledger]
I --> J[Completion gate]

Structured decoding & typed contracts

Evaluation & observability frameworks

Recommended use

Use deterministic trajectory assertions for objective process requirements and semantic judges only for requirements whose meaning cannot be verified from traces/artifacts alone.

Security lane

Formal requirements & traceability

Provenance & receipts

flowchart LR
S[Verbatim source span] --> O[OBL / KEF / KER / UI / Means ID]
O --> P[Plan/WBS item]
P --> A[Agent action]
A --> E[Evidence event]
E --> V[Evaluator verdict]
V --> D[Policy decision]
D --> R[Decision receipt]
R --> T[Test / metric record]
S --> R
O --> R
V --> R

Architecture bake-off

flowchart TD
A[Frozen cases + human oracle] --> B[Versioned adapters]
B --> C1[Exact policy]
B --> C2[Encoder/NLI]
B --> C3[Open System-One]
B --> C4[Direct logits]
B --> C5[Native Jev]
B --> C6[AR judge]
C1 --> D[Raw per-case records]
C2 --> D
C3 --> D
C4 --> D
C5 --> D
C6 --> D
D --> E[Critical FNR / FPR]
D --> F[Brier / ECE / log loss]
D --> G[P50/P95/P99 / memory / cost]
D --> H[Order / paraphrase / distractor / OOD drift]
E --> I[Risk-coverage + architecture-by-job decision]
F --> I
G --> I
H --> I

The comparison unit is the decision mechanism, not a vendor leaderboard.

CASE-JEV-LONG-001

flowchart TD
A[30.6 KB verbatim user feedback] --> B[Deterministic labelled-block segmentation]
B --> C[Source spans + hashes]
C --> D[Candidate obligation extraction]
D --> E[Exception / condition / conflict pass]
E --> F[Persistent OBL registry]
F --> G[Coverage audit against every source block]
G --> H[Human / strong-model adjudication of residual ambiguity]
H --> I[Plan/WBS mapping]
I --> J[Execution evidence ledger]
J --> K[Completion matrix: mentioned / represented / satisfied / evidenced]

Calibration & selective autonomy

flowchart LR
A[Raw evaluator scores] --> B[Calibration split]
B --> C[Temperature / isotonic / other justified map]
C --> D[Untouched test set]
D --> E[Brier + log loss + ECE + reliability]
E --> F[Threshold / conformal risk sweep]
F --> G[Risk-coverage curve]
G --> H{Operational band}
H -- safe --> I[AUTO]
H -- recoverable --> J[REPLAN]
H -- uncertain --> K[STRONG REVIEW]
H -- critical --> L[HUMAN]

KPI framework

OKR framework

These are project acceptance objectives; numeric risk tolerances remain hypotheses until RUN-JEV-06.

MCDM, veto & expected loss

Baselines, external evidence & economics

Cost-model identity

Adversarial & distribution-shift suite

Research-run comparison

Claim conflict register

v06 does not pretend conflicting source claims disappeared. Each conflict has one canonical operational position and remains addressable.

Evidence taxonomy

Canonical corrections

Search saturation

Another generic census has diminishing value. Targeted delta search remains necessary, but the project should now spend most effort on frozen fixtures, human labels, live route experiments and local reproduction.

Gap registry

Dedicated next runs

Execution roadmap

flowchart LR
R1[RUN-01 Reconcile claims] --> R2[RUN-02 77-obligation oracle]
R1 --> R3[RUN-03 Native Jev routes]
R2 --> R4[RUN-04 Local bake-off]
R2 --> R5[RUN-05 Exact policy]
R3 --> R6[RUN-06 Calibration]
R4 --> R6
R5 --> R7[RUN-07 Adversarial + multilingual]
R6 --> R7
R7 --> R8[RUN-08 Harness portability]
R6 --> R9[RUN-09 Legal/privacy]
R8 --> R10[RUN-10 Load/economics]
R9 --> R10

Acceptance hypotheses

These are deliberately not production thresholds. RUN-JEV-06 must replace them with measured risk/coverage policy.

Canonical source registry

149 canonical source records after the 26 Sep v10 refresh. Live catalog pages are timestamped; project/vendor numbers remain explicitly attributed.

ID continuity & immutability

Machine-readable state

The exact v06 canonical state is embedded in this HTML as <script type="application/json" id="jev-v06-data">.

{
  "document": {
    "id": "DOC-JEV-GOV-LAB",
    "version": "v06",
    "date": "2026-09-23"
  },
  "counts": {
    "options": 54,
    "sources": 75,
    "findings": 20,
    "conflicts": 14,
    "gaps": 16,
    "runs": 10,
    "kpis": 22,
    "okrs": 6
  },
  "findings": [
    {
      "id": "FND-JEV-001",
      "status": "VERIFIED",
      "impact": "ARCH CHANGE",
      "finding": "Deterministic veto/trace checks must run before learned evaluators.",
      "consequence": "Hard facts such as tool-call presence, file deletion, test exit code, all-ID coverage and protected paths should not consume semantic-model risk.",
      "refs": "SRC-JEV-021;SRC-JEV-023;SRC-JEV-057"
    },
    {
      "id": "FND-JEV-002",
      "status": "VERIFIED",
      "impact": "ARCH CHANGE",
      "finding": "Completion gating is two-stage: exact coverage/evidence first, semantic satisfaction second.",
      "consequence": "A missing obligation disposition or required trace event is deterministic failure; only meaning/equivalence needs a learned judge.",
      "refs": "TST-JEV-LOCAL-001;TST-JEV-LOCAL-002"
    },
    {
      "id": "FND-JEV-003",
      "status": "VERIFIED",
      "impact": "ARCH CHANGE",
      "finding": "Agent-native policy is a separate family from generic authorization.",
      "consequence": "Invariant Guardrails, AgentSpec and Progent operate nearer to agent traces/tool calls than ordinary OPA/Cedar policies.",
      "refs": "SRC-JEV-025;SRC-JEV-026;SRC-JEV-027"
    },
    {
      "id": "FND-JEV-004",
      "status": "VERIFIED",
      "impact": "ARCH CHANGE",
      "finding": "Jev is one fast semantic layer, not the policy engine or completion oracle.",
      "consequence": "Use it for bounded semantic questions inside a deterministic enforcement and escalation architecture.",
      "refs": "SRC-JEV-008;SRC-JEV-059"
    },
    {
      "id": "FND-JEV-005",
      "status": "VERIFIED",
      "impact": "ARCH CHANGE",
      "finding": "There are now multiple real-Jev transports and integrations.",
      "consequence": "Vercel, OpenRouter, LangChain/LangSmith and MCP integrations reduce deployment friction, but transport/version/data-policy differences remain relevant.",
      "refs": "SRC-JEV-001;SRC-JEV-003;SRC-JEV-009;SRC-JEV-010;SRC-JEV-011"
    },
    {
      "id": "FND-JEV-006",
      "status": "VERIFIED",
      "impact": "LEGAL",
      "finding": "TypeSafe outputs must not be used as imitation/distillation training targets under current terms.",
      "consequence": "Benchmark all evaluators against independent/human ground truth instead of training local models on Jev answers.",
      "refs": "SRC-JEV-005"
    },
    {
      "id": "FND-JEV-007",
      "status": "PROJECT CLAIM",
      "impact": "ARCH CHANGE",
      "finding": "Purpose-built local System-One families are broad enough for a real bake-off.",
      "consequence": "Mapika, Kev, Von, Laya, Zefan Open-Jev and poorjev represent distinct mechanisms and should be benchmarked separately.",
      "refs": "SRC-JEV-012;SRC-JEV-013;SRC-JEV-014;SRC-JEV-015;SRC-JEV-060;SRC-JEV-061"
    },
    {
      "id": "FND-JEV-008",
      "status": "VERIFIED",
      "impact": "ARCH CHANGE",
      "finding": "Direct option-logit scoring is a separate architecture, not merely 'structured LLM output'.",
      "consequence": "mini-Jev, dasein open-jev and parallel constrained decoding can avoid prose generation but still require token-bias controls and calibration.",
      "refs": "SRC-JEV-016;SRC-JEV-017;SRC-JEV-021"
    },
    {
      "id": "FND-JEV-009",
      "status": "VERIFIED",
      "impact": "ARCH CHANGE",
      "finding": "Long-prompt governance is primarily an obligation-accounting problem.",
      "consequence": "The benchmark must distinguish mentioned, represented/planned, satisfied and evidenced; long context alone does not guarantee obligation preservation.",
      "refs": "CASE-JEV-LONG-001"
    },
    {
      "id": "FND-JEV-010",
      "status": "VERIFIED",
      "impact": "ARCH CHANGE",
      "finding": "Quote-first obligation extraction is the correct source-of-truth pattern.",
      "consequence": "Derived normalized obligations preserve exact source spans/quotes, modality, conditions, exceptions, lifecycle and evidence type.",
      "refs": "SRC-JEV-035;OPT-REQ-001;OPT-REQ-003"
    },
    {
      "id": "FND-JEV-011",
      "status": "VERIFIED",
      "impact": "ARCH CHANGE",
      "finding": "Calibration/abstention is a first-class control layer.",
      "consequence": "Raw Jev distributions, NLI scores, logit shares and LLM self-confidence are not assumed to be probabilities of correctness on this project.",
      "refs": "SRC-JEV-037;SRC-JEV-038;SRC-JEV-039;SRC-JEV-040"
    },
    {
      "id": "FND-JEV-012",
      "status": "VERIFIED",
      "impact": "ARCH CHANGE",
      "finding": "Critical governance aggregation must be non-compensatory.",
      "consequence": "A critical veto cannot be averaged away by hundreds of green rules; expected loss and MCDM operate only after veto logic.",
      "refs": "OPT-POL-001;OPT-POL-002"
    },
    {
      "id": "FND-JEV-013",
      "status": "VERIFIED",
      "impact": "ADD/REFINE",
      "finding": "Trace-evaluation frameworks already solve much of process verification.",
      "consequence": "Promptfoo, MLflow, Phoenix, Inspect and DeepEval reduce custom infrastructure for tool-use, sequence and evidence checks.",
      "refs": "SRC-JEV-041;SRC-JEV-042;SRC-JEV-043;SRC-JEV-044;SRC-JEV-045;SRC-JEV-046;SRC-JEV-068"
    },
    {
      "id": "FND-JEV-014",
      "status": "VERIFIED",
      "impact": "ARCH CHANGE",
      "finding": "Critical natural-language rules can sometimes be compiled into formal derived artifacts.",
      "consequence": "ReqIF, EARS and FRET can improve persistence/normalization/monitoring while the verbatim OUP remains authoritative.",
      "refs": "SRC-JEV-047;SRC-JEV-048;SRC-JEV-049;SRC-JEV-050"
    },
    {
      "id": "FND-JEV-015",
      "status": "VERIFIED",
      "impact": "ARCH CHANGE",
      "finding": "Windows/Intel local evaluation should explicitly benchmark OpenVINO/OVMS.",
      "consequence": "Do not assume Linux-first vLLM results transfer to the user's Windows Intel GPU/NPU environment.",
      "refs": "SRC-JEV-051;SRC-JEV-052;SRC-JEV-053"
    },
    {
      "id": "FND-JEV-016",
      "status": "VERIFIED",
      "impact": "ARCH CHANGE",
      "finding": "Security is a separate defense-in-depth lane.",
      "consequence": "CaMeL/LlamaFirewall/guardrails address prompt/tool-output attacks; they do not replace authorization or governance compliance.",
      "refs": "SRC-JEV-028;SRC-JEV-029;SRC-JEV-054;SRC-JEV-069"
    },
    {
      "id": "FND-JEV-017",
      "status": "HYPOTHESIS",
      "impact": "ADD/REFINE",
      "finding": "Signed/hashed decision receipts are useful provenance even without adopting an unstable external protocol.",
      "consequence": "Use internal append-only hashes immediately; track emerging receipt protocols without depending on them.",
      "refs": "SRC-JEV-055"
    },
    {
      "id": "FND-JEV-018",
      "status": "DESIGN CONCLUSION",
      "impact": "ARCH CHANGE",
      "finding": "The decisive optimization target is critical false-negative risk under selective autonomy, not aggregate accuracy.",
      "consequence": "A high-average-accuracy evaluator that misses one critical obligation is unacceptable for the governance objective.",
      "refs": "SRC-JEV-038;SRC-JEV-039"
    },
    {
      "id": "FND-JEV-019",
      "status": "DESIGN CONCLUSION",
      "impact": "ARCH CHANGE",
      "finding": "Benchmark decision architectures, not only brand/model names.",
      "consequence": "Exact rules, encoders, NLI, open System-One, direct logits, native Jev and strong LLM judges should face the same frozen cases and human oracle.",
      "refs": "TST-JEV-017"
    },
    {
      "id": "FND-JEV-020",
      "status": "DESIGN CONCLUSION",
      "impact": "ROADMAP",
      "finding": "The next phase is experimental evidence, not another generic technology census.",
      "consequence": "Run CASE-JEV-LONG-001, native routes, local bake-off, exact-policy bake-off, calibration and adversarial/multilingual tests.",
      "refs": "RUN-JEV-01..10"
    }
  ],
  "conflicts": [
    {
      "id": "CON-JEV-001",
      "claim": "Jev context: 32K vs 64K",
      "state": "RESOLVED",
      "canonical": "Treat 32K as the state + longest-question / gateway context listing; some TypeSafe surfaces describe a larger aggregate request budget. Benchmark against the exact route and pinned version; never collapse the two numbers into one generic context limit.",
      "refs": "SRC-JEV-001;SRC-JEV-073"
    },
    {
      "id": "CON-JEV-002",
      "claim": "Qwen-2.5-1B-RLCD described as RLCD-trained",
      "state": "RESOLVED",
      "canonical": "Treat the currently inspected artifact as a parallel constrained decoding / KV-cache technique over stock Qwen unless a pinned weight artifact proves RLCD training. The filename is not evidence of training method.",
      "refs": "SRC-JEV-020;SRC-JEV-021"
    },
    {
      "id": "CON-JEV-003",
      "claim": "GLiNER2.5 called 'unlimited context/span'",
      "state": "RESOLVED",
      "canonical": "Boundary/span design can represent long spans within the encoder/chunking strategy; it is not an infinite-context semantic judge. Long documents still need segmentation or extract_long-style processing.",
      "refs": "SRC-JEV-035"
    },
    {
      "id": "CON-JEV-004",
      "claim": "Laya context and latency differ across reports",
      "state": "OPEN-BY-VERSION",
      "canonical": "No universal number is promoted. Pin exact Laya checkpoint/version and measure its actual tokenizer/context/hardware. Treat published latency/context values as project claims until reproduced.",
      "refs": "SRC-JEV-013"
    },
    {
      "id": "CON-JEV-005",
      "claim": "Von latency reported as <15, <25, or 25–300 ms",
      "state": "RESOLVED-AS-CLAIM",
      "canonical": "All are hardware/project measurements, not portable constants. v06 stores only 'encoder-limited; benchmark locally' as canonical operational guidance.",
      "refs": "SRC-JEV-014"
    },
    {
      "id": "CON-JEV-006",
      "claim": "Jev marketed as calibrated vs independent calibration concerns",
      "state": "OPEN-EMPIRICAL",
      "canonical": "Native distributions are useful but not assumed P(correct) on this governance domain. Require project-specific Brier/ECE/reliability/risk-coverage and optional recalibration.",
      "refs": "SRC-JEV-072;SRC-JEV-037;SRC-JEV-038"
    },
    {
      "id": "CON-JEV-007",
      "claim": "TypeSafe/Vercel ZDR surfaces disagree",
      "state": "OPEN-CONTRACT",
      "canonical": "Treat retention/ZDR as route-specific and unverified for confidential governance until exact endpoint contract is documented in writing.",
      "refs": "SRC-JEV-002;SRC-JEV-006;SRC-JEV-007;SRC-JEV-056"
    },
    {
      "id": "CON-JEV-008",
      "claim": "JevBench 231-task and 242-decision results differ",
      "state": "RESOLVED-AS-DIFFERENT-SNAPSHOTS",
      "canonical": "Do not merge them. Record benchmark commit/snapshot, task count and adapters separately. Compare only within the same frozen benchmark snapshot.",
      "refs": "SRC-JEV-019;SRC-JEV-020"
    },
    {
      "id": "CON-JEV-009",
      "claim": "'Cannot hallucinate'",
      "state": "RESOLVED",
      "canonical": "Closed-set/typed output prevents free-form fabrication outside the schema; it does not prevent selecting the wrong allowed answer.",
      "refs": "SRC-JEV-004;SRC-JEV-074"
    },
    {
      "id": "CON-JEV-010",
      "claim": "Vercel promo dates differ in secondary material",
      "state": "RESOLVED-OPERATIONALLY",
      "canonical": "Always record the live provider price at run start. Historical secondary dates are not authoritative. Current benchmark logs must carry the price snapshot and access timestamp.",
      "refs": "SRC-JEV-002;SRC-JEV-003"
    },
    {
      "id": "CON-JEV-011",
      "claim": "OpenJev name collision",
      "state": "RESOLVED",
      "canonical": "Use repository-qualified names: daseinlabs/open-jev, Zefan-Cai/Open-Jev, SemIf/OpenJev, mini-Jev, etc. Never cite 'OpenJev' alone.",
      "refs": "SRC-JEV-017;SRC-JEV-018;SRC-JEV-061"
    },
    {
      "id": "CON-JEV-012",
      "claim": "Search saturation",
      "state": "RESOLVED",
      "canonical": "Family-level saturation is medium-high; implementation-level saturation is medium; live-metric saturation is low. Generic census yields diminishing returns, but new projects continue to appear.",
      "refs": "SRC-JEV-019"
    },
    {
      "id": "CON-JEV-013",
      "claim": "Promptfoo acquisition / Braintrust exclusivity claims from Lumo",
      "state": "EXCLUDED-PENDING-PRIMARY",
      "canonical": "Not promoted into canonical architecture in v06 because no primary-source verification is present in the supplied canonical registries. Keep only as future search leads.",
      "refs": ""
    },
    {
      "id": "CON-JEV-014",
      "claim": "CASE-JEV-LONG-001 availability",
      "state": "RESOLVED-FOR-PROJECT",
      "canonical": "Some external research runs lacked the fixture; this conversation/project does contain the 30,609-byte source file. Remaining blocker is a reviewed gold obligation oracle, not fixture availability.",
      "refs": ""
    }
  ],
  "options": [
    {
      "id": "OPT-JEV-001",
      "family": "Cloud System-One",
      "name": "TypeSafe Jev 1.13",
      "stage": "fast semantic judge / routing",
      "locality": "Cloud",
      "license": "Commercial service terms",
      "context": "32K",
      "probability": "Native typed distributions",
      "integration": "TypeSafe API; OpenRouter; Vercel",
      "status": "VERIFIED",
      "fit": "High",
      "caveat": "Not ZDR by evidence found; service may update; legal restriction on distillation/imitator training.",
      "sources": [
        "SRC-JEV-001",
        "SRC-JEV-005",
        "SRC-JEV-006"
      ]
    },
    {
      "id": "OPT-JEV-002",
      "family": "Gateway",
      "name": "Vercel AI Gateway → Jev",
      "stage": "transport / observability",
      "locality": "Cloud",
      "license": "Gateway + TypeSafe terms",
      "context": "32K current listing",
      "probability": "Pass-through typed probabilities",
      "integration": "AI SDK; TypeSafe client; HTTP",
      "status": "VERIFIED",
      "fit": "High",
      "caveat": "Vercel provider directory currently labels TypeSafe AI as ZDR, but the model-specific Jev page examined leaves its ZDR cell blank. Treat ZDR as route/contract-specific until verified for the exact endpoint.",
      "sources": [
        "SRC-JEV-002",
        "SRC-JEV-003",
        "SRC-JEV-056"
      ]
    },
    {
      "id": "OPT-JEV-003",
      "family": "Gateway",
      "name": "OpenRouter → Jev 1.13",
      "stage": "transport / provider abstraction",
      "locality": "Cloud",
      "license": "OpenRouter + provider terms",
      "context": "32K",
      "probability": "Structured decisions",
      "integration": "OpenRouter API",
      "status": "VERIFIED",
      "fit": "High",
      "caveat": "Pin exact model ID for reproducibility; alias drift is unacceptable for governed runs.",
      "sources": [
        "SRC-JEV-001"
      ]
    },
    {
      "id": "OPT-JEV-004",
      "family": "Agent integration",
      "name": "LangChain Jev harness / middleware",
      "stage": "pre-tool / loop evaluation",
      "locality": "Cloud evaluator",
      "license": "Framework OSS + provider terms",
      "context": "Evaluator-dependent",
      "probability": "Jev typed distributions",
      "integration": "LangChain middleware",
      "status": "VERIFIED",
      "fit": "High",
      "caveat": "Middleware integration does not make semantic verdicts deterministic.",
      "sources": [
        "SRC-JEV-008"
      ]
    },
    {
      "id": "OPT-JEV-005",
      "family": "MCP integration",
      "name": "Composio Jev MCP",
      "stage": "agent access to evaluator",
      "locality": "Cloud MCP",
      "license": "Composio + TypeSafe terms",
      "context": "Jev-dependent",
      "probability": "Noul/Choice/Score",
      "integration": "Codex; Claude Code; other MCP clients",
      "status": "VERIFIED",
      "fit": "Medium-High",
      "caveat": "Adds a third-party control plane and credential/data path; evaluate privacy and failure modes.",
      "sources": [
        "SRC-JEV-009",
        "SRC-JEV-010",
        "SRC-JEV-011"
      ]
    },
    {
      "id": "OPT-S1-001",
      "family": "Open System-One",
      "name": "Mapika decider-2b v10",
      "stage": "local fast semantic judge",
      "locality": "Local/self-host",
      "license": "Apache-2.0 repo; base-model terms also apply",
      "context": "up to 32K project claim",
      "probability": "One-pass typed distributions; calibration-aware training",
      "integration": "TypeSafe-compatible HTTP",
      "status": "PROJECT CLAIM",
      "fit": "High experimental",
      "caveat": "Promising new family, but project benchmarks are not independent and hard-tier overconfidence remains a key risk.",
      "sources": [
        "SRC-JEV-012"
      ]
    },
    {
      "id": "OPT-S1-002",
      "family": "Open System-One",
      "name": "Mapika decider-35b-a3b",
      "stage": "stronger local typed judge",
      "locality": "Local/self-host",
      "license": "Apache-2.0 repo + base model",
      "context": "project-dependent",
      "probability": "One-pass typed distributions",
      "integration": "TypeSafe-compatible HTTP",
      "status": "PROJECT CLAIM",
      "fit": "Medium",
      "caveat": "Much heavier memory footprint; must benchmark on actual Intel/NVIDIA hardware before selecting.",
      "sources": [
        "SRC-JEV-012"
      ]
    },
    {
      "id": "OPT-S1-003",
      "family": "Encoder System-One",
      "name": "Laya typed decisions",
      "stage": "short-context classifier/judge",
      "locality": "Local",
      "license": "Repository/model-card terms",
      "context": "~512–1024 class depending checkpoint",
      "probability": "Encoder classification distributions",
      "integration": "Python/local service",
      "status": "PROJECT CLAIM",
      "fit": "Medium",
      "caveat": "Context ceiling prevents direct 30K-governance use; distribution-shift calibration failure is a central test case.",
      "sources": [
        "SRC-JEV-013"
      ]
    },
    {
      "id": "OPT-S1-004",
      "family": "Encoder System-One",
      "name": "Von",
      "stage": "short-context local judge",
      "locality": "Local",
      "license": "Apache-2.0 project claim",
      "context": "encoder-limited",
      "probability": "Typed classification",
      "integration": "Python/local",
      "status": "PROJECT CLAIM",
      "fit": "Medium",
      "caveat": "Needs independent accuracy/calibration reproduction and multilingual stress test.",
      "sources": [
        "SRC-JEV-014"
      ]
    },
    {
      "id": "OPT-S1-005",
      "family": "NLI System-One",
      "name": "poorjev",
      "stage": "CPU fallback / NLI judge",
      "locality": "Local CPU",
      "license": "MIT project claim",
      "context": "encoder-limited",
      "probability": "Temperature-scaled + conformal claim",
      "integration": "Python; MCP roadmap/project status evolving",
      "status": "PROJECT CLAIM",
      "fit": "Medium-Low until reproduced",
      "caveat": "Rapidly evolving repo and inconsistent historical status make commit pinning mandatory.",
      "sources": [
        "SRC-JEV-015"
      ]
    },
    {
      "id": "OPT-LOGIT-001",
      "family": "Direct logits",
      "name": "mini-Jev candidate-logit scoring",
      "stage": "cheap forced-choice baseline",
      "locality": "Local",
      "license": "MIT repo + base model terms",
      "context": "base-model dependent",
      "probability": "Next-token option logits; not calibrated by construction",
      "integration": "Transformers-style local",
      "status": "PROJECT CLAIM",
      "fit": "Medium",
      "caveat": "Tokenization/label bias; option-letter logit shares are not correctness probabilities.",
      "sources": [
        "SRC-JEV-016"
      ]
    },
    {
      "id": "OPT-LOGIT-002",
      "family": "Direct logits",
      "name": "daseinlabs/open-jev option-sequence scorer",
      "stage": "local option scoring",
      "locality": "Local",
      "license": "Repository/base-model terms",
      "context": "base-model dependent",
      "probability": "Softmax over candidate sequence likelihoods",
      "integration": "MLX / TypeSafe-compatible service",
      "status": "PROJECT CLAIM",
      "fit": "Medium-High",
      "caveat": "Length normalization, candidate tokenization, model/base choice and calibration all materially affect results.",
      "sources": [
        "SRC-JEV-017"
      ]
    },
    {
      "id": "OPT-LOGIT-003",
      "family": "Direct logits",
      "name": "SemIf / OpenJev browser scoring",
      "stage": "private local decision experiments",
      "locality": "Browser/local",
      "license": "Project-specific",
      "context": "model-dependent",
      "probability": "Direct option scores",
      "integration": "Web/browser",
      "status": "PROJECT CLAIM",
      "fit": "Experimental",
      "caveat": "Benchmarks need independent reproduction; browser execution has practical memory/performance limits.",
      "sources": [
        "SRC-JEV-018"
      ]
    },
    {
      "id": "OPT-ENC-001",
      "family": "Encoder / IE",
      "name": "GLiNER2 / GLiNER2.5",
      "stage": "obligation extraction / applicability / records",
      "locality": "Local",
      "license": "Apache-2.0 project/model terms",
      "context": "encoder-limited",
      "probability": "Per-label/task scores",
      "integration": "Python / HF",
      "status": "VERIFIED",
      "fit": "High for extraction, not whole-state judging",
      "caveat": "Per-label scores are not automatically a categorical posterior; long governance must be segmented/hierarchical.",
      "sources": [
        "SRC-JEV-035"
      ]
    },
    {
      "id": "OPT-ENC-002",
      "family": "Few-shot classifier",
      "name": "SetFit",
      "stage": "project-specific applicability/classification",
      "locality": "Local",
      "license": "Apache-2.0 framework; base-model terms",
      "context": "encoder dependent",
      "probability": "Classifier probabilities; calibrate separately",
      "integration": "HF/Python",
      "status": "VERIFIED",
      "fit": "Medium-High after labels",
      "caveat": "Requires representative project labels and shift monitoring.",
      "sources": [
        "SRC-JEV-036"
      ]
    },
    {
      "id": "OPT-ROUTE-001",
      "family": "Semantic routing",
      "name": "vLLM Semantic Router",
      "stage": "cascade / model selection",
      "locality": "Local or service",
      "license": "Project terms",
      "context": "router/model-dependent",
      "probability": "Routing scores",
      "integration": "vLLM ecosystem",
      "status": "VERIFIED",
      "fit": "High orchestration fit",
      "caveat": "Routing must be recall-first. Never allow a miss to suppress evaluation of critical rules.",
      "sources": [
        "SRC-JEV-034"
      ]
    },
    {
      "id": "OPT-STRUCT-001",
      "family": "Constrained decoding",
      "name": "XGrammar",
      "stage": "output shape enforcement",
      "locality": "Local/runtime library",
      "license": "Project terms",
      "context": "runtime-dependent",
      "probability": "N/A",
      "integration": "vLLM/SGLang/TensorRT-LLM/MLC/etc.",
      "status": "VERIFIED",
      "fit": "High contract layer",
      "caveat": "Guarantees allowed syntax/structure, not semantic truth, evidence, or calibration.",
      "sources": [
        "SRC-JEV-030"
      ]
    },
    {
      "id": "OPT-TYPED-001",
      "family": "Typed outputs",
      "name": "TypeChat",
      "stage": "typed evaluator contract",
      "locality": "Provider-portable",
      "license": "MIT",
      "context": "model-dependent",
      "probability": "Model-dependent",
      "integration": "TypeScript / schemas",
      "status": "VERIFIED",
      "fit": "Medium-High",
      "caveat": "Validation/retry can ensure conformance but does not prove semantic correctness.",
      "sources": [
        "SRC-JEV-031"
      ]
    },
    {
      "id": "OPT-TYPED-002",
      "family": "Typed outputs",
      "name": "Instructor",
      "stage": "typed evaluator contract / retries",
      "locality": "Provider-portable incl. local",
      "license": "Project terms",
      "context": "model-dependent",
      "probability": "Model-dependent",
      "integration": "Pydantic / multiple providers",
      "status": "VERIFIED",
      "fit": "High",
      "caveat": "Retries add latency/cost; semantic correctness still needs independent judge/evidence.",
      "sources": [
        "SRC-JEV-032"
      ]
    },
    {
      "id": "OPT-STRUCT-002",
      "family": "Schema validation",
      "name": "JSON Schema 2020-12",
      "stage": "deterministic contract validation",
      "locality": "Local",
      "license": "Standard",
      "context": "N/A",
      "probability": "N/A",
      "integration": "Universal",
      "status": "VERIFIED",
      "fit": "Very High",
      "caveat": "Structural validity only.",
      "sources": [
        "SRC-JEV-033"
      ]
    },
    {
      "id": "OPT-POL-001",
      "family": "Policy-as-code",
      "name": "OPA / Rego",
      "stage": "hard deterministic gate",
      "locality": "Local/sidecar/Wasm",
      "license": "Apache-2.0 project",
      "context": "Structured facts",
      "probability": "Deterministic",
      "integration": "HTTP/embedded/Wasm/CI",
      "status": "VERIFIED",
      "fit": "Very High",
      "caveat": "Semantic facts must be supplied by trusted extraction/evaluation; unsupported Wasm built-ins need host implementation.",
      "sources": [
        "SRC-JEV-021",
        "SRC-JEV-022"
      ]
    },
    {
      "id": "OPT-POL-002",
      "family": "Authorization",
      "name": "Cedar 4.5",
      "stage": "principal/action/resource authorization veto",
      "locality": "Embedded/service",
      "license": "Apache-2.0 project",
      "context": "PARC request + entity data",
      "probability": "Deterministic Allow/Deny",
      "integration": "Application authorizer",
      "status": "VERIFIED",
      "fit": "Very High for tool permission layer",
      "caveat": "Best for authorization-shaped rules, not arbitrary semantic requirements.",
      "sources": [
        "SRC-JEV-023",
        "SRC-JEV-024"
      ]
    },
    {
      "id": "OPT-POL-003",
      "family": "Agent-native policy DSL",
      "name": "Invariant Guardrails",
      "stage": "trace/data-flow/tool-call enforcement",
      "locality": "Local or gateway proxy",
      "license": "Project terms",
      "context": "Agent trace/events",
      "probability": "Rules deterministic; optional detectors probabilistic",
      "integration": "MCP/LLM proxy; Python",
      "status": "PROJECT CLAIM",
      "fit": "Very High experimental",
      "caveat": "Separate deterministic rule semantics from detector scores; test bypass/coverage.",
      "sources": [
        "SRC-JEV-025"
      ]
    },
    {
      "id": "OPT-POL-004",
      "family": "Agent-native policy DSL",
      "name": "AgentSpec",
      "stage": "runtime constraints",
      "locality": "Research implementation",
      "license": "Paper/code dependent",
      "context": "Agent events",
      "probability": "Rule enforcement",
      "integration": "Research prototype",
      "status": "VERIFIED",
      "fit": "High research input",
      "caveat": "ICSE research result; production readiness and ecosystem integrations must be independently assessed.",
      "sources": [
        "SRC-JEV-026"
      ]
    },
    {
      "id": "OPT-POL-005",
      "family": "Privilege policy",
      "name": "Progent",
      "stage": "least-privilege tool gate",
      "locality": "Research prototype",
      "license": "Research code terms",
      "context": "Tool policy + action",
      "probability": "Deterministic policy/fallback",
      "integration": "Agent tool layer",
      "status": "VERIFIED",
      "fit": "High research input",
      "caveat": "Research prototype, not a turnkey control plane.",
      "sources": [
        "SRC-JEV-027"
      ]
    },
    {
      "id": "OPT-SEC-001",
      "family": "Secure agent architecture",
      "name": "CaMeL",
      "stage": "capability/data-flow separation",
      "locality": "Local architecture",
      "license": "Research repo terms",
      "context": "Agent/tool flows",
      "probability": "N/A",
      "integration": "Architecture pattern",
      "status": "VERIFIED",
      "fit": "High concept",
      "caveat": "Research artifact warns it may contain bugs and is not a maintained Google product.",
      "sources": [
        "SRC-JEV-028"
      ]
    },
    {
      "id": "OPT-SEC-002",
      "family": "Security guardrails",
      "name": "LlamaFirewall",
      "stage": "prompt injection / misalignment / code scan",
      "locality": "Local/serviceable",
      "license": "Project terms",
      "context": "Messages/code/traces",
      "probability": "Scanner-dependent",
      "integration": "Agent guardrail layer",
      "status": "VERIFIED",
      "fit": "Medium-High defense-in-depth",
      "caveat": "Not a replacement for authorization or deterministic governance gates.",
      "sources": [
        "SRC-JEV-029",
        "SRC-JEV-054"
      ]
    },
    {
      "id": "OPT-CAL-001",
      "family": "Calibration",
      "name": "Temperature scaling",
      "stage": "post-hoc probability calibration",
      "locality": "Local",
      "license": "Method",
      "context": "Held-out labeled decisions",
      "probability": "Calibrated logits",
      "integration": "Any scorer exposing logits/probs",
      "status": "V
... [display truncated; full JSON remains embedded below]

About v10

v07 authorship & project identity

Author / Project Manager: Karolis Valickas. Project codename: “We have Jev at home” :D. Authorship is visible here and in the footer/header metadata, and is also embedded in JSON-LD, XML and hidden document metadata.

AI-oriented document preparation

There is no universal web standard saying that an HTML research report should be “XML-tagged for AI.” v07 therefore uses a layered approach instead of pretending one exists: semantic HTML and accessible headings/tables; persistent immutable IDs; Schema.org JSON-LD authorship/CreativeWork metadata; the existing machine-readable JSON registry; data-ai-*/data-wbs attributes; and a hidden <script type="application/xml" id="ai-document-map"> semantic map. XML-style structure is particularly useful when this document is fed into assistants as context, while ordinary semantic HTML remains the canonical browser structure.

AI map ID: ai-document-map · JSON state ID: jev-v07-data · Schema.org metadata ID: schema-org-metadata.

Board-orientation layer

v08 adds 0 Start Here without renumbering sections 1–10. It is deliberately decision-oriented: which tab answers which executive question, what to read first, and what not to read yet.

Benchmark cap

The tournament catalog is capped at 100 cases. Future discoveries normally enter as mutations/evidence/replacements unless they reveal a genuinely new decision family or modality.

Benchmark Tournament Lab

flowchart TD
A[Real use-case seed] --> B[Freeze task + labels + evidence]
B --> C{Tournament family}
C --> R[Routing & triage]
C --> S[Scoring & priority]
C --> G[Tool & action gates]
C --> L[Agent-loop decisions]
C --> COV[Completion & governance]
C --> D[Documents & content]
C --> RT[Real-time control]
C --> F[Finance & risk]
C --> ROB[Robustness & calibration]
C --> SYS[Scale & economics]
R --> E[Common evaluator adapters]
S --> E
G --> E
L --> E
COV --> E
D --> E
RT --> E
F --> E
ROB --> E
SYS --> E
E --> M[Raw metrics + evidence]
M --> P[Pareto frontier by tournament]
P --> H[Homebrew recipe / deployment choice]

The lab now contains 100 benchmarks in 10 comparable tournaments, the requested cap. v08 preserves the original 80 and adds 20 official/open/academic cases.

sequenceDiagram
autonumber
actor U as Human benchmark owner
participant S as Source/seed
participant H as Homebrew harness
participant O as Human/gold oracle
participant E as Evaluator adapters
participant M as Metrics
U->>S: Select sourced use case
S-->>H: State shape + bounded questions + edge cases
H->>O: Build independent labels/evidence
O-->>H: Frozen gold set
loop every entrant
H->>E: Same state/questions/case IDs
E-->>H: Raw typed results + latency + usage
end
H->>M: Compare against gold
M-->>U: FNR/FPR + calibration + coverage + latency + cost
U->>U: Select Pareto candidate, not headline winner

40 source-derived benchmark seeds

The original 20 v07 seeds are preserved; v08 adds 20 official/open/academic seeds, led by TypeSafe cookbooks and Grok-discovered reproducible workloads.

TRN-JEV-A · Routing & triage

flowchart LR
A[Message / form / document / task] --> B[Defined destinations or rubric]
B --> C[Evaluator returns typed answer + probability]
C --> D{Above automation threshold?}
D -- yes --> E[Act automatically]
D -- no --> F[Fallback model / human]
E --> G[Record corrected outcome later]
F --> G
G --> H[Confusion matrix + calibration + review load]

TRN-JEV-B · Priority & scoring

flowchart LR
A[Message / form / document / task] --> B[Defined destinations or rubric]
B --> C[Evaluator returns typed answer + probability]
C --> D{Above automation threshold?}
D -- yes --> E[Act automatically]
D -- no --> F[Fallback model / human]
E --> G[Record corrected outcome later]
F --> G
G --> H[Confusion matrix + calibration + review load]

TRN-JEV-C · Tool & action gates

sequenceDiagram
autonumber
participant A as Main agent
participant X as Exact policy
participant J as Fuzzy judge
participant T as Tool runner
participant H as Human
A->>X: Proposed tool + args
alt Exact deny/allow
X-->>A: Deterministic decision
else Fuzzy risk
X->>J: Tool state + clear/caution criteria
alt Clear
J-->>T: Auto approve
else Caution / uncertain
J->>H: Approval packet
H-->>T: Approve / reject
end
end
T-->>A: Result + trace