Measured and reproducible.

A benchmark number is only as credible as the command, configuration, and environment that produced it. Below are the reproducibility notes for each project. Missing or preliminary numbers are labeled, not hidden.

Benchmark classification

Synthetic microbenchmark: No I/O, no network, no persistence. Measures pure computation.
Test-suite benchmark: Runs as part of the test suite. Measures in-process performance.
End-to-end benchmark: Includes HTTP, signing, persistence. Closer to real dispatch.

OVARA

Under Development

Policy evaluation microbenchmarks

In-process testing.B microbenchmarks for policy evaluation.

Type: Synthetic microbenchmark - in-process
Source: runtime/gateway/internal/handlers/benchmark_test.go
Excludes: HTTP, network, persistence, receipt signing.

Reproduce

cd OVARA/runtime/gateway && go test -bench=. ./internal/handlers/

HTTP load test

In the current HTTP load test, approximately 4,010 requests/sec were observed, with approximately 19.76% errors under the tested workload. p50 ~3 ms, p99 ~98 ms.

Type: End-to-end HTTP load test
Source: runtime/gateway/tests/load/load_test.go
Caveat: This result should not be interpreted as a production throughput guarantee. The error rate indicates tuning is needed.

Reproduce

cd OVARA/runtime/gateway && go test ./tests/load/ -run TestLoad

No published reproducibility artifact (hardware spec, pinned config, raw output). Not independently reproduced.

MAP

Released

Policy evaluation, no I/O

MAP synthetic microbenchmark results
Measurement	Result
Policy evaluations (5 rules)	~2.47M/sec
Policy evaluations (100 rules)	~599K/sec
Policy evaluations (500 rules, worst-case scan)	~61K/sec
Full executions (no I/O)	~163K/sec
Approval decisions	~288K/sec

Type: Synthetic microbenchmark - no I/O
Source: src/test/benchmark.test.ts
Excludes: Network, adapter execution, signing, persistence, approval workflow, external services. These numbers measure pure policy evaluation in process. They are not representative of production dispatch throughput.

Reproduce (benchmark subtests)

cd micro-agent-protocol && npm run build && npm test

No end-to-end agent execution benchmark published. Not independently reproduced.

DASH

Working / Active Development

Performance benchmark

DASH performance benchmark results
Measurement	Result
Semantic retrieval	~90 ops/sec
WAL-bound ingest	~133 ops/sec
ANN build, 10k vectors	~30s

Type: End-to-end performance benchmark
Source: docs/benchmarks/performance.md
Method: Hardware, release build, single-threaded, fixtures, and percentile methodology disclosed in the source doc. perf_bench emits BENCH_JSON: machine-readable results.

Reproduce

cd DASH && cargo run -p benchmark-smoke --bin perf_bench --release -- --all

Large-scale targets (1B claims, 99.95% availability) are design goals, not measured. Not independently reproduced.

POLICY.md

Working Specification

No benchmark suite. 21 vitest tests pass (validate, agentsmd, policyManager, installer, codex adapter).

Reproduce

cd POLICY.md && npm test

Sidian

Beta

No reproducible benchmark or build artifact is currently public. The product is live at sidian.dev but its internals cannot be independently inspected.

Read with

Claims & evidenceEvery public claim, its status, and the source that supports it.Non-guaranteesWhat each project explicitly does not do.Threat modelWhat the stack protects against, what it records, and where it stops.