# PlatPhorm Evals

> Registry-driven evaluation mesh for the PlatPhorm ecosystem.

## Product Role
- Evals is the evaluation brain for PlatPhormNews.
- It defines, discovers, runs, grades, compares, publishes, and gates evaluations across the network.
- Integrations such as MCP, Claws, Spec, Sandbox, BrowserOps, AgentUI, Trace, Docs, Sheets, Catalog, and Monitor are evaluated by Evals; they are not Evals' product identity.

## Live Registry Counts
- Services tracked: 216
- Active suites: 11
- Capabilities indexed: 802
- Eval runs today: 0
- Count source: merged

## Public API
- Health: GET https://evals.platphormnews.com/api/health
- Versioned health: GET https://evals.platphormnews.com/api/v1/health
- API docs: GET https://evals.platphormnews.com/api/docs
- OpenAPI YAML: GET https://evals.platphormnews.com/openapi.yaml
- OpenAPI JSON: GET https://evals.platphormnews.com/openapi.json
- MCP descriptor: GET https://evals.platphormnews.com/api/mcp
- Registry status: GET /api/v1/registry/status
- Targets: GET /api/v1/targets
- Capabilities: GET /api/v1/capabilities
- Suites: GET /api/v1/suites
- Runs: GET /api/v1/runs
- Scorecards: GET /api/v1/scorecards
- Release gates: GET /api/v1/release-gates
- Integrations: GET /api/v1/integrations/status
- Agent policy: GET /api/v1/agent-policy

## Public Evaluation Endpoints
- Discovery eval: POST /api/v1/evaluate/discovery
- OpenAPI eval: POST /api/v1/evaluate/openapi
- MCP eval: POST /api/v1/evaluate/mcp
- AgentUI, Claws, workflow, regression evals return honest degraded states unless a real provider run is configured.

## Protected Actions
- Registry sync: POST /api/v1/registry/sync
- Suite create/update/delete: POST/PATCH/DELETE /api/v1/suites
- Expensive BrowserOps, Sandbox, LLM judge, release gate, and report export actions require PLATPHORM_API_KEY.
- Accepted auth: Authorization: Bearer $PLATPHORM_API_KEY or X-PlatPhorm-API-Key: $PLATPHORM_API_KEY.

## Public MCP Tools
- get_eval_info
- get_dashboard
- get_registry_status
- list_targets
- get_target
- list_capabilities
- get_capability
- list_suites
- get_suite
- list_runs
- get_run
- get_run_results
- get_run_evidence
- get_scorecard
- list_templates
- get_template
- list_benchmarks
- get_integration_status
- evaluate_discovery
- evaluate_openapi
- evaluate_mcp
- evaluate_agent_policy
- get_agent_policy
- list_agent_platforms
- get_agent_platform
- get_health
- get_route_compliance
- get_discovery_compliance
- compare_eval_runs
- detect_regressions

## Protected MCP Tools
- sync_registry
- sync_network_registry
- import_targets_from_mcp
- import_targets_from_spec
- create_eval_suite
- create_suite
- update_suite
- delete_suite
- generate_eval_cases
- create_dataset
- create_grader
- run_eval_suite
- run_suite
- rerun_eval
- cancel_eval
- evaluate_browserops
- evaluate_sandbox
- evaluate_claws
- evaluate_agentui
- evaluate_workflow
- evaluate_llm_judge
- run_release_gate
- generate_scorecard
- gate_release
- publish_scorecard
- create_docs_report
- create_sheet_report
- create_deck_summary
- update_agent_policy

## Templates
- required-route-compliance: Evaluate all required public discovery and platform routes.
- sitemap-no-dead-link-compliance: Validate sitemap URLs and ensure no protected mutation route is listed.
- rss-feed-validity: Validate RSS and Atom public-safe feed output.
- llms-readability: Check llms.txt, llms-full.txt, and llms-index.json readability and counts.
- openapi-validity: Validate OpenAPI schema, auth, examples, and public/protected route clarity.
- mcp-initialize-list-call: Validate public MCP JSON-RPC methods and protected rejection.
- protected-action-rejection: Confirm protected operations require PLATPHORM_API_KEY.
- agentui-form-render: Render MCP tool schemas and inspect required fields.
- browserops-page-load: Run page-load and no-stuck-loading checks through BrowserOps.
- sandbox-command-execution: Run a bounded command and validate real command output.
- claws-tool-orchestration: Validate public Claws tool registry and orchestration metadata.
- docs-publish: Publish a public-safe evaluation methodology or report to Docs.
- sheets-export: Export run matrices and scorecards to Sheets.
- opencontent-ingest: Validate OpenContent public submit and docs export workflow.
- dictionary-search-submit: Check dictionary search and protected submission boundary.
- emoji-score-proposal: Check emoji scoring and protected proposal flow.
- reader-translation: Check public reader translation degraded states and protected actions.
- echo-content-monitor: Check content monitor public status and alert delivery boundaries.
- calendar-kanban-task-flow: Evaluate task handoff across calendar and kanban surfaces.
- platphorm-cli-harness-run: Validate CLI-generated evidence against Evals scorecard requirements.

## Benchmarks
- platform-contract-baseline: Required public route, discovery, auth, and policy checks for every trusted site.
- mcp-tool-quality: JSON-RPC correctness, schema validity, public-safe introspection, and protected action rejection.
- release-control: Scorecard and gate criteria for release promotion decisions.

## Public Pages In Sitemap
- /
- /dashboard
- /registry
- /targets
- /capabilities
- /suites
- /runs
- /scorecards
- /release-gates
- /benchmarks
- /templates
- /integrations
- /methodology
- /docs
- /faq
- /api/docs
- /openapi.yaml
- /openapi.json
- /llms.txt
- /llms-full.txt
- /llms-index.json
- /robots.txt
- /.well-known/agents.json
- /.well-known/agent-policy.json
- /.well-known/ai-policy.json
- /.well-known/trust.json
- /.well-known/security.txt
- /.well-known/mcp.json

## Evidence Rules
- Deterministic checks take precedence over model-assisted grades.
- Model-assisted grades must be labeled with prompt, version, and model when configured.
- BrowserOps screenshots, Sandbox output, Claws workflow results, and release approvals are never fabricated.
- Raw x-vercel-ja4-digest is fingerprint-adjacent metadata and is redacted or hashed before public display.