Evidence & Trust
Every star above 1★ is gated by evidence — not declaration. This page covers the full evidence system: the deprecated single-axis Evidence Class, the current two-axis model (Evidence Type + Evidence Grade), how scores accumulate into an Overall Trust Grade, and how to attach evidence to a skill via the CLI.
Overview
Gaia uses evidence to verify that a skill is real, reproducible, and battle-tested. Evidence is attached to a skill node and is the primary input to star progression — a skill's stars are derived from its evidence, never declared.
The evidence system was revised with the #646 trust model. The old single-letter Evidence Class conflated two orthogonal questions — where evidence comes from and how good it is — into one field. The new model separates them: Evidence Type (provenance) and Evidence Grade (quality).
Critical: The deprecated Class letters (C/B/A) and the new Grade letters (S/A/B/C) are not equivalent. Class A ≠ Grade A. See Common pitfalls.
Deprecated: Evidence Class
The legacy Evidence Class was a single letter covering both provenance and quality:
| Class | Description | Typical source |
|---|---|---|
C | First sighting — skill observed at least once in the wild. | GitHub repo, demo, casual reference |
B | Reproducible — reliably reproducible with documentation. | SKILL.md with examples, recorded benchmarks |
A | Battle-tested, peer-reviewed — widely used and externally validated. | arXiv paper, widely-used library |
The problem: a GitHub repo (Class B) with 100k stars carries far more confidence than one with 10 — but both mapped to the same letter. The new model separates these signals.
Evidence Type
Evidence Type is the provenance of one demonstration — where it comes from,
not how good it is. Types are kebab-case strings driven from meta.json → evidence.types,
so new sources extend the list without a schema change.
Always write the full phrase Evidence Type; never the bare word "type", which refers to the Basic / Extra / Unique / Ultimate taxonomy field.
| Type value | What it represents | URL format / flags |
|---|---|---|
| repo-own | Artifact volume from the contributor's own repository — commit and contributor counts. | Must use blob/branch/subpath — never tree/. Pass --commits / --contributors. |
| github-stars-own | Stars on the skill's own repository. Discounted when the repo bundles multiple skills (mothership discount). | Bare repo URL: https://github.com/owner/repo. Pass --stars and, if applicable, --skill-count-in-repo. |
| arxiv | Academic citation count for a published paper demonstrating the capability. | Abstract URL: https://arxiv.org/abs/<id> — not the PDF link. Pass --citations. |
| peer-review | Structured walkthrough with written critique by 4★+ reviewers — distinct from a Verifier attestation. | URL to the review thread/writeup. Pass --reviewers. |
| verifier-attestation | A 4★+ Verifier signs an attestation that the skill is real and graded fairly. | URL to the attestation record. No self-attestation — the evaluator must hold a Verifier-tier Named Skill. |
| benchmark-result | A reproducible benchmark on a public dataset, scored as percentile rank within the field. | URL to the benchmark/leaderboard entry. Requires a percentile value (0–100) — a raw score alone does not resolve to a magnitude. |
| fusion-recipe | Auto-derived for every fused/Ultimate skill from its suiteComponents. Only origins graded ≥C count toward the tally. |
Not hand-submitted — written by gaia dev fuse / merge when a suite skill is built. |
| proxy-containment | The skill's artifact is bundled inside a separately-starred external project. | URL to the containing project. Max 3 entries per skill; magnitude plateaus on repeated entries. |
| social-signal | Creator/audience evidence — video views, engagement, topical authority from external creators. | URL to the video/post. Pass --views; a view-count floor applies before this counts toward any magnitude. |
| self-attestation | Contributor declaration that the skill exists and is real. Flat, low magnitude — a placeholder, not proof. | Any URL documenting the claim. Max one entry per skill. |
The full list lives in meta.json → evidence.types, alongside the magnitude formula and cap for each. New types extend that list without a schema version bump.
Evidence Grade
Evidence Grade is the quality of one demonstration on an S / A / B / C axis — Platinum (S), Gold (A), Silver (B), Bronze (C) — derived from a numerical trust number. A demonstration whose trust number falls below the C threshold is ungraded: on the record but counting toward no gate.
| Grade | Label | Trust # | What qualifies |
|---|---|---|---|
| S | Platinum | ≥ 250 | Definitive, peer-reviewed, widely-cited evidence. Reserved for capabilities established beyond reasonable doubt in literature — extremely rare. |
| A | Gold | ≥ 100 | Battle-tested in production; corroborated by multiple independent sources. A library with 10k+ stars backed by a SKILL.md and a supporting paper qualifies. |
| B | Silver | ≥ 50 | Reproducible and documented. A SKILL.md with working examples and a verifiable demo at 100+ stars typically lands here. |
| C | Bronze | ≥ 20 | First sighting — observed at least once, basic documentation present. The minimum to count toward Named (2★) status. |
| — | Ungraded | < 20 | On the record but does not count toward any star gate. Useful for logging an observation without making a quality claim. |
Trust Number
The trust number is an internal 0–100 score that drives grade assignment. It is never user-facing — public surfaces show the grade label (S/A/B/C) that the trust number yields, not the raw score. Do not call it "trust score"; the canonical term is trust number.
Thresholds live in meta.json → evidence.gradeThresholds and can be
recalibrated without a schema change. The trust number is computed by the build pipeline from
evidence metadata (type, external signals like star counts, verification state) and is not stored
in registry nodes.
Overall Trust Grade
An individual evidence entry has its own Evidence Grade. A skill's aggregate standing is called its Overall Trust Grade — the accumulation of all its evidence grades that establishes the capability "beyond reasonable doubt."
- Computed at build time from the full evidence inventory — never stored on a registry node.
- Materialises only in generated catalogs (
registry/named-skills.json,docs/graph/gaia.json). - Distinct from a single entry's grade — three Bronze entries can yield a Gold Overall Trust Grade in aggregate.
- Never hand-assigned — writing this field to a node directly violates Programmatic-First and CI will reject the PR.
Verification States
Each evidence entry passes through verification states independently of its grade. Verification attests that a demonstration is real; grading measures how strong it is. The two axes never substitute for each other.
evidence.verified: false (default)
On the record but not yet reviewed by a 4★ Verifier. All new entries start here.
evidence.verified: true
Confirmed real by a contributor holding a 4★ (Extra/Unique) or higher Named Skill. Unlocks the Community Verified certification level.
evidence.disputed: true
A Verifier has flagged the entry as questionable. It stays on the record with a demerit flag in the catalog until resolved.
Check your Verifier status with gaia whoami. The via: verifier path means you are authorized to set evidence.verified: true or evidence.disputed: true.
Adding Evidence via CLI
Use gaia dev evidence to attach evidence to a skill node. This is a Verifier-gated command — confirm authorization with gaia whoami first.
Legacy form (Class — deprecated, still accepted)
gaia dev evidence registry-curation "https://github.com/owner/repo/blob/main/skills/foo/SKILL.md" --class B
New form (Type + Trust)
Grade is never set directly — it is auto-derived from the --trust number you pass
(S≥250, A≥100, B≥50, C≥20; below 20 the entry stays on record but ungraded).
# repo-own evidence, trust 50 → Grade B (Silver) — always use blob/, not tree/ gaia dev evidence registry-curation "https://github.com/owner/repo/blob/main/skills/foo/SKILL.md" \ --type repo-own \ --trust 50 \ --commits 400 --contributors 6 # arXiv paper, trust 100 → Grade A (Gold) gaia dev evidence multimodal-reasoning "https://arxiv.org/abs/2504.00001" \ --type arxiv \ --trust 100 \ --citations 500 # verify an entry (Verifier-only — a separate subcommand, not a flag on `evidence`) gaia dev verify registry-curation --index 0 # dispute an entry (Verifier-only) gaia dev verify registry-curation --index 0 --dispute
Check gaia whoami before writing. Evidence writes directly to a registry node under Verifier authorization — a bad URL or wrong skill ID produces invalid JSON that will fail CI. Pass --no-build to skip the doc/graph rebuild while iterating.
URL format rules per type
repo-own: Useblob/branch/subpath. GitHub directory URLs usetree/— convert manually. The resolver rejectstree/with a schema error.arxiv: Use the abstract URL (https://arxiv.org/abs/<id>), not the PDF link. The build pipeline normalises to the abstract for deduplication.github-stars-own: Bare repo URL (https://github.com/owner/repo). Star count is read live at build time.
Migration Guide
When auditing nodes with legacy class: A/B/C and converting to the new model,
use this table as a starting point. The mapping is approximate — the correct Grade
depends on the actual evidence, not just the old Class letter.
| Deprecated Class | Approx. Type | Approx. Grade range | Notes |
|---|---|---|---|
class: C |
repo-own | C Bronze | First sighting. Verify the URL still resolves before converting. |
class: B |
repo-own | C – B | Grade B if repo has examples + demos + ≥100 stars. Grade C if thin. Check star count at migration time. |
class: A (arXiv) |
arxiv | B – A | Confirm the URL before assigning type. High-citation papers with reproduction evidence can reach Grade A. |
class: A (repo) |
repo-own + github-stars-own | A – S | Battle-tested repos (10k+ stars, independent reproductions). Verify evidence still exists before upgrading. |
Do not auto-migrate. Converting class: A to grade: A is always wrong — the letters are not equivalent. Each entry must be evaluated individually.
Common Pitfalls
Class letters ≠ Grade letters
The single most common mistake. The axes share letters but measure different things at different calibrations:
| Old Class | Old meaning | Grade | Why they differ |
|---|---|---|---|
C | First sighting | C Bronze | Closest mapping — both mean first sighting. |
B | Reproducible | B Silver | Similar concept, but Grade B requires a higher trust number than Class B implied. |
A | Battle-tested, peer-reviewed | A Gold | Class A was the top of a 3-rung ladder. Grade A is second-highest on a 5-rung ladder. Converting 1:1 is a downgrade for some skills, an upgrade for others. |
| — | (no equivalent) | S Platinum | Grade S (≥ 250) has no Class equivalent — it represents certainty that Class A rarely achieved. |
Using tree/ instead of blob/
GitHub's directory view uses tree/branch/path. The evidence resolver and installer only accept blob/branch/path. The CLI rejects tree/ URLs with a schema error.
Using github-stars-own as sole evidence
Star counts are volatile and subject to farming. Never use github-stars-own as the sole evidence for a skill — always pair it with at least one repo-own entry pointing to a SKILL.md.
Passing --grade instead of --trust
gaia dev evidence has no --grade flag — Grade is always auto-derived from the --trust number you supply (see thresholds above). There is also no --dry-run mode for this subcommand; it writes directly to the registry node under Verifier authorization, so confirm the skill ID and URL before running it. Pass --no-build to skip the doc/graph rebuild while iterating.