# Assigner scorecard

Vulnerability-Lookup ranks the numbering authorities whose records it holds by how
well they fill them. Every CVE Numbering Authority (CNA) of the CVE Program and every
GCVE Numbering Authority (GNA) gets a score, a grade and a badge, computed from its
own publications against the rules on this page.

The scorecard is at `/assigners/scorecard` (CNAs) and `/assigners/scorecard?kind=gna`
(GNAs); each assigner has a page at `/assigners/<name>` and a badge at
`/assigners/<name>/badge.svg`. The same figures are served by
`/api/stats/scorecard/*`.

## What is scored

A record is scored on its **CNA container only**: the part of a CVE JSON 5 record the
assigner wrote. A CVSS or a CWE added afterwards by an Authorized Data Publisher (ADP)
such as CISA's vulnrichment is not the assigner's work. It is counted separately, as
*enrichment dependency*, and shown next to the score: a CNA whose severity is always
supplied by someone else is a different fact from one that states none, and the two
must stay tellable apart.

Both CNAs and GNAs publish CVE JSON 5 records, so one engine scores both. A CNA is
identified by the record's `assignerShortName`; a GNA is a source of its own
(`gna-<id>`) whose every record is its own work, whatever the record says.

## The rules

Each rule returns a credit between 0 and 1. A record's score is the weighted sum, out
of 100; an assigner's score is the mean over its records in the period. Four pillars,
25 points each:

**Describe** -- can a reader tell what the vulnerability is?

| Rule | Points | Credit |
| --- | --- | --- |
| Description | 10 | An English description that is not a placeholder: 1.0 from 200 characters, 0.7 from 80, 0.4 from 40, 0 below. |
| Title | 5 | A title. |
| CWE | 10 | A weakness as a CWE identifier in `problemTypes`; a free-text problem type without one earns 0.25. |

**Scope** -- can a reader tell whether they are affected?

| Rule | Points | Credit |
| --- | --- | --- |
| Affected | 10 | An affected entry naming both a vendor and a product; one side missing or `n/a` earns 0.5. |
| Versions | 8 | An affected entry stating versions, as a version or a bound; a `defaultStatus` alone earns 0.5. |
| CPE | 7 | A CPE on an affected entry, or a `cpeApplicability` block. |

**Assess** -- can a reader tell how bad it is?

| Rule | Points | Credit |
| --- | --- | --- |
| Severity | 15 | A severity statement in the CNA container: a CVSS v3 or v4 vector, an SSVC decision, or a vendor rating as an `other` metric. A CVSS v2 vector alone earns 0.5. |
| Modern severity | 5 | A CVSS v4 vector or an SSVC decision. |
| Impacts | 5 | An `impacts` entry: a CAPEC identifier or an impact description. |

**Act** -- can a reader tell what to do about it?

| Rule | Points | Credit |
| --- | --- | --- |
| References | 7 | At least one reference. |
| Reference tags | 5 | At least one reference carrying tags. |
| Remediation | 8 | A reference tagged `patch`; one tagged `vendor-advisory`, `mitigation` or `release-notes` earns 0.6. |
| Solutions | 5 | A solution or a workaround. |

Grades from the mean score: **A** from 90, **B** from 75, **C** from 60, **D** from 45,
**E** below.

Three choices behind these rules are deliberate:

- **Severity is any severity statement, not CVSS.** Several active CNAs (curl,
  OpenSSL, Go, Apple, Mozilla, Chrome...) publish no CVSS as a matter of policy. A
  scorecard reading that as negligence would rank the strength of an opinion about
  CVSS rather than the care put into a record. A record that states no severity of
  any kind still earns nothing here.
- **Rules are graded, not binary.** A description of forty characters and one of
  four hundred are not the same record, and a check that only asks whether the field
  is non-empty awards full marks for one-line descriptions.
- **Timeliness is displayed, never scored.** The delay between a reservation and its
  publication is a coordinated-disclosure embargo as often as it is a backlog, and
  no rule can tell the two apart. The median and 90th percentile are shown on each
  assigner's page as context.

## Ranking, window and threshold

The ranking is built on a **rolling window** of the last 180 days of publications,
so it reflects current practice rather than a decade's. An assigner needs **10
publications in the window** to be ranked; below that its score is shown and marked
unranked, because three perfect records are not evidence of a practice and a
leaderboard of them would be won by whoever publishes least. Both figures are
options of the preparing pass and are reported with every answer.

Each assigner's page also shows its all-time tallies and its score per publication
year, which is where a CNA's progress (or the opposite) is visible.

## Context figures

Shown beside the score and never in it:

- **Severity / weakness supplied by an ADP**: the share of records whose CVSS or CWE
  was added by an ADP because the CNA container carries none.
- **No severity statement**: the share of records with no severity of any kind.
- **Descriptions under 100 characters.**
- **Known exploited**: records listed by a known-exploited catalogue, and how many of
  them were published without a severity statement.
- **Reservation to publication**: median and 90th percentile, in days.
- **Rejected records** in the period.
- For GNAs: how many records state the CVE they correspond to (or a relationship to
  one), and how many still use the legacy identifier placement of GCVE-BCP-05.

## Hall of fame, wall of shame, badges

The **hall of fame** lists the ranked assigners graded A.

The **wall of shame** does not list the bottom of the ranking, which would name
nothing to change. It lists named practices with an objective threshold, over the
window, among the ranked assigners:

- no severity statement on any record;
- no CWE on any record;
- severity supplied by an ADP for more than half the records;
- descriptions under 100 characters on more than 30% of the records;
- affected products missing or `n/a` on more than half the records;
- known-exploited vulnerabilities published without a severity statement.

It is **off by default**: naming publishers is a stance an instance takes
deliberately. Enable it with `WEB_MODULES["wall_of_shame"] = True` in
`config/website.py`. The ranking, the grades, the badges and the per-assigner pages
are shown either way, and the signals stay readable from
`/api/stats/scorecard/signals`.

Each assigner has an embeddable **badge** at `/assigners/<name>/badge.svg` showing
its grade and score over the window. It reads "unranked" under the threshold rather
than failing, so a README embedding it does not break the day its publisher goes
quiet. The assigner's page gives the Markdown to paste.

## The API

| Endpoint | Answers |
| --- | --- |
| `GET /api/stats/scorecard/ranking?source=cvelistv5[&period=window\|all\|YYYY][&limit=N][&output=markdown]` | Every assigner of the source, ranked ones first, best first, with the preparation metadata. |
| `GET /api/stats/scorecard/gnas[&period=...]` | The GNAs ranked against each other, one entry per GNA source with a prepared scorecard. |
| `GET /api/stats/scorecard/assigner?source=cvelistv5&assigner=<name>` | One assigner: credit rate and points per rule, pillars, context figures, the per-year trend. |
| `GET /api/stats/scorecard/signals?source=cvelistv5` | The wall of shame signals and the ranked assigners tripping each. |

For a GNA, `source` is its source name and `assigner` the same name, e.g.
`?source=gna-1988&assigner=gna-1988`. Like every prepared figure, an unprepared
scorecard answers `available: false` rather than an empty leaderboard, and the
methodology carries a version: figures prepared under older rules are reported as
unavailable rather than read against rules they were not computed with.

## Preparing the figures

Nothing on the read side walks a record. The figures are prepared by

```bash
# the CVE list and every GNA source with records, whole history
$ poetry run scorecard

# only the rolling window the ranking is built on: a few tens of thousands of
# records rather than the corpus, the cadence to run hourly
$ poetry run scorecard --window-only

# one source, and the two knobs every answer reports
$ poetry run scorecard --source cvelistv5 --window-days 180 --min-records 10
```

`index_vulnerabilities` runs the full pass as part of every reindex, so an instance
that reindexes nightly needs only the hourly refresh. Both replace the previous
figures in one transaction, so a page reading them mid-run sees the previous run's
figures rather than none. The keys are `stats:<source>:scorecard:*`, documented in
`vulnerabilitylookup/scorecard.py`.

## Inspiration

The idea of scoring CNAs on record completeness comes from
[CNA ScoreCard](https://cnascorecard.org/) by RogoLabs, whose five binary checks
over a six-month window showed the approach works. This implementation differs in
what it scores (the CNA container only, with the ADP dependency shown apart), in
grading rather than binary checks, in accepting any severity statement, in a volume
threshold, in covering GNAs, and in being prepared inside the platform that already
holds the records.