An open fingerprint database should let another researcher disagree with it.
That requires more than a hash and an application name. The row needs to show which bytes produced the fingerprint, how the label was established, where the traffic was observed, which software generated the value and what competing explanations remain plausible.
Most public databases were built for a narrower purpose. Historical JA3 lists made values easy to share. Threat feeds made malware observations easy to match. Nmap and p0f made signature rules executable. TLS observatories count what appears at their vantage points. Those are sensible designs for their jobs.
They do not yet add up to a reusable open ground-truth corpus.
Start with observations, not verdicts
The fundamental database object should be an observation:
{
"observation_id": "...",
"observed_at": "2026-07-12T00:00:00Z",
"capture": {
"position": "client-facing edge",
"collector": "mercury",
"collector_revision": "3172786...",
"source": "controlled-client-run"
},
"fingerprints": [],
"labels": [],
"evidence": [],
"licence": "..."
}
The observation can then carry several fingerprint methods and several candidate labels. This is safer than making the hash the primary truth and forcing one application name into the same row.
Store the reversible material
Each fingerprint entry should contain the published identifier and the material used to derive it:
{
"method": "ja4",
"implementation": "FoxIO Python",
"implementation_revision": "...",
"value": "t12d2709h2_..._...",
"raw_value": "t12d2709h2_002f,...",
"source_message_digest": "sha256:...",
"truncated": false
}
For JA3, retain the five-part source string beside the MD5. For JA4, retain JA4_r where collection policy permits it. For Mercury, retain the complete versioned NPF string rather than only its hash nickname. For HASSH, retain the algorithm lists. For a matcher rule, retain the response bytes or sanitised fixture that satisfied it.
This permits field-level comparison, implementation testing and migration when a definition changes. It also exposes incompatible values hidden behind a common field name.
Make labels many-to-many
Labels should be separate evidence-backed assertions:
{
"type": "process",
"value": "example-client",
"version": "1.2.3",
"source": "endpoint-telemetry-join",
"review_state": "reviewed",
"confidence": 0.94,
"first_seen": "...",
"last_seen": "..."
}
One fingerprint can have several process labels because applications share libraries. One process can have several fingerprints because versions, platforms and configurations differ. The schema should preserve counts and conflict instead of selecting one winner during ingestion.
Cisco Mercury's public resource schema points in this direction: fingerprint entries contain candidate processes, observation counts, operating systems and destinations. The production database is private, but the many-to-many model is the right starting point.
Describe how ground truth was produced
Use a controlled vocabulary for evidence sources:
controlled_capture
endpoint_process_join
malware_sandbox
reviewed_pcap
active_probe_match
passive_observation
community_submission
derived_from_external_database
Each type needs its own metadata. An endpoint join should record the join window and ambiguity rules. A controlled capture should name the client build, operating system and generation script. A malware sandbox label should identify the sample and distinguish the sandbox process from the malware process. A community submission should identify what a reviewer actually checked.
Derived labels should never masquerade as independent evidence. If a mapping was imported from an older JA3 list, cite that row and keep its original uncertainty.
Separate prevalence from classification
Prevalence belongs in observation aggregates:
{
"source": "university-vantage-a",
"first_seen": "...",
"last_seen": "...",
"count": 184233
}
It should not silently increase application-label confidence. A common fingerprint is not necessarily well-labelled; a rare fingerprint is not necessarily suspicious.
The TLS Fingerprint Observatory demonstrates the value of publishing counts even when labels are missing. Its large unlabelled TLS corpus is more useful for prevalence research than a catalogue filled with guesses.
Represent threat observations explicitly
A malware feed should attach an observation, not overwrite the fingerprint's identity:
{
"type": "malware-sandbox-observation",
"family": "example-family",
"sample_sha256": "...",
"observed_at": "...",
"known_good_evaluation": "not-performed"
}
This retains the warning published by sources such as SSLBL. It allows a consumer to ask whether a fingerprint was seen in malware without treating every matching benign process as infected.
Publish validation sets and negative evidence
A useful database should include:
- controlled positive captures;
- known-good observations that share supposedly malicious fingerprints;
- deliberately conflicting labels;
- client upgrades that changed fingerprints;
- malformed and truncated handshakes;
- captures before and after a TLS-terminating proxy;
- implementation conformance cases.
Negative and conflicting evidence is not database dirt. It tells consumers where a label stops working.
Version the data and the interpretation
Every release should state:
- schema version;
- database snapshot identifier;
- fingerprint implementation revisions;
- collection time range;
- label additions, removals and merges;
- retired mappings;
- licence changes;
- reproducible validation results.
Consumers should be able to pin a snapshot and explain which database caused a decision. A live service may remain convenient, but an incident review needs the state that existed when the event was classified.
Make licensing part of the schema
Fingerprint method, implementation and dataset rights are separate.
Core JA4 is BSD-licensed. Other JA4+ methods use different FoxIO terms. Nmap's data files use the Nmap Public Source License. A combined historical repository may contain rows imported under different licences. An API may permit lookup but prohibit bulk redistribution.
Store a licence or source-rights reference for each imported dataset and, where necessary, each record. If the rights are unclear, publish a pointer and transformation recipe rather than republishing the row.
Provide a confidence model that can be audited
A score without calibration is decoration. For each label type, document:
- what the score estimates;
- the labelled validation set;
- true-positive and false-positive definitions;
- treatment of unseen applications;
- time and environment holdouts;
- how conflicts and stale observations affect the score.
Where calibration is unavailable, use review states such as submitted, reproduced, reviewed and contested instead of inventing numeric precision.
The minimum viable open record
If full packet evidence cannot be published, a useful minimum is:
method and rule version
implementation revision
compact and raw fingerprint
candidate label, including version/platform
ground-truth method
capture position
first and last seen
observation count
confidence or review state
evidence reference
licence
That is more expensive than a CSV containing hash,name. It is also what makes the mapping useful outside the environment and assumptions of its original collector.
The public ecosystem already contains most of the parts: JA4DB's cross-method model, the TLS observatory's prevalence data, Mercury's candidate-process and destination schema, FingerprinTLS's retained ClientHello fields, and executable fixtures in matcher projects such as Rapid7 Recog. The missing step is to combine those strengths without erasing provenance.
For the current resource landscape, see Public Network Fingerprint Databases and What They Cover. For the import checklist, see How to Evaluate a Network Fingerprint Database.
