Phenotype & disease · gene eval

disease_associations_discovery

Gene-disease association temporal discovery — train on associations known at the T0 release (2022-06-02), evaluate on gene->disease links that first appeared by the T1 release (2026-05-30); positive associations only (is_not_implicated_in excluded), is_a-ancestor propagated (true-path), terms with 15-500 annotated genes.

Phenotype & disease Term membership (multi-label) gene_function_classification temporal discovery

Spec

Target	Gene-disease association temporal discovery — train on associations known at the T0 release (2022-06-02), evaluate on gene->disease links that first appeared by the T1 release (2026-05-30); positive associations only (is_not_implicated_in excluded), is_a-ancestor propagated (true-path), terms with 15-500 annotated genes.
Axis	Phenotype & disease — Phenotype ontologies and gene–disease associations.
Shape	Term membership (multi-label) · metric family: f_max
Task	gene_function_classification / gene_function
Dataset	schriml_2025_nucleic_acids_research/disease_associations_discovery@0.0.1
Publication	schriml_2025_nucleic_acids_research
Metrics	gene_function_term_auprc gene_function_term_auroc gene_function_micro_auprc gene_function_fmax gene_function_smin
Temporal discovery	yes — splits by time (T0 → T1)
License	Declared per SPDX in the dataset's recipe README ↗ (commercial-use gated by CI).
References	doi:10.1093/nar/gkad1042 doi:10.1093/nar/gkae1086 linear:TECH-4594

IO contract

ic_column: ic
pairing_key: gene_term
required_metadata: gene_id, term
target_kind: binary

What the prediction looks like

This eval has the term membership (multi-label) shape. The figure is seeded to it — drag the control to watch the score respond.

Shape 1term membership · multi-labelFmax

A gene × term grid — which terms does the gene carry?

Used by GO · pathways · HPO · disease · domains · cell-type & tissue markers · lineage · temporal discovery

autophagy

cGAS–STING

pyroptosis

ferroptosis

necroptosis

efferocyt.

ATG9B

STING1

GSDMD

ZBP1

TREM2

truth — gene really carries the term model calls it present (p ≥ threshold)cell shade = predicted probability

call present when p ≥ 0.50

Precision89%

Recall89%

F1 @ thr0.89

Fmax0.95

Truth is sparse — most cells are 0. The model emits a probability per cell (the shading); Fmax sweeps the threshold to the best-F1 operating point, so no single arbitrary cutoff is punished. Cell-type, lineage and temporal-discovery evals reuse this exact grid.

Run it

Fetch the censored splits, predict, score — task and dataset filled in.

import prima_bench as pb

# Downloads on demand into the local cache; returns local Paths.
train_path, test_path = pb.fetch_data(task="gene_function_classification", dataset="schriml_2025_nucleic_acids_research/disease_associations_discovery@0.0.1")
train = pb.load(train_path, task="gene_function_classification", split="train", dataset="schriml_2025_nucleic_acids_research/disease_associations_discovery@0.0.1")
test = pb.load(test_path, task="gene_function_classification", split="test", dataset="schriml_2025_nucleic_acids_research/disease_associations_discovery@0.0.1")

# Your model produces predictions aligned to the eval input.
predictions = my_model(test)

# Score against the censored solution over this task's metric panel:
#   gene_function_term_auprc, gene_function_term_auroc, gene_function_micro_auprc, gene_function_fmax, gene_function_smin
run = pb.Evaluation(task="gene_function_classification", dataset="schriml_2025_nucleic_acids_research/disease_associations_discovery@0.0.1")
run.update(predictions=predictions)   # one-shot; or stream shard-by-shard
result = run.compute()                # -> EvalResult
print(result)

Results

Awaiting scored models — the row scaffold is live; numbers land with results.json.

Awaiting results · prima-bench

disease_associations_discovery · leaderboard

1 evals · 1 datasets · no models scored yet

Evals

graded

Datasets

sources

Models

—

pending

Best score

—

pending

No results.json wired up yet. The eval scaffold below is live; drop a document matching src/data/results.schema.md at src/data/results.json (or pass a results prop) and every bar fills in. Each track's tick marks the chance baseline.

Gene Function

Eval	Score	awaiting
disease_associations_discovery`schriml_2025_nucleic_acids_research/disease_associations_discovery@0.0.1`	—fmax	—