prima·bench
Repo ↗

Gene-level evals · Gene function / characteristic prediction

One gene in, one label out.

A gene-level eval asks one thing: given a gene, predict a property of it. No cells, no sequences at inference — just the gene identity and what your model believes about it. That framing is narrow on purpose, and it is what lets every eval below share a single contract.

42 evals 5 axes 4 shapes results leaderboard →
·

One shape, one contract

Every eval here is a table with one row per gene. The property changes — a GO term, a phenotype, an essentiality score — but the shape never does. prima-bench never touches your model: you hand in a per-gene prediction in the expected shape, and the benchmark scores it against a censored solution. Score one, and you know how to score all 42.

I

The landscape

What a model can know about a gene splits into 5 axes. Pick an axis to filter the catalog below, or jump straight to any eval.

Gene function / characteristic prediction 42 evals

Gene regulatory network (GRN) inference 0

Predicting the regulatory wiring between genes — who activates or represses whom.

none yet — planned

Prototypes This view rebuilds two prima-artifacts references on our tokens: landscape ↗ · results ↗.

II

The shape of each eval

5 axes, but only 4 shapes of question. Each figure is live — switch the example to see a concrete eval, and drag the control to watch the score move.

Shape 1term membership · multi-labelFmax

A gene × term grid — which terms does the gene carry?

Used by GO · pathways · HPO · disease · domains · cell-type & tissue markers · lineage · temporal discovery

truth — gene really carries the term model calls it present (p ≥ threshold)cell shade = predicted probability
Precision89%
Recall89%
F1 @ thr0.89
Fmax0.95

Truth is sparse — most cells are 0. The model emits a probability per cell (the shading); Fmax sweeps the threshold to the best-F1 operating point, so no single arbitrary cutoff is punished. Cell-type, lineage and temporal-discovery evals reuse this exact grid.

Shape 2scalar score · regressionSpearman ρ

One number per gene — does your ranking match the truth?

Used by essentiality (DepMap) · constraint (LOEUF) · dosage (pHaplo / pTriplo / HIPred)

-2.3-1.6-0.88-0.160.55Chronos gene-effect · ← more essentialRPL13APSMD1KRASEGFRTP53OR2T1● true score (above)○ prediction (below)
true score model predictiononly the rank order is scored — not the absolute value
Spearman ρ0.94
Genes shown6
Full eval17–18k genes

Each gene has one continuous truth value; the metric is Spearman ρ — pure rank agreement, so a model with the ordering right scores well even when its absolute numbers drift. Short vertical connectors mean the ranking held; long slants are rank errors.

Shape 3binary call · classificationAUROC + MCC

Yes or no — and where do you draw the line?

Used by is-TF · constrained (LOEUF<0.6) · ClinGen haplo / triplo

called + →0.000.250.500.751.00model P(gene is a TF)transcription factors (8%)other coding genes
Sensitivity95%
Specificity98%
Precision78%
Positives8%

Two populations, one score axis. Moving the threshold trades sensitivity for specificity. When positives are rare (is-TF is ~8%), accuracy misleads and precision gets brutal — so the panel scores threshold-free AUROC plus MCC and balanced accuracy.

Shape 4one class of K · multiclassMCC + ancestor-aware F1

Pick the single right family — sometimes credit for getting close.

Used by PANTHER family / class · TF DBD & TFClass family · essentiality 3-class

PredictionPTHR11361
TruthPTHR11361
Resultcorrect ✓

The model scores every class; the arg-max is its single call, graded by multiclass MCC and balanced accuracy — honest even when a few classes dominate the gene count.
III

The catalog

All 42 gene evals, grouped by axis. Filter by axis, shape, task or temporal-discovery; every card opens that eval's detail page.

Axis
Shape
Task
Discovery

42 of 42 evals · 5 of 5 axes

Gene function / characteristic prediction

Function & pathways

11

Molecular function, biological process, and pathway membership.

Phenotype & disease

4

Phenotype ontologies and gene–disease associations.

Structure & family

7

Protein domains, families, and classes.

Fitness & dosage

8

Essentiality, constraint, and dosage sensitivity.

Cellular context

12

Cell-type / tissue marker and lineage specificity.