01
Spec
| Target | As `cell_type_markers`, restricted to CL cell types present in the pretraining reference-map vocabulary (the aligned subset, TECH-4885 UC2). |
|---|---|
| Axis | Cellular context — Cell-type / tissue marker and lineage specificity. |
| Shape | Term membership (multi-label) · metric family: f_max |
| Task | gene_function_classification / gene_function |
| Dataset | tabula_sapiens_2022_science/cell_type_markers_reference_aligned@0.0.1 |
| Publication | tabula_sapiens_2022_science |
| Metrics |
|
| Temporal discovery | no — held-out genes |
| License | Declared per SPDX in the dataset's recipe README ↗ (commercial-use gated by CI). |
| References |
|
IO contract
- ic_column
- ic
- pairing_key
- gene_term
- required_metadata
- gene_id, term
- target_kind
- binary
02
What the prediction looks like
This eval has the term membership (multi-label) shape. The figure is seeded to it — drag the control to watch the score respond.
A gene × term grid — which terms does the gene carry?
Used by GO · pathways · HPO · disease · domains · cell-type & tissue markers · lineage · temporal discovery
beta cell
hepatocyte
T cell
endothelial
cardiomyo.
neuron
INS
ALB
CD3D
PECAM1
MYH6
truth — gene really carries the term model calls it present (p ≥ threshold)cell shade = predicted probability
Precision89%
Recall89%
F1 @ thr0.89
Fmax0.95
Truth is sparse — most cells are 0. The model emits a probability per cell (the shading); Fmax sweeps the threshold to the best-F1 operating point, so no single arbitrary cutoff is punished. Cell-type, lineage and temporal-discovery evals reuse this exact grid.
03
Run it
Fetch the censored splits, predict, score — task and dataset filled in.
import prima_bench as pb
# Downloads on demand into the local cache; returns local Paths.
train_path, test_path = pb.fetch_data(task="gene_function_classification", dataset="tabula_sapiens_2022_science/cell_type_markers_reference_aligned@0.0.1")
train = pb.load(train_path, task="gene_function_classification", split="train", dataset="tabula_sapiens_2022_science/cell_type_markers_reference_aligned@0.0.1")
test = pb.load(test_path, task="gene_function_classification", split="test", dataset="tabula_sapiens_2022_science/cell_type_markers_reference_aligned@0.0.1")
# Your model produces predictions aligned to the eval input.
predictions = my_model(test)
# Score against the censored solution over this task's metric panel:
# gene_function_term_auprc, gene_function_term_auroc, gene_function_micro_auprc, gene_function_fmax, gene_function_smin
run = pb.Evaluation(task="gene_function_classification", dataset="tabula_sapiens_2022_science/cell_type_markers_reference_aligned@0.0.1")
run.update(predictions=predictions) # one-shot; or stream shard-by-shard
result = run.compute() # -> EvalResult
print(result)
04
Results
Awaiting scored models — the row scaffold is live; numbers land with results.json.
Awaiting results · prima-bench
cell_type_markers_reference_aligned · leaderboard
Evals
1
graded
Datasets
1
sources
Models
—
pending
Best score
—
pending
No
results.json wired up yet. The eval scaffold below is live; drop a document matching src/data/results.schema.md at src/data/results.json (or pass a results prop) and every bar fills in. Each track's tick marks the chance baseline.Gene Function
1| Eval | Score | awaiting |
|---|---|---|
cell_type_markers_reference_alignedtabula_sapiens_2022_science/cell_type_markers_reference_aligned@0.0.1 | — |