01
Spec
| Target | ClinGen triplosensitivity binary classification (K=2, dosage_insensitive/triplosensitive): predict whether a gene is triplosensitive. Positives = `ts_ordinal >= 3`; negatives = `ts_code == 40` ONLY (same sentinel discipline as the HI eval). Wired to the sample_classification task; AUPRC primary. EXTREMELY low-N — ClinGen has only a handful of high-confidence TS calls — so this is a stringent probe, not a balanced benchmark. Paralog/family-aware split. |
|---|---|
| Axis | Fitness & dosage — Essentiality, constraint, and dosage sensitivity. |
| Shape | Binary classification · metric family: auroc |
| Task | sample_classification / sample_classification |
| Dataset | riggs_2012_clinical_genetics/dosage_triplosensitivity@0.0.1 |
| Publication | riggs_2012_clinical_genetics |
| Metrics |
|
| Temporal discovery | no — held-out genes |
| License | Declared per SPDX in the dataset's recipe README ↗ (commercial-use gated by CI). |
| References |
|
IO contract
- operating_threshold_column
- operating_threshold
- optional_metadata
- cohort, disease, assay, split, donor_id
- pairing_key
- sample_id
- prediction_format
- (n, K) per-class scores; (n, 1) accepted at K=2
- required_metadata
- sample_id
- target_kind
- multiclass_index
02
What the prediction looks like
This eval has the binary classification shape. The figure is seeded to it — drag the control to watch the score respond.
Yes or no — and where do you draw the line?
Used by is-TF · constrained (LOEUF<0.6) · ClinGen haplo / triplo
Sensitivity93%
Specificity86%
Precision86%
Positives50%
Two populations, one score axis. Moving the threshold trades sensitivity for specificity. When positives are rare (is-TF is ~8%), accuracy misleads and precision gets brutal — so the panel scores threshold-free AUROC plus MCC and balanced accuracy.
03
Run it
Fetch the censored splits, predict, score — task and dataset filled in.
import prima_bench as pb
# Downloads on demand into the local cache; returns local Paths.
train_path, test_path = pb.fetch_data(task="sample_classification", dataset="riggs_2012_clinical_genetics/dosage_triplosensitivity@0.0.1")
train = pb.load(train_path, task="sample_classification", split="train", dataset="riggs_2012_clinical_genetics/dosage_triplosensitivity@0.0.1")
test = pb.load(test_path, task="sample_classification", split="test", dataset="riggs_2012_clinical_genetics/dosage_triplosensitivity@0.0.1")
# Your model produces predictions aligned to the eval input.
predictions = my_model(test)
# Score against the censored solution over this task's metric panel:
# sample_auroc, sample_sensitivity, sample_specificity, sample_accuracy, sample_f1, sample_mcc, sample_balanced_accuracy, sample_auprc, sample_score_correlation
run = pb.Evaluation(task="sample_classification", dataset="riggs_2012_clinical_genetics/dosage_triplosensitivity@0.0.1")
run.update(predictions=predictions) # one-shot; or stream shard-by-shard
result = run.compute() # -> EvalResult
print(result)
04
Results
Awaiting scored models — the row scaffold is live; numbers land with results.json.
Awaiting results · prima-bench
dosage_triplosensitivity · leaderboard
Evals
1
graded
Datasets
1
sources
Models
—
pending
Best score
—
pending
No
results.json wired up yet. The eval scaffold below is live; drop a document matching src/data/results.schema.md at src/data/results.json (or pass a results prop) and every bar fills in. Each track's tick marks the chance baseline.Sample Classification
1| Eval | Score | awaiting |
|---|---|---|
dosage_triplosensitivityriggs_2012_clinical_genetics/dosage_triplosensitivity@0.0.1 | — |