Structure & family · gene eval

panther_protein_class

PANTHER protein class classification (single-label multi-class) — gene -> one PANTHER protein class (PCxxxxx) over the shallow PC ontology; classes floored at >=10 annotated genes, by-gene 80/20 split.

Structure & family Multiclass classification gene_classification

Spec

Target	PANTHER protein class classification (single-label multi-class) — gene -> one PANTHER protein class (PCxxxxx) over the shallow PC ontology; classes floored at >=10 annotated genes, by-gene 80/20 split.
Axis	Structure & family — Protein domains, families, and classes.
Shape	Multiclass classification · metric family: accuracy
Task	gene_classification / gene_classification
Dataset	thomas_2022_protein_science/panther_protein_class@0.0.1
Publication	thomas_2022_protein_science
Metrics	gene_class_accuracy gene_class_top_k_accuracy gene_class_f1_macro gene_class_f1_weighted gene_class_balanced_accuracy gene_class_mcc gene_class_accuracy_ancestor_aware gene_class_f1_macro_ancestor_aware
Temporal discovery	no — held-out genes
License	Declared per SPDX in the dataset's recipe README ↗ (commercial-use gated by CI).
References	doi:10.1002/pro.4218 doi:10.1093/nar/gkaa1106 linear:TECH-4602

IO contract

class_parents_column: class_parents
pairing_key: gene_id
prediction_format: (n, K) per-class scores
required_metadata: gene_id
target_kind: multiclass_index

What the prediction looks like

This eval has the multiclass classification shape. The figure is seeded to it — drag the control to watch the score respond.

Shape 4one class of K · multiclassMCC + ancestor-aware F1

Pick the single right family — sometimes credit for getting close.

Used by PANTHER family / class · TF DBD & TFClass family · essentiality 3-class

PTHR11361 · bHLH TF0.61arg-max ✓ truth
PTHR24012 · C2H2 Znf0.16
PTHR45631 · homeobox0.10
PTHR19818 · forkhead0.07
…281 more classes0.06

PredictionPTHR11361

TruthPTHR11361

Resultcorrect ✓

The model scores every class; the arg-max is its single call, graded by multiclass MCC and balanced accuracy — honest even when a few classes dominate the gene count.

Run it

Fetch the censored splits, predict, score — task and dataset filled in.

import prima_bench as pb

# Downloads on demand into the local cache; returns local Paths.
train_path, test_path = pb.fetch_data(task="gene_classification", dataset="thomas_2022_protein_science/panther_protein_class@0.0.1")
train = pb.load(train_path, task="gene_classification", split="train", dataset="thomas_2022_protein_science/panther_protein_class@0.0.1")
test = pb.load(test_path, task="gene_classification", split="test", dataset="thomas_2022_protein_science/panther_protein_class@0.0.1")

# Your model produces predictions aligned to the eval input.
predictions = my_model(test)

# Score against the censored solution over this task's metric panel:
#   gene_class_accuracy, gene_class_top_k_accuracy, gene_class_f1_macro, gene_class_f1_weighted, gene_class_balanced_accuracy, gene_class_mcc, gene_class_accuracy_ancestor_aware, gene_class_f1_macro_ancestor_aware
run = pb.Evaluation(task="gene_classification", dataset="thomas_2022_protein_science/panther_protein_class@0.0.1")
run.update(predictions=predictions)   # one-shot; or stream shard-by-shard
result = run.compute()                # -> EvalResult
print(result)

Results

Awaiting scored models — the row scaffold is live; numbers land with results.json.

Awaiting results · prima-bench

panther_protein_class · leaderboard

1 evals · 1 datasets · no models scored yet

Evals

graded

Datasets

sources

Models

—

pending

Best score

—

pending

No results.json wired up yet. The eval scaffold below is live; drop a document matching src/data/results.schema.md at src/data/results.json (or pass a results prop) and every bar fills in. Each track's tick marks the chance baseline.

Gene Classification

Eval	Score	awaiting
panther_protein_class`thomas_2022_protein_science/panther_protein_class@0.0.1`	—mcc	—