Benchmarks

Tasks with a fixed evaluation and a baseline to beat, many from established benchmark suites. Routes change the model, the training, the data, the search or the system; the project’s verifier scores every commit, and the best route holds the record anyone can build on.

Predict human plasma protein binding rate (%) from a SMILES string

Plasma protein binding rate from structure

Predict the percentage of a drug bound to human plasma proteins, scored by mean absolute error in percentage points.

—MAE (percentage points bound)

The baseline, 7.44, still stands

Predict lipophilicity (log-ratio) from a SMILES string

Lipophilicity from structure

Predict octanol-water distribution (lipophilicity) for 4,200 AstraZeneca compounds, scored by mean absolute error.

—MAE (log-ratio)

The baseline, 0.456, still stands

Classify human intestinal absorption from a SMILES string

Human intestinal absorption from structure

Classify whether an orally administered drug is absorbed from the human intestine, scored by AUROC on TDC’s scaffold split.

—AUROC

The baseline, 0.993, still stands

Classify hERG channel blockade from a SMILES string

hERG channel blockade from structure

Classify whether a compound blocks the hERG cardiac ion channel, scored by AUROC on 648 drugs.

—AUROC

The baseline, 0.88, still stands

Predict human drug half-life from a SMILES string

Drug half-life from structure

Rank 667 drugs by their human half-life from structure, scored by Spearman correlation on TDC’s scaffold split.

—Spearman correlation

The baseline, 0.576, still stands

Classify drug-induced liver injury from a SMILES string

Drug-induced liver injury from structure

Classify whether a drug can cause liver injury, scored by AUROC on an FDA-derived set of 475 drugs.

—AUROC

The baseline, 0.956, still stands

Classify CYP3A4 inhibition from a SMILES string

CYP3A4 inhibition from structure

Classify whether a compound inhibits the CYP3A4 enzyme, scored by AUPRC over 12,328 compounds.

—AUPRC

The baseline, 0.916, still stands

Predict Caco-2 cell effective permeability from a SMILES string

Caco-2 cell permeability from structure

Predict a drug’s Caco-2 cell effective permeability from its SMILES string; TDC scores mean absolute error.

—MAE (log cm/s)

The baseline, 0.256, still stands

Classify blood-brain barrier penetration from a SMILES string

Blood-brain barrier penetration from structure

Classify whether a compound crosses the blood-brain barrier, scored by AUROC on TDC’s scaffold split of 1,975 drugs.

—AUROC

The baseline, 0.924, still stands

Predict aqueous solubility (log mol/L) from a SMILES string

Aqueous solubility from structure

Predict aqueous solubility in log mol/L for 9,982 compounds from AqSolDB, scored by mean absolute error.

—MAE (log mol/L)

The baseline, 0.741, still stands

Classify Ames mutagenicity from a SMILES string

Ames mutagenicity from structure

Classify whether a compound is mutagenic in the Ames test, scored by AUROC over 7,255 compounds.

—AUROC

The baseline, 0.871, still stands

Predict rat acute oral toxicity LD50 from a SMILES string

Acute oral toxicity (LD50) from structure

Predict rat acute oral toxicity LD50 for 7,385 compounds, scored by mean absolute error in log(1/(mol/kg)).

—MAE (log(1/(mol/kg)))

The baseline, 0.552, still stands