Plasma protein binding rate from structure

Only the unbound fraction of a drug in blood can reach its target, so the percentage bound to plasma proteins decides how much of a dose does any work. This benchmark asks for that percentage from the structure.

Task

Given a drug as a SMILES string, predict its human plasma protein binding rate as a percentage. The labels in the benchmark files run from $10.1$ to $100$, with a median of $94.7$. Submissions are scored by mean absolute error in percentage points, so lower is better.

Background

The data come from a ChEMBL assay deposited by AstraZeneca as part of its 2016 release of in vitro DMPK measurements. Two features make it awkward in a useful way. The labels crowd against the ceiling — most drugs are highly bound — so the interesting range is a few percentage points wide where it matters most; and the assay was run across five species, of which TDC’s benchmark keeps the human subset.

Data and evaluation

TDC ships the group as one archive (admet_group, Harvard Dataverse file 4426004), loaded with admet_group(path='data/') from the tdc.benchmark_group module. The group’s ppbr_az folder holds 2,231 train_val and 559 test rows. These are measurements, not compounds: 1,797 distinct SMILES (the count TDC’s leaderboard shows), of which 691 appear more than once with different values, so repeated measurements of the same compound sit in the same split.

The split is a scaffold split: TDC holds out a fixed 20% test set (here 559 of 2,790 rows in the group’s files), and the remaining train_val part is split again per seed by group.get_train_valid_split(benchmark=name, split_type='default', seed=seed) into 87.5% training and 12.5% validation. The leaderboard asks for five independent runs with seeds 1, 2, 3, 4 and 5; group.evaluate_many then reports the mean and the standard deviation of the per-seed test scores, rounded to three decimals. The test set never changes, so only the training split moves with the seed.

The metric is mean absolute error, $\mathrm{MAE} = \frac{1}{n}\sum_{i=1}^{n} |y_i - \hat{y}_i|$, computed by TDC’s Evaluator(name='mae') on the test set; lower is better.

Official evaluation code: tdc/benchmark_group/base_group.py in the TDC repository (MIT licence), whose evaluate and evaluate_many methods are what the leaderboard numbers come from. The dataset page gives the licence as "Not Specified. CC BY 4.0".

What is known

• Best on the leaderboard: Gradient Boost, MAE $7.440 \pm 0.024$ percentage points (submitted by Rob Learsch); its write-up describes a scikit-learn gradient-boosting regressor over Morgan-fingerprint similarities and RDKit descriptors.
• Next: MapLight + GNN, $7.526 \pm 0.106$, then MapLight, $7.660 \pm 0.058$ (Notwell and Wood, 2023).
• Twenty entries are listed, from MAE $7.440$ to $12.848$. No method has got below $7$: with most labels above 90% bound, an error of seven percentage points is large relative to the range that matters, and the repeated measurements put a floor under what any model can reach.

What would count as progress

• A mean MAE on TDC’s fixed test set that beats Gradient Boost (MAE 7.44), averaged over the official seeds 1-5 and produced by group.evaluate_many, with the five per-seed numbers reported as well.
• The training and inference code, the environment and the fitted model, so that the five runs can be repeated end to end from the train_val part alone.
• Evidence that the gain is larger than run-to-run noise: the gap between the top two entries here is 0.09 percentage points, so a difference smaller than the reported standard deviation is not a result.
• Error on the highly bound compounds (above 95%) reported separately, since that is where a percentage point changes the free fraction most.

References

1. AstraZeneca, "Experimental in vitro DMPK and physicochemical data on a set of publicly disclosed compounds" (2016), deposited in ChEMBL as AZ_dataset. doi:10.6019/CHEMBL3301361
2. Notwell, J. H. and Wood, M. W., "ADMET property prediction through combinations of molecular fingerprints", arXiv:2310.00174 (2023). arxiv.org/abs/2310.00174
3. Huang, K. et al., "Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development", NeurIPS Datasets and Benchmarks (2021), arXiv:2102.09548. arxiv.org/abs/2102.09548

Source

Imported from TDC ADMET benchmark group, PPBR_AZ, curated by Therapeutics Data Commons (Harvard). Open according to the source as of 2026-10-20. The statement above is written in our own words; see the source for the original wording, history and discussion.