Plasma protein binding rate from structure

Predict the percentage of a drug bound to human plasma proteins, scored by mean absolute error in percentage points.

shoal State the problem, from Therapeutics Data Commons ADMET benchmark group b38c662 · 2周前历史

55 行 · 4.8 KB 原始文件永久链接历史

1	# Plasma protein binding rate from structure
2
3	Only the unbound fraction of a drug in blood can reach its target, so the percentage bound to plasma proteins decides how much of a dose does any work. This benchmark asks for that percentage from the structure.
4
5	## Task
6
7	Given a drug as a SMILES string, predict its human plasma protein binding rate as a percentage. The labels in the benchmark files run from $10.1$ to $100$, with a median of $94.7$. Submissions are scored by mean absolute error in percentage points, so lower is better.
8
9	## Background
10
11	The data come from a ChEMBL assay deposited by AstraZeneca as part of its 2016 release of in vitro DMPK measurements. Two features make it awkward in a useful way. The labels crowd against the ceiling — most drugs are highly bound — so the interesting range is a few percentage points wide where it matters most; and the assay was run across five species, of which TDC’s benchmark keeps the human subset.
12
13	## Data and evaluation
14
15	TDC ships the group as one archive (`admet_group`, Harvard Dataverse file 4426004), loaded with
16	`admet_group(path='data/')` from the `tdc.benchmark_group` module. The group’s `ppbr_az` folder holds 2,231 `train_val` and 559 `test` rows. These are measurements, not compounds: 1,797 distinct SMILES (the count TDC’s leaderboard shows), of which 691 appear more than once with different values, so repeated measurements of the same compound sit in the same split.
17
18	The split is a scaffold split: TDC holds out a fixed 20% test set (here 559 of 2,790 rows in
19	the group’s files), and the remaining `train_val` part is split again per seed by
20	`group.get_train_valid_split(benchmark=name, split_type='default', seed=seed)` into 87.5% training
21	and 12.5% validation. The leaderboard asks for five independent runs with seeds 1, 2, 3, 4 and 5;
22	`group.evaluate_many` then reports the mean and the standard deviation of the per-seed test scores,
23	rounded to three decimals. The test set never changes, so only the training split moves with the seed.
24
25	The metric is mean absolute error, $\mathrm{MAE} = \frac{1}{n}\sum_{i=1}^{n} \|y_i - \hat{y}_i\|$, computed by TDC’s `Evaluator(name='mae')` on the test set; lower is better.
26
27	Official evaluation code: [`tdc/benchmark_group/base_group.py`](https://github.com/mims-harvard/TDC/blob/main/tdc/benchmark_group/base_group.py)
28	in the TDC repository (MIT licence), whose `evaluate` and `evaluate_many` methods are what the
29	leaderboard numbers come from. The dataset page gives the licence as "Not Specified. CC BY 4.0".
30
31	## What is known
32
33	- Best on the leaderboard: Gradient Boost, MAE $7.440 \pm 0.024$ percentage points (submitted by Rob Learsch); its write-up describes a scikit-learn gradient-boosting regressor over Morgan-fingerprint similarities and RDKit descriptors.
34	- Next: MapLight + GNN, $7.526 \pm 0.106$, then MapLight, $7.660 \pm 0.058$ (Notwell and Wood, 2023).
35	- Twenty entries are listed, from MAE $7.440$ to $12.848$. No method has got below $7$: with most labels above 90% bound, an error of seven percentage points is large relative to the range that matters, and the repeated measurements put a floor under what any model can reach.
36
37	## What would count as progress
38
39	- A mean MAE on TDC’s fixed test set that beats Gradient Boost (MAE 7.44), averaged over the official
40	seeds 1-5 and produced by `group.evaluate_many`, with the five per-seed numbers reported as well.
41	- The training and inference code, the environment and the fitted model, so that the five runs can be
42	repeated end to end from the `train_val` part alone.
43	- Evidence that the gain is larger than run-to-run noise: the gap between the top two entries here is
44	0.09 percentage points, so a difference smaller than the reported standard deviation is not a result.
45	- Error on the highly bound compounds (above 95%) reported separately, since that is where a percentage point changes the free fraction most.
46
47	## References
48
49	1. AstraZeneca, "Experimental in vitro DMPK and physicochemical data on a set of publicly disclosed compounds" (2016), deposited in ChEMBL as AZ_dataset. [doi:10.6019/CHEMBL3301361](https://doi.org/10.6019/CHEMBL3301361)
50	2. Notwell, J. H. and Wood, M. W., "ADMET property prediction through combinations of molecular fingerprints", arXiv:2310.00174 (2023). [arxiv.org/abs/2310.00174](https://arxiv.org/abs/2310.00174)
51	3. Huang, K. et al., "Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development", NeurIPS Datasets and Benchmarks (2021), arXiv:2102.09548. [arxiv.org/abs/2102.09548](https://arxiv.org/abs/2102.09548)
52
53	## Source
54
55	Imported from [TDC ADMET benchmark group, PPBR_AZ](https://tdcommons.ai/benchmark/admet_group/08ppbr/), curated by Therapeutics Data Commons (Harvard). Open according to the source as of 2026-10-20. The statement above is written in our own words; see the source for the original wording, history and discussion.