Lipophilicity from structure

Lipophilicity — how a compound divides itself between oil and water — sets much of the rest of a drug’s behaviour: how it dissolves, how fast it is metabolised, where it ends up. This benchmark asks for the measured value from the structure.

Task

Given a compound as a SMILES string, predict its lipophilicity, reported by TDC as a log-ratio (the labels run from $-1.5$ to $4.5$). Submissions are scored by mean absolute error on the held-out test set, so lower is better.

Background

The dataset comes from AstraZeneca’s 2016 release of in vitro DMPK and physicochemical measurements on publicly disclosed compounds, and reached wide use through MoleculeNet; TDC includes 4,200 of those compounds. It is measured on one assay in one laboratory, which makes it unusually consistent for a public ADMET set — the residual error is closer to real model error than to disagreement between data sources.

Data and evaluation

TDC ships the group as one archive (admet_group, Harvard Dataverse file 4426004), loaded with admet_group(path='data/') from the tdc.benchmark_group module. The group’s lipophilicity_astrazeneca folder holds 3,360 train_val and 840 test rows (4,200 distinct SMILES, matching TDC’s page).

The split is a scaffold split: TDC holds out a fixed 20% test set (here 840 of 4,200 rows in the group’s files), and the remaining train_val part is split again per seed by group.get_train_valid_split(benchmark=name, split_type='default', seed=seed) into 87.5% training and 12.5% validation. The leaderboard asks for five independent runs with seeds 1, 2, 3, 4 and 5; group.evaluate_many then reports the mean and the standard deviation of the per-seed test scores, rounded to three decimals. The test set never changes, so only the training split moves with the seed.

The metric is mean absolute error, $\mathrm{MAE} = \frac{1}{n}\sum_{i=1}^{n} |y_i - \hat{y}_i|$, computed by TDC’s Evaluator(name='mae') on the test set; lower is better.

Official evaluation code: tdc/benchmark_group/base_group.py in the TDC repository (MIT licence), whose evaluate and evaluate_many methods are what the leaderboard numbers come from. The dataset page gives the licence as "Not Specified. CC BY 4.0".

What is known

• Best on the leaderboard: MiniMol, MAE $0.456 \pm 0.008$ (Kläser, Banaszewski et al., 2024).
• Next: Chemprop-RDKit, $0.467 \pm 0.006$, then plain Chemprop, $0.470 \pm 0.009$ (Yang et al., 2019).
• Twenty-one entries are listed, from MAE $0.456$ to $0.743$. The leading four sit within $0.025$ of each other, and graph neural networks hold all of them.

What would count as progress

• A mean MAE on TDC’s fixed test set that beats MiniMol (MAE 0.456), averaged over the official seeds 1-5 and produced by group.evaluate_many, with the five per-seed numbers reported as well.
• The training and inference code, the environment and the fitted model, so that the five runs can be repeated end to end from the train_val part alone.
• Evidence that the gain is larger than run-to-run noise: the gap between the top two entries here is 0.011 log units, so a difference smaller than the reported standard deviation is not a result.

References

1. AstraZeneca, "Experimental in vitro DMPK and physicochemical data on a set of publicly disclosed compounds" (2016), deposited in ChEMBL as AZ_dataset. doi:10.6019/CHEMBL3301361
2. Wu, Z. et al., "MoleculeNet: a benchmark for molecular machine learning", Chemical Science 9 (2018), 513–530. doi:10.1039/C7SC02664A
3. Kläser, K., Banaszewski, B. et al., "MiniMol: A Parameter-Efficient Foundation Model for Molecular Learning", arXiv:2404.14986 (2024). arxiv.org/abs/2404.14986
4. Yang, K. et al., "Analyzing Learned Molecular Representations for Property Prediction", Journal of Chemical Information and Modeling 59 (2019), 3370–3388. doi:10.1021/acs.jcim.9b00237

Source

Imported from TDC ADMET benchmark group, Lipophilicity_AstraZeneca, curated by Therapeutics Data Commons (Harvard). Open according to the source as of 2026-10-20. The statement above is written in our own words; see the source for the original wording, history and discussion.