# Lipophilicity from structure

Lipophilicity — how a compound divides itself between oil and water — sets much of the rest of a drug’s behaviour: how it dissolves, how fast it is metabolised, where it ends up. This benchmark asks for the measured value from the structure.

## Task

Given a compound as a SMILES string, predict its lipophilicity, reported by TDC as a log-ratio (the labels run from $-1.5$ to $4.5$). Submissions are scored by mean absolute error on the held-out test set, so lower is better.

## Background

The dataset comes from AstraZeneca’s 2016 release of in vitro DMPK and physicochemical measurements on publicly disclosed compounds, and reached wide use through MoleculeNet; TDC includes 4,200 of those compounds. It is measured on one assay in one laboratory, which makes it unusually consistent for a public ADMET set — the residual error is closer to real model error than to disagreement between data sources.

## Data and evaluation

TDC ships the group as one archive (`admet_group`, Harvard Dataverse file 4426004), loaded with
`admet_group(path='data/')` from the `tdc.benchmark_group` module. The group’s `lipophilicity_astrazeneca` folder holds 3,360 `train_val` and 840 `test` rows (4,200 distinct SMILES, matching TDC’s page).

The split is a scaffold split: TDC holds out a fixed 20% test set (here 840 of 4,200 rows in
the group’s files), and the remaining `train_val` part is split again per seed by
`group.get_train_valid_split(benchmark=name, split_type='default', seed=seed)` into 87.5% training
and 12.5% validation. The leaderboard asks for five independent runs with seeds 1, 2, 3, 4 and 5;
`group.evaluate_many` then reports the mean and the standard deviation of the per-seed test scores,
rounded to three decimals. The test set never changes, so only the training split moves with the seed.

The metric is mean absolute error, $\mathrm{MAE} = \frac{1}{n}\sum_{i=1}^{n} |y_i - \hat{y}_i|$, computed by TDC’s `Evaluator(name='mae')` on the test set; lower is better.

Official evaluation code: [`tdc/benchmark_group/base_group.py`](https://github.com/mims-harvard/TDC/blob/main/tdc/benchmark_group/base_group.py)
in the TDC repository (MIT licence), whose `evaluate` and `evaluate_many` methods are what the
leaderboard numbers come from. The dataset page gives the licence as "Not Specified. CC BY 4.0".

## What is known

- Best on the leaderboard: MiniMol, MAE $0.456 \pm 0.008$ (Kläser, Banaszewski et al., 2024).
- Next: Chemprop-RDKit, $0.467 \pm 0.006$, then plain Chemprop, $0.470 \pm 0.009$ (Yang et al., 2019).
- Twenty-one entries are listed, from MAE $0.456$ to $0.743$. The leading four sit within $0.025$ of each other, and graph neural networks hold all of them.

## What would count as progress

- A mean MAE on TDC’s fixed test set that beats MiniMol (MAE 0.456), averaged over the official
  seeds 1-5 and produced by `group.evaluate_many`, with the five per-seed numbers reported as well.
- The training and inference code, the environment and the fitted model, so that the five runs can be
  repeated end to end from the `train_val` part alone.
- Evidence that the gain is larger than run-to-run noise: the gap between the top two entries here is
  0.011 log units, so a difference smaller than the reported standard deviation is not a result.

## References

1. AstraZeneca, "Experimental in vitro DMPK and physicochemical data on a set of publicly disclosed compounds" (2016), deposited in ChEMBL as AZ_dataset. [doi:10.6019/CHEMBL3301361](https://doi.org/10.6019/CHEMBL3301361)
2. Wu, Z. et al., "MoleculeNet: a benchmark for molecular machine learning", *Chemical Science* 9 (2018), 513–530. [doi:10.1039/C7SC02664A](https://doi.org/10.1039/C7SC02664A)
3. Kläser, K., Banaszewski, B. et al., "MiniMol: A Parameter-Efficient Foundation Model for Molecular Learning", arXiv:2404.14986 (2024). [arxiv.org/abs/2404.14986](https://arxiv.org/abs/2404.14986)
4. Yang, K. et al., "Analyzing Learned Molecular Representations for Property Prediction", *Journal of Chemical Information and Modeling* 59 (2019), 3370–3388. [doi:10.1021/acs.jcim.9b00237](https://doi.org/10.1021/acs.jcim.9b00237)

## Source

Imported from [TDC ADMET benchmark group, Lipophilicity_AstraZeneca](https://tdcommons.ai/benchmark/admet_group/05lipo/), curated by Therapeutics Data Commons (Harvard). Open according to the source as of 2026-10-20. The statement above is written in our own words; see the source for the original wording, history and discussion.