Predicting perovskite formation energy from structure

Predict the DFT heat of formation of a 5-atom perovskite cell in eV from its structure, over 18,928 cells.

shoal State the problem, from Matbench 0383993 · 2周前历史

42 行 · 4.5 KB 原始文件永久链接历史

1	# Predicting perovskite formation energy from structure
2
3	Perovskites are a small structural family with a huge compositional space, which makes them a natural target for computational screening. This task asks for the computed heat of formation of a 5-atom perovskite cell from its structure, on 18,928 cells generated for a water-splitting screen.
4
5	## Task
6
7	Given a crystal structure, predict the heat of formation of the whole five-atom perovskite cell in electronvolts, as computed with RPBE GGA-DFT. Scoring is the mean absolute error over the five fixed folds. Note the reference state: oxygen was referenced to its chemical potential in water vapour, not to the oxygen molecule.
8
9	## Background
10
11	The data was generated by Castelli and co-workers, who swept cubic perovskite compositions for one- and two-photon water splitting in the Computational Materials Repository. Because every entry shares one structure type, the task isolates composition-driven energetics within a fixed lattice, which is a different problem from the suite's general formation-energy task and a good test of whether a model has learned chemistry rather than geometry.
12
13	## Data and evaluation
14
15	- Data: `matbench_perovskites.json.gz` from MPContribs-ML (18,928 records, each a pymatgen Structure and a heat of formation in eV per unit cell), at <https://ml.materialsproject.org/projects/matbench_perovskites> or via `task.load()` in the `matbench` package; the calculations are from Castelli and co-workers.
16	- Split: five folds fixed in `matbench_v0.1_validation.json`, built with scikit-learn's `KFold(n_splits=5, shuffle=True, random_state=18012019)`. Train on each fold's training set, tune inside it only, predict its test set, report the mean. Changing the folds invalidates the comparison.
17	- Metric: mean absolute error in eV/unit cell over the five test folds, lower is better. The leaderboard also lists the fold-to-fold standard deviation, RMSE and worst single-sample error.
18	- Official evaluation code: the `matbench` package (MIT), <https://github.com/materialsproject/matbench>; the task's `record(fold, predictions)`, as in `mb.matbench_perovskites.record(...)` on a `MatbenchBenchmark` `mb`, scores a fold and `mb.to_file(...)` writes the submission file.
19
20	## What is known
21
22	- Top of the task leaderboard: coGN, mean MAE 0.0269 eV per unit cell (standard deviation 0.0008).
23	- Runners-up: ALIGNN and DeeperGATGNN both at 0.0288, and coNGN at 0.0290.
24	- A dummy regressor scores 0.5660 eV, the target's mean absolute deviation, so the best models cut the error by a factor of twenty.
25	- Divided by the five atoms in the cell the leader's error is about 5 meV/atom, so further gains need care about what the labels themselves can support.
26	- The leaderboard's repository has merged no submission since 20 January 2024; newer ones wait as open [pull requests](https://github.com/materialsproject/matbench/pulls), so the official numbers may lag what has been achieved.
27
28	## What would count as progress
29
30	- A mean MAE below 0.0269 eV/unit cell on the five fixed folds, beating coGN, submitted as the `matbench` benchmark file so every fold's predictions can be re-scored.
31	- Code and a pinned environment that reproduce the run from the fixed folds.
32	- Per-fold numbers, not just the mean, so that a gain can be compared with the leader fold by fold on the same folds; a small gain should hold on each fold or over repeated training runs. Reproducing the current leader counts too, since the board has not moved since 2024.
33
34	## References
35
36	1. I. E. Castelli et al., "New cubic perovskites for one- and two-photon water splitting using the computational materials repository", Energy & Environmental Science 5, 9034–9043 (2012). <https://doi.org/10.1039/C2EE22341D>
37	2. A. Dunn, Q. Wang, A. Ganose, D. Dopp and A. Jain, "Benchmarking materials property prediction methods: the Matbench test set and Automatminer reference algorithm", npj Computational Materials 6, 138 (2020). <https://doi.org/10.1038/s41524-020-00406-3>
38	3. R. Ruff, P. Reiser, J. Stühmer and P. Friederich, "Connectivity optimized nested line graph networks for crystal structures", Digital Discovery 3, 594–601 (2024). <https://doi.org/10.1039/D4DD00018H>
39
40	## Source
41
42	Imported from [Matbench v0.1 matbench_perovskites](https://matbench.materialsproject.org/Leaderboards%20Per-Task/matbench_v0.1_matbench_perovskites/), curated by Materials Project (Dunn et al.). Open according to the source as of 2026-10-20. The statement above is written in our own words; see the source for the original wording, history and discussion.