加入
Realistic simulators for spatial transcriptomics
Simulate spatial transcriptomics data that match ten real datasets on 23 distributional and downstream-analysis metrics.
shoal 2周前提交
0 个父提交
提交 f1f96e6072e0d50e355186837f96d0fa8f028f63
+44 −0 README.md 新增 查看文件
| @@ -0,0 +1,44 @@ | ||
| 1 | +# Realistic simulators for spatial transcriptomics | |
| 2 | + | |
| 3 | +New methods for spatial transcriptomics are often developed and tested on simulated data, so those tests are only as good as the simulator. This benchmark asks for a simulator whose synthetic data match real spatial datasets, judged on many statistical properties and on how downstream analyses behave. | |
| 4 | + | |
| 5 | +## Task | |
| 6 | + | |
| 7 | +A method receives a real spatial transcriptomics dataset: counts per spot or cell, with spatial coordinates. It must return a simulated dataset of the same form. The simulated data are compared with the real data, and with a matched single-cell reference, on 23 metrics: distributions of gene- and spot-level statistics, spatial statistics, and the results of downstream tasks run on real versus simulated data. | |
| 8 | + | |
| 9 | +## Background | |
| 10 | + | |
| 11 | +Simulation lets developers test methods on data with known structure, but many simulators were built for dissociated single-cell data and ignore space. SpatialSimBench (Liang et al., 2025) assessed 13 simulation methods on ten spatial datasets and introduced simAdaptor, which extends single-cell simulators with spatial variables. Its authors, Xiaoqi Liang, Yue Cao and Jean Yang, brought the framework to Open Problems as a living benchmark. | |
| 12 | + | |
| 13 | +## Data and evaluation | |
| 14 | + | |
| 15 | +- **Data:** ten public spatial datasets, each paired with a single-cell reference of similar tissue: mouse brain, olfactory bulb, gastrulation and regenerating hindlimb muscle, a mouse fibrosarcoma sample from the DestVI study, a seqFISH+ cortex dataset, and human breast cancer, osteosarcoma, pancreatic ductal adenocarcinoma and prostate. Files are in `s3://openproblems-data/resources/task_spatial_simulators/datasets/`. The task code is MIT-licensed; dataset licences are not restated on the task page. | |
| 16 | +- **Split:** there is no train/test split; each simulator is fitted to and compared with the same real dataset. | |
| 17 | +- **Metric:** 23 metrics in release v0.2.0. Most are kernel-density two-sample statistics (`ks::kde.test`, lower is better) comparing real and simulated distributions of, for example, library size, fraction of zeros, mean–variance relation, gene–gene correlation, Moran's I and nearest-neighbour correlation. The rest score downstream tasks: cell-type deconvolution error (Jensen–Shannon divergence, RMSE), spatially variable gene recall and precision, and spatial cross-correlation (cosine, Mantel). Each is rescaled per dataset between the worst and best control method (a positive control and two negative controls that shuffle the data or draw normal noise), $s=\min\{1,\max\{0,(x-c_{\text{worst}})/(c_{\text{best}}-c_{\text{worst}})\}\}$, and the overall score is the mean over the 10 × 23 pairs, with failed runs counted as 0. The leaderboard shows it to two decimals. | |
| 18 | +- **Code:** [openproblems-bio/task_spatial_simulators](https://github.com/openproblems-bio/task_spatial_simulators). | |
| 19 | + | |
| 20 | +## What is known | |
| 21 | + | |
| 22 | +On the v0.2.0 leaderboard (as of 2026-10-20): | |
| 23 | + | |
| 24 | +- SRTsim, a simulator built for spatial data, leads at 0.90 (0.9021 before rounding, recomputed from the published results). | |
| 25 | +- scDesign3 with a negative binomial model follows at 0.84, then scDesign2 (0.79), zinbwave (0.77) and SPARsim (0.75). | |
| 26 | +- Splatter (0.59) and SymSim (0.57), two single-cell simulators, come last. | |
| 27 | +- The positive control scores 0.98 and the shuffling control 0.41, which bounds the room for improvement. | |
| 28 | + | |
| 29 | +## What would count as progress | |
| 30 | + | |
| 31 | +- An overall score above 0.9021 on the v0.2.0 datasets and metrics, computed with the task's `run_benchmark` workflow, with code that reproduces it. | |
| 32 | +- A simulator that beats SRTsim on the downstream-task metrics (deconvolution, spatially variable genes, cross-correlation), which show how the simulated data behave when used to test analysis methods. | |
| 33 | +- A gain that holds on each of the ten datasets, not only on average. | |
| 34 | + | |
| 35 | +## References | |
| 36 | + | |
| 37 | +1. Liang X, Torkel M, Cao Y, Yang JYH. Multi-task benchmarking of spatially resolved gene expression simulation models. *Genome Biology* 26, 57 (2025). [doi:10.1186/s13059-025-03505-w](https://doi.org/10.1186/s13059-025-03505-w) | |
| 38 | +2. Zhu J, Shang L, Zhou X. SRTsim: spatial pattern preserving simulations for spatially resolved transcriptomics. *Genome Biology* 24, 39 (2023). [doi:10.1186/s13059-023-02879-z](https://doi.org/10.1186/s13059-023-02879-z) | |
| 39 | +3. Song D, Wang Q, Yan G, et al. scDesign3 generates realistic in silico data for multimodal single-cell and spatial omics. *Nature Biotechnology* 42, 247–252 (2024). [doi:10.1038/s41587-023-01772-1](https://doi.org/10.1038/s41587-023-01772-1) | |
| 40 | +4. Luecken MD, Gigante S, Burkhardt DB, Cannoodt R, et al. Defining and benchmarking open problems in single-cell analysis. *Nature Biotechnology* 43, 1035–1040 (2025). [doi:10.1038/s41587-025-02694-w](https://doi.org/10.1038/s41587-025-02694-w) | |
| 41 | + | |
| 42 | +## Source | |
| 43 | + | |
| 44 | +Imported from [Open Problems in Single-Cell Analysis: Spatial Simulators (task_spatial_simulators)](https://openproblems.bio/benchmarks/spatial_simulators), curated by the Open Problems in Single-Cell Analysis consortium. Open according to the source as of 2026-10-20 (leaderboard release v0.2.0). The statement above is written in our own words; see the source for the original wording, history and discussion. | |