sentinel-crop-pipeline
A reproducible Sentinel-2 data preparation pipeline for crop-classification research, where every discarded pixel is accounted for.
pip install sentinel-crop-pipelineThe dataset is the experiment
Most of the reproducibility problem in remote-sensing machine learning sits upstream of the model. Which scenes were selected and why, how many pixels the cloud mask removed, whether two patches from the same field ended up on opposite sides of the split — these decisions determine the result, and they are usually undocumented.
This pipeline treats them as first-class outputs. Every stage writes a timestamped JSON summary, and the patch manifest records the split assignment and invalid-pixel percentage of every patch.
Five stages, each one auditable
Stages run individually or as sentinel-crop run-all. The illustrations show what each stage does to the data, not what it is called.
What each command produces
STAC search with deterministic scene selection; writes per-scene accept and reject decisions.
AOI-cropped retrieval through the CDSE Process API rather than whole tiles.
SCL masking, reflectance normalisation, NDVI / NDRE / NDWI where bands allow.
Fixed-size patches with spatially blocked train, validation and test assignment.
Rasterises externally prepared ground-truth polygons into uint8 masks.
Whole grid blocks are assigned to a single split, which reduces the spatial autocorrelation that makes random splits flatter the model.
The audit trail is part of the deliverable
logs/run_<stage>_<timestamp>.json— per-stage summaries.logs/selection_results.json— why each scene was kept or dropped.data/patches/index.csv— the patch manifest: split, paths, invalid percentage.data/patches/cog/— georeferenced GeoTIFFs you can open and look at.- Training exports as COG/TIFF, NPY, and optional TFRecord.
A validated live run over an Urla area of interest produced 990 training patches from 9 Sentinel-2 L2A scenes in June 2025.
The interim Urla AOI is a rectangle, not an official boundary. Spatial blocks reduce leakage but do not eliminate it where fields cross block edges. The label stage requires externally prepared ground-truth polygons; nothing is downloaded automatically. And the pipeline prepares data — it makes no claim about downstream model accuracy.