← Lillie AcademyCourse contentslillieearthintelligence.com

Module 1: The Cloud-Native Geospatial Stack

Phase: I, Foundations of the data, physics, and radar
Level: Intermediate
Estimated time: ~12 hours (about 4 h concepts, 6 h lab, 2 h write-up)
Prerequisites: Phase 0 complete (toolkit, EO mental model, geospatial fundamentals, the eo-portfolio repo).
Portfolio thread: starts Portfolio Project A. You build cube.py, a STAC-to-xarray cube fetcher, and produce a cloud-masked composite over your study-area fire AOI.

Why this module

Every remote-sensing and geospatial-ML job description lists the same stack: rasterio, xarray, GeoPandas, STAC, dask. The mindset shift this module delivers is to stop thinking of imagery as files you download and start thinking of it as lazily evaluated, labeled arrays with a coordinate reference system, fetched on demand from cloud catalogs. That shift is what separates someone who downloads a scene and opens it in QGIS from someone who can turn a search over a petabyte archive into an analysis-ready cube in a few lines. It is also the backbone every later module stands on, so time spent here pays off all course long.

Learning outcomes

After this module you can:

  1. Explain the STAC model (catalog, collection, item, asset) and query a STAC API by space, time, and properties.
  2. Explain cloud-optimized GeoTIFF (COG) and Zarr, and why cloud-native formats allow reading a window without downloading the whole file.
  3. Load a STAC search into a dask-backed xarray cube with odc-stac, then clip, reproject, and composite it.
  4. Use dask for lazy, chunked, parallel computation, and judge when it helps and when it hurts.
  5. Write a COG and reason about when a Zarr time cube is the better store.
  6. Work provider-agnostically by running the same workflow against more than one STAC API.

Concept lessons (about 4 hours)

Lesson 1: STAC, the index that makes archives usable

SpatioTemporal Asset Catalog (STAC) is the standard that makes petabyte archives searchable with a few HTTP calls. Its hierarchy:

You query with pystac-client: give it a collection, a bounding box, a date range, and a property filter (for example cloud cover below twenty percent), and it returns the matching items. This replaces hunting through folder trees and per-provider download portals with one uniform, scriptable interface.

Lesson 2: Cloud-native formats, COG and Zarr

Two formats make cloud-native EO possible.

Rule of thumb: COG for single-scene imagery and for outputs other tools will consume; Zarr for dense multi-date time cubes you will analyze repeatedly.

Lesson 3: xarray and dask, labeled and lazy

When dask helps: data larger than RAM, many scenes, embarrassingly parallel work. When it hurts: small data where the scheduling overhead outweighs the work, or workflows with poor chunk alignment that force expensive reshuffles. The skill is choosing sensible chunk sizes (often aligned to the COG tiling and your AOI) and not reaching for dask when a plain numpy array fits in memory.

Lesson 4: From search to a clean cube

The pipeline you will build:

  1. Search a STAC API for scenes over the AOI and date range, filtered by cloud cover.
  2. Load the matching items into an xarray cube with odc-stac, requesting only the bands, resolution, and bbox you need, chunked for dask. Use groupby="solar_day" so scenes from the same pass are merged and overlaps de-duplicated.
  3. Apply the cloud mask that ships with the product (for Sentinel-2 Level 2A, the Scene Classification Layer, SCL) so clouds and shadows are excluded.
  4. Composite over time (a per-pixel median is a robust default) to get one clean, gap-reduced image.
  5. Clip to the AOI, confirm the CRS, and write a COG.

Because the whole thing runs off STAC, the same code works against different providers by swapping the catalog URL, which is what "provider-agnostic" means in practice.

Guided lab (about 6 hours): build cube.py

Open in Colab Open in GitHub Codespaces

You will build a reusable STAC-to-xarray cube fetcher and produce a cloud-masked composite over your fire AOI. Primary data source is the Copernicus Data Space STAC API, which is open and current.

Step 1: Search helper

"""cube.py: STAC to analysis-ready xarray cube."""
from __future__ import annotations
from pystac_client import Client

# Verify these before use: CDSE is the reliable open STAC API as of 2026.
STAC_CDSE = "https://stac.dataspace.copernicus.eu/v1"

def cloud_query(max_cloud: float) -> dict:
    """Pure helper: the STAC property filter for max cloud cover (unit-testable)."""
    return {"eo:cloud_cover": {"lt": max_cloud}}

def search_scenes(bbox, datetime, max_cloud=20,
                  collection="sentinel-2-l2a", url=STAC_CDSE):
    """Return STAC items over bbox/date under the cloud threshold. (network)"""
    client = Client.open(url)
    search = client.search(collections=[collection], bbox=bbox,
                           datetime=datetime, query=cloud_query(max_cloud))
    return list(search.items())

Step 2: Load a dask-backed cube with odc-stac

import odc.stac

def load_cube(items, bbox, bands=("B04", "B03", "B02", "B08", "SCL"),
              resolution=10, chunks=None):
    """Load STAC items into a dask-backed xarray cube. (network)

    Band/asset names differ by provider: CDSE uses B04/B03/B02/B08 and SCL.
    Verify the asset keys of your collection before running.
    """
    chunks = chunks or {"x": 1024, "y": 1024}
    return odc.stac.load(
        items, bands=bands, bbox=bbox, resolution=resolution,
        groupby="solar_day", chunks=chunks,
    )

Step 3: Cloud mask and composite (pure, testable)

The Sentinel-2 SCL band labels each pixel; classes for cloud, cloud shadow, and cirrus should be dropped. The median over time then yields a clean image.

# SCL classes to exclude: 3 shadow, 8 cloud med prob, 9 cloud high prob, 10 cirrus, 11 snow
_BAD_SCL = (3, 8, 9, 10, 11)

def mask_and_composite(ds, scl_band="SCL", bad=_BAD_SCL):
    """Mask cloudy pixels via SCL, then median-composite over time.

    Pure array logic (no network); unit-testable on a synthetic cube.
    """
    import numpy as np
    good = ~ds[scl_band].isin(bad)
    optical = ds.drop_vars(scl_band).where(good)
    return optical.median(dim="time", skipna=True)

Step 4: Write a COG

def write_cog(composite, path):
    """Write an xarray dataset/array to a Cloud-Optimized GeoTIFF."""
    # composite must carry a CRS (rioxarray). Convert dataset to a band-stacked array as needed.
    arr = composite.to_array(dim="band") if hasattr(composite, "data_vars") else composite
    arr.rio.to_raster(path, driver="COG")

Step 5: Tests that need no network

import numpy as np, xarray as xr
from eo_portfolio.cube import cloud_query, mask_and_composite

def test_cloud_query():
    assert cloud_query(20) == {"eo:cloud_cover": {"lt": 20}}

def test_mask_and_composite_drops_clouds():
    # two timesteps: t0 clean value 10, t1 cloudy (SCL=9) value 999
    t, y, x = 2, 2, 2
    red = xr.DataArray(np.array([[[10,10],[10,10]], [[999,999],[999,999]]], dtype="float32"),
                       dims=("time","y","x"))
    scl = xr.DataArray(np.array([[[4,4],[4,4]], [[9,9],[9,9]]]), dims=("time","y","x"))
    ds = xr.Dataset({"B04": red, "SCL": scl})
    out = mask_and_composite(ds)
    # cloudy timestep masked out, so median equals the clean value
    assert float(out["B04"].mean()) == 10.0

Run pytest -q until green.

Step 6: The notebook and write-up

In notebooks/03_cube.ipynb: search a one-month window over your fire AOI, load the cube, mask and composite, plot a true-color image, and write the COG. In the repo README, add a short note explaining which provider you chose and why, and your COG-versus-Zarr decision for this output.

Step 7: Commit

git add src/eo_portfolio/cube.py tests/test_cube.py notebooks/03_cube.ipynb
git commit -m "Module 1: STAC-to-xarray cube fetcher with cloud-masked composite"
git push

Checkpoint

Self-check (answers below).

  1. Name the four levels of the STAC hierarchy and what an item versus an asset is.
  2. How does a COG let you read a 512 by 512 window from a 1 GB scene without downloading the whole file?
  3. You call odc.stac.load(...) and it returns instantly on a huge date range. Why, and when does the actual reading happen?
  4. Why is a per-pixel median a robust way to composite a stack of partly cloudy scenes?

Interview-style questions (practice out loud).

Answers. (1) Catalog, collection, item, asset; an item is one scene with datetime and footprint, an asset is an actual file (usually a COG band) the item points to.
(2) The COG is internally tiled with overviews and served over HTTP range requests, so the client fetches only the byte ranges for the tiles covering that window (or a coarse overview), not the whole file.
(3) odc-stac builds a lazy dask-backed cube; nothing is read until you compute or write, so construction is instant and the reads happen on .compute() or on writing output.
(4) The median ignores the extreme values that clouds and shadows introduce (after masking, remaining outliers are damped), so it recovers a clean surface value per pixel across dates.

Deliverable

A tested cube.py (search, load, mask-and-composite, write-COG), a notebooks/03_cube.ipynb that produces a true-color, cloud-masked monthly composite over your fire AOI written as a COG, and a README note on provider choice and the COG-versus-Zarr decision. This is section 1 of Portfolio Project A.

What a hiring manager sees

The competency bar for this field literally includes "go from STAC query to cloud-masked analysis-ready cube in under thirty lines, lazily, for any provider." A clean cube.py that does exactly that, provider-agnostic and tested, is the first strong signal that you work cloud-native rather than downloading everything to a laptop. It is also the reusable foundation every other project in your portfolio will import.

Currency note

Verified August 2026. The Copernicus Data Space STAC API (stac.dataspace.copernicus.eu, collection sentinel-2-l2a) is the reliable, open primary source; confirm the exact API root and collection name at study time. Microsoft's public Planetary Computer data and STAC APIs remain open and available (only the hosted Hub notebook service was retired, in June 2024), while Microsoft's newer enterprise product is the paid Planetary Computer Pro; if you use the public Planetary Computer STAC as a second provider, confirm the collection you need is still populated before relying on it. odc-stac and stackstac are both maintained; this module uses odc-stac for its groupby="solar_day" de-duplication and strong mosaic throughput, and stackstac is a fine alternative. Band and asset names differ by provider (CDSE uses B04, B03, B02, B08, SCL), so check the asset keys of your collection, and pin all versions in your Module 0.1 environment file.

Previous0.4 Git and GitHubNext2 Optical Radiometry and Burn Severity