Module 11: MLOps and Cloud Deployment

Phase: IV, Production and the job
Level: Advanced
Estimated time: ~12 hours (about 3 h concepts, 8 h lab, 1 h write-up)
Prerequisites: Module 10 (firewatch), Phases I to III.
Portfolio thread: deploy and schedule firewatch, publish a live demo map, and write Public write-up #3.

Why this module

The single biggest gap between "scientist" and "ML engineer" on job descriptions is not modeling; it is Docker, orchestration, object storage, and CI. A candidate who can containerize a geospatial pipeline, schedule it in the cloud, serve the outputs as tiles and a STAC catalog, and show a green CI badge is immediately more hireable than one with a slightly better model and no deployment story. A week spent here is worth more to your prospects than another percentage point of IoU. This module turns firewatch from a script into a running service.

Learning outcomes

After this module you can:

Containerize a geospatial application with a multi-stage Docker build on a GDAL-capable base image.
Use cloud object storage and batch compute within a free tier or credits.
Orchestrate a scheduled, observable pipeline with retries and alerts.
Serve outputs cloud-natively with a tile server, a STAC catalog, and Zarr time cubes.
Package a model for portable inference and reason about drift monitoring.
Set up CI and deploy the firewatch service with a public demo map.

Concept lessons (about 3 hours)

Lesson 1: Containerization

Geospatial dependencies (GDAL and the native stack) are the classic "works on my machine" trap, so containers matter more here than in most fields. Use a multi-stage Docker build on a base image that already has GDAL, install your package, and produce a small, reproducible image. The image is the unit you deploy, so anything that runs in it runs identically in the cloud.

Lesson 2: Cloud building blocks

Four primitives cover most needs: object storage (S3 or GCS) for data and outputs, batch compute (AWS Batch, or GCP Cloud Run jobs) for bursty processing, a container registry (ECR or Artifact Registry) for your image, and IAM for least-privilege access (grant only what the job needs). Stay inside the free tier or your credits by shutting compute down when idle, the same budget discipline as any GPU work.

Lesson 3: Orchestration

A pipeline that must run on a schedule, retry on failure, and alert you when something breaks needs an orchestrator. Prefect is fast to adopt and gives you scheduled flows, retries, and observability; Airflow is the heavyweight alternative, and knowing Flyte and Temporal by name helps in interviews. Carry the idempotency and graceful-degradation habits from Module 10 into the flow so a rerun is always safe.

Lesson 4: Cloud-native serving

Do not ship static PNGs; serve outputs so others can use them. A dynamic tile server (titiler, or TiTiler-xarray for Zarr) renders your COGs and cubes to map tiles on demand. Publish your outputs as a STAC catalog (with pystac and stac-fastapi) so they are discoverable and queryable the same way you consumed input data in Module 1. For dense time cubes, store them as Zarr v3 with Icechunk, which adds ACID transactions, versioning, and time travel to array storage, so your cube has a safe, versioned history. A minimal web map (leafmap or MapLibre) then browses the events.

Lesson 5: Model packaging and monitoring

For portable, fast inference, export the model to ONNX so it runs without your training framework, and use batch inference patterns for large areas. In production, watch for drift: the input distribution can shift (a new sensor, a new season) and performance can decay, so log input statistics and periodic accuracy checks. You do not need a full monitoring platform for the portfolio, but you must be able to talk about it.

Guided lab (about 8 hours): deploy `firewatch`

Step 1: Containerize and push

Write a multi-stage Dockerfile on a GDAL base image, build the firewatch image, and push it to a registry (ECR or Artifact Registry).

Step 2: Serving helpers with tests

The pure logic you can test without any cloud:

"""serve.py: STAC item builder and a retry decorator (testable)."""
from __future__ import annotations
import functools

def build_stac_item(item_id, bbox, datetime_iso, cog_href, collection="firewatch"):
    """Minimal, valid STAC Item for an output COG."""
    w, s, e, n = bbox
    return {
        "type": "Feature",
        "stac_version": "1.0.0",
        "id": item_id,
        "collection": collection,
        "bbox": [w, s, e, n],
        "geometry": {"type": "Polygon", "coordinates": [[[w, s], [e, s], [e, n], [w, n], [w, s]]]},
        "properties": {"datetime": datetime_iso},
        "assets": {"data": {"href": cog_href,
                            "type": "image/tiff; application=geotiff; profile=cloud-optimized",
                            "roles": ["data"]}},
        "links": [],
    }

def retry(tries=3, exceptions=(Exception,)):
    """Retry a flaky call (network, cloud API) a few times before giving up."""
    def deco(fn):
        @functools.wraps(fn)
        def wrap(*a, **k):
            last = None
            for _ in range(tries):
                try:
                    return fn(*a, **k)
                except exceptions as exc:
                    last = exc
            raise last
        return wrap
    return deco

# tests/test_serve.py
from eo_portfolio.serve import build_stac_item, retry

def test_stac_item_is_valid():
    it = build_stac_item("e1", (0, 0, 1, 1), "2026-01-01T00:00:00Z", "s3://b/x.tif")
    assert it["type"] == "Feature" and len(it["bbox"]) == 4
    assert it["properties"]["datetime"] == "2026-01-01T00:00:00Z"
    ring = it["geometry"]["coordinates"][0]
    assert ring[0] == ring[-1]                      # closed polygon
    assert it["assets"]["data"]["href"].endswith(".tif")

def test_retry_succeeds_after_failures():
    calls = {"n": 0}
    @retry(tries=3)
    def flaky():
        calls["n"] += 1
        if calls["n"] < 3:
            raise ValueError("transient")
        return "ok"
    assert flaky() == "ok" and calls["n"] == 3

Step 3: Orchestrate

Write a Prefect flow: hourly FIRMS ingest, event trigger, analysis, and report written to object storage, with retries on transient failures and an alert on failure. Run the analysis steps as batch jobs.

Step 4: Serve and map

Publish outputs as a STAC catalog on object storage, stand up a titiler endpoint, store the event time cube as Zarr v3 with Icechunk, and build a minimal MapLibre or leafmap web map that browses events. This is your public demo.

Step 5: CI and commit

Add CI that runs tests, lint, and an image build on every push, including one integration test against a tiny fixture AOI. Then write Public write-up #3, "From FIRMS hotspot to damage report in under an hour," with an architecture diagram.

git add Dockerfile src/eo_portfolio/serve.py tests/test_serve.py flows/ .github/workflows/ writeups/03_firewatch_pipeline.md
git commit -m "Module 11: containerized, scheduled firewatch with STAC/titiler serving and CI"
git push

Checkpoint

Self-check (answers below).

Why do containers matter more for geospatial code than for many other domains?
What does an orchestrator give you that a cron job does not?
Why publish outputs as a STAC catalog and serve them with a tile server instead of shipping PNGs?
What does Icechunk add on top of plain Zarr, and why does it matter for a running pipeline?

Interview-style questions (practice out loud).

Design the deployment for a near-real-time EO pipeline: containers, storage, compute, scheduling, and serving.
How would you keep cloud costs bounded for a bursty satellite-processing workload?
How do you monitor a deployed model for drift, and what would trigger a retrain?
Walk through your CI: what runs on every push, and why the integration test on a tiny AOI?

Answers. (1) The native GDAL and geospatial stack is notoriously hard to install consistently, so a container that pins the whole environment removes the largest source of "works on my machine" failures.
(2) Scheduling with retries, dependency management between steps, observability and logging, alerting on failure, and safe reruns, none of which a bare cron job provides.
(3) A STAC catalog makes outputs discoverable and queryable by space and time, and a tile server renders them on demand at any zoom, so consumers pull exactly what they need rather than receiving fixed images.
(4) Icechunk adds ACID transactions, versioning, and time travel to Zarr v3, so concurrent writes are safe and the cube has a consistent, revertible history, which a running pipeline that updates data needs.

Deliverable

A containerized firewatch deployed and scheduled in the cloud within free tier or credits, publishing a STAC catalog and titiler-served tiles with a public web map, backed by a Zarr and Icechunk event cube, with CI (tests, lint, image build) green, plus Public write-up #3 and an architecture diagram. The tested serve.py proves the serving logic.

What a hiring manager sees

This is the module that most changes how a resume reads. A deployed, scheduled service with object storage, a tile server, a STAC catalog, CI, and a public demo map is the concrete evidence that you are an ML engineer, not only a modeler. Most applicants cannot show this, so it is frequently the deciding factor between two otherwise similar candidates.

Currency note

Verified August 2026. Icechunk reached 1.0 and then 2.0.0 (released 8 April 2026), an open-source ACID transactional storage engine for Zarr v3 by Earthmover; it requires Zarr v3. titiler and TiTiler-xarray, stac-fastapi and pystac, Prefect, Docker, and the AWS and GCP primitives referenced here are stable, but their APIs and free-tier terms change, so confirm current usage and pricing before the lab and pin versions in your Module 0.1 environment file. Keep cloud spend inside free tier or credits by shutting compute down when idle.