Geospatial in Cloud: AWS

AWS Geospatial Services

Eight AWS services matter for geospatial. Most production workloads use only four. Here is the full map so you know what exists - and what to ignore.

Glue: The Orchestrator

AWS Glue is where most geospatial pipelines should start. It comes in three flavours, each suited to a different scale of spatial workload. Python Shell for small data (under 1GB) - think geocoding a CSV or buffering a few thousand parcels. Spark ETL for heavy lifting - spatial joins on millions of features using Apache Sedona. And Visual ETL for drag-and-drop data transforms when your team includes analysts who do not write code.

The critical distinction: Glue Python Shell is single-node (max 1 DPU, 16GB memory). For anything involving spatial joins at scale, distributed raster processing, or datasets exceeding a few gigabytes, you need Glue Spark ETL. This is where Sedona comes in - Glue 4.0 supports Sedona natively, giving you distributed spatial SQL across a managed Spark cluster without provisioning a single server.

Sedona on Glue requires Spark ETL, not Python Shell. This is the single most common mistake we see in production. Teams set up a Python Shell job, try to import Sedona, and get cryptic import errors. Python Shell does not run Spark. For Sedona spatial SQL, you need a Spark ETL job with the Sedona JAR as a dependency.

Glue jobs install Python dependencies via the --additional-python-modules flag - comma-separated package names like geopandas, shapely, fiona, rasterio. For Sedona on Glue 4.0, the Spark JARs are pre-installed - you only need --additional-python-modules: apache-sedona for the Python API. On older Glue versions, use --extra-jars with S3 URIs pointing to JARs you have uploaded. A critical lesson from production: passing Maven coordinates instead of S3 URIs to --extra-jars causes a URISyntaxException that tells you nothing useful about the actual problem.

Code structure Glue expects: a main.py entry point (the ScriptLocation) that imports flat step modules listed in --extra-py-files as S3 URIs. No package hierarchy - Glue cannot resolve nested imports. The main.py initialises GlueContext and Sedona, parses runtime arguments via getResolvedOptions, then calls each step in sequence. This is fundamentally different from how local Python projects are structured, and it is the gap that causes most deployment failures when migrating from desktop ArcPy to cloud Glue.

Glue replaces three legacy tools simultaneously: FME for ETL transforms (Glue + Sedona covers the same spatial operations), scheduled ArcPy scripts (Glue jobs with EventBridge cron triggers), and file geodatabase management (GeoParquet on S3 with Glue Data Catalog for schema discovery). The Glue Data Catalog acts as a metadata layer that makes your S3 data queryable from Athena without any additional infrastructure.

Lambda for Processing

The cold start problem is the number one issue nobody talks about openly. GDAL + rasterio + numpy together form a ~150MB Lambda layer. The first invocation after a period of inactivity takes 8-12 seconds to initialise. That is unacceptable for real-time APIs but perfectly fine for batch processing and event-driven pipelines.

Warm invocations run in 200-400ms for typical raster tile generation. The trick is deciding whether you need provisioned concurrency ($0.015/GB-hour) or can accept cold starts for batch and event-driven processing.

The production pattern is straightforward: Lambda receives an S3 event with the bucket and key of a newly uploaded raster. It opens the COG via GDAL's virtual filesystem (which issues HTTP range requests to S3), reads only the spatial window needed using the bounding box from the event payload, and writes the result back to S3. The entire operation touches only the pixels it needs - not the full file.

Deployment tip: use Docker images, not zip layers. The 250MB unzipped size limit for Lambda layers is too small for GDAL + rasterio + numpy. Docker-based Lambda images support up to 10GB, and community base images like lambgeo/lambda-gdal include GDAL, rasterio, and numpy pre-compiled for Lambda's Amazon Linux runtime. This eliminates the most common deployment failure.

S3 + COG Storage

Why COG matters on S3: HTTP range requests mean Lambda reads only the pixels it needs. A 50GB satellite image can serve a 1km x 1km tile in 50-200ms without downloading the entire file. This is the pattern that makes serverless geospatial economically viable.

Storage cost: $0.023/GB/month on S3 Standard. A 1TB raster catalogue costs $23/month. For archival data accessed less than once a month, S3 Glacier Instant Retrieval drops this to $0.004/GB/month - an 83% reduction.

In practice, you open a COG on S3 directly using GDAL's virtual filesystem driver. When you request a windowed read - say, the 1km x 1km area around a specific bounding box - GDAL translates that into HTTP range requests against S3. It reads only the internal tiles that overlap your window, typically 50-200ms for a 50GB raster. The rest of the file is never touched. This is why COG format matters: without internal tiling and overviews, GDAL would need to download the entire file to extract a small region.

S3 layout convention matters for cost. Partition your data by date and region using Hive-style prefixes (raw/sentinel-2/year=2024/month=06/) so Athena and Glue crawlers skip irrelevant files. Store processed outputs separately from raw inputs. And always use GeoParquet for vector data - it gives you columnar storage with spatial metadata that both Athena and Glue can exploit for predicate pushdown.

The STAC catalogue pattern ties this together: index your COGs on S3 with a STAC API. Query by date, bounding box, cloud cover - then access individual COGs directly via range requests. No database, no file server, just S3 + metadata.

For detail on COG, GeoParquet, and STAC formats, see our cloud-native geospatial formats guide.

Athena Spatial SQL

Athena lets you query GeoParquet data directly on S3 with standard SQL. No database to manage, no infrastructure to provision. Point Athena at an S3 bucket, define an external table via Glue Data Catalog, and query.

The workflow: Glue computes H3 indices and writes them as a column in your GeoParquet files. Athena queries filter by H3 cell values using simple equality or range predicates. No geometry comparison at query time, no spatial index to maintain. A query on "all parcels in Berlin" becomes a filter on H3 cells that intersect Berlin's boundary - computed once during data preparation, queried thousands of times in Athena.

Coordinate order warning: H3 functions expect (latitude, longitude) - the opposite of most GIS tools. If your H3 cells produce zero matches on data you know overlaps, check the axis order first. This is the same trap on Databricks Mosaic. On GCP (BigQuery), it is the reverse: ST_GEOGPOINT expects longitude first, latitude second. Every platform has its own convention.

Cost: $5 per TB of data scanned. With GeoParquet's columnar format, you typically scan 10-20% of the total data (only the columns you reference), so effective cost is $0.50-$1.00 per TB of actual data stored. For 100 queries a month on a 1TB dataset, that is roughly $5-10/month total.

The limitation: Athena is not a real-time database. Query execution takes 3-15 seconds depending on data volume and complexity. For sub-second spatial queries serving a web application, use PostGIS on RDS. Athena is for ad-hoc analysis and batch reporting.

Step Functions Pipelines

Geospatial workflows are rarely a single operation. Satellite imagery arrives, needs validation, conversion to COG, quality checks, and catalogue updates. Step Functions orchestrates this without you managing any servers - and it coordinates both Glue jobs and Lambda functions in the same pipeline.

The key pattern from production: use the .sync suffix on Glue resource ARNs so Step Functions waits for job completion before advancing. Pass S3 URIs between states (never inline data - there is a 256KB payload limit). Set explicit TimeoutSeconds for long-running raster operations. And use Parallel branches when steps are independent - processing vectors and rasters simultaneously rather than sequentially cuts pipeline time in half.

A real production pipeline - satellite imagery ingestion:

Real Benchmarks

These numbers come from production workloads running in us-east-1. Lambda configured with 1769MB memory (1 full vCPU). S3 Standard storage class. We include the cold start numbers because pretending they do not exist helps nobody.

Cost Analysis

The cost advantage of AWS for geospatial is real - but only if you have the engineering capability to build and maintain the stack. Here is the side-by-side comparison for a mid-sized team processing 10K tiles per day with 1TB of raster storage.

When NOT to Use AWS for Geospatial

We build geospatial pipelines on AWS for clients. We still tell some of them not to. These are the scenarios where AWS is the wrong choice:

Reference Architecture

This is the production architecture we deploy for clients processing 1TB+ of geospatial data. Four layers, four AWS services per layer, no over-engineering.

The STAC catalogue sits alongside this: a Lambda-backed API with DynamoDB storage that indexes all processed data. Users query the catalogue by bounding box, date range, or sensor type, then access individual COGs directly from S3 via range requests. Total infrastructure cost for the catalogue: under $10/month.