Cloud-Native Geospatial Formats

What Problems Do Cloud-Native Geospatial Formats Solve?

Cloud-native formats solve three problems: inefficient full-file downloads, analytics platform integration gaps, and data discovery challenges. Traditional formats like Shapefile and GeoTIFF require downloading entire files to access a small region. COG, GeoParquet, and STAC enable HTTP range requests, columnar analytics, and searchable metadata. Before diving into solutions, let's be precise about what we're solving.

The precision access principle: extract only what you need from the whole.

How Does Cloud-Optimized GeoTIFF (COG) Work?

COG reorganises standard GeoTIFF with internal tiling, overviews, and header-first structure, enabling HTTP range requests to read selected regions without downloading the full file. The retained example compares reading 1km2 from a 50GB raster by full download and by range request. Its 14-minute and 2.3-second timings are dated illustrations, not current benchmarks. COG isn't a new format - it's a GeoTIFF organised in a specific way that can support efficient cloud access. In 2023, COG was adopted as an official OGC standard.

How COG Works

A standard GeoTIFF stores pixels in sequential strips. To read any pixel, you typically need to read from the beginning of the file. A COG reorganises this structure with three key features:

Tile-based access: request only the cell you need, leave the rest untouched.

Creating COGs: What Actually Matters

gdal_translate input.tif output_cog.tif \
  -of COG \
  -co COMPRESS=DEFLATE \
  -co OVERVIEW_RESAMPLING=LANCZOS \
  -co BLOCKSIZE=512

Compression: DEFLATE can provide a useful balance for some data. JPEG may fit imagery where controlled quality loss is acceptable; measure file size and output quality for the target data.

Tile Size: 512x512 can improve performance for large-area queries at the cost of slightly more data transfer for small queries.

For the Python code needed to read and write COG files with Rasterio, see our detailed ArcPy to Rasterio migration guide.

GeoParquet in Depth

If COG solves a raster access problem, GeoParquet addresses cloud-native vector data. It is a columnar format with different limits and query behaviour from Shapefile.

The Columnar Advantage

Shapefiles and File Geodatabases store data row by row. To read any attribute of any feature, you typically scan through records sequentially. Parquet stores data column by column - optimised for analytics queries that access specific attributes, aggregate across records, or filter based on values.

Row vs column: scattered access versus precise selection of what you need.

Creating GeoParquet: Practical Code

import geopandas as gpd

# Read source data
gdf = gpd.read_file("input.shp")

# Write to GeoParquet
gdf.to_parquet(
    "output.parquet",
    compression="snappy",  # Fast decompression
    index=False
)

If you're transitioning from ArcGIS, our ArcPy to GeoPandas translation guide covers the complete workflow conversion process.

STAC (SpatioTemporal Asset Catalog) in Depth

COG and GeoParquet solve access and analytics problems. STAC solves discovery.

STAC is a specification for metadata, not a data format. It defines a standard JSON structure for describing geospatial assets - what they contain, where they are, when they were captured, and how to access them.

Why This Matters at Scale

Consider: 10 years of aerial imagery, satellite data from 3 providers, LiDAR from 5 acquisition projects, and derived products like DEMs. Without STAC, finding relevant data requires knowing folder structures, understanding naming conventions, manually checking date ranges. With STAC, you query a single API:

from pystac_client import Client

catalog = Client.open("https://your-stac-api.com")

# Find all Sentinel-2 imagery for summer 2023
results = catalog.search(
    collections=["sentinel-2-l2a"],
    bbox=[-122.5, 47.5, -122.0, 48.0],
    datetime="2023-06-01/2023-08-31",
    query={"eo:cloud_cover": {"lt": 20}}
)

for item in results.items():
    print(f"{item.id}: {item.datetime}")

When NOT to Use These Formats

Cloud-native formats aren't universally superior. Here's when traditional approaches may be appropriate:

Integration with Modern Data Platforms

The strategic value of cloud-native geospatial formats is integration with enterprise data infrastructure.

df = spark.read.format("geoparquet").load("s3://bucket/parcels/")
df.select(st_area("geometry"), st_centroid("geometry")).show()

SELECT parcel_id, ST_AREA(geometry) as area_m2
FROM parcels_ext
WHERE ST_CONTAINS(aoi_polygon, geometry);

The modern stack: cloud-native formats work directly with enterprise data infrastructure.

Migration Strategy: A Practical Roadmap

Migration isn't a weekend project. Here's a phased approach that minimises risk. Define the conversion, validation, catalogue, rollback, and ownership boundaries before selecting implementation support.

The migration path: each step lighter than the last.

Questions to Ask Before You Start