2025 FOSS4G NA | Cloud Optimized Shapefiles - Calvin Metcalf

In this session from FOSS4G NA 2025, Calvin Metcalf of the Sanborn Map Company takes a deep dive into an unlikely hero of the modern web: **Cloud Optimized Shapefiles**. While shapefiles are often seen as a relic of the past, Calvin demonstrates how these multi-file structures can be modernized to work natively on the web through clever indexing and byte-range requests.

Calvin breaks down the anatomy of shapefiles—from the .shp and .dbf to the often-overlooked .shx, .prj, and .cpg files—and explains the engineering required to make them "cloud optimized." By implementing a QIX (spatial index) file and leveraging Hilbert curves for spatial ordering, he shows how to structure geospatial data so users only download the specific parts they need, significantly reducing data overhead and server requests.

Highlights:

🛰️ What is a Shapefile? A breakdown of the three to five files that make up the "yesterday's format of tomorrow"

🔄 Anatomy of Cloud Optimization: How byte-range requests allow for downloading specific parts of a file without the whole package

🧩 The Power of the QIX Index: Using tree-based nodes to skip irrelevant data and jump directly to spatial bounding boxes

⚙️ Big Endian vs. Little Endian: Navigating the "split the baby" compromise in shapefile data structures

🌐 The Hilbert Curve Advantage: Why space-filling curves are the superior method for keeping spatially close shapes adjacent in the file

🏢 Practical Implementation: A look at the open-source library and command-line tool for building and reordering cloud-optimized shapefiles

🏗️ Browser Hurdles: Addressing the specific Firefox bug that impacts byte-range requests on compressed files

For more content like this check out www.projectgeospatial.com

#Geospatial #FOSS4G #Shapefile #CloudOptimized #GIS #WebMapping #OpenSource #DataEngineering #HilbertCurve #SanbornMapCompany #SoftwareDevelopment #ProjectGeospatial #VectorData #WebDev #SpatialAnalysis