Experimental run record
Run df7df63d
An inspectable execution record: configuration, deterministic outcomes, attempts, artefacts, and failure traces. Comparison applies across the configurations in this run only.
· pack geoworkbench-0.1 · accepted · sealed snapshot 0cbcce31
- Trials
- 12
- Accepted
- 6
- Configurations
- 4
- Tasks
- 3
- Cost USD
- 0.02673
In shortKimi K2.6 led this run: 2/3 tasks accepted (66.7%) for 0.0039 USD total. This ranking holds inside this run only — it is not a general model ranking.
| # | Model | Architecture | Accepted | Rate | First-attempt success | False-success rejected | Cost USD | Cost / accepted | Median ms | p95 ms |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Kimi K2.6 | single-shot | 2/3 | 66.7% | 66.7% | 1 | 0.0039 | 0.0020 | 28078 ms | 42037 ms |
| 2 | GLM-4.7 Flash | single-shot | 2/3 | 66.7% | 66.7% | 1 | 0.0046 | 0.0023 | 35526 ms | 71853 ms |
| 3 | GLM-4.7 Flash | single-retry | 1/3 | 33.3% | 33.3% | 2 | 0.0090 | 0.0090 | 75372 ms | 83968 ms |
| 4 | Kimi K2.6 | single-retry | 1/3 | 33.3% | 33.3% | 2 | 0.0091 | 0.0091 | 73802 ms | 95205 ms |
Trials
attempt traces expand beneath each trial| Task | Configuration | Retry | Result | Attempts | Cost USD | Latency ms |
|---|---|---|---|---|---|---|
| crs-reproject-munich | GLM-4.7 Flash | single-shot | accepted | 1 | 0.0015 USD | 35526 ms |
Attempt trace undefined# attempt undefined — model: @cf/zai-org/glm-4.7-flash — exit 0 — accepted — cost 0.0015 USD — 35526 ms # verifier: PASS — CRS reproject to WGS84 verified # artifact: download (591 bytes) import geopandas as gpd import pyproj # Read the input file gdf = gpd.read_file("input.geojson") # Diagnose the CRS # The coordinates are large (millions), indicating a projected CRS in metres. # Based on the task description, we determine this is EPSG:3857 (Web Mercator). gdf.crs = "EPSG:3857" # Reproject to WGS84 (EPSG:4326) gdf_wgs84 = gdf.to_crs("EPSG:4326") # Write the result to output.geojson gdf_wgs84.to_file("output.geojson", driver="GeoJSON") | ||||||
| crs-reproject-munich | GLM-4.7 Flash | single-retry | rejected | 2 | 0.0036 USD | 83968 ms |
Attempt trace undefined# attempt undefined — model: @cf/zai-org/glm-4.7-flash — exit 0 — rejected — cost 0.0019 USD — 40707 ms
# verifier: FAIL — verifier did not emit JSON; stdout tail:
# no artifact produced
Attempt trace undefined# attempt undefined — model: @cf/zai-org/glm-4.7-flash — exit 0 — rejected — cost 0.0017 USD — 83968 ms # verifier: FAIL — output not in WGS84 Munich coords # artifact: download (582 bytes) import geopandas as gpd # Load the input file gdf = gpd.read_file("input.geojson") # The coordinates are large (millions), indicating a projected CRS. # As per the task instructions, we determine this is EPSG:3857. if gdf.crs is None: gdf.crs = "EPSG:3857" # Reproject to WGS84 (EPSG:4326) gdf_wgs84 = gdf.to_crs("EPSG:4326") # Write the output gdf_wgs84.to_file("output.geojson", driver="GeoJSON") | ||||||
| raster-zonal-mean | GLM-4.7 Flash | single-shot | rejected | 1 | 0.0019 USD | 71853 ms |
Attempt trace undefined# attempt undefined — model: @cf/zai-org/glm-4.7-flash — exit 0 — rejected — cost 0.0019 USD — 71853 ms
# verifier: FAIL — stats.json unreadable
# no artifact produced
| ||||||
| raster-zonal-mean | GLM-4.7 Flash | single-retry | rejected | 2 | 0.0039 USD | 75372 ms |
Attempt trace undefined# attempt undefined — model: @cf/zai-org/glm-4.7-flash — exit 1 — rejected — cost 0.0017 USD — 32013 ms
# verifier: FAIL — stats.json unreadable
# no artifact produced
import rasterio
import numpy as np
import json
from shapely import geometry
def main():
# Load the raster
with rasterio.open('grid.asc') as src:
data = src.read(1)
transform = src.transform
shape = src.shape
nodata = src.nodata
# Load the polygon geometry
with open('zone.geojson', 'r') as f:
geojson_data = json.load(f)
# Extract geometry from GeoJSON feature
geom = geometry.shape(geojson_data['features'][0]['geometry'])
# Create a boolean mask for pixels inside the polygon
mask = rasterio.features.geometry_mask(
[geom],
transform=transform,
out_shape=shape,
invert=False
)
# Apply the mask to the raster data
masked_values = data[mask]
# Filter out nodata values
if nodata is not None:
masked_values = masked_values[masked_values != nodata]
# Filter out NaN values
masked_values = masked_values[~np.isnan(masked_values)]
# Calculate the mean
if len(masked_values) > 0:
mean_val = np.mean(masked_values)
else:
mean_val = None
# Write the result to stats.json
result = {"mean": mean_val}
with open('stats.json', 'w') as f:
json.dump(result, f)
if __name__ == "__main__":
main()
--- stderr ---
Traceback (most recent call last):
File "/tmp/arena/solution.py", line 51, in <module>
main()
File "/tmp/arena/solution.py", line 22, in main
mask = rasterio.features.geometry_mask(
^^^^^^^^^^^^^^^^^
AttributeError: module 'rasterio' has no attribute 'features'. Did you mean: '_features'?
Attempt trace undefined# attempt undefined — model: @cf/zai-org/glm-4.7-flash — exit 0 — rejected — cost 0.0022 USD — 75372 ms
# verifier: FAIL — stats.json unreadable
# no artifact produced
| ||||||
| vector-buffer-1km | GLM-4.7 Flash | single-shot | accepted | 1 | 0.0013 USD | 24848 ms |
Attempt trace undefined# attempt undefined — model: @cf/zai-org/glm-4.7-flash — exit 0 — accepted — cost 0.0013 USD — 24848 ms # verifier: PASS — 1000m buffers verified # artifact: download (5935 bytes) import geopandas as gpd from pyproj import CRS # Read input data gdf = gpd.read_file("sites.geojson") # Define coordinate reference systems src_crs = CRS.from_epsg(4326) tgt_crs = CRS.from_epsg(3857) # Reproject to projected CRS for accurate buffering gdf_proj = gdf.to_crs(tgt_crs) # Create 1000m buffers gdf_proj['geometry'] = gdf_proj.buffer(1000) # Reproject back to WGS84 gdf_final = gdf_proj.to_crs(src_crs) # Write output gdf_final.to_file("buffers.geojson", driver="GeoJSON") | ||||||
| vector-buffer-1km | GLM-4.7 Flash | single-retry | accepted | 1 | 0.0015 USD | 30656 ms |
Attempt trace undefined# attempt undefined — model: @cf/zai-org/glm-4.7-flash — exit 0 — accepted — cost 0.0015 USD — 30656 ms # verifier: PASS — 1000m buffers verified # artifact: download (5903 bytes) import geopandas as gpd # Read the input file gdf = gpd.read_file('sites.geojson') # Define coordinate reference systems # Source: WGS84 (EPSG:4326) # Target: Web Mercator (EPSG:3857) for accurate meter-based buffering src_crs = 'EPSG:4326' dst_crs = 'EPSG:3857' # Reproject to projected CRS to ensure 1000m distance is accurate gdf_projected = gdf.to_crs(dst_crs) # Create 1000m buffers gdf_projected['buffer'] = gdf_projected.geometry.buffer(1000) # Extract buffer geometries and reproject back to WGS84 buffers_gdf = gpd.GeoDataFrame(geometry=gdf_projected['buffer'], crs=dst_crs) buffers_gdf = buffers_gdf.to_crs(src_crs) # Write the output file buffers_gdf.to_file('buffers.geojson', driver='GeoJSON') | ||||||
| crs-reproject-munich | Kimi K2.6 | single-shot | accepted | 1 | 0.0009 USD | 20975 ms |
Attempt trace undefined# attempt undefined — model: @cf/moonshotai/kimi-k2.6 — exit 0 — accepted — cost 0.0009 USD — 20975 ms # verifier: PASS — CRS reproject to WGS84 verified # artifact: download (591 bytes) import geopandas as gpd gdf = gpd.read_file("input.geojson") gdf = gdf.set_crs(epsg=3857, allow_override=True) gdf = gdf.to_crs(epsg=4326) gdf.to_file("output.geojson", driver="GeoJSON") | ||||||
| crs-reproject-munich | Kimi K2.6 | single-retry | rejected | 2 | 0.0039 USD | 73802 ms |
Attempt trace undefined# attempt undefined — model: @cf/moonshotai/kimi-k2.6 — exit 0 — rejected — cost 0.0019 USD — 41038 ms
# verifier: FAIL — verifier did not emit JSON; stdout tail:
# no artifact produced
Attempt trace undefined# attempt undefined — model: @cf/moonshotai/kimi-k2.6 — exit 0 — rejected — cost 0.0020 USD — 73802 ms
# verifier: FAIL — verifier did not emit JSON; stdout tail:
# no artifact produced
| ||||||
| raster-zonal-mean | Kimi K2.6 | single-shot | rejected | 1 | 0.0019 USD | 42037 ms |
Attempt trace undefined# attempt undefined — model: @cf/moonshotai/kimi-k2.6 — exit 0 — rejected — cost 0.0019 USD — 42037 ms
# verifier: FAIL — stats.json unreadable
# no artifact produced
| ||||||
| raster-zonal-mean | Kimi K2.6 | single-retry | rejected | 2 | 0.0038 USD | 95205 ms |
Attempt trace undefined# attempt undefined — model: @cf/moonshotai/kimi-k2.6 — exit 0 — rejected — cost 0.0019 USD — 45507 ms
# verifier: FAIL — stats.json unreadable
# no artifact produced
Attempt trace undefined# attempt undefined — model: @cf/moonshotai/kimi-k2.6 — exit 0 — rejected — cost 0.0019 USD — 95205 ms
# verifier: FAIL — stats.json unreadable
# no artifact produced
| ||||||
| vector-buffer-1km | Kimi K2.6 | single-shot | accepted | 1 | 0.0012 USD | 28078 ms |
Attempt trace undefined# attempt undefined — model: @cf/moonshotai/kimi-k2.6 — exit 0 — accepted — cost 0.0012 USD — 28078 ms # verifier: PASS — 1000m buffers verified # artifact: download (5935 bytes) import geopandas as gpd gdf = gpd.read_file("sites.geojson") gdf_proj = gdf.to_crs("EPSG:3857") gdf_proj["geometry"] = gdf_proj.buffer(1000) gdf_out = gdf_proj.to_crs("EPSG:4326") gdf_out.to_file("buffers.geojson", driver="GeoJSON") | ||||||
| vector-buffer-1km | Kimi K2.6 | single-retry | accepted | 1 | 0.0014 USD | 31172 ms |
Attempt trace undefined# attempt undefined — model: @cf/moonshotai/kimi-k2.6 — exit 0 — accepted — cost 0.0014 USD — 31172 ms # verifier: PASS — 1000m buffers verified # artifact: download (5935 bytes) import geopandas as gpd sites = gpd.read_file("sites.geojson") if sites.crs is None: sites.set_crs(epsg=4326, inplace=True) buffers = sites.to_crs(epsg=3857) buffers.geometry = buffers.buffer(1000) buffers = buffers.to_crs(epsg=4326) buffers.to_file("buffers.geojson", driver="GeoJSON") | ||||||
Cite this record
Axis Spatial. "Experimental run df7df63d." Axis Spatial Arena, run df7df63d-6f59-4b78-85fe-65676d5f7543. Benchmark pack geoworkbench-0.1. https://www.axisspatial.com/arena/runs/df7df63d-6f59-4b78-85fe-65676d5f7543 (accessed 19 Sep 2026).
@misc{axis-spatial-arena-run-df7df63d,
author = {{Axis Spatial}},
title = {Experimental run df7df63d},
howpublished = {Axis Spatial Arena, run: https://www.axisspatial.com/arena/runs/df7df63d-6f59-4b78-85fe-65676d5f7543},
note = {Benchmark pack geoworkbench-0.1},
year = {2026}
}Glossary
full protocol in the methodology- Pack version
- The pinned set of task definitions a run executed. Results from different pack versions are never compared directly.
- Configuration
- A complete comparison unit: model, prompt version, tools, retry policy, and pinned task, data, and verifier versions. A model name alone is not comparable.
- Lane
- One configured route through which a model is reached. Duplicate upstream models across lanes count as adapter evidence, not extra capability.
- False-success rejection
- A trial whose output looked plausible but failed independent deterministic checks, such as an empty file or a fabricated value.
- Token-cost proxy
- Cost estimated from reported token usage and list prices at recording time; actual billing may differ.
- Sealed snapshot
- An immutable, content-addressed copy of a leaderboard computed from a finished run.