Experimental run record
Run 4a0b7258
An inspectable execution record: configuration, deterministic outcomes, attempts, artefacts, and failure traces. Comparison applies across the configurations in this run only.
· pack geoworkbench-0.1 · accepted · sealed snapshot 243b17d4
- Trials
- 12
- Accepted
- 5
- Configurations
- 4
- Tasks
- 3
- Cost USD
- 0.02806
In shortKimi K2.6 led this run: 3/3 tasks accepted (100%) for 0.0082 USD total. This ranking holds inside this run only — it is not a general model ranking.
| # | Model | Architecture | Accepted | Rate | First-attempt success | False-success rejected | Cost USD | Cost / accepted | Median ms | p95 ms |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Kimi K2.6 | single-retry | 3/3 | 100% | 33.3% | 0 | 0.0082 | 0.0027 | 70451 ms | 77379 ms |
| 2 | GLM-4.7 Flash | single-shot | 1/3 | 33.3% | 33.3% | 2 | 0.0051 | 0.0051 | 37812 ms | 49096 ms |
| 3 | GLM-4.7 Flash | single-retry | 1/3 | 33.3% | 33.3% | 2 | 0.0090 | 0.0090 | 79605 ms | 80983 ms |
| 4 | Kimi K2.6 | single-shot | 0/3 | 0% | 0% | 3 | 0.0058 | 0.0000 | 48695 ms | 49410 ms |
Trials
attempt traces expand beneath each trial| Task | Configuration | Retry | Result | Attempts | Cost USD | Latency ms |
|---|---|---|---|---|---|---|
| crs-reproject-munich | GLM-4.7 Flash | single-shot | rejected | 1 | 0.0019 USD | 37812 ms |
Attempt trace undefined# attempt undefined — model: @cf/zai-org/glm-4.7-flash — exit 0 — rejected — cost 0.0019 USD — 37812 ms
# verifier: FAIL — verifier did not emit JSON; stdout tail:
# no artifact produced
| ||||||
| crs-reproject-munich | GLM-4.7 Flash | single-retry | rejected | 2 | 0.0037 USD | 80983 ms |
Attempt trace undefined# attempt undefined — model: @cf/zai-org/glm-4.7-flash — exit 0 — rejected — cost 0.0016 USD — 50853 ms # verifier: FAIL — output not in WGS84 Munich coords # artifact: download (582 bytes) import geopandas as gpd # Load the input file gdf = gpd.read_file('input.geojson') # Diagnosis: Determine the CRS based on coordinate magnitude. # Large coordinates (millions) indicate a projected CRS. # Web Mercator (EPSG:3857) covers approximately -20,000,000 to 20,000,000. if gdf.crs is None: # Check if any coordinate is in the millions if (gdf['geometry'].x.abs() > 1_000_000).any(): gdf.crs = 'EPSG:3857' # Reproject to WGS84 (EPSG:4326) gdf_wgs84 = gdf.to_crs(epsg=4326) # Write the output gdf_wgs84.to_file('output.geojson', driver='GeoJSON') Attempt trace undefined# attempt undefined — model: @cf/zai-org/glm-4.7-flash — exit 0 — rejected — cost 0.0021 USD — 80983 ms
# verifier: FAIL — verifier did not emit JSON; stdout tail:
# no artifact produced
| ||||||
| raster-zonal-mean | GLM-4.7 Flash | single-shot | rejected | 1 | 0.0019 USD | 49096 ms |
Attempt trace undefined# attempt undefined — model: @cf/zai-org/glm-4.7-flash — exit 0 — rejected — cost 0.0019 USD — 49096 ms
# verifier: FAIL — stats.json unreadable
# no artifact produced
| ||||||
| raster-zonal-mean | GLM-4.7 Flash | single-retry | rejected | 2 | 0.0038 USD | 79605 ms |
Attempt trace undefined# attempt undefined — model: @cf/zai-org/glm-4.7-flash — exit 0 — rejected — cost 0.0019 USD — 38163 ms
# verifier: FAIL — stats.json unreadable
# no artifact produced
Attempt trace undefined# attempt undefined — model: @cf/zai-org/glm-4.7-flash — exit 0 — rejected — cost 0.0019 USD — 79605 ms
# verifier: FAIL — stats.json unreadable
# no artifact produced
| ||||||
| vector-buffer-1km | GLM-4.7 Flash | single-shot | accepted | 1 | 0.0013 USD | 27210 ms |
Attempt trace undefined# attempt undefined — model: @cf/zai-org/glm-4.7-flash — exit 0 — accepted — cost 0.0013 USD — 27210 ms # verifier: PASS — 1000m buffers verified # artifact: download (5935 bytes) import geopandas as gpd gdf = gpd.read_file("sites.geojson") gdf_projected = gdf.to_crs(epsg=3857) gdf_projected['geometry'] = gdf_projected.buffer(1000) gdf_final = gdf_projected.to_crs(epsg=4326) gdf_final.to_file("buffers.geojson", driver="GeoJSON") | ||||||
| vector-buffer-1km | GLM-4.7 Flash | single-retry | accepted | 1 | 0.0015 USD | 49203 ms |
Attempt trace undefined# attempt undefined — model: @cf/zai-org/glm-4.7-flash — exit 0 — accepted — cost 0.0015 USD — 49203 ms # verifier: PASS — 1000m buffers verified # artifact: download (5903 bytes) import geopandas as gpd from pyproj import CRS # Read the input file gdf = gpd.read_file("sites.geojson") # Reproject to a projected CRS (EPSG:3857) to ensure 1000m is calculated correctly gdf_projected = gdf.to_crs("EPSG:3857") # Create 1000 metre buffers gdf_projected['buffer'] = gdf_projected.geometry.buffer(1000) # Extract the buffer geometries and reproject back to WGS84 (EPSG:4326) buffers = gdf_projected[['buffer']].copy() buffers = buffers.set_geometry('buffer') buffers = buffers.to_crs("EPSG:4326") # Write the result to the output file buffers.to_file("buffers.geojson", driver="GeoJSON") | ||||||
| crs-reproject-munich | Kimi K2.6 | single-shot | rejected | 1 | 0.0019 USD | 47932 ms |
Attempt trace undefined# attempt undefined — model: @cf/moonshotai/kimi-k2.6 — exit 0 — rejected — cost 0.0019 USD — 47932 ms
# verifier: FAIL — verifier did not emit JSON; stdout tail:
# no artifact produced
| ||||||
| crs-reproject-munich | Kimi K2.6 | single-retry | accepted | 2 | 0.0033 USD | 70451 ms |
Attempt trace undefined# attempt undefined — model: @cf/moonshotai/kimi-k2.6 — exit 1 — rejected — cost 0.0019 USD — 41471 ms
# verifier: FAIL — verifier did not emit JSON; stdout tail:
# no artifact produced
import geopandas as gpd
from pyproj import CRS
gdf = gpd.read_file('input.geojson')
source_crs = CRS.from_epsg(3857)
target_crs = CRS.from_epsg(4326)
gdf = gdf.set_crs(source
--- stderr ---
File "/tmp/arena/solution.py", line 7
gdf = gdf.set_crs(source
^
SyntaxError: '(' was never closed
Attempt trace undefined# attempt undefined — model: @cf/moonshotai/kimi-k2.6 — exit 0 — accepted — cost 0.0014 USD — 70451 ms # verifier: PASS — CRS reproject to WGS84 verified # artifact: download (591 bytes) import geopandas as gpd from pyproj import CRS gdf = gpd.read_file('input.geojson') source_crs = CRS.from_epsg(3857) target_crs = CRS.from_epsg(4326) gdf = gdf.set_crs(source_crs, allow_override=True) gdf = gdf.to_crs(target_crs) gdf.to_file('output.geojson', driver='GeoJSON') | ||||||
| raster-zonal-mean | Kimi K2.6 | single-shot | rejected | 1 | 0.0019 USD | 49410 ms |
Attempt trace undefined# attempt undefined — model: @cf/moonshotai/kimi-k2.6 — exit 0 — rejected — cost 0.0019 USD — 49410 ms
# verifier: FAIL — stats.json unreadable
# no artifact produced
| ||||||
| raster-zonal-mean | Kimi K2.6 | single-retry | accepted | 2 | 0.0035 USD | 77379 ms |
Attempt trace undefined# attempt undefined — model: @cf/moonshotai/kimi-k2.6 — exit 0 — rejected — cost 0.0019 USD — 41332 ms
# verifier: FAIL — stats.json unreadable
# no artifact produced
Attempt trace undefined# attempt undefined — model: @cf/moonshotai/kimi-k2.6 — exit 0 — accepted — cost 0.0016 USD — 77379 ms # verifier: PASS — zonal mean verified by independent recompute # artifact: download (14 bytes) import json import numpy as np import rasterio import rasterio.mask with open("zone.geojson") as f: geojson = json.load(f) if geojson["type"] == "FeatureCollection": geometry = geojson["features"][0]["geometry"] elif geojson["type"] == "Feature": geometry = geojson["geometry"] else: geometry = geojson with rasterio.open("grid.asc") as src: nodata = src.nodata out_image, _ = rasterio.mask.mask(src, [geometry], crop=True) band = out_image[0] valid_mask = np.ones(band.shape, dtype=bool) if nodata is not None: valid_mask &= band != nodata valid_mask &= ~np.isnan(band) valid_pixels = band[valid_mask] mean_value = float(np.mean(valid_pixels)) if valid_pixels.size > 0 else None with open("stats.json", "w") as f: json.dump({"mean": mean_value}, f) | ||||||
| vector-buffer-1km | Kimi K2.6 | single-shot | rejected | 1 | 0.0019 USD | 48695 ms |
Attempt trace undefined# attempt undefined — model: @cf/moonshotai/kimi-k2.6 — exit 0 — rejected — cost 0.0019 USD — 48695 ms
# verifier: FAIL — verifier did not emit JSON; stdout tail:
# no artifact produced
import geopandas as gpd
gdf = gpd.read_file('sites.geojson')
gdf = gdf.set_crs(epsg=4326, allow_override=True)
gdf = gdf
| ||||||
| vector-buffer-1km | Kimi K2.6 | single-retry | accepted | 1 | 0.0013 USD | 31421 ms |
Attempt trace undefined# attempt undefined — model: @cf/moonshotai/kimi-k2.6 — exit 0 — accepted — cost 0.0013 USD — 31421 ms # verifier: PASS — 1000m buffers verified # artifact: download (5935 bytes) import geopandas as gpd gdf = gpd.read_file('sites.geojson') if gdf.crs is None: gdf.set_crs(epsg=4326, inplace=True) gdf = gdf.to_crs(epsg=3857) gdf.geometry = gdf.buffer(1000) gdf = gdf.to_crs(epsg=4326) gdf.to_file('buffers.geojson', driver='GeoJSON') | ||||||
Cite this record
Axis Spatial. "Experimental run 4a0b7258." Axis Spatial Arena, run 4a0b7258-6a07-4da3-a8cb-83c98c43af3e. Benchmark pack geoworkbench-0.1. https://www.axisspatial.com/arena/runs/4a0b7258-6a07-4da3-a8cb-83c98c43af3e (accessed 19 Sep 2026).
@misc{axis-spatial-arena-run-4a0b7258,
author = {{Axis Spatial}},
title = {Experimental run 4a0b7258},
howpublished = {Axis Spatial Arena, run: https://www.axisspatial.com/arena/runs/4a0b7258-6a07-4da3-a8cb-83c98c43af3e},
note = {Benchmark pack geoworkbench-0.1},
year = {2026}
}Glossary
full protocol in the methodology- Pack version
- The pinned set of task definitions a run executed. Results from different pack versions are never compared directly.
- Configuration
- A complete comparison unit: model, prompt version, tools, retry policy, and pinned task, data, and verifier versions. A model name alone is not comparable.
- Lane
- One configured route through which a model is reached. Duplicate upstream models across lanes count as adapter evidence, not extra capability.
- False-success rejection
- A trial whose output looked plausible but failed independent deterministic checks, such as an empty file or a fabricated value.
- Token-cost proxy
- Cost estimated from reported token usage and list prices at recording time; actual billing may differ.
- Sealed snapshot
- An immutable, content-addressed copy of a leaderboard computed from a finished run.