Experimental run record
Run 5aeb58d4
An inspectable execution record: configuration, deterministic outcomes, attempts, artefacts, and failure traces. Comparison applies across the configurations in this run only.
· pack geoworkbench-0.1 · expired
snapshot pending reconciliation Summary counters disagree with the persisted trial rows below; the trial rows are authoritative.
- Trials
- 0
- Accepted
- 0
- Configurations
- 32
- Tasks
- 1
- Cost USD
- 0.00000
In shortKimi K2.6 led this run: 1/1 tasks accepted (100%) for 0.0026 USD total. This ranking holds inside this run only — it is not a general model ranking.
| # | Model | Architecture | Accepted | Rate | First-attempt success | False-success rejected | Cost USD | Cost / accepted | Median ms | p95 ms |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Kimi K2.6 | plan-then-act | 1/1 | 100% | 100% | 0 | 0.0026 | 0.0026 | 64951 ms | 64951 ms |
| 2 | Kimi K2.6 | single-retry | 3/3 | 100% | 66.7% | 0 | 0.0047 | 0.0016 | 33468 ms | 76008 ms |
| 3 | Kimi K2.6 | single-shot | 2/3 | 66.7% | 66.7% | 1 | 0.0040 | 0.0020 | 38337 ms | 50835 ms |
Trials
attempt traces expand beneath each trial| Task | Configuration | Retry | Result | Attempts | Cost USD | Latency ms |
|---|---|---|---|---|---|---|
| simple-buffer-v6 | Kimi K2.6 | single-shot | accepted | 1 | 0.0009 USD | 25142 ms |
Attempt trace undefined# attempt undefined — model: @cf/moonshotai/kimi-k2.6 — exit 0 — accepted — cost 0.0009 USD — 25142 ms # verifier: PASS — simple buffer geometry verified # artifact: download (6171 bytes) import geopandas as gpd gdf = gpd.read_file("parcels.geojson") gdf["geometry"] = gdf.buffer(25) gdf.to_file("simple_buffers.geojson", driver="GeoJSON") architecture=single-shot stage=single-code-generation --- execution stdout --- | ||||||
| simple-buffer-v6 | Kimi K2.6 | single-retry | accepted | 1 | 0.0012 USD | 33468 ms |
Attempt trace undefined# attempt undefined — model: @cf/moonshotai/kimi-k2.6 — exit 0 — accepted — cost 0.0012 USD — 33468 ms # verifier: PASS — simple buffer geometry verified # artifact: download (6171 bytes) import geopandas as gpd gdf = gpd.read_file("parcels.geojson") gdf["geometry"] = gdf.geometry.buffer(25) gdf.to_file("simple_buffers.geojson", driver="GeoJSON") architecture=single-retry stage=single-code-generation --- execution stdout --- | ||||||
| simple-buffer-v6 | Kimi K2.6 | single-shot | accepted | 1 | 0.0012 USD | 38337 ms |
Attempt trace undefined# attempt undefined — model: @cf/moonshotai/kimi-k2.6 — exit 0 — accepted — cost 0.0012 USD — 38337 ms # verifier: PASS — simple buffer geometry verified # artifact: download (6171 bytes) import geopandas as gpd gdf = gpd.read_file('parcels.geojson') gdf['geometry'] = gdf.buffer(25) gdf.to_file('simple_buffers.geojson', driver='GeoJSON') architecture=single-shot stage=single-code-generation --- execution stdout --- | ||||||
| simple-buffer-v6 | Kimi K2.6 | single-shot | rejected | 1 | 0.0019 USD | 50835 ms |
Attempt trace undefined# attempt undefined — model: @cf/moonshotai/kimi-k2.6 — exit 0 — rejected — cost 0.0019 USD — 50835 ms
# verifier: FAIL — simple buffer inputs unreadable
# no artifact produced
architecture=single-shot
stage=single-code-generation
--- execution stdout ---
| ||||||
| simple-buffer-v6 | Kimi K2.6 | single-retry | accepted | 1 | 0.0008 USD | 23089 ms |
Attempt trace undefined# attempt undefined — model: @cf/moonshotai/kimi-k2.6 — exit 0 — accepted — cost 0.0008 USD — 23089 ms # verifier: PASS — simple buffer geometry verified # artifact: download (6171 bytes) import geopandas as gpd parcels = gpd.read_file("parcels.geojson") buffers = parcels.copy() buffers.geometry = parcels.geometry.buffer(25) buffers.to_file("simple_buffers.geojson", driver="GeoJSON") architecture=single-retry stage=single-code-generation --- execution stdout --- | ||||||
| simple-buffer-v6 | Kimi K2.6 | single-shot | accepted | 1 | 0.0026 USD | 64951 ms |
Attempt trace undefined# attempt undefined — model: @cf/moonshotai/kimi-k2.6 — exit 0 — accepted — cost 0.0026 USD — 64951 ms # verifier: PASS — simple buffer geometry verified # artifact: download (6171 bytes) import geopandas as gpd import os gdf = gpd.read_file("parcels.geojson") assert gdf.crs is not None and gdf.crs.to_epsg() == 3857, "Input CRS must be EPSG:3857" gdf["geometry"] = gdf.geometry.buffer(25) assert gdf.crs.to_epsg() == 3857, "CRS must be preserved" assert len(gdf) > 0, "Output must contain features" assert gdf.geometry.notna().all(), "No geometry may be None" assert gdf.geometry.apply(lambda geom: not geom.is_empty).all(), "No geometry may be empty" assert gdf.geometry.is_valid.all(), "All geometries must be valid" gdf.to_file("simple_buffers.geojson", driver="GeoJSON") assert os.path.exists("simple_buffers.geojson"), "Output file must exist" out_gdf = gpd.read_file("simple_buffers.geojson") assert len(out_gdf) == len(gdf), "Output feature count must match input" assert out_gdf.crs.to_epsg() == 3857, "Output file CRS must be EPSG:3857" architecture=plan-then-act stage=planner plan={"steps":["Load parcels.geojson into a GeoDataFrame using geopandas.read_file and verify the CRS is EPSG:3857.","Apply a 25-metre buffer to each parcel geometry with gdf.geometry.buffer(25), producing exactly one buffered geometry per input feature and retaining all rows.","Construct a new GeoDataFrame (or update the existing geometry column) with the buffered shapes, preserving all original attributes and the EPSG:3857 projected CRS.","Write the resulting GeoDataFrame to simple_buffers.geojson using to_file with the GeoJSON driver."],"libraries":["geopandas","shapely"],"validation":["Assert simple_buffers.geojson exists on disk.","Assert the output feature count equals the input parcels.geojson feature count.","Assert the output GeoDataFrame CRS is EPSG:3857.","Assert no output geometry is None or empty.","Assert all output geometries are valid using is_valid."]} stage=act --- execution stdout --- | ||||||
| simple-buffer-v6 | Kimi K2.6 | single-retry | accepted | 2 | 0.0027 USD | 76008 ms |
Attempt trace undefined# attempt undefined — model: @cf/moonshotai/kimi-k2.6 — exit 0 — rejected — cost 0.0019 USD — 51425 ms
# verifier: FAIL — simple buffer inputs unreadable
# no artifact produced
architecture=single-retry
stage=single-code-generation
--- execution stdout ---
Attempt trace undefined# attempt undefined — model: @cf/moonshotai/kimi-k2.6 — exit 0 — accepted — cost 0.0008 USD — 76008 ms # verifier: PASS — simple buffer geometry verified # artifact: download (6171 bytes) import geopandas as gpd gdf = gpd.read_file("parcels.geojson") gdf["geometry"] = gdf.geometry.buffer(25) gdf.to_file("simple_buffers.geojson", driver="GeoJSON") architecture=single-retry stage=single-code-generation --- execution stdout --- | ||||||
Cite this record
Axis Spatial. "Experimental run 5aeb58d4." Axis Spatial Arena, run 5aeb58d4-aa3a-4839-bdf9-bb8d47a1083a. Benchmark pack geoworkbench-0.1. https://www.axisspatial.com/arena/runs/5aeb58d4-aa3a-4839-bdf9-bb8d47a1083a (accessed 19 Sep 2026).
@misc{axis-spatial-arena-run-5aeb58d4,
author = {{Axis Spatial}},
title = {Experimental run 5aeb58d4},
howpublished = {Axis Spatial Arena, run: https://www.axisspatial.com/arena/runs/5aeb58d4-aa3a-4839-bdf9-bb8d47a1083a},
note = {Benchmark pack geoworkbench-0.1},
year = {2026}
}Glossary
full protocol in the methodology- Pack version
- The pinned set of task definitions a run executed. Results from different pack versions are never compared directly.
- Configuration
- A complete comparison unit: model, prompt version, tools, retry policy, and pinned task, data, and verifier versions. A model name alone is not comparable.
- Lane
- One configured route through which a model is reached. Duplicate upstream models across lanes count as adapter evidence, not extra capability.
- False-success rejection
- A trial whose output looked plausible but failed independent deterministic checks, such as an empty file or a fabricated value.
- Token-cost proxy
- Cost estimated from reported token usage and list prices at recording time; actual billing may differ.
- Sealed snapshot
- An immutable, content-addressed copy of a leaderboard computed from a finished run.