Part of our From Vibe to Viable series: our colleague Fan built an agent simulation project largely through GitHub Copilot, then ran it through Artemis to see what was left on the table.
The project
The code was generated the way most code is generated now — describe the intent, accept the suggestion, iterate until it runs. It worked. It passed. By the normal standard of "done", it was done.
What the benchmark showed
Then it was measured. Between the generated version and the evolved version, the difference was 35× faster. Not a rewrite in a different language, not new hardware — the same program, with the parts that dominated runtime replaced by variants that were built, run, and proven quicker.
The takeaway
AI-generated code is optimised for looking correct, because that is what the model was rewarded for. Nothing in the generation loop measures speed, memory, or cost. Until something does, "it works" and "it is efficient" remain completely different claims — and only one of them was ever checked.
