As large language models (LLMs) continue to improve at coding, the benchmarks used to evaluate their performance are steadily becoming less useful. That's because though many LLMs have similar high ...

In recent years, it has become common for developers to use coding AI in software development, and various benchmarks exist to measure the performance of coding AI. Now, a new benchmark called ...

Most AI coding benchmarks still ask the question: did the agent produce code that passes the current tests? This is a useful question, but it is too narrow. Software development is iterative.

Researchers from Stanford, Princeton, and Cornell have developed a new benchmark to more accurately evaluate the coding abilities of large language models (LLMs). Called CodeClash, the new benchmark ...

To fix the way we test and measure models, AI is learning tricks from social science. It’s not easy being one of Silicon Valley’s favorite benchmarks. SWE-Bench (pronounced “swee bench”) launched in ...

DeepSWE is quickly becoming the AI coding benchmark developers trust most. The new testing system exposed major flaws in older evaluations and showed some leading AI models may have looked stronger ...

Kimi K2.7-Code claims 30% fewer thinking tokens and a drop-in API swap path, but independent benchmarks show kernel regressions and no DeepSWE submission.

OpenAI's GPT-5.6 Sol advanced another step toward general availability on July 6 when an OpenAI Codex engineer publicly confirmed that Sol Ultra — the model's highest-compute tier — would ship inside ...