devtake.dev
AI Security Open Source Hardware Apple Policy

#swe-bench

RSS
Related: #ai-coding · 1 #ai-models · 1 #benchmarks · 1 #claude-opus · 1 #evaluations · 1 #llm · 1 #openai · 1
OpenAI just retired SWE-bench Verified. The headline coding benchmark of 2025 is officially saturated.
AI·

OpenAI just retired SWE-bench Verified. The headline coding benchmark of 2025 is officially saturated.

OpenAI says SWE-bench Verified is saturated and contaminated, and 60% of remaining problems are unsolvable. Here's what comes next, and why every coding leaderboard is suspect.