Old software can be awkward to read and still do something a business depends on. A rewrite that looks cleaner can quietly change that behaviour. OpenAI’s October 19 research announcement focuses on this problem, alongside coding in languages with less training material than Python or JavaScript.
The company describes two evaluations. MultiLangBench asks models to build a SQLite-compatible database engine across 20 languages. A separate migration evaluation asks them to translate COBOL programs while preserving their observable outputs. OpenAI reports improvements from its newer models; these are publisher-reported results, not tests we ran.
OpenAI has published the MultiLangBench setup and tests on GitHub. The repository explains how an implementation receives a database and a query, then has its resulting database checked against SQLite. That makes the test procedure inspectable instead of leaving readers with only a headline score.
The migration evaluation uses hidden tests and requires all of them to pass. OpenAI highlights details such as decimal handling that can change a financial result even when a rewritten program runs successfully. Its announcement explicitly says production use still requires expert review and testing against the system’s business rules and dependencies.
Engineering
