🔍 Read the full analysis: The Top-Scoring AI Setup Isn’t The One Powering My News Pipeline on ThorstenMeyerAI.com
Get smart everyday buys delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
A test of 71 AI configurations for a news-writing pipeline found GPT-6 Astra had the highest quality scores, while GPT-6 Luna offered a lower-cost production option. The publisher also reported that two fallback models produced invented quotes or unusable output and removed them from the route. The results come from one publisher’s test and do not establish how the models perform on other tasks or workloads.
GPT-6 Astra scored highest in Thorsten Meyer’s test of 71 AI configurations for writing news articles, but Meyer says his production pipeline now uses GPT-6 Luna because it performed well at a much lower reported cost. The test also led him to remove two fallback models after they produced invented quotes or output his pipeline could not use.
Meyer says the evaluation ran from September 8 to October 13, 2026, using the same prompt and real news sources as his production workflow. It covered API, subscription, free-tier and locally run open-weight models. Each source was turned into both an English and a German article, with two AI judges scoring source fidelity, fabrication, clarity about uncertainty, news value, structure and polish; German articles were also scored for language quality.
In the standard test, GPT-6 Astra scored 4.00 out of 5 in both languages and received 5.00 for source fidelity. In a further run of 12 articles, it averaged 3.93, with a reported 95% bootstrap interval of 3.83 to 4.02. Meyer reports GPT-6 Luna averaged 3.67 in that run, with an interval of 3.59 to 3.76 and source fidelity of 4.71. He says every Luna article in the run produced valid output for the pipeline.
The reported cost and speed differed substantially: Astra cost about $0.12 per article and took around 50 seconds, while Luna cost about $0.0012 in the comparison and took around 17 seconds. Meyer says production costs over the past week averaged about $0.0013 per article at a median of 17 seconds. These are figures from his workflow, not independently verified prices for all users.
Infographic 01 · Who writes the best news article?
The best writer is not the one in production
71 model configurations, one real job: turn a news source into an article for an English and a German edition. GPT-6 Astra writes the best articles. GPT-6 Luna writes them for about a hundredth of the price.
Robustness run: 12 real news sources, English edition
Dot = mean score (1–5), bar = 95% bootstrap interval. German edition mean below each score. Overlapping bars are not a real difference.
Two judges, two caveats: GPT-6.1 Sol and Claude Opus 5.5 grade every article and were also tested as writers, so they partly grade themselves. Claude models ran with effort medium unless noted.
Source: Thorsten Meyer AI model-scout benchmark, 8 Sep – 13 Oct 2026. Task: write a news article from a real source with the production prompt, English and German editions. Judges: GPT-6.1 Sol (reasoning high) and Claude Opus 5.5, both with the source in context; score 1–5, mean of both judges. Robustness run: 2 test articles plus 10 more real sources, mostly short like production; mean with 95% bootstrap interval.
Powered by Thorsten Meyer AI
Cost and Accuracy in Production
The results highlight a practical trade-off for publishers using AI to produce articles at scale: the highest-scoring model in one evaluation may not be the most workable production choice. Meyer selected GPT-6 Luna for its reported balance of quality, cost and speed, rather than paying substantially more for Astra’s higher scores.
The reported failures in the fallback chain also point to a separate operational risk. A backup model can affect published output when the primary service is unavailable, so Meyer treated fabricated quotations and invalid formatting as reasons to remove models, not merely as lower scores. The findings apply to this pipeline and test design; they do not show that the same models will rank similarly for other publishers or tasks.
As an affiliate, we earn on qualifying purchases.
How the Pipeline Test Worked
The evaluation used two source items: a longer product filing of 6,340 characters and a short political news item of 619 characters. Meyer says most of his production sources are short, making the thinner item an important part of the test. The strongest configurations then wrote 10 additional articles from real sources, mostly short ones, in what he describes as a robustness run.
Both judges saw the original source and scored articles on a five-point scale. Meyer says he treated differences below about 0.3 points in the initial two-article test as potentially noisy, and relied more on the larger follow-up run. He also counted an article only if it yielded valid output after the same repair step used in production. The methodology is publisher-reported, and the judges were AI systems rather than human editors.
“The right model is the cheapest one that clears your quality bar reliably, not the one at the top of the chart.”
— Thorsten Meyer
AI language model API subscription
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Limits of the Reported Results
The results are Meyer’s own evaluation, not an independent audit. The source does not provide the full article outputs, detailed score breakdowns for every configuration, or enough information to reproduce all costs across other providers and account plans. The judges were GPT-6.1 Sol and Claude Opus 5.5, so the assessment relied on AI scoring rather than a disclosed panel of human editors.
The report also does not establish how the models perform on a broader range of topics, longer source documents, breaking-news updates or other language editions. Meyer says all Luna articles in the robustness run were valid, but that result does not guarantee future output will be error-free. The supplied account ends with an incomplete description of Luna’s current routing position, so the full failover order after the removals is not clear.
As an affiliate, we earn on qualifying purchases.
Further Testing and Pipeline Changes
Meyer says GPT-6 Luna remains the production model and that the two fallback models were removed from the news route on October 7. His reported tests suggest that publishers considering similar systems would need to evaluate models on their own sources, prompts, languages and formatting requirements rather than rely on general leaderboard positions.
The source does not set out a date for a new test or say which models, if any, will replace the removed fallbacks. Further results would be needed to show whether the current quality and cost pattern holds over a larger and more varied set of news articles.
As an affiliate, we earn on qualifying purchases.
Key Questions
Which model scored highest in the test?
GPT-6 Astra scored 4.00 out of 5 in both English and German in the standard test and averaged 3.93 across 12 articles in the robustness run, according to Meyer.
Why is GPT-6 Luna used in production instead?
Meyer says Luna performed consistently well while costing about $0.0012 per article in the comparison, versus about $0.12 for Astra. He also reported a median production time of 17 seconds for Luna.
What happened to the fallback models?
Meyer says GPT-4.1 nano produced invented quotations and scored 1.00 for source fidelity, while local gpt-oss-120b produced no valid articles in four attempts and also generated attributed quotes. Both were removed from the route on October 7.
Did higher reasoning effort improve Luna’s articles?
Meyer tested six effort settings and reported that score differences were within the test’s noise range. The maximum setting cost about 12 times as much and took about 12 times as long as the no-effort setting, without a reported quality gain.
Do these results show which AI is best for all newsrooms?
No. The evaluation covered one publisher’s prompt, sources, languages and pipeline. Other newsrooms would need their own tests to assess accuracy, formatting, cost and performance on their content.
Source: ThorstenMeyerAI.com
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
