🔍 Read the full analysis: Can AI Models Resist Adversarial Distillation? on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
OpenAI reports that updated safeguards in GPT-6.2 Astra lowered success in an internal test of protected-reasoning extraction from 54.0% to 0.2%. A separate internal study found that student models trained on exact reasoning traces often performed better on cyber tasks than those trained on summaries, though the results lack published benchmark scores and independent replication.
The original analysis reports that OpenAI says updated safeguards in GPT-6.2 Astra reduced the success rate of an internal test for extracting protected reasoning from 54.0% to 0.2%. The company also reports that, in a separate internal study, student models trained on exact reasoning traces generally transferred more cyber capability than students trained on summaries, raising questions about how frontier-model outputs could be reused to train systems with different safeguards.
The extraction result came from an internal adaptive evaluation using automated red-teaming. OpenAI compared GPT-6 Astra with its original safeguards against GPT-6.2 Astra with updated protections, testing whether an attacker could recover protected reasoning generated while the model answered an earlier prompt. The company has not supplied the evaluation’s full methodology, sample size or independent replication, so the reported rates describe this test rather than a general estimate of extraction risk.
OpenAI says its defenses combine model training, classifiers and API-level protections. It describes checks on individual requests as well as campaign-level monitoring, which looks for patterns across requests and accounts that could indicate coordinated extraction. The company says investigation findings and automated red-team results feed back into training and testing; it did not provide details of specific investigations or enforcement actions.
In the cyber-capability study, researchers started each training run with the same pretrained student model. Supervised fine-tuning data mainly consisted of trajectories from Astra-level teacher models working on coding and defensive cybersecurity tasks, including task steps, tool interactions and final responses. Students trained on exact traces generally outperformed summary-trained students after fine-tuning. Following an additional round of reinforcement learning, the exact-trace student scored higher at the largest inference budget tested on both reported benchmarks, while the summary-trained student did better at lower budgets. OpenAI did not name the benchmarks or publish numerical scores in the supplied account.
Combatting Adversarial Distillation
OpenAI reports stronger safeguards against attempts to extract protected reasoning, alongside evidence that exact reasoning traces can transfer cyber capability to student models.
How defenses work
OpenAI describes safeguards that look at individual requests and coordinated activity across requests and accounts.
Model training, classifiers and API-level protections aim to block reasoning extraction.
Patterns across requests and accounts can help identify organized extraction attempts.
Investigation findings and automated red-teaming feed back into training and testing.
What student models learn
OpenAI’s internal study compared training on exact reasoning traces with training on reasoning summaries.
- Students generally outperformed summary-trained students after supervised fine-tuning.
- After further reinforcement learning, the exact-trace student scored higher at the largest tested inference budget on both reported benchmarks.
- The summary-trained student scored higher at lower inference budgets after reinforcement learning.
- The study did not provide benchmark names or numerical scores.
Why it matters
Distillation can pass a teacher’s capabilities to a student model that may not have comparable safeguards.
Dual-use skills
Coding and defensive security work can also teach techniques applicable to attacks.
Sensitive research
Biological research can have both beneficial and harmful uses.
Why Reasoning Data Can Transfer Risk
The findings frame adversarial distillation as a safety issue beyond copying a model’s visible answers. If training data includes detailed traces of how a teacher solves tasks, those traces may help a student acquire useful skills, potentially without inheriting the teacher’s protections. OpenAI’s study suggests exact traces can transfer more cyber capability than summaries under the tested conditions, but it does not establish that this holds across models, datasets or training methods.
The concern is relevant to dual-use fields. Coding and defensive cybersecurity work can teach techniques that also apply to attacks, while biological research can support beneficial or harmful applications. A student model with transferred capabilities could be used in settings where the original teacher would refuse or block a request. That is a risk scenario described by the source, not a demonstrated outcome of the reported tests.
For model providers, the report points to a tension between protecting detailed outputs and enabling legitimate research, evaluation and model development. Restricting access to reasoning traces may reduce one route for capability transfer, but the supplied evidence does not show whether summaries or other data sources can be used to achieve similar results. Independent evaluations would help determine how much practical protection the measures provide.
Products Worth Considering

Logitech MX Keys S Combo Wireless Keyboard and Mouse, Graphite

Samsung Odyssey G55C 32-Inch QHD Curved Gaming Monitor

Apple iPad 11-inch (A16), 128GB, Wi-Fi, Silver
As an affiliate, we earn on qualifying purchases.
How Distillation Enters the Picture
In distillation, a student model learns from data generated or labeled by a teacher model. Depending on the method, that data can include answers, judgments or step-by-step reasoning. OpenAI uses “adversarial distillation” for unauthorized use of a frontier model’s outputs to train another system that may not have comparable safeguards.
The company’s study compared training on exact reasoning traces and reasoning summaries, then assessed student models after supervised fine-tuning and after lightweight reinforcement learning in cyber and coding environments. The reported pattern varied with inference budget: exact-trace training had an advantage at the largest tested budget after reinforcement learning, while summary-trained students scored higher at lower budgets. The source does not provide enough detail to compare the size of those differences or judge their statistical significance.
OpenAI’s extraction test addressed a related but distinct question: whether protected reasoning could be elicited from the teacher itself. The company presents request-level and campaign-level safeguards as parts of its response. The available account does not state when the updated protections were introduced or whether they are available to all users.
What the Internal Tests Leave Open
The reported 0.2% extraction success rate cannot be assessed fully from the supplied information. OpenAI has not disclosed the test’s sample size, exact success-rate calculation, full attack methodology or independent replication. The source also does not say when the safeguards were introduced or whether they apply to every user or access route.
For the cyber study, the benchmark names and numerical scores are absent, and the account does not establish how results generalize beyond the particular teachers, students, tasks, training recipes and inference budgets tested. It remains unclear how much capability summaries transfer under other conditions, or whether other sources of training data could provide similar capabilities. The source also gives no details about identified real-world extraction campaigns or enforcement actions.
Further Testing Across Models
OpenAI says it plans to continue studying capability transfer across different student models and distillation methods. It also says automated red-teaming and investigations will inform future safeguards, but the supplied account sets no timetable for additional results.
Useful next evidence would include fuller evaluation methods, benchmark names and scores, details on safeguard availability, and testing by independent teams across more models and attack approaches. Until those are available, the figures should be read as company-reported results from internal evaluations, not proof of a general reduction in extraction risk or a universal effect of exact reasoning traces.
Key Questions
What is adversarial distillation?
It is the unauthorized use of a teacher model’s outputs to train a student model, which may not have safeguards comparable to the teacher’s. The training material can include answers, judgments or reasoning traces.
What did OpenAI report about protected-reasoning extraction?
OpenAI says an internal adaptive test’s success rate fell from 54.0% to 0.2% when it compared GPT-6 Astra’s original safeguards with updated safeguards in GPT-6.2 Astra. The company did not provide the full methodology or independent replication in the supplied account.
Did exact reasoning traces always produce better student models?
No. OpenAI reports that exact-trace students generally performed better after supervised fine-tuning and scored higher at the largest tested inference budget after reinforcement learning. Summary-trained students scored higher at lower budgets.
Are the results independently verified?
The supplied source describes both evaluations as internal and does not report independent replication. It also omits key methodological details and, for the cyber study, benchmark names and numerical scores.
What is OpenAI testing next?
OpenAI says it plans further study of capability transfer across different student models and training methods. It has not provided a timetable for results in the supplied account.
Primary source: OpenAI · via ThorstenMeyerAI.com
