# "Open weights" tells you the license, not whether anyone can rebuild the model.

A model lands with a press post that says "open." You download a few hundred gigabytes of weights, confirm the license lets you fine-tune and redeploy, and you have what the announcement promised: an open-weights model. Now try to do the thing the word "open" is supposed to license — rebuild it. Reproduce the run from scratch, on your own hardware, and arrive at a model you can defend as equivalent. You cannot, and not because you lack the GPUs. You lack the training data, the data-mixing and filtering code, the recipe — the schedule, the hyperparameters, the order the corpus was fed in. The weights are a snapshot of an answer. The announcement implied you were getting the working that produced it, and you were not.

This is the conflation worth dismantling: "open weights" and "reproducible model" are treated as one claim, and they are two. One is a statement about a license attached to a file. The other is a statement about whether the components needed to rebuild that file were released. A model can satisfy the first completely — permissive license, no usage restrictions, weights anyone can pull — and satisfy the second not at all. The marketing bar and the reproducibility bar are not the same height, and almost every "open" model clears the first while failing the second. The point of this essay is that the distance between those two bars is not a rounding error. It is most of the model.

## Openness is graded, and weights sit at the bottom

The instinct to treat "open" as a yes/no flag is the first thing that has to go. Openness in a trained model is not one property; it is a stack of them, and you can release any subset. The Linux Foundation's _Model Openness Framework_ ([_The Model Openness Framework_](https://arxiv.org/abs/2403.13784) (arXiv 2403.13784)) makes this concrete. It defines 17 components across the model development lifecycle — data, code, the trained artifact, the documentation around each — grouped into three ascending classes. In the paper's words, "the 17 components are categorized into three distinct classes," and "each class builds upon the previous one, with Class III being the least complete and Class I being the most complete." The ladder runs Class III "Open Model," then Class II "Open Tooling," then Class I "Open Science" at the top.

Here is the detail that matters: "open weights" is not somewhere in the middle of that ladder. It is the floor. Shipping a weights file and a license puts you in the least-complete class, with the data, the training and evaluation code, and the recipe — the things that turn a snapshot into something a researcher can rebuild — sitting in the tiers above, unshipped. The framework is explicit that this is the prevailing practice and not enough: "to achieve full transparency, reproducibility, and extensibility, we argue that model producers must go beyond just releasing their model and the trained weights and biases, which is currently the norm." The norm is the floor. The word "open," in usage, points at the whole building.

So when an announcement says "open," the honest question is not whether the claim is true but which rung it is true on. A weights drop is a true Class III release — and also the weakest form of openness the framework can name, with the gap to a reproducible model being the two classes the announcement did not mention.

## The word for shipping the floor and calling it the building

There is a name for the move, and it is not a flattering one. The same framework calls it openwashing: the practice where "ML models whose weights are made publicly-available for download and downstream use are being falsely promoted as 'open-source.'" The paper states the underlying gap directly — "most models lack the necessary components for full understanding, auditing, and reproducibility, and some model producers use restrictive licenses whilst claiming that their models are 'open source.'" Some releases withhold the components needed to rebuild the model; some additionally attach restrictive licenses while still claiming the label. Either way the word is doing work the artifact does not back.

The FAccT analysis of generative-AI openness ([Rethinking open source generative AI: open-washing and the EU AI Act](https://facctconference.org/static/papers24/facct24-120.pdf)) puts a number on how common this is. Surveying more than 45 generative-AI systems across text and text-to-image, the authors find that "while the term open source is widely used, many models are 'open weight' at best and many providers seek to evade scientific, legal and regulatory scrutiny by withholding information on training and fine-tuning data." "Open weight at best" is the empirical finding, not a rhetorical jab — most of the systems calling themselves open clear the weights bar and stop there. The paper names the strategy as a deliberate one: openwashing is "collect brownie points for openness without disclosing critical information of training and tuning procedures, thereby largely escaping the scientific scrutiny and legal exposure that would come with full openness."

The same paper supplies the methodological diagnosis of why the single word fails. Openness, it argues, "is necessarily composite (consisting of multiple elements) and gradient (coming in degrees)," and it warns against "the risk of relying on single features like access or licensing to declare models open or not." It is blunt about the specific error: "to single out only a single measure and base an openness classification on that ... is in effect what a focus on open weight models or on open licences accomplishes." Pinning the verdict to one dimension — the license, or the downloadability of the weights — is the named mistake, and exactly the dimension the marketing word picks.

This is a question about what the word certifies, and it is a different question from the one [weights provenance](/blog/weights-provenance/) asks. Provenance asks whether the weight file you received is the genuine, untampered artifact — a supply-chain-integrity question about specific bytes. Reproducibility asks whether anyone can rebuild the model at all from what was released. The two are orthogonal: you can have a cryptographically provable, untampered weights file (perfect provenance) that is completely unreproducible (no data, no code, no recipe), and the reverse is equally possible. "Open" as commonly used answers neither cleanly — it answers a third question, what the license permits, and lets readers hear the other two.

## The hidden dimensions are exactly the reproducibility-relevant ones

If the gap between "open weights" and "reproducible" were filled with minor documentation, the conflation would be venial. It is not. The dimensions releases systematically withhold are precisely the inputs you would need to rebuild the model — and we know which, because the work measuring foundation-model transparency has isolated them.

The _2025 Foundation Model Transparency Index_ ([_The 2025 Foundation Model Transparency Index_](https://arxiv.org/abs/2512.10169) (arXiv 2512.10169)) scores developers across 100 indicators and reports where the opacity concentrates: "companies are most opaque about their training data and training compute as well as the post-deployment usage and impact of their flagship models." Training data and training compute are, in the index's framing, the two critical inputs for building a model. They are also the two things a weights download does not expose and cannot be reverse-engineered from — the most-hidden dimensions and the most-reproducibility-critical ones are the same.

The earlier edition makes the same point from the other direction. The _2024 Foundation Model Transparency Index_ ([_The 2024 Foundation Model Transparency Index_](https://arxiv.org/abs/2407.12929) (arXiv 2407.12929)) — which scored 14 developers against the 100 indicators carried over from the 2023 edition — identified "regions of sustained and systemic opacity such as on copyright status, data access, data labor, and downstream impact." Copyright status, data access, data labor: the provenance and composition of the corpus, none of which falls out of a weights file. The same edition notes that "on average, developers disclosed information related to 16.6 indicators that was not previously public" — much of even the disclosed pipeline had been secret by default until someone built a scorecard and asked.

The cross-model survey reaches the same conclusion from a larger sample. The analysis of transparency across SoTA LLMs ([_Comprehensive Analysis of Transparency and Accessibility of ChatGPT, DeepSeek, And other SoTA Large Language Models_](https://arxiv.org/abs/2502.18505) (arXiv 2502.18505)) examines over 100 models through, in its words, "two perspectives: open-source vs. open-weight models" — the exact distinction this essay is built on — and finds the gap holds even in the best cases: "while some models are labeled as open-source, this does not necessarily mean they are fully open-sourced. Even in the best cases, open-source models often do not report model training data, and code as well as key metrics." Openwashing, it concludes, "limits the reproducibility, bias mitigation, and domain adaptation of these models" — the label promises rebuild, audit, adapt, and the withheld components are exactly what you would need to exercise any of them.

## The trend is backwards, which is the part that should worry you

A static gap between label and capability is a definitional problem. A widening gap is a drift, and drift is worse: the word moves away from the thing it is taken to certify while the certification stays in place.

The 2025 index is the anchor here, and the headline reverses the optimistic read of the prior year: "the average score out of 100 fell from 58 in 2024 to 40 in 2025." In the fuller phrasing, "scores in 2025 (average = 40.69) have declined from their 2024 levels (average = 58), reversing the progress observed in 2024 back to the 2023 levels when the Index first launched (average = 37)." Two years of measurement, and transparency did not climb toward the openness the label implies — it fell back to where the index started. The 58 the 2024 edition reported as a 21-point gain was, it turns out, a peak and not a trajectory.

The decline is not a few laggards dragging an average. It is broad and it is deliberate: "most companies scored in the past two years have decreased their score in the past year with Meta cutting its score in half and Mistral by more than two-thirds." Even the willingness to engage when asked directly fell: of the 23 companies contacted for the 2025 edition, "7 agreed to submit transparency reports — a decrease from the 14 participants" the year before — half as many developers willing to even fill out the scorecard.

That is what makes the conflation actively dangerous rather than merely sloppy. If "open" reliably meant "Class III weights drop, nothing more," a careful reader could mentally subtract the missing tiers. But the word does not hold a fixed referent, and the referent it gestures at is shrinking. The marketing word reads the same whether the underlying transparency is 58 or 40 — and nothing in the label tells you which year's level of disclosure you are getting.

## The honest limits of this argument

Three places this case is narrower than the slogan, stated so it does not get used past its evidence.

First, "open weights" is genuinely valuable. A permissively licensed weights file you can fine-tune, inspect, and self-host is a real good — the floor of the ladder, but a floor is still something you can stand on. The fault is not in releasing weights; it is in calling the floor the whole building. A Class III release honestly labeled as Class III is fine. The argument is against the word, not the artifact.

Second, the measurement instruments carry their own subjectivity, and the cleanest critique cuts against my use of them. Which indicators the index uses, and how each is graded, is a set of judgment calls; the framework's 17 components are a chosen decomposition, not a law of nature. A different rubric could produce a different slope — and the 2025 index did revise its 100 indicators, so part of the 58-to-40 drop could be the ruler changing rather than the field. The index says as much, listing "different indicators" among the things that "may contribute to this decline." What keeps the trend from being a rubric artifact is that the index controls for it: holding the company set fixed to the nine scored in both 2024 and 2025, "7 scored lower in 2025 than they did in 2024." That same-developer comparison, plus four independent efforts converging on the same gap, is the load-bearing evidence — not raw cross-edition scores, and a skeptic may still fairly ask how much of the rest is the instrument.

Third — and this cuts against the developers, so I will not hide behind it — some opacity has defensible causes the score does not credit. Training data is increasingly withheld for litigation exposure, not just to dodge scrutiny, and "we cannot publish the corpus our lawyers are arguing about" is a real constraint that still lands in the index as a transparency failure. That explains _why_ disclosure is falling; it does not rescue the word. A weights file withheld for a defensible reason is exactly as unreproducible as one withheld for an indefensible one — the reproducibility verdict does not care about the motive.

## The label now has legal teeth

One more reason precision here is not pedantry: the drift has acquired regulatory consequences. The FAccT paper flags what it calls a potentially alarming exemption in the EU AI Act — the Act "allows models released under open licences to forego detailed disclosure of training data and fine-tuning methods, while outsourcing the definition of what is open to a (yet to be established) EU AI Office." The single dimension the framework warns against anchoring on — the license — is the one a legal regime now uses to grant relief from disclosing the dimensions that matter for reproducibility, with the definition of "open" left to a body that has not yet drawn the line. The label's looseness is no longer just a procurement hazard. It is a load-bearing term in law, pointed at the weakest dimension of the thing it names.

## What to ask instead of "is it open"

The cure is the same shape as the cure for any word that certifies more than it checks: stop asking the binary and name the dimension. "Is this model open" invites the press-post answer; the questions that separate a license claim from a reproducibility claim are specific, and a release either answers them or it does not.

- Is the **training data** released, or described precisely enough to reassemble — sources, mixing ratios, filtering? This is the input the indices report as most often hidden, and the one a weights file cannot reveal.
- Is the **data-processing and training code** released — not just an inference harness, but the pipeline that built the corpus and ran the optimization?
- Is the **recipe** published — hyperparameters, schedule, compute — at the level where an independent run could land in the same place?
- Which **MOF class** does the release actually sit in: Class III (weights and license), Class II (open tooling), or Class I (open science)? Name the rung, not the building.
- Is "open" pinned to the **license alone**, or backed by the composite of dimensions? A single feature — the licence, the download link — carrying the whole openness claim is the exact single-measure error the literature names.

If those answers are present, the model is reproducible and "open" is fair. If they are absent, "open weights" is still true and still worth having — it is just a license claim, to be read as one, not as a promise that anyone could rebuild what was shipped. The same discipline applies downstream: a model derived by [distillation](/blog/distillation-keep-the-eval/) inherits whatever opacity its teacher carried, and a model produced by [decentralized training](/blog/decentralized-training-2026/) is reproducible only to the extent its distributed recipe was published. Openness does not improve as you move down the pipeline; it can only decay.

## Reading list

- [The Model Openness Framework](https://arxiv.org/abs/2403.13784) — establishes that openness is graded across 17 components in three classes, with "open weights" at the bottom tier and naming the gap as openwashing.
- [The 2025 Foundation Model Transparency Index](https://arxiv.org/abs/2512.10169) — the backwards-trend anchor: average transparency fell from 58 in 2024 to 40 in 2025, with training data and compute the most-hidden dimensions.
- [The 2024 Foundation Model Transparency Index](https://arxiv.org/abs/2407.12929) — sets the 58/100 baseline the next year reverses and locates the systemic opacity in data access, copyright, and data labor.
- [Comprehensive Analysis of Transparency and Accessibility of ChatGPT, DeepSeek, And other SoTA Large Language Models](https://arxiv.org/abs/2502.18505) — separates open-source from open-weight across 100+ models and shows even the best cases omit training data and code.
- [Rethinking open source generative AI: open-washing and the EU AI Act](https://facctconference.org/static/papers24/facct24-120.pdf) — finds most "open" systems are open-weight at best, defines openness as composite and gradient, and flags the EU AI Act's open-licence exemption.

"Open weights" is a true and useful statement about a license and a file. It is just not the statement most people hear — that the model could be rebuilt — and in a field where the disclosure that would make that true is shrinking year over year, the only safe move is to read "open" as the license claim it actually is.