← Home

Machines Can’t Create

A convex hull is a set of weighted averages. Nobody has averaged their way to a theorem, and in fifty dimensions nobody, human or machine, stays inside the hull.

Defendants
Ted Chiang (2023, 2024); Noam Chomsky, Ian Roberts and Jeffrey Watumull (2023); with Ada Lovelace (1843) as precedent
Claim
A language model only rephrases, averages or interpolates the human work it was trained on. In the mathematical version, it operates inside the convex hull of its training data and never goes beyond it.
Charge
Bringing a charge, “just a mashup”, that is true of every result by anyone, and dressing it up as geometry when ideas do not combine by averaging and new points almost never lie inside the hull anyway.
Verdict
Guilty. Machines have produced matrix-multiplication algorithms and combinatorial constructions that beat everything in the literature, checked by proof or computation. Chiang is not guilty on what he says about art and choices, which is a different claim.

This rebuke was suggested by Bernd Johannes Wuebben (Global Head, Fixed Income Systematic Investing), whose article “Beyond the Convex Hull: AI and the Nature of Human Creativity” (LinkedIn, 10 October 2026) supplies the geometric objection in Rebuke 2 and several of the examples in Rebuke 1.

The argument, in their words

Wuebben states the claim he is answering:

A criticism of AI that I hear increasingly often is that it cannot really create anything original because everything it produces must ultimately be some combination of ideas already present in its training data. Sometimes this argument is expressed in mathematical language: AI is said to operate within the convex hull of human knowledge, capable of interpolating between existing ideas but unable to venture genuinely beyond them. — Wuebben (2026)

The best-known statement of it is Ted Chiang’s essay in The New Yorker, “ChatGPT Is a Blurry JPEG of the Web” (February 2023). A model is a lossy compression of the Web, and when it fills a gap it does what an image decoder does:

When an image program is displaying a photo and has to reconstruct a pixel that was lost during the compression process, it looks at the nearby pixels and calculates the average. This is what ChatGPT does when it’s prompted to describe, say, losing a sock in the dryer using the style of the Declaration of Independence: it is taking two points in “lexical space” and generating the text that would occupy the location between them. — Chiang (2023)

For a writer, he concludes, “starting with a blurry copy of unoriginal work isn’t a good way to create original work.” The essay closes:

In the meantime, it’s reasonable to ask, What use is there in having something that rephrases the Web? — Chiang (2023)

In “Why A.I. Isn’t Going to Make Art” (August 2024) the averaging is explicit. A program writing a story from a short prompt has to fill in the choices the user did not make:

One is to take an average of the choices that other writers have made, as represented by text found on the Internet; that average is equivalent to the least interesting choices possible, which is why A.I.-generated text is often really bland. — Chiang (2024)

Noam Chomsky, Ian Roberts and Jeffrey Watumull, in The New York Times a month after Chiang’s first essay, called the same systems “a lumbering statistical engine for pattern matching” that works by “extrapolating the most likely conversational response”. Of one of its answers they wrote that it works “by a kind of super-autocomplete” and shows “plagiarism and apathy and obviation”. And they make the limit permanent:

For this reason, the predictions of machine learning systems will always be superficial and dubious. — Chomsky, Roberts and Watumull (2023)
In short, ChatGPT and its brethren are constitutionally unable to balance creativity with constraint. — Chomsky, Roberts and Watumull (2023)

The claim is older than any of them. Ada Lovelace wrote in 1843 that the Analytical Engine “has no pretensions whatever to originate anything”. Her note, and Turing’s reply to it, are on the abduction page.

Rebuke 1: every result is a mashup, so the word distinguishes nothing

In the accuser’s sense, everything is interpolation. Every result takes theory A and applies technique B to it. So the charge “it’s just a mashup” can be brought against any result by anyone, and it will be true.

Try it on the human canon. Donaldson’s work on 4-manifolds is the Yang–Mills equations from physics applied to smooth topology. Atiyah and Bott’s paper on the Yang–Mills equations over Riemann surfaces is gauge theory plus Morse theory. Wiles’s proof of Fermat’s Last Theorem is modularity plus Galois representations. Scholze’s perfectoid spaces carry problems in mixed characteristic over to characteristic p, where older tools work. Each description is accurate. Each is useless, because it is equally accurate of a dull paper that combines the same two things.

An accusation that can always be made tells you nothing about the case in front of you. The stochastic-parrot page makes the same point about “remix” (its Rebuke 6 is about descriptions that fit every outcome). The test we apply to people is different: is the result new, is it correct, and is it useful? Rebuke 4 applies that test to machines.

Their response

Chiang anticipated a version of this in 2024:

Some individuals have defended large language models by saying that most of what human beings say or write isn’t particularly original. That is true, but it’s also irrelevant. — Chiang (2024)

His reason is that a sincere apology or a work of art has value because of who says it and what life it comes from, not because it is novel. That is a fair point about meaning, and I return to it under Scope. It answers a different defence. The point here is not that humans are unoriginal. It is that the best human work is also a combination, so being a combination cannot be what separates creating from copying.

Rebuke 2: ideas do not combine by averaging

The mathematical version tries to make the mashup charge precise. It fails in a different way. The convex hull of a set of points is the set of their weighted averages, with weights that are nonnegative and sum to one. Wuebben asks what that would mean for ideas:

But why should the space of ideas have such a geometry? What would it even mean to take a weighted average of algebraic geometry and quantum field theory, or to combine thirty percent of topology with seventy percent of number theory? — Wuebben (2026)

“More importantly,” he goes on, “combining mathematical structures is not the same as averaging them.” Composing two functions does not give a point between them. Neither does applying a theorem from one field to an object from another, or taking the dual of a construction. Each gives an object of a different kind, which can be combined again.

Chiang’s own example shows this. A sock lost in the dryer, written in the style of the Declaration of Independence, is not halfway between a laundry note and the Declaration. It takes the content of one and the form of the other. That is a composition. A text literally halfway between them would be neither about socks nor in Jefferson’s style.

Averages also have a property the results in Rebuke 4 lack. An average cannot be more extreme than everything it averages. A cap set larger than any known cap set is not a weighted average of smaller ones.

Rebuke 3: even where the hull exists, nothing new is inside it

Suppose the data really are points in space, as images and embeddings are. Then the convex hull is well defined, and one can ask how often a new point falls inside it. Randall Balestriero, Jérôme Pesenti and Yann LeCun asked in 2021:

We empirically and theoretically argue against those two points and demonstrate that on any high-dimensional (>100) dataset, interpolation almost surely never happens. — Balestriero, Pesenti and LeCun (2021), abstract

Here “interpolation” is the hull definition: a sample interpolates if it lies inside or on the boundary of the training set’s convex hull, and extrapolates otherwise. Their conclusion is that the definition is useless as a measure of how a model generalises, because by it every model extrapolates nearly all the time.

This is easy to check. Draw 1,000 training points from a standard Gaussian in d dimensions. Then draw fresh points from the same Gaussian, so that none of them is unusual, and test each one for membership of the hull. Membership is a linear program: find weights w ≥ 0 summing to one with XTw = x. The script does this with scipy.optimize.linprog for 200 fresh points per dimension, and runs in under a minute:

Dimension dFresh points inside the hull of 1,000 training pointsFraction
2199 of 2000.995
5165 of 2000.825
1053 of 2000.265
200 of 2000.000
500 of 2000.000

By twenty dimensions, a point drawn from the very distribution the training set came from is outside its hull every time. Language-model embeddings have thousands of dimensions. Adding data helps slowly: the number of points needed to keep new samples inside grows exponentially with the dimension of the data, which is the paper’s theoretical result.

So the dressed-up claim is false as stated, for machines and for people. Every sentence you have ever written, encoded as a vector, almost surely lies outside the hull of the sentences you had read. That does not make it original. It shows that “inside the hull” was never the line between copying and creating.

Rebuke 4: the record

The fair test is the one we apply to people. Is the result new, is it checkable, and does it beat what the field had?

Matrix multiplication, 2022. Multiplying two 4×4 matrices by applying Strassen’s 1969 method twice takes 49 multiplications. DeepMind’s AlphaTensor, a reinforcement-learning system, found an algorithm that uses 47 in arithmetic modulo 2:

Particularly relevant is the case of 4 × 4 matrices in a finite field, where AlphaTensor’s algorithm improves on Strassen’s two-level algorithm for the first time, to our knowledge, since its discovery 50 years ago. — Fawzi et al., Nature (2022)

Cap sets, 2023. A cap set is a set of points in (ℤ/3)n with no three on a line. FunSearch paired a pretrained language model with a program that scores its output, and found a cap set of size 512 in dimension 8, larger than any previously known. It also raised the lower bound on the cap set capacity from 2.2180 to 2.2202, which the authors call “the largest improvement in 20 years to the asymptotic lower bound”. The output is a program, which mathematicians could read:

This demonstrates that it is possible to make a scientific discovery—a new piece of verifiable knowledge about a notorious scientific problem—using an LLM. — Romera-Paredes et al., Nature (2023)

Matrix multiplication again, 2025. AlphaEvolve, a coding agent built on language models, found a way to multiply two 4×4 complex-valued matrices with 48 multiplications:

For 56 years, designing an algorithm with rank less than 49 over any field with characteristic 0 was an open problem. AlphaEvolve is the first method to find a rank-48 algorithm to multiply two 4 × 4 complex-valued matrices. — Novikov et al. (2025), §3.1

The paper adds a footnote that matters: there are algorithms using fewer than 49 multiplications that are not tensor decompositions and cannot be applied recursively, so the record is for the kind of algorithm Strassen’s is. The same day, Dumas, Pernet and Sedoglavic posted a version with rational coefficients, which they obtained by projecting the machine’s complex decomposition. Human mathematicians recombined a machine’s result into a new one. Nobody suggested theirs was not original either.

Go, 2016. AlphaGo’s move 37 against Lee Sedol, which Fan Hui called “not a human move”, is on the abduction page.

Each of these is new by the test the defendants would apply to a person, and each was checked by proof, by computation or, for the Go move, by the game, rather than by taste. None of them is a point between two points in lexical space.

A claimed result, October 2026. Wuebben’s article opens with a more recent case. On 6 October 2026 OpenAI published a repository of several hundred mathematical manuscripts produced by an internal model. Problem 180 claims a proof of Barnette’s conjecture, that every 3-connected cubic bipartite planar graph has a Hamiltonian cycle. Jake Boggan, who had worked on the conjecture on and off for 24 years, wrote on Hacker News that night: “But it’s supposedly proven here - problem 180. I don’t know what to think exactly.” Later the same day he described the argument as resting on “some clever partition function sums which are intellectually tractable but simply too far out of anything I’d seen before”, though he had also written that on the surface it looked like an approach he had considered and abandoned.

This one is a claim. The paper has not been refereed. The repository lists a Lean formalisation for it, and I found no sorry in the Lean files for this problem, but I have not compiled them, and Boggan wrote on the night that there was no Lean proof. The repository’s own history records three other manuscripts withdrawn the next day for a sign error. Nothing on this page depends on Problem 180.

Rebuke 5: Chiang set a test, and it has been passed

To his credit, Chiang said what would change his mind:

If and when we start seeing models producing output that’s as good as their input, then the analogy of lossy compression will no longer be applicable. — Chiang (2023)

In the systems above, it already is. FunSearch works by feeding the model’s best programs back into its prompts, which its authors call “best-shot prompting”. The model’s output becomes its input, and the loop ends with a cap set larger than anything in the literature it was trained on. A copy cannot be sharper than the original. This one is.

Chiang’s related suggestion was that a company’s willingness to train new models on a model’s text would be a sign of its quality, and he suspected that would need “a major breakthrough”. Four months later Microsoft trained phi-1 partly on “synthetically generated textbooks and exercises with GPT-3.5”. That is weaker evidence, since a small model learning from a large one is closer to distillation than to surpassing the source. The cap sets and matrix algorithms are the clean case.

Chomsky and his co-authors said these systems are “constitutionally unable to balance creativity with constraint”. FunSearch is that balance built as a machine: a language model proposes, an evaluator rejects what fails, and the paper introduces the design as an answer to the model’s tendency to confabulate. Their prediction that the output would “always be superficial and dubious” does not survive a 512-point cap set.

Scope

The claim on trial is “can’t”. Most of what language models produce is derivative, much of it is bland, and Chiang is right about that. So is most of what people produce. The rebuke needs only some machine results that are new by a fair test, and they exist.

The machines did not work alone. FunSearch evolved one function inside a program skeleton that humans wrote, and AlphaEvolve’s authors describe the division of labour as “Human defines ‘What?’” and “AlphaEvolve figures out ‘How?’”. Choosing the problem and the scoring rule is real work. It is also what a supervisor does for a student, and we still credit the student.

Wuebben draws his own lines, and they are the right ones: “Nor should we confuse an impressive-looking argument with a correct proof, or a correct proof with a conceptual breakthrough.” A record number of multiplications is new and correct, and is not a new field of mathematics. Nothing here shows a machine doing what Grothendieck did. It shows that the hull argument cannot be the reason it hasn’t.

Chiang’s later argument, that art is made of many choices and gets its meaning from the life of the person who makes it, is not refuted here. It is a claim about what makes a work worth attending to, and it is compatible with machines producing things that are new.

In their favour

Chiang’s compression analogy explains hallucination well: a decoder that fills gaps plausibly will produce confident errors. That part has aged better than the rest.

There is real evidence that feeding models their own output can go badly. Shumailov et al. (2023) found that training on generated data makes the tails of the original distribution disappear, which they call model collapse. The successful loops above all have an external check, and that supports the defendants’ instinct that the model alone is not enough.

Training-data attribution and copyright are real concerns and separate ones. Chiang quotes Simon Willison calling model training “money laundering for copyrighted data”. Whether a result is new and whether its makers paid for their inputs are different questions, and this page answers only the first.

Measuring creativity is hard, and most benchmarks for it are weak. The examples here were chosen because they have an objective score: a count of multiplications, the size of a set. Open-ended creative writing has no such score, and claims in either direction there are harder to settle.

Disclosure: this page was researched and drafted by a language model, which on the defendants’ account could only have averaged the sources. Check the quotations.

References