What generative software actually is

The question behind the question

Nobody actually asks what generative software is. What they ask is some version of “can I trust this”, and the honest answer requires knowing roughly what is happening between the box you type in and the thing that comes back.

This is the explanation we give new colleagues in their first week. It is not complete and it is not the version a researcher would write, but it is accurate enough to reason with and short enough to finish.

A very good guess, made in pieces

A generative model is a system that has been shown an enormous amount of material and tuned, over and over, to predict what comes next. Next word, next patch of image, next frame. That is the whole trick. The reason the results feel like understanding is that predicting the next thing well turns out to require an astonishing amount of structure.

Two consequences follow immediately, and almost every practical question about this technology is downstream of one of them.

It is a guess. A very well-informed guess, but nothing in the process checks the output against the world. The system that produces a correct answer and the system that produces a confident invention are the same system running the same way.

It is made in pieces. Output is generated incrementally, each piece conditioned on the pieces before it. That is why a small change early in a prompt can reshape everything after it, and why models drift over long outputs.

What a prompt actually does

A prompt is not an instruction in the way a function call is an instruction. It is context — material that shifts which continuations are likely.

This is why prompt advice that sounds like superstition often works. Telling a model it is an experienced editor does not grant it expertise; it makes text that resembles experienced editing more probable than text that resembles a forum reply. The effect is real and the mechanism is mundane.

Prompt:   "Summarise this in two sentences."
Context:  ← everything above shifts what "summary" looks like here
Output:   generated one piece at a time, each conditioned on the last

Where the workspace fits

A model on its own is a raw capability. Everything between it and useful work is software: choosing which family handles a job, holding a house style across a sequence, retrying the chunk that failed, keeping a record of what produced what, and stopping the bill at a number you chose.

That layer is unglamorous and it is where most of the actual value lives. A team with a mediocre model and excellent tooling reliably outperforms a team with the reverse. We have watched it happen often enough to stop being surprised.

So can you trust it

Not in the way you trust a calculator. You can trust it the way you trust a fast, capable, occasionally overconfident colleague: give it work where being wrong is visible and cheap, check the work where being wrong is expensive, and never put it in a position where nobody would notice a confident mistake.

That framing has held up better than any list of capabilities, and it survives each new generation of models without needing to be rewritten.

The one-paragraph version

Generative software predicts what comes next, one piece at a time, based on everything it has seen and everything you gave it. It has no mechanism for knowing whether it is right. Used where verification is cheap, that is an enormous amount of leverage. Used where verification is expensive and skipped, it is a liability. Almost every good decision in this field is a decision about which of those two situations you are in.