What a synthetic panel cannot tell you

The fastest way to make synthetic research untrustworthy is to ask it the questions a field answers. A synthetic panel built on public survey data carries the structure of that data. It does not carry a fresh probability sample, and it cannot create one.

Four questions sit outside what it can answer.

1. Incidence of a behaviour

How many people did a thing last month? Incidence requires a sample drawn from the population, with known inclusion probabilities. A panel of personas is drawn from a distribution, and the distribution is the answer you would be trying to measure. Asking it for incidence returns the assumption of the model, dressed as a measurement.

2. Market share

Market share is incidence with brands attached, plus a share-of-wallet question. It needs a sample of buyers, not a sample of profiles. The personas can tell you which arguments a segment finds credible. They cannot tell you how much of the market a brand holds.

3. Margin of error

A margin of error is a statement about sampling. It has no meaning for a synthetic panel, because there is no sampling frame and no randomness in the draw that a confidence interval would describe. Reporting one is not optimistic; it is a category error.

4. Prevalence of a rare behaviour

Rare behaviours are rare in the source data too. If a behaviour appears in 1.5 percent of the calibration sample, a panel that reflects that sample contains too few cases to say anything segment by segment. The usual workaround is to oversample the rare group, which then breaks the distribution it was supposed to represent.

What it can do instead

The defensible uses are about the instrument and the structure of opinion:

• Item diagnosis. Does this item measure the dimension it claims, and does its wording move the answer?
• Scale behaviour. Does the scale survive a reversal, and do the points separate?
• Segment structure. Which groups differ, and by how much relative to the spread within groups?
• Sensitivity. Which item is the one whose wording changes the conclusion?

Each of those is a statement about the questionnaire and about relative structure, not about level in a population.

Abstention is an output

When the calibration data leaves a segment ambiguous, the persona abstains. An abstention is a usable result: it marks the boundary of what the data supports. A fabricated answer is not, because it enters the report indistinguishable from a measured one.

The rule is simple and worth stating plainly: a synthetic panel is a pretesting instrument. It tells you whether the question can be answered. It does not answer it for the population.