The Counterintuitive Goal of Data Fusion

Data Fusion is the art of matching individual records from one dataset (the donor dataset) with records in another dataset (the recipient dataset), allowing us to infer relationships across datasets, even though we are dealing with entirely different records.

Within market research, Data Fusion is commonly used to match individuals from one survey with individuals from another, so that the two surveys can be cross-tabulated. The objective is usually to reduce fieldwork costs, improve data quality and avoid logistic and privacy issues.  

The idea could be clumsily stated as, “Similar people are more likely to say similar things than not”. When matching individuals on a handful of datapoints, we do not expect this to be remotely true, however as we add more commonalities into the mix, the number of combinations grows exponentially, and results become eerily accurate.

On the surface, this sounds like a problem best solved with precise matching alone. That is, the closer the matches are, the better the fusion will be. Right?

Admittedly, it makes intuitive sense: The best outcome is achieved when donors and recipients are perfectly aligned. After all, if we’re merging datasets, shouldn’t we strive for the most precise pairing possible?

Not Quite

Aiming exclusively for perfect matches leads to unintended and hidden consequences. Instead of preserving the dataset’s richness and complexity, an excessive focus on matching precision flattens variability and introduces systematic bias.

The “Average Joe” Trap.

Imagine a dataset where a small group of donors serve as near-perfect matches for the vast majority of recipients. These ‘Average Joes’ are great matches because recipients themselves are average, but because their characteristics are broadly representative of so many, in different ways. While every donor pool mathematically always includes some “Average Joes,” this is in truth part of a much broader spectrum. Because average profiles fit well across multiple recipients, they inevitably become popular donors. We’re not only considering the overall average, but pockets of average cohorts throughout the data.

Essentially, an over-reliance on “Average Joes” to ensure strong matching causes everyone else to disappear from the dataset entirely.

Said in different ways, this causes:

·   The dataset to lose diversity/heterogeneity.

·   Variance to be underestimated.

·   Real-world complexity to be lost

Fusion: More of a balancing act than a match-making game.

In reality, Data Fusion is not about maximising similarity but rather about increasing similarity while preserving the structural associations with the dataset. In this, it makes more sense to think of fusion as a sampling technique.

Recall that, when sampling, we aim for a sample size large enough, and as we sample more, our sample’s mean gradually approaches the true population mean. (This is why researchers fixate on healthy base sizes). Also, as we add more sample, we want to progressively account for more variation, so that the variation in our sample approximates the true variance. This is the second part of the equation which determines our confidence intervals. The part that tells us the difference between the signal and the noise. Though this mechanism is the fundamental concept which allows the field of Statistics (and so Quantitative Research) to exist, it is sometimes overlooked, especially with the recent rise of synthetic data generated by language models.

That is, if I were to deliver a synthetic dataset which mimics the average true answer by way of delivering average Joe’s in every cohort, I can imagine something about my data wouldn’t sit right with you. You might not less confident about basing business decisions on just what the average person said, and you’d be right to do so.

Really? Hold the phone. That’s the goal of Quantitative Research?

While that may appear this is exactly what decision makers are doing, the silent part is that for that average to be meaningful, it must first be representative of something – including the extremes, not just the centre. In truth, the ‘mean score’ is only one statistic which analysts use to describe central tendency of data. While easy to understand, and so the most popular statistic; if considered alone, it is dangerously misleading. Hitting the average answer isn’t our only goal.

A robust fusion therefore must mirror real-world variability to also respect the rules of statistical inference. To achieve robust Data Fusion, the analyst must:

Search out the strongest possible matches (for reasons cited in the first paragraph) while also:

Controlling donor usage – to avoid over-representing average profiles
Preserving variance – so the dataset retains real-world distributions and associations.

This is the art of Data Fusion.

This careful balance is precisely why Data Fusion cannot be fully automated, nor effectively solved by a single universal algorithm.Let’s consider this precision versus representation play another way. Assume our two datasets have 40 datapoints in common, measured by say, 15 variables. It stands to reason that some of these datapoints are more relevant than others, in describing how individuals may be similar, or more strictly, how similar they are in the context of what the merged dataset will be used for.By arbitrarily seeking perfect matches on a single datapoint, the sampling pool of donors is halved. That is just as bad as it sounds. That said, if this datapoint is decisively important, the resulting fusion may indeed improve, despite suffering a reduced donor pool. By forcing more identical matches, the donor pool is too quickly reduced to zero. In doing so, the analyst is missing out on making better matches on the variables which remain.

In other words, the mechanism by which fusion works, is not by matching very well on a few variables, it is by matching reasonably well on several. In this sense, “close enough” or even “nothing alike, but broadly similar”, is better than “perfect”.

Data Fusion is not about precision alone; it’s about producing meaningful, reliable, and representative data that can safely inform decision-making.