The author suggests that discrepancies in reports with identical metrics often stem from differences in calculation signatures rather than technical errors. A calculation signature includes parameters such as population rules, aggregation formulas, data snapshots, time zones, and access scopes. Without such a standardized description, metrics are effectively redefined in every report.
Common causes of divergence are grouped into layers: semantic differences where the same name refers to different questions, data expansion (fan-out) during JOIN operations, and timing issues. Fan-out occurs when joining a table with one row per object to a table with multiple rows, distorting sums and averages even if distinct counts remain correct.
Time-related discrepancies often arise from the difference between event time (when an action occurred) and processing time (when it was recorded). The author emphasizes the need for a defined cutoff date and time zone to ensure reproducibility, especially in streaming processing using tools like Apache Flink.
The management of historical data also plays a role. Reports may use current classifications to analyze the past or restore the classification active at the time of the event. Additionally, different restatement policies—whether to update closed periods or keep them fixed—can lead to conflicting values between reports.
Finally, identity models affect uniqueness metrics. For example, in mobile products, a single user may be identified by an installation ID, a device ID, or an account. If reports use different identifiers or profile merging rules, the resulting 'unique users' count will differ.
