Metraipolis › archive › dc_86b2a28f508e
Sparse departures from independence in two-way tables: a heteroscedasticity profile and detection boundary, an adaptive higher-criticism gate, and an assumption-lean… — arXiv:2608.07979
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.07979
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T09:03:21.442Z
- SHA-256 of the archived text
86d28da516749732f6d3180f78b3ea21c41e90f97e8dc3d70c1665f6631da8a9- Stable URL of this page
https://metraipolis.com/documento/dc_86b2a28f508e
How to cite
Sparse departures from independence in two-way tables: a heteroscedasticity profile and detection boundary, an adaptive higher-criticism gate, and an assumption-lean… — arXiv:2608.07979 — arXiv (en), https://arxiv.org/abs/2608.07979 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_86b2a28f508e (Metraipolis archive dc_86b2a28f508e, sha256 86d28da516749732f6d3180f78b3ea21c41e90f97e8dc3d70c1665f6631da8a9)
BibTeX
@misc{dc86b2a28f508e,
title = {Sparse departures from independence in two-way tables: a heteroscedasticity profile and detection boundary, an adaptive higher-criticism gate, and an assumption-lean… — arXiv:2608.07979},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.07979},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_86b2a28f508e; sha256 86d28da516749732f6d3180f78b3ea21c41e90f97e8dc3d70c1665f6631da8a9},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.07979. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Two-way contingency tables are tested for independence throughout applied statistics (genomics, network and text co-occurrence, pharmacovigilance, ecology, survey cross-tabulation), and the routine test reads asymptotic Gaussian tail probabilities off the table cell by cell. On the tables people actually analyze this is badly miscalibrated: across 5,543 real public two-way tables, two-thirds have counts small enough or margins heterogeneous enough that the asymptotic maximum-cell independence scan false-positives at a mean 33% against a 0.05 target, while an exact margin-conditional anchor holds at about 1%. The failure is sharpest when the departure is sparse, the association concentrated in a few cells, where the omnibus chi-square is inefficient and the sharp object is the detection boundary. In three parts we answer where a sparse signal can be seen, which combiner attains that limit, and how to calibrate at small counts. Part I specializes the sparse-detection theory of Ingster (1997), Donoho and Jin (2004), and Chhor, Mukherjee and Sen (2024) to independence: the table is a heteroscedastic Gaussian sequence whose variance profile is the expected-count table, fixed by the margins via CVe^2 = (1 + CVr^2)(1 + CVc^2) - 1, giving an exact signal map and a closed-form separation radius valid under a moderate-deviation growing-count condition. Part II shows higher criticism, with its closed-form Jaeschke-Eicker null, attains that boundary adaptively over unknown sparsity. Part III measures the small-count calibration failure (size 0.3 to 0.6 even under uniform margins, so the driver is the per-cell tail), removes it with exact margin-conditional laws (size 0.003 to 0.009), and gives a routing rule sending large-count tables to the asymptotic gate and small-count tables to the exact anchor. Every claim is reproduced from openly deposited, deterministically seeded code. Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-11 in stat.ME). Categorie arXiv: Methodology (stat.ME) ; Statistics Theory (math.ST). Pagina del lavoro: https://arxiv.org/abs/2608.07979 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T09:02:02.351Z dalla pagina https://arxiv.org/catchup/stat.ME/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).