Sparse decompositions for exploratory pattern analysis

We define and verify the utility of a pattern analysis procedure called sparse decomposition. This technique involves sequentially ``peeling'' sparse subsets of patterns from a pattern set, where sparse subsets are sets of patterns which possess a certain degree of regularity or compactness as measured by a compactness measure c. If this is repeated until all patterns are deleted, then the sequence of decomposition ``layers'' derived by this procedure provides a wealth of information from which inferences about the original pattern set may be made. A statistic P is derived from this information and is shown to be powerful in detecting clustering tendency for data in reasonably compact sampling windows. The test is applied to both synthetic and real data.