Journal article
Speeding up correlation search for binary data
Pattern recognition letters, Vol.34(13), pp.1499-1507
10/01/2013
DOI: 10.1016/j.patrec.2013.05.027
Abstract
Searching correlated pairs in a collection of items is essential for many problems in commercial, medical, and scientific domains. Recently, a lot of progress has been made to speed up the search for pairs that have a high Pearson correlation (phi-coefficient). However, phi-coefficient is not the only or the best correlation measure. In this paper, we aim at an alternative task: finding correlated pairs of any "good" correlation measure which satisfies the three widely-accepted correlation properties in Section 2.1. In this paper, we identify a 1-dimensional monotone property of the upper bound of any "good" correlation measure, and different 2-dimensional monotone properties for different types of correlation measures. We can either use the 2-dimensional search algorithm to retrieve correlated pairs above a certain threshold, or our new token-ring algorithm to find top-k correlated pairs to prune many pairs without computing their correlations. The experimental results show that our robust algorithm can efficiently search correlated pairs under different situations and is an order of magnitude faster than the brute-force method. (c) 2013 Elsevier B.V. All rights reserved.
Details
- Title: Subtitle
- Speeding up correlation search for binary data
- Creators
- Lian Duan - New Jersey Institute of TechnologyW. Nick Street - University of IowaYanchi Liu - New Jersey Institute of Technology
- Resource Type
- Journal article
- Publication Details
- Pattern recognition letters, Vol.34(13), pp.1499-1507
- Publisher
- Elsevier
- DOI
- 10.1016/j.patrec.2013.05.027
- ISSN
- 0167-8655
- eISSN
- 1872-7344
- Number of pages
- 9
- Language
- English
- Date published
- 10/01/2013
- Academic Unit
- Bus Admin College; Nursing; Computer Science; Business Analytics
- Record Identifier
- 9984380495102771
Metrics
4 Record Views