Efficient Discovery of Confounders in Large Data Sets
Professor Hui Xiong
Management Science and Information Systems Department
Rutgers University
Abstract :
Given a large transaction database, association analysis is concerned with efficiently finding strongly related objects. Unlike traditional associate analysis, where relationships among variables are searched at a global level, we examine confounding factors at a local level. Indeed, many real-world phenomena are localized to specific regions and times. These relationships may not be visible when the entire data set is analyzed. Specially, confounding effects that change the direction of correlation is the most significant.
Along this line, we propose to efficiently find confounding effects attributable to local associations. Specifically, we derive an upper bound by a necessary condition of confounders, which can help us prune the search space and efficiently identify confounders. Experimental results show that the proposed CONFOUND algorithm can effectively identify confounders and the computational performance is an order of magnitude faster than benchmark methods.
Bio:
Dr. Hui Xiong received his Ph.D. from the University of Minnesota and the B.E degree from the University of Science and Technology of China (USTC). He is currently an Associate Professor at Rutgers University, where he received a two-year early promotion/tenure (2009), the Rutgers University Board of Trustees Research Fellowship for Scholarly Excellence (2009), an IBM ESA Innovation Award (2008), the Junior Faculty Teaching Excellence Award (2007) and the Junior Faculty Research Award (2008) at the Rutgers Business School. His general area of research is data and knowledge engineering, with a focus on developing effective and efficient data analysis techniques for emerging data intensive business applications. He is an Associate Editor of the Knowledge and Information Systems journal. He has served regularly in the organization committees and the program committees of a number of international conferences and workshops. More detailed information is available at
http://datamining.rutgers.edu.