TNT: An Effective Method for Finding Correlations Between Two Continuous Variables

摘要

Determining whether two continuous variables are relevant, either linearly or non-linearly correlated, is a fundamental problem in data science. To test whether two continuous variables have a linear correlation is simple and has a much complete solution, but to judge whether they are in nonlinear correlation is far more difficult. Here, we propose a novel method, Tight Nearest-neighbor prediction correlation Test (TNT), to determine whether two continuous variables are nonlinearly correlated. TNT first use the values of one variable to construct a tight neighborhood structure to predict the value of the other variable and then use the sum of squared errors to measure how well the prediction is. A permutation test based on the sum of squared errors is employed to determine whether two continuous variables are relevant. To evaluate the performance of TNT, we performed extensive simulations comparing with seven existing methods. The results on both simulation and real data demonstrate that TNT is an efficient method to test nonlinear correlations, particularly for some nonlinear correlation which existing methods cannot solve, such as "ring".