In this paper, John Elder describes the power of Target Shuffling to evaluate the validity of your discovery. It’s a method that is particularly useful
for identifying false positives, or when two events or variables occurring together are perceived to have a cause-and-effect relationship, as opposed to a coincidental one. In data mining the more variables you have in your predictive model, the easier it becomes to ‘oversearch’ and identify false patterns among them; What Dr. Elder calls the ‘vast search effect‘.