But with this power we can also see some problems, and of late I am noticing a trend that gives me pause. Specifically, I see occasional requests to basically abuse data files by crunching all possible combinations of various data elements, with the results sorted by the degree of significance found in the relationship. With the many thousands (or even millions) of relationships that can be tested, the opportunity for spurious correlations jumps off the charts.
Most people understand that this opportunity for false positive findings really does not do their research any good, and will try to get a bit more focused when pushed on the validity of doing research in this fashion. But others insist on going down this path. I will decline such business. To me, this seems to be an approach to research similar to how a drunk uses a lamp post -- for support as opposed to illumination.