As someone who crunches numbers for a living, you can believe me when I say that the growth in computing power provided by technological advances in the past 30 years is something that I appreciate -- really appreciate. It has made so much of my work easier, faster, and far more accurate. (But then I can remember when we used card sorters and programmable, mechanical calculators!)
But with this power we can also see some problems, and of late I am noticing a trend that gives me pause. Specifically, I see occasional requests to basically abuse data files by crunching all possible combinations of various data elements, with the results sorted by the degree of significance found in the relationship. With the many thousands (or even millions) of relationships that can be tested, the opportunity for spurious correlations jumps off the charts.
Most people understand that this opportunity for false positive findings really does not do their research any good, and will try to get a bit more focused when pushed on the validity of doing research in this fashion. But others insist on going down this path. I will decline such business. To me, this seems to be an approach to research similar to how a drunk uses a lamp post -- for support as opposed to illumination.
But with this power we can also see some problems, and of late I am noticing a trend that gives me pause. Specifically, I see occasional requests to basically abuse data files by crunching all possible combinations of various data elements, with the results sorted by the degree of significance found in the relationship. With the many thousands (or even millions) of relationships that can be tested, the opportunity for spurious correlations jumps off the charts.
Most people understand that this opportunity for false positive findings really does not do their research any good, and will try to get a bit more focused when pushed on the validity of doing research in this fashion. But others insist on going down this path. I will decline such business. To me, this seems to be an approach to research similar to how a drunk uses a lamp post -- for support as opposed to illumination.
RSS Feed