Mangen Research Associates, Inc.
Email us!
  • Home
  • Pricing and Forecasting
  • Product Design
  • ROI
  • Cultural Adjustments
  • Blog
  • About
  • Contact

A Trend That Concerns Me

7/22/2014

0 Comments

 
Picture
As someone who crunches numbers for a living, you can believe me when I say that the growth in computing power provided by technological advances in the past 30 years is something that I appreciate -- really appreciate.  It has made so much of my work easier, faster, and far more accurate. (But then I can remember when we used card sorters and programmable, mechanical calculators!)

But with this power we can also see some problems, and of late I am noticing a trend that gives me pause. Specifically, I see occasional requests to basically abuse data files by crunching all possible combinations of various data elements, with the results sorted by the degree of significance found in the relationship.  With the many thousands (or even millions) of relationships that can be tested, the opportunity for spurious correlations jumps off the charts.

Most people understand that this opportunity for false positive findings really does not do their research any good, and will try to get a bit more focused when pushed on the validity of doing research in this fashion.  But others insist on going down this path.  I will decline such business.  To me, this seems to be an approach to research similar to how a drunk uses a lamp post -- for support as opposed to illumination.

0 Comments

    David J. Mangen

    I'll use this space to make some occasional comments about statistics, numbers and research issues as seen in the world today.

    Archives

    July 2021
    October 2020
    July 2020
    April 2020
    December 2019
    September 2019
    February 2018
    January 2018
    December 2017
    January 2017
    December 2016
    November 2016
    March 2016
    January 2016
    October 2015
    July 2015
    March 2015
    January 2015
    November 2014
    September 2014
    July 2014
    June 2014
    May 2014
    January 2014

    Categories

    All
    Big Data

    RSS Feed

Welcome to the Mangen Research Associates, Inc. web site
Copyright 2022 Mangen Research Associates, Inc.