If by some chance you are interested in internship opportunities with a statistics focus, consider taking a look at this set of opportunities published by the American Statistical Association in Amstat News.  There are a lot of good opportunities here.stattrak.amstat.org/2016/12/01/2017internships/

I have been absent from posting here for quite a while.  It is nice to be busy, but that sometimes means that I neglect other things that I really should do.

Today I ran across some material developed by Pierre-Antoine Kremp who is doing work on forecasting the outcome of the 2016 election.  It is interesting analytical work based on Bayesian probability models, and is available on Slate.  See http://www.slate.com/features/pkremp_forecast/report.html for the details.

​What I wanted to show you was this chart where he plots the predicted probabilities, by state, for the candidates.  I may use something like this to present the results of propensity modeling.

This graph really does illustrate how few states really are "in play" for this election.

The following has been directly copied from an email released today by the American Statistical Association.  It represents a position that I heartily endorse.

Today, the American Statistical Association Board of Directors issued a statement on p-values and statistical significance. We intend the statement, developed over many months in consultation with a large panel of experts, to draw renewed and vigorous attention to changing research practices that have contributed to a reproducibility crisis in science.
"Widespread use of 'statistical significance' (generally interpreted as 'p < 0.05') as a license for making a claim of a scientific finding (or implied truth) leads to considerable distortion of the scientific process," says the ASA statement (in part). By putting the authority of the world's largest community of statisticians behind such a statement, we seek to begin a broad-based discussion of how to more effectively and appropriately use statistical methods as part of the scientific reasoning process.
In short, we envision a new era, in which the broad scientific community recognizes what statisticians have been advocating for many years. In this "post p < .05 era," the full power of statistical argumentation in all its nuance will be brought to bear to advance science, rather than making decisions simply by reducing complex models and methods to a single number and its relationship to an arbitrary threshold. This new era would be marked by radical change to how editorial decisions are made regarding what is publishable, removing the temptation to inappropriately hunt for statistical significance as a justification for publication. In such an era, every aspect of the investigative process would have its appropriate weight in the ultimate decision about the value of a research contribution.
Is such an era beyond reach? We think not, but we need your help in making sure this opportunity is not lost.
The statement is available freely online to all at The American Statistician Latest Articles website. You'll find an introduction that describes the reasons for developing the statement and the process by which it was developed. You'll also find a rich set of discussion papers commenting on various aspects of the statement and related matters.

Just a brief note today -- coupled with this lovely sunset -- to wish you and yours a prosperous 2016.

Research released from the National Institute on Retirement Security provides some stark data on the extent to which Americans are relying on Social Security for their economic well-being in  retirement.  Using data from the Federal Reserve's Survey of Consumer Finance, they estimate that 38 million households do not have any assets in retirement accounts.  The full study is available here.

Now some of these differences may be attributed to definitions -- note, for example, that the report focuses on assets in retirement accounts.  If you aren't using a tax-deferred IRA or 401(k) or 403(b) plan your assets -- which could be considerable -- wouldn't count as retirement account assets.  Nonetheless, this result does paint a rather dismal picture.

As the chart above -- taken from the report -- illustrates, even those closest to retirement often have little set aside in retirement savings.

Yes, it is very easy to lie with statistics, but it is perhaps even easier to lie with graphs.  We recently saw a situation where an unscrupulous politician, intent on pandering to one of his interest groups, briefly displayed the following graph on the screen during a committee hearing on the funding of Planned Parenthood.

The display on the screen was brief, and thus sought to communicate that abortions out-number cancer screening and prevention services.  But wait -- look at the actual numbers in the graph (which to their credit they did include):  When did 327,000 become greater than 935,573?  Or, 935,573 approximately equal to 289,750?  Or 2 million approximately equal to 327,000?

This gets my vote as one of the most distorted graphs of the year, and the Bubba who used it should be tossed out of office for either his fundamental ignorance or his crass willingness to distort the data while pursuing his political agenda.

Tomorrow morning have a piece of pie -- preferably at about 9:26 local time.

The other day I was doing some work in the early evening and I received a call from a group purporting to be an independent political polling firm, and they asked if I minded participating in their survey of the MN electorate.  I was somewhat surprised because we just finished the silly season of political gamesmanship a couple of months ago, but OK, I agreed to participate in the poll.

Question 1: Are you a registered voter?  OK, yes I am.
Question 2: Do you consider yourself a Republican, Democrat, or Independent.  Independent.

I'd like to ask your opinion about some issues:

Question 3: An agree-disagree flash-point ideological test issue for one of the political parties.  In the interests of protecting the guilty I won't identify which party.
Question 4: Second agree-disagree flash-point ideological  test issue for the same political party.
Question 5: My last question is...  and we get the 3rd ideological test for the same party, 


Now I answered all of the questions with my opinions, and told the interviewer that their poll had zero validity simply because (a) questions 3-5 were leading and (b) a blind pig could figure out the political affiliation of the person or organization that was sponsoring the poll.  I suspect that a certain state senator was the sponsor. 


Which gets me to the point of this blog post.  The data from that little study will have no validity whatsoever as a gauge of measuring the interests of the citizenry.  It might be able to get a decent percentage indicator of the "party faithful" in a given geographic area.  But in terms of helping to understand the issues that are important to the public at this point in time, it is worthless.  And no amount of analytics will help overcome the fact that the data are fundamentally garbage.

The election in the USA is finally over, and the obnoxious political ads have stopped.  The election was, on the whole, a fairly clear and dramatic victory for the Republican Party in the states.  But is this election a mandate as some are claiming?

Regardless of the party that purports to have received one, the use of the word "mandate" in most political contexts is quite annoying to me.  Why?  Because rarely do the numbers to support that assertion.  Let's look at and personalize the numbers for a fairly common electoral margin that would often produce the assertion of a mandate-- a 55% to 45% victory for one or the other of the parties. A 10 point victory is pretty dramatic, right?

If we look at this in a more personalized context however, that 10 point margin of victory becomes a bit more shallow.  With the holidays coming up, many of us will be having celebrations in our homes where we'll have 20 or so family members coming to visit.  If we apply that 55% to 45% margin to the group of 20 family members, that translates into a split of 11 to 9.  If one person changes their mind, the "mandate" has become a dead heat.  Sorry, but that hardly reflects a mandate.

What would I call a mandate?  If you're getting into the range of 2-1 -- 67% to 33% -- then we can start to talk about a mandate.  But please, do not use the Electoral College to claim your mandate.