Bankers not as dishonest as purported

January 5 shoter-term update: see here for a partial list of citations of this article.

Bankers are a unique group of liars, somehow a problem running the financial system?  Cohn, Fehr, and Maréchal, in “Business culture and dishonesty in the banking industry” (doi: 10.1038/nature13977, 516, 86-89 December of 2014) purport to show a few industry “scandals” came from corrupt cultures, which they suggest give the industry a tarnished reputation.  A temporary Nature advisor brought this article to my attention given my position and independent background.  Bankers have worked alongside their communities, to improve residents’ lives, before and after the recent global financial crisis.  A small number of unique actors in the industry, doesn’t prove the entire industry is worthless.

People in all industries sometimes cut-ahead.  Recent scandals have occurred in government, autos, technology, and academic research.  Some employees are not important, for their dishonesty to rise to scandal-level.  Highlighting bankers, now, is short-sighted.

At a top-level, there’s a typical age difference between the studied “volunteer” bankers at 11.5 years of experience, and non-bankers (e.g., doctors) at 14.8 years.  Underlying data, for Figures and Extended “Data”, were not given when requested, making it difficult to clearly understand the experiment’s results.  Nature agreed with me on this and consequently told me they will push on the article's authors to release their data.

See this self-constructed illustration:

The article focuses you only on bankers’ results, rising from 52%, to 58%.  In Figure 3, they then pivot to a contrast with the difference within non-bankers by only showing -4% and -2%.  They prevent the reader from noticing non-bankers were already cheating at high levels, in the baseline and subject to a slight downward bias from an asymmetric posterior (Ch.6: Statistics Topics).  And it shows no liars’ selection bias of choosing finance.  Identified bankers reported flip results of 58%, though non-bankers start the experiment >58%, in the control: 60% for non-bankers; 58% for students!  If one looks at a composite per group (control+identity) we see a lower cheating level among bankers, with flips of: 55% for bankers, 58% for non-bankers, 57% for students.  Or ~3% improvement of bankers versus everyone else.  The “significance” standard varies throughout the article, but a confidence interval shown at +4.5% pooled for bankers, and about +3% for a composite of non-bankers and students.

It inconveniently works against the main thrust of the paper that professional/banking identity, when salient to non-bankers, has cheating levels improved!

We see the banker-control group performed statistically close to chance, versus the treatment group.  But the article doesn’t explore the idea further, as to why the banker-control group was more related to chance, versus anyone else.  Put another way, instead of treating this as given, why was it not explored: what it is about the statistical significance in other groups that creates the dichotomous mathematical result?

After conversations with the authors and Nature, there remains a criticism that significance of reported coin flips is driven partly based upon some cheating outliers (they erroneously ignore this for any non-parametric statistics).  Those bent on getting the maximum payoff allowed ($200 for professionals; a lower $50 for students.)  The authors still don’t communicate the distribution of banker-identified results of <$100.  We can see it is definitely at near 50% for the sample, but they impossibly state to me it is >54%.  Again without the data, it becomes unstylishly opaque for readers.

Now there’s several ways to consider outlier treatment. A spectrum ranging from removing them from the banker-identified group, to adding them to the banker-control group while jointly removing data from the distribution center where random chance would result at ~50%. The statistical significance that was used isn’t noted in the article, but the levels noted were about 2%-3% on a two-tail (and since the study did not have paired results the significance of using a rank-sum technique is much stronger than otherwise can be). So depending on outlier treatment (see Supplemental results below), the difference between the banker-identified and banker-control have a greatly reduced statistical significance.

Leaving math techniques aside, it’s unconvincing that bankers are a unique group of liars.  Other groups (e.g., doctors) don’t not lie, and while they have a Hippocratic Oath, financial boards have ethical standards as well.  As we consider how this study applies to real-world matters, note bankers are already under significant oversight, from Treasury and Federal Reserve.  Their careers are not private games, but rather comply with regulations brought down from the bank CEOs. 

Nature looked at this paramount industry issue, considering interesting angles.  But the researchers missed the opportunity to give a robust treatment, and so crammed onto populist anger between occupations.

Supplemental results

control	banker-identity	control w. outlier	notes
40	40	40
60	40	60
60	60	60
60	60	60
60	60	60
60	60	60
60	80	60
60	80	60
60	80	80
80	80	80
80	80	80
80	80	80
80	80	80
80	80	80
80	80	80
80	100	80
80	100	80
80	100	80
80	100	80
80	100	80
80	100	80
80	100	80
80	100	100
80	100	100
100	100	100
100	100	100
100	100	100
100	100	100
100	100	100
100	100	100
100	120	100
100	120	100
100	120	100
100	120	100
100	120	100
100	120	100
100	120	120
100	120	120
100	120	120
100	120	120
120	120	120
120	120	120
120	120	120
120	120	120
120	120	120
120	120	120
120	140	120
120	140	120
120	140	120
120	140	140
120	140	140
120	140	140
120	140	140
140	160	140
140	160	160
140	160	160
140	160	180
140	160	200
140	160	200
160	200	200
160	200	200
180	200	200
200	200	200
200	200	200
-5.2 (~0, ~0)			original article statistic (p-value 1-tail, p-value 2-tail)
	-2.2 (>.01, >.03)		outlier treatment statistic (p-value 1-tail, p-value 2-tail)