On social science about complex stuff

Don't divide your no from your yes.

People are complicated. Society is complicated. Relationships that hold in one context don’t always hold elsewhere. “Absence makes the heart grow fonder”, but then again “out of sight, out of mind”. (Jon Elster gave that example.)

Picking a random article from the American Economic Review:

Does Information Break the Political Resource Curse? Experimental Evidence from Mozambique

Natural resources can have a negative impact on the economy through corruption and civil conflict. This paper tests whether information can counteract this political resource curse. We implement a large-scale field experiment following the dissemination of information about a substantial natural gas discovery in Mozambique…. information targeting citizens… increases local mobilization and decreases violence. By contrast, when information reaches only local leaders, it increases elite capture and rent-seeking.

The title asks a general question: “does information break the resource curse”? The abstract rows back a little, asking “whether information can counteract” the curse. It may offer an existence proof: such and such a process can happen, because it did happen this one time. (Fair enough.) The last sentences are ambiguous. They are statements about what happened in their experiment in Mozambique. But they could also be scientific laws.

The tension between specificity and generality is endemic to any discipline that aims to generalize. Smart researchers understand that. I am sure the article’s conclusions are appropriately cautious.

An experimental design has a problem when it aspires to generality, but tells you more about how the experiment maps to the social space than about true parameter values across the social space. In that case, it may not be persuasive. If the experiment shows X, you think “yup, that can sure happen sometimes”. If it finds no evidence for X, you think “well, not this time”. This is not a case of indestructible priors. It reflects our large store of knowledge about human possibilities, born from common sense, experience or good novels.

Experiments in an artificial environment (i.e. a laboratory) are particularly vulnerable: they deliberately strip away context, which makes sense when you are searching for a context-free rule. Fields that study complex behaviour – social psychology, experimental economics and political economy spring to mind – are also at risk. I hunted for some examples. These are just the first picks in fields I know a bit about. There are probably many worse examples, and the papers themselves may be excellent.

Fairness considerations in joint venture formation

[t]his paper shows that fairness concerns result in failures to undertake profitable joint production opportunities. … we demonstrate that while subjects do not engage in unprofitable joint ventures, they frequently fail to engage in profitable ones. This failure is more likely when equal sharing of firm profits is not individually rational for one of the two participants.

Sounds plausible sometimes.

Friend or foe? Social ties in bribery and corruption

All in all, people in power may make biased decisions in response to bribes or as a consequence of favoritism. … Our results show little effect of social ties on the decision to bribe, nor on the amount bribed. More than two-thirds of the performers send a bribe, but this varies little with the social ties between performers and evaluator.

Hmm, but I read an investigative journalist who said the biggest cause of corruption wasn’t money, it was friendship. Was he wrong?

The strength of weak leaders: an experiment on social influence and social learning in teams

[W]e address the question of how the selection of the team leader affects the performance of social learning in the team. Is it necessary that the central person is the one with the highest expertise? …Answering these questions can be informative for the design of successful organizations…. We show that selection of leaders by accuracy or confidence does not outperform random selection.

Is that a rule? (How would you choose a chief engineer to build a bridge?)

‘Everybody’s doing it’: on the persistence of bad social norms

One view is that bad norms thrive even when people are informed of the preferences of others, since the bad norm is an equilibrium of a coordination game. The other view is based on pluralistic ignorance, in which uncertainty about others’ preferences is crucial. In an experiment, we find clear support for the pluralistic ignorance perspective.

Are these “alternative views”? Can only one be right?

Third-party manipulation of conflict: an experiment

The present paper uses a controlled experiment to study whether and how conflict between two rival players can be manipulated by an interested third party through public communication. Can a third-party provocateur increase the likelihood of conflict by making a strategic provocation? Can a third-party peacemaker reduce its likelihood by making public calls for peace?

I bet they can sometimes!

You get the idea.

This point is a kind of external validity concern. External validity fails when in the experiment X causes Y, but not in the real world. The problem here happens when we reasonably expect that in the real world, X sometimes causes Y, and sometimes doesn’t. In that case, the experiment tells you that it is mapping to that part of the world where X causes Y. So we have learned something about the experimental environment. But we have not yet learned anything about the world.