ABA ROLI Presentation, June 8, 2016

This notebook was prepared for a presentation at the ABA's Rule of Law Institute, and contains all the code necessary to re-run the data analysis presented therein.

Replication information:
This document contains working Python code and output, mixed with text. For convenience, the two csv files containing the relevant data may be downloaded here and here. The slides underlying this presentation are available here. All libraries and tools are packaged into the excellent free Anaconda scientific Python distribution, which I highly recommend. I used Python 2, but it should work on the more modern Python 3 too (untested).

If you're done here, you can go back to The Rule of Law in the Real World.

In [20]:
%matplotlib inline
In [21]:
import numpy as np
import pylab as plt
import pandas as pd
import sklearn 
rlrw = pd.read_csv("rlrw_scaled.csv")
wjp_bti = pd.read_csv("wjp_bti_scaled.csv")

rlrw is a collection of centered and scaled data, including the scores from The Rule of Law in the Real World, same-year scores from the World Bank and Freedom House, World Justice Project subfactor scores (wjp1...8 are the eight main factors from WJP data for 2012), the first principal component of the World Justice Project 2012 scores, and measurements of GDP per capita (World Bank), individual liberty (from Freedom House), property rights protection (Heritage Foundation), and democracy (Freedom House).

In [22]:
rlrw.head()
Out[22]:
Unnamed: 0 State rol_score wjppc gdppc elec_pros pol_plur per_auto hprop fh_rol wb_rol log_gdp
0 0 Albania -0.583004 -0.661588 -0.578904 -0.203537 -0.340063 -0.493828 -0.690925 -0.025308 -0.619088 -0.864895
1 1 Argentina 0.007954 -0.470926 -0.184115 0.625800 0.850157 0.704527 -1.282219 0.435291 -0.762398 -0.203482
2 2 Australia 1.358174 1.726488 2.704461 0.902245 0.850157 1.303705 1.674249 1.126190 1.673396 1.309538
3 3 Austria 1.349949 1.542930 1.649975 0.902245 0.850157 1.303705 1.674249 1.356489 1.768312 0.974550
4 4 Bangladesh -1.280892 -1.227155 -0.744639 0.072909 -0.102019 -0.493828 -1.085121 -0.485907 -0.954713 -1.365078
In [23]:
wjp_bti.head()
Out[23]:
Unnamed: 0 bti wjp_overall wjp1 wjp2 wjp3 wjp4 wjp5 wjp6 wjp7 wjp8 pca2015 diffs Country
0 0 -1.518167 -1.670134 -0.642546 -1.500013 -0.850605 -1.275612 -2.242860 -1.221828 -1.777739 -1.442546 -1.617419 0.099252 Afghanistan
1 1 -0.263356 0.015918 0.228852 -0.628317 0.162616 0.436090 0.499963 -0.360404 -0.052078 0.006591 -0.025672 0.237683 Albania
2 2 0.138184 0.015918 -0.246456 0.176326 0.612936 0.902918 -0.710106 -0.551831 0.427273 -0.298490 0.030387 0.107798 Argentina
3 3 -0.614703 -0.975877 -0.642546 -1.231799 -0.400285 -0.964393 -0.387421 -1.126114 -1.106648 -0.756112 -1.030266 0.415563 Bangladesh
4 4 -1.267205 0.115097 -1.355508 0.310433 -0.512865 -0.653175 0.903319 0.596734 1.098363 0.387943 0.105652 1.372857 Belarus
In [24]:
# make plotting easier
import matplotlib 
matplotlib.rcParams.update({'font.size': 60})
pd.options.display.mpl_style = 'default'
def plot(x, y, dataset):
    dataset.plot(kind='scatter', x=x, y=y, figsize=(20,10), s=200)
In [25]:
# correlation matrix.  GDP appears to work much better on a log scale than on a linear scale.
rlrw[['rol_score','log_gdp','elec_pros', 'per_auto','hprop']].corr()
Out[25]:
rol_score log_gdp elec_pros per_auto hprop
rol_score 1.000000 0.871797 0.590591 0.891206 0.889864
log_gdp 0.871797 1.000000 0.499732 0.833674 0.834366
elec_pros 0.590591 0.499732 1.000000 0.777183 0.510440
per_auto 0.891206 0.833674 0.777183 1.000000 0.803828
hprop 0.889864 0.834366 0.510440 0.803828 1.000000

List of states:

Albania, Argentina, Australia, Austria, Bangladesh, Belgium, Bolivia, Bosnia and Herzegovina, Botswana, Brazil, Bulgaria, Burkina Faso, Cambodia, Cameroon, Canada, Chile, China, Colombia , "Cote d'Ivoire, Croatia, Czech Republic, Denmark, Dominican Republic, Ecuador, Egypt, El Salvador , Estonia, Ethiopia, Finland, France, Georgia, Germany, Ghana, Greece, Guatemala, Hong Kong, Hungary, India, Indonesia, Italy, Jamaica, Japan, Kazakhstan, Kenya, Kyrgyzstan, Lebanon, Liberia, Macedonia, Madagascar, Malawi, Malaysia, Mexico, Moldova, Mongolia, Morocco, Nepal, Netherlands, New Zealand, Nicaragua, Nigeria, Norway, Pakistan, Panama, Peru, Philippines, Poland, Portugal, Romania, Russia, Senegal, Serbia, Sierra Leone, Singapore, Slovenia, South Africa, South Korea, Spain, Sri Lanka, Sweden, Tanzania, Tunisia, Turkey, Uganda, Ukraine, United Kingdom, United States, Uruguay, Uzbekistan, Venezuela, Zambia, Zimbabwe

In [26]:
plot("rol_score", "log_gdp", rlrw)
In [27]:
plot("rol_score", "per_auto", rlrw)
In [28]:
plot("rol_score", "hprop", rlrw)
In [29]:
# democracy does less well 
plot("rol_score", "elec_pros", rlrw)
In [30]:
rlrw[["rol_score", "wjppc", "fh_rol", "wb_rol"]].corr()
Out[30]:
rol_score wjppc fh_rol wb_rol
rol_score 1.000000 0.976609 0.875058 0.941708
wjppc 0.976609 1.000000 0.851513 0.967053
fh_rol 0.875058 0.851513 1.000000 0.858182
wb_rol 0.941708 0.967053 0.858182 1.000000
In [31]:
plot("rol_score", "wjppc", rlrw)

This is good news for measurement: while the World bank and Freedom House measures are less theoretically sound, they seem to serve as pretty good proxies. The World Justice Project really seems to capture the same rule of law ideas that I capture with a subset of their data.

It turns out that the World Justice Project listened---since I did the data analysis for this book (which was several years ago), they also produced a single aggregate score from their factors. It looks (from fairly casual examination) like it just takes a mean of the factors, rather than using any kind of dimensionality reduction technique.

I also would like to look at the BTI. Since there's a 2015 WJP score, which probably covers a similar period to the 2016 BTI, and since I couldn't use the BTI in the previous comparison (as it misses about 20 countries represented in my rule of law score), let's just look at them together.

In [32]:
plot("pca2015", "wjp_overall", wjp_bti)
In [33]:
wjp_bti[["pca2015", "wjp_overall", "bti"]].corr()
Out[33]:
pca2015 wjp_overall bti
pca2015 1.000000 0.998781 0.693348
wjp_overall 0.998781 1.000000 0.705156
bti 0.693348 0.705156 1.000000

It looks like the first principal component of the WJP factors is extremely close to the aggregate score the WJP used. It also looks like it's pretty far away from the BTI's index. In light of the 2012 analysis, this is some (weak) reason to think that the BTI rule of law score ought to be less trusted than the WJP's new aggregate score.

Note that this is based on just the BTI's rule of law subscore, not the entire index score.

In [34]:
# WJP-BTI countries
print ", ".join([x for x in wjp_bti["Country"]])
Afghanistan, Albania, Argentina, Bangladesh, Belarus, Bolivia, Bosnia and Herzegovina, Botswana, Brazil, Bulgaria, Burkina Faso, Cambodia, Cameroon, Chile, China, Colombia, Costa Rica, Cote d'Ivoire, Croatia, Czech Republic, Dominican Republic, Ecuador, Egypt, El Salvador, Estonia, Ethiopia, Georgia, Ghana, Guatemala, Honduras, Hungary, India, Indonesia, Iran, Jamaica, Jordan, Kazakhstan, Kenya, Kyrgyzstan, Lebanon, Liberia, Macedonia, FYR, Madagascar, Malawi, Malaysia, Mexico, Moldova, Mongolia, Morocco, Myanmar, Nepal, Nicaragua, Nigeria, Pakistan, Panama, Peru, Philippines, Poland, Republic of Korea, Romania, Russia, Senegal, Serbia, Sierra Leone, Singapore, Slovenia, South Africa, Sri Lanka, Tanzania, Thailand, Tunisia, Turkey, Uganda, Ukraine, United Arab Emirates, Uruguay, Uzbekistan, Venezuela, Vietnam, Zambia, Zimbabwe
In [35]:
# can we get any more insight about how the WJP and BTI scores differ?
plot("pca2015", "bti", wjp_bti)

It looks like there's a lot of disagreement as we get to states that score higher with the WJP. Let's see where the disagreement is. The "diffs" variable in here is the absolute difference between the scaled first principal component of the WJP factors and the BTI score.

In [36]:
wjp_bti[["Country", "pca2015", "bti", "diffs"]].sort_values("diffs", ascending=False).head(n=20)
Out[36]:
Country pca2015 bti diffs
64 Singapore 2.953962 0.138184 2.815778
74 United Arab Emirates 1.647602 -0.765280 2.412882
4 Belarus 0.105652 -1.267205 1.372857
78 Vietnam -0.158569 -1.518167 1.359598
14 China -0.328890 -1.618552 1.289662
58 Republic of Korea 2.676028 1.392996 1.283032
5 Bolivia -1.037110 0.238569 1.275679
72 Uganda -1.078753 0.138184 1.216937
36 Kazakhstan -0.222975 -1.367590 1.144615
29 Honduras -0.988580 0.138184 1.126765
69 Thailand -0.007683 -1.116627 1.108945
48 Morocco -0.016033 -1.116627 1.100595
62 Serbia -0.176958 0.891071 1.068029
35 Jordan 0.531524 -0.514318 1.045841
52 Nigeria -1.047447 -0.012393 1.035054
67 Sri Lanka -0.112136 -1.116627 1.004492
31 India -0.109861 0.891071 1.000932
76 Uzbekistan -0.621788 -1.618552 0.996764
46 Moldova -0.498048 0.489531 0.987579
8 Brazil 0.177566 1.142033 0.964467

What immediately leaps out from that is that the BTI is substantially more generous with fairly democratic/liberal countries and less generous with fairly undemocratic/illiberal countries---particularly of note here are Singapore, UAE, South Korea, and, in the other direction, India---cases in which, intuitively, the BTI rule of law score tracks liberal-democraticness in general rather than rule of law in particular.

I'm not sure why this should be the case, except perhaps that the comprehensive nature of the BTI's codebook orients the experts whom they survey to thinking about liberal democracy as a whole, rather than the rule of law in particular, even when answering rule of law specific items. Unfortunately, the current country reports do not appear to actually be downloadable from the BTI website.

Conclusions

  • Equality as justification and basis for development

  • Theoretically grounded rule of law measures work at least as well as existing leaders, better than others. (But really, trust the WJP)

The Rule of Law in the Real World

http://rulelaw.net

data vis

(Use discount code GOWDER16 for 20% off direct from Cambridge until July 1)