leidenlawmethodsportal

Official Statistics in Empirical and Legal Research

Last update:

Official statistics, meaning databases or aggregated figures and indicators compiled by a governmental organization, can play a valuable role in empirical research projects and legal arguments. The quality of the data is often high, and government agencies have unique access to data pools such as population census or tax returns.

Legal and empirical arguments can be supported by government data, for example regarding income, poverty, health, and education (Pogge, 2005); tax avoidance and foreign investment (Bolwijn et al. 2018), and trials and crimes (Kritzer 2004).

Which number to use?

When using an official database or indicator (like GDP, inequality in a country, number of crimes), it is sometimes necessary to have a close look which measure to use for a particular research question, since many aggregate indicators are subject to debate (Mügge, 2016). A good case in point are indicators of economic inequality, where over time more measures beyond the well-known Gini coefficient have spread, some of them said to represent better changes in the relative wealth of rich vs. poor (Cobham et al., 2016). It often depends on the specific research question which indicator is best. However, a good practice is to use several different indicators and check whether all yield the same result with regards to the research question.

Which number to trust?

Especially when undertaking internationally comparative research, it happens frequently that statistical offices from different countries use different definitions or methods to compile statistics. For example, I once found four different figures for a same statistic (foreign direct investment flows from a certain country to another) published by four different agencies. Sometimes, agents collecting data have more or less ample discretion when compiling data (Boivin and Cordeau, 2011).

International organizations such as the United Nations, the Organisation for Economic Co-operation and Development and Eurostat, however, undertake efforts to harmonize methods and definitions across countries. Nevertheless, this can mean that definitions and/or methods are not always consistent over time. Another issue highlighted by researchers is that, in some instances, official statistics can be subject to political influence (Aragão and Linsi, 2020): economic growth and unemployment figures in the pre-election year, high or low crime rates… Approaching statistics with a critical mindset is thus good research practice.

Whether one should discard statistics because of such issues, however, always depends on whether the issues matter for the research question, in a way that they could falsify the result. Sometimes, investigating the production of an official statistic itself could be a very insightful and rewarding research process.

Resources

  • Dutch statistical institute

  • Eurostat

  • OECD

  • World Bank

  • UN

  • Statistics on the activities of the European Court of Justice

  • Aragão, R., Linsi, L., 2020. Many shades of wrong: what governments do when they manipulate statistics. Review of International Political Economy 1–26.

    This paper critically examines statistics and their production.

  • Boivin, R., Cordeau, G., 2011. Measuring the impact of police discretion on official crime statistics: A research note. Police Quarterly 14, 186–203.

    This paper critically examines statistics and their production.

  • Cobham, A., Schlögl, L., Sumner, A., 2016. Inequality and the tails: the Palma proposition and ratio. Global Policy 7, 25–36.

    This paper critically examines statistics and their production.

  • Mügge, D., 2016. Studying macroeconomic indicators as powerful ideas. Journal of European Public Policy 23, 410–427. https://doi.org/10.1080/13501763.2015.1115537

    This paper critically examines statistics and their production.

  • Bolwijn, R., Casella, B., Rigo, D., 2018. An FDI-driven approach to measuring the scale and economic impact of BEPS. Transnational Corporations 25, 107–143.

    An example of research and legal argument using statistics.

  • Kritzer, H.M., 2004. Disappearing trials? A comparative perspective. Journal of Empirical Legal Studies 1, 735–754.

    An example of research and legal argument using statistics.

  • Pogge, T., 2005. Recognized and violated by international law: the human rights of the global poor. LJIL 18, 717.

    An example of research and legal argument using statistics.