Economic Impact and Statistical Significance: Interpreting Accounting Research for Evidence-Based Policy Making

Barone, E. and Mcleay, S. (2019) Economic Impact and Statistical Significance: Interpreting Accounting Research for Evidence-Based Policy Making. Discussion Paper. Chartered Accountants’ Trustees Limited.

Abstract

This briefing, funded by ICAEW’s charitable trusts, is addressed to practitioners such as financial analysts, finance directors, audit partners, ‘technical’ departments in audit firms and all who are involved in standard-setting. Many academic studies rely on statistical significance to draw conclusions from empirical analysis about the impact of policy issues, such as the adoption of IFRS or the change in the particular requirements of a Standard. This approach has received criticism because of a failure to focus on, or demonstrate, the economic significance of the change. We set out to re-interpret the evidence base of published quantitative accounting research in terms of its economic significance. Results were collected from 40 published research studies, mostly amongst the 200 surveyed by Brian Singleton-Green in The Effects of Mandatory IFRS Adoption in the EU: A Review of Empirical Research, ICAEW (2015). We found that economic impact was discussed for a wide range of factors that might be influenced by regulatory change (eg, cost of capital, analyst following, earnings management, and many others). Generally, however, authors infer economic impact in terms of an expected change in the variable of interest, for the average firm. Amongst these studies, we did not find any extrapolations of economic impact that were quantified at the level of the economy or economic sector as a whole. Unfortunately, when we attempted to estimate the economic impact on the corporate sector on the basis of published research, we met several barriers that prevented us from doing so to our satisfaction. Most of the obstacles were technical in nature, relating to statistical method and the conventions of journal paper publication. We describe these within this briefing. It is important to note, nevertheless, that each of the papers surveyed was fully transparent in the justification of its statistically significant results. To the practitioner, the conclusion - that we experienced difficulty in quantifying the full extent of economic impact - may seem surprising, given that the large samples commonly used by researchers account for much corporate economic activity. For policy-makers, and accounting researchers with an interest in the political and economic implications of regulation and standardization, who may wish to model the kind of ‘effects analysis’ considered by EFRAG, IASB and FASB, a useful set of research questions emerges from our own attempts to unravel the connections between statistically significant results and their economic impact. These are summarised in the concluding section.

Actions (login required)

Edit Item Edit Item