SROI has grown in popularity within the not-for-profit sector, and more recently the private sector too, but is yet to be used seriously by a government or international organisation. We feel that this in part may be due to the fact that an extensive academic literature in fields such as economics and philosophy has yet to really make its mark on SROI, although there are a number of key lessons from these literatures for the SROI methodology. We seek to set out some of these main challenges and lessons in our new research paper ‘The Seven Principle Problems of SROI’. The paper also provides some suggestions on areas that need addressing in SROI. SROI has a number of positive attributes and we hope this paper can guide its development to make it a more robust method in the future.
Here is a summary of our main findings. We acknowledge that some of these issues have already been highlighted and acknowledged by the SROI field and literature. The full paper can be downloaded here.
1. SROI lacks a clear principled normative approach
SROI lacks a clear definition of ‘social value’ making it hard to determine the meaning of the SROI ratio. Other approaches such as cost-benefit analysis (CBA) have well-defined foundations centred around social wellbeing.
2. SROI is silent on the issue of interpersonal comparisons and perversely places greater weight on the outcomes of the rich
A key challenge for all social impact methods relates to the rules used to aggregate impacts over different people in society. This is the issue of interpersonal comparisons and it determines how we weight impacts on different individuals and how we deal with important issues such as fairness and equality in social impact measurement. A huge literature in economics and philosophy exists on this but has not been acknowledged fully in SROI. A perverse but unintentional result is that the values of outcomes for richer groups are weighted higher in SROI whilst best-practice in other methods such as CBA is to use equal weighting across all individuals.
3. SROI’s views on stakeholders can be too narrow
Involving stakeholders is one of SROI’s core principles. However, it is (correctly) recognised within SROI that stakeholder involvement is not always possible, but SROI could go further to provide guidance on non-stakeholder valuation methods such as benefits transfer techniques, which can deliver a robust social impact assessment without recourse to stakeholders if necessary. SROI also needs to acknowledge and incorporate in to its framework non-use values associated with outcomes, which are values that may be held by people who are not stakeholders. Non-use values can make up a large proportion of the overall social value of an intervention, but they cannot be ascertained from stakeholders themselves.
4. The ratio calculation is susceptible to biases
For most interventions there will be negative outcomes for some groups. How these negative outcomes (which are costs) are treated in the ratio calculation has significant impacts on the overall results in SROI and the interpretation of these results. Explicit guidelines on different approaches to measuring ratios and their interpretation should be provided in SROI.
5. Statistical methods for inferring causality are problematic in SROI
Understanding the relationship between cause and effect is critical to measuring social impact. The SROI principles and guidelines go some way to covering this issue but on the whole SROI has not kept up with major developments in the fields of statistics and econometrics and is also silent on the issue of statistical inference testing. It is likely that many SROI studies overstate impact because the appropriate statistical tools have not been applied. The literature on best-practice methodology for estimating impact is vast and can and should be applied to SROI to improve the methodology with immediate effect.
6. Valuation theory and methodology in SROI are outdated and incomplete
SROI approaches the task of valuation without a comprehensive theory of valuation. It relies heavily (entirely) on valuation methods developed in economics and CBA, but has ignored best-practice guidelines and new developments in this field leaving valuation in SROI somewhat ad-hoc and outdated. This is evidenced by the frequent (mis)use of inaccurate or inappropriate values in SROI studies.
7. The meaning of the SROI ratio is vague
SROI must be clear from the outset as to whether costs are related to society as a whole (like public sector policies), in which case the opportunity costs of the intervention must be considered, or whether they relate just to a specific organisation. This affects the interpretation of results and if the main focus in SROI is on costs to the organisation the methodology would not be suitable for public policy evaluations.
Many of these issues simply require a clarification of standpoint in SROI, while other issues are more involved but can be addressed with the tools, knowledge and vast body of research that we have at our disposal today. SROI should seek to address these challenges, but at the same time make sure that it maintains its own unique advantages and approach. This will increase its reputation among policymakers with the potential to develop SROI into a serious alternative to other currently accepted methods of social impact measurement such as CBA.
At Simetrica we will be working with key stakeholders and practitioners in SROI to pioneer an Advanced SROI Framework with Social Value UK. If you would like to be involved or contribute to this we would love to hear from you via: