Return on Investment (ROI) is an economic term used to describe the profits accrued in relation to an initial cost of a product. This enables a figure to be calculated that can be compared with other similar products. For example, I spend £20 on Crate A of apples, and £20 on Crate B of apples. I then sell Crate A for £40, and Crate B for £60. Here Crate B has the better ROI, double that of Crate A. In this simple example, it would be seen as more prudent to invest in Crate B because the apples generate a bigger profit than Crate A. A similar example would be where I spend less on Crate B apples than Crate A apples, and they both sell for £40. All things equal, I would want to invest again in Crate B.
Some people talk about the ROI of attending a conference. For example, Edelheim et al (2017) have discussed the ROI of conferences in terms of the time, money and effort taken to attend a conference (see also Nicholas Rowe’s Conference Inference interview on this issue). These are all variables that can be measured, (which might be a red light to those who disapprove of metrics). University management will expect staff to provide full costings of any funding request to attend a conference. More often than not, the person making the request will be expected to outline what they anticipate to be the perceived (and somewhat ambiguous) benefits from attending the conference; usually ‘presenting a paper’, and/or ‘networking’, and/or ‘career development’. These are outcomes that are probably not amenable to quantification (other than scoring ‘yes’/’no’).
Thankfully, I was never asked to provide conclusive evidence that attending a conference at the cost of my host Department had a direct ROI. In hindsight I do not know how I could show that the time/money/effort taken to travel to, and attend and present at a conference had a tangible ROI. I would provide a detailed written report on the conference; although it would most likely have been evident if I got anything out of the conference from my enthusiasm, or not, when talking about it.
My main issue is that I do not see conferences as being amenable to cause and effect. Therefore, it is difficult for me to understand how a conference organiser could show ROI. Here the costs might run into hundreds of thousands of pounds, and take years of planning and preparation before its execution, usually over the course of four or five days. One outcome that might indicate conference ROI is “Impact”. I do not consider profit as a ROI for a conference, because in theory, some Universities hold a charitable status. (In addition, it is important to note that this might not be relevant for conferences that are run by Industry, especially if they seek a ROI). Because I worked in an applied field of research (Health Services Research), ‘Impact’ was something that was always at the centre of the projects I worked on. Therefore, when I began my empirical work on conferences, the Impact of a conference was something I was interested to tease out. This reflects my academic background, where the precise measuring and evaluation of intervention and outcome are the norm. But I now see the idea of conference Impact as problematic. The perceived Impact of a conference will probably vary widely amongst those organising it – and similarly for those attending. And as was noted in my 2017 book, Academic Conferences as Neoliberal Commodities, Impact is probably an alien concept to conferences in the Humanities (though this may be changing with the spread of Impact discourses across disciplines, in the UK context and beyond). Therefore, I am now prepared to say outright that I consider ROI is not a helpful concept for gauging the value of conferences.
Methodologically, I think it is difficult to show for some outcomes, that a conference has clearly met its investment and delivered on target what was a priori expected. It will be simple enough to say if a conference has drawn in the numbers of people that it sought to do. (This might be a proxy measure that a conference has broken even). But I fail to see how a conference can be shown to have engaged with each and every attendee, benefitting their being there.It seems to me that the temporality of a ROI is important to consider. For example, I have been to many conferences where any benefit was not immediately felt in the here and now of the conference, but rather developed with time; for example, months could pass and then something would trigger my recognition of what I’d heard at a conference, giving me a better understanding of the present. Lastly, and probably most crucially; it is doubtful to me if there is a clear and reliable measure that can show a conference has had a ROI, other than monetary, which for most universities, is at least in principle, not relevant.
I consider there are also ideological reasons to be critical of conference ROI. Academic life is precarious enough these days without a conference organiser being criticised if the conference is not perceived to have delivered. If a conference does not have the desired ROI, this should not have implications for the employment of the conference organisers, or the attending researcher.
As such then it is probably nigh on impossible to compare and contrast the ROI between different conferences. The lack of clarity on a ROI might be tied to the more general confusion around what is the purpose of conferences. These points highlight the difficulty with applying economic and effectiveness concepts to conferences; (which might be further reason to be critical of the Neoliberal Academy). But clearly, we need a better conceptual basis for understanding conferences. For Conference Scholars, this is our ongoing challenge.
About Donald Nicolson
Donald Nicolson has worked in academic research since 2001 and is still an independent scholar, much to the chagrin of himself and his family. In July 2018 he gave an invited keynote address to the Association for Borderland Studies conference in Vienna, from which this piece arose. He can be approached on Twitter @the_mopster. His first book, Academic Conferences as Neoliberal Commodities, was published by Palgrave Macmillan. Some people think it is not bad.
Conference Inference 
Return on Investment (ROI) is a measure used to evaluate the efficiency of an investment, looking at the amount or return relative to the investment cost. Understandably, ROI has featured mainly in financial and investment domains and is seen as an indicator of profitability, numerically expressed as a percentage or ratio, and uses a basic equation:
(Gain (what you got for your investment)–Investment costs (what it cost you to make the investment))/(Investment costs (what it cost you to make the investment))
or
Net Profit ÷ Cost of Investment x 100
So, in the example of investing money in stocks and shares: if you bought $1,000 worth of stocks and then sold them for $ 1,200, your basic ROI would be 20% or a return ratio of 1:1.2. Again in the financial and investment domains, while ROI has been seen as a flexible way of expressing performance, it does not take time into account, so an investment that takes one year to give a 20% ROI is clearly preferable to one that takes two years to give the same return. The quantification of performance by ROI has developed since the mid- 1960s and become a focal interest in the current millennium.
However, the evaluation of investment and return also has social and environmental applications that lie outside financial value, and since the 2000s a metric of Social Return on Investment (SROI) has been developed to reflect this position. In the evaluation of conference ‘worth’, this may be a better tool that ROI.
SROI = (social impact value – initial investment amount) ÷ initial investment amount x 100%
To convert and organize qualitative information into quantitative values for the purposes of examining conference activities, an Analytical Hierarchy Process (AHP) can been followed to take into account specific elements of the study area, and organise them into:
Inputs: the inputs and resources that the activity entails, such as the costs of undertaking the activity, and the costs of running the activity.
Outputs: the direct or tangible products of the activity (e.g. knowledge contributions or experiential sharing).
Outcomes: the resulting change that the activity contributes (e.g. tangible knowledge dissemination, changes in thinking or practice).
Impact: a measure of what has been produced or developed, less an estimate of what would have happened without the activity (e.g. would the research have developed in the same way or been disseminated without having participated in a specific event).
It is acknowledged that this line of research is foundational, and the economics of conferences have not been widely studied beyond the event provision studies of the Meetings industry. However, this literature fails to differentiate the academic, scientific and professional (ASP) community as a focal sector, makes no analysis of their investment, and fails to assess what they gain from attending events, beyond a rudimentary feedback level.
Building on a paper delivered at the SRHE 2017 ARC, I conducted an empirical analysis of a UK educational conference over three years (N=1,261), examining the financial cost elements of fees, travel, accommodation and financial support that are common to international conference delegates. The findings show significant individual expenditures for both domestic and international delegates (£1,565 for a domestic delegate, £2,214 for an international delegate), but these will not be of any great surprise to conference goers. However, when the figures are extrapolated to published global researcher populations (UNESCO), the costs indicate a multi-billion annual expenditure of a level that cannot be justifiably ignored, and ROI (or better SROI) is a fair way to examine this. In monetary terms and when applied to UNESCO’s global researcher areas, even the most conservative estimations of our expenditures on conferences run to £22 billion+ pa, with area expenditures of OECD £12.7 billion, G20 19.2 billion, Europe £6.8 billion, and even the smallest of areas spending in the hundreds of millions. Because of the simple and conservative calculation of these figures, they cannot be called into question, and serve as our current ‘best available evidence’.
Because of a lack of data, offering concrete measures for Outputs, Outcomes and Impact is difficult, and this again reflects the foundational nature of the research. However, the little research we have does not make comfortable reading. Firstly, the estimated parameters of input measures used in the work are based on best available evidence, but demonstrably conservative. Secondly, the ‘outputs’ of conferences are hard to find, reach an unpredictable amount of people (especially if we consider our global community), and vary in terms of quality, depth, accessibility, etc. The ‘outcomes’ of conferences are mainly considered at a subjective level, and best available evidence shows our conference papers have a publication development rate of less than 37% (dropping from the 44.5% found by the same authors in 2007). As a further observation – in the 3-year study, the conference travel of delegates increased their carbon footprint by more than 6.7 times the normal EU daily level of production. Hicks (2013) calculated the CO2 footprint of a US conference, with an average of 1.2 tons per delegate, and this is 3 times higher than the UK event presented. However, 98% of this was related to air travel, and the UK event accounted for 0.9063 tons per delegate (+ their road travel). So, the emission rates may in fact be similar if differences in travel modes and distances are accounted for. This observation bares particular importance when viewed in conjunction with the recent severe warnings on climate change (United Nations, 2018), and any activity that creates such excesses without tangible need requires immediate consideration. The evidence of negative impacts is clear to see, but we need to furbish counter-evidence that supports what we do.
In face of this empirical evidence, the ‘impact’ of our conference is questionable to a level that I feel exceeds our subjective opinions and any claims that analysing conferences from this perspective is simply neoliberal and reductive to metric analysis. Rather, I think it urges us to develop our conference practices to fill these gaps, and to ensure (the same way that we do with our research in general) that our endeavours are beneficial to as wide an audience as possible, and make a positive contribution to our (peer) societies. As a closing observation, especially in the U.S., the concept of ‘Taxpayer ROI’ has emerged to frame the return that taxpayers get for their non-voluntary contributions. No argument has been put forward to refute that firstly, conference attendees often receive some form of funding from their institutions or research funders to attend and present at conferences. This funding originates primarily from taxpayer contributions, distributed through education/research allocations, and to some degree, private/philanthropic funding. The ‘return’ such investment yields is not yet visible, and although REF assessments take into account our mainstream publications to demonstrate ‘excellence’, the quality and contribution our conference activities make to our global peer society is seldom recognized.
As a group that holds a specific research interest in conferences, it is perhaps appropriate to consider what tangible benefits (return) the tax paying public get for a global investment of over £22 billion per year, and how this can be demonstrated on a more visible level? As a research community I feel that investigating conferences has great potential in many areas, but as their full scope and cost is emerging, it is important to look at our practices not so much against the backdrop of ‘what we have always done’, but more with a view to ‘this is what we could do’ and ‘this is the difference we could make for so many people’.
LikeLike
I was only alerted to the reply to my blog this the first week in February, so my apologies for any apparent tardiness. My points may or may not help add to this discussion.
Reading Nick Rowe’s lengthy exposition of economics – spoiler alert – I claim nowhere near the same level of knowledge. My understanding of economics has in the last few years been framed by a classic paper by John Searle [Searle, John R. “What is an institution?.” Journal of institutional economics 1.1 (2005): 1-22.]. In it, Searle makes a number of useful points which should lead us to look on economics with a little more doubt than is the norm. The crux of his argument is that “economic theory was dependent on human beliefs and other attitudes in a way that was totally unlike the reality described by physics or chemistry”. Looking at economics from an ontological perspective, “most of the phenomena we discuss in economics, such as money, financial institutions, corporations, business transactions, and public offerings of stock are all observer relative. One can say that, in general, the natural sciences are concerned with observer independent phenomena and the social sciences with observer relative phenomena”. I think based on this reading that we should be a little less convinced by the objectivity of economics. And we may or may not want to consider this when referring to the study of conferences.
The main crux of my reply is (with apologies for any repetition) to restate some points I have made elsewhere, in a hope to clarify my position. Based on my reading Richard Rorty (I have a soft spot for contemporary US philosophy) in a 2017 publication [Nicolson, Donald J. “Perspectives: Do conference presentations impact beyond the conference venue? ” Journal of Research in Nursing 22.5 (2017): 422-425.] I have argued that we might want to consider research is a literature – a way of telling a story – which reflects my research background in systematic reviews. I think this has a particular consequence for conferences, giving them a role for discussing and convincing others of one’s research and one’s own credibility. As I said, if we accept this, then research stories (presentations of “Evidence”), like reputations and research tools, have commodity value and are traded at a conference.
As I explained in my book [Nicolson, Donald J. Academic Conferences as neoliberal commodities. Springer, 2016.], a conference IS a marketplace for trading knowledge, ideas, reputations, or industry products. I think we can understand this as the further encroachment of neoliberal values on Academia. This gives the conference a political edge, be it at a ground or birds eye level. As I said, I would find it worrying if there were negative consequences for people running a conference if it did not have a perceived ROI. Academia is a human endeavour and so there is a need to consider the consequences for people.
I am fond of the argument Parker and Weik [Parker, M., & Weik, E. (2014). Free spirits? The academic on the aeroplane. Management Learning, 45(2), 167–181.] made that the value of conferences lies in their highlighting new trends and directions for research, etc etc. I have suggested this might be a means for finding the elusive impact, and explained how this can be achieved if a conference has a solid infrastructure bringing on board politicians. However, as I said in my editorial, I have concluded for now that conferences are better at being marketplaces than being an agent for change. I think we have some way to go to change this, but I too hope that the conference experience can be improved.
I suppose we Band of Conference Scholars all share the same goal of having a better understanding of the conference as a social institution. Like Nick, I think that for now it is better to ask where conferences can make a difference. How any of us better understand this, will probably be by a different route with different background scenery. Our journeys I hope will be equally valuable and enjoyable.
LikeLike