I used to lead a big data product that specialises in research data.
Our primary customers were universities, governments, and corporate R&D. In that role, I learned a lot about how careful you need to be when introducing any new metric to assess research quality or success.
Metrics are powerful, and the risk of unintended consequences is huge.
Here’s a provocative but not crazy outcome I see on the horizon.
The focus on translation & commercialisation could lead to “startup or perish”
In the last decade, there has been an increased focus on the translation and commercialisation of research.
Governments and other funders, quite rightly, want to ensure that universities share the valuable research and knowledge they generate.
They want great ideas to create value for society and the economy.
However, attention has recently turned to how to measure this.
- A good example is the UK’s new Knowledge Exchange Framework (KEF) – the first results were published March 2021. The KEF evaluates universities across 7 areas. One of these areas is “IP and commercialisation”.
- In the US, universities are under pressure to articulate their value to state and federal government. University spinouts and graduate startups are one angle many are choosing (example).
Why does measurement matter?
If you set a specific goal, people tend to optimise for that objective regardless of the consequences (Goodhart’s Law).
You’re changing the system, not just measuring the system.
So far, academia doesn’t have a long history of evaluating universities based on the number & value of startups they produce. But we’re on the cusp of it becoming an important formally-evaluated measure.
It’s one thing for a professor to be on a couple of startup boards because they’ve mentored some promising biotech entrepreneurs.
It’s quite different for a professor to know that they may be assessed based on how many startup boards they’re on.
When “tracking” turns to “metrics-based evaluation”, you change the behaviour.
Metrics can be powerful tools to incentivise desired behaviour change
It’s non-controversial to want to help translate research out of universities into real-world impact.
It’s the methods used to measure & evaluate translation that we need to be careful about.
We want to maximise positive outcomes, and minimise negative ones.
What do you think?
Do we risk burdening academics with additional obligations?
Or are these translation and commercialisation measures helping to focus attention on important outcomes that would otherwise be ignored?
I’d love to hear your thoughts. Send me a DM on Twitter.
Photo by Possessed Photography on Unsplash