NEW RESEARCH | How to evaluate new-age innovation policies

In order to evaluate innovation policies for a more sustainable society, we need to leave old methods behind. But the transition has major challenges ahead.

The main objective of innovation policy is to create favorable conditions for bringing new ideas to the market. Traditional evaluation of such efforts is roughly based on counting the number of publications and patents.

– But we are facing a systemic change, a so-called transformative change, as we move to a more sustainable system to address societal problems such as climate change and natural resource depletion. The innovation policy needed for this shift looks different and also needs completely different methods for evaluation,” says Carolina Resende Haddad, Chalmers University of Technology, who in her doctoral thesis has studied what these methods can look like.

5 new aspects of the strategy

An innovation policy strategy focusing on systemic change builds on previous strategies, but distinguishes itself by taking into account five other aspects:

• focuses science, technology and innovation policy on addressing key societal challenges and going beyond the sole objective of growing economies.
  
• Sees the need to involve more stakeholders in the innovation process, such as civil society and NGOs.
  
• Wants to connect local, regional, national and international levels of governance to accelerate change.
  
• Sees the need to combine science, technology and innovation with climate and environmental policy.
  
• Calls for action to immediately end practices, regulations and technologies that sustain the fossil fuel industry.

Several challenges with new method

One of the challenges that evaluators face in assessing what should be included in this third generation innovation policy is to assess how interventions enable technological change in society to address sustainability problems. It also requires a lot from policy makers and evaluators as they need to be aware of the types of outcomes that interventions should target and develop.

– And is transformative change even possible – can government programs really transform the system towards sustainability? No one has yet been able to answer this question and it is one that the innovation agency Vinnova is grappling with.

Carolina Haddad argues that the type of outcomes that the new innovation policy should aim for are complex and consist of several interlinked activities and processes.

– For example, there is a need to understand how interventions contribute to knowledge development, create new markets and support new companies to enter the market. It is therefore not sufficient to measure the number of publications and patents generated by the actions.

She argues that it is more a matter of understanding how and why interventions contribute to the transformation of technology and societies towards sustainability.

– It also means that we can no longer demand accountability, i.e. that resources used to implement policies have been used correctly and effectively to achieve the desired results.

If in the future we have an innovation policy for a more sustainable society, what would be the characteristics of the evaluation of such a policy?

– In the best of all worlds, an evaluation strategy that combines different types of evaluations must first be developed. There is also a need to go beyond accountability and focus on learning and ongoing evaluation. Finally, you need to be able to show how your efforts contribute to changing society and technology in a sustainable direction,” says Carolina Resende Haddad.

Contact carol.resende.haddad@gmail.com

More about the thesis
Carolina Resende Haddad recently defended her thesis at Chalmers with the thesis Transformative innovation policy evaluation: characteristics, challenges, and lessons from practice.

Read also: Opinion | Time for a new innovation authority