Back to Blog

Exploring what COVID-19 means for changes to our evaluation practice

While COVID-19 has significantly disrupted our lives, a closer inspection shows we are not all impacted in the same ways. For example, some groups have experienced enormous financial and social impacts, while others have not.  Recently, practitioners in economic development and evaluation made sense of the fast-shifting contexts they find themselves in. The two events I attended were: the Economic Development New Zealand (EDNZ) online conference and the Australian Evaluation Society’s onl...

October 8, 2020

Contracting public health and social services: Insights from complexity theory for Aotearoa New Zealand

Public health and social services are often hard to specify, complex to deliver and challenging to measure. This research uses a complexity theory-informed lens to explore the challenges and opportunities of contracting out for public health and social services in Aotearoa New Zealand. This qualitative study considers the implications of complexity concepts with ten public sector managers experienced in contracting out for public health and social services. Our newly published findings show that...

September 8, 2020

Videoconferencing: the new normal for getting work done

Working online and working at home are not new to us, and probably aren’t to you either. But suddenly we find ourselves doing it more, and under different circumstances. For example, meetings, workshops and interviews that a couple of weeks ago we’d have planned to hold face-to-face, are now moving to videoconference. In these new situations, we’re finding that we need to work a bit differently. We’re learning that with the right technology, skilled facilitation and some adaptation,...

March 21, 2020

Envirolink Evaluation completed

The Ministry of Business, Innovation and Employment (MBIE) commissioned Pragmatica to review the Envirolink Fund. The review aimed to inform MBIE of how well the Envirolink Fund operates and whether it achieved its intended outcomes and provided value for money. Pragmatica found that: “Envirolink makes a worthwhile and valuable contribution in the way it supports select regional councils to engage with and use environmental science research and technology. According to respondents, Envirolink ...

December 9, 2019

Rubrics — a tool for unboxing evaluative reasoning

Judy Oakden presented a session about rubrics with Kinnect Group colleagues at the 2019 Australian Evaluation Conference. Rubrics are an intuitive way of implementing the evaluation-specific methodology. Rubrics can help to unbox, demystify and democratise evaluative reasoning. In addition, rubrics help facilitate a clear, shared understanding of how quality, value and effectiveness are defined. Kinnect Group members Judy Oakden, Julian King, Nan Wehipeihana, Kate McKegg, Adrian Field, and colle...

October 1, 2019

Evaluation Building Blocks: A guide

With colleagues from Kinnect Group we published an e-book on the use of rubrics in evaluation. Download your copy here The Kinnect Group developed the ideas in this book over ten years of collaboration. In the introduction they observed: “There are many other guides to evaluation and different ways to do it. This approach emphasises the place of evaluative reasoning in evaluation. It draws on the work of many evaluation theorists, as well as our practice-based body of knowledge. We find that ...

March 14, 2019

Chapter on Evaluation in Social Science Research in New Zealand: An Introduction

Julian King and I recently published a chapter about evaluation in Martin Tolich and Carl Davidson’s introductory social research textbook, Social science research in New Zealand: An introduction. For more information please click here...

December 6, 2018

“Making sense of the mess” Adaptive Action Lab

Change is quickening for individuals, communities, and institutions around the world. Often change brings excitement and opportunity, but sometimes it challenges us. How do we adapt to that change and cope with the messiness that results? Judy Oakden recently collaborated with Glenda Eoyang, founder of the Human Systems Dynamics Institute to run this virtual Adaptive Action Lab. The Adaptive Action Lab was a three-session online training programme. Most of the participants were from New Zealand...

November 5, 2018

Applying complexity theory to contracting out…the journey so far

As part of the early stages of her Master of Philosophy, Judy Oakden was encouraged by her supervisors to attend the Operations Research Society Conference, in Lancaster UK, in September 2018.  As this was the 60th Conference of the Operations Research Society, the organising committee worked hard to attract a wide range of systems thinkers from all over the world. The conference was well supported, and many of the leading systems thinkers, including Gerald Midgely, Mike Jackson, and Peter ...

September 13, 2018

Dealing with complex change in evaluation

Glenda Eoyang, the Human Systems Dynamics (HSD) Institute founder and Judy Oakden, Director of Pragmatica, recently ran a half-day workshop at the ANZEA conference. What set the workshop apart was that Glenda led the session via Zoom from Minneapolis, US, while Judy ran the session in the room in Auckland. The HSD Institute’s depth of experience delivering training globally, both in person and remotely, made this session possible. In addition, the technology was surprisingly easy to set up, an...

July 17, 2018

Regional Growth Programme Evaluation published

Pragmatica completed an evaluation for MBIE and MPI of the Regional Growth Programme (RGP). First, the review assessed whether the RGP worked as intended. Then, it considered some aspects of the RGP implementation, systems and processes. Finally, it also evaluated the value of the results so far. This evaluation focused on ways government agencies worked with each other. It also considered ways central agencies liaised with the regional stakeholders and with Māori in the regions. It was not an ...

April 3, 2018

Understanding the components of evaluative rubrics – new thoughts

Judy Oakden wrote this e-book for practitioners and those wanting to use evaluative rubrics. In her mentoring work, she observed that sometimes certain types of rubrics are not well suited for some kinds of evaluation. In these instances, evaluators could get ‘stuck’. Does this sound like you? “We spent weeks writing up the descriptions for ratings categories. Soon the rubrics became bigger than Ben Hur, taking up pages and pages of tables filled with minuscule writing…For us, the ...

February 27, 2018

Putting the fun and useful back into systems approaches in evaluation practice

Judy Oakden co-facilitated a workshop session with Marah Moore and Jan Noga at the American Evaluation Association Conference in Washington. It was a great chance to connect with others to think about applying systems thinking in our evaluation practice. We used an appreciative inquiry approach to help participants reflect on when systems thinking works well in their evaluation practice. First, we reflected on the questions that matter in developing evaluative criteria. We used four lenses: moti...

November 14, 2017

Adaptive evaluation paper recently published in Emergence: Complexity and Organisation journal

Glenda Eoyang and Judy Oakden recently had a paper called Adaptive Evaluation: A synergy between complexity theory and evaluation practice published in the journal Emergence: Complexity and Organisation.To download the artlcle, click here...

June 28, 2017

You may know what evaluative rubrics are, but how do you do them in practice?

Joint presentation with Jane Davidson and other Kinnect Group colleagues at CES Vancouver. Along with Jane Davidson, Kinnect Group members, Judy Oakden, Julian King, Nan Wehipeihana and Kate McKegg presented Evaluative Rubrics – delivering well reasoned answers to real evaluative questions recently at the Canadian Evaluation Society in Vancouver, May 2017.You may know what evaluative rubrics are, but how do you do them in practice?How do you use them to: engage stakeholders...

May 20, 2017

Evaluation rubrics look easy but can be hard to do well: lessons from the field

Judy Oakden presented a paper on evaluation rubrics, and some lessons learned over the last five years’ in their use at the American Evaluation Association conference in Chicago. Drawing on some ideas from Human Systems Dynamics this paper was developed in collaboration with Glenda Eoyang....

November 30, 2015

Delivering Developmental Evaluation workshops in Australia

Judy Oakden co-presented two full day Developmental Evaluation workshops with Kate McKegg at the Australian Evaluation Association Conference. For more information on these workshops in Melbourne and Canberra click here....

September 10, 2015

Reaching across boundaries in high stakes collaboration

Judy Oakden, along with Kate McKegg and Viv Twyford presented a paper about our learnings about evaluating high stakes collaboration in local government settings at the Australasian Evaluation Society conference.Download PDF here....

September 10, 2015

So now you’ve got the data – what next?

udy Oakden, along with Irene Guijt and Kate McKegg, presented a paper on collaborative sensemaking to navigate diversity at the Australasian Evaluation Society conference. Click here to download PDF....

September 10, 2015

Presenting at ANZEA conference

On 7 July Judy Oakden spoke at the Aotearoa New Zealand Evaluation Association (ANZEA) conference in Auckland on three complexity approaches in Human Systems Dynamics that evaluators can easily use in their work. Her presentation was called ‘A new compass for dealing with complexity when we are all at sea’....

July 7, 2015

Received Human Systems Dynamics professional certification

Judy Oakden received Human Systems Dynamics professional certification from the Human Systems Dynamics Institute (USA). She has expertise in complexity theory and uses HSD in her evaluation practice. In particular she finds:  HSD provides useful process for figuring out what to do next at each stage in the evaluation HSD provides approaches which can be used in developing useful evaluative rubrics which can be used to make judgments of the quality, value and importance of the evaluand ...

June 1, 2015 Posts 1-21 of 21 | Page