Implementation Enablers — Monitoring and Evaluation

Monitoring and evaluation can be used to improve both the intervention and the implementation process.

Monitoring is the routine and systematic collection and use of information against a plan. It makes use of existing data and information, with a view to ongoing cycles of improvement.

Evaluation is a planned investigation of a project, programme, or policy used to answer specific questions. It can be related to the design, implementation, and results (cause and effect) of an intervention.

Monitoring and evaluation use data to determine if indicators are being met and outcomes achieved. This information is used to improve both the intervention and the implementation process. Doing this well helps to identify ‘quick wins’. It also builds credibility, support, and momentum for an intervention.

Involving relevant stakeholders and effective planning are crucial aspects of monitoring and evaluation. This helps clarify what outcomes are useful to measure. It also helps to agree specific responsibilities for monitoring and evaluation.

Monitoring and evaluation need appropriate reporting and review mechanisms. It is important to assess existing data collection processes before introducing new ones.


Resources

The Measuring Outcomes Guidebook (Strengthening Nonprofits, 2010) is a guide for organisations interested in learning more about implementing or improving their procedures for measuring and monitoring outcomes. Click here


The Magenta Book: Guidance on what to consider when designing an evaluation (HM Treasury, 2012) is the recommended central government guidance on evaluation in the UK. It presents standards of good practice in conducting evaluations. Click here

The A—Z of Implementation

There are a range of terms used when writing or talking about implementation and in implementation science. This glossary provides a short definition for each of these terms.

Adaptable components

Elements of an intervention which may be tailored to local settings during implementation without undermining the integrity of the intervention itself.

Barriers

Factors which hinder the implementation process and reduce the probability of successful implementation.

Capacity

The ability or power to do, understand or absorb something. This can apply to an individual, a team, an organisation or a whole system.

Coaching

A formal, typically short-term, arrangement between a coach and an individual focused on developing work-related skills or behaviours.

Community

A group of people living in an area or having characteristics in common (e.g. city, neighbourhood, organisation, service, business, professional association); the larger socio-political-cultural context in which an intervention is intended to operate.

Consultation

The action or process of formally discussing something with stakeholders, generally asking stakeholders a relevant question and receiving answers to that question. While the views of stakeholders may then be used to influence decisions, there is no commitment or requirement to do so.

Context

The set of circumstances or unique factors in which implementation takes place. This can refer to both the wider, systemic context, as well as the specific setting in which a specific intervention will be implemented.

Continuous Improvement Cycles

Ongoing use of emerging data and evidence on outcomes and implementation, and using that information to learn from experience, inform future implementation and improve outcomes. Progress is, therefore, achieved in an incremental manner over time.

Core Components

Indispensable elements of an intervention or implementation plan, which cannot be changed without undermining it. All core components should be delivered with fidelity.

Data-Based Decision Making

Using processes for collecting and analysing different types of data to guide decisions with the aim of improving outcomes on an ongoing basis.

Diffusion

A process by which an intervention is communicated through certain channels over time. The spread of ideas through diffusion is generally a passive process, following an unpredictable, unprogrammed, emergent and self-organising path, e.g. word of mouth.

Dissemination

An active, negotiated and influenced means of spreading an intervention or information about an intervention to relevant target groups.

Enablers

Factors which increase the probability of successful implementation.

Evaluation

A planned investigation of a project, programme, or policy used to answer specific questions. It can be related to design, implementation, results, and outcomes (cause and effect) of an intervention.

Evidence-Based Interventions

Practices, programmes, policies, strategies or other activities that have been empirically shown through scientific research and evaluation processes to improve outcomes to some degree.

Fidelity

Delivering an evidence-based intervention exactly as intended by those who developed it.

Framework

A structure, overview, outline, or system consisting of various descriptive categories and the presumed relationships between them.

Implementation

The carrying out of planned, intentional activities that aim to turn evidence and ideas into policies and practices that work for people in the real world. It is about putting a plan into action; the ‘how’ as well as the ‘what’.

Implementation Plan

A plan outlining the key activities, responsibilities, timelines and other important information required to achieve the desired outcomes from implementing an intervention.

Implementation Readiness

The extent to which organisations and individuals are both ‘willing’ to, and ‘capable’ of, implementing any specific intervention.

Implementation Science

The scientific study of how interventions are incorporated into service settings. It seeks to identify activities, contexts and other factors that increase the likelihood of successful implementation and lead to improved outcomes for people.

Implementation Strategy

Any specific method or action aimed at overcoming barriers, increasing the pace and effectiveness of implementation, and sustaining interventions over time.

Implementation Team

A group or structure that oversees, supports and attends to, moving an intervention through the stages of implementation. They actively use strategies to facilitate implementation.

Inputs

Resources needed to carry out activities and outputs.

Intervention

Any policy, practice, service or programme that is being implemented. It can be a change to something already in existence, or an entirely new intervention.

Leadership

The action of leading a group of people, or the ability to do this. This does not just apply to leading a whole organisation or system – leadership can take multiple forms and can occur at any level of an organisation or system.

Logic Model

An adaptable tool that describes the theory of change underpinning an intervention, a programme or a policy. It allows the user to systematically work through the connections between the components of an intervention or process, usually in graphical format on a single page.

Mentoring

A formal or informal arrangement which typically involves an ongoing relationship of support for significant transitions in knowledge, thinking and skills.

Model

A deliberate simplification of a phenomenon. Models are intended to be descriptive and need not be a completely accurate representation of reality to have value.

Monitoring

The routine and systematic collection of information against a plan. It makes use of existing data and information about inputs, outputs, outcomes, or about outside factors affecting an organisation or project, with a view to ongoing cycles of improvement.

Needs Assessment

A process which clarifies the extent to which needs, as well as enablers and barriers to meeting those needs, are accurately known and prioritised by an organisation or group of people.

Outcomes

Intended or unintended changes that occur as a result of implementing interventions. These changes can occur at the level of individuals, groups, organisations or population, and can occur in the short-, medium- or long-term.

Outputs

Key activities and areas of work that will help to achieve the desired outcomes.

Organisational Culture

The norms, values and beliefs that exist and govern behaviour within an organisation.

Resources

A stock or supply of money, materials, staff, and other assets that can be drawn on by a person or organisation in order to effectively implement an intervention.

Stakeholders

Anyone who is affected by or is involved in the intervention being implemented. This includes staff, public, clients, managers, professional bodies, unions, educators, policy-makers and funders.

Strategies

Methods or actions that aim to overcome barriers, increase the pace and effectiveness of implementation, and sustain interventions over time.

Sustainability

An intervention can be considered to be sustainable when not only have the process and outcomes changed, but the thinking and attitudes behind them are fundamentally altered and the systems surrounding them are transformed as well. In other words, the intervention has become an integrated or mainstream way of working rather than something ‘added on’.

Synthesis

A structured process where relevant information and evidence on a topic is gathered, reviewed, assessed and brought together to support decision making.

Theory

A set of analytical principles or statements designed to structure our observation, understanding and explanation of the world.

Theory of Change

An explicit, step-by-step statement of the expected relationship between the intervention and the outcome, i.e. why providing input X should lead to a change in outcome Z, by way of output Y.

Vested Interests

A special interest in maintaining or controlling an intervention, arrangement or institution, usually for personal gain.