In this blog, Dr Nick Maguire from the University of Southampton proposes a meaningful
outcomes framework as the antidote for some of the challenges faced by commissioners and
providers when determining how to measure change.
Measuring change has always been a significant challenge for homelessness services, and
has arguably been brought into sharp relief by the move from centralised funding to grants
with attached outcomes. Through our work in evaluating Psychologically Informed
Environments, we have observed the difficulties and considered alternative methodologies.
The problems
One issue may be considered an artefact of commissioning processes that define what
‘effective change’ looks like. Many homelessness service commissioning processes specify
some, or all of, the change required. This might include service user engagement in
statutory services, or perhaps entering training or employment.
This can result in a paradoxical process whereby service users are less likely to engage in
change, as the organisations struggle to hit the outcomes specified by the commissioning
process. Where organisations do not insulate frontline workers from these expected
outcomes, support staff may encourage those behavioural change that lacks meaning to
individuals and which, whilst achieving outcomes, may actually be a threat to engagement.
Well-evidenced psychological theory and practice (e.g. motivational interviewing) indicate
the value of autonomy in behavioural change, i.e. that the person themselves choose to
make those changes. A worker trying to make a person change, according to an outcome
meaningless to them, erodes that autonomy. Top-down, outcomes-driven commissioning
may therefore actually make interventions less effective.
Related to this is Campbell’s Law which states:
‘The more any quantitative social indicator is used for social decision-making, the
more subject it will be to corruption pressures and the more apt it will be to distort
and corrupt the social processes it is intended to monitor’ (Campbell, 1979)
This suggests that setting outcomes for an organisation to achieve can result in those
outcomes either being ‘gamed’ (manipulated to the situation) or not achieved at all, which
may result in some fudging in reporting. The danger is that these unhelpful processes are
not used to improve commissioning and service delivery, and we continue to do what we’ve
always done as no clear alternative is apparent. Thus we repeatedly miss opportunities to
empirically reflect on and improve the quality of the services we offer people who are
homeless.
Whilst there are other ways of evaluating service effectiveness and individual outcomes,
such as through case studies and other qualitative evidence, they do not provide the overall
outcomes that are usually required as a way of gauging change for the whole group of
people being served.
A solution?
A way of addressing the issues highlighted above, whilst building a fundamentally person-
centred approach, may be through implementing a meaningful outcomes framework. The
question is, how can we design an outcomes framework which specifies increments of
change that are meaningful for the individuals concerned?
This is easier than might be imagined. First we need to ask people accessing services the
following, carefully designed questions: ‘What is getting in the way of where you want to go
and what would you like to be doing more of/less of? These questions will help us to
discover what people aspire to as well as the behaviours they engage with that are barriers
to them achieving their goals.
We can next gauge the incidence rate of those behaviours by counting how often they
happen over a period of time, getting a sense of the average occurrence over a day or week.
Any changes in these behavioural incidence rates demonstrates change related to outcomes
that have been identified as meaningful by those being supported. This approach can also
improve how support is provided as can highlight any environmental or individual triggers
which are hindering the individual’s progress towards their goals.
Adopting this approach is useful for commissioning. It provides evidence of change in the
form of outcomes that is not only meaningful to individuals but is likely to be more
sustainable in the longer term and is concrete and observable. It can also be a really useful
rudimentary method of the economic analysis of the effectiveness of interventions. By
enabling people to tell us how often a particular behaviour is happening, we can estimate
the incidence of the social impact and associated costs of the behaviour. For example, if
someone is struggling with emotions and copes in a way that results in an A&E visit costing
between £45 and £400 depending on severity, any reductions in the incidence of behaviours
resulting in such attendance can be easily calculated.
Most importantly of course is that this approach values the individual’s view about the
change important to them and is therefore a fundamentally person-centred, meaningful
approach to outcomes. Having these conversations can be difficult as people may be unused
to expressing what they want to change and need help to work it out. Using a values-based
approach can be useful to help people identify the behaviours they want to do more and
less of. Asking the person what they value about themselves and others can unlock a
conversation about behaviours that are in service of, or contrary to, their values.
However we measure it, a trusting, quality relationship between service user and worker is
fundamental to enabling change. Commissioning is slowly evolving to make use of
outcomes, but it’s a big – and essential – step for such processes to be based around the
views of the service-user. A golden opportunity for collaboration between service-users,
providers and commissioners is not to be missed!
Nick Maguire
Associate Professor in Clinical Psychology, University of Southampton
Director, OutcomeHome

Nick Loved this Blog!
I am looking to construct a person-centred outcomes framework- on three levels.
One that identifies outcomes at an organisational level, outcomes for the work workforce and thirdly for outcomes for individuals accessing service.
I am based in Glasgow and Edinburgh, and a Practice Development Manger for Homelessness and Trauma Informed Practice and trying to link homeless services in alignment with the new Scottish Government outcomes MONITOR which provided wide outcome domains across the homeless sectors in Scotland.
Those incremental measures of change I agree are vital.
If you are aware of any good working example anywhere,I would be obliged if you could signpost me to.
I have watching very carefully the work of CHRP( mainly via x and linkedIn) over the last year and been hugely impressed by your own research, and the work of Becky Ward and Jennifer Tarabay.
My warm regards to you all.
Mark
Mark Cairney