Andy Gibson has developed an Evaluation framework as a way of thinking about and evaluating public involvement. The tool is designed to be an easy, participative, and enjoyable way to evaluate the quality of public involvement.
Participants are asked to evaluate PPI (in a project) by positioning 4 sliders to answer questions relating to 4 axis. The questions and 4 axis (that participants need to choose a slider position for) are:
Q1) Is there just one way to be involved e.g. face to face meetings or can you contribute in different ways?)
One way to be involved → Many ways to be involved
Q2) A strong voice participates in discussions and influences decision-making. A weak voice participates in discussions but has little chance of influencing decisions? (Andy could you clarify if this is the recommended wording for this)
Weak voice → Strong voice
Q3) Who sets the agenda? To what extent are public concerns valued relative to organisational concerns?
Organisations concerns → Public Concerns
Q4) To what extent has the organisation decided to, and has actually, changed as a result of involvement?
Organisation resistant to change → Organisation has changed
In addition to positioning a slider for each of these 4 continua participants have the option of adding notes relating to each question.
The position (and colour) of the point in the cube relating to each slider and the notes can be used to enrich the conversation about the PPI project in question.
PPI and research project leads are likely to report on the evaluation data from PPI participants.
Reporting on notes and verbal comments provided by participants is straightforward, however particular consideration is needed when reporting on the individual and aggregate positioning/data of participants’ choice on the slider relating to each of the 4 axis.
The slider position creates a numerical value that is available to researchers. Given that the data collected relates to ordinal data on relatively abstract and subjective scales, these numbers could be more misleading than useful, particularly if aggregated.
It would be technically possible to create an aggregate score on each axis and plot a group aggregate / average point in the cube. This discussion space is for consideration about how, if at all, to aggregate / produce an average from the data; how to express this, if it is possible (without losing the integrity of the model); and any other thoughts about how to create summary statistics relating to the 4 dimensions.