9. Recommendations

On this page:

• 9.1 Recommendations to improve outcomes measurement
• 9.2 Other recommended actions

This section suggests recommendations for CHAC on how to improve outcomes measurement for the future and other actions that build on the insights from the analysis.

9.1 Recommendations to improve outcomes measurement

At the commencement of this project, there had been little data collected over time to indicate the changes experienced by stakeholders and quantities of stakeholders experiencing those changes. Through stakeholder consultation, and working with the CHAC staff, we developed an understanding of the changes experienced by stakeholders and constructed a dataset of changes experienced by stakeholders that had been involved with the program in the past.

To indicate the quantity of young people that will experience the outcomes we:

• Mapped the experience of past and current clients - start date, end date, status, stage of development reached
• Projected the stage of development reached by the current clients in the next two and five years
• Grouped clients into three categories based on the length of time spent in the program
• Modelled the projected stage of development reached by future clients (based on the experience of past and current clients).

We have made the following professional judgements in our modelling of the changes young people will experience:

• If clients do not progress through one or more stages of development, they do not experience change
• If clients do progress through one or more stages of development, they experience all four of the material outcomes
• The outcomes are experienced differently across the stages of development
• Eight new young people join the program per annum
• The movement of future clients through the program will match the movement of past clients.

This approach is described in detail in Section 5.1.

The analysis included in this report is reliant on estimates made by the organisation, based on the case manager's extensive knowledge of the young people involved. We cross-checked the estimates and assumptions with other stakeholders, including our notes from interviews with young people themselves, and tested them against population data (where relevant). The willingness and ability of young people to engage in the consultation process (for example, due to cognitive impairment) was also a limitation of this analysis.

To improve the rigour of future analyses, CHAC collect data on the activity delivered (outputs) and the changes experienced by stakeholders as a result of this activity (outcomes) on an on-going basis. CHAC should focus on answering these five questions:

1. Who is changed?
2. How do they change?
3. How do you know that they have changed?
4. How important are the changes?
5. How much is as a result of you?

To order to answer these questions, CHAC should track for each young person that they work with:

• Date commenced with the program (start date) and date exited the program (end date)
• Program engagement (e.g. active, inactive)
• Activities that the program provides (e.g. recreation, mentoring)
• Progress towards the achievement of each of the outcomes (increased self-esteem; increased engagement in meaningful activity; more positive connections to others; Reduced likelihood of detention or incarceration)
• Any unintended or negative outcomes that arise
• Number of years that the outcome is experienced for
• Background (e.g. education level; employment history; mental health issues; housing situation; alcohol and drug use; offending history)
• Other support services accessed (i.e. to indicate who else is contributing to change, this will be used to calculate attribution in the future)

A Social Outcomes Measurement and Evaluation Tool has been developed as part of this project.

Due to the nature of the program and the number of young people in the program, the Social Outcomes Measurement and Evaluation Tool was developed with the following objectives in mind:

• Simple - not over-burdensome on program staff or undermine their ability to deliver the project, including the critical relationship development aspect of the program;
• Meaningful - helping the program staff to deliver optimally against their objectives;
• Timely - allowing for regular collection and monitoring of data to allow for course corrections;
• Transparent - negative findings are identified, acknowledged and addressed;
• Context-aware - clear about how changes in the environment of the project impact on project outcomes (e.g. changes to youth justice approaches in different states and territories); and Provide value - to young people, to program staff, as well as funders.
• The Social Outcomes Measurement and Evaluation Tool is designed so that the young people self-select whether they are experiencing change as a result of the program. For Outcome 1.1 (Increased self-esteem), the young people are asked to answer a questionnaire based on Rosenberg's Self-Esteem scale, which has been proven to provide accurate representation of an outcome of self-esteem. Based on these responses, the program manager can record whether the young person's self-esteem is very low, low, normal or high. Similarly, for Outcome 1.3 (More positive connections to others) the young person is asked to answer questions about their connection to the people around them which were designed with reference to The Household, Income and Labour Dynamics in Australia survey conducted by the University of Melbourne, which is widely used by Australian and International researchers and by the Australian Government. The Social Outcomes Measurement and Evaluation Tool will enable objective and subjective indicators to be used.
• It is recommended that data be collected on intake of the young person, at six monthly intervals and at exit from the program. Regular monitoring of data will be useful for demonstrating to the young people what progress that they have made and to engage them in their journey through the program. It will also help them to demonstrate the importance of these changes to stakeholders and how much is as a result of the project, as the tool allows for information to be recorded about other services that the young person is using. The tool should also assist the program manager to better plan how each young person is supported and how to balance competing needs of different young people with limited resources.
• The Social Outcomes Measurement and Evaluation Tool has been developed to be integrated into other monitoring and reporting that the program manager already undertakes. Figure 9.1 indicates the proposed timing for using the Social Outcomes Measurement and Evaluation Tool (the green crosses) and how it can be used as a resource for the program manager to assist with other information collection.

Figure 9.1: Proposed timing of capturing data through the Social Outcomes Measurement and Evaluation Tool

CHAC staff should also capture the number of families and adults in South Inverell that it interacts with and periodically assess the changes and the value they derive, as a result of the program. This could help to identify ways how these stakeholders should be involved in the program in the future, to generate the most value for both the young people and these stakeholder groups. This could be achieved through a short face-to-face or telephone survey conducted on an annual basis. These steps will aid in communicating the full impact that the program is having across the different stakeholder groups.

In future it may be possible to compare the outcomes achieved by the program with the outcomes achieved by similar programs, and with changes in the population. This will help to understand the relative effectiveness of the program.

9.2 Other recommended actions

As a result of the insights from the analysis, the following actions are also recommended for the Youth Prevention and Diversion program.

Funding the program

1. Secure funding for the full costs of the program for a five year period in recognition of the time taken to generate changes for the young people involved

The Department of the Prime Minister and Cabinet contribute the majority of the funding to CHAC. This investment supports 1.3 FTE case workers to work a standard 38 hour working week (1,824 hours per year). On average, the program staff have been working an additional 300 hours per year. This extra time equates to $170k in in-kind investment into the program. It is recommended that CHAC seeks funding that reflects the full cash and in-kind investment required for the program to ensure that it can be delivered on a sustainable basis.

There is a range of risk factors that may make young people more likely to engage in antisocial behaviours. The young people in the Youth Prevention and Diversion program are exposed to many of these risk factors, such as low socio-economic conditions; low education; lack of employment opportunities; mental health issues; and lack of appropriate housing.¹⁷ As a result they require intense, long-term support to avoid re-offending and to develop the sense of identity necessary to maintain a positive path in life. The average length of time that has been spent in the program is five years. To ensure continuity of support for these young people, it is recommended that the program is funded from periods greater than two years, ideally for five years or more.

2. Seek funding from the Tasmanian Government in recognition of the significant justice system cost savings forecast to be generated by the program

The Youth Prevention and Diversion program is expected to generate significant cost savings for the justice system. The justice system is administered by the Tasmanian Government. It is recommended that CHAC share the findings of the analysis with the state government and seek funding to support its activities. As the current primary funder, the Department of the Prime Minister and Cabinet could play a role in communicating the credentials of the program to the relevant section of the Tasmanian Government.

3. Resource the program with more than one person to ensure program sustainability and increase impact

The success of the program is heavily reliant on one key figure, the case worker. The strength of the relationship between the case worker and the young people is critical to the success of the program. However, it poses a risk to the sustainability of the program. Proactive succession planning is required and could be achieved by bringing on-board a trainee to be skilled up to support the work of the case worker. CHAC had a young person who was employed as a trainee in the past, which was reported to be hugely successful, however, this person had to be let go due to reduced program funding.

Demonstrating the value of the program

4. Share knowledge of the program with other organisations focused on youth justice diversion

Youth Prevention and Diversion program has many of the characteristics recognised as best practice in programs for Indigenous youth at risk.¹⁸ These include:

• Long-term mentoring relationship between the case worker and the young people, that continues as young people consolidate positive changes
• Consistent, regular contact between the case worker and the young people (up to 10-20 hours per week), depending on the young person's needs
• Integration of the mentoring relationship into broader programs
• Strong partnerships between the program and other community, education, health and welfare services
• Involving Elders in supporting the young people
• Involvement of Indigenous parents in the mentoring relationship (where appropriate), to improve parent-child relationships
• The case worker / Elder having experienced similar challenges to those facing the young people and have proven their success in overcoming negative life circumstances.

CHAC should seek out opportunities to share its program design, implementation and impact with other organisations focused on supporting young people at risk, particularly Aboriginal young people. This could take the form of conference presentations, seminars, or research studies.