4. Understanding the change
On this page:
- 4.1 Stakeholder engagement
- 4.2 Other sources of data used
- 4.3 Stakeholder Outcomes
- 4.4 The program logic that emerged from stakeholder consultations
4.1 Stakeholder engagement
An SROI analysis requires that the changes are described, measured and valued. The purpose of stakeholder engagement was to understand the relative importance of changes (or outcomes), how the stakeholders would prove and measure change, how they would place value on outcomes, the duration of outcomes and what proportion of the outcome is attributable to others or would have taken place anyway.9
Consultation
Based on previous experience with similar projects, and initial consultations with the case manager and the Department, it was determined that face-to-face interviews (between 30-60 minutes) of one to two people would be the most appropriate method for engaging most of the stakeholder groups.
All stakeholder groups considered to experience material changes have been consulted (see Table 4.1). SVA Consulting conducted the majority of the interviews in Smithton. Some supplementary telephone interviews were held where stakeholders were not available during the project team's visit to Smithton, Tasmania. Observation of program activities (e.g. interactions between the program staff and young people in the program) also took place.
Information from each interview was recorded by hand or on a computer, which was then transferred into a spreadsheet containing the interview notes for each stakeholder. These were referred to throughout the analysis.
The interviews focused on the relative importance of outcomes, how the stakeholders would prove and measure change, how they would place value on outcomes, the duration of outcomes and what proportion of the outcome is down to others or would have taken place anyway.10 The project team continued to interview stakeholders until no new themes were emerging. The age and cultural background of the young people make it difficult to test possible financial proxies.
Alternative engagement techniques, such as surveys, were considered to try to engage with a larger number of program participants and their family / significant others, however, it was determined that this would not be suitable for these stakeholder groups.
Below is a summary of stakeholder groups, the size of the group (currently) and the number of stakeholders engaged per group. All stakeholder groups included in the analysis were engaged through the consultation process.
| Stakeholder Group | Size of group | Number involved in consultations |
|---|---|---|
Stakeholder 1: Young people |
Approximately 35 young people currently engaged through the program |
Three active clients |
Stakeholder 2: Families and significant others |
Approximately 35 family / carer units (not all actively engaged with the program) |
Two carers of active clients |
Stakeholder 3: Community mentors |
Approximately 25 community mentors actively engaged in supporting the young people |
One community mentor who provides support to the 7Up Youth activity centre |
Stakeholder 4: Justice system |
Not available Included members of police, courts, juvenile justice, child protection and correctional/corrective services |
One Youth Liaison Offices, Burnie Police One Youth Justice representative Two Child Protection Officers |
Stakeholder 5: CHAC staff |
38 staff |
Five staff members |
Stakeholder 6: Department of the Prime Minister and Cabinet |
2,530 employees at the Department |
One representative from the Department accompanied the analyst during the site visit Additional 10 representatives of the Department engaged through workshops, project check-points, as well as provided feedback at key project junctions Shared SROI methodology and preliminary findings with approximately 30 staff from the Department through presentations |
Other local service providers |
Not available Includes welfare agencies (e.g. Anglicare, Mission Australia), employment agencies, Men's Shed, counsellors and mental health support agencies |
One Anglicare staff member |
Total |
― |
17 interviews 10 Department representatives involved throughout the project 30 Department staff presented to |
Due to historical and social factors, many of the young people and their parents and carers are reluctant to engage with people from outside of the Circular Head community. Despite the best efforts of the project team, there was a low sample size for the primary beneficiaries (young people) and their families and significant others. Factors that limited engagement with these groups include including a number of young people not showing up for an interview and another not feeling comfortable with speaking to the interviewers. The sample size used was based on the availability of stakeholders during the consultation period for the project.
The young people that did take part in the consultations struggled to express themselves. A language barrier, as well as limited vocabulary appeared to be limiting factors in these conversations. It was particularly challenging in trying to understand why things mentioned in the interviews were important to them and how the program supported them to get there.
Because of the limited engagement with the young people and difficulty in mapping the chain of change based on the conversations that were had, the forecast outcomes had to also be informed through the observations from other stakeholders about the changes that are likely to be experienced by the young people. The most meaningful way to gauge the changes experienced by these stakeholders and understand which were material changes was through the program staff, who are part of the community and have established trust with these stakeholders over a long period of time.
During the analysis, we worked with the case manage rover three to four months, who was introduced to SROI principles and became deeply engaged in the SROI process and methodology. As a result, the case manager understood the need to collect information from stakeholders and did this throughout this period, which was then conveyed to us during our regular check-ins over the phone. As the case manager engages with each of the young people on a daily or weekly basis and records his observations in case notes, he was in a strong position to explain the changes experienced by the young people in the program to supplement the information obtained from the young people during interviews. The case manager conveyed this information by talking through each young person involved in the program individually, to ensure understanding of the specific changes experienced by him or her. This is captured in the assumptions about the quantity of young people that will experience outcomes (see Section 5.1), and was used to project the changes experienced by young people currently in the program, and those that will join the program, in the future.
The information provided by the case manager was verified by interviews with other stakeholders that had direct contact with the young people (for example, local service providers). Engaging a diverse range of other stakeholders in understanding the changes for the young people, ensured that the forecasts were not overly reliant on the opinion of the organisation staff and helped us identify strong trends / common themes. The project team continued to interview stakeholders until no new themes emerged. In addition, other sources of information were used to verify the stakeholder consultation, which are outlined in section 4.2.
In future, the young people will be engaged on an ongoing basis by the case manager to understand the changes that they are experiencing as a result of the program. This will involve the young people self-reporting on these changes and the data being recorded in a Social Outcomes Measurement and Evaluation Tool which has been developed as part of this analysis. For further detail, refer to recommendation 9.1.
Verification of results
CHAC staff were involved in the verification of results at four main points: stakeholder consultations (through feedback on the program logic); the measurement phase (through feedback on the measurement approach); the valuation phase (through feedback on the calculation of the value of outcomes); and the reporting phase (through feedback on the draft report. In addition, the CHAC Board provided verbal feedback on the SROI findings at the reporting phase. Relevant staff from the Department were also involved at each stage of the project. It is anticipated that the report findings and recommendations will be shared with other relevant stakeholders of the program which will help to verify the results and embed the recommendations for future outcomes measurement and evaluation.
4.2 Other sources of data used
Other data sources used to supplement consultation are outlined in the table below.
| Data source | Description | Use in the SROI analysis |
|---|---|---|
1. Data provided by the Department of the Prime Minister and Cabinet |
|
|
2. Data provided by the case worker |
|
|
3. Secondary research |
|
|
4.3 Stakeholder Outcomes
The stakeholder outcomes are judgements based on the data collected by the program, stakeholder consultation throughout this project, secondary research and SVA Consulting analysis. Throughout the data collection process attention was paid to all possible consequences that will arise as a result of the activity: intended and unintended, positive and negative.
This section outlines the stakeholder outcomes for the following stakeholders:
- Stakeholder 1: Young people
- Stakeholder 2: Families and significant others
- Stakeholder 3: Community mentors
- Stakeholder 4: Justice system
The outcomes included in the SROI analysis are considered "material", that is, they are the significant and relevant changes that stakeholders experienced due to the Youth Prevention and Diversion program activities. Materiality is a concept that is borrowed from accounting. In accounting terms, information is material if it has the potential to affect the readers' or stakeholders' decision. According to the SROI Guide, a piece of information is material if leaving it out of the SROI would misrepresent the organisation's activities.
Defining the material outcomes for stakeholder groups is complex. When defining the material outcomes for each stakeholder group, an SROI practitioner must ensure that each outcome is unique or it would be considered double counting. This is difficult as the outcomes for each stakeholder group are necessarily related because they describe all of the changes experienced by the stakeholder. For example, people do not compartmentalise the different changes they experience. Outcomes also happen at different times throughout the period being analysed with different levels of intensity. There are also complex relationships between outcomes for different stakeholder groups.
Stakeholder outcomes were determined by applying the materiality test to the range of consequences identified in the program logic. This was done through initial consultations with the relevant stakeholders and employees of CHAC. The materiality of outcomes was again tested when the number of people experiencing the changes were measured and valued (see Sections 5 and 6). No negative outcomes or unintended outcomes were found to be material.
The following sections outline the outcomes that are forecast to be experienced by each stakeholder group and the anticipated impact of these changes over the five year investment period.
Stakeholder 1 - Young people
The young people involved in the program are Aboriginal young people, generally aged between 12-24 years and live in Smithton or surrounding areas. They are referred to the program mostly by Juvenile Justice Tasmania or police on the basis that they have been in juvenile detention or are at risk of being placed in juvenile detention. It is expected that the type of young people engaged through the program in the future will remain the same.
The table below summarises inputs (investment in the program), outputs (summary of activity) and outcomes (changes) that are forecast to be experienced by the young people.
| Inputs | Outputs | Material outcomes |
|---|---|---|
Not material |
Approximately 69 young people are expected to participate in the program over the forthcoming five year period. Approximately 34% are expected to receive support for 1 to 2 years; 43% for 3 to 4 years and 23% for 5 or more years. The following activities are expected to be undertaken by the program to support the young people:
The exact quantity and the type of these activities will be dependent on the individual needs of the young people participating in the program. |
1.1 Increased self-esteem 1.2 Increased engagement in meaningful activity 1.3. More positive connections to others 1.4 Reduced likelihood of detention or incarceration in the future |
Material Outcomes
The following section describes how outcomes are forecast to be experienced by young people. Each material outcome is being developed over time as young people progress through the program. The detail on the different stages the young people go through during the program is provided in Section 4.4 and 5.1.
1.1 Increased self-esteem
Young people engaged in the program increase their self-esteem by first feeling accepted and included by the case worker. The relationship that the case worker builds with the young people is an essential factor in helping them move closer towards realising their potential.
"...The time it takes to develop trust depends on the past trauma that these kids are dealing with. Some would click within a couple of months, others might take years. They decide when it is time, you cannot expedite it..."
Anglicare Tasmania
A variety of activities that the case worker engages the young people in provide opportunities to model positive behaviours, as well as reinforcing these through praise and rewards. Other individuals that support young people in the program (e.g. community Elders) are encouraged to recognise young people's positive changes, no matter how small.
"... All of my free time is dedicated to the kids. I talk to kids one on one. I would bring them to our home for dinner or a sleep in. This allows them to see how a loving and understanding household functions."
Case worker, CHAC
This is a marked difference to how many of these young people would have been treated in the past by their parents, carers and other figures of authority. In the past many were often led to believe that they are worthless and unworthy of respect, recognition or praise. By treating them in a positive way, the young people start to build their self-esteem.
"…A lot of young people do not care about themselves because they are told that they do not matter. We are trying to reverse that. We tell them that they do matter but also we show them that. It is important that there are no surprises in how we interact with them. If we say we will do something we have to do it, otherwise we are risking of not only undermining all the efforts but have them reject us completely…"
Anglicare Tasmania
"…The main change is more self-confidence - I can talk to strangers, perform at school assembly, don't care what other people think, ask people for help when I need it rather than avoiding doing something or trying to do it by myself…"
Young person 1
"…CHAC helps break the cycle of kids being abandoned by their family and ending up in the system because no one cared enough about them to show what is right and what is wrong. A lot of problems that we are dealing with are generational. Family lacks parenting capacity. They are themselves often have been to prison, live on the dole, drink excessively or always high…"
Child Protection Services (NSW)
"…He did not use to talk at all. Now he is a very confident young man…"
Case Worker (CHAC)
The next change that is seen in these young people is that they start believing in their own self-worth, become more confident in themselves and what they have to offer to the world, and expand their understanding of possibilities available to them in the future.
"…Praising these kids is very important in helping them develop self-esteem and confidence. We use different activities we engage them in to show that they could be good at something, that they could achieve things that they might have not expected of themselves. For example, we would organise an exhibition of paintings done by the young people. They receive praise for their work and are able to positively engage with community. The community changes their perception of these young people…"
Anglicare Tasmania
This is followed by the young people being able to start thinking about their future, dream big and set goals for how they will achieve their dreams. All of this adds to an increased self-esteem which was identified as the most important ingredient for the future success of the young people.11
"…CHAC [the program] have shaped who I am going to be. CHAC helped me realise what I hide and what I need to do to get where I want to be. They have shown me that I have talents…"
Young person 1
"…When we start with them their goals do not go further than just waking up the next day. We help them think about education, jobs and other aspirations; having nice things; being someone in this work and to be accepted in the community they live in…"
Anglicare Tasmania
1.2 Increased engagement in meaningful activity
One of the first things that the case worker works on with the young people is helping them to embed a routine and some stability in their lives. This could be as little as calling them at the same time of the day to check in on their wellbeing, getting them out of bed to take them to school, arranging appointments that they have to attend or simply spending time with them.
The young people who join the program have often been very neglected by their parents or carers and often have quite an unstable life which could include not having a proper place to live, being expelled from school and facing juvenile court. The CHAC case worker often becomes the only person in their lives that offers a sense of calmness and security.
"…I am a lot better with my anger. Baldy [the case worker] taught me to count to 10 when I get really angry and about to do something stupid. He taught me that it takes a better man to walk away than to fight. I try very hard to listen to his advice. It is not always possible, like yesterday someone hit me first so I had to respond. But I am less angry generally so definitely I have changed…"
Young person 2
When the young person is ready, the case worker might start linking him/her into other activities, providing support to access other services, such as drug and alcohol counselling, and start setting higher benchmarks for adhering to the rules of behaviours which he has established. At this point the case worker often also advocates on the young person's behalf to the school to facilitate their return and to be provided with the necessary support when the young person is at school.
"…As a result of the trust that is created with the case worker, many young people show willingness to re-engage in school. The case worker facilitates repair of the relationship the young person has with school and provides them with a 'safety plan' to be able to self-regulate and respond to triggers in a proactive way, instead of letting the situation get out of control…"
Anglicare Tasmania
"…I started going to school, got my Ls and have developed a tight circle of friends at school…"
Young person 1
"…James's school attendance has improved dramatically. He is starting to enjoy school, so he goes because he wants to…"
Case worker (CHAC)
Through this work and with the help of others, the young person is then able to gain insight into their trauma and start developing the essential life skills such as self-regulation, self-care, relationship-building and communication.
"…The kids we work with behave exactly how they think. They are not able to understand what different emotions mean and how to manage these. The program helps them address their emotional immaturity and encourage new behaviours…"
Anglicare Tasmania
These changes provide the foundations for reduced reliance on drugs and alcohol to self-soothe. The young people reduce their intake of drugs and alcohol as a result of counselling supplemented by support from the case worker, as well as a change in habits and behaviour as a result of obligations placed on them by school and training.
As they progress through the stages of development, the young peoples' attitude towards schooling and education changes. They are more engaged and show a desire to apply themselves to learning. Those that are ready to move into employment or choose that path instead of schooling also make a concerted effort to succeed in their chosen area.
This increased engagement in a meaningful activity, whatever that might be for each individual young person going through the program, is a key outcome experienced by the young people. The young people who succeed after the program are no longer roaming the streets or sitting at home doing nothing, instead these young people are more likely to be back at school and applying themselves to their studies, or pursuing a training course, or gaining work experience, or working.
"…I want to get a job on the farm. I no longer go to school - it is just not for me. But I love gardening, fishing or doing other things outside or with animals. My life would be a lot better thanks to Baldy [the case worker]…"
Young person 2
1.3 More positive connections to others
Young people in the program build more positive connections to other people by first developing a positive relationship with the CHAC case worker, who provides young people with a safe place to access help, non-judgemental support and positive social interaction. The case worker tries to spend time with the young person in activities that create positive shared memories, such as going to a movie or having an ice-cream.
"…It is important that the same person does the same job. When workers change it creates a new trauma for these kids…"
Anglicare Tasmania
"…I would call Baldy [the case worker] straight away if there is anything happening in my life - good and bad things. He is like my uncle and Di [CHAC centre CEO] is like my aunty…"
Young person 2
"…Socialisation of the kid is a big part of what I try to do. He has no social skills. He grew up on a farm so he has no skills to even do some basic things for himself or understands how things are done…"
Case Worker, CHAC
This exercise shows young person how positive relationships are created and the joy that this could bring to their lives. Once trust has been built, young people start looking up to the case worker, become more eager to please him and start modelling his behaviour.
"…I am trying very hard to be good. I now get into a lot less trouble. I am more confident to say "no" to friends and refuse to take blame for someone else…"
Young person 2
"…There is also an aspect of wanting to please me. I guess I play a role of a mentor or a role model that he looks up to. All of these kids see me as part of their family to a degree. I guess you could say that I have a very big family!..."
Case Worker, CHAC
"…The support provided by Baldy [the case worker] is just astronomical! He really gets these kids and is always there when they need him. He spends a lot of time with them. We do not have resources to provide such as intensive case management, plus we are not in Smithton so it is hard to come see the kids we work with too often…"
Anglicare Tasmania
This relationship becomes an anchor for the development of other positive relationships. This is particularly important for how the young people start interacting with their family, and could be a very powerful change for them especially when the parents / carers engage in the process.
"…When kids come in at the start you could see that they are very stressed and emotionally do not understand a calm state. When they start interacting with someone like Baldy [the case worker] they feel joy again. Often this is the very first time that they realise what a positive relationship might entail…"
Anglicare Tasmania
Some young people also start expressing interest in understanding better where they come from and wanting to engage with the Elders in the community.
"…Young Koori boys respond well to older men of their culture taking them under their wing. These men are able to utilise a concept of shame very effectively and we start seeing powerful changes. Having an older Aboriginal mentor from who the child can get support and balance from leads to powerful changes…"
Child Protection Services (NSW)
Later in their development, the young people are able to identify the support they need when required and know how to access it. They are also a lot more connected into other support networks around them such as school and the broader community. Young people are also able to demonstrate respect and empathy for others, which would eventually facilitate establishment of deep positive relationships. This outcome is captures the extent to which the young people are able to relate and connect to others. This is different to the outcome 1.1 which captures their internal feeling about themselves; or outcome 1.2 which is about the young people actually doing something meaningful.
1.4 Reduced likelihood of detention or incarceration in the future
Because of their participation in the Helping Hand and Linking Youth program, the young people reduce their likelihood of being in detention or incarcerated in the future. This starts out when the case worker attends court and other legal appointments with the young person, and supporting them through this process. The young person is unlikely to have been supported in this way before by someone who has some degree of knowledge of the court system and knows how to support the young person to achieve the best outcome for them. This process enables the young person to be part of the program and not in juvenile detention. Through this, they gain an appreciation of the legal process as it applies to them.
"…Baldy [the case worker] helped to convince the judge to allow me to go to the Live Free Tasmania camp for six months instead of going to Ashley [youth detention centre] for three months…"
Young person 2
"…We couldn't do our work without someone like CHAC [the program] - we don't have the time, resources or cultural skills…"
Child Protection Services (NSW)
Once the young person begins engaging with the program and accepting the boundaries placed on them by the case worker, such as making sure they attend all their legal and other appointments, they begin to develop an appreciation of the consequence of their actions. The case worker works intensively with the young person to think about the impact of their behaviour on others and on themselves. As a result of this, as well as the fact that the young person has developed a bond with the case worker and wants to please him, their behaviour improves and the incidence of violation of any court orders imposed on the young person is reduced.
"…I used to steal, break into places and even assaulted the police officers, but I am now working very hard to not do bad things…"
Young person 2
"…Help from Baldy [the case worker] to keep me out of trouble is very important to me. I do not want to go to jail…"
Young person 2
This has a flow on effect for the incidence of antisocial behaviours, which are also reduced. The young person has increased their hope and motivation through working with the case worker and is beginning to see the benefits of not living a life of crime. The young person is also more occupied in that he or she is attending school or training more, and is likely to be playing sport or engaging in cultural activities through the program. As a result, the amount of minor offences and misdemeanours that often result from young people being bored and under-occupied decline.
"…If it wasn't for Baldy [the case worker] I would be in prison or doing something stupid on the streets…"
Young person 2
When a young person reaches the point of exercising their own agency as a result of the program, they are less likely to be detained in the future as they are no longer exhibiting antisocial behaviour and offending, and have experienced lasting positive change through the program. They have reached a point where they are unlikely to relapse into criminal behaviour, or be influenced by others to do so. This requires motivation on the part of the young person and support from the case worker and other people in their support networks, all of which has been largely as a result of their participation in the program.
This is a change that is important to young people in and of itself because it is a foundation for positive change in the future. This change captures the shift in the behaviour, attitude and beliefs of the young people who can now not only distinguish the difference between right and wrong, but also understand the importance placed on being law-abiding in a society and self-regulate own behaviours that is aligned with expectation of the community.
"…We are hoping to see changes in child's beliefs about norms and behaviours that are acceptable. This 10 year old child is already talking about things that would definitely lead to him going to jail. For example, talking about bashing people to get what you want, hiding identity to escape police finding him and smoking weed. We want to build a strong network around the child to help break his existing pattern of behaviour and thinking. CHAC [the program] is particularly important to us as the child is placed with an Aboriginal family and they are able to work with them in a culturally appropriate manner…"
Child Protection Services (NSW)
Excluded Outcomes
Two outcomes - the creation of space to be outside normal pressures and access to fun recreational activities - were excluded from the analysis on the basis of relevance. These outcomes were found to be less relevant than other outcomes that were measured based on the information collected from the stakeholder interviews. Elements of these outcomes are also experienced though another outcome that was measured - more positive connection to others - where the young people feel a sense of belonging to a community and feel joy as part of this outcome.
Through stakeholder consultation, a negative outcome that may be experienced by the young people -'family conflict as a result of confronting hard issues' - was found to be not significant as the quantity of young people who experienced this change was low.
Stakeholder 2 - Families and significant others
Families and significant others includes parents and their partners, carers, siblings, aunts, uncles, cousins, grandparents and any other members of the family who are involved in the program.
The table below summarises inputs, outputs and outcomes forecast to be experienced by families and significant others, including the estimated number of people who will experience the outcomes.
| Inputs | Outputs | Material outcomes |
|---|---|---|
Not material |
Approximately 35 family members and significant others are expected to engage with the case manager who will link them to other services which are relevant to the needs of the young person in their care as well as their individual circumstances. Families and significant others will be linked in to other services which are relevant to their individual circumstances. These could include:
The case worker will also coach the families and significant others to provide better support to the young person in their care. The amount of support received by this stakeholder group will depend on how much they want to engage with the case worker and the program. |
2.1 Improve communication between family members 2.2 Increase engagement in lives of the young people in their care |
Material Outcomes
The following is a description of the outcomes forecast to be experienced by families and significant others due to their involvement with the program.
2.1 Improve communication between family members
One of the aims of the program is to improve the relationships of the young people and their families and significant others so the chances of the young person living a healthy and fulfilled life are increased. This can be extremely difficult to achieve as many of the young people regularly experience family violence, child abuse and neglect. Family is a barrier to success for many of the young people as most of them have drug and alcohol issues and are unemployed, which makes it difficult for the young people to be motivated, engage in school and training and develop self-esteem.
"…In 90% of cases we work it is the parent's fault what is happening to these young people. That is why we cannot just deal with the kids. Engaging the family is critical to our success. One of the kids I work with used to be flogged with a poly-pole by his dad if he came home without drugs. How can we succeed when he is coming home to this?..."
Case Worker, CHAC
The young people involved in the program often live away from their parents, or live with different combinations of family members and friends over time. It is not always appropriate for the young person to be re-engaged with their families (for example, due to a history of abuse or neglect). In these situations, it is important that the young person establishes strong relationships with other significant adults in their lives.
"…Stability at home is key to helping the program achieve significant changes for the young people. Therefore, the program proactively tries to link them in with the support they need to provide a stable and safe place for young people to live in…"
Anglicare Tasmania
"…CHAC has been very good to us. I do not know where I would have been without them…"
Mother of a young person
The stakeholder consultations with CHAC staff and other services providers indicate that as a result of the program, and where appropriate, the young people are experiencing improved communication with family members. The program has had a positive influence on relations between family members despite the inherent obstacles in achieving this.
"…Most families get to the stage where they want to re-build their relationships. Families are engaged in a similar way through the program, perhaps just not as intensively. Because of not being able to support the families in a more intensive way, we do not see them changing as much. However, the change that does happen such as improved communication is key to maintaining the changes we are trying to achieve with the young people…"
Anglicare Tasmania
The family members or significant others of approximately one-third of the young people involved in the program (32%) are expected to experience this outcome to a significant degree. Over the five-year forecast period this equates to approximately 22 family members or significant others.
2.2 Increase engagement in lives of the young people in their care
An aim of the program is to actively involve the families and significant others of the young people in the program, the Centre and the young persons' development. Often the experience of families with government and other services has not been a positive one so they are wary to engage with the program. However, the community surrounding the Centre which includes many of the families of the young people have learnt to trust CHAC staff and view the program positively.
Involving families and significant others occurs through the case worker encouraging them to attend meetings with the young person, encouraging them to drop into the Centre, visiting the young person at home, encouraging family members to participate in cultural and other activities at the Centre, providing support where necessary (for example, support dealing with welfare agencies, counselling services and legal services) and advocating on behalf of the parents and carers to other service providers such as education providers.
"…Without CHAC families would see each other less often…"
Child Protection Services (NSW)
As a result of the support provided by the program, families and significant others increase their engagement with the program and, more importantly, the lives of young people. This change is significant for the families and significant others as they develop a more positive relationship with the young person in their lives. They benefit from being able to take more responsibility for the young person and enjoy playing a more active role.
The family members or significant others of approximately one-third of the young people involved in the program (32%) are expected to experience this outcome to a significant degree. Over the five-year forecast period this equates to approximately 22 family members or significant others.
"…Readiness of the parents to change is hugely important in helping us to see an ongoing and lasting change. Once we gain their trust in us and the process, many of them are eager to step up to the plate and become more engaged in the lives of their children…"
Anglicare Tasmania
"…Johnny does not like to come see me when I am drinking wine. He now visits me more often because I stopped drinking now for more than a month. It is very important that he wants to come spend time with me…"
Mother of a young person
"…One kid recently came back to us to ask if we could help him get back into high school. My initial thought was that he just wanted to sell drugs there, so initially I would not have a bar of it. But then I have seen that his mum has straightened up. If the family is doing better, kid would also want to straighten up. Suddenly you see a hope that things could be better for him…"
Case worker, CHAC
Excluded Outcomes
Other outcomes became evident through stakeholder consultations and were included in the chain of change, however, they were not at the end of the outcome chain which is a point at which outcomes were measured.
Excluded outcomes were:
- Access to necessary support (e.g. welfare, counselling, legal)
- Follow routine and rules established by case worker
- Less worried about the young persons' wellbeing (especially related to them going to detention)
Access to the necessary support by families and significant others was not considered to be a material change during the investment period because it was relevant but not significant. This support is received sporadically and is more likely to be received towards the beginning of the investment period than throughout. For some families and significant others no support is received. For this reason, this outcome was not considered to be significant to the SROI analysis.
Following the routine and rules established by the case worker is a change that is mostly experienced by the families and significant others when these are imposed on the young people through the program. This includes attending sessions with the case worker and the young person, attending appointments with the young person, attending their own appointments and ensuring that the young person attends school or training. In discussions with stakeholders, this was not seen as a material outcome of the program as it was more focussed on the activity rather than what results from the activity. For this reason, this outcome was excluded as not relevant to the SROI analysis.
Family and significant others being less worried about the young persons' wellbeing (especially related to them going to detention) came through consultation with this stakeholder group as the program provides them with comfort that the young person is being cared for and supported. However, in further discussions with stakeholders this was not seen as a material outcome for families and significant others as it was related to the young person's experience. For this reason, this outcome was excluded as not relevant to the SROI analysis.
Stakeholder 3 - Community mentors
Members of the Smithton community are involved in mentoring the young people in the program in an informal way through a men's shed, art, cultural activities and sporting activities. Members of the community including Aboriginal Elders come together with the young people to guide the future generation and assist with their personal development. The activities are usually semi-structured and are focused around establishing connections with the young people and passing down knowledge, stories and culture.
The table below summarises inputs, outputs and the outcome forecast to be experienced by Community mentors.
| Inputs | Outputs | Material outcome |
|---|---|---|
Not material |
Approximately 43 community mentors will mentor the young people in an informal way through a men's shed, art and cultural activities, and sporting activities. |
3.1 Increased sense of pride from contribution to community |
Material Outcomes
The following is a description of the outcome forecast to be experienced by the community due to their involvement with the program.
3.1 Increased sense of pride from contribution to community
"…I would take him shopping with me, so that people see him as normal. He used to knick things from everywhere so eventually places would not allow him to come in…"
Case worker, CHAC
The positive changes that happen to the young people was observed by numerous members of the community and reported during stakeholder consultations. Many community members - including Aboriginal Elders, police officers, and other service providers - referred to changes in the young people and described how this had shifted their perception of them.
The impact of this change on community mentors was particularly pronounced. By bringing younger and older people (who can have a particularly negative view) together there are opportunities to learn from each other and to challenge stereotypical views and misconceptions. The community mentors benefit from this interaction by being able to use their own life experiences to guide the young people. They feel a sense of satisfaction that their experiences and knowledge are valuable and feel pride that they are contribute to young people's development and stronger community relations.
"…Community also sees this kid in a different light…"
Case worker, CHAC
Approximately 43 community mentors are forecast to experience this outcome in the forthcoming five-year period.
Excluded Outcomes
Other outcomes became evident through stakeholder consultations which are the necessary pre-cursors (i.e. they are not at the end of the outcome chain) for other more important changes to be realised.
Excluded outcomes were:
- Access to a meeting place
- Sharing cultural experiences with young people
- Establishing a cultural connection with the young people where knowledge, stories and culture can be passed down
Access to a meeting place came through stakeholder consultations as an outcome of the program as the centre where the program is based provides people in the community with a friendly place where young people and the community mentors can come together. However, this outcome was not considered material on its own. Therefore, it was excluded for the purposes of the SROI analysis.
Sharing cultural experiences with the young people occur when members of the Aboriginal community engage in cultural activities with the young people such as storytelling and art making. This outcome was excluded from the SROI analysis as it was not material on its own.
Establishing a cultural connection with the young people where knowledge, stories and culture can be passed down is relevant to the SROI analysis but was not considered to be significant and was therefore excluded. Stakeholder consultation indicated that this outcome would only be experienced by a small number of stakeholders during the investment period because of barriers to achieving this outcome including loss of knowledge and stories within the Aboriginal community and a lack of willingness to participate on the part of some of the young people.
Stakeholder 4 - Justice System
The justice system includes police, courts, juvenile justice, and correctional/corrective services.
The table below summarises inputs, outputs and outcomes forecast to be experienced by the justice system.
| Inputs | Outputs | Material outcomes |
|---|---|---|
None |
Not applicable |
4.1 Reduction in anti-social behaviour 4.2 Decreased number of young people offending 4.3 Decreased number of young people in detention |
"…Youth Justice Tasmania doesn't have the contacts, cultural context to work with these kids. Having CHAC means they get linked in to the support they need to get on the right track…"
Youth Justice Tasmania
These outputs were determined through stakeholder consultation with local police, Juvenile Justice Tasmania, child protection services and CHAC staff.
Material Outcomes
The following is a description of the outcomes forecast to be experienced by the justice system as a result of the program.
4.1 Reduction in anti-social behaviour
The objective of the justice system is to reduce crime and increase community safety. Young people that become involved in the justice system typically start displaying anti-social behaviour, progress to low-level offending and then on to more serious offending that results in juvenile detention.
As a result of the program, fewer young people engage in anti-social behaviour, which can range from minor offensive or harmful acts, to more serious criminal activity.12 The stakeholder consultations indicated that this occurs because the program and the activities that it supports the young people to do "gives the kids something to do" and "keeps them off the streets", which means antisocial behaviour is less likely to occur. The local Major has shared with the CHAC CEO that he believed that as a result of the work that the program has done there was less property damage in the area, particularly the incidences of graffiti and minor property theft. As a result of these changes, the local police have observed less frequent callout and less time spent on patrolling are streets to monitor community and property safety.
"…[Name] is a good kid so I don't understand why he keeps doing all these bad things. Baldy [the case worker] is the only person that was able to influence him. I just do not know what to do with him...If it wasn't for him, [Name] would have been in jail…"
Mother of a young person
"…Western Police district, where the program works, has seen a 29% drop in young people being charged by police this year. Kids that are in trouble in the past are now misbehaving less, getting into trouble with police less…"
Youth Liaison Officer, Burnie Police
One of the most significant reasons that this change has occurred is because the program is designed to re-engage the young people in education or training. For those young people in the program that are able to attend traditional high schools, it has been observed that their behaviour has improved since joining the program.
Over the five-year forecast period it is estimated that this outcome will be associated with approximately 47 young people reducing their anti-social (68% of participants).
4.2 Decreased number of young people offending
The data collected by the program and the stakeholder consultations with the Youth Prevention and Diversion program staff, local police, Juvenile Justice Tasmania and others indicate a clear link between the work done by the program and a decrease in the number of young people offending.
Between 2010 and 2013, 55 per cent of the young people involved in the program did not reoffend. This compares favourably to multiple studies of youth recidivism that have found the juvenile reoffending rate to be 68 per cent (i.e. only 32 per cent did not reoffend).13 Of the participants that did offend, 36 per cent re-offended only once.
The collaboration established between service providers was seen by stakeholders as crucial to ensuring that the young people do not offend or (in most cases) re-offend. For example, Juvenile Justice Tasmania can usually only provide a limited support for a young person due to its limited resources and being located a one hour drive out of Smithton. Because of this, Juvenile Justice Tasmania sees the Youth Prevention and Diversion program as an essential partner to achieve successful outcomes for the young people.
"…Don't get a lot of kids in youth justice from Smithton compared to other areas…"
Youth Justice Tasmania
Similarly, the local police in Smithton regard the program as an essential element to the reduction of young people in the community offending. They rely on the case worker to act as an "unofficial conduit" between the community and the police force because of the "street credibility" that the program has. In collaboration with CHAC, the police are able to deal with potential offenders in a more targeted and holistic way which has the effect of lowering overall rates of offending. This has cost implications for the justice system, with potentially less resources needed the monitor, apprehend and rehabilitate offenders.
"…Less time is needed to go to Smithton — we can just phone up CHAC now and check in, we have an immediate point of contact, rapport and respect. We don't have time to build rapport…"
Youth Justice Tasmania
Over the five-year forecast period it is estimated that this outcome will be associated with approximately 32 young people not offending (46% of participants).
4.3 Decreased number of young people in detention
The number of young people from Smithton and surrounding areas in detention is less as a result of the program. The program has influenced the way that the local court is sentencing young people. For example, the case worker described how the courts are starting to include participation in the program as part of a young person's bail conditions. In this way, the program is facilitating a reduction of young people in detention immediately when they commence in the program. As a result it is expected that fewer young people will experience detention in the future. This is based on the high rates of recidivism amongst young people that have been in detention.14 This change has cost implications for the justice system, with potentially less resources required for juvenile detention.
Numerous stakeholders were confident that but for the program, the young people are likely to be in detention.
"…Baldy [the case worker] helped me to get out of trouble with police. He supported me in court. Instead of going to detention, I am going to "Free Tasmania" camp for 6 month. I am very excited. It will be good to be away from home for a while. Time at the camp would allow me to think about what I have done. I have been there once for one week and it was wicked. At camp I cannot get in trouble. It is in my nature to find myself in trouble. I often get into fights or do stupid things…"
Young person 2
Over the five-year forecast period it is estimated that this outcome will be associated with approximately 37 young people avoiding detention immediately after joining the program (54% of participants), 13 young people not being in detention during the program (19% of participants) and 16 young people not being in detention after the program.
Excluded Outcomes
Other outcomes became evident through stakeholder consultations and were included in the program logic as outcomes but were not considered to be material for the SROI analysis. These were:
- Decreased number of call-outs to investigate minor crimes
- Decreased number of young people charged by police
- Police receive more respect
- More reporting to police
- Increased awareness of issues in the community
"[There is a] really good relationship between police and young people now. Police can help young people out of abuse (e.g. young person was a target of paedophile and got them out), more reporting to police, police more aware of issues in the community, able to act on the information"
Youth Liaison Officer, Burnie Police
When the expected quantity of change for each of these outcomes was considered it was found to be low, which indicates that the outcomes are not significant for the purposes of the SROI analysis. On their own, these outcomes are not material, however, they are important indicators that other more significant changes are likely to occur.
4.4 The program logic that emerged from stakeholder consultations
The program logic (or theory of change) tells the story of change that takes place as a result of Youth Prevention and Diversion program. The program logic includes information on:
- The issue that the Youth Prevention and Diversion program is seeking to address
- The key participants in the Youth Prevention and Diversion program
- The activities involved in the program
- The outcomes of activities that occur through the program, for various stakeholders
- The overall impact of these outcomes.
The program logic that emerged from the stakeholder consultations was that the inputs of the program (monetary and non-monetary investment) will be collectively used to deliver the program activities. The combination of activities and the frequency of activities will be tailored to the needs of the young person. As a result of the activities, young people are expected to experience four material outcomes (described in section 4.3). These outcomes are anticipated to occur concurrently and to reinforce each other. For example, a young person who experiences increased self-esteem will increase their engagement in meaningful activity, which in turn increases their self-esteem.
The changes experienced by young people are expected to lead to outcomes for their families and significant others, the community mentors that are involved in the program and the justice system. The overall impact of these outcomes is anticipated to be that young people take a positive pathway to adulthood by becoming healthier adults who are able to take responsibility for their own lives; living with more purpose; and are more grounded. Meanwhile, the community is expected to benefit through less burden on the justice system, healthier families and increased community cohesion and safety.
Development of the program logic
The first iteration of the program logic was developed with staff from the Department. Following this, substantial changes to the program logic took place during the initial stakeholder engagement. In particular, the outcomes experienced by young people emerged from consultations with them. In consultations with young people we heard about the importance of what might be seen as relatively small changes that have resulted from the program, such as "not reacting to people as much", "going up and talking to strangers at school" and asking people for help when needed rather than avoiding doing things". As a result we restructured the program logic to add more foundational immediate outcomes.
Our conversations with the case manager and CEO of CHAC, and the local service delivery organisations, helped us to describe more precisely the activities that are delivered. We revised the program logic to take into account their input. We also tested the negative outcomes identified by the Department with the stakeholders in Smithton and found that these were either not occurring to any significant extent, or were not occurring as a result of the program.
The concept that young people move through stages of development was identified in discussions with the CHAC staff. This concept was tested with other local service providers and against the notes collected during interviews with young people and their families and significant.
The final iteration of the program logic is included in Figures 4.1, 4.2 and 4.3 on the following pages. This represents engagement of all stakeholders. The outcomes described in the program logic are directly related to the outcomes expected to be experienced by different stakeholders in the SROI analysis, which are described in Section 4.3. The blue banner at the top of Figure 4.2 shows the stages of development that young people progress through during the program. These stages are described in Section 5.1. The outcomes that are numbered are those that either occur at the end of the chain of change, and there was sufficient evidence available to quantify how many stakeholders experienced the outcome. These are known as the material outcomes (discussed further in Section 5.1).
This is a forecast analysis, therefore it captures the consequences that are expected to be realised in the future as a result of the investment made into the program during the forecast period. Since information about what will happen in the future is not currently available, the short-term and the long-term consequences of the program are either assumed to be similar to the consequences observed for the stakeholders who have been in the program in the past or are inferred through the logic that some of the long-term consequences for stakeholders will occur in the future. No negative or unintended outcomes were identified.
Figure 4.1 - Youth Prevention and Diversion summary program logic
Figure 4.2 - Youth Prevention & Diversion program logic - Outcomes for the young people
Figure 4.3 - Youth Prevention and Diversion program logic - Other stakeholders
