Importantly, mātauranga informs tikanga; the processes and policies which guide our work. Māori youth development scholars (Keelan, 2003, 2014; Ware, 2009; Ware & Walsh-Tapiata, 2010) have complained about the invisibility of mātauranga Māori and youth voice in mainstream youth development literature, especially when “there is a mātauranga Māori of youth development and it does not sit in negative statistics” (Keelan, 2014, p. 7).

Of all the Māori values outlined in this arotake tuhinga, mātauranga most closely aligns with the YDSA Principle 6, “youth development needs good information”. It emphasises information derived from research and evaluation, but as discussed in the earlier sections of this arotake, the mātauranga used to inform youth development policy and practice in the past was substantially limited.

Effective youth development policy and practice must be informed by multidisciplinary, multicultural, multimethod and multiple stakeholder perspectives. In this section, we summarise what we have learned from youth development research reviews conducted prior to this one, we share our insights of the youth development mātauranga we have synthesised here, and we discuss the implications associated with privileging certain knowledge bases of youth development over others.

Other Research Reviews

Research reviews are useful in terms of synthesising current knowledge of a specific topic and the youth development sector has benefitted from several reviews produced since McLaren’s (2002) Building Strengths report was disseminated. Ball (2010) produced a literature review based on both national and international evidence of best practices for mental health promotion programmes targeting taiohi Māori. She argued that approaches aligning with good PYD practice with respect to boosting strengths and focusing on holistic development and wellbeing were more effective than those focused on reducing problematic behaviours, and that both universal and targeted programmes were needed. Ineffective approaches included solely information- based and fear-inducing interventions and unstructured activities.

MYD conducted a youth development programming review in 2009, again drawing predominantly on international evidence with some commentary on domestic initiatives to the relevancy of the structured programmes it supports. This review emphasised the problems associated with such a diffuse definition of “youth development”, arguing that “the current conceptualisation of what constitutes appropriate youth development activity is so broad that very few things could actually be considered inconsistent or out-of-scope” (p.7). This highlights the importance of being clear about the youth development principles that operationalise the work of our sector, and what sits outside of our youth development practice. MYD’s review provides a contrasting tone to Ball’s in that it suggests the huge popularity of youth development programmes far outweigh the evidence. With good quality implementation, such programmes typically produce modest positive effects. Raising the collective standard of practice is thus imperative to increasing the sector’s impact. Growing the Aotearoa New Zealand evidence base, upon which best practice guidelines should be based, forms a vital part of this work.

Two systematic review projects are also of note. Systematic reviews collect all available research on a topic and then synthesize the data to provide a large-scale assessment of the evidence. In this way, they report on the quality of the research as well as the content. Farruggia and colleagues conducted such a review on the effectiveness of youth mentoring programmes first reported on in 2010, then followed by related publications (Farruggia, Bullen, Davidson et al., 2011; Farruggia, Bullen, Soloman et al., 2011). Their systematic review included 26 evaluations of programme effects. A vast majority of programmes showed some degree of effectiveness. This review provided some assurances that youth mentoring in Aotearoa New Zealand was on the right track, but also highlighted some important gaps which needed to addressed (such as more robust evaluation procedures and increased funding to programmes to ensure evaluation takes place).

Their systematic review also explored the cultural context of mentoring. Many of the programmes included in that review work with significant numbers of Māori and Pasifika young people, and those programmes with a greater proportion of Māori and Pasifika youth were less culturally appropriate. Moreover, there was a negative association between appropriateness and effectiveness, as programmes higher in cultural appropriateness tended to be lower in effectiveness, and vice versa. The authors suggest that programmes need to balance established best practices with cultural appropriateness, to ensure young people are receiving the benefits of both. They also suggest that goal-orientation (e.g., interpersonal, academic, or cultural goals) can influence effectiveness and perhaps programmes with high numbers of Māori and Pasifika youth are targeting goals which are more difficult to change or measure. Finally, they posited that a mismatch with research methodologies may mean the effects of programmes with many Māori and Pasifika youth are not captured.

Also in 2010, a systematic review of impact of youth work on young people was published (Fouché, Elliott, Mundy-McPherson, Jordan, & Bingham, 2010). The review had specific parameters: evaluations of the effect of youth work interventions on young people aged 12–24. Unfortunately, the review itself was ‘empty’ as no studies met the inclusion criteria of the review. We note that their operationalisation of “youth work” for the purposes of the review was very narrow. The researchers only included studies where the authors explicitly labelled their work as “youth work”. This does not align with the broad definition offered in the Code of Ethics (Ara Taiohi, 2011) where most youth development programmes (paid or voluntary) involve relationships of a youth work nature. The authors noted that no evidence does not mean there is no effect; only that the type of evaluation targeted by their review was lacking. The authors concluded that while there are indicators in the field to show youth work works – such as young people’s participation in and positive feeling towards the services they are offered – the youth work sector would benefit from practice-based, high quality evidence which shows the effect youth work can have. Generating this evidence has challenges however, not least with respect to accessing funding for such endeavours.

Insights about Youth Development Research from our Synthesis

In this review, we have had the opportunity to peruse a swathe of local literature. In doing so, several trends have become evident and are worth discussion here as suggestions for ways to move the collective body of knowledge forward. Since 2002, there has been a plethora of research done by Aotearoa New Zealand researchers about young people in Aotearoa New Zealand. As indicated earlier in this review, although we have included a lot of this research, there is still much more that was not able to be included. Data has been compiled from a range of sources: peer-reviewed journal articles, books, postgraduate student theses, organisational reports, submissions and information sheets. The research ranges from conceptual and theoretical pieces to empirical studies using small case studies to large scale, national projects with thousands of young people. All manner of research methodologies based on kaupapa Māori, qualitative, quantitative, and mixed-methods frameworks have been employed and we have drawn on literature from the disciplines of psychology, sociology, anthropology, health and education. In short, Aotearoa New Zealand youth development research is innovative, diverse, and exciting.

Already in this review we have highlighted some of the large-scale projects. Of note, the Youth 2000 surveys present the most comprehensive picture of secondary school health and wellbeing that we have to date, and the robust representative design of these studies means that they provide the most accurate overview of the “big picture”. Not only this, but they provide rare insights into population- based experiences of marginalised secondary school students, without which these students would continue to be invisibilised. Likewise, Munford and Sanders’ work with young people living in chronic risk conditions provide unique insights about the lives of hard to reach young people. Continued support of projects such as these is essential if we are to understand the diverse experiences of young people across the country.

Although large-scale projects are often associated with quantitative analyses and generalised findings (such as AHRG), Munford and Sanders’ work includes mixed-methods and qualitative research being done with significant sample sizes (e.g., Munford & Sanders, 2015a; Sanders, Munford, & Boden, 2017). There are also many smaller projects utilising robust methodologies from kaupapa Māori, quantitative, qualitative, and mixed-methods paradigms. Some have argued that certain qualitative methodologies – such as interviews, focus groups, oral histories, and case studies – are most appropriate for doing research with Māori, as they more closely align with how Māori have traditionally passed knowledge on (Bullen et al., 2019; Walker, Eketone, & Gibbs, 2006) and seem to resonate more with Māori participants (Heke, 2005). Indeed, these approaches appear to be popular in the Aotearoa New Zealand youth development literature, emphasising the voices and subjective experiences of marginalised peoples.

There are numerous youth development programmes in Aotearoa New Zealand, working with thousands of young people across the country. A critical part of programme practice involves evaluating whether the programme is not only doing what is says it does (in other words, that it is effective), but also that it is safe. However, the systematic reviews led by both Farruggia and Fouché and their colleagues in 2010 point to the scarcity of youth programme evaluations available at that time. The limited scope of both reviews (on mentoring and narrowly defined youth work initiatives) may have contributed to the low numbers; it may also be that evaluation in the youth development landscape has changed very quickly in the last 10 years. Numerous reports have been produced in recent years and recent research demonstrates that there is a growing evidence-based movement in Aotearoa New Zealand, which is almost certainly a key driver of this increased activity (Bullen et al., 2019; Deane & Harré, 2016). Several evaluation studies have been described in this arotake, where specific outcomes were pertinent to issues in other sections, and we expect that there are many more that will contribute importantly to the evidence base on youth development programming in this country.

The increased evaluation activity in the sector is encouraging considering youth development organisations experience significant resource constraints (Martin, 2006) and are amongst many social and community services that acknowledge numerous barriers to engaging more deeply with evidence. This includes a need for internal evaluation capacity-building (Bullen et al., 2019). Investment in these activities is crucial because Farruggia et al. (2010) demonstrate that programmes with a history of evaluation and that align with best practice principles show greater levels of effectiveness.

Many of the evaluation studies included in this review, particularly those published in peer review journals, provided useful and transparent details regarding the theorised programme mechanisms driving change (i.e. aspects of the programme theory), the methods employed, and the limitations. These details are essential with respect to demonstrating evaluation and programme quality thus we encourage greater attention to these aspects in all evaluation reports. The NZYMN Guide to Effective and Safe Practice (McDonald et al., 2016) includes a section on evaluation which describes the benefits of evaluation, as well as the different types of evaluation. This is a useful resource for any youth development programme, as is the What Works website.

There is an abundance of postgraduate theses available in the ‘grey literature’. Clearly, many students are interested in the lives of young people in this country and they are exploring a multitude of dimensions of youth development. There are now open online repositories of theses completed by students at tertiary institutions, making most of this research available to the sector at large. Some thesis content is eventually published in peer- reviewed journals and we encourage more of this. Publication of this work in peer-reviewed outlets provides an extra level of quality assurance, and also forces the distillation of arguments and narratives into a more digestible length. We appreciate that publisher paywalls restrict access to this information for many segments of the youth development sector unless choosing Open Access options, some of which do attract high fees. Nevertheless, there are freely available Open Access options (e.g. Kōtuitui: New Zealand Journal of Social Sciences Online published by the Royal Society Te Apārangi) that provide good outlets for youth development research. We also encourage research dissemination in a wide range of other open access forms, such as presentations, resources, blogs and podcasts. Increased dissemination activity is necessary to advance the sector.

3 http://whatworks.org.nz/

The Politics of Youth Development Knowledge

We conclude by reiterating that an exclusive focus on psychological knowledge of youth development, which has largely privileged Western constructions of adolescence, wellbeing and development and originates from deficit theorising of norm deviant youth (Beals, 2008b; 2015; France, 2012), does not serve young people in Aotearoa New Zealand well. Certainly, ecological, cultural and critical psychological perspectives of youth development exist and these knowledge bases eschew biologically reductionist and individualistic views of human development. Nevertheless, it is the former approaches, undergirded by brain science (France, 2012) and a positivist, evidence-based movement (Bullen et al., 2019) that have gained the greatest currency in political spheres by convincingly offering methodological solutions that pinpoint causality, and hence “What Works” for effective youth development. Fostering positive youth development in an inclusive and culturally responsive way in a diverse country like Aotearoa New Zealand is not that simple and the political power of knowledge has real implications for young people. France (2012) argues that knowledge construction is also subject to neoliberal conditions of market competition. Consequently, this had led to power and resource imbalances whereby “hard” science secures greater gains from financial investment. This has the flow on effect of greater visibility in policy making, perpetuating a vicious cycle that pushes social science knowledge further and further to the periphery when it comes to influencing youth policy. The youth development sector will gain more from a collective knowledge base that builds a systemic, dynamic, and interdisciplinary picture of young people and their world. It is our hope that by incorporating diverse research in this arotake that we have taken a small step towards this goal.