Process Evaluation of The Nsw Safer Drivers Course: Final Report

Research output: Book/ReportCommissioned ReportOther

Abstract

In NSW, as elsewhere in Australia and internationally, provisional licensed drivers are over-represented in fatal and injury crashes. They are among the most vulnerable road users, particularly during their first months of unsupervised driving. Addressing the over-representation of young novice drivers in casualty crashes requires implementation of a range of good practice initiatives. The Graduated Licensing Scheme (GLS) has been a widely adopted approach for which evaluation has demonstrated significant road safety benefits (Senserrick & Whelan 2003). The Safer Driver Course (SDC) is identified within the NSW Road Safety Strategy 2012-2021 as a key program for improving the safety of young drivers. It was launched on 1 July 2013 as an optional accredited component under the Graduated Licensing Scheme to assist learner drivers to become safer drivers as they progress to driving solo as a provisional licence holder. The Course was developed by a Ministerial appointed board of experts and sits among multiple road safety education programs, licencing requirements and options that interact with the aim of increasing positive road safety outcomes for young drivers in NSW. The SDC is a combined theoretical and practical course designed for young learner drivers under 25 years of age who have completed at least 50 hours of the required 120 hours of supervised driving experience.Transport for NSW (TfNSW) is the SDC policy owner and responsible for: the design of course curriculum materials; course training materials based on the Course Curriculum Framework developed by the board; the business rules; the evaluation of the course, and related road safety policy and future curriculum updates. Through the Community Road Safety Fund, TfNSW provides Roads and Maritime Services (RMS) funding to deliver the Course. The governance responsibilities of TfNSW and RMS are set out in the Policy, program management and delivery framework for the NSW Safer Drivers Course.RMS is the SDC administrator and is responsible for managing the overall delivery of the Course in NSW, including provider management, curriculum management, quality assurance processes and administration including payment to providers. Details of these key functions and related responsibilities are set out in the Administrator Guide and Policy, program management and delivery framework for the NSW Safer Drivers Course.The SDC is delivered by external Providers who are selected through a public tender process managed by RMS. Providers engage Trainers (facilitators and coaches) who hold specified qualifications and experience to deliver the course modules. During the analysis period of the evaluation, TfNSW engaged a Training Provider to deliver a training program for facilitators and coaches, governed by a licensing agreement and training materials. Provider responsibilities are set out in the Accreditation Agreement and Provider Guide. Trainers are required to deliver modules 1 and 2 as per respective delivery guides.In order to ensure overall effective delivery of the course, the Centre for Road Safety (CRS) commissioned a process evaluation of the SDC in the first instance, to be followed by an outcome evaluation in the future.The specific evaluation questions for the Process Evaluation were:•To what extent is the SDC operating effectively? •What (if any) improvements are necessary to meet the course objectives?The overall aims of both process and future outcomes evaluations of the SDC are to:•Inform any necessary refinements of course components, delivery model and business rules for continuous improvement purposes;•Assess the extent to which the SDC has achieved its intended objectives of assisting young learner drivers to:oImprove their capabilities to drive safely by reducing and managing their road safety risks;oAcquire additional safe driving skills, knowledge and insights into their driving vulnerabilities to complement their on-road driving experience within the context of the log book driving requirement; andoTo identify improvements necessary to meet the course objectives. For the process evaluation, a cohort study approach with a mixed-method quantitative and qualitative design was employed. It included a series of surveys and interviews from learner drivers who participated in the SDC, their parents, a group of provisional (P1) drivers who had not undertaken the SDC, SDC providers, facilitators and coaches, and a group of key program stakeholders. Observations of SDC Modules 1 and 2 delivery were also undertaken. A total of 10 key stakeholder representatives (from TfNSW, RMS, Youthsafe, Aboriginal Affairs NSW, and the NSW Department of Education), 17 service providers, 65 facilitators, 90 coaches, 1,320 SDC participants (534 of these completing all 3 surveys administered), 426 parents of SDC participants and 118 non-SDC P1 drivers participated in the evaluation. Course participant and non-participant surveys were conducted between November 2015 and May 2017, while stakeholder, provider and trainer surveys and observations were conducted between March and May 2017.To inform the process evaluation questions, data collection was designed to gather information on the following key aspects of the course: •Development and implementation•Marketing and promotion•Management and administration •Recruitment and training of facilitators and coaches•Course structure, content and delivery•Reach, access and completion•Impact on the licensing journey•Position of the program in young driver initiativesTO WHAT EXTENT IS THE SDC OPERATING EFFECTIVELY?The evaluation found the SDC was operating effectively in most aspects of management, course structure and delivery, and access and completion. Strengths in the operation of the SDC included course delivery documents, booking processes, course completion rates and high participant satisfaction ratings for most aspects of in-class and practical components. The evaluation also identified some issues that would benefit from refinement, including aspects of administration, training for facilitators and coaches, and some aspects of course content and delivery (discussed in the next section). While the SDC has achieved reasonable reach and access, findings suggest there is greater potential to reach young learners in regional and remote areas and from low socioeconomic and non-English speaking backgrounds. Some concerns were raised around the focus on the 20 credit hours incentive, including participants attending “just for the 20 hours” undermining the learning experience of others (with no minimum requirement for participation, engagement or demonstrated learning other than completing both modules), and the possible implications of allowing less supervised driving experience. An outcome evaluation is needed to test whether there is safety benefit for those that complete the course. WHAT IMPROVEMENTS ARE NECESSARY TO MEET THE COURSE OBJECTIVES?The findings indicate there is a need to refine aspects of the course components, delivery model, business rules, and administration to improve processes that contribute to achieving the course objectives.Key suggested improvements to consider regarding administrative issues include: improve communication strategies between RMS and course providers; increase flexibility of business rules for providers related to participant numbers, particularly in low demand areas; establish a fit-for-purpose Customer Relations Management System for Providers; revise pre-requisites, content, delivery and assessment of the ‘train-the-trainer’ course; and assist providers with ongoing support and development for facilitators and coaches.Key suggested improvements to consider regarding course content and delivery include: removing unnecessary information and repetition from Module 1, updating videos and statistics, and allowing for more opportunity for interaction and activities; refining lower performing Module 2 components and review the coaching guide; more explicit linking of Modules 1 and 2; and revising content of participant feedback forms and considering an online version.More generally, opportunities should be explored to: •Broaden course delivery in regional and remote areas.•Develop SDC-based resources for immigrants and those from non-English speaking backgrounds.•If the safety benefits of the SDC are proven in an Outcome Evaluation, consider making it a compulsory element of the GLS.•Require a minimum level of participation or engagement by participants as part of SDC completion.•Develop a defined QA and audit schedule.•Review the impact of the incentive of 20 credit hours on driver safety as part of the Outcome Evaluation.
Original languageEnglish
Place of PublicationMelbourne Vic Australia
PublisherMUARC
Commissioning bodyTransport for NSW (New South Wales)
Number of pages115
Publication statusPublished - Sep 2018

Keywords

  • Young drivers
  • Road safety
  • Training and development

Cite this