Calgary AfterSchool Evaluation - page 4

Calgary AfterSchool Evaluation
Summary Report
Guyn Cooper Research Associates
Participation in programs is voluntary, with a City-led recruitment campaign targeting children, youth,
and parents undertaken early in the fall each year, supplemented with an on-line program directory and
recruitment efforts by the individual programs. Examples of the types of available programs range from
access to City of Calgary Leisure Centres, to dance, cooking, and music instruction, to drop-in mixed
recreation programs, to leadership and other personal skills development programs. Programs provided
by non-profit organizations are required to have a developmental, rather than a recreational focus.
4. Evaluation Overview
The evaluation sought to answer two primary evaluation questions:
1. Did CAS increase the participation of children and youth in high-quality after-school programming?
2. Did participation in CAS programs contribute to improvements in participants’ friendships and
social skills, emotional well-being, and/or school engagement?
The sources of quantitative data for the evaluation were an intake/registration form completed for each
participant, child and youth activity questionnaires, youth “constructive use of time” questionnaire, nine
developmental outcome questionnaires for children, and 10 developmental outcome questionnaires for
youth. The outcome questionnaires were developed using questions from a range of instruments used
in other, large, high-quality research conducted by Statistics Canada (National Longitudinal Survey on
Children and Youth); Girl Scouts of the USA; Public/Private Ventures; theCenter for Research on Education,
Diversity & Excellence; and the National (U.S.) Outcome Workgroup – Children, among others. Most
of the instruments had also been used extensively in Calgary. It was not expected that each program
would, or even could, influence developmental outcomes in all of these areas. Therefore, programs were
directed to collect pre- and post-participation data on the specific developmental outcomes that they
expected their program to influence.
Programs were provided with and trained to administer survey instruments to collect all participant data
in the developmental domains in which they sought to have positive impacts. Programs were directed to
administer the appropriate developmental survey(s) along with the mandatory use of time survey at the
time of program registration (pre-test) and at the conclusion of the program (post-test). Questionnaires
were not administered to children in grade 1 to grade 3 as their reading proficiency was likely inadequate
to answer the questions without help from an adult, which was not likely to be available in the programs.
Data from all sources were entered by CAS program staff into a master on-line database developed and
owned by The City of Calgary. To protect participant confidentiality, participants were assigned a unique,
logarithmically-generated identification number. All data sources were linked using this number. The data
were provided to the evaluators and analyzed using SPSS (Statistical Package for the Social Sciences).
Non-parametric Wilcoxon tests were used to statistically assess change after program participation. To
estimate effect size, a statistical measure called Cohen’s D (CD) was calculated for the overall average
score for sets of indicators, and Cramer’s V (CV) was used for individual questions.
a,b,1,2,3 5,6,7,8,9,10
Powered by FlippingBook