Calgary AfterSchool Evaluation - page 5

Guyn Cooper Research Associates
Calgary AfterSchool Evaluation
Summary Report
5
With a view to determining whether particular types or aspects of programs were associated with
particular developmental outcomes, a brief program quality assessment instrument was administered
to each program semi-annually. The program quality assessment instrument was derived from the Youth
Program Quality Assessment (YPQA) developed by the High/Scope Educational Research Foundation
specifically for youth in grades 4 to 12. These included questions derived from the YPQA on safety,
accessibility, staff qualifications, and elements of programming, along with questions to assess program
intentionality.
As it was such a large and complex outcome measurement initiative with many diverse programs,
thousands of participants, 22 different survey instruments, and a large number of staff, data collection
was an ongoing challenge. Ultimately, a complete data set, consisting of registration data, pre-test data,
and post-test data, was available for about 20 per cent of program registrations, with percentages varying
among the different sets of developmental indicators. The participants for whom a complete data set
was available comprise the study groups. Although the total study group of 4,237 individuals is not a
random sample of all participants, it is demographically representative of the whole CAS population. In
addition, the pre-test scores of the study group did not differ significantly from the pre-test scores of the
other participants who completed a pre-test but not a post-test. Therefore, it is a reasonable assumption
to generalize the results to the entire participant population.
A few programs amassed sufficient data over four years to analyze. In addition, issues with the collection
and entry of questionnaire completion dates meant that it was not possible to confidently calculate
duration of participation (i.e., program “dose”). Continuous work was done to improve data quality,
including the implementation in fall 2012 of a newdatabase systemat TheCity of Calgary that incorporated
extensive data checking criteria at time of entry, meaning that this problem should not arise in the future.
5. Summary of Evaluation Findings
Evaluation Question 1: Did CAS increase the participation of children and youth in high-quality
after-school programming?
CAS succeeded in increasing the participation of both children (in elementary school) and youth (in
junior and senior high school) in after-school programming. Increases in participation were consistent
among boys and girls, diverse population groups, and immigrant children and youth, including those
who were new to Canada. CAS had the greatest impact on children who had not been involved after-
school programming before CAS: 65 per cent of these children and 58 per cent of these youth increased
their participation in after-school programming. CAS clearly provided opportunities and benefits to
young people who would not otherwise have had them.
In addition, CAS significantly increased the amount of time spent in constructive activities and reduced
the amount of time spent in less constructive activities by youth who had not been using their time
constructively (CD=0.34). Because of CAS, youth who had been spending a lot of time at home alone or
a,b,1,2,3,4 6,7,8,9,10
Powered by FlippingBook