On this page, I’m sharing my writing for PME 802, a course in Program Inquiry and Evaluation.

My Program Evaluation Design website

Step #1. Select and describe program context: 

Focus and Goals
The social program I’ve chosen to evaluate is the Concours d’art oratoire, run by the Canadian Parents for French (CPF). As a Core French teacher, this is a program that connects directly to my teaching context, providing opportunities for both me and my students to get involved. I moved to a new school a year and a half ago, and am now working in a school that also houses a French Immersion program. This provides me with a first-hand opportunity to observe aspects of French Immersion programming and its appeal to or impact on students.

The focus of the Concours d’art oratoire is aligned with the umbrella organization’s mandate of “furthering bilingualism [in Canada] by promoting and creating opportunities for students to learn and use French.” The Concours d’art oratoire is the largest annual French public speaking program in Canada for second language students in grades 6-12, in both French Immersion and Core French programs, as well as Francophone students. Although the Concours is a national program, it is structured to begin in local classrooms. The event is organized by parents and school staff, and requires a high number of volunteers, as close to 10,000 students across Canada participate each year. Volunteers also include university and college professors, teachers, native Francophones, and former Immersion students. Students who are successful at each level progress through school district, provincial and national level competitions. The stated aim of this program is “to stimulate the interest of students learning French, to improve their speaking skills, and to give them experience presenting in public.” I have been involved previously at the classroom and district levels, and my interest was piqued by the potential to evaluate this program in light of some questions I have (more on that below).

Size of staff
The Concours is a volunteer-run event, and depends on teachers to volunteer to begin the initial phases. Teachers serve as coaches, coordinators and judges within their own schools, and must follow the rules and evaluation rubric laid out on the CPF website. Once the school-based competition has determined the winners, those students progress to the school district level. In my district, this competition is held on the campus of a local university, where more volunteers coordinate the event. If a teacher is sending a student to a competition, he or she is requested to serve as a judge in order to help support the event.

Resources
The CPF website lists a number of resources for teachers to use at the classroom level to prepare for Concours, supporting students in selecting topics, preparing speeches, providing lessons connected to public speaking, as well as additional resources and evaluation criteria.  Not directly related to Concours, the CPF website also lists more general resources for youth, parents and educators, promoting second-language literacy as well as collaboration between educators and parents in support of youth.

Given that the Concours begins in schools, the instruction and facilities are integrated into programs of instruction and take place largely within the context of a school day. Once the higher levels of competition are reached, events take place at local schools and universities.

Community Demographics
I am choosing to look at this program through the lens of its use and impact in my school district. I work in Surrey, BC, which is home to the largest school district in the province. There are 125 schools district-wide, including elementary and secondary schools. There are currently almost 74,000 students registered in Surrey schools, and the district offers Core French, Intensive French, and French Immersion programs. The popularity of French Immersion programs in particular continues to increase province-wide. There is a local chapter of Canadian Parents for French who primarily communicate through their Facebook page, which is used to organize and promote local activities for students and parents. The Twitter account for CPF-BC is used in both English and French to promote stories of interest, news, and events.

Surrey is a diverse and multicultural district, with 195 languages other than English represented in our schools. Of those other languages, those with the highest percentage of representation within the student body are Punjabi, Tagalog, Mandarin, Hindi and Arabic. A speech and film contest for Punjabi-speaking students is in its third year of operation in Surrey schools, and is co-sponsored by OMNI TV and community organizations. Given the multicultural nature of our community, it is important to consider the impact of Concours d’art oratoire in schools as well as its appeal and impact on students from a wide range of cultural and linguistic backgrounds.

Step #2. Identify purpose for evaluation and specify evaluation questions: 

The purpose of my evaluation is to look at the impact of the stated goal of Concours on the learning of Core French students in my district. I have seen some increasing resistance to participation on the part of both students and teachers, and I would like to investigate the following questions:

  1. Given that it requires class time to run and spans several months of a school year, how well does Concours align with BC’s revised curriculum?
  2. Is Concours providing a meaningful opportunity for Surrey Core French students to learn and use French?
  3. What recent trends have emerged in Surrey School District Core French student participation in Concours?
  4. What do these trends tell us about the impact of Concours in our community? Does this impact tell us whether and how anything should be changed in the way we are inviting students to participate?

Sources

Canadian Parents for French

Canadian Parents for French – BC and Yukon Chapter

CPF Surrey Facebook page

Surrey School District website

2019/2020 FACT SHEET – Surrey Schools

Step # 3 Construct a program theory: 

In the table below, I have summarized a basic model of what the Concours d’art oratoire would look like at the school level, which is the level at which it will impact the greatest number of students.

ActivityIF the activity is provided, THEN what should be the result for participants? WHY do you believe the activity will lead to this result?WHAT evidence do you have that this activity will lead to this result (data from your own or other programs, published literature, etc.)?
Students participate in French language public speaking activities run by teachers in their school on topics of interest to the students. Teachers will use the resources listed here on the Canadian Parents for French website.If this activity is provided, then the result for students should be increased skill and confidence in public speaking in French.It is generally accepted that when people practice a skill, they will likely improve in their performance of that skill over time. With the support of a teacher and peers, as well as resources from the Canadian Parents for French (and other resources the teacher may have at their disposal) in the familiar setting of their own classroom, this improvement is more likely to occur. Speaking is a normal part of second language programs, and is incorporated in the BC Core French curriculum.
Students select topics of interest and write and memorize speeches on those topics, in French, with assistance from their teachers.By selecting topics of personal interest, students will be more engaged and motivated. In writing and memorizing their speeches, students will engage directly in the process of learning to deliver their message to their intended audience.Bandura’s social learning theory supports the interaction of the processes of attention, retention, motor reproduction and motivation as being at the heart of social learning, which incorporates both thinking and behaviour. Selecting a topic of personal importance will also lead to a more meaningful experience for student participants.
Students will enter a school-based French language public speaking competition to be judged by teachers and other adult volunteers, using criteria from the Concours d’art oratoire. Students will also respond in French to questions from the judges relating to the content of their speech.Students will be motivated to revise, memorize, and perform well at a school-based competition and will practice responding to questions on their speeches.Self-regulated learning, which is in part built from the theories developed by Albert Bandura, supports student learning, motivation and efficacy through repetition of the cycle of planning, monitoring performance, reflecting on performance, and then using the results to guide the next performance. This cycle is at the heart of instructional design for teachers, and supports student learning, motivation and performance in tasks of this nature.
The winners of the Concours at each grade level will go on to compete, using the same speech, at the district level.Students will be more confident after having won at the school level, and will be more motivated to improve their performances for the district level competition.As mentioned above, social learning theory and self-regulated learning will support student efforts at this level. In addition, the added motivation of winning at the next level increases the competition and motivation of students to do well.

Step #4: Identify, describe, and rationalize your evaluation approach:

Program Theory – rationale, description and approach

My goal with this program evaluation is to look at the impact of the Concours d’art oratoire on Core French classes in my school district. Although the Concours is a national program, I am looking specifically at my local context.

For this evaluation, I am using the Evaluation for Learning and Use model explained by Dr. Jennifer Green in this video. The program I am looking at has been in existence in Canada for more than 30 years, and I want to know if it is having its intended impact on students in my district.

The table above outlines the activities contained in basic model of the Concours d’art oratoire operating at the school level, which is where most students would be impacted by it. The activities are expressed mostly from a student perspective, although there is significant input required from teachers as well. The activities that teachers will engage in at the school level may or may not be integrated into their regular classroom instruction, which could potentially impact their ability or willingness to participate. The model (as laid out by the parent organization) presupposes that teachers will either volunteer time outside of class to run the program, or will find ways to integrate it into their classroom instruction. Regardless of which option teachers choose, the model also assumes that there will be a fit with programs and provincial curricula that will prepare students for public speaking and oral presentation on a topic of interest to them, rather than a topic integrated into a specific textbook or program. In my experience, these are reasonable expectations, which is also supported by the longevity of this program.

Because I am not looking at the setup of a new program, I am not looking at aspects of program setup that would involve looking for volunteers and finding facilities. This program has an existing infrastructure that supports those aspects. My purpose is to contribute to the quality of the program and its impact on students in my district. With this in mind I am also considering aspects of stakeholder theory as described by Chen, specifically the ones impacted by this program: the mission and philosophy of the organization, the budget and personnel restraints, the theoretical justification, and the base of evidence. As part of considering the impact of this program on my community, it will also be helpful to consider its relevance to the majority of students enrolled in Core French programs. This aspect is also incorporated by Dr. Green, when she recommends that evaluations of this type should examine how well the program under consideration addresses the priority needs of the diverse program participants.

Revised Questions

  1. Given that it requires class time to run and spans several months of a school year, how well does Concours align with BC’s revised curriculum? How well does it align with the ability and willingness of Surrey teachers to participate?
  2. Is Concours providing a meaningful opportunity for Surrey Core French students to learn and use French?
  3. What recent trends have emerged in Surrey School District Core French student participation in Concours?
    1. Are we seeing diversity in the students choosing to participate in this program?
    2. Are we seeing diversity in the students who are successful at the school or district level?
    3. Do the participation rates indicate that this program is reaching its intended audience?
  4. What do these trends tell us about the impact of Concours in our community? Does this impact tell us whether and how anything should be changed in the way we are inviting students to participate?

References:

  1. www.instructionaldesign.org/theories/social-learning/
  2. https://serc.carleton.edu/sage2yc/self_regulated/what.html
  3. Greene, Jennifer. “Approaches to Evaluation.” YouTube, Education at Illinois, 12 Apr. 2017, www.youtube.com/watch?v=zpwB_nSv6HM.
  4. Huey-Tsyh, C. (2005). Practical program evaluation: Assessing and improving planning, implementation, and effectiveness. Thousand Oaks, CA: Sage Publications