BPS has a strong commitment to high-quality science instruction. The resources provided by BSI's Science in the Schoolyard program have helped BPS teachers actively engage students in 'doing science,' helping them develop good observation skills, make predictions, talk about science with each other, and connect science concepts to their everyday lives. With BSI, BPS science teachers have proudly led the way in using outdoor teaching to support rigorous science learning.

—Pam Pelletier, Senior Program Director,
Science Department, Boston Public Schools

In a new University of Chicago study, LaForce and Bancroft (2014) found that elementary science teachers in Boston Public Schools (BPS) are taking science outdoors across the district, benefiting students and science instruction overall. The study looked at the extent to which the Boston Schoolyard Initiative's Science in the Schoolyard (SSY) program is being implemented in BPS schools and the corresponding outcomes for student learning and science teaching in the district. It centered around four questions:

• To what extent are science teachers in BPS incorporating outdoor science lessons?
• How does outdoor science instruction affect student learning?
• How does outdoor science instruction affect science teaching in the district?
• How has the Science in the Schoolyard™ program affected science teaching in BPS?

In brief, the study found:

1. Of the 100 elementary teachers from 55 schools who participated in the study, 70% reported incorporating outdoor science lessons in their spring 2013 classes.
2. The benefits of outdoor lessons for students included: higher levels of interest in science; better science communication skills (including science vocabulary and student to student science talk); higher rates of identifying connections between lessons; and greater risk-taking, independence, and curiosity.
3. Teachers who took lessons outdoors were significantly more likely to engage in several specific practices identified by the science department as indicators of high quality science instruction. These include: supporting science talk in their classes; structuring lessons to allow students to work at their own level; and asking students to make observations in science.
4. Teachers who had participated in SSY professional development were significantly more likely to incorporate outdoor science lessons in their classes and to do so with greater frequency than teachers who had not had SSY training.
5. Teachers who had participated in SSY professional development also demonstrated a greater commitment to outdoor science and had greater confidence in their ability to take science outdoors. Of those with SSY training, 92% said their "science teaching skills had improved inside and outside of the classroom" as a result of the training.

Measuring the impact of a program like SSY in a large urban district is challenging. "In large-scale innovations, particularly in districts with a lot of other challenges, it's often difficult for programs to show strong and direct effects," Melanie LaForce, lead evaluator explained. "I think it's particularly difficult to find effects with programs that are implemented diversely across a district where there are also many other variables (different student populations, economic variability, etc.) that come into play. So, considering the rigor of our methods and the nature of the SSY program, I think it's very encouraging to see some effects."

About the Survey

To evaluate the impact of SSY, researchers looked at science instruction across BPS not just at the SSY program. They made every attempt to ensure that the collected data represented the full spectrum of science teachers in the district. The final data represent science specialists and classroom teachers, teachers with and without SSY training, and classroom observations in classes, which did and did not include outdoor lessons in nearly equal numbers.

All 200 K–5 science teachers in the district were surveyed. Of the 97 who responded, approximately half were science specialists (47%) and half were classroom teachers serving as their students "primary science teacher" (51%). Survey respondents included teachers who had had no SSY training (41%) as well as teachers who had taken SSY courses (59%).

Of the eight schools that were randomly selected for data collection, none declined to participate. At these schools, evaluators collected qualitative data through in-depth interviews with selected teachers and the school principal, teacher and student focus groups, and classroom observations. Six of the eight schools had an outdoor classroom, and five had a science specialist who had participated in SSY training. Seven of the 15 classroom lessons observed included an outdoor component.

One thousand students (grades 3–5) were surveyed resulting in 800 useable surveys, half of which were from schools with an outdoor classroom and/or SSY trained science teacher and half of which had neither.

To overcome social desirability and confirmation biases, LaForce, lead evaluator, said, "we worked hard to create instruments where teachers didn't just answer what they thought they should answer, or answer questions that they knew were linked to their experience with the SSY program." For example, using the first science class they taught each week, teachers were asked to respond to a wide array of questions about the teaching practices they use. In separate sections about classes held indoors and outdoors, teachers were asked questions such as the following: "How often did you ask students to discuss lesson content and activities with each other?" and "How often did you ask students to articulate what they are doing in a science investigation and why they are doing it?" In other sections of the survey, teachers were asked to describe the behaviors their students (in the first class of the week) had engaged in, with reference to both indoor and outdoor classes. For example,

• How many of your students discussed lesson content and activities with each other?
• How many of your students articulated what they were doing in a science investigation and why they were doing it?

Extent of Outdoor Science Teaching in BPS

Of the teachers who reported incorporating outdoor science lessons in the spring of 2013, 83% reported going out multiple times. Teachers reported using outdoor lessons to support all 18 elementary science units, in schools across the city, independent of number of years teaching. Correspondingly, of the 800 students surveyed in grades 3–5, only 26% said they never go outside for science.

Increased Interest in Science

Students who had science lessons outdoors showed significantly higher interest in science than those who didn't have outdoor lessons. They were significantly more likely to agree that learning science was fun, and that they wanted to learn more science; and were significantly more likely to report that they do science activities at home "even when I don't have to." They were also more likely to agree that they could do well in science even if a new science topic was hard.

In addition, the study found, outdoor lessons may help students become independent learners. Teachers who take science lessons outdoors were significantly more likely than teachers who didn't go out, to report that their students ask questions stemming from their own curiosity. The more often teachers went out for science, the higher the number of students they reported asking such questions.

Teachers who took science lessons outdoors more frequently also reported higher numbers of students connecting concepts from one lesson to another.

Increased Science Talk

Students who reported having outdoor science lessons also reported significantly higher frequencies of using science words—both with peers and their teacher—and ranked their own listening skills during science significantly higher than students who didn't go out. Students reported feeling that they were freer to talk to each other outdoors, "I like doing science outside because you're allowed to talk about your experiments to your partners."

Teachers concurred, reporting that when they are outdoors, students talk more to each other about science, and use science vocabulary more. One teacher explained:

My third grade class, they're learning about the water cycle and it's hard for them, it's abstract concepts, really technical language [...] and getting them to use the vocabulary was hard, you know, precipitation, condensation, evaporation—they all sound very similar. But I took them to the outdoor classroom recently and we looked for signs of the water cycle outside so they could see exactly where it happens and would happen. [...] I'm noticing them using the vocabulary correctly and in context since we went outside.

Impact of Outdoor Instruction on Science Teaching in General

The BPS Science Department has developed guidelines for the classroom practices that serve as indicators of high quality science instruction. Teachers who took students outdoors were significantly more likely to engage in three of the teaching practices identified in the guidelines compared to teachers who didn't.

• Structuring activities to allow students to work at their own level.
• Promoting science talk (for example, 50% of teachers taking science outdoors said they "always revisit vocabulary learned in a previous lesson", compared to 29% for teachers who didn't teach outdoors).
• Asking students to make observations in science.

Students who reported going outside for science also reported conducting investigations at higher rates and were 10% more likely than students who didn't go out, to say they "always" make observations in science.

The Role of SSY

The study highlights the importance of professional development in general and the Science in the Schoolyard program in particular. Teachers who had participated in SSY trainings were significantly more likely than teachers who hadn't to take students outdoors for science, even when controlling for years of teaching, presence of an outdoor classroom, number and length of science classes, other outdoor PD, and fidelity of kit use.

Teachers with SSY training were more confident in their ability to teach outdoors. For instance, they were significantly more likely to agree that they had the content knowledge to implement outdoor lessons, and the ability to handle the unpredictability of teaching outdoors. They were also more committed to the goals of outdoor teaching, and more likely to report that outdoor teaching supported English language learners and students with special needs. Ninety two percent of teachers rating the SSY trainings agreed that their "science teaching skills had improved both inside and outside of the classroom" as a result of SSY.

Throughout SSY's history all professional development has been designed by BPS science teacher leaders in conjunction with BSI staff. These teacher leaders, continue to provide leadership and training through the BPS Science Department. They bring a deep understanding of how outdoor instruction increases student interest in science, improves science proficiency, and leads to high-quality science instruction. Their understanding and skill will grow and develop (along with teachers in schools across the district) as they continue to build on the foundation laid by BSI. To learn more about BSI, or to read the evaluation in full, visit www.schoolyards.org.

Reference

• LaForce, M., & Bancroft, L. (2014). Science in the Schoolyard Evaluation. Outlier Research & Evaluation, CEMSE. The University of Chicago.