FOSSconnect


FOSS Assessment Corner: FOSSmap

Kathy Long, FOSS Assessment Coordinator, Lawrence Hall of Science
March 07, 2014 | Assessment

Research has shown that frequent formative assessment is a key to improved learning (Black and Wiliam, 1998; Wiliam, 2011). But developing prompts, analyzing test data, and developing next-step strategies can take a significant amount of time. Enter FOSSmap.

FOSSmap, the FOSS assessment system online computer program, provides an easy way for teachers to record evidence about student progress for each part of every investigation. Evidence can come from student notebooks. The science notebook is an important part of teaching FOSS Third Edition, and it is also an important authentic artifact for assessing students on a frequent basis. Suggestions for which notebook prompts to use to make students' thinking visible are in the Getting Ready section of each FOSS investigation part. The Guiding the Investigation section tells you where the assessment is embedded in each lesson and what to look for when you review student notebooks after class.

Why is frequent data so important? Dylan Wiliam provides a great analogy by comparing assessment to taking a trip on an airplane. When you get on the plane, you expect the pilot to take off and aim the plane towards the destination, but you also expect him/her to be checking the flight path frequently along the way. You would not want to be on a plane in which the pilot pointed the plane in the right direction, but then didn’t check the final destination until after the plane had completed the allotted flight time. Wind currents and other variables affect the path of the plane along the way. If the pilot didn't check frequently, you would most likely end up somewhere you didn't want to be. The same thing can happen in instruction.

The notes page for the FOSS Weather on Earth Module: Investigation 1, Part 1.

The notes page for the FOSS Weather on Earth Module: Investigation 1, Part 1./p>

FOSS has developed a formative assessment process called Reflective Assessment Practice. The idea is that you spend 10 minutes after each investigation part reviewing student work, as suggested in the Investigations Guide. (Ten minutes is what ASK Project teachers told us they could devote to science assessment on a frequent basis. Armed with this information, we conducted a small study as a part of the ASK Project to find out if 10 minutes of review could make a difference. We found out that it can, up to a 30% difference on the posttest. See "FOSS Assessment Corner," FOSS Newsletter, Fall 2013 for more information on the ASK Project.)

As you review students' work—reading an answer to a focus question, reviewing a response sheet, for example—you simultaneously go online to FOSSmap for that embedded assessment and enter reflections in the "add notes" page for that investigation part. You will find that all of the students are checked off as "got it!" The reason for this is time. You know that every keystroke taken to record that a student "got it" takes a few seconds. In our studies we found that students more often than not did get it, so to save time the system is set up so you only have to uncheck the box if a student didn't get it. Then you can write a short note to describe the problem as well as type a sticky note to print for that student.

Code Frequency Chart

Code Frequency Chart

When you have reviewed the work, you reflect on patterns of learning you noticed for the class in general and record those in a textbox below the student data. This is a great place to keep notes about what you did to help students move forward if they were having problems or to make notes about how you might do things differently next year. When you're finished recording data, notes, and reflections, you can print out the embedded assessment report, as well as sticky notes for selected students or all students. There is a default sticky note in the program for students who did get it. (You entered customized text to provide sticky-note feedback to help students who did not get it.) Reports are archived online for five years and provide an excellent resource to contemplate and improve instruction from year to year.

When it's time to take an I-Check, FOSSmap has been designed (Third Edition only) to allow students to take the assessments online. The program records students' answers as well as automatically codes students' answers that are responses from multiple-choice or multiple answer questions. FOSSmap also codes student responses to short answer questions. FOSSmap can interpret short answers that are spelled correctly, and those that are close approximations, but can't always give credit for all the ways students invent spelling. You'll want to spot-check short answer responses.

NOTE: In FOSS we code students' answers rather than score them or "give points." The purpose for this is to categorize the level of conceptual knowledge that students exhibit at that time. Look for more about coding vs. scoring in a future assessment corner.

Teachers must code the open response items, but if students took the assessments online, the responses are all displayed on one page with a coding guide to make it easy for the teacher to read them one right after another. If students take the benchmark assessments offline (using paper and pencil) you will need to enter their codes by hand into the FOSSmap system in order to generate reports. If you are a Second Edition FOSS user, you will need to enter codes by hand for all items.

Another of the beauties of FOSSmap is the array of reports that you print for your students, yourself, families, and administrators. Here's the sequence we recommend for looking at reports.

For the Teacher

Class by Item Assessment Report

Class by Item Assessment Report

After entering data for a benchmark assessment, print the Code Frequency Chart first. This may look a little intimidating, but it's actually very simple to interpret. You only need to pay attention to the colored bars. There is one colored bar for each item. That bar will be green, yellow, or red.

  • A green bar indicates that at least 70% of the students got the answer correct.
  • A yellow bar indicates that between 50 to 70% of students got the answer correct.
  • A red bar indicates that 50% or fewer of the students got the answer correct.
  • The red bars identify the problem areas you need to help students with their understanding.
  • The black bars indicate the percentage of students who received codes below the highest possible.

The Class By Item report provides the detail you need to determine what students need help with. The items on this report are ordered by concept and then by item difficulty. There is a narrative description for each code and each item that will tell you what each student knows or needs help with. This report can help you determine items that need some self-assessment reflection with students.

For Each Student

There is a Student by Item report available for each student after each benchmark assessment. This report shows correct answers, student answers, and a description of what this shows students know or need to work on. This is a report that parents love because it gives them information about what their child knows and needs to learn.

Student by Item Assessment Report

Student by Item Assessment Report

There are six reports in all, and you can go to FOSSmap.com to see them for yourself.

When you're ready to dive into FOSSmap, first, and foremost, take the time to go through the tutorials found on the FOSSmap.com home page. These tutorials explain in great detail how to use all aspects of FOSSmap. Although the program is quite intuitive and easy to use, spending a few minutes on the tutorials can save you time and frustration as you start using the program.

As always FOSS staff are here to help you if you run into any problems or have questions about using the program. You can email me directly at klong@berkeley.edu or you can email the FOSSmap programmers directly at the help email provided in FOSSmap itself. We hope everyone will give this new aspect of FOSS a try and send us feedback. All feedback is welcome. As with all new programs, there may be a few glitches along the way. The only way we can fix those in a timely manner is to hear from you! And of course, we want to hear your success stories as well!