Do you have an achievement gap or an instructional problem?

For years, it looked like our school had a really big achievement gap based on race. By third grade, the gap was huge. Our students of color were way more likely to be in our bottom two tiers. But as I dug into the data, I found a really different story.

This is how I learned that an instructional problem can appear like ONLY an achievement gap problem and hide a more pervasive problem. 

I am an IB coordinator at my school and have been asked to run our reading RTI program in my spare time. (When I'm not teaching Spanish, doing duty, managing our IB program etc.)

My second year in the job, I noticed that our kindergarten students were not making a a half years of growth in reading. This was not difficult to figure out. If you do some sort of a screener (our school uses DIBELS), you can look at the percentile change from fall to winter to spring. 

Imagine you take your child to the Dr. She does a physical and measures her height. Your child is exactly at the 50th percentile. 

• If year later, they are still at the 50th percentile a year later: they grew an average amount over the year. 
• If they are now at the 90th percentile - they had a growth spurt. 
• If they are now at the 10th percentile, you might want to check in and make sure everything is ok. 

This is how I use our school's DIBELS data. I look at changes in percentile for every single child on the composite score.  

Our average kid starts kindergarten with really fantastic scores - around the 80th percentile. Our teachers basically have all league players, with a few players with weak skills mixed in. Just like our students, our teachers are generally really strong in terms of classroom management, building relationships with students and families and being well-liked by their peers. 

But, year in, year out, the scores were dropping. Our students would come in around the 80th percentile, and by the end of the year, be around the 40th percentile. 

This drop was across the board. 

But it appeared like an achievement gap program. This is why. Many of our black and brown students came in with scores around the 50th percentile. After a drop, they were now tier 2 and tier 3. Most of our white students entered around the 90th percentile. After a 40 percentile drop, they were still tier 1. When we just looked at our tiers, and not the changes in percentile, it looked like we had an achievement gap problem centered on race. 

When we looked at percentile changes, it was clear we had an instructional problem that was impacting everyone. Our children of color were the canaries in the mine, thanks to where their scores were located entering kindergarten. (This is of course due to systemic racism and a million other factors.) Our average student was moving from the 80th percentile to the 40th percentile in one year of school as measured by DIBELS.

Why is it a big deal that we realized it was an instructional program and not a race- specific achievement gap problem? 

• It changed the way we approached the problem.  

What did we do? 

• I looked at the data and realized we had a problem. After looking at changes by race, I saw that the problem was impacting all children. Our kids of color were more impacted, but the trend affected everyone. 
• We talked with the district literacy coach and received very little guidance. 
• The principal offered to let me buy around $300 worth of materials, but would not attend the meetings with the team to discuss what I was seeing in the data. 
• I talked to other IB coordinators at other schools. I talked with colleagues at different schools to see if they agreed that this was a serious problem. 
• Through this we learned that many schools had the same drops, especially schools where the kids were entering high overall. 
• We also learned, there were plenty of schools where students were not dropping.
• I talked with our teachers and looked at the data and we didn't get very far. 
• We tried a variety of interventions over multiple years and our data didn't budge in any meaningful way. 
• We talked with teachers from schools where there scores were not dropping and even went and observed at those schools.
• I asked who the best kindergarten teachers were at these schools (where the scores were not dropping.) I listened to what the teachers said was making a difference. The one thing that kept on coming up was a phonemic awareness program called Heggerty. 
• The teachers agreed to try 10-15 minutes of Heggerty phonemic awareness instruction a day. 
• The first year with help of a EA push-in we used the new Heggerty intervention the last 9 weeks of school. We dropped all of our old interventions. 
• The children's scores continued to drop in percentiles (even though the raw score was increasing). The average drop was not quite as bad as in previous years for the last term of school.  
• In classroom one with the 9 week intervention, we stopped the slide for many students, and some students started to return toward their baseline percentiles.  This was a sign that at least with the right modifications and skilled teaching, the program would work for our students. 
• Those students in first grade who received the intervention last year for only 9 weeks continue to make significant growth this year in reading. 
• The teacher in classroom two said that she felt the program had a lot of potential. 
• This year, the teachers have had no support with an EA due to scheduling. 

This year, our winter data shows the following. 

• In class one, the average student went up 10 or more percentile points.  
• In class two, most of the students stayed at their current percentile rank or improved.  
• In class three, the students continued to drop. At first it looked like nothing had changed. 
• Many students in class three dropped 30 - 40 percentile points in 4 months of schooling. Again, their raw scores showed growth, but it was slower than average growth. 
• BUT, when we just looked at the phonemic awareness sub score, that score was better. It actually was the best it had ever been. So, there had been some nice growth, just not enough to show up on the overall composite score. 

I have observed repeatedly in all three classrooms and have seen the program implemented regularly. 

All three teachers made minor adjustments for their students, as anyone would do with any program they teach. None of the adjustments seemed major or significant in any way. It just seemed like the teachers were adapting the program to fit their style of teaching. I spent some time observing in all three classrooms.  

What happened with this data? 

• I celebrated with the team. 
• I asked teachers to share with each other the modifications that they were making and their thinking behind the modifications. 
• Teachers agreed to try some additional modifications with the program. Go amazing life-long learners! 

What happened on an administration level? 

• Our administration went after teacher three. Even though she had made gains, was working hard and doing her best. 
• It was the perfect example of how not to use data and how to erode relationships with staff.
• It meant that I needed to spend hours helping her and protecting her from administration instead of doing my job coaching. 
• Why can't our administration take the same stance with our teachers that it takes with at least some of our most challenging students? Why not show the same level of confidence and belief and support with our teachers? 

Update a Few months later: 

• In classroom three, now all but one student is at grade level for phonemic awareness. We put in more supports, talk with parents, taught families to support at home, got in some volunteers and  focused on ensuring no one was slipping through the cracks. 
• We are supporting that one child and family and discussing potential next steps for what that child needs to be successful in school. 
• In one of the teachers, the majority of students are above the 90th percentile. I know that COVID has now hit and we are homeschooling. Amazingly, we had almost everyone at end of year goals by March! 
• Teachers still led inquiry-based IB learning the majority of the time. This program was at most 20 minutes of the day. 

What are my conclusions? 

• We need to, as a school system, honor that our best teachers CHANGE programs for the better. 
• We need to listen to our best teachers and ask them HOW they are modifying programs, and not criticize them for "not teaching with fidelity." 
• We need to treat all teachers who are working hard like they are valuable members of the team. 

What should I read next?

Further thought experiments on your most difficult students.

Use the simple view of reading to improve instruction tomorrow
Free Classroom Management plan for Zoom or Google Meet
COVID-19 and moving past the Fearscape of my mind