What the Star Rating System Tells Us, Part 1

In August of 2013, Michael Petrilli posted an article in the online magazine Education Next,entitled “The Problem with Proficiency”. In the post, he argued that proficiency levels are determined largely by demographics, and advocated for a system in which schools are judged largely on student academic growth. Petrilli wrote: “To be sure, proficiency rates should be reported publicly, and parents should be told whether their children are on track for college or a well-paying career. (That’s one of the great benefits of a high standard like the Common Core.) But using these rates to evaluate schools will end up mislabeling many as failures that might in fact be doing incredible work at helping their students make progress over time.”

In the same month, Chester E. Finn, Jr. responded with “Let’s Hear it for Proficiency”, in which he argued that proficiency has its rightful place as an indicator of readiness, especially for college, and should not be dropped: “Not, at least, so long as it matters greatly in the real world. Do you want the pilot of your plane to be proficient at take-offs and landings or simply to demonstrate improvement in those skills? (Do you want to fly on an airline that uses only “growth measures” when hiring pilots?) How about dining in restaurants that use only growth measures when selecting chefs? Having your chest cut open by thoracic surgeons who showed “gains” on their surgical boards but didn’t actually “pass” them?”

The Idaho Star Rating system was initiated by the Idaho State Department of Education in 2012. The first school ratings were published in the 2012-13 school year, combining growth and proficiency measures. Though many states have opted for an “A-F” rating system, Idaho’s SDE decided to use a rating system similar to that used for hotels – 5 is the highest rating a school can receive, and 1 is the lowest. The Star Rating system was implemented as part of Idaho’s NCLB waiver request , submitted in September of 2012.

Though the Star Rating system itself is complicated, suffice it to say that the points earned by schools in the system are based primarily on school performance on the ISAT; in fact, 100% of the ratings for elementary and middle/junior high schools are based on test scores. In 2014-15, the system will replace the ISAT with scores from the Smarter Balanced assessments which are being piloted statewide this year.

At elementary and middle/junior high levels, the distribution of Star rating points is as follows:

• ISAT Achievement – 25%
• ISAT Growth – 75%

This distribution allows schools that are achieving at a relatively low level to gain Star Rating points by demonstrating that students are “growing” from grade to grade at a high level. In other words, if the schools in a system are showing excellent growth, outcomes should be better over time for that system.

Members of the Star Rating Advisory Committee (established this year by the State Department of Education) recommended that a more balanced distribution of points between achievement and growth be implemented beginning in 2014-15, in order that schools with high achievement receive more credit for their performance. However, the basic structure of the elementary/junior high system will remain intact.

At the senior high school level, the system contains a few more variables. Here is the basic structure:

• ISAT Achievement – 20%
• ISAT Growth – 50%
• High School Graduation Rate – 15%
• SAT College Readiness – 7.5%
• Advanced Coursework – 7.5%

The ISAT was taken by 9th and 10th graders across Idaho, which meant it counted in the Star Rating system for freshmen and sophomores at 4-year high schools, and for just sophomores at 3-year high schools. In the current state testing plan, the SBAC will count for 9th, 10th, and 11th graders in 4-year high schools, for sophomores and juniors in 3-year high schools

The Star system at the high school level relies on growth for half of the rating. The result is that it provides a misleading view of high school performance when considered against measurement of progress on the Idaho State Board of Education goal:

“60% of young Idahoans, ages 25-34, will have a post-secondary degree or
certificate of value by 2020.”

In light of this goal, It would be helpful to know how Idaho schools are doing in preparing students for college. The high school Star Ratings don’t tell us that. If we use the SAT performance of Idaho juniors and cross-reference Star Ratings for Idaho high schools, this is what we find:



So, while Timberlake High School (Lakeland District) and Moscow High School are among the top high schools in the state at preparing students for college math, they are “4-star schools”, and American Falls High School and Marsing High School, with far fewer students prepared, are “5-star schools”.

This is caused primarily by the weighting of the factors in the Star Rating system. Grade-to-grade student growth counts for half of the Star Ratings for high schools. If the system were more outcome-based, that is, if it put more weight on preparation for college and career in Idaho high schools (as Finn recommends), we’d have a better picture of how we measure up in preparing students. In Part 2 of this post, we’ll examine an idea for a different weighting system that would be more reflective of outcomes and preparation.