2017 ADVANCED PLACEMENT UPDATE:
WHICH HIGH SCHOOLS PROVIDE 
ACCESS TO RIGOR?

Each year, we update the data from the College Board's Advanced Placement Program for the State of Idaho and for school districts and high schools. Advanced Placement coursework is the "gold standard" for rigor in high schools around the country.  Advanced Placement is by far the most popular program, and offers an unmatched set of challenging courses from which students can choose. 

Statewide AP Data

Here's the percentage of statewide AP exams given in districts around the state:

As you can see, Boise gave over a third of statewide exams, while enrolling about 9% of the state's students. In fact, Boise and West Ada (which has expanded its program recently) gave over half the exams in the state.

Insofar as student participation is concerned, here is the same comparison:

Again, Boise students took almost 30% of the statewide exams, with 9% of the students.


High School AP Exams and Senior Enrollment

However, a better way to judge whether students have access to and take advantage of rigorous coursework is to look at AP exams in comparison to enrollment in the district or for a particular high school. Let's have a look at the high school data.

The Washington Post's Challenge Index looks at data submitted by high schools around the country, and uses a ratio system to rank their efforts with rigor. It's a pretty simple comparison - the number of AP or IB exams given divided by the number of senior class graduates - if the ratio is higher than 1.0, the schools makes the list. For our comparison, we used the senior class enrollment for fall, 2016 (State Department of Education data) and AP exams (State Board of Education data).

Timberline and Boise were basically in a dead heat in 2017 in this comparison - Timberline's student participation grew significantly from 2016. Next was McCall, which has a thriving small-district AP program, followed by Capital and Wood River (Blaine County).

Ridgevue is the Vallivue District's new high school (pretty amazing participation for a first-year school) Borah and Century (Pocatello) have long been fixtures on the Washington Post list, and are followed closely by Centennial (West Ada). Especially interesting are Vallivue, Caldwell  and Columbia (Nampa), three high schools with very high rates of free/reduced lunch, which nonetheless had relatively high exam numbers.

One special note - Coeur d'Alene Charter Academy is a charter school in northern Idaho, located within the boundaries of the Coeur d'Alene District. The school is not a comprehensive high school in our definition - percentages of free/reduced and Special Education students are quite low, and student enrollment numbers decline significantly as respective cohorts progress through the grades, so we have not included them in the charts. Nonetheless, it's important to note that CDA charter would be near the top in these charts if it was included.

Student Participation

We've struggled with how to characterize AP student participation. At one point, we used the total number of students divided by enrollment of juniors and seniors, but more and more sophomores are taking the exams. We thought about the four-year high school enrollment (used by the Idaho High School Activities Association for classification purposes), but very few 9th graders take AP exams in Idaho. Eventually, we settled on AP student participation divided by the number of enrolled sophs, juniors and seniors.

Again, Boise and Timberline have nearly 50% of students taking exams, followed by McCall-Donnelly at 39%. Then, within a couple of percentage points are Wood River, Century, Borah and Capital with about 1/3 of students participating. Centennial and Ridgevue are next, with another 11 schools between 21% and 15%. 

At one time, Advanced Placement coursework was reserved for the small number of students who met a "cutoff score" on an assessment, as identified by districts or schools. With the adoption of the AVID program and a new philosophy of encouragement for students wanting tot take the AP challenge, the percentage of students taking at least one exam has changed. Using our new standard of  exam participants divided by 10th-12th enrollment, here's what we have seen in Boise.

A significant change, to be sure. Many more Boise students are taking advantage of the opportunity to experience rigorous coursework and take challenging, college-level assessments. 

Next, a look behind the Advanced Placement exam numbers in the Boise District.

IRI READING READINESS SCORES 
ON 5-YEAR SLIDE

The Fall Kindergarten administration of the Idaho Reading Indicator is often used as a gauge for the reading preparation of  Idaho's youngsters as they enter school. The assessment, which measures letter identification and letter-sound recognition, is a simple screener administered individually to students  at the beginning of the school year.

For the past five years, the percentage of students identified by the Fall kindergarten IRI as "ready to read" has decreased. Over that period of time, the percentage has declined from 56.1% to 51.4%. That may not seem like much of a drop, but it means over 1,700 fewer students entered kindergarten "ready to read" than did in 2012-13. The percentage of "economically disadvantaged" proficient kindergartners declined even more steeply, from 43% to 37.2%.

In fact, analysis of the same pattern for the 20 largest districts in the state shows the same trend - every one of the districts saw a decline in fall kindergarten proficiency percentages over the last five years.

These twenty districts enroll about 2/3 of the state's kindergartners each year. You can see that there is a substantial difference between the districts that have relatively high preparation levels - West Ada, Boise, Madison, and those with preparation levels that are really very low - Caldwell, Jerome, Blackfoot, Minidoka County.

However, most districts have at least a few elementary schools, where very few students arrive prepared for reading. In 2016, there were examples across the state of schools where few of the kindergartners are prepared- from Sacajawea (27%) to Lewis and Clark (18%) and Washington (16%) in Caldwell, to American Falls (18%), to Central (28%) in Nampa, Falls Valley (38%) in Bonneville, Erickson (22%) in Idaho Falls, Dworshak (28%) in Cassia County, Paul (24%) in Minidoka County, Jefferson and Koelsch (39%) in Boise, Centennial and Whitman (Lewiston) 39%, and many more.

It seems clear that the issue of kindergarten preparation is becoming more critical as each year passes. It's time to look at a pilot of Pre-K in Idaho schools which have the greatest need. If just 10-20 schools have the opportunity to implement Pre-K, it will give the state a chance to evaluate progress and see if a difference can be made for students in Idaho. Given our experience at Hawthorne and Whitney, we think it will.

WHY CONSIDERING STUDENT MOBILITY IS IMPORTANT FOR THE IRI (AND EVERY OTHER ACCOUNTABILITY MEASURE)

In 2002, the Idaho Comprehensive Literacy Plan law was modified in Senate Bill 1412 to include a provision that only students who were enrolled 90% of the schools day between fall and spring IRI testing were to be counted for purposes of goal attainment on the IRI. The change was made so that schools and teachers were not held accountable in a given year for kids who showed up a week before the test, for example. The new provision was in place until SB 1614 was put in place last year - the new law had no provision for considering mobility.

The State Department of Education recently provided data to school districts about student mobility, which has been very helpful. However,  the public site which has IRI data dating back to 2006 does not include any consideration of student mobility.

So why is this important? Let's take a look at the data for 2 Boise schools, using cohort IRI data.

The Assessment

First, it's important to understand the scoring of the IRI. The test is administered on an individual basis in fall and spring of a given year by trained administrators (often former teachers), and takes between 15 and 25 minutes for most students. The results are scored on a 3 point scale.  A score of "3" is considered "proficient", "2" is basic, and "1" is below basic. The IRI does not measure "above grade level" skill in reading.

The kindergarten assessment is meant to measure growth in letter naming and letter sound fluency. In first grade, letter - sound fluency is measured in the fall along with fluent reading on a basic passage. Beginning in the spring of first grade, each assessment measures reading fluency - fluent reading rate on increasingly complex passages.

Mobility Case Studies

For these studies, we looked at 2013 fall kindergarten and 2017 spring 3rd grade data, and analyzed the changes in student population and IRI performance.

Horizon Elementary

Horizon Elementary is a large elementary school on the Boise bench which has high mobility and about 66% free and reduced lunch. Horizon was built in 1992 to relieve overcrowding in Boise's West End.

In the fall of 2013, Horizon had 115 kindergartners, each of whom took the Idaho Reading Indicator. Of that total group, 50% scored at "grade level" ( a score of "3"). The statewide fall kindergarten "at grade level" percentage was 54.5%.

We wanted to know how many of those kindergartners were still around in the spring of 3rd grade, and how they did on the IRI.

In the fall of 2013, 115 kindergartners took the Idaho Reading Indicator at Horizon. Half of those students were ready to read, according to the IRI, 5% fewer than in the state as a whole. Of those students, 49 (43%) were still at Horizon in the spring of 3rd grade. Of those 49, 81% were at grade level on the spring 3rd grade IRI, 6% better than the state average. 66 of the original 115 (57%) were no longer enrolled at Horizon.

Of the 66 students who moved, 23 did so within the Boise District. 36 moved within the state of Idaho (16 to West Ada), and 7 moved out of state.

In the spring of 2017, there were 49 third graders who had moved in since kindergarten. 16 had moved from another Boise school, 22 came from within the state (14 from West Ada), and 11 came from outside the state of Idaho. One student arrived during kindergarten, 7 arrived during 1st grade, 14 during 2nd grade, and 27 during third grade.  67% of the "new" students scored at grade level on the IRI at the end of third grade.

Roosevelt Elementary

Roosevelt is an elementary school of about 300 students in northeast Boise just off of Warm Springs Avenue. Roosevelt had a free/reduced lunch percentage of 18% in May, 2017. The school was originally built in 1920 and received a complete renovation in 2010.

In fall, 2013, 40 kindergartners took the IRI at Roosevelt. 92% of those students scored at grade level on the assessment. In spring of 3rd grade, 27 of those students (68%) were still enrolled at Roosevelt. 96% of those remaining scored a "3" on the spring 3rd grade IRI.

13 students (32%) transferred after the kindergarten assessment. Of those students, 8 changed schools within the District, 2 within the state, and 3 moved to homeschooling or to a charter.

Twenty-one (21) students came to Roosevelt during or after kindergarten. Of those students, 7 came from within the District, 5 came from in-state (4 from West Ada), and 9 came from outside the state (4 from California).  Two came during kindergarten, 7 in first grade, 6 in second grade, and 3 in the third grade. 91% of these students scored at "grade level on the IRI in spring of 3rd grade.

The Point of All This

We live in a tremendously mobile society. Teachers are faced with the enormous task of dealing with constant mobility among their students. At Horizon, for example, a "cohort" of students may actually feature more than 50% mobility from grades k-3. Sp over half the students in kindergarten will transfer during that period of time, and will be replaced by other students moving into the school. Some students may attend 4 or 5 schools during that time.

Apart from the challenges of socialization for so many students, starting and stopping at a number of schools often affects achievement among those kids. But since we have no control over the movement of families and students, the least we can do is to consider the achievement of students who are non-mobile. At Horizon, for example, 81% of non-mobile students were at grade level in spring of 3rd grade - in kindergarten, the percentage was 50 - truly remarkable growth. For Roosevelt, the percentage of non-mobile students who were at grade level at the end of 3rd grade was 96% - up four percent from kindergarten.

The fact that consideration of mobility is no longer a part of the IRI legislation is truly unfortunate. The old law required that it be considered each year. It's really not fair to hold teachers and schools accountable for the achievement of students who moved in just before the test, whether they achieved at grade level or below. If they continue in the school, then it makes sense - this kind of consideration of mobility is part of the federal testing requirement for ESSA - it should be for state assessments, as well.

IDAHO READING INDICATOR

GROWTH IN COHORT PERCENTAGE OF 
"GRADE LEVEL" READERS

The Idaho Reading Indicator has been around for a while. The K-3 screener originated after a 1997 Reading Study Committee's findings led to the passage of the Idaho Comprehensive Literacy Act in 1999. The Literacy Act mandated training for pre-service and practicing teachers, provided funding for remediation, and established a reading assessment to be given twice a year to all students in grades k, 1, 2, and 3.

In 2001, legislation sponsored by senator Darrel Diede (R-Caldwell) passed the legislature and was signed by the Governor. SB 1116 set goals for the percentage of students, including that 85% of 3rd graders would read at grade level as determined by the IRI by 2006.

The "New" Law

In 2016, a new bill was passed, and became Idaho Code 33-1614. The law contained this language:

"Each school district shall report to the state department of education by October 1 of each year. The report shall contain the following information on the prior school year:

(a)  By grade, the number and percentage of all students in grades K-3 performing at the basic or below basic level on local and statewide assessments in reading; and

(b)  By grade, the number and percentage of all students in grades K-3 performing at the proficient or higher level on local and statewide assessments in reading."

Idaho Board Rule 08.02.01.801 set goals for schools based on the law, and specified that "year over year" comparisons be made:

"Statewide Trajectory Growth Targets

Statewide trajectory annual growth targets are based on aggregated student performance on the spring administration of the statewide reading assessments. Local growth targets are set by the LEA based on the LEA’s available resources and student demographics. Statewide trajectory growth targets indicated the statewide goal for year over year increases in the percentage of students reading at grade".

The comparisons, then, are of the combined percentage of students reading at or below grade level for kindergarten, first grade, second grade, and third grade in a given year for schools, districts, and for the state. So four different groups of students are combined and compared in a "year over year" fashion.

Recently, Idaho Education News, in accordance with the new law and rule, ran an article portraying the overall "percent at grade level" for districts on last spring's IRI testing and compared it with the percentage from 2016. Unfortunately, the overall percentage of K-3 "at grade level" students does not accurately reflect growth on the IRI, and is an inappropriate measure of progress.

Why? Well, because fall and spring percentages of "on grade level" reading on the IRI vary widely, and are always lower after kids come back from the summer break and take the fall test.  Presumably the fall measure reflects expectations for the next grade level, but the drops in percentages are also presumably due to "summer loss" among the student population, and especially among students who have less access to literacy-rich environments during the summer.

To illustrate, here is a "cohort" chart of the percentage of students at "grade level" on the IRI for the 2017 3rd grade class in 5 urban districts and in the state of Idaho.

The marker for the state of Idaho is the green circle. Note that about 55% of the state's kindergartners scored a "3" on the IRI in the fall of 2013, and 79% were at that level the next spring. But when they came back for first grade? Only 62% were at grade level. If you follow the cohort you will see the peaks during the school year and valleys into fall testing, and that  to the end of 3rd grade (far right) you'll see that 75% of the cohort students were at grade level - 21% higher than in the fall of kindergarten. 

Large District Cohort Performance

The next chart shows the cohort growth percentage for the 3rd grade classes of 2016 and 2017 for the 20 largest school districts in the state.

The data in the chart are sorted by the 2013 fall kindergarten percentage of "grade level" scores on the IRI, from highest to lowest. Note that:

1. As a general rule, districts that had higher percentages of cohort  "grade level" scores in fall of kindergarten have lower growth percentages. This makes sense, in that there are fewer kids who are not reading at grade level, so it's more difficult to make substantial progress. So, for example, Caldwell's fall 2013 K "grade level" percentage was 30% (!), while West Ada's was 70% - much more room to grow in Caldwell (though their growth percentage of 31% is amazing).
2. Districts up north (Lakeland, Post Falls, Coeur d'Alene) have few Limited English students. Limited English students, who are learning a new language and  learning to read - often progress much more slowly.

It makes far more sense, then, to look at cohort growth from fall  of kindergarten to spring of 3rd grade on the IRI when evaluating whether districts are making progress, while considering the above factors, than to calculate the overall percentages for k-3 and compare "year over year" performance. 

Rural District Performance

The Ed News article was titled "Reading Scores Suggest a Widening Urban Rural Gap". When we look at the scores of cohort groups, is that the case?

Well, not really. The same pattern exists here - as a general rule, districts that start with a percentage of "grade level" Kindergartners make less growth than districts that start out lower. 

The Mobility Effect

Even though the cohort method of evaluating IRI progress is better than the "year over year" method prescribed by the new reading law, it suffers from a shortcoming that all assessments bear - family mobility in our society is quite high, and it disproportionally affects achievement in schools with a high percentage of low income students . It's also not accounted for in IRI calculations.

We'll have a look at that phenomenon in the next post.