TEACHER EVALUATIONS - THE SAGA CONTINUES

Recently we wrote about the dustup revolving around the McRel audit (or desk report) on a sample of 2014-15 educator evaluations, in which two elements that were not required in 2014-15 were used as part of the analysis, causing 99% of the evals to be judged as incomplete or inaccurate. 

Well, the resulting furor has caused confusion among legislators and State Board members, and unnecessarily made the long-term future of the Career Ladder a bit murky. It's also resulted in a huge imposition upon districts around the state, as the State Department of Education and the State Board of Education have now both requested educator evaluation information.

The SBOE has requested a sampling of 2015-16 evaluation documents for particular administrators, in order to do an audit of those evals. The SDE has requested information with respect to all evaluations done in 2015-16.

In this post we will focus on the SDE request, which once again includes elements not required for some 2015-16 evaluations. This time, the "student success indicators" and "professional learning plans" are listed as elements that school districts must report for all Pupil Personnel Certificate holders.

However, when the Career Ladder first became a law, "Pupil Services" personnel (such as counselors, school psychologists, nurses, speech and language pathologists, social workers, and audiologists) were not a part of it because suitable measures of student success as required by the law had not yet been developed. These pupil services personnel became a part of the Career Ladder in the current school year (2016-17).

When the data were submitted to the SDE for 2015-16 evals, districts were required to answer "No" for the areas of submission required for Pupil Services employees. But those elements were not required for these employees because they had not yet been placed on the Career Ladder.

As you can see, some educators did not receive an evaluation because of extenuating circumstances. When teachers retire, resign, or are on leave, we typically do not complete and evaluation. When they are hired late in the year, we normally conduct their first evaluation the following year. In any event, the total number of educator evals conducted was 1783.

There has been some confusion across the state as to which employees in "support" roles should be included on the Career Ladder (Athletic Directors, Librarians, etc.). Here's a recounting of those evals compared to the total number of evaluations. Note that if all "incomplete" evals were counted against us, our percentage of "accurate" and "complete" evals would have been 1640 out of 1784, or 86%.

However, as you can see in the chart above, fully 197 staff were missing components that were not required for them in 2015-16. That leaves 44 employees, 17 of whom likely should have been on the Career Ladder, but for whom the District was awaiting guidance, and 27 others whose evaluations were truly not complete.

So even if the misclassified employees are included as incomplete, just 44 of 1783 evals were not done correctly, meaning that 1739, or 97.4% were "complete" and "accurate". Though we'd like to be at 100%, and will work toward that goal, 97% is a good start.

Incidentally, of the 1784 educator evals completed, 1764 were rated as "Proficient", and 19 were rated as "Basic" or Unsatisfactory". We have written previously that we know the vast majority of District teachers are doing a great job in the classroom. The "probationary" period for teachers is in the first 3 years, and especially in the first year, when 5% - 8% of teachers opt out of the profession or are counseled not to return for the following year.

ADVANCED PLACEMENT AND COLLEGE-GOING

Followers of this blog are aware that enrollment in Advanced Placement coursework has exploded in the past two decades, as the District has added coursework, implemented programs such as AVID, and taken down barriers to enrollment.

PROGRAM GROWTH

Here's a chart showing the Boise students taking one or more AP tests expressed as a percentage of juniors and seniors over the years, compared with the "pass rate" (exams scored 3 and above) for those same students.

So, as we anticipated, the overall "pass rate" for AP tests taken in the Boise District has declined from about 3/4 to about 2/3 in the past twenty years (the 2016 national rate was 58%; the Idaho rate (without Boise) was 60%.

However, less than 1 in 10 students (as a percentage of juniors and seniors) took an exam in 1995. Last year, the ratio was 52%. This chart is truly illustrative of the progress we've made in providing rigorous opportunities for kids.

COLLEGE-GOING RATES

The State Board of Education posted a Dual Credit report in 2015 in which they wrote about the college-going habits of high school students enrolling in dual credit classes. 71% of students passing dual credit classes enrolled in college, and 70% of those students returned for a second year. For non-dual credit students, the figures were 45% enrolling and 55% returning for a second year. So, almost 50% of dual credit students were still enrolled in sophomore year, while only a quarter of non-dual credit students were still enrolled.

We were curious how this kind of comparison played out for students who took one or more Advanced Placement tests during their high school careers. We took a look at the class of 2013, and analyzed the "go-on" rate for the class and the percentage of those that went to college who were still enrolled in the fall of 2016.

The results of the study were impressive. Of seniors in the class of 2013 who took one or more AP tests in high school, 91% enrolled in college, and 81% were still enrolled or have graduated 3 years later. 

Here are the data by Boise School District high school:

By way of explanation, the first column in the above chart reflects the percentage of class of 2013 students who took one or more AP tests and enrolled in college after graduation (Boise High percentage - 94%). The second column is the college enrollment percentage for all 2013 graduates (Boise - 70%). The third column is the percentage of graduate AP test takers who were enrolled after 3 years (Boise - 89%), and the fourth column is the percentage of the 2013 class enrolled after 3 years (Boise - 68%).

Now, it makes sense that a higher percentage of AP test-takers would have enrolled in college than of the total graduating class. Many AP students come from supportive families that have the resources to assist students with college expenses. That would have especially been true back in the time when only a small percentage of Boise high school students took AP coursework.

However, with the growth of the Advanced Placement program, more students than ever are accessing rigorous coursework, and the percentage of those students attending and staying in college is extremely high. In fact, if most of the class of 2013 students who've been in college for three years graduate, the percentage with degrees and certificates will far exceed the State Board's goal of 60% of students ages 25-34 having a post-secondary degree or certificate. 

No matter the college or university, students who took one or more AP tests in high school stay in college at far higher rates than do students in the overall population of the college or university.

AP test-takers also go to out-of-state colleges more often than Boise's college attendees as a whole. In general, about 2/3 of Boise college-goers attend in-state colleges. Among AP test-taker, the percentage is 55% in-state ad 45% out-of-state.

THIS AND THAT FOR JANUARY

Here are some interesting articles and opinions from around the web.

We are alerted to some of these articles via Peter Greene's excellent blog, Curmudgucation. However, this first one is about two of our refugee students and their ongoing successes.

Two Brothers' Quest to Change Hearts and Minds about Refugees, by Kulsoom Rizvi

This is a feature from the International Rescue Committee's online journal that will touch your heart. Two brothers, Patrick and George Ngalamulume, former Borah students, are enrolled at the University of Idaho. Patrick is a senior and George a freshman. They both are originally from Zambia, and were in the Borah Bridge program. George was one of our designated AVID program speakers at Rotary, IBE, and other organizations. 

Patrick and George are truly ambassadors for refugees and for opportunities that have been made available to them in America. Patrick wants to be an attorney and George a doctor. Their stories are inspirational.

Among quotes from the brothers:

"To have the doors [shut] on refugees will greatly have a negative impact on the United States," George says. “You’ll miss out on so many opportunities. The next Einstein is not going to be here because America closed the doors. You’d miss out on having me as a doctor and [my brother] as a lawyer.”

The Red Queen, by Jennifer Berkshire (Edushyster)

Jennifer Berkshire, an education blogger from Massachusetts, writes as Edushyster. She traveled to Michigan to conduct interviews with a number of people affected by Betsy Devos' efforts to undermine public education in the state. Fascinating read.

"...The real goal of the DeVos family is to crush the state’s teachers unions as a means of undermining the Democratic party, weakening Michigan’s democratic structures along the way. And on this front, our likely next Secretary of Education has enjoyed measurable, even dazzling success."

Time to Eliminate 3rd Grade Reading Retention by Rob Miller (View from the Edge)

Miller is a teacher in the state of Oklahoma, one of a number of states that retain third graders who do not read on grade level. There's plenty of research demonstrating the negative effects of retention, but Miller makes an eloquent argument for providing the resources to help these students succeed instead of punishing them with retention. 

Third grade retention was part of the original reading bill in the 2016 legislature, but was removed after significant outcry. The version of the bill that passed the legislature invests in providing resources to improve reading skills.

Inside Sears' Death Spiral..., by Hayley Peterson (Business Insider)

This one is not directly related to education, but anyone thinking that "running schools like a business" is a good idea should read this post, which details the mismanagement and decline of this venerable company.

Peterson writes, "Interviews with dozens of store-level and corporate employees over the past year yielded a common refrain: (Sears CEO) Lampert is out of touch with reality.

"He refuses to put a dime in updating stores," one former vice president said. "You walk in and you are embarrassed as an employee when the ceilings are leaking and the floors are cracked."

"No one believes in Eddie's vision," this person said. "He has just gone rogue."

Business Insider spoke to several store-level employees who said the stores are severely understaffed, with some operating on fewer than half of the employees they need. That has led to widespread complaints among shoppers that they can't find an employee to check them out, so they end up leaving the store empty-handed."

ADVANCED PLACEMENT PARTICIPATION

IN IDAHO DISTRICTS AND HIGH SCHOOLS

Across the state of Idaho, AP participation and exams are showing growth. But which district and high schools are providing the opportunity for rigor through AP to their students?

There are several ways to look at the data. Here is a chart showing student exam participation in Idaho districts with more than 50 students taking 1 or more exams.

Well, not really a fair comparison. Boise and West Ada have the largest participation rates, but the two are the largest districts in the state, as well. Also, Coeur d'Alene's data includes Coeur d'Alene Charter Academy's students, which inflates the total for the district.

It's interesting to look at growth in student AP exam participation in examining the effort districts are making to improve their AP programs, no matter how large or small the district or participation, no matter how large or small the numbers (as long as they are greater than 50).

The Nampa District appears to be making a concerted effort to grow its AP program, with most of the increase coming at Columbia and Skyview High Schools. Blaine County, Madison, and Twin Falls also have seen increases in participation of over 30%.

Next,  let's have a look at student participation and exams taken by high school, again using the 50 student bar.

The four comprehensive Boise high schools lead the way, followed by Vallivue, Rocky Mountain (West Ada), Centennial (West Ada), Century (Pocatello)  and Blaine County.

The picture is not much different when we look at exams taken by high school.

Again, though, these are not really fair comparisons, since most of the high schools in the chart are large. 

So we took the student participation numbers at the high schools we examined and compared them with the number of juniors and seniors enrolled in 2016, and expressed the resulting fraction as a percentage. Here's what the comparison revealed:

This chart more fairly expresses the AP participation by school. Note that Coeur d'Alene Charter, McCall-Donnelly, and Wood River (Blaine County) are now in the upper reaches of the chart. What the comparison misses is that a number of sophomores, and a few 9th graders, take AP exams. However, it does get at the issue of school size.

We realize that CDA Charter is a special case - the school serves a select group of students who are looking for rigorous curricula, and, from 7th to 12th grade, we see the school's enrollment drop substantially. Still, when we examined the array of AP courses offered at the school, we were impressed. We counted 13 Advanced Placement courses taught in the school.

McCall-Donnelly and Blaine County offer multiple AP courses, as well, proving that relatively small schools can offer an impressive array of opportunities. 

Of course, in the Boise District, students can choose from among 25 Advanced Placement offerings, regardless of the high school they attend. In the Vallivue District, there are 17 offerings, and students attending Ridgevue (the district's new high school) and Vallivue High School have equivalent opportunities for rigor.

In many high schools across the state, there are no Advanced Placement offerings. While dual credit opportunities typically abound in these districts, students cannot take the "gold standard" offerings provided in other districts, except through online opportunities from the Idaho Digital Learning Academy and others. 

IDAHO ADVANCED PLACEMENT PROGRAMS 

SHOW GROWTH

There was an interesting op-ed in the Idaho Statesman the other day, from a former Advanced Placement teacher, asserting that dual credit classes are no substitute for college classes. For many Boise District alumni who have benefited from the credits they have earned from AP tests, that will come as a surprise, especially since they have earned credit for core classes at prestigious universities and colleges, and started college as second semester freshman or even as sophomores. AND they will tell you they felt well-prepared for the subsequent courses they took.

AP courses are, of course, not intended to satisfy advanced college coursework requirements in major areas. They are, instead, meant to allow students to get a head start on their college careers by passing exams that certify their competence in content knowledge in core college academic courses.

The reason we consider Advanced Placement to be the "gold standard" among college preparation programs is that students must take and pass rigorous exams which serve as a national benchmark accepted by the vast majority of colleges nationwide. About 2/3 of exams from the Boise District were judged to meet that standard in 2016. The overall Idaho percentage was 63% (60% without Boise. The national 2016 "passing" percentage was 58%.

Most students do not use AP course credit to graduate early from college. Instead, they take on double majors, add a minor area, or use the flexibility afforded by the credits they have earned to travel abroad or to ease the load as they participate in activities and sports in college.

Idaho Student AP Participation Has Grown Dramatically

Participation in Idaho's AP testing program has increased steadily since 1995, and is now almost 6 times what it was 21 years ago. These are students who took one or more exams in the given year's AP testing.

However, the Boise District's AP participation as a percentage of the statewide student count has also increased:

So when we take out Boise's numbers, we get an idea of what's happened with AP participation across the state of Idaho outside of Boise. The increase is still impressive, with over 4 times as many students participating in the program as did in 1995.

The Number of Idaho Exams has Increased Substantially, as Well

Since 1995, the number of AP exams taken in the state of Idaho has increased by over 600%.

The percentage of exams taken by Boise students steadily increased for a number of years, and has been stable since 2010. In 2016, the percentage of exams taken by Boise students was 35%. Interestingly, among 2016 Boise District seniors, just under 10% had taken 10 or more exams during their high schools careers, and almost half took 5 or more.

Here is the increase in Idaho exams when the data are considered without the Boise District. The rate of increase without Boise is almost as high as it is when Boise's exams are included - a sign of a healthy, growing program around the state.

Next on the blog - which districts and high schools lead the way in AP, and who is making great progress?

BOISE DISTRICT AP EXAMS, STUDENT PARTICIPATION REACH NEW HIGHS (AGAIN)

Boise District students took 35% of the statewide Advanced Placement tests in 2016, setting a new high of just less than 4000 exams. District students represent about 9% of the statewide student population.

In the past twenty years, the exam total has grown from just over 500 to 3963, growth of almost 800%.

Student participation has increased in similar fashion, as well. In 1995, 203 students took at least one Advanced Placement exam. In 2016, 1992 students did so. 30% of the statewide student participation came from the Boise District.

The District Advanced Placement program began at Boise High School in 1980, under the leadership of Principal Jack Craven. It has since spread across the District to all 4 comprehensive high schools. Boise High still has the highest participation and gives the most exams (13% of the state total in 2016).

Though Boise High had the highest percentage of its students taking an AP exam in 2016 (44%), Timberline has made great gains - 40% of the student population took at least one exam in 2016. Borah and Capital were at 31% and 29%, respectively, in 2016.

It's interesting to note that the 3 most popular AP courses (and tests) satisfy graduation requirements in the Boise District and for the state. For example, AP English Language satisfies the junior English requirement, AP English Literature the senior English requirement, and AP US History the junior Social Studies requirement. AP Calculus AB satisfies one of the three required math courses, as well. The true electives on the "most popular" list are AP World History, AP Psychology, and AP Human Geography. 

In those three courses, Boise students take a much higher percentage of statewide AP exams. In World History, 51% of the state's exams were taken by Boise students, in Psychology 66%, and in Human Geography, 86%.

EVALUATING THE MCREL 
TEACHER EVALUATION AUDIT

McREL International, a Denver-based non-profit which has developed its own teacher evaluation system, was hired by the Idaho Department of Education to review Idaho's progress in implementing its evaluation system, which is based on the Charlotte Danielson evaluation model.

McREL was asked to audit 225 randomly selected teacher evaluations from across the state (36 of the 225 came from Boise, which has about 12% of the state's teachers). A State Department of Education committee, the Principal Evaluation Review Committee, provided the questions that McREL was to use in preparing its report.

Idaho Education News greeted the publication of the audit data with the headline "Audit Finds 99% of Teacher Evaluations were Inaccurate or Incomplete". Idaho Board of Education President Emma Atchley said soon after the publication of the article, “The audit raises serious concerns regarding the teacher evaluation process conducted during the 2014-2015 school year.” 

But did it? Let's take a look at the questions provided by the committee and used by the auditors and how they relate to Idaho Code and practice.

Question 1 - What are the components that were on the Individual Professional Learning Plan (IPLP)?

The audited evaluations were from the 2014-15 school year. The IPLP was not required that year; it came into effect in 2015-16. No wonder a large number of evaluations missed the mark in this area.

Question 2 - Does the professional practice portion include all 22 components of the Charlotte Danielson Framework (2nd ed.)?

State Board  rule requires that “each district evaluation model shall be aligned to state minimum standards that are based on Charlotte Danielson’s Framework for Teaching Second Edition domains and components of instruction. The rule, by design, does not require districts to adopt verbatim the Danielson Framework domains and components as was implied in the questions given to McREL for the evaluation.

For example, the Bonneville District uses a different evaluation system which is aligned the the Danielson system, but did not satisfy the criteria in the question, and is not required to. The Bonneville system was approved by the Idaho Department of Education.

Question 3 - Record the levels of performance for each component (1, 2, 3 or 4)

Again, State Board rule requires districts to establish a “rating system with a minimum of three (3) rankings (not 4) used to differentiate performance of teachers and pupil personnel certificate holders including: Unsatisfactory being equal to “1”; Basic being equal to “2”; and Proficient being equal to “3”. For example, Boise uses 3 categories, as is required. Even though every one of Boise's evals met State Board rule, they would all have missed the mark on this question.

Question 4 -What are the dates of the two documented observations?

Idaho teacher evaluations must include a minimum of two documented observations, one of which shall be completed prior to January 1 of each year. In the Boise School District, we require our principals to complete, at a minimum, one formal observation prior to January 1st in compliance with the law as well as a second formal observation that becomes incorporated into the final formal evaluation. The dates may not have been noted on the evaluation but in every case two observations or more were made.

Further, in two cases, the audit identified employees who did not have an evaluation because they were on leave during the second semester of the school year, one for family leave and the other for medical reasons.  However, the auditors did not ask for the reasons for missing evaluations.

Question 5 - Which additional measure(s) was included to inform professional
practice?

• Student Input
• Parent Input
• Portfolio
• None

The Boise District uses parent input to satisfy this State Board required element.

Question 6 - Which measures were used for student achievement?

The data elements provided to McREL for this review come from the options to measure

student achievement under the Career Ladder which was not part of the state law and was not required until 2015­-2016 with the introduction of the Career Ladder Legislation that was established during the 2015 Legislative Session. In 2014­-2015, districts had to include ISAT data in the evaluation as well as one or more of multiple objective measures of growth in student achievement, which could be determined by the board of trustees. Districts could choose for the ISAT to count for 1% up to 32% of the 33% of the evaluation that was to be

based on student achievement.

Additionally, the rule states that “growth in student achievement may be considered as an optional measure for all other school based and district based staff, as determined by the local board of trustees.”

As a result of this provision, any evaluations that were reviewed by McREL that were not instructional staff, may not have had student achievement data included because it did not have to be included in accordance with State Board rule. This was the case for two evaluations that were randomly selected as part of the Boise School District sampling.

Question 7 - What is the summative rating?

Question 8 - Does the summative rating include combining professional practice (67%) and student achievement (33%) 

The Boise School District requires a separate rating for the 33% of the

evaluation that is based on growth in student achievement and an overall

summative rating that includes combining the 67% and 33%.

Question 9 - What is the date of the summative evaluation?

Question 10 - Was it completed by May 1st?

Three (3) 2014-2015 Boise District audited evaluations were submitted between May and June 1. For 2015-16, that submittal date was changed by the legislature to June 1 because May 1 was deemed too early.

Question 11 - Is there a written evaluation policy?

The State Department of Education did not ask for the District's policy, or we would have provided it.

In summary, McREL was asked in the 2014-15 audit to:

• conduct analyses of elements not required by law or under Idaho Code
• analyze adherence to deadlines that had already been determined unreasonable by the legislature
• summarize without asking for explanation:
• observations required by law and by local policy
• missing evaluations that may have been due to unforeseen circumstances

In order for legislators to have an understanding of the errors inherent to the audit process as they make decisions about the Career Ladder, it makes sense that the State Department of Education confer with districts involved in the audit as soon as possible. We believe we have done a thorough job of outlining the flaws of the audit. Now is the time to set it right.