Wednesday, August 26, 2015

JUDGING FROM AFAR

There are a number of school ranking systems out there these days. Each has criteria upon which its rankings are based. Most school rating systems examine a small slice of a school's characteristics, namely standardized test scores. For example, Great Schools uses standardized test scores as their primary rating tool, as does Schooldigger, which also includes information about a school's free/reduced lunch percentage.

Two of the nation's most prominent school ranking systems focus on high schools, those of Newsweek magazine and U.S. News and World Reports. These two ranking systems use fairly complex systems which consider standardized test scores and a few other variables, including how well the schools serves its underprivileged population. A third, the Washington Post's "Most Challenging High Schools" list, relies on a much simpler formula for its rankings.

U.S. News High School Rankings

Kevin Welner, the Director of the National Education Policy Center at the University of Colorado-Boulder, wrote a critique of the U.S. News high school rankings which was published in the Answer Sheet, Valerie Strauss' blog for the Washington Post newspaper. The critique, which spanned two posts, can be found here (Part 1) and here (Part 2). In Part 1, Welner looks primarily at the methodology of the U.S. News rankings, and in Part 2 he examines the characteristics of some of the top-ranked schools.

From Welner's critique:

"There is an important lesson here about rankings in general—not just the U.S. News rankings. A disproportionate focus on outcomes will always reward those schools that excel at enrolling high-achievers. Perhaps this is more obvious when we look outside the school realm. For instance, University of Kentucky basketball coach John Calipari is such a fantastic recruiter that when looking at his success it’s almost impossible to disentangle his recruiting and coaching talents."

and:

"This year, the nation’s top-ranked high schools are: (1) Dallas’ “School for the Talented and Gifted,” a magnet school; and (2) BASIS Scottsdale (AZ), a charter school. The Dallas school has an application process that begins with a GPA screen and a test score screen, then the applicant student must “design and carry out a school-related or other project that demonstrates extended effort and creativity,” and then the applicant student must attend an “application session.” At this session, the applicant student must complete a timed (1 ½ hours) hand-written, prompted essay; must complete a “30-minute timed logic and reasoning activity;” and finally is given a 15-minute scored interview. This highly competitive process yields an entering class of 65 students.

The Scottsdale charter school is not selective in the same way; admission is by lottery. But the selectivity is arguably just as potent as with the Dallas school. Parents are warned in no uncertain terms that only the most gifted and committed students will survive. Just a handful of students are admitted after the early grades (e.g., a parent was told that there would be a total of four slots open for entering eighth graders). Moreover, as described by the mother of a student in another BASIS charter, the warnings appear to understate the extremity of the BASIS approach."

U.S. News responded with a written statement about Welner's critique, which read in part:

"Mr. Welner’s overall argument is incorrect. Schools that do better and are awarded medals in the U.S. News rankings are the ones that are doing more (better than the average in the state or way better than the average in the state) than what is expected given their level of poverty or proportion of economic disadvantaged."

Newsweek Rankings

Newsweek ranks the top 500 high schools in the country, based on its own analysis. Westat, a Maryland research company, provided an accompanying document to the rankings, entitled "Identification of Newsweek's 2015 Public High School Rankings", which explained the methodology of the rankings. In the Section called "Limitations", the authors make the following remarks:

"There are limitations associated with conducting this analysis, many of which stem from the availability of data and their suitability for comparing schools in different states. We have no information about a range of school factors that may influence school performance, such as fiscal resources, teacher quality and effectiveness, school leadership, and school climate. These school factors could potentially contribute to student achievement and college readiness, but data are unavailable for a variety of reasons.

Additionally, the rankings are still dependent on self-reported data, which has implications for our sample and data collection standards. There are circumstances in which the variables may not have been reported consistently, and for 2015, this issue may be particularly problematic for the new dual enrollment items."

and this:

"We did not control for any other factor besides student socioeconomic status. Many of the top schools on the relative list are magnet and charter schools and may have an application process that allows them to select high achieving students. These types of schools would have an advantage in our ranking methodology over schools that do not have an application process."

So, Newsweek's rankings suffer from some of the flaws noted by Mr. Welner on the U.S. News rankings. Further, some of the magnet and charter schools he referenced are among the highest ranked schools on both lists.

Washington Post Most Challenging Schools Index

The Newsweek rankings were formerly the "Most Challenging High School Rankings" until the Washington Post Company sold Newsweek in 2010. Since then, Post education columnist Jay Mathews has continued with his rankings, using two simple factors for the evaluation of "Challenging High Schools" : If the number of Advanced Placement or International Baccalaureate Exams given in a particular year at the school divided by the number of school's graduating seniors that year is greater than 1.0, the school makes the list.

To be sure, the Post rankings suffer from some of the same shortcomings as the other ranking systems. The top-ranked schools are typically magnets or charter high schools. And, on the Post rankings, the number of exams and graduates are self-reported.

In Idaho last year, 10 high schools applied for and made the Washington Post list. Here they are, with their ratio (AP or IB/Grads) and free/reduced lunch percentages:


Especially impressive on the Post list are the achievements of Borah, Capital, Century, and Vallivue, which have successfully emphasized rigorous coursework and achievement with high numbers of disadvantaged students.

On balance, we prefer the Washington Post list to the others, since it is simple and understandable. It was a District goal in the early 2000's to have all 4 comprehensive high schools on the Post list, simply to prove to ourselves that we could do it. Now that all four have made the list for 6 straight years, our challenge is to expand opportunities for all students to access rigorous coursework and preparatory opportunities.

Clearly, it's extremely difficult to put together a ranking system that reflects all of the characteristics of a successful school.Witness the difficulty Idaho has had with the establishment of the "Five Star System", which relied only on achievement and produced school  rankings which were unstable and unreliable.


The combined value of school culture, achievement and depth of offerings such as music, art and athletics may be harder to quantify, but is perhaps more important to student success than any annual school ranking list can adequately portray.










THIS AND THAT FOR AUGUST


Here are some interesting stories from summer, 2015:

Teacher Shortage News

Peter Greene, English teacher and author of the Curmudgucation blog, put together a post entitled Teacher Shortage Coast to Coast back in late July which summarized the data from the U.S. Department of Education report on the subject. The Real Reasons Behind the Teacher Shortage, from Valerie Strauss' Answer Sheet blog, has some additional information, and the USDOE report is embedded.

Dramatic Increase in UW Computer Science and Engineering Majors 

Seattle Schools Community Forum blog is written by a parent in the school district, who makes some interesting posts.. In June, the author, Melissa Westbrook, wrote about Computer Science/Engineering trends at the University of Washington. We located the original article, Trends in the First-Choice Majors of Incoming UW freshmen

Two charts from the article show the dynamic changes in first choice selections of UW students
.

 Pretty incredible growth when compared to other choices of other majors in the College of Engineering...


And when compared to other popular majors, including Business, Psychology, and Biology. In explaining the growth in interest, the author of the article said, "Students are smart. They understand which field offers the greatest opportunity to change the world!"

We are attempting to gather some data from  Idaho universities abut first choice majors of incoming students.

A Different Take

Washington Post columnist Fareed Zakaria wrote an interesting column in the paper entitled Why America's Obsession with STEM Education is Dangerous. In the opinion, he lays out arguments for continuing in the American tradition of a broad-based curriculum, liberal including this:

My point is not that it’s good that American students fare poorly on these tests. It isn’t. Asian countries like Japan and South Korea have benefited enormously from having skilled workforces. But technical chops are just one ingredient needed for innovation and economic success. America overcomes its disadvantage — a less-technically-trained workforce — with other advantages such as creativity, critical thinking and an optimistic outlook. A country like Japan, by contrast, can’t do as much with its well-trained workers because it lacks many of the factors that produce continuous innovation.

Zakaria makes some of the same arguments made by Yong Zhao, the University of Oregon professor whose most recent book, Who's Afraid of the Big Bad Dragon we are reading currently.

PDK Gallup Poll Results

The 47th Phi Delta Kappa/Gallup Poll of the Public's Attitude Toward Public Schools results came out last week. The poll results are very reader-friendly, but here are a few of the key results:

  • 64% of those polled feel there is too much standardized testing in public schools.
  • Higher percentages of the sample thought that:
    • how engaged students are with their coursework (78%)
    • the percentage of students that feel hopeful for the future (77%)
    • the percentage of students that graduate from high school (69%)
    • the percentage of high school grads who "Go On" (38%)
    • the percentage of hs grads who immediately get jobs (27%)
were more important indicators of quality than test scores (14%).

Further, poll takers thought that examples of the student's work, written observations by the teacher, and student grades were all better indicators of student progress than standardized test scores.

The results with respect to "opting out" of standardized tests were more mixed, with 44% saying parents should not be allowed to opt their children out of testing, and 41% saying they should be allowed to do so.

55% of those polled were opposed to using standardized test results to evaluate teachers, while 43% favored their use in evaluations.

54% opposed use of the Common Core standards, while 24% favored their use in schools.

While 64% of those polled favored allowing choice and charters, only 31% favored the use of vouchers.








Tuesday, August 18, 2015

THE IDAHO HIGH SCHOOL GRADUATING CLASS OF 2005  TEN YEARS OUT

The staff at the Idaho Board of Education began tracking the college status of Idaho high school graduates with a pilot for the class of 2005.  Data has been collected for 85 high schools  across the state for the past 9 1/2 years, and is displayed on this website. The website also features data about "Go-on' rates at various stages of progress for the classes of 2011-15. However, in this post we'll focus on that longitudinal data set for the class of 2005.

State of Idaho Graduation Rate


Here's the overall status of the class (85 schools), as of March, 2015:





As you can see, 36% of the class has graduated from a 2-year of 4-year institution, or acquired a certificate. This percentage is likely a bit low; some 2-year and 4-year colleges are not members of the National Student Clearinghouse, which supplies the data to the state.  

What's interesting about the data is the increase in the percentage of graduates between year 6 and year 10. Only 26% had graduated by 2011; by 2015, 36% had completed.  Eight (8) percent of the class is also currently enrolled at an institution.


Also, though only 36% of the class has completed a program, 71% of the class of 2005 has competed some college as of March 2015. Here is a PDF document with college degree progress portraits for twelve high schools and for the state of Idaho.


In Signature Report 7: Some College, No Degree, the National Clearinghouse says this on page 6 about the length of time taken for college:


"More than one in four potential completers (25.7 percent) enrolled continuously or intermittently for seven years or longer. While 38.1 percent enrolled continuously or on-and-off over three years or less, a similar percentage (36.3 percent) spread their enrollments over four to six years. These results suggest that standard cutoffs for

measuring student graduation rates (typically 150 percent of program length) are inadequate: significant numbers of students continue to make substantial progress toward a credential for many years longer."

In fact, the NCES College Navigator publishes information about 4, 6, and 8-year graduation rates for colleges on its website. For example, at BSU,  the 6-year entering first-time, full-time class of 2006 graduation rate was 29%, while the 8-year grad rate was 36%.At Idaho State, the 6-year rate was 31%, and the 8-year 37%. Idaho's percentages were 56% and 59%, respectively.

Sample of High School Graduation Rates


For the past several years, seven of the top-performing large high schools on the SAT have been Boise, Timberline, Moscow, Lakeland, Madison, Sandpoint and McCall-Donnelly. Moscow and Sandpoint did not participate in the class of 2005 study; however, we have information about the status of the other 5 high schools.



Fifty-four percent (54%) of the Boise High class of 2005 has completed, as of March, 2015, and another 6% are enrolled in some form of program. Eighty-four percent of the class has attended college at one time or another in the past 9 1/2 years.

About 50% of Boise graduates attend school at out-of-state institutions. Among the most popular are the University of Utah (59% six-year grad rate, 73% eight-year). Western Washington University (67%, 69%), University of Colorado-Boulder (68%, 70%), Gonzaga University (81%, 82%), and the University of Oregon (67%, 70%). Many students attending these universities from Boise consistently enroll in a full-time capacity.



In a typical year, about 65% of Timberline students enroll at Idaho colleges. The University of Utah and Brigham Young University (77%, 86%) are also popular choices, but the vast majority of Timberline students enroll at BSU and Idaho. Forty-four percent (44%) of Timberline students had completed by March, 2015, and 80% had attended some college.



Lakeland's percentage of college completers stood at 21% 5 years after the class graduated from high school, comparable with Timberline (21%) . However, the percentage of graduates at Timberline more than doubled from 2011 to 2015, while Lakeland's grew by only 5%.

Gauging the Impact of Church Missions

The "Direct to college" Go-on rate, or even the percentage in college a year or two after high school graduation, is irrelevant for a school district like Madison, where many students serve a church mission at some point after high school. While Madison has a low direct Go-on rate, the district's percentage of students graduated after 9 1/2 years is among the state's highest.



Madison's college completion percentage in 2009, four years after high school graduation, was 16%; in 2013 it was 45%.  The higher education schools of choice for Madison students are BYU-Idaho (located in Rexburg) and BYU-Provo.

A Small District with a Large Graduation Rate



Among smaller districts, McCall-Donnelly stands out, with a college completion rate the same as Boise High School. Seventy-nine percent (79%) of McCall's class of 2005 graduates have completed some college.