ADVANCED PLACEMENT: THE BEAT GOES ON

Student participation in the Boise District's Advanced Placement program continues to grow, both in terms of tests taken and number of students taking tests.

The District offers the most complete Advanced Placement program in the state of Idaho; each comprehensive high school offers at least 24 A.P. courses, allowing students to choose coursework in a variety of subjects. Here's a complete list of AP coursework offered in the District.

Students

Student participation in the AP exam program has increased dramatically since the inception of the program in 1980, when the program, then just at Boise High, had 50 testing participants. Here's the data n student participation since 1995.

The growth in student participation in the AP exam program is truly remarkable. 


Here is the same information broken down by high school. The information for the year 2000 is interesting because Timberline opened in 1998, so Boise's participation was lower in 2000 than in 1995. Participation rates for all four comprehensive high schools have grown substantially since then. It's also important to note that enrollment at the 4 high schools at the time of testing was Boise 1524, Borah 1474, Capital 1243, and Timberline 1107.

And there is something new this year: 24 9th graders at Hillside Junior High School took the course and the exam for AP Human Geography. It's our first effort at offering an AP course for our junior high students.

Exams

Many students take more than one Advanced Placement examination; in fact, a number of students took five exams this year, and a few took six or even seven. So the exam number is much higher than the number of students taking exams.

And the number of exams has grown dramatically. In 2016, Boise District students took 8 times as many AP exams as they did 20 years ago.

Here's the exam information. This is a story of continuous improvement, and of providing access for more and more students to rigorous coursework as time has gone by. 

So, how does this information impact the college entry level of students? Since over 95% of colleges nationwide provide credit pr waivers for passage of AP tests, the impact is considerable. Here are a couple of hypothetical examples of the total credits that students would have earned in 2016, using the AP credit policies of the University of Idaho.  Of course, not all students would have been able to obtain credit for particular courses - depending on major fields, some credits would not count in fulfilling prerequisites or the core requirements. But this is an indicator of the impact passing AP tests can have for our students. (Note: Boise State and Idaho State have similar policies - this is just an example.)

So, students could have the opportunity to obtain over 9000 college credits based on AP test performance. And, since AP credits are accepted by almost all universities and colleges, students don't have to worry about acceptance policies at particular schools - they can check it out on-line at the AP Central Credit Policy website.

WHAT THE SAT ITEM ANALYSIS TELLS US 

(AND THE SBAC DOESN'T)

THE SAT

The "new" SAT was administered to juniors across the state of Idaho in April. The test has been revised substantially. The major changes are:

• alignment with the Common Core
• two subtests ( Math and Evidence-based Reading and Writing) instead of three (Reading, Math and Writing)
• improved resources for improvement of instruction
• new "cut" scores indicating probability of success in college (EBRW = 480, Math = 530)
• altered definition of readiness for college that includes 2 and 4 year colleges

The College Board, which administers the SAT, has released the actual test for a number of years after that year's administration is complete. However, though they have offered an item analysis for the PSAT (Preliminary SAT) for several years, this year was the first time the College Board has provided question analysis for the SAT.

The item analysis will prove extremely valuable for the improvement of instruction. Here's why.

For each of the 154 questions on the SAT (96 EBRW and 58 Math, the College Board provides access to the answer given by each student who took the test, district, school, and state performance on that item, percentages of students who chose distractors and the correct answer, the difficulty level of the problem, and the general question category. 

The item analysis is available for each high school, as well as for the district and for the state as a whole. The College Board also provides an explanation of the correct answer for each question. 

Here is the display of the information for question 9 on this year's Math subtest:

How does the distribution of student answers help us to improve instruction?

Well, 28% of District juniors answered "D". While we might speculate as to why they chose that answer, the fact is that 55+9 does not equate to √55+√9. Can we improve instruction so that more students understand this principle? Sure. And the item analysis gives us the tools to improve instruction and student performance.

College and university personnel can also use the statewide item analysis to isolate performance on each question and then use performance on a series of questions to determine placement in Math or Language Arts courses.

THE SBAC

Comparatively, the SBAC gives only top-level information that is of little use in improving instruction, Here's an example of what teachers receive as feedback from the SBAC:

It's clear that the SAT provides a much more useful examination of student responses that can help us with improving instruction in high school than does the SBAC. We need an assessment at the elementary and junior high level that provides the same level of feedback.

States are required under federal law (ESSA) to test all students in math and language arts in grades 3-8 and once in high school. At each decision point about the adoption of assessments, we must consider use of data for improvement of instruction first. Then and only then will we adopt an appropriate assessment.

2016 SAT STATEWIDE RESULTS

The SAT Schoolday exam was administered to Idaho juniors in April. Over 17,000 students took the revised exam, which consists of two subtests, Evidence-Based Reading ad Writing and Mathematics. Each subtest has a maximum score of 800, for a total possible score of 1600.

We requested statewide district and school scores from the State Department of Education, along with SAT participation numbers. The SDE also provided recent 11th grade enrollment numbers from across the state. Kelly Everitt from the State Department of Education graciously provided these results. Then, we added free/reduced lunch percentages for each school and district to the mix. 

It's important to consider indicators of poverty in analyses such as ours, since the SAT (and the SBAC) are highly correlated with poverty. Then, we can assess the status of schools in comparison with the expectations that might be associated with the level of poverty at that school/in that district, and use data from some "outliers" to help those that might not have fared so well.

Results

Here are scattergrams portraying SAT results for the total test, and for the two subtests. We used SAT "percent prepared" percentage as the "y axis", and free/reduced lunch percentages on the "x axis"

These percentages are often misused. It's important to remember that SAT scores are just one indicator of preparedness for post-secondary pursuits. In fact, grades are the best predictor, and should be the first indicator used in assessing preparedness. Rigor of coursework, level of math courses taken and passed, extra-curricular participation, and other factors are also important. However, many colleges and universities use SAT and ACT scores as part of their admissions processes.

We used a threshold of 75% participation for including schools in the analysis we did, which left out three high schools (Lakeland - 71%, Wood River - 69% - and Weiser, - 71%). In addition, we excluded alternative high schools and magnet schools, since we were interested in comprehensive high school performance.  Two charters, Compass Academy, and Idaho Virtual Academy, made the 75 student cutoff and exceeded the 75% participation mark.

Evidence-based Reading and Writing

Here are results of the Evidence-based Reading and Writing subtest. Boise, Timberline, and Moscow high schools are typically among the high performing high schools in the state, and continued that pattern in 2016, and Compass Charter's performance was quite good, as well. However, when we factor in free/reduced lunch percentages, some other schools show excellent performance: Twin Falls, Timberlake (Lakeland District), Capital, Century (Pocatello), Sandpoint, Centennial, and Post Falls did well in this comparison among schools with 30-50% free/reduced populations. And Jerome, Canyon Ridge (Twin Falls District), Columbia, Vallivue, and Idaho Virtual were high performers among districts with 60-80% free/reduced lunch.

One factor that's not considered in this comparison is the Limited English population of a school. Though the vast majority of LEP students qualify for free/reduced lunch, they are also just learning the English language, and the SAT and SBAC present a complex range of tasks that often make the test almost impossible. With the focus in Common Core on reading even in the math section, these Core-aligned tests are extremely difficult for our LEP students, and few do well. However, as these students grow more comfortable with the English language, and participate in programs such as AVID (available in Boise and Vallivue), many do well in college pursuits.

Schools such as Timberlake, Sandpoint, Coeur d'Alene, Post Falls and many others have few LEP students, or may even have none.

That being said, for a school like Borah, with the highest percentage of LEP students in the state, the scores of the LEP population artificially deflate the school's performance. When Borah's LEP scores are taken from the total, it's "readiness" percentage would be much higher. Even with the scores included for the LEP students (some of whom are in their first or second year in the U.S.), Borah's performance is above expectations.

Math

In the math comparison, many of the same schools performed well. Madison, which has historically performed very well in math, shows up in this comparison. On the 50%+ free/reduced comparison, Emmett and Buhl appear as high-performers, as does Caldwell, which has nearly 80% free/reduced, as well as a high percentage of LEP students.

Since SAT and the SBAC results correlate highly with poverty, it's important that teachers receive specific results from the test that allow them to pinpoint curricular strengths and weaknesses so that they might improve results over time.

Do the two tests provide such information? That's the subject of our next blog post.