Mississippi’s Progress Not a Surprise; It was Part of a Plan

When the 2019 results of the National Assessment of Educational Progress (NAEP) were released last week, the good news was that fourth-grade students in our neighbor state, Mississippi, scored at the national average in both reading and math and that eighth graders there made significant gains in reading and math as well.

On all four measures, the average score for Mississippi students exceeds those of Alabama students, despite Mississippi’s higher level of poverty and higher percentage of students of color. The interactive chart below traces the average scale scores for Alabama and Mississippi students on the NAEP since 2003 on each of the four measures. The green line represents the average of public school students nationally. Other tabs in the chart allow you to explore other ways of looking at the data, including comparing demographically similar groups of students across states. In each of the main demographic and economic categories, Mississippi students are outperforming Alabama’s. The NAEP reading and math assessments are given every two years to a sample of students in each state, the sample representing the demographics of the state. It is the same assessment from year to year, and it is administered nationally. Known as the Nation’s Report Card, NAEP serves as the principal national measure of academic proficiency for U.S. education.

Mississippi’s rise was not a surprise; it was part of the plan: a serious, sustained, and strategic effort to improve. In a 2018 report, Leadership Matters, commissioned by Business Education Alliance, PARCA described Mississippi’s strategic plan for educational improvement.

Mississippi has had a consistent and cohesive educationally-focused leadership in the Governor’s office, the State Legislature, and at the Mississippi Department of Education. Current superintendent Carrie Wright was recruited in 2013 from a leadership position in the District of Columbia’s rapidly improving public schools. Under Wright, and with the backing of an appointed state school board, Mississippi has operated under a plan that includes goals, objectives, and clearly articulated strategies aimed at meeting those goals. To track progress on the implementation of strategies and progress toward goals, the superintendent delivers an annual update that details actions taken to advance toward those goals.

Mississippi borrowed some aspects of its approach to improving reading from Alabama’s Reading Initiative. And more recently, Alabama borrowed from the Mississippi model. Earlier this year, the Alabama Legislature adopted a Literacy Act, similar to a Literacy-Based Promotion Act Mississippi adopted in 2013. Mississippi developed its own student assessment test for grades 3-8, MAAP, which was first deployed in the 2015-2016 school year. Alabama hopes to deploy its own state-developed assessment in the spring of 2020.

Over the long term, both Alabama and Mississippi have made progress in both reading and math. However, during Superintendent Wright’s tenure in Mississippi, Alabama has had five different superintendents. The State Department of Education did develop a state plan for education, Plan 2020, in 2012, but it was never fully developed and implemented and was shelved as subsequent superintendents came and went.

Reading

In 2011, just eight years ago, Alabama enjoyed the national spotlight when NAEP was released. Alabama fourth graders scored at the national average, having made the largest improvements in the U.S. That growth coincided with and has been generally attributed to the Alabama Reading Initiative (ARI). ARI emphasized a schoolwide commitment to getting all students reading at grade level, with an emphasis on Kindergarten through third grade. ARI placed a reading coach in every Alabama elementary school and required intensive professional development to teachers on research-based approaches to teaching reading.

However, after that achievement, and in the face of a constrained budget, funding for ARI was reduced, and schools were allowed to repurpose the state-funded reading coaches for other purposes. Reading scores on NAEP began drifting and in 2019 dropped sharply in both fourth and eighth grades.

Earlier this year, the Legislature adopted the Literacy Act, which will require that third graders be able to read, or they will be held back to repeat third grade. The Literacy Act, modeled after similar legislation in Mississippi and other Southeastern states, is expected to add urgency to reading instruction and to addressing reading challenges like dyslexia. ARI’s funding was also increased by $6.5 million, though at $51 million per year, that’s still far from the $64 million it received at its peak in 2008. Education leaders say the program will be restored to fidelity.

Math

While the 2019 reading results on NAEP were distressing for the severity of the drop, the math results for Alabama students were equally disturbing. Alabama school children in both the fourth and eighth grade had the lowest average test scores in the United States. It’s a familiar position for Alabama. Alabama ranked behind all other states in 2015. In 2017, Alabama students climbed a couple of notches in the rankings, but slipped back into last this year.

The state’s strategy for addressing math is less clear. In March of 2019, Gov. Kay Ivey put on hold new math standards, which had been developed by a statewide panel of educators. Ivey postponed adopting the changes to the math course of study after some conservative groups, who are opponents of the Common Core state standards, voiced their objections. In a letter to State Superintendent Eric Mackey, Ivy asked that Alabama’s new proposed math curriculum be compared to the math course of study for the top six performing states on the NAEP: Massachusetts, Minnesota, the Department of Defense’s educational system, Virginia, New Jersey, and Wyoming.

Considering Mississippi’s results, its approach to math should be examined as well. Mississippi did adopt the Common Core standards. Judging by national results, it safe to say that Common Core did not cause NAEP scores to leap. But it’s also true that states that did not adopt the Common Core have seen declines on the NAEP as well. Other factors may be contributing to an overall stagnation in educational progress. However, with Mississippi bucking the trend, Alabama would be well served to take note and inspiration from our neighbor’s progress.


ACT WorkKeys – An Assessment of Workforce Readiness Among High School Graduates in Alabama

The WorkKeys Assessment is a standardized test given to 12th graders in Alabama public schools. The assessment is meant to measure skills relevant to many of today’s work environments.

In 2018:

  • 64 percent of Alabama high school graduates in 2018 were deemed workforce ready as measured by the ACT WorkKeys assessment, a year over year improvement of a half percent.
  • At 94 percent, Hartselle had the highest percentage of workforce ready graduates, as measured by WorkKeys.
  • High fluctuations occurred among the different certificate levels, with Platinum (highest level) and Gold dramatically increasing, but Silver decreasing. Bronze (below workforce ready) increased, though the percent not earning a certificate decreased.

What is WorkKeys?

The WorkKeys assessments are meant to provide a meaningful assessment of applied cognitive skills useful in contemporary work settings. It is also one of the components of Alabama’s College and Career Ready measure.

The assessments do not measure a student’s attitudes about work, dependability, interpersonal skills, teamwork, communication skills, or instincts for creativity, innovation, or leadership. They also do not provide insight about a student’s competency for a job requiring specialized knowledge and skills.

The Assessments. The assessments consist of three tests of applied cognitive skills which are relevant, according to ACT’s research, to over 20,000 occupations:

  • The Applied Math test measures critical thinking, mathematical reasoning, and problem-solving techniques for situations in today’s workplace.
  • The Graphic Literacy test measures the skill needed to locate, synthesize, and use information from charts and graphs. 
  • The Workplace Documents test measures the skills needed to read and understand written text such as memos, letters, directions, signs, notices, bulletins, policies, and regulations on the job.

Students are awarded a National Career Readiness Certification in they score a Platinum, Gold, Silver, or Bronze score on the WorkKeys.

Platinum: These are students with the highest level of applied cognitive skills.  According to ACT, students at this level have demonstrated applied foundational skills for 96 percent of the occupations in the ACT jobs dataset.

Gold: Those earning a Gold level certificate should have the applied foundational skills for 90 percent of jobs in the database.

Silver: Students scoring at the Silver level should have the applied foundational skills for 71 percent of jobs in the ACT database.

Bronze: Students earning a Bronze certificate are judged to be ready for 16 percent of jobs.

In Alabama students earning a Silver certificate or above are considered career ready.

2018 Assessment Results

The following charts show the percent of graduates in Alabama who demonstrated workforce readiness on WorkKeys assessments at the state, local system, and school level.

Percent Workforce Ready Remained the Same. The first chart shows that in 2018 64 percent of high school graduates in the state were deemed workforce ready as measured by WorkKeys. The percent steadily increased from 58.8 percent in 2015 to 60.8 percent in 2016 and 63.5 percent in 2017. The increase from 2017 to 2018 was comparatively small, with both years rounding at 64 percent workforce ready.

Workforce Ready at the System Level  

Listed below are the top ranked systems based on workforce readiness assessed through WorkKeys:

  • Hartselle City – 94 percent of students
  • Mountain Brook – 91 percent of students
  • Cullman – 88 percent of students
  • Oneonta – 86 percent of students
  • Guntersville -85 percent of students

There does appear to be some correlation between performance on the WorkKeys and the ACT exam, but not an exact one-to-one match. For example, some systems achieved a comparable state ranking on both sets of assessments:

  • Mountain Brook was number 1 on the ACT and 2 on WorkKeys
  • Shelby County was 8th on the ACT and 8th on WorkKeys.

However, other systems saw a larger separation.

  • Vestavia Hills was 2nd on the ACT and 16th on WorkKeys.
  • Madison City ranked 4th on the ACT and 18th on WorkKeys.
  • Trussville was 6 and 25, respectively.

All of the systems in the top 10 on the ACT are in the top 25 on WorkKeys, except Auburn.

At the same time, a number of less affluent systems demonstrated progress on the WorkKeys assessment over the previous year. Those systems showing the most improvement over 2017 included:

  • Perry County – 28 percent increase
  • Elba – 27 percent increase
  • Alexander City – 25 percent increase
  • Thomasville – 21 percent increase
  • Sheffield – 18 percent increase

Change in Certificate Levels

Significant Growth in Platinum and Gold Certificates. Students are deemed workforce ready if they achieve certification at the Platinum, Gold, or Silver levels. The charts show that the percent at each of these levels from 2015 through 2017 increased moderately each year and the distribution of students across the different levels remained about the same.  However, ACT’s decision to change one of the tests for a new one apparently led to dramatic changes in the scoring of the WorkKeys test, producing far more Gold and Platinum level certificates:

  • Platinum certificates dramatically increased from essentially zero percent all three previous years to 10 percent of students in 2018. 
  • Gold certificates, remaining fairly stable around 15 percent in the previous three years also showed a more significant increase to 19 percent in 2018.
  • Silver certificates dropped to 35 percent in 2018 after increasing to from 43 percent in 2015 to 48 percent in 2017.

Overall, this resulted in roughly the same percent workforce ready, but with a positive trend toward higher certification levels.  Furthermore, while the percent of Bronze certificates increased, the percent with no certificate decreased. This is a positive trend with more students edging toward the readiness threshold.  

Change in Certificate Levels at the System Level

Platinum: None of the systems decreased at the Platinum level. While a fair number showed no growth, the vast majority increased. In 2017, Sheffield generated the highest percent of Platinum level students of any system in the state at only two percent of its students. All other systems were at zero or one percent. In 2018, Mountain Brook increased from one percent to 41 percent, followed by Homewood, which increased from one percent to 29 percent.  Each of the remaining schools in the 2018 top ten increased from one percent or less to 20-29 percent.  Sheffield increased from two to 11 percent. 

Gold: Cullman generated the highest percentage of Gold Certificates, followed by Hartselle. The percentage receiving Gold increased for most systems, though not at the level of change experienced for Platinum. Fourteen systems generated fewer Gold Certificates in 2018, including some of the top academic systems. Supposedly more of their high performing students moved into the Platinum level. Statewide, Thomasville generated the highest increase in Gold Certificates, moving them into the top five in overall state rankings.

Silver: Finally, most systems decreased in the percentage of students receiving Silver Certificates, the threshold for being considered workforce ready. The highest gains were in Perry County and Elba.  The highest decreases were in Mountain Brook, Jasper, Cullman, Marion County, and Brewton. Rounding this out, the percent of students receiving Bronze Certificates increased in the majority of systems, while the majority of systems have a lower percentage of students who did not receive a certificate.

Possible Causes for the Change

A variety of explanations can be considered for the changes in WorkKeys results:

  • Changes in the WorkKeys assessments;
  • Stronger alignment between WorkKeys teacher training, test preparation, and test questions; and
  • Stronger concerted efforts in schools to prepare students for the assessments.

In 2018 WorkKeys underwent a number of changes, though the only test section that involved significant content change was the Locating Information test, which is now called Graphic Literacy. The names used for the other two assessments were changed to their current titles, Applied Math and Workplace Documents, though apparently no significant content changes occurred in these assessments. 

Changes in an assessment often lead to scoring changes and other issues that can affect results. The new Graphic Literacy test may account for the leap in higher certificates at the Platinum and Gold levels, but the new test is supposed to be more rigorous. While higher rigor would usually not be associated with higher scores, higher relevance in an improved test could produce better scores.

Aligned with the changes in the actual assessments are changes in teacher training and student prep tools, including practice exams. These are potentially a better fit with the formal assessments being rolled out than was available in preparation for the prior assessment.

More systems may also be using the ACT WorkKeys Curriculum, which is aligned with the WorkKeys assessments. The courses are delivered through a mobile-based learning management system. It provides students and teachers with a customized study schedule and detailed instructional content. While the curriculum can improve test performance, it is primarily designed to develop workplace-ready skills in students.

Subgroup Analysis

Analysis of WorkKeys results for student subgroup performance shows continuing disparity between subgroups. Use the filters to see how systems differ in subgroup performance. Some schools may be better at assisting struggling groups than others.

In looking at trends, all racial groups are showing progress from year to year, especially Asians, Native Americans, and black students. The gap between Asian students and all other races is growing. The gap between white and Hispanic students is also growing, while the gap between white and black students has remained about the same – but not closing. Black students are gradually closing the gap with Hispanic students.


College and Career Readiness in Alabama

In 2012, the Alabama State Board of Education adopted Plan 2020, which embraced a vision for the state education system led by the motto: “Every child a graduate. Every graduate prepared.” The plan called for raising Alabama’s high school graduation rate to 90 percent, while at the same time producing graduates who are better prepared for college and the workplace. Since that time, significant progress occurred in raising the graduation rate from 72 percent in 2011 to 90 percent in 2018.

While the high graduation rate is laudable, state education leaders have raised concerns about the gap between the percent graduating and the percent prepared for college or work. The other half of the motto — “Every graduate prepared” — came under question.

The following chart shows Alabama high schools are closing the gap between the percentage of students graduating and the percentage of seniors demonstrating they are ready for college and the workforce.

According to yet-to-be finalized data from the Alabama State Department of Education, significant progress has been made over the past three years. Final and complete data are expected to be published later this year:

  • In 2016, Alabama graduated 87 percent of its students, though only 66 percent were college and career ready. 
  • In 2017, the gap closed, with 89 percent graduating and 71 percent college and career ready. 
  • In 2018, improvement continued with 90 percent graduating and 75 percent college and career ready. 

Though the gap is still large, it is improving.

Continuing to close that gap is vital. The state has a goal of adding 500,000 highly-skilled workers to the workforce by 2025. To meet that goal, virtually all high school graduates will need to be prepared for education beyond high school or prepared to enter the workforce directly after high school.

The 2018 CCR data shows:

  • Career Technical Education (CTE) certificates are the fastest-growing means for classifying students as college and career ready.
  • Qualifying scores on the ACT and WorkKeys assessments are the two most common measures used to classify students as college and career ready.
  • Systems and schools leverage different strategies for preparing students – reflecting varying strengths, resources, and goals for education.
  • Some systems are very strong in particular areas and weak in others, which may not meet the needs of all students.
  • Disparities in performance exist across schools and student subgroups that may go beyond poverty.

Alabama’s College and Career Readiness Measure

The Alabama College and Career Strategic Plan (a component of Plan 2020) articulated a vision in which all Alabama students graduate high school college and career ready. The plan defines college and career readiness as:  

“…a high school graduate [that] has the English and mathematics knowledge and skills necessary to either (1) qualify for and succeed in entry-level, credit-bearing college courses without the need for remedial coursework, or (2) qualify for and succeed in the postsecondary job training and/or education necessary for their chosen career (i.e. technical/vocational program, community college, apprenticeship or significant on-the-job training).”

High school graduates are classified as college and career ready (CCR) if they meet at least one of the following criteria.

  1. Score college ready in at least one subject on the ACT
  2. Score at the silver level or above on the WorkKeys Assessment
  3. Earn a passing score on an Advanced Placement or International Baccalaureate Exam (college-level courses delivered in high schools)
  4. Successfully earn a Career Technical Education credential
  5. Earn dual enrollment credit at a college or university
  6. Successfully enlist in the military

Some of these measures are more aligned with college preparation and others with career preparation.

The state now provides data on the overall CCR rate and data on the individual metrics that create the measure. Detailed analysis is found in the interactive charts below, which allow users to explore college and career readiness percentages for high school seniors in 2018 at the state, school system, and high school level.

Graduation and CCR Rates

The first chart shows the percentage of students graduating, followed by the percentage of seniors who are college and career ready, followed by the percentage achieving readiness on the various performance measures that compose the CCR rate.  While preset for the state in 2018, the filters can be used to produce the same chart for individual school systems in 2017.

Statewide, the percentage of seniors testing “ready” is highest for the WorkKeys assessment, followed closely by ACT. Those are the two main channels through which a CCR rating is achieved, though a growing number of students are deemed CCR by earning a credential in a Career Technical Education (CTE) field.  Earning college credit or a qualifying score on an AP exam are also used by a smaller percentage of students. A very low percentage of seniors achieve a CCR rating by getting a passing score on an International Baccalaureate exam or successfully enlisting in the military.  Exploring these measures at the system and school level suggests the use of different strategies across school systems reflects different goals for education, local needs and strengths, and characteristics of the community.  

Graduation – CCR Gap by Local System

Chart 2 shows the graduation rate, CCR rate, and gap between these two rates in each system. They are listed in the order of CCR rate from highest to lowest. The Piedmont City School System is No. 1 in the state for CCR — the only system where 100 percent of seniors are classified as college and career ready. The system also graduates close to 100 percent of its students.

Some systems actually have negative gaps where the percent of seniors who are college and career ready exceeds the percent who graduate after four years in high school. This includes the Coffee County System, and city systems in Opp, Arab, Satsuma, and Piedmont. 

On the other end of the spectrum, systems with the lowest CCR rates tend to have the highest gaps between graduation and CCR rates, though not always. 

Graduation – CCR Gap by School

Individual schools are showing similar trends, with wide disparity between high and low performing schools.

  • Keith Middle-High School in Dallas County graduates 90 percent of its students, matching the state average, but only 19 percent of its seniors are measured to be college and career ready.
  • Barbour County High School graduates 77 percent of its students, but only 12.5 percent of seniors are measured as college and career ready.

Having both high graduation rates and CCR rates indicates that the diplomas issued by those schools have credibility and value. Where graduation rates are high but CCR rates are low, there is cause for concern.

Individual Components of CCR by System and School

The remaining charts in this section display individual components of college and career readiness by system and school. The final chart shows the percent change in CCR rates from 2017 to 2018.

Overall for the state, the largest change occurred in the percent of students earning career technical education credentials, increasing from 22 to 29 percent. The state and individual systems have put an increased focus on providing career-related coursework in high school, and the increases here may reflect that emphasis. At the same time, it is important for policy-makers to monitor what career credentials students are earning. For this to be a meaningful measure of career readiness, those credentials need to be recognized and valued by employers and should be in a field in which a student is likely to obtain work or more advanced training.

Other increases occurred in students earning college credit (10% to 13%), and in WorkKeys readiness (55% to 57%).

As cited earlier, when looking at individual systems and schools, it becomes apparent that different places achieve college and career readiness through different strategies.

  • Mountain Brook is No. 1 in ACT, WorkKeys, and AP, but lower on college credit and career technical credentials.
  • Vestavia is second on ACT readiness, among the top schools on the AP exam, and in the top 25 percent in career technical credentials, but much lower on WorkKeys and dual enrollment.
  • Opp City exceeds the state average on all measures of CCR but is especially high-achieving in students earning college credit, where they are No. 1 in the state, and in career technical credentials, where they are No. 5. 
  • The system with the highest CCR rating, Piedmont City Schools, is at the state average on ACT and WorkKeys, but far and above other systems in credentials.

Are schools meeting the needs of all students? One concern this analysis raises is that some systems may not be meeting the varying needs of all students. Those systems scoring high or at least moderately high on a balance of college and career measures are providing a breadth of services that can help students shine where they show interest and potential. The lack of balance in some systems or schools may reflect an intentional emphasis on what they value most: college preparation or career readiness. It is important for schools to assess whether they are providing options that fit the needs and interests of the diverse array of students they serve.

Conclusion

As a composite of various academic and career indicators, Alabama’s College and Career Ready metric reflects three important concepts.

  • Every student needs either a post-secondary education or credible career-focused training in high school.
  • Post-secondary education need not be a traditional four-year college degree.
  • There are many different pathways for students.

The gap between Alabama’s graduation rate and the number of graduates deemed college and career ready has been a concern, but one with optimism given progress in closing that gap.

Career Technical Education certificates are the fastest-growing measure through which students are earning the CCR marker. These credentialing programs are meant to prepare students for workforce opportunities in high-demand fields right out of high school.  They combine academics with work-based learning as a strategy to address the widening gap between job applicants’ skills and the skills employers need. The state will need to continuously ensure that all courses and concentrations are of high quality and relevant to the workforce needs in the state and in local communities.  

Beyond preparing students with skills for specific jobs, an array of academic, extracurricular, and work-based learning opportunities can develop the student as a whole person capable of thoughtful decision-making and meet the unique needs and preferences of each student. Academics, career training, life skills, and the cultivation of passions and interests can all come together to support college, career, and life readiness.

Alabama is assessing progress on part of this, but not all. The state has made a good faith effort to evaluate college and career readiness through a variety of measures such as the ACT, college dual enrollment, WorkKeys, and Career and Technical Education (CTE) certification. Still, this is a changing and growing field. Skills and attributes needed in various careers are continually changing. Alabama should remain alert to more rigorous and authentic measures of college and career readiness that may emerge.