ACT WorkKeys – An Assessment of Workforce Readiness Among High School Graduates in Alabama

The WorkKeys Assessment is a standardized test given to 12th graders in Alabama public schools. The assessment is meant to measure skills relevant to many of today’s work environments.

In 2018:

  • 64 percent of Alabama high school graduates in 2018 were deemed workforce ready as measured by the ACT WorkKeys assessment, a year over year improvement of a half percent.
  • At 94 percent, Hartselle had the highest percentage of workforce ready graduates, as measured by WorkKeys.
  • High fluctuations occurred among the different certificate levels, with Platinum (highest level) and Gold dramatically increasing, but Silver decreasing. Bronze (below workforce ready) increased, though the percent not earning a certificate decreased.

What is WorkKeys?

The WorkKeys assessments are meant to provide a meaningful assessment of applied cognitive skills useful in contemporary work settings. It is also one of the components of Alabama’s College and Career Ready measure.

The assessments do not measure a student’s attitudes about work, dependability, interpersonal skills, teamwork, communication skills, or instincts for creativity, innovation, or leadership. They also do not provide insight about a student’s competency for a job requiring specialized knowledge and skills.

The Assessments. The assessments consist of three tests of applied cognitive skills which are relevant, according to ACT’s research, to over 20,000 occupations:

  • The Applied Math test measures critical thinking, mathematical reasoning, and problem-solving techniques for situations in today’s workplace.
  • The Graphic Literacy test measures the skill needed to locate, synthesize, and use information from charts and graphs. 
  • The Workplace Documents test measures the skills needed to read and understand written text such as memos, letters, directions, signs, notices, bulletins, policies, and regulations on the job.

Students are awarded a National Career Readiness Certification in they score a Platinum, Gold, Silver, or Bronze score on the WorkKeys.

Platinum: These are students with the highest level of applied cognitive skills.  According to ACT, students at this level have demonstrated applied foundational skills for 96 percent of the occupations in the ACT jobs dataset.

Gold: Those earning a Gold level certificate should have the applied foundational skills for 90 percent of jobs in the database.

Silver: Students scoring at the Silver level should have the applied foundational skills for 71 percent of jobs in the ACT database.

Bronze: Students earning a Bronze certificate are judged to be ready for 16 percent of jobs.

In Alabama students earning a Silver certificate or above are considered career ready.

2018 Assessment Results

The following charts show the percent of graduates in Alabama who demonstrated workforce readiness on WorkKeys assessments at the state, local system, and school level.

Percent Workforce Ready Remained the Same. The first chart shows that in 2018 64 percent of high school graduates in the state were deemed workforce ready as measured by WorkKeys. The percent steadily increased from 58.8 percent in 2015 to 60.8 percent in 2016 and 63.5 percent in 2017. The increase from 2017 to 2018 was comparatively small, with both years rounding at 64 percent workforce ready.

Workforce Ready at the System Level  

Listed below are the top ranked systems based on workforce readiness assessed through WorkKeys:

  • Hartselle City – 94 percent of students
  • Mountain Brook – 91 percent of students
  • Cullman – 88 percent of students
  • Oneonta – 86 percent of students
  • Guntersville -85 percent of students

There does appear to be some correlation between performance on the WorkKeys and the ACT exam, but not an exact one-to-one match. For example, some systems achieved a comparable state ranking on both sets of assessments:

  • Mountain Brook was number 1 on the ACT and 2 on WorkKeys
  • Shelby County was 8th on the ACT and 8th on WorkKeys.

However, other systems saw a larger separation.

  • Vestavia Hills was 2nd on the ACT and 16th on WorkKeys.
  • Madison City ranked 4th on the ACT and 18th on WorkKeys.
  • Trussville was 6 and 25, respectively.

All of the systems in the top 10 on the ACT are in the top 25 on WorkKeys, except Auburn.

At the same time, a number of less affluent systems demonstrated progress on the WorkKeys assessment over the previous year. Those systems showing the most improvement over 2017 included:

  • Perry County – 28 percent increase
  • Elba – 27 percent increase
  • Alexander City – 25 percent increase
  • Thomasville – 21 percent increase
  • Sheffield – 18 percent increase

Change in Certificate Levels

Significant Growth in Platinum and Gold Certificates. Students are deemed workforce ready if they achieve certification at the Platinum, Gold, or Silver levels. The charts show that the percent at each of these levels from 2015 through 2017 increased moderately each year and the distribution of students across the different levels remained about the same.  However, ACT’s decision to change one of the tests for a new one apparently led to dramatic changes in the scoring of the WorkKeys test, producing far more Gold and Platinum level certificates:

  • Platinum certificates dramatically increased from essentially zero percent all three previous years to 10 percent of students in 2018. 
  • Gold certificates, remaining fairly stable around 15 percent in the previous three years also showed a more significant increase to 19 percent in 2018.
  • Silver certificates dropped to 35 percent in 2018 after increasing to from 43 percent in 2015 to 48 percent in 2017.

Overall, this resulted in roughly the same percent workforce ready, but with a positive trend toward higher certification levels.  Furthermore, while the percent of Bronze certificates increased, the percent with no certificate decreased. This is a positive trend with more students edging toward the readiness threshold.  

Change in Certificate Levels at the System Level

Platinum: None of the systems decreased at the Platinum level. While a fair number showed no growth, the vast majority increased. In 2017, Sheffield generated the highest percent of Platinum level students of any system in the state at only two percent of its students. All other systems were at zero or one percent. In 2018, Mountain Brook increased from one percent to 41 percent, followed by Homewood, which increased from one percent to 29 percent.  Each of the remaining schools in the 2018 top ten increased from one percent or less to 20-29 percent.  Sheffield increased from two to 11 percent. 

Gold: Cullman generated the highest percentage of Gold Certificates, followed by Hartselle. The percentage receiving Gold increased for most systems, though not at the level of change experienced for Platinum. Fourteen systems generated fewer Gold Certificates in 2018, including some of the top academic systems. Supposedly more of their high performing students moved into the Platinum level. Statewide, Thomasville generated the highest increase in Gold Certificates, moving them into the top five in overall state rankings.

Silver: Finally, most systems decreased in the percentage of students receiving Silver Certificates, the threshold for being considered workforce ready. The highest gains were in Perry County and Elba.  The highest decreases were in Mountain Brook, Jasper, Cullman, Marion County, and Brewton. Rounding this out, the percent of students receiving Bronze Certificates increased in the majority of systems, while the majority of systems have a lower percentage of students who did not receive a certificate.

Possible Causes for the Change

A variety of explanations can be considered for the changes in WorkKeys results:

  • Changes in the WorkKeys assessments;
  • Stronger alignment between WorkKeys teacher training, test preparation, and test questions; and
  • Stronger concerted efforts in schools to prepare students for the assessments.

In 2018 WorkKeys underwent a number of changes, though the only test section that involved significant content change was the Locating Information test, which is now called Graphic Literacy. The names used for the other two assessments were changed to their current titles, Applied Math and Workplace Documents, though apparently no significant content changes occurred in these assessments. 

Changes in an assessment often lead to scoring changes and other issues that can affect results. The new Graphic Literacy test may account for the leap in higher certificates at the Platinum and Gold levels, but the new test is supposed to be more rigorous. While higher rigor would usually not be associated with higher scores, higher relevance in an improved test could produce better scores.

Aligned with the changes in the actual assessments are changes in teacher training and student prep tools, including practice exams. These are potentially a better fit with the formal assessments being rolled out than was available in preparation for the prior assessment.

More systems may also be using the ACT WorkKeys Curriculum, which is aligned with the WorkKeys assessments. The courses are delivered through a mobile-based learning management system. It provides students and teachers with a customized study schedule and detailed instructional content. While the curriculum can improve test performance, it is primarily designed to develop workplace-ready skills in students.

Subgroup Analysis

Analysis of WorkKeys results for student subgroup performance shows continuing disparity between subgroups. Use the filters to see how systems differ in subgroup performance. Some schools may be better at assisting struggling groups than others.

In looking at trends, all racial groups are showing progress from year to year, especially Asians, Native Americans, and black students. The gap between Asian students and all other races is growing. The gap between white and Hispanic students is also growing, while the gap between white and black students has remained about the same – but not closing. Black students are gradually closing the gap with Hispanic students.


College and Career Readiness in Alabama

In 2012, the Alabama State Board of Education adopted Plan 2020, which embraced a vision for the state education system led by the motto: “Every child a graduate. Every graduate prepared.” The plan called for raising Alabama’s high school graduation rate to 90 percent, while at the same time producing graduates who are better prepared for college and the workplace. Since that time, significant progress occurred in raising the graduation rate from 72 percent in 2011 to 90 percent in 2018.

While the high graduation rate is laudable, state education leaders have raised concerns about the gap between the percent graduating and the percent prepared for college or work. The other half of the motto — “Every graduate prepared” — came under question.

The following chart shows Alabama high schools are closing the gap between the percentage of students graduating and the percentage of seniors demonstrating they are ready for college and the workforce.

According to yet-to-be finalized data from the Alabama State Department of Education, significant progress has been made over the past three years. Final and complete data are expected to be published later this year:

  • In 2016, Alabama graduated 87 percent of its students, though only 66 percent were college and career ready. 
  • In 2017, the gap closed, with 89 percent graduating and 71 percent college and career ready. 
  • In 2018, improvement continued with 90 percent graduating and 75 percent college and career ready. 

Though the gap is still large, it is improving.

Continuing to close that gap is vital. The state has a goal of adding 500,000 highly-skilled workers to the workforce by 2025. To meet that goal, virtually all high school graduates will need to be prepared for education beyond high school or prepared to enter the workforce directly after high school.

The 2018 CCR data shows:

  • Career Technical Education (CTE) certificates are the fastest-growing means for classifying students as college and career ready.
  • Qualifying scores on the ACT and WorkKeys assessments are the two most common measures used to classify students as college and career ready.
  • Systems and schools leverage different strategies for preparing students – reflecting varying strengths, resources, and goals for education.
  • Some systems are very strong in particular areas and weak in others, which may not meet the needs of all students.
  • Disparities in performance exist across schools and student subgroups that may go beyond poverty.

Alabama’s College and Career Readiness Measure

The Alabama College and Career Strategic Plan (a component of Plan 2020) articulated a vision in which all Alabama students graduate high school college and career ready. The plan defines college and career readiness as:  

“…a high school graduate [that] has the English and mathematics knowledge and skills necessary to either (1) qualify for and succeed in entry-level, credit-bearing college courses without the need for remedial coursework, or (2) qualify for and succeed in the postsecondary job training and/or education necessary for their chosen career (i.e. technical/vocational program, community college, apprenticeship or significant on-the-job training).”

High school graduates are classified as college and career ready (CCR) if they meet at least one of the following criteria.

  1. Score college ready in at least one subject on the ACT
  2. Score at the silver level or above on the WorkKeys Assessment
  3. Earn a passing score on an Advanced Placement or International Baccalaureate Exam (college-level courses delivered in high schools)
  4. Successfully earn a Career Technical Education credential
  5. Earn dual enrollment credit at a college or university
  6. Successfully enlist in the military

Some of these measures are more aligned with college preparation and others with career preparation.

The state now provides data on the overall CCR rate and data on the individual metrics that create the measure. Detailed analysis is found in the interactive charts below, which allow users to explore college and career readiness percentages for high school seniors in 2018 at the state, school system, and high school level.

Graduation and CCR Rates

The first chart shows the percentage of students graduating, followed by the percentage of seniors who are college and career ready, followed by the percentage achieving readiness on the various performance measures that compose the CCR rate.  While preset for the state in 2018, the filters can be used to produce the same chart for individual school systems in 2017.

Statewide, the percentage of seniors testing “ready” is highest for the WorkKeys assessment, followed closely by ACT. Those are the two main channels through which a CCR rating is achieved, though a growing number of students are deemed CCR by earning a credential in a Career Technical Education (CTE) field.  Earning college credit or a qualifying score on an AP exam are also used by a smaller percentage of students. A very low percentage of seniors achieve a CCR rating by getting a passing score on an International Baccalaureate exam or successfully enlisting in the military.  Exploring these measures at the system and school level suggests the use of different strategies across school systems reflects different goals for education, local needs and strengths, and characteristics of the community.  

Graduation – CCR Gap by Local System

Chart 2 shows the graduation rate, CCR rate, and gap between these two rates in each system. They are listed in the order of CCR rate from highest to lowest. The Piedmont City School System is No. 1 in the state for CCR — the only system where 100 percent of seniors are classified as college and career ready. The system also graduates close to 100 percent of its students.

Some systems actually have negative gaps where the percent of seniors who are college and career ready exceeds the percent who graduate after four years in high school. This includes the Coffee County System, and city systems in Opp, Arab, Satsuma, and Piedmont. 

On the other end of the spectrum, systems with the lowest CCR rates tend to have the highest gaps between graduation and CCR rates, though not always. 

Graduation – CCR Gap by School

Individual schools are showing similar trends, with wide disparity between high and low performing schools.

  • Keith Middle-High School in Dallas County graduates 90 percent of its students, matching the state average, but only 19 percent of its seniors are measured to be college and career ready.
  • Barbour County High School graduates 77 percent of its students, but only 12.5 percent of seniors are measured as college and career ready.

Having both high graduation rates and CCR rates indicates that the diplomas issued by those schools have credibility and value. Where graduation rates are high but CCR rates are low, there is cause for concern.

Individual Components of CCR by System and School

The remaining charts in this section display individual components of college and career readiness by system and school. The final chart shows the percent change in CCR rates from 2017 to 2018.

Overall for the state, the largest change occurred in the percent of students earning career technical education credentials, increasing from 22 to 29 percent. The state and individual systems have put an increased focus on providing career-related coursework in high school, and the increases here may reflect that emphasis. At the same time, it is important for policy-makers to monitor what career credentials students are earning. For this to be a meaningful measure of career readiness, those credentials need to be recognized and valued by employers and should be in a field in which a student is likely to obtain work or more advanced training.

Other increases occurred in students earning college credit (10% to 13%), and in WorkKeys readiness (55% to 57%).

As cited earlier, when looking at individual systems and schools, it becomes apparent that different places achieve college and career readiness through different strategies.

  • Mountain Brook is No. 1 in ACT, WorkKeys, and AP, but lower on college credit and career technical credentials.
  • Vestavia is second on ACT readiness, among the top schools on the AP exam, and in the top 25 percent in career technical credentials, but much lower on WorkKeys and dual enrollment.
  • Opp City exceeds the state average on all measures of CCR but is especially high-achieving in students earning college credit, where they are No. 1 in the state, and in career technical credentials, where they are No. 5. 
  • The system with the highest CCR rating, Piedmont City Schools, is at the state average on ACT and WorkKeys, but far and above other systems in credentials.

Are schools meeting the needs of all students? One concern this analysis raises is that some systems may not be meeting the varying needs of all students. Those systems scoring high or at least moderately high on a balance of college and career measures are providing a breadth of services that can help students shine where they show interest and potential. The lack of balance in some systems or schools may reflect an intentional emphasis on what they value most: college preparation or career readiness. It is important for schools to assess whether they are providing options that fit the needs and interests of the diverse array of students they serve.

Conclusion

As a composite of various academic and career indicators, Alabama’s College and Career Ready metric reflects three important concepts.

  • Every student needs either a post-secondary education or credible career-focused training in high school.
  • Post-secondary education need not be a traditional four-year college degree.
  • There are many different pathways for students.

The gap between Alabama’s graduation rate and the number of graduates deemed college and career ready has been a concern, but one with optimism given progress in closing that gap.

Career Technical Education certificates are the fastest-growing measure through which students are earning the CCR marker. These credentialing programs are meant to prepare students for workforce opportunities in high-demand fields right out of high school.  They combine academics with work-based learning as a strategy to address the widening gap between job applicants’ skills and the skills employers need. The state will need to continuously ensure that all courses and concentrations are of high quality and relevant to the workforce needs in the state and in local communities.  

Beyond preparing students with skills for specific jobs, an array of academic, extracurricular, and work-based learning opportunities can develop the student as a whole person capable of thoughtful decision-making and meet the unique needs and preferences of each student. Academics, career training, life skills, and the cultivation of passions and interests can all come together to support college, career, and life readiness.

Alabama is assessing progress on part of this, but not all. The state has made a good faith effort to evaluate college and career readiness through a variety of measures such as the ACT, college dual enrollment, WorkKeys, and Career and Technical Education (CTE) certification. Still, this is a changing and growing field. Skills and attributes needed in various careers are continually changing. Alabama should remain alert to more rigorous and authentic measures of college and career readiness that may emerge.


How Do States Choose State Boards of Education and State Superintendents?

The Alabama Legislature is currently considering a bill that would potentially overhaul state education governance and policymaking as it relates to K—12 public schools in the state. The bill, which has already passed the Senate, would abolish the elected State Board of Education and the Board-appointed position of State Superintendent of Education. The bill would create a Governor-appointed Commission, the Alabama Commission on Elementary and Secondary Education. The Commission would appoint a Secretary of Elementary and Secondary Education to be approved by the Senate. If approved by the Legislature, the proposal will be placed on the statewide ballot as an amendment to the Alabama Constitution.  

State commissions or boards of education and chief executive officers, whether they be superintendents or secretaries are central to state education governance. The process used for their selection has implications for accountability, decision-making, and setting priorities for a state’s K—12 education system.

All but two states (Minnesota and Missouri) have a school board or commission. Eleven states have elected school boards (12 if including the district of Columbia). The rest have appointed boards, most of which are appointed by the governor. States with elected school boards or commissions are listed below.

States with Elected Boards

  • Alabama
  • Colorado
  • Kansas
  • Louisiana
  • Michigan
  • Nebraska
  • New Mexico – only an advisory group
  • Nevada – Mixture of elected and appointed members
  • Ohio
  • Texas
  • Utah

The proposal before the Legislature resembles the governance structure currently in place in 12 states in which the governor appoints the school board and the board appoints the superintendent.

Education Governance

Why does this matter? States have the responsibility for implementing federal education law and developing, implementing, and managing state-level policies. For this to work well, several institutions must work well together.   

State legislatures must pass effective legislation.

Governors can propose education legislation and have the statutory authority to approve or veto legislation. As the state’s chief executive, the governor carries out the laws passed by the legislature. They can also play an important role in shaping the priorities of a state board when they have the power to appoint. In some states, as in Alabama, the governor serves as president of the school board.

State boards of education are responsible for statewide curriculum standards; high school graduation requirements; qualifications for professional education personnel; state accountability and assessment programs; standards for accreditation of local school districts; preparation programs for teachers and administrators; administration of federal assistance programs; and the development of rules and regulations for the administration of state programs. State boards are often seen as the lay representative of the state’s population and as the liaison between professional educators and policymakers. Boards should play a role as advocates for education and, in some states, have been influential in building consensus on state education policy.[1]  

Finally, state superintendents are responsible for administrative oversight of state education agencies and implementation of state law and board policies. Policy making can occur as superintendents interpret laws and policies they are responsible for implementing.

A report issued by the Education Commission of the States (ECS) outlines how state boards and state superintendents are chosen in varying states, and how the majority of states fall into one of four models. These models, described below, have implications for how state leaders work together in setting priorities and implementing policies.[2]

Model 1: Governor Appoints Board and Superintendent

In 10 states (Delaware, Iowa, Maine, New Hampshire, New Jersey, Pennsylvania, South Dakota, Tennessee, Vermont, and Virginia), the governor has the most power in setting priorities and ensuring they are implemented.

Consequently, the superintendent and board should both be aligned with the governor, though the superintendent may feel more independent of the board than in other models where the board appoints them. The governor is accountable to the voters and can be held more directly accountable for the status and effectiveness of education in the state

Model 2: Governor Appoints Board and Board Appoints Superintendent

In 12 states (Alaska, Arkansas, Connecticut, Florida, Hawaii, Illinois, Kentucky, Maryland, Massachusetts, Missouri, Rhode Island, and West Virginia), the governor still has power in shaping the education agenda but has less direct control over the implementation of policies through the superintendent’s office, as compared to Model 1. The Board and Superintendent would potentially have a closer relationship than found in Model 1. The proposed Commission in Alabama fits this model.

Model 3:  Governor Appoints Board while Superintendent is Elected

In 10 states (Arizona, California, Georgia, Idaho, Indiana, Montana, North Carolina, North Dakota, Oklahoma, and Wyoming), voters may see different platforms for education supported by the governor and the superintendent. The governor appoints the board, and this becomes a channel through which policy is formed. The superintendent may exercise more autonomy in interpreting those policies and how they will be implemented in the state.

Model 4:  Board is Elected and Appoints the Superintendent

In Alabama and five other states (Colorado, Kansas, Michigan, Nebraska, and Utah), the governor and the board are both directly accountable to voters. Since the board appoints the superintendent, this increases their power. In this model, the governor is likely in the weakest position to craft or control the education agenda, compared to the other models. According to ECS, states using this model potentially face stronger challenges aligning and collaborating across state leadership, unless the voting public is clear in its desires. When alignment is not present, states will likely face limitations in pushing for ambitious policy changes. At the same time, an elected board will be highly responsive to voters and will seek out their opinion, preferences, and needs.

According to ECS, twelve additional states (Louisiana, Minnesota, Mississippi, New Mexico, New York, Nevada, Ohio, Oregon, South Carolina, Texas, Washington and Wisconsin function under modified versions of the above models. Five of these elect their board, though New Mexico’s is only an advisory commission. Two of these states elect their state superintendent. No state elects both their state board and superintendent.

Elected and Appointed State Boards: Strengths and Weakness

The process for selecting the board and superintendent can influence the goals for these officials. Some groups, including the National Association of State Boards of Education (NASBE), emphasize the role of state boards in representing the interests of the lay public in accordance with democratic principles. Whether elected or not, NASBE contends that:

“State boards of education are integral to the governance of public education in the United States. State Boards, operating as a lay body over state education, are intended to serve as an unbiased broker for education decision-making, focusing on the big picture, articulating the long-term vision and needs of public education, and making policy based on the best interests of the public and the young people of America.”

Elected board members are charged with asking important and challenging questions that lead to good policy. The question is whether boards effectively play this role.

Proponents say elected boards are more responsive to the public will. As elected officials, board members have their rightful place and, ideally, are only responsible to the people who elected them. They should be more empowered to oppose what they believe is not in the interests of the state’s schools and children.

At the same time, as elected officials, re-election is an important goal, if not the central goal. Thus elected board members may find themselves where the interests and desires of voters conflict with policies, programs, and practices that best serve children.

Conversely, proponents of appointed boards cite the strength of the vetting process in creating boards with knowledgeable, skilled, effective board members. An appointment process allows the governor to consider the needs of the board and the qualities different candidates would bring. Others cite that governor-appointed boards and appointed superintendents create a more efficient, aligned, and harmonious system for setting and implementing education priorities. Ambitious and substantive changes to a state’s school system are more feasible in a more efficient system that encourages collaboration and strengthens the governor’s capacity to effect change. However, while somewhat insulated, appointed boards are not immune from political pressure. 

Conclusion

The selection process for state school boards and state superintendents is important, and there are reasonable arguments for both elections and appointments. Regardless, the selection process will not remove politics. The nature of the task — setting and implementing the state’s K—12 education policy — means state school boards will likely always be politicized to some degree.

Thus, it is essential to establish both an effective governing structure and qualified leaders committed to strengthening teaching and learning in Alabama.

A quality education is how dreams are realized and the people’s voice is strengthened.


References

[1] National Association of State Boards of Education (2019). “State Board of Education Responsibilities”

[2] Education Commission of the States (2017). 50-State Comparison: K-12 Governance Structures.”


How Are Alabama Students Performing? 2018 Scantron Results

Less than half of Alabama public school students in grades 3-8 scored proficient on grade level tests of reading and math in 2018, according to the test data released early this year by the Alabama State Department of Education.

Considering all results across the grades, 48 percent of students were proficient in math and 46 percent were proficient in reading. Only in 3rd-grade math did a majority of students, 57.6 percent, test above the benchmark for proficiency.

The overall proficiency rate for science, which is tested in the fifth and seventh grade, was much lower, at 38 percent. While selected local systems exceed expectations in science, the lower statewide average is a concern given science’s role in developing the future workforce.

New State Assessments

2018 was an important transition year from ACT Aspire, which has been the primary tool for measuring academic progress in Alabama public schools since the 2013-2014 school year. Back in 2014 ACT Aspire replaced the Alabama Reading and Math Test (ARMT) to provide a better measure of students’ grade-level proficiency on a path to college readiness. While Aspire was applauded for being a more “honest” assessment of student performance and for its functional tie to the commonly used ACT college readiness test, Aspire also generated criticism for technical difficulties in test administration and results that were not consistently delivered in a timely manner.

In June 2017, the State Board of Education voted to cancel Alabama’s contract with ACT and move toward developing another set of tests, scheduled to be launched in the 2019-20 school year. In the meantime, the state has continued to assess performance in grades 3-8 using tests provided by a testing company named Scantron, which has experience working with local school systems in Alabama. Initial results on Scantron were higher but the scores were mathematically adjusted to provide continuity with Aspire results. For more information on the history and issues shaping this transition, see PARCA’s report, Student Achievement Matters: the Future of Assessment is Now.

Before diving more deeply into the 2017-18 Scantron results, let’s quickly step back and see how proficiency rates across these tests compare.

Scantron scores are from 2018, ACT Aspire from 2017, ARMT from 2013, and the NAEP trendline from 2013 through 2017. Though Scantron and Aspire are different tests, this chart shows that proficiency levels are somewhat comparable, especially when compared to ARMT and NAEP. The percent proficient in NAEP is lower than found in all Alabama state tests. Given the respect accorded to NAEP as a national baseline, the extremely high proficiency rates from ARMT showed signs of a watered down test, which failed to serve as a respectable measure of authentic proficiency and progress in building college and work readiness. Results from ACT Aspire proved to be more comparable to NAEP, as do the proficiency levels from Scantron in 2018. When the new set of tests are launched in 2019, the results will establish a new baseline from which future improvement and growth can be measured.

Scantron Results for 2018

Comparing Subjects. Among the three subjects assessed, math generated the highest percentage of students scoring at or above proficiency (48 percent) across grades 3-8 combined. Reading was not far behind at 46 percent proficient. It was also true of the Aspire that students in grades 3-8 combined demonstrated higher proficiency in math than in reading. Science was only tested in the 5th and 7th grades and saw the lowest proficiency scores among the three subject on the 2018 Scantron tests, with only 38 percent proficient. With ACT Aspire, the percent proficient ranged from 35 to 39 percent.

Comparing Grade Levels. On Scantron, proficiency levels are generally higher in the lower grades. The highest single average is for 3rd-grade math at 58 percent proficient. The average across all grades for math, not counting the 3rd grade, is 45 percent. The average for reading, not counting 3rd grade, is now slightly higher than math at 46 percent. During the early grades, math and reading concern basic skills, but in the higher grades, math becomes more complex and introduces algebra. Reading instruction also becomes more demanding as students move from learning to read to reading for understanding.

The higher growth in reading from grade to grade is only marginal but might suggest that either students are more effectively learning the basics in reading as a foundation for later grades, or it may mean the material in math becomes a harder to master. By the time Alabama’s students reach high school, their college ACT scores in reading are higher than in math, and proficiency scores on the state Aspire test for 10th graders significantly drop, as low as 19 percent proficient in 2016-17.

The percent of students proficient in science drops from 40 percent of 5th graders to 36 percent of 7th graders. The low scores in science are worthy of concern. In addition to important content, science relates to problem-solving skills, reasoning, curiosity, critical thinking, good measurement skills, and applied learning. These are foundational skills for the workforce of tomorrow in Alabama.

Comparing Systems and Schools – How Did Your School Do? The tabs above list the percent proficient in each subject for all local systems and schools. Click on any one of those tabs and look up schools or local systems most important to you and see how they compare to other systems.

Clustering occurs where systems and schools perform similarly in all three subjects, especially among the wealthiest systems and poorest systems, but you can also find variation. For example, Sheffield City System is below the state average in science, with only 28 percent of its students demonstrating proficiency, and below in reading, with 39 percent proficient, but right at the state average in math, with 48 percent proficient. It would be useful to learn what is causing these differences and why some systems and schools are stronger in some subjects than others. Note that the vertical line shown in these charts represents the state average, providing a quick way to see how each system or school compares to the state as a whole.

In Alabama and across the country, differences in proficiency rates among various subgroups of students remains a concern.

  • Similar to Aspire, the percentage of students who are economically disadvantaged scoring proficient is about 25 percentage points lower than the percentage of non– economically disadvantaged students scoring proficient. This wide gap is the same across all three subjects, and holds up nationally as well.
  • At the same time, when comparing all subgroups, students who are African-American, English Learners, and in special education all perform at a lower level than do economically disadvantaged students as a group. Latinos, except those who are English learners, consistently attain higher levels of proficiency than African Americans. Among all the sub-groups English Learners appear to be struggling the most. They are ranked low in all subjects, especially science and reading – where they fall below all racial groups, economically disadvantaged students, and students in special education.
  • Wide gaps continue to exist between African-American students and both Asian and White students, and between Latino students and both Asian and White students. Differences in parental education level and income are factors that help explain these gaps, and differences in the quality of schools can also make a difference. Asian students are the highest performing racial group in math, reading, and science. Both African-American and Latino students score their best in math.
  • Female students in the state performed higher than males in math and significantly higher in reading, while male students scored slightly higher in science.
  • Finally, military students are among the higher performing groups in the state, though PARCA research has found gaps among racial groups within the military.

Students growing up economically disadvantaged are less likely to be read to in the early years, are exposed to fewer words, and are more likely to be exposed to health problems that can affect their capacity to learn in school and perform on tests. The education level and income of a student’s parents becomes a significant predictor of performance on standardized tests such as Scantron.

But some schools are better equipped to help all students learn, and some do better with less.

The scatterplot charts found in this section show the general correlation between proficiency levels and poverty levels. A school system’s proficiency rate determines its vertical position on the chart — the higher on the chart, the higher the proficiency rate. A system’s percent of economically disadvantaged students, measured by the percentage of students qualifying for free meals under the National School Lunch Program, determines the system’s position on the horizontal axis. Systems with a higher percentage of economically disadvantaged students will to the left of the chart and those with a lower percentage will be to the right of the chart.

Exceeding Expectations or Falling Short. The line displayed in the scatterplot is the average proficiency level for a given level of poverty. Those systems and schools above the line are exceeding expectations given their percent of economically disadvantaged students, and those below the line are falling short of expectations. These charts show that systems with similar poverty levels often show very different proficiency levels. In other words, school systems can and do exceed expectations through effective teaching, student support, and school organization and culture.

As an example, consider the chart on Scantron math performance and percent poverty. Saraland is exceeding expectations with 75 percent of their students scoring math proficient while 37 percent are economically disadvantaged. In contrast, in Pike Road only 16 percent of students are economically disadvantaged but only 47 percent are considered proficient in math. You can learn about the scores of each of these systems by clicking on the circles in the chart.

Conclusion

As Alabama rolls out its next suite of assessments it will be important to establish a baseline and provide feedback on the validity, reliability and usefulness of the new assessments. In the meantime, Scantron provides an assessment of performance that appears to be roughly comparable to Aspire in identifying strengths and where more attention is needed. The gaps that exist between school systems and among student subgroups continues to be an area where more work is needed, especially if Alabama is to fulfill its vision as a state characterized by a vibrant, innovative and relevant workforce.