More Resources Letter on No Child Left Behind Implementation Progress

As evidenced by the diversity among the approved state accountability plans and state-consolidated applications, states have great flexibility in the design of their systems and implementation of particular No Child Left Behind (NCLB) provisions. Presented as a checklist of items, states considered many issues when designing accountability systems, providing options for parents and defining highly qualified teachers. The following list represents almost 40 separate issues under the control and responsibility of state and local education agencies.

Standards and Assessments

• Standards and assessments used to provide the substance for Adequate Yearly Progress (AYP) definitions
• Definition of advanced, proficient and basic achievement levels

Elements of the AYP Definition

• Minimum group size for accountability, participation and reporting
• Minimum group size for students with disabilities
• Definition of full academic year
• Definitions of major racial and ethnic subgroups
• Timeline for reaching 100 percent proficiency by 2013-2014 (i.e., how often the annual measurable objectives will increase)
• Achievement goals set by grade span for AYP decisions
• Uniform averaging procedure (i.e., how many years of data will be included in AYP decisions)
• Statistical procedures, such as confidence intervals
• Selection of other academic indicators and what it means to make AYP on each indicator
• Use of an index in making AYP decisions
• Scores of students not participating in assessments
• Use of results from early-administered assessments

Adequate Yearly Progress and State Accountability Systems

• Integration of AYP with previously existing state accountability systems
• Same-subject identification
• Format of report cards
• Publication dates for school, district, and state report cards
• Processes and timing for releasing AYP decisions to schools and the public
• System of rewards, sanctions and instructional interventions
• Use of state school-improvement funds

Students With Special Instructional Needs

• Alternate achievement standards for students with significant cognitive disabilities
• Off-level assessments (lower grade level assessments for students with disabilities)
• Accommodations and alternative assessments for LEP students
• Definition of LEP subgroup

AYP for Unique Schools

• Accountability for small schools
• Accountability for schools serving only grade levels for which assessments are not required

Definition of "Persistently Dangerous"

• State definition of persistently dangerous schools

Highly Qualified Teachers

• Content and achievement standards for rigorous state tests for new elementary school teachers
• Methods for determining subject matter competency for new secondary school teachers
• Development and use of a "high, objective, uniform State standard of evaluation" for current teachers
• State certification requirements
• Plan for increasing the number of highly qualified teachers
• Parent notification of teacher quality
• Use of Title II and other federal funds to meet these requirements
• Alternate means of achieving full certification and licensing
• State definition and evaluation of qualified paraprofessionals

Standards and assessments are the foundation on which state accountability systems are built. These critical ingredients are developed, designed, and determined by states. States have always had and continue to have at their discretion the determination of their content and achievement standards and the design of their assessments. At the high school level, for example, some states have designed their standards and assessment system around particular courses that all students must take (e.g., course-by-course standards for algebra and biology and end-of-course assessments). States using this approach include Tennessee, Mississippi, Oklahoma, Arkansas and Virginia. Other states have designed their high school standards and assessment systems more broadly (e.g., high school math and reading standards and high school math and reading assessments). This includes states such as Alaska, Michigan and Missouri.

Additionally, states have the flexibility to add student "stakes" to their standards and assessment systems (e.g., students must achieve a particular score on an assessment to advance to the next grade or to graduate from high school). Massachusetts requires students to pass the high school assessments as a condition of receiving a diploma. Colorado requires students to achieve at certain levels to be promoted to subsequent grades. It is important to note that student stakes are not a NCLB requirement--a point that often confuses local educators.

Finally, states have the flexibility in the naming of their achievement levels (e.g., basic, proficient, advanced; or not meeting standard, meeting standard, exceeding standard) and the number of achievement levels. For example, Delaware has five levels. Kentucky likely has the most achievement levels with four general achievement levels--Novice, Apprentice, Proficient and Distinguished--and additional levels within the Novice and Apprentice levels (i.e., Novice non-performance, Novice medium, Novice high, Apprentice low, Apprentice medium and Apprentice high) for a total of eight achievement levels.

Top

Minimum group size. State flexibility extended to the setting of minimum group sizes for AYP determinations, reporting, and the participation rate. States had to consider a number of state and local factors when making these decisions, such as validity and reliability of AYP decisions, student privacy, average size of schools, number of tested grades, history with subgroup accountability, and extent of diversity among students. For example, states with primarily smaller schools leaned toward smaller group sizes; North Dakota and Montana set no minimum group size and are relying exclusively on a confidence interval. Another consideration was whether or not to use a confidence interval: States might select a smaller group size (and therefore increase subgroup accountability) while ensuring reliable decisions with the application of a confidence interval. For example, Maryland chose an AYP group size of five with the use of a confidence interval. One issue with a confidence interval is that it can be difficult to explain. In responding to this trade-off, other states selected larger group sizes with no confidence interval; an example is Virginia with an AYP group size of 50 without a confidence interval. California, with its larger and highly diverse schools, selected an AYP group size of 50, when those 50 are at least 15 percent of the student body, or 100. In California, if there are 50 students within a group, there can be no more than 333 students in the school or those students will not form a separate subgroup.

Additionally, states have the flexibility to set a different AYP group size for special education students. Nebraska and Puerto Rico, for example, set a higher group size for their special education students.

Definition of full academic year. In calculating AYP determinations for schools and districts, states can only use the assessment scores of students enrolled in a school and district for a full academic year. A few States (such as Colorado) have defined full academic year as enrollment from test administration to test administration. Other states (including Michigan and New Mexico) have defined full academic year as enrollment from some pre-determined head count date in the fall to test administration. The application of this definition means that schools are only held accountable for those students they have an opportunity to teach for at least a full academic year.

Definition of major racial and ethnic subgroups. States have the flexibility to determine what constitutes a major racial or ethnic subgroup. NCLB did not identify the major racial or ethnic groups for states but instead called upon states to make this determination based upon demographic factors within their state borders. Texas, for example, only designates subgroups as major racial or ethnic groups when they constitute a certain percentage of the state population. In practice, there are only three ethnic subgroups in Texas' AYP definition: African American, Hispanic and white.

Timeline for reaching 100 percent proficiency by 2013-2014. While states have to ensure that their intermediate goals increase in equal increments over the NCLB timeline, states have great flexibility in determining how often their intermediate goals increase. States can raise their intermediate goals every year or every two or three years.

• Alaska and a number of other states, including Ohio and Arizona, created a trajectory that is more aggressive in the second half.
• Missouri's trajectory increases in equal increments every three years.
• New Jersey's trajectory increases every three years, where the increases are calculated based on an equal percent growth expectation.
• Illinois' goals increase in 2005, 2007 and then annually until 2014. Described as the "Illini Plan," it anticipates slower growth in the earliest and latest years with more accelerated progress in the intervening years.
• Arkansas' and Washington's goals increase every year until 2014.

States also have the flexibility to set the same trajectory for all their schools or to establish different trajectories based on grade level.

Grade spans for AYP decisions. The regulations allowed states to set starting points, and by implication, annual measurable objectives and intermediate goals, by grade span. For example, some states have chosen to have the same trajectory and annual measurable objectives for the elementary, middle and high school levels. Other states have separate starting points and annual measurable objectives for the elementary, middle, and high school levels. Use of this flexibility resulted in a number of configurations across the states. Wisconsin set one goal for all grade levels in both reading and mathematics. West Virginia set starting points and goals for each grade level in elementary, middle and high school. The District of Columbia set a starting point for schools with any grades 3-8, and then a separate starting point for high schools. Mississippi set starting points for each tested grade and devised a formula for combining such information when making AYP decisions. Vermont, with its rural and uniquely configured schools, established starting points and goals for each type of school (such as schools that include only elementary grades or those that include all grades K-12).

Uniform averaging procedure. States have the flexibility to determine how many years of data will be used to make an AYP determination. States may use one, two, or three years of data in calculating AYP. Further, states have the flexibility to compare one year of data to two or three years of data in making final AYP determinations. This flexibility enables a state to give schools the benefit of recent improvements (with one year) or limit the effect of a poor achievement in one year (with two or three years). States using the uniform averaging procedure include Alabama, Hawaii, Tennessee, and Rhode Island.

Statistical tests in making AYP determinations. States have the flexibility to apply a statistical test, such as a confidence interval, to their AYP calculations. Additionally, states have the flexibility in determining the level of confidence that will be used. For example, some states are applying a 90 percent confidence interval, (Iowa), 95 percent confidence interval (Maine and New Hampshire), or 99 percent confidence interval (Arizona and Oregon). Minnesota is using a sliding scale confidence interval (from 95-99 percent) that increases dependent upon the number of subgroups present in an individual school.

AYP Targets and Confidence Intervals

The best way of explaining this is to tap into people's understanding of poll results. A poll result might report that, with 95 percent confidence, 80 percent of Americans want the Cubs to play in the World Series, with +/- 5 percentage points. This means that we can be 95 percent confident that somewhere between 75-85 percent of Americans want the Cubs to play in the World Series.

The same example works for a school's AYP rating. A group of students may have had a 50 percent proficiency rate in reading. Once the confidence interval is applied, we can be 95 percent confident that this group of students met the state goal of 55 percent proficiency, because their 50 percent proficiency rating was within the +/- band of 5 points. The decision is that this group of students met AYP.

The actual band, or +/- range, depends on how many students are being considered. With fewer students, the band is larger (e.g., +/- 20 points). With more students, the band is smaller (e.g., +/- 3 points).

Other academic indicators. States have great flexibility in selecting their other academic indicators and for setting the AYP criteria for these indictors. At the elementary and middle school levels, the other academic indicators vary widely among the states (e.g., attendance rate; retention rate; achievement on science, writing, and social studies assessments; performance on the state's current accountability system; or a menu of additional academic indicators). Georgia, for instance, allows its districts to select from a menu of indicators that will be used in elementary and middle school AYP decisions over a three-year period of time. This menu includes retention rate; achievement in writing, science, and social studies; and increases in the percentage of students scoring at advanced levels. Districts thereby have flexibility to focus on different issues, as befitting the students in their schools. Although at the high school level, the statute requires that, at a minimum, states use the graduation rate as the additional academic indicator, states have the flexibility to choose another academic indictor for the high school level as well. As an example, Florida selected the high school writing assessment for this purpose. Further, until a state can calculate and disaggregate a four-year graduation rate, states have the flexibility to craft an interim graduation rate or another academic indicator as a proxy. Additionally, states have complete flexibility in setting the criteria for meeting the additional academic indicators. Some states set the criteria as a straight target (e.g., to make AYP a school must meet a pre-defined target), and other states set the criteria as either meeting the goal or making progress towards the goal. Finally, states have the flexibility to keep the goals the same over the NCLB timeline or to raise them over time. Illinois set its initial goal for the percentage of students who will graduate with a regular diploma at 60 percent (increasing to 85 percent in 2014), while Indiana set its goal at 95 percent or improvement over the previous year.

Use of an index in making AYP decisions. States have the flexibility to take into account the percentage of students who improve from being "below basic" to achieving a "basic" level of knowledge through the development of an index. Allowing schools to receive credit for improving all students' achievement is typically calculated through an "index" score. For example, New York is using its index scores to calculate AYP. New York, Oklahoma and Vermont, for example, had all used an index system as part of their past state accountability systems and continue to do so with NCLB. The Department allows states to base AYP on an index score, but only when certain conditions are met: 1) the index does not give extra weight to students scoring above proficiency so as to mask performance in the lower achievement levels; 2) the index can be calculated separately for reading and mathematics and for each relevant student subgroup; 3) the index does not allow schools to make AYP without also increasing the percent of students who are proficient; and 4) the index, as reflective of the annual measurable objectives and intermediate goals, must follow the relevant regulatory provisions.

Scores of students not participating in assessments. While all students not participating in state assessments must be counted against the participation rate for a school, states have the flexibility also to count these students as not proficient when calculating the school's AYP rating. For example, as a disincentive for its schools to exclude students from state assessments, Maryland made the decision to count students not participating against both the participation rate and against the proficiency rating for the school.

Use of results from early-administered assessments. For AYP purposes states must count the assessment from the first official administration given when they expect all students to have learned the content standards. The first administration is the first time an assessment is officially administered to measure student achievement on the state content standards in a given grade and subject, at a time when the state expects students to have learned the content standards or achieved proficiency. Students who have scored at proficient or higher on assessments taken earlier than the first official administration, however, may "bank" those scores and would not have to retake the test at a later date. Ohio can administer the fourth-grade reading test early in the school year and include in AYP calculations any proficient results from that fall administration.

Top

Integration of existing state accountability systems with NCLB. States can continue using their current accountability designs while integrating the AYP methodology into their systems. This approach allows states to continue using school labels, scores and other accountability elements (including pre-existing state sanctions and rewards) that educators, parents and the public already understand. At the same time, the more finely tuned analysis offered through AYP provides even greater detail and incentives to ensure every child reaches grade-level standards in reading and mathematics. States have great flexibility in joining their current accountability systems with NCLB. Some states (e.g., California, Louisiana) used their current accountability systems as additional academic indicators. Other states (e.g., Virginia, North Carolina) decided to give their schools two separate ratings - their state rating and the NCLB AYP rating. A few states are using AYP as a conditional requirement for achieving the highest state rating. For example, Ohio schools can only receive the top state rating if they have not been identified for improvement.

Same-subject identification. In identifying schools for improvement, corrective action and restructuring, states have the flexibility to make these identifications based on the school missing AYP in the same subject for two or more years. Most states chose to identify schools for improvement only after that school missed in the same subject for consecutive years. Only Louisiana identifies its schools for improvement irrespective of the subject matter in which the school did not make AYP.

Report cards. NCLB lists the information that must be provided on state report cards. States determine how to present this information and what additional information to include for the public to have an even better understanding of public education. The release of these report cards is determined by the state as is the medium for delivering the information (e.g., newspaper, Internet). To help states with this important activity, the U.S. Department of Education has entered into a public-private venture with Standard and Poors, the Broad Foundation and Just for the Kids. This joint collaboration will provide states with analytical and reporting tools necessary to implement NCLB.

Identification of schools that need to improve. States must identify schools for improvement prior to the beginning of the school year. To help states with this goal, the Department allows flexibility in this process. States can make these identifications on the basis of preliminary analysis, identify the schools directly with public announcements to follow, or roll out the identification of schools. South Carolina made preliminary identifications in mid-August, with the public release of information in late September. Texas worked directly with schools-in-improvement to begin offering choice and supplemental services before school began, while the public release of information was finalized for later publication. Connecticut will roll out information due to its testing schedule wherein elementary and middle schools test in the spring and high schools test in the fall. Similarly, Idaho first released its list of Title I schools identified for improvement in early August and followed with preliminary AYP reports for all schools in November.

System of rewards, sanctions and instructional interventions. States have great flexibility in fashioning their systems of rewards and sanctions for both Title I and non-Title I schools. While the statute stipulates required actions (including school choice and supplemental services for Title I schools identified for improvement, corrective action and restructuring), states and districts have incredible flexibility in shaping the exact school-improvement strategies and interventions for these schools. In other words, states can focus the type and level of assistance they provide to struggling schools depending on the nature of the school's difficulty. For example, states and districts can vary the technical assistance and professional development activities dependent upon the specific subgroups that missed AYP and the number of subgroups that missed AYP. Additionally, states have the flexibility to require all schools, regardless of Title I status, to offer school choice and supplemental educational services or to have a different set of sanctions for its non-Title I schools. Regarding their system of rewards, states have flexibility to determine what these rewards might be. In some states, rewards are banners, plaques or ceremonies, while in other states rewards come in the form of financial benefits for schools and teachers.

Use of state school-improvement funds. With the record increases in Title I and other NCLB funding, states have a dedicated source of funds that are specifically directed towards helping schools and districts improve and meet the goals of NCLB. For the 2004 fiscal year, the set-aside (which is equal to four percent of the Title I allocation) may result in almost $500 million dollars for school-improvement efforts nationally. States may either allocate these funds directly to districts and schools identified for improvement or may arrange for services at the state level.

Top

Use of alternate achievement standards for students with the most severe cognitive disabilities. Under the Department's transition policy, states have the flexibility to include proficiency results (up to 1 percent) from assessments based on alternate achievement standards in AYP calculations. While many states have chosen to take advantage of this flexibility, other states (e.g., South Dakota) are holding all students, regardless of disability, to the same content and achievement standards. This regulation was published on December 9, 2003. In the final regulation when measuring AYP, states, school districts, and schools will have the flexibility to count the "proficient" scores of students with the most significant cognitive disabilities who take assessments based on alternate achievement standards. The number of those proficient scores may not exceed 1 percent of all students in the grades tested (about 9 percent of students with disabilities) at the district and state levels. Without this flexibility, those scores would have to be measured against grade level standards and considered "not proficient."

Off-level assessments and students with disabilities. As states begin to transition away from off-level assessments for some students with disabilities, states were given the flexibility to consider students who took an off-level assessment as participating but not scoring proficient (e.g., Mississippi, South Carolina). Later in the summer of 2003, states were allowed (for one year only) to include results from off-level assessments positively in AYP decisions. Secretary Paige used his transition authority to provide states with this flexibility for the 2002-2003 school year. It was a state's decision whether or not to use this authority. Ohio allowed school districts to include, as proficient, student scores from off-level assessments that reflected proficiency on the tested material. Moving forward, these assessments will be subject to the requirements specified in the alternate assessment regulation referred to above.

Use of accommodations and alternative assessments for LEP students. States have wide flexibility in determining how best to assess their LEP students. States can offer a menu of accommodations (e.g., use of bilingual dictionaries, extra time, the use of translators) or an alternative assessment (e.g., a native-language version of its assessment or a simplified English version of its assessment). NCLB allows states a minimum of three years to test LEP students in language arts using a native language assessment, with an additional two years if needed on a case-by-case basis. For mathematics, NCLB does not restrict the number of years an LEP student can take a native language assessment. New York translates its Regents exams into Spanish, Haitian-Creole, Russian, Chinese and Korean for all subject matters except English and language arts. Illinois has developed a "plain language" version of its mathematics assessments. To summarize, states decide how best to test LEP students, which accommodations to use, how long to allow the use of native language assessments, and how to transition LEP students into the English versions of statewide assessments.

Definition of the LEP subgroup. The NCLB Title IX definition of a limited English proficient student gave states great flexibility in defining the students who constitute the LEP subgroup. For example, a state has the flexibility to define narrowly the LEP subgroup as only those students receiving direct, daily LEP services. An example of this is North Carolina that chose to define its LEP subgroup as students receiving direct LEP services. A state could also define the group more widely to include both students receiving direct services and students being monitored by virtue of their achievement on several important assessments. For instance, California LEP students will remain in this category until students score proficient on the state's assessment of English language proficiency and proficient on the state's academic content assessments for multiple years.

Top

AYP determinations for small schools. Although states and districts must make an annual AYP determination for all their public schools, regardless of their size, states have incredible flexibility in determining the methodology for small school AYP determinations. Some states will average data over multiple years to make an AYP determination while other states are employing some sort of small-school review process whereby a team of outsiders will look at classroom assessments and other measures of academic performance in addition to student performance on the state's academic assessments. Vermont, with its larger than average number of small schools, has tailored a process that provides a typical AYP result for schools with 15-39 students that is followed up with a more thorough analysis of data when this decision indicates that the school has not made appropriate progress. For schools that are very small (less than 15 students), Vermont state educators will analyze the available data and student achievement results and then provide an AYP rating.

AYP determinations for schools without tested grades. Again, while states must make an annual AYP determination for all its public schools, states have great flexibility in designing the methodology for making AYP determinations for schools without tested grades (e.g., K-2 schools). Some states will use a back-mapping technique whereby the scores of third graders are assigned back to their K-2 schools. Wyoming paired each of its schools without tested grades to its feeder school, thus encouraging educators to work together and provide meaningful accountability for all schools. Other states have chosen to use other academic assessments they have designed for the K-2 grade level.

Top

State definition of "persistently dangerous" schools. Students must be allowed to transfer to another safer public school when their current school is determined to be "persistently dangerous." States, in consultation with districts, have the responsibility for establishing the criteria for identifying such unsafe schools. In practice, only a few states (such as Pennsylvania, Oregon and New Jersey) have identified any schools under this provision. Forty-four states and the District of Columbia report having no "persistently dangerous" schools.

Top

State definition of highly qualified teachers. NCLB specifies that each teacher must have a bachelor's degree, have subject-matter competency, and be a fully licensed teacher. There are some differences between expectations for new teachers versus teachers in the classroom before NCLB.

New elementary school teachers. New elementary school teachers must hold a bachelor's degree, be fully licensed, and demonstrate (through a rigorous state test) subject knowledge and teaching skills. States set their own standards on how well teachers must do on these state tests to demonstrate subject matter competency.

New secondary school teachers. New secondary school teachers must also hold a bachelor's degree, be fully licensed, and demonstrate their subject matter competency. New secondary school teachers have several avenues for establishing their subject competency, including: (1) completing an academic major, graduate degree, coursework, or advanced certification; or (2) passing a rigorous state academic subject test. States have the flexibility to determine how well teachers must do on these state tests to demonstrate subject matter competency.

Kansas and Iowa have made efforts to link university programs to academic standards in an effort to ensure that new teachers will meet these standards. The education departments in both states work closely with their colleges and universities to achieve this goal.

Current teachers. Current teachers must also hold a bachelor's degree, be fully licensed and demonstrate subject-matter competency. In determining subject-matter competency, states have the flexibility to determine the status of current teachers through the use of the "high, objective, uniform state standard of evaluation" (HOUSSE). Factors such as college coursework, advanced credentials, professional development, involvement in curriculum-related activities, student achievement and years of experience may be included as components of the HOUSSE model.

Kentucky has developed a Web-based calculator for teachers to determine their highly qualified status for each subject they teach (http://wd.kyepsb.net/cfusion/hq).

Certification requirements. In all instances, states have full authority over certification requirements. They may redesign these at any time according to state law. States may use alternate certification programs and other non-traditional routes towards achieving state certification. The American Board for Certification of Teacher Excellence (ABCTE) has also been designated as another path towards certification that states may use if they wish. The ABCTE project allows teachers to achieve full certification by passing a rigorous assessment. Pennsylvania and Idaho are using the ABCTE process to help more teachers meet certification requirements.

Improving the number of highly qualified teachers. States were asked to establish a plan for improving the quality of teachers and increasing the number of highly qualified teachers. This information was submitted to the U.S. Department of Education during the consolidated application process in June 2002. Under the statute states have the flexibility to develop these plans in accordance with previous state reform efforts and existing state law. One way of increasing the number of highly qualified teachers is to implement more alternate paths towards teaching. Oklahoma, Florida, Colorado and New Jersey have successful alternate certification programs for new prospective teachers. Kansas has initiated an on-line alternative path towards receiving full certification and meeting other teacher quality provisions http://www.ksde.org/cert/TransitionTeaching.htm.

Parent notification of teacher quality. At the beginning of each school year, a district that accepts Title I funding must notify parents of students in Title I schools that they can request information regarding the qualifications of their child's teacher. Title I schools must let parents know if their child has been taught for four consecutive weeks by a teacher who is not highly qualified. States and districts may determine the content of this parental notification. Nevada provides districts with sample letters to aid them in their implementation of this provision.

Use of Title II and other funds. States may use their Title II funds to ensure that all teachers meet the highly qualified teacher provisions. These funds provide considerable flexibility in terms of what may be offered to teachers, including professional development, stipends for professional development, bonuses, and recruitment and retention efforts. Other funds (such as Title I, Title V local technology funds) may also be used to meet these requirements. Funds from the Individuals with Disabilities Education Act may be used to help special education teachers meet these provisions; similarly funds from Title III may be used to help teachers of English language learners meet these provisions. Georgia, for example, is using its Title II funds to provide consultants. These consultants are charged with helping districts improve professional development and increase the number of highly qualified teachers. Both Florida and Alabama are using the flexibility provisions in NCLB to provide extensive, scientifically based professional development for elementary teachers in reading.

State definition of qualified paraprofessionals. NCLB requires that paraprofessionals funded with Title I funds and who provide instructional support meet certain standards. Iowa continues to build on past traditions, and currently only a handful of paraprofessionals do not meet these standards. Hawaii relies on community members, referred to as Kapunas, to provide cultural training for students. These individuals who provide valuable cultural information are not required to meet the paraprofessional requirements since they do not provide instructional support in reading, writing or mathematics. However, paraprofessionals who do provide critical instructional support and meet these standards receive a state certification.

Qualified paraprofessionals must have completed two years of study at an institution of higher education; obtained an associates degree; or met a rigorous standard of quality and demonstrate knowledge of certain core subjects. States have the flexibility to define the rigorous standard of quality and means for demonstrating knowledge of core subjects. New paraprofessionals must meet these requirements prior to being hired, and current paraprofessionals have until January 2006 to meet these requirements.

###

Top

Back to January 2004