1. Understanding the Revised National Reporting System for Adult Education (NRS)

1. Understanding the Revised National Reporting System for Adult Education (NRS)

Section Summary

While the U.S. Department of Education’s Office of Career, Technical, and Adult Education (OCTAE) will provide extensive formal guidance and regulations for WIOA and the revised NRS, this toolkit will provide basic terminology and information that can provide practical support to both state adult education staff and local providers as they navigate changes in their state. In this section, you will review the basics of the impact of WIOA on the NRS including the resulting changes to the NRS reporting tables. Additionally, you will learn how these changes affect the reporting timeline.

By the end of this section you will understand

  • key term changes as a result of WIOA that affect data collection and reporting,
  • the impact of WIOA on the NRS tables, and
  • when various criteria and indicators need to be collected and reported.

Overview of NRS Changes

TIP: Share this section with local providers to ensure all staff are working with the same definition of key terms.

As mentioned in the LEAP, Part I toolkit, WIOA aimed to streamline the following core federal education and workforce training programs through a unified state plan of coordinated services:

  1. adult, youth, and dislocated worker employment training and Wagner-Peyser services;
  2. adult education and literacy programs;
  3. and vocational rehabilitation services.

WIOA also included new accountability requirements to reflect increased emphasis on employment and postsecondary education, and more tracking of participants after their exit. These new requirements mean changes to the NRS to state and local data systems and the need for additional staff training. In any system change, it is helpful for all parties involved to be working with the same set of terms and definitions so as to reduce misinterpretation. This type of confusion can lead to challenges and frustration further along in the process and hinder progress. For the remainder of this section we will review the most common and relevant key terms for adult education. The good news is that everything is not new. So what is new and what is not? Let’s begin with what remains the same.

A Review of Common Unchanging Terms

Participant – A participant is an individual who has 12 or more contact hours. This definition is consistent with the criteria we have always used.

Measurable Skill Gains – While this term is defined in a new way under WIOA (see below), the procedures and policy for pre-post-testing and assessment remain the same.

Program entry – As always, this refers to the date of an individual’s enrollment in the adult education program or services.

Program exit – This refers to the last date the participant received services and has no further services. Program exit cannot be determined until at least 90 days have elapsed since the participant last received services.

Credential attainment indicator – This is another term that will be redefined below; however, there is an element that remains unchanged, receipt of a secondary credential.

A Review of New Terms

Reportable individual – This is an individual who has taken action demonstrating an intent to use program services and who meets reporting criteria, including providing identifying information, but has achieved less than 12 contact hours.

Period of participation – A period of participation begins each time an individual enters adult education and gets at least 12 hours of service, even if there are multiple entries during the same program year. A participant may have multiple periods of participation a year and performance is measured and reported on NRS tables for each period.

Measurable Skill Gains (MSG) – This is a WIOA performance indicator, which includes educational functioning level gain through pre- post testing, entry into postsecondary education after program exit and awarding of Carnegie Units for progression from ABE Level 5 to Level 6. MSG also includes receipt of a secondary credential. Only one of the outcomes is reported (the last one) on NRS tables for each period of participation.

Barriers to employment – WIOA defines 11 types of participant characteristics that can create barriers to employment. The joint ICR reporting form breaks out performance measures for each type of participant by barrier. See the Day 1 PowerPoint for a list of the 11 barriers to Employment under WIOA.

Credential attainment indicator – This includes both receipt of a postsecondary credential and receipt of a secondary credential while enrolled or within one ear of exit. However, receipt of a secondary credential is only counted if the participant also is employed or is enrolled in an education or training program leading to a recognized postsecondary credential within 1 year after exit from the program. The secondary credential only applies to participants who exit at the ninth grade equivalent or higher; postsecondary credential only applies to participants in postsecondary education.

Second quarter employment indicator – This refers to whether participants are employed in the second quarter after exit.

Fourth quarter employment indicator – This refers to whether participants are employed in the fourth quarter after exit.

Median earnings indicator – This indicator refers to the median of all participants employed in the second quarter after exit

We will look at three of the above terms—measurable skill gains, period of participation, and credential indicator—more in depth in a later section, Defining Key Elements of NRS Reporting.

Summary of NRS Table Changes

TIP: Copy and paste Figure 1 into a separate document for an easy handout to share with state and local program staff.

The above changes have direct implications for NRS tables and reporting and the NRS has updated reporting tables to collect new data. Several tables have minor changes, and a few tables are optional. Tables 4 and 5 have the most extensive updates and were discussed in depth at the LEAP, Part II regional trainings. For a full review of the table changes and their implications on data systems, refer to the training materials in the appendix. Figure 1 provides a summary of changes to the NRS tables listed by current versus new table number and title.

Figure 1. Summary of Changes by Current and New NRS Tables

Current NRS Table Change New NRS Table
Table 1: Participants by Entering Educational Functioning Level, Ethnicity, and Sex No change Table 1: Participants by Entering Educational Functioning Level, Ethnicity, and Sex
Table 2: Participants by Age, Ethnicity, and Sex Age 45-59 category broken up into 45-54 and 55-59 Table 2: Participants by Age, Ethnicity, and Sex
Table 3: Participants by Program Type and Age

(1) Age 45-59 category broken up into 45-54 and 55-59;

(2) ESL named English Language Acquisition;

(3) separate reporting for Integrated EL and Civics participants; breakout for IET for all categories

 Table 3: Participants by Program Type and Age
Table 4: Measurable Skill Gain and Attendance by Educational Functioning Level

(1) EFL gain expanded to be Measureable Skill Gain, defined as (a) EFL gain by pre-post-testing, completion of Carnegie Units or entered postsecondary education or training; and (b) obtained a secondary credential

 

(2) Number completed a level and advancing a level no longer separately reported

(3) New columns for period of participation reporting

Table 4: Measurable Skill Gain by Entry Level
Table 4B: EFL Gains and Attendance for Pre- and Post-tested Participants Number completed a level and advancing a level no longer separately reported. “Completed” changed to “achieving EFL gain”. Table 4B: EFL Gain and Attendance for Pre- and Post-tested Participants
Table 4C: Measurable Skill Gain and Attendance by EFL for Participants in Distance Education Same as Table 4 but for reporting distance education participants. Table 4C: Measurable Skill Gain by Entry Level EFL for Participants in Distance Education
Table 5: Core Follow-up Outcome Achievement

(1) Outcomes changed to WIOA required indicators

(2) rows and columns for sampling eliminated

(3) rows and columns for number used for data matching or survey eliminated; new columns added for period of participation reporting

 Table 5: Core Follow-up Outcome Achievement
Table 5A: Core Follow-up Outcome Achievement for Participants in Distance Education Same as Table 5 but for reporting distance education participants. Table 5A: Core Follow-up Outcome Achievement for Participants in Distance Education
Table 6: Participant Status and Program Enrollment:

(1) Status categories of disabled, on public assistance and living in rural areas dropped

(2) New employment status category added of “employed with notice of termination”

(3) Optional secondary status categories dropped

(4) Program types of homeless and work-based project learner dropped

(5) Changed “GED” to “high school equivalent”

(6) changed “some college” to some postsecondary education” and “high school” changed to “secondary school equivalent”

 Table 6: Participant Status and Program Enrollment:
Table 7: Adult Education Personnel by Function and Job Status No change Table 7: Adult Education Personnel by Function and Job Status
Table 8: Outcomes for Adults in Family Literacy Programs (optional) Identical to new Table 5 with a row added for measurable skill gain; includes optional measures of increased involvement in children’s education and increased involvement in children’s education Table 8: Outcomes for Adults in Family Literacy Programs (optional)
Table 9: Outcomes for Adult in Workplace Literacy Programs Old Table 9 is eliminated and replaced by now former Table 11 without work-based project learner category. For optional reporting of left public assistance, achieved citizenships skills, increased involvement in children’s education, voting or registered to vote, increased community involvement Table 9: Secondary Outcome Measures (Optional)
Table 10: Outcomes for Adults in Correctional Education Programs Revised to be identical to new Table 5 with a row added for measurable skill gain; for separate reporting of participants in correctional education programs Table 10: Outcome Achievement for Adults in Correctional Education Programs
Table 11: Secondary Outcome Measures (Optional) Now Table 9 See new Table 9
Table 12: Work-based Project Learner by Age, Ethnicity and Sex Eliminated NA
Table 14: Local Grantee by Funding Source Column added to include IEL/EL Civics providers Table 14: Local Grantee by Funding Source

Note: Table 13 was eliminated in 2012.

Activities

The activities described below and in each section were conducted in the NRS regional training by state teams. These activities can be replicated with additional state staff who were not in attendance. In addition, we have provided suggested modifications to use the activities with local program staff.

Determining the Implications of WIOA for Data Collection

States used this activity to examine the new WIOA terminology and NRS table changes as a state team. In state teams, they then reviewed each individual table and summary of its requirements for completion and considered the impact these would have on the six common topics identified in LEAP, Part I: 1) intake, 2) measurable skill gains, 3) follow-up, 4) data systems, 5) program monitoring, and 6) training, communication and leadership. After identifying the impact of WIOA changes on each topic, state teams brainstormed potential questions local providers may have regarding the changes. This activity is a useful one to revisit in order to both help new staff learn necessary elements for completing the NRS tables and also as a way of continually identifying needs of local program staff and areas for additional training.

Adaptations

For use with local program staff: Understanding the overall purpose of why certain reporting elements are required can help ensure fidelity and efficiency of reporting at the local program level. Use this activity with local program staff as a way to help them become familiar with the larger federal reporting and how their local program reporting fits in with the larger purpose. In place of the section brainstorming potential questions related to the tables, you may want to compare state responses to the implications chart with how the local program staff completed the same task as a way to connect the state and local focus of data reporting.

Key terms Jeopardy®

It can often take multiple exposures to new information in different formats in order to remember and understand how it is used. Taking dry information and putting it into a format that is engaging can help this process. The key terms Jeopardy® game was created with this in mind. Information in the areas of some of the key terms was put into a Jeopardy® format and state teams faced off against the other team at their table responding to the answers in the traditional question format.

Adaptations

For use with local program staff: This technique is useful with a variety of people, but may be best suited for a longer training and used in order to break up longer blocks of discussion. Provide the same link to the game as is, or copy the game and adapt the questions and answers to other specific local program information. Following the game, provide time for teams or groups to discuss the answers they got wrong and why. Then as a whole group, discuss any remaining questions or use the whole group time to review some of the more complex questions.

Communication Guidance/Six Key Issues

In this activity, state teams reviewed the characteristics of effective communication which were discussed in depth in LEAP, Part I. Each group was then provided a question prompt related to the six key issues around WIOA (as noted in the training) and one of the key terms under WIOA. Participants then individually wrote a message to local programs describing the topic in their own words. Teams provided feedback on whether the message was clear, if anything was left out, or how else to revise the messages for clarity and understanding which was then incorporated into one final message. This activity is helpful in showing whether a participant understands the information that was presented.

Adaptations

For use with local program staff: This activity works well as a gallery walk with messages written on chart paper and feedback provided using post-it notes. Have participants at a face-to-face training create messaging in groups for the same items or others around WIOA and NRS reporting as if they were explaining the process to new staff. Assign teams a particular format for their message, such as newsletter article, webinar, or interoffice memo to help narrow their focus. Feedback can be provided in a whole group discussion.

Table Changes in Action

During the training, participants were provided Handout 6. After each set of tables was presented, state teams discussed a possible scenario related to data collection for the tables, and determined the appropriate response for how to complete the corresponding NRS table.

Adaptations

For use with local program staff: This activity can be completed as is with local programs. Additionally, state training staff may want to identify real-life examples from local programs to supplement this activity. As a group discuss why answers are or are not correct for a particular situation.

Additional Resources

The following resources provide further reading about WIOA implementation, NRS Performance Reporting and other sources of information.