Leap II: Accessible Text

Leap II: Accessible Text Version

2. Defining Key Elements of NRS Reporting

Section summary

The previous section touched upon broad elements of WIOA changes and the implications for the NRS reporting tables. This section takes a deeper look at three key elements that have an impact on multiple data collection and reporting topics and issues.

By the end of this section you will have a clearer understanding of the following topics:

  • measurable skill gains,
  • periods of participation, and
  • credential indicator.

Measurable Skill Gains (MSG)

Adults enroll in adult education programs for many different reasons, but most often they attend to gain the skills they need to be successful in school, the workplace, and their everyday lives. Under the Workforce Innovation and Opportunity Act (WIOA), programs are able to determine their success in helping students gain those skills, using the measurable skill gains (MSG) indicator. This section explains what MSG is, how it is measured, who is included in the indicator, and how it is different from the outcome measures that state and local adult education programs previously collected.

What is measurable skill gains indicator?

MSG is a WIOA-required indicator used to demonstrate participants’ progress toward achieving a credential or employment.

How is measurable skill gains measured?

For adult education programs, participants can demonstrate MSG in two ways: educational functioning level gain and receipt of a secondary credential (as seen in Figure 1, below).

  1. Educational functioning level gain can be demonstrated in one of three ways:
    • Comparing a participant’s pretest with the participant’s posttest, using an NRS-approved test
    • Awarding Carnegie Units or credits in an adult high school program (enough to move to 11th- or 12th-grade status according to state rule)
    • Enrollment in postsecondary education and training after exit
  2. Receipt of a secondary credential, which can be demonstrated by obtaining passing scores on state-approved high school equivalency tests or obtaining a secondary diploma or state-recognized equivalent.

Participants in other WIOA partner programs can also demonstrate MSG by demonstrating progress from a secondary or postsecondary transcript, progress toward milestones, and passing a technical/occupational knowledge–based exam (shown in Figure 1).

Who is included in the measurable skill gains?

MSG is reported for all participants (adults who receive 12 or more contact hours) for each period of participation. However, participants who are forced to exit the program due to the following extenuating circumstances are excluded:

  • Incarceration or entry into a 24-hour support facility such as a hospital or treatment center
  • Medical treatment that lasts more than 90 days
    Being called into active duty in the National Guard or other armed services for at least 90 days
  • Death

Figure 1.  Five Types of Measureable Skill Gains Under WIOA

Figure 2. Five Types of Measurable Skill Gains Under WIOA

Where are data reported for measurable skill gains?

The NRS has four tables for reporting MSG:

  • Table 4—for all participants
  • Table 4C—for participants in distance education
  • Table 8—for participants in family literacy programs (optional table)
  • Table 10—for participants in correctional education programs

Each table breaks down—by NRS educational functioning level—the total number of participants, the number and percentage who achieve an MSG, the total number of periods of participation, and the total number of MSGs by period of participation. Only the most recent MSG achieved is reported per period of participation, even if more than one MSG is achieved.

Where are data reported for measurable skill gains?

Entry into postsecondary education at exit and receipt of a secondary credential count as outcomes, as they did before, but now are counted as an MSG rather than as separate measures. Although measureable skill gains is a new measure, pre- and post-testing, and assessment procedures and policy remain the same as prior reporting for measuring gains in an educational functioning level.

Periods of Participation (POP)

Performance calculations under the new Workforce Innovation and Opportunity Act (WIOA) of 2014 require that indicators be calculated for participants and according to their periods of participation. This section explains participants and periods of participation, and their implications for data collection and reporting to the NRS. In the regional training, Leap Into WIOA, Part II, participants delved deeper into these definitions to understand their impact on state reporting.

What is a participant?

Under WIOA, an individual who completes at least 12 contact hours of service in an adult education program qualifies as a Title II “participant” and any individual who receives adult education services without completing at least 12 contact hours is considered a “reportable individual.” Only participants are reported for WIOA performance indicator purposes. Participant status must be achieved any time an individual enters, or reenters, an adult education program. Participants who continue receiving services across program years do not need to requalify as a participant in a new program year since exit has not occurred.

What are periods of participation?

A period of participation begins each time a participant enrolls in adult education—even when multiple enrollments occur during the same program year. Subsequent enrollments during a program year result in a new period of participation. Therefore, a participant may have more than one period of participation in a program year.

Exit date is the last day of service for participants, but this date cannot be determined until 90 days have elapsed since the person last received services and there are no future services planned. Services do not include self-service, information-only services, activities, or follow-up services

For example, a student who enters an adult education program in September, exits in December, and re-enrolls in May has two periods of participation for the purpose of reporting MSG, and only one POP for the purpose of reporting exit-based indicators because only one exit is reported. A participant who enters an adult education program only once within a program year has one period of participation for the purpose of reporting both MSG and exit-based indicators (upon their exit). For all performance indicators, each period of participation is counted separately, resulting in duplicate reporting of individual participants with more than one period.

How are periods of participation used for calculating performance indicators?

State performance on all core outcome measures under WIOA, excluding median earnings, is calculated as a percentage of the total number of outcomes achieved by the total number of periods of participation.

Do the WIOA performance measures apply for each period of participation?

Every period of participation is treated as a separate event for a participant and performance indicators apply separately to each period. This means that for each period of participation, the state must report measurable skill gains and conduct follow-up on the participant after exit from each period to collect data on the appropriate post-exit indicators. Each exit date from a period of participation is used to determine the follow-up time. For example if a student exits in December, reenrolls in April, and exits again in June, the state would report on appropriate post-exit indicators for both exits. The second-quarter employment outcome, for example, would be reported in the April through June quarter for the December exit, and in the following October through December quarter for the June exit.

How do periods of participation affect pre- and post-testing of participants?

At entry into each period of participation, a participant must be placed in an NRS educational functioning level (EFL) on the basis of an approved NRS assessment. EFL placement in a new period of participation can carry over from a prior period, or a new placement test may be administered. However, it is important to note that policy and procedures for assessing students should not change because of periods of participation. Proper assessment practices should always be followed.

Policies and procedures regarding the assessment of individuals who return after an exit or an extended absence must be detailed in the state’s assessment policy, and those policies and procedures must follow test publishers’ guidelines, when available.

If the test publisher does not have retesting guidelines for returning students, the state must establish a policy for retesting that local programs must follow. These procedures will provide uniformity to testing practices within the state. After a participant is placed in an EFL in the new period of participation, either by a new assessment or carryover from a prior assessment, an EFL gain (via pre-/ post-testing) in the new period of participation requires that a new posttest be given. A posttest used in the additional period of performance may not be used to also reflect a gain in a prior period of performance.

Must barriers to employment and student demographic data be collected for each period of participation?

Yes, all barriers to employment data must be collected for each period of participation. Each new program entry is a new period of participation, and after 12 contact hours, the enrolled individual is treated as a “new” participant.

Demographic data—age, sex, employment status, and race/ethnicity—are not likely to change over the year. Therefore, depending on how a state requires programs to keep demographic information on file, participants may simply verify that all the information in their record, from the first period of participation, is still current.

Credential Indicator

Adult education programs have always tracked the number of students obtaining a secondary credential as a follow-up outcome measure for NRS reporting. However, the Workforce Innovation and Opportunity Act (WIOA) establishes a credential indicator that includes secondary and postsecondary credentials. This section explains the credential indicator, who is included in the measure, how it is calculated, how it differs from previous measures, and how it is reported.

What is the credential indicator?

The credential attainment indicator measures two types of credentials:

  • Receipt of a secondary school diploma or recognized equivalent during participation or within one year after exit from the program. The receipt of this credential is only counted if the participant also enters postsecondary education or training, or employment within one year after exit.
  • Receipt of a recognized postsecondary credential during participation or within one year after exit from the program.

Who is included in the credential indicator?

The secondary school diploma credential component of the indicator applies only to participants enrolled in a secondary education program at or above the ninth grade level who exited the program and who did not have a secondary school diploma or its equivalent. The postsecondary education credential component of the indicator applies only to participants who were enrolled in a postsecondary education or training program who exited.

Participants in correctional education programs (WIOA Section 225) are excluded from the indicator. In addition, participants are excluded from the indicator if they are forced to exit the program due to the following extenuating circumstances:

  • Incarceration or entry into a 24-hour support facility such as a hospital or treatment center
  • Medical treatment that lasts more than 90 days
  • Being called into active duty in the National Guard or other armed services for at least 90 days
  • Death

How is the credential indicator calculated?

The credential indicator is calculated as a percentage of the total number of participants who exited during the reporting period who achieved either credential as defined above divided by the total number of participants (1) who exited and were enrolled in a postsecondary education or training program plus (2) all participants who exited and were enrolled in a secondary education program at or above the ninth grade level and did not have a secondary school diploma or its equivalent.

How is this different from previous reporting?

Participants must be enrolled at the ninth-grade equivalent or higher during participation in the program to be included in the measure, and they must be employed or enrolled in postsecondary education within one year after exit in order to be counted for achieving a secondary credential. Attainment of a postsecondary credential is a new performance indicator for the NRS under WIOA.

Where are data reported for the credential indicator?

NRS Table 5 reports the indicator and other WIOA-required exit-based measures. This indicator is also included in the following tables:

  • Table 5a—for participants in distance education
  • Table 8—for participants in family literacy programs (optional)
  • Table 10—for participants in correctional education programs

Performance on this indicator cannot be measured until one year after a participant’s exit.

Activities

Table Changes in Action

During the training, participants were provided Handout 6. After each set of tables was presented, state teams discussed a related scenario and determined the appropriate response for how to complete the corresponding NRS table. To adapt this activity, go through the tables and note which of the tables uses measurable skill gains, period of participation, or credential indicators in its reporting. Highlight the most common mistake or concern you have seen from local programs regarding each of these issues and completing the tables. Craft a communication that explains to local programs how each of these elements works in conjunction with completing the tables.

Adaptations

For use with local program staff: For this activity, after local program staff identify which tables use each of these key elements, brainstorm common questions or concerns and develop answers in small groups. Then discuss the results as a whole group and identify any misinterpretations or areas for clarification.

Additional Resources