NRS Data Systems Requirement Checklist
NRS Data System Requirements Checklist
This checklist provides sample specifications and basic information needed for a data system that helps states meet National Reporting System (NRS) requirements under the Workforce Innovation and Opportunity Act (WIOA). The checklist is designed to assist states in determining their data system’s alignment with the requirements.
Table of Contents
Overview
This document identifies the data system specifications needed to meet NRS requirements. It can support data system development, guide a vendor through a request for proposals (RFP), or help state staff review their existing system. Because NRS data systems often operate in an interagency context, collaboration among adult education, workforce, system development, and information technology staff—as well as other professionals—may be required to provide information on agency-wide IT requirements, consistent data definitions, security, and other key factors.
The elements of a set of specifications for developing a data system rely upon several factors, including
whether the plan is to implement a custom-built or off-the-shelf option;
whether development will be performed in-house or by a vendor; and
the level of expertise of the development team.
The art and science of crafting specifications involve thoughtful consideration of the system elements, the characteristics of the system to be communicated, and who will use the specifications and for what purpose. Given these considerations, it is not required that each element be checked off the list; rather, this checklist is designed to give users a better sense of what the system being reviewed contains and how it may be enhanced.
Data System Functions
To meet user needs, the data system should include the components described in this section. Each component will have specific functions for managing student records, generating reports and alerts, preparing accountability reports, setting up the data system, and so on.
Managing the Data System Setup
The “data system setup manager” provides functions for setting up the data system for programs to use. It allows a user to add information about new instructors, classes, programs, and assessments.
Information | Can Information Be Added? | Notes |
|---|---|---|
| ||
| ||
| ||
| ||
|
Additional notes:
Managing Student Records
The “student records manager” provides a way to enter basic information about individuals who enter an adult education program. It consists of a single data entry screen to capture student name, Social Security number, date of birth, ethnicity, gender, address, home phone, mobile phone, and email address. As specified in the Business & Validation Rules and Glossary sections of this document, data entry validation checks will prevent the entry of erroneous and improperly formatted data.
Function | Available? | Notes |
|---|---|---|
| ||
| ||
| ||
| ||
| ||
| ||
|
Additional notes:
Managing Student Assessments
The “assessment manager” part of the data system provides functions for entering student assessment scores. It consists of an entry screen that lists all pre-/posttest scores for a student during their current period of participation, and a selection screen from which instructors or program administrators can view a list of students in their program (or class); access should be read-only except for program administrators who are correcting errors. The listing of students includes the student’s name, pre-test score for each subject area, initial Educational Functioning Level (EFL) score, posttest eligibility date, date of last assessment, latest test score for each subject area, and current Educational Functioning Level (EFL). It can be arranged by name, posttest eligibility date, or date of most recent assessment.
The entry page includes past assessment scores for read-only access. For each assessment, the date, score, and instrument used must be entered. EFLs are generated automatically. The data system should validate entries to ensure consistency with formats and ranges specified by the assessment’s publisher according to the Business & Validation Rules section of this document.
Function | Available? | Notes |
|---|---|---|
| ||
| ||
| ||
| ||
| ||
|
Additional notes:
Managing Attendance
The “attendance manager” part of the data system provides functions for entering and updating student attendance hours. It consists of an entry screen that lists a student’s attendance history and a screen where instructors or program administrators can view a list of students in their class and select which student’s attendance to update. The listing of students includes the student’s name, identification number, enrollment date, and last date when attendance hours were entered.
The entry page includes past attendance hour entries, for read-only access. For each entry, the number of contact hours, and the period to which they apply, must be entered manually or based on student sign-in, with the proxy contact for distance learning customized to different distance learning products. A comment may be added to an entry, including indicating personal contact time. Entries are validated consistent with formats and ranges specified in the data dictionary, and according to the rules specified in the Business & Validation Rules section of this document. Once entered, entries may not be deleted. However, adjusting entries are allowed and must be accompanied by a note explaining why it was necessary.
Function | Available? | Notes |
|---|---|---|
| ||
| ||
| ||
|
Additional notes:
Managing Outcomes
The “outcomes manager” part of the data system allows entry and updates of students’ educational and employment outcomes. The outcomes manager summarizes the information for each period of participation and indicates if the student has achieved a Measurable Skill Gain (MSG; e.g., for Table 4) or Outcome Measure (e.g., for Tables 5, 5a, and 10) based on the evidence added to the student record.
It consists of an entry screen that lists any student outcomes for the current period of participation, and a screen where instructors or program administrators can view a list of students in their program or class and select one for updating.
The list includes each student’s name, identification number, enrollment date, date of the latest outcome, and a brief description of the outcome.
The entry page includes a student’s outcomes for the current period of participation, for read-only access. For each entry, the period of participation to which they apply must be entered. Notes about required follow-up may be added to an entry. Because NRS under WIOA defines certain reportable outcomes, the data system can provide a dropdown list from which an outcome may be selected.
The “outcomes manager” also should accept information about student outcomes from a data-match file. Each row in the file should include the student identification number for matching, outcome type, date of the outcome, and supporting documentation/notes.
Entries are validated consistent with formats and ranges specified in the data dictionary and according to the Business & Validation Rules section of this document.
Function | Available? | Notes |
|---|---|---|
| ||
| ||
| ||
| ||
| ||
|
Additional notes:
Managing Follow-Up With Students
The “follow-up manager” allows entry and updates of follow-up information related to student outcomes. It consists of an entry screen that lists any student outcomes for the current period of participation, and a screen from which instructors or program administrators can view a list of students in their program or class and select one for entering follow-up information. Follow-up information can also be imported via data match.
The selection list of students includes student name, identification number, enrollment date, date of the latest outcome, a brief description of the outcome, and date of latest follow-up—whether manually entered or data matched.
Function | Available? | Notes |
|---|---|---|
| ||
| ||
| ||
| ||
| ||
|
Additional notes:
Inputs
To provide the data system functions described, and generate required NRS tables and other outputs, the data system must be able to manage certain types of information. The NRS Technical Assistance Guide describes fundamental data items for a WIOA data system.
Business & Validation Rules
The data system will be capable of validating data when entered by an individual user or imported from another source, such as data matching. Although states must use a data validation strategy, the specific design, implementation, and periodic evaluation of that strategy is left to the discretion of the state so long as those strategies or procedures are consistent with federal guidelines (e.g., OCTAE Memo 19-1). The following is a list of data checks and validation rules to be implemented.
Data Element | Requirement Met? | Notes |
|---|---|---|
| ||
| ||
| ||
| ||
| ||
|
Additional notes:
NRS Reporting Requirements
NRS reporting under WIOA requires an understanding of data items that must be collected and maintained by the state data system, rules for determining whether and how data are maintained, as well as specific reporting criteria and formulas.
This list of business rules specifies the types of data that the data system must maintain about programs, staff, and participants and the conditions that govern how and when that data may be entered. In addition to these federal business rules, each state may develop additional rules for entering and maintaining intake, enrollment, attendance, advancement, and outcome data for local programs. To ensure accuracy and consistency of student records and program data, data checks are advised within the data system based on federal and state business rules.
Data Element | Requirement Met? | Notes |
|---|---|---|
| ||
| ||
| ||
| ||
| ||
| ||
| ||
| ||
| ||
| ||
| ||
| ||
| ||
| ||
| ||
| ||
| ||
| ||
| ||
Outcomes: Integrated English Literacy and Civics Education (IELCE) |
| |
| ||
| ||
| ||
|
Additional notes:
Users and Roles
Adult education staff will use the data system as a tool for managing student records, generating accountability reports for the NRS under WIOA, supporting local and state adult education program operations, and providing tools to help managers make more strategic decisions to improve program quality.
It is expected that the data system will have multiple types of users who play different roles in data collection, including staff members in the state office, local program directors, administrative assistants, and instructors. Each local provider will have its own set of student records and reporting capabilities that are separate from the records of other local providers. State staff will be able to review local student records, monitor the local data entry process, and generate individual provider and aggregate statewide reports.
The following section outlines basic functions that data system users will require.
State Staff
Function | Available? | Notes |
|---|---|---|
| ||
| ||
| ||
| ||
|
Local Program Directors/Staff
Function | Available? | Notes |
|---|---|---|
| ||
|
Program Administrators
Function | Available? | Notes |
|---|---|---|
| ||
| ||
|
Instructors
Function | Available? | Notes |
|---|---|---|
| ||
| ||
| ||
|
Additional notes:
Outputs
The data system needs to generate reports, such as NRS reporting tables, to support state compliance with NRS requirements under WIOA. All student data must sync to a period of performance. In addition, it should provide reports and tools to support program operations and state planning.
Accessing Reports
The “report manager” provides access to the data system’s reports. It consists of a menu of available reports with links to each one. Each report, including mandatory NRS tables and WIOA reports, can be run for specific date ranges, student demographics (age, ethnicity, gender), type of program (ABE, ESL), outcomes, and program location. The data system must be able to generate the NRS reporting tables.
Report | Available? | Notes |
|---|---|---|
Joint Statewide Performance Report (SPR) |
| |
Joint Measurable Skill Gains (MSG) Table |
| |
NRS Table 1 |
| |
NRS Table 2 |
| |
NRS Table 2A |
| |
NRS Table 3 |
| |
NRS Table 4 |
| |
NRS Table 4A |
| |
NRS Table 4B |
| |
NRS Table 4C |
| |
NRS Table 5 |
| |
NRS Table 5A |
| |
NRS Table 6 |
| |
NRS Table 7 |
| |
NRS Table 8 |
| |
NRS Table 9 |
| |
NRS Table 10 |
| |
NRS Table 11 |
|
Report | Available? | Notes |
|---|---|---|
NRS Table 14 |
|
|
Student Profile Report |
|
|
Students Needing Posttests |
|
|
Class Contact List |
|
|
Inactive Students Report |
|
|
Student Attendance Report |
|
|
Students Needing Follow-up Report |
|
|
|
| |
|
| |
|
| |
|
|
Additional notes:
Final Thoughts
Please note any additional questions, considerations, or issues here.
Glossary
Accountability Reporting | State staff will be able to generate all NRS and WIOA tables, plus reports to monitor local program activities including enrollment, advancement, achievement, separation, and follow-up for students. |
Add Contact Hours | Function to allow entry of contact hours and dates for which they apply. These can include personal contacts. |
Add Follow-up | Function to add follow-up information for an existing outcome, including date, follow-up requirement being met (employed, median earnings, a credential earned, etc.), and notes/supporting documentation. The data system should provide checks to prevent data entry or business rule issues. Supervisory approval should be required to update the entry. |
Add Outcome | Function to add information about an outcome, including date and type of outcome, and notes/supporting documentation. The data system should provide checks to prevent data entry or business rule issues. |
Add Test Score | Function to add information about a pre/posttest, including date and score. The data system should provide checks to prevent data entry or business rule issues. EFL should be calculated automatically. |
All Follow-up Outcomes Report | Report that contains all achievement outcomes entered into the data system, with one line of record for each. |
All FY Data Report | Report that contains summary data from the fiscal year (FY) for inclusion on the NRS tables and FY data searches. |
Assessment | Assessment scores must be within range for the test and form administered. Students must be pretested before enrollment in a class. The posttest date cannot be less than the number of hours since the last assessment specified in the state assessment policy. Assessments and forms must be appropriate for the program type. Student score changes greater than 10% will be flagged for possible entry errors and reported. |
Assessment Completion | Instructors will use the data system to record the results of student pre- and posttests. |
Assessment Readiness | Instructors will use the data system to obtain reports on students needing posttesting. |
Assessments | Information available on assessments should include assessment name, forms, score ranges, mapping to EFL, and the ability to import results of the assessment. |
Attendance | Instructors will use the data system to record contact hours for their students. The data can be entered manually and be based on student sign-in. Attendance hours for a student may not exceed contact hours for the class during the same period. Nonattendance will be reported if there are no attendance entries, indicating that no services have been received for more than 90 days, and the student will be exited from the program. |
Class Information | Information needed to set up a class associated with a site. This may include class name, program type, identifier, teacher ID, and hours. |
Class Setup | Program administrators will enter information about managed enrollment and unstructured classes into which students are enrolled. |
Contact Hours | Contact hours are entered only for reportable individuals and participants in the following ways:
|
Data Match | Function to import a data match file. The user enters the data match file name, then clicks submit. The data system must have a mechanism in place to validate match information and save it in the student records database. |
Data Matching and Sharing | State staff will be able to access WIOA-relevant data from other agencies needed to generate accountability reports and provide support for local program operations. |
Data System Configuration | Setup of parameters, operational rules, and data for assessment, intake, and follow-up activities, including allowable tests, assessment timelines, and information required for follow-up activities. |
Dates of Birth | Date of birth will be entered and stored in the format YYYY-MM-DD. The data system will ensure proper formatting, reasonable ranges, and appropriate student ages. Students must be 16 years of age or older. |
Decision Support (for Program Directors and Staff) | Program directors and staff will use dashboards and data analysis tools to review progress toward outcome-related goals and underlying factors that might affect that progress. |
Decision Support (for State Staff) | State staff will have tools for understanding demographic, achievement, and operational efficiency at the program and state levels. Ideally, data exploration, dashboards, and business intelligence tools will provide the flexibility and insight to support program improvement activities. |
Education Level | One of the following education statuses recorded at intake:
U.S. or non-U.S.-based schooling
|
EFL | Individuals who have been tested are assigned a single Educational Functioning Level (EFL) based on their assessed score, or Carnegie Units completed. A student is assigned an EFL each time assessed or at completion of an appropriate number of Carnegie Units to advance.
|
Enroll Student in Class | Function to add a student to an existing class. |
Enrollment | The format of the enrollment date will be validated. The date of enrollment cannot be before the intake date. |
Group Designation | Function to identify group designations for follow-up measures and maintain information about follow-up cohorts and outcomes. |
Individual Age | Individuals must be no younger than 16 at intake. |
Individual Employment Status | One of the following employment statuses are recorded at intake:
Income for jobs the individual acquired while participating the in the program or after exit. |
Individual Exclusions | Participants who exit the program due to the following circumstances may be excluded from the WIOA primary indicators of performance:
|
Individual Participation Status | Participation status is recorded as individuals progress through a program, as follows:
|
Individual Period of Participation Dates | Program entry and exit dates for individuals are recorded as follows:
– Date of final contact* *Final contact before lapse of 90+ days |
Individual Race/Ethnicity | The race/ethnicity of individuals may be reported as one of the following:
|
Instructional Environment | Participants may be enrolled in a program operating in one of the following environments:
|
Level Student | Automatic function that identifies student level based on test scores. |
Manage Periods of Participation | Function to maintain information about students’ enrollments and exits. |
Management Reports | Program administrators will run management reports on behalf of program directors and staff. |
Measurable Skill Gains | Measurable skill gains (MSG) for participants may be entered when:
|
Monitor Operations | Program directors and their staff will have the ability to review the status of key operational activities including enrollment, attendance, assessment, advancement, and follow-up tracking. |
Move Assessment Forward | Function allows for moving an assessment forward from the preceding fiscal year to the current fiscal year, provided it falls within the allowable time range for moving forward according to state assessment policy. |
New Student Entry | Function allows entry of information for a new student, after checking that the student is not already entered into the data system (provides alert for possible duplicate entry). Each program will have its own set of student records, but the data system will attempt to populate new student records with existing contact and demographic information if the individual is found in another program’s records. |
Optional Achievement Outcomes | Function allows entry and tracking of achievements made by the student during and after participation. Optional achievement outcomes are used to populate Tables 8 (Family Literacy) and 9 (Participants in Integrated English Literacy and Civics Education). |
Outcomes: Credential | Credential outcomes may be entered for participants as follows:
Participants who did not possess a secondary school credential or recognized equivalent and exited during the program year who entered at, or advanced into, a secondary-level program (ASE) who earned a secondary school diploma/equivalency AND either enrolled in postsecondary within 1 year of receiving a secondary credential OR achieved employment outcome within 1 year of receiving a secondary credential.
Participants who were dual-enrolled in a postsecondary education or training program (participants dual-enrolled in a postsecondary course, including IET), and exited that postsecondary training program and adult education who receive a postsecondary credential before or within a year of exit. Report this outcome for each period of participation.
Participants who EITHER
AND earned either a secondary school diploma/equivalent or a post-secondary credential. |
Outcomes: Employment | Employment outcomes must be reported in the calendar quarters required by each measure according to the exit date of each period of participation and verified/entered as of
The state must report whether a participant who exits in November was employed in the second (ending June 30) and fourth (ending December 31) quarters following this exit. If this individual re-enters the program, becomes a participant, and exits in the same program year, the state must report the same post-exit data elements after the second exit during the same program year. The state would follow up again with the participant to determine employment in the second and fourth quarters following the second exit (or to determine employment/education/training for youth participants).
Participants are included in the calculation of median earnings only if employed in the second quarter after exit. Multiple employment outcomes may be entered. *Only one (1) outcome counts per Period of Participation *Review Exhibit 2.9 – Quarterly Periods for Collecting Employment and Earnings Indicator in the NRS Technical Assistance Guide |
Outcomes: Family Literacy | The following outcomes may be recorded for participants in family literacy programs:
|
Outcomes: Integrated English Literacy and Civics Education (IELCE) | The following outcomes may be recorded only for participants who are counted as Integrated English Literacy and Civics Education program participants.
|
Paid Staff Experience and Credentials | Years of experience and the following types of credentials for Paid Staff members for programs administered under the Adult Education and Family Literacy Act (AEFLA) are maintained:
|
PoP Summary/Measurable Outcomes | Report shows a summary of the information for each period of participation and indicates if the student has achieved a Measurable Skill Gain (Table 4) or Outcome measure (Tables 5, 5a, 8, 9, 10, and 11) based on evidence added to the student record. PoP Summary data includes the following:
|
Postsecondary Education and/or Training Report | This report allows Postsecondary Education and/or Training information related to the student record to be viewed (populates Tables 4 and 5 for enrolled in PSET MSG, measurable outcome, and achieved PSE credential while enrolled or within a year of exit). |
Program Content | Students may be enrolled in one or more of the following areas:
|
Program Information | Information needed to set up a program. This may include program name, identifier, and location. |
Program Setup | State staff will have the ability to set up student records management capabilities for individual programs and provide access for local program staff. |
Program Types | Participants may be enrolled in only one of the following programs during a period of participation:
|
Race/Ethnicity Values | The data system will allow entry only of standard NRS race/ethnicity codes. |
Remove Outcome | Function to remove the record that was entered erroneously. Supervisory approval should be required to remove it. |
Remove Student from Class | Function to remove a student from a class in which they are enrolled. |
Remove Test Score | Function to remove the test score that was entered erroneously. Supervisory approval should be required to remove it. |
Select Student | Function to search and/or select a student from the class or program list of students. |
Separate Student | Function to mark the student as separated from the program. It provides for the required entry of the separation date and reason for separation. Once separated, students no longer appear in participant or class lists. |
Separation | Students are separated from a program 90 days after their last service was provided. |
Site Information | Information needed to set up a site associated with a program. This may include site name and location. |
Staff Employment Status | Staff members for programs administered under AEFLA are designated by one of the following employment statuses:
|
Staff Organizational Level | Staff members for programs administered under AEFLA are designated by one of the following organizational levels:
|
Staff Professional Development Report | This report provides a cumulative list of professional development topics and hours for all program staff. |
Staff Profile | Information about staff members including name, identification number, and contact information. |
Staff Roles | Staff members for programs administered under AEFLA are characterized by one of the following job functions:
|
Student Barriers to Employment | The data should be captured at intake and include the following barriers-to-employment fields:
|
Student Lookup | Function to search for students by name, student ID, email address, or phone number. |
Student Management | Instructors will use the data system to obtain student profile information. |
Student Work History | Initial work history is captured at the time of intake, and the start of the work history record will be the same as the student intake date. A new line of work history is added for each follow-up, even if the employment status remains the same. |
Teacher Information Management | Program administrators will enter basic contact information about teachers serving students in the program. |
Update Contact Hours | Function to allow adjusting entries. Must be accompanied by a note explaining why the adjustment was necessary. |
Update Outcome | Function to update information about an outcome, consistent with data validation and business rules, with the intent of correcting erroneous entries. Updates should require supervisory approval. |
Update Student Entry | Function to make changes to a student’s contact or demographic information. |
Update Test Score | Function to update information about a test score, consistent with data validation and business rules, with the intent of correcting erroneous entries. Updates should require supervisory approval. |
This document was created under U.S. Department of Education Contract No. GS-00F-347CA by the American Institutes for Research. The content of the publication does not necessarily reflect the views or policies of the U.S. Department of Education nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government. The U.S. Department of Education does not control or guarantee the accuracy, relevance, timeliness, or completeness of any outside information included in these materials.