Processes of data collection, entry, reporting, and security are verified for compliance during site visits, through continuous desk monitors, and via sub-recipients' quarterly reports.

• Generally during onsite visits, the required student-file documentation (including intake form, program of study, assessments, and release-of-information signature) is randomly verified. Furthermore, the MIS has functionality which allows the agency and the providers to generate detailed reports for analyzing participants, dates, FY hours, hours-between-assessment, test-instruments, levels, forms, scale scores, measurable skills gains, periods of participation, completions, and more.
• Desk monitoring typically provides the agency ample opportunity to identify data-related issues and to correct policy-related procedures. Because of the state's program-size, consistent and constant contact with the local providers' data specialists is commonplace.
• Quarterly reports from each provider are reviewed for accuracy by reconciling aggregate information, data sets, and individual records within the MIS. The state agency also uses customized queries, student diagnostics, dashboards, and fiscal-year searches to verify the accuracy of local reports.

In addition to the regular [plenary] webinar trainings, PY2018 marked the seventh year the state has delivered provider-specific webinar reviews; each sub-recipient has the opportunity to schedule 45-minute sessions with the state and the vendor to explore local data-quality issues (without their statewide colleagues in attendance).

This approach has afforded the state, the MIS vendor-trainers, and the respective local providers the means to scrutinize topics in highly specific context, while often building rapport and eliciting more effective communication.