Search

Differential Monitoring

NARA's approach to Differential Monitoring is grounded in regulatory science, focusing squarely on Fiene's theory of regulatory compliance which clearly demonstrates the efficacy of the risk assessment and compliance (key) indicator methodologies as being the most effective and efficient ways to implement differential monitoring. Click HERE to learn more.

  • Differential monitoring is a method used to determine the frequency of inspections needed and the scope that will be required based on the setting’s licensing history. This method focuses measuring tools to determine the number and scope of inspections that should be conducted at a licensed setting.

  • Integrated Monitoring is another approach to program monitoring which integrates regulatory compliance and quality into a continuous approach. Quality standards are the focal point of integrated monitoring systems.

Focused measuring tools are licensing inspection instruments that increase the effectiveness and
efficiency of a regulatory oversight agency without producing recurring operational costs. In other
words, focused measurement tools maximize performance and efficiency.
Click HERE to learn more.

  1. Weighted Risk Assessment assigns weighted scores to licensing regulations based on the level of risk to persons in care in the event of regulatory noncompliance. Click HERE to learn more.

  2. Compliance Indicator Systems identify a subset of licensing regulations from an
    existing set of regulations that statistically predict regulatory compliance with the entire
    set of regulations.  Click HERE to learn more.

  3. Quality Indicator Systems identify a subset of licensing regulations and other non-regulatory factors to predict the overall quality of care provided in a licensed setting. Click HERE to learn more.

The purpose of a licensing indicator system is to increase the efficiency and effectiveness of an existing licensing system by refocusing the emphasis of the licensing process. It is intended to complement, and not replace, an existing licensing measurement system.

Additional information, including the Schedule of Services for Measurement Tools, can be viewed on NARA – The Right Choice. 

 

Foundational Publications

Regulatory Science (General) - click to explore

Regulatory science in the human care industry is a field that uses
scientific methods to evaluate or develop tools, methods, standards, and systems
that support a better understanding of safety, quality, and effectiveness of licensing
systems. Articles in this section address a wide array of information specific to
licensing systems in the human care industry.

 

Differential Monitoring - click to explore

Differential monitoring is defined as a customized approach to
regulatory oversight that adjusts the amount and frequency of monitoring activities
based on a regulated entity's compliance history and identified risk profile. Articles
in this section address the holistic differential monitoring systems inclusive of
foundational research and current trends including Compliance (Key) Indicators,
weighted Risk Assessments and Quality Indicators.

 

Studies and Projects

Foundational Studies

  • Colorado, 2014 - This study analyzed Colorado's quality rating system, the Qualistar Rating, for generating key indicators. Read the full study.
  • Georgia, 2014 - Validation of Georgia's process for determining if state-regulated child care facilities are compliant with basic state health and safety requirements. Read the full study.
  • Oregon, 2013 - This study provided Oregon's OCC with a basic risk factor analysis comparing its child care center rules to Stepping Stones (SS) standards. Read the full study.
  • Head Start, 2013  - This project provided the Office of Head Start (OHS) Key Indicators for their Head Start Performance Standards. Read the full study.

Differential Monitoring Projects (NARA) 

  • Saskatchewan 2022 - 2023
  • Focus: The Saskatchewan Ministry of Education has completed the design, development, and validation of a new Program Quality Indicator (PQI) Tool/Scale. Saskatchewan Program Quality Indicators Tool This is the last component to their design of a comprehensive differential monitoring system.
    Project Activities: Key indicators methodology, Quality indicator development, regulatory analysis assessed with program quality, demonstration of the regulatory compliance ceiling effect, use of the regulatory compliance scale, data collection and data analysis for the development of the quality key indicators, center based infant, toddler and preschool classrooms, use of the ERS scales for validation purpose, validation and reliability testing of scale, independent observers/data collectors utilizing PQI and ERS scales, and development of the PQI scale. Read the Full Report here. 
  • Florida, 2021
    Focus: Development of a Differential Monitoring Key Indicator System (KIS) for the Florida Department of Children of Families, Office of Child Care (OCC).
    Project Activities: stakeholder feedback sessions, data collection and analysis, development of an inspection checklist, policy and procedure development, and staff and stakeholder training. Read the Full Report here.
  • Alberta, Canada, 2020 - 2021
    Focus: Development of Alberta Child Care’s licensing Key Indicator System (KIS) as well as additional analyses and recommendations based on the findings for Alberta Child Care’s Child Care Centers, Preschool Programs, and Out of School Care (OSC).
    Project Activities: data collection and analysis to determine key indicators. Read the Licensing Key Indicator Predictor Rule Results Report here - coming soon.
  • Washington, 2019 - 2020
    Focus: Validation of Washington State’s Child Care Risk Assessment and Licensing Decision Making Tiered System
    Project Activities: validation study of Washington State’s Department of Children, Youth and Families child care Risk Assessment Licensing Decision Making Tiers System (RALDMTS). The validation involves two key components: 1) Validation of the measurement strategy used to determine the licensing decision making for child care centers and family child care homes; 2) Validation of the licensing system in juxtaposition to the program quality measures (ERS & CLASS) as part of their QRIS – Quality Rating and Improvement System utilized in Washington. Read the full report here.
  • Washington, 2019 - 2020
    Focus: Licensing Measures and Outputs Validation Study for the Department of Children, Youth, and Families Child Care
    Project Activities: Validation of the measurement strategy used to determine the licensing decision making for child care centers and family child care homes; Validation of the licensing system in juxtaposition to the program quality measures (ERS & CLASS) as part of their QRIS – Quality Rating and Improvement System. Read the full report.
  • Saskatchewan, Canada, 2019 - 2021
    Focus: Key Indicator study and Weighted Risk Assessment study for the Ministry of Education in the Province of Saskatchewan
    Project Activities: Key Indicator study as well as trends in regulatory compliance; Home-based Weighted Risk Assessment.  Read the full Key Indicator report. Read the full Weighted Risk Assessment report. Read the full CCC/FCC Validation Study Report here.Read the Full Report here.
  • Minnesota, 2019
    Focus: Family Child Care Key Indicator Study for Minnesota Department of Human Services Licensing Division
    Project Activities: Key Indicator Data Analysis. Read the full report. Follow-up: Minnesota Family Child Care Task Force - Interim Legislative Report, March 2020
  • Indiana, 2018 - 2019
    Focus: Key Indicators for the Family and Social Services Administration, Office of Early Childhood and Out of School Learning
    Project Activities: Focus Groups; Survey & Data Collection; Statistical Analysis; Indicator Work Tools & Manual; Training. Read the full report.
  • Montana, 2016 - 2017
    Focus: Development of Key Indicators for Department of Public Health and Human Services
    Project Activities: Statistical Analysis; Policy Development; Staff Training; Stakeholder Meetings. Read the full report.
  • Michigan, 2016 - 2017
    Focus: Development of Key Indicators for licensed child care and group homes and registered family homes
    Project Activities: Data Collection & Analysis; Policy Development; Work Tools and Instruction Manual; Staff Training; Stakeholder Meetings; Pilot Program; Statewide Implementation. Read the full report.
  • Ontario, 2015 - This project outlined recommended methodology for developing an abbreviated inspection tool; a tiered licensing approach to child care licensing. Read the full report.
  • Illinois, 2013 - 2014
    Focus: Development of a Weighted Licensing and Key Indicator System
    Project Activities: Determination of Eligibility Criteria; Data Collection & Analysis; Identification of Additional Factors; Development of Indicator Work Tools & Instructional Manual; Staff Training; Stakeholder Meetings. Read the full report. Follow-up: Illinois DCFS - Day Care Licensing Annual Report to the General Assembly, FY 2019
  • Kansas, 2012 - 2013
    Focus: Develop a Key Indicator System
    Project Activities: Data Capacity Building and Analysis; Policy Development; Stakeholder Meetings. Read the full report.
  • Washington, 2011 - This project provided Washington's Department of Early Learning a differential monitoring process to ensure efficiency while keeping children in licensed safe care. Read the full study.