logo

Resource Library: Implementation Science & Scaling Up


“System-wide change will only happen if there are multiple drivers, multiple partnerships, multiple teams and many individuals working together in new ways to create a new culture.”  -Alma Harris (2010)

 

SISEP Active Implementation Hub: Online Modules and Resources.  "This is a free, online learning environment for use by any stakeholder  involved in active implementation and scaling up of programs and innovations. The site goal is to increase the knowledge and improve the performance of persons engaged in actively implementing any program or practice.” 

Assessments
Coaching
Implementation Drivers
Implementation Stages
National Meeting Presentations and PLC and Directors' Webinars
Publications and Research Articles
Tools and Resources

 

Assessments

Overview of Stage-Based Measures of Implementation Components (PDF, SISEP, 11/2011). This table outlines the stage-based assessments of implementation activities in practice, to be used (modeled) by Regional Implementation Team members during District or project work with District Implementation Teams or Project Teams in the State Department of Education. Subsequently useful to District Leadership and Implementation Team members as they work with groups of schools or building leadership and implementation teams.

ImpleMapping. Exploration Stage Assessment of Implementation Capacity (PDF, SISEP, 11/2010). This is part of SISEP's series of Stage-Based Measures of Implementation Components.

Installation Stage Assessment (PDF, SISEP, 2/2011). This is part of SISEP's series of Stage-Based Measures of Implementation Components.

Installation Stage Action Planning Guide for Implementation (PDF, SISEP, 2/2011). This is part of SISEP's series of Stage-Based Measures of Implementation Components.

Full Implementation Stage Assessment (PDF, SISEP, 2/2011).
This is part of SISEP's series of Stage-Based Measures of Implementation Components.

Stages of Implementation Analysis (PDF, SISEP, Adapted 1/2012).
This tool provides the team with the opportunity to plan for and/or assess the use of stage-based activities to improve the success of implementation efforts for EBPs or evidence-informed innovations. The tool can be used to assess current stage activities (e.g. “We are in the midst of Exploration”) or past efforts related to a stage (e.g. “We just completed most of Installation? How did we do? What did we miss?).
 

Coaching

Overview of Running Start and evaluation activities (PDF, Utah, 3/2012). Utahs' Running Start is a special education new teacher induction program designed to prepare licensed and unlicensed special education teachers to teach on the first day of school. This PDF provides an overview of Running Start with an emphasis on the evaluation activities and outcomes associated with Running Start and the generalization of vital teaching behaviors to the classroom. Additional emphasis will be placed on the role of micro-teaching, coaching, and on-going performance feedback as critical attributes of sticky professional development. Running Start Coaches Checklist (PDF, Utah, 3/2012)   

Running Start 2009: Ready to Teach on Day 1 [PPT, March 2010, Schroeder, B., Utah SPDG Director] Using the coaching best practices to inform your work. Implementation Conversations.

Leading for Reading: An Introductory Guide for K-3 Reading Coaches [PDF, October,2008, Center on Instruction] Chapter 1 – Defining the K-3 Coaching Process

Coaching: A Strategy for Developing Instructional Capacity [PDF, 2010, Neufeld, B. & Roper, D.] Explains what coaching is, what coaches do, the kinds of supports that coaches need & the potential benefits to both educators & students. The paper is co-published by the Annenberg Institute & the Aspen Institute Program on Education. A print copy of this report can be ordered via the web site of the Annenberg Institute.

Developing and Supporting a Cadre of Coaches (Systems & Instructional) (PDF, 7/2012, Developed by Utah) Most SEAs and LEAs understand the value of evidence-based professional development, which includes effective coaching systems.  However, establishing infrastructures (funding sources, training, policies, etc.) designed to support such systems appears to be a challenge. Some LEAs have been much more successful than others at implementing these systems. This document was developed as a SWOT analysis of Utah PDC on who can help facilitate implementation of MTSS in LEAS.


Implementation Drivers

 

 

Implementation Drivers Best Practices Analysis and Discussion Tool [Adapted from ©National  Implementation Research Network, September 2010, Blase, K.A., Van Dyke, M.K., Duda, M. & Fixsen, D.L.]

Using the Coaching Best Practices to Inform Your Work [Schroeder, B., Utah SPDG Director] Wiki page with the Implementation Conversations links & information.

 

Implementation Stages

Process for Quick Sorting Schools into Categories of Support (DOC, 8/2012, MiBLISi). Target Audience: District leadership teams, state professional development who provide support to schools using the stagets of implementation.

Presenter: Steve Goodman, Michigan MiBLSi Director

Five Stages of Implementation and Implementation Drivers. Developed by National Implementaion Research Network (NIRN).

Exploration Phase:

Scaling-up Brief: Readiness for Change (PDF, 2/2009). State Implementation & Scaling-up of Evidence-based Practices.

Minnesota's Terms of Reference: A Tool and Process for Clarity and Communication (PPT, 6/2009). Karen A. Blase, PhD & Allison Metz, PhD, Frank Porter Graham Child Development Institute, University of North Carolina – Chapel Hil

Stage-Based Measures of Implementation Components: Initial Implementation Stage Assessment. (DOC, February 2011). National Implementation Research Network.

Stage-Based Measures of Implementation Components (DOC, October 2010). National Implementation Research Network. Dean Fixsen, Karen Blase, Sandra Naoom, & Melissa Van Dyke.

 

National Meeting Presentations and PLC and Directors' Webinars

North Dakota Scaling-up and Implementation Science Framework (PPT, 10/2012. Oas, B., and Dughman, R.)  Brenda Oas and Ron Dughman will present how North Dakota built a supportive system to implement a state wide evidence-based initiative. They’ll cover details on who was involved, what it required of them to facilitate collaborative connections across the system levels and the lessons learned over time.(SESSION RECORDING)

PRESENTATION: Pennsylvania’s Paraprofessional Training Initiative: From Pilots to Statewide Scale-Up and Implementation (PPT, 3/2012) This PPT provides and overview of what Pennsylvania accomplished in changing the culture of professional development for paraprofessionals over the last three SIGs/SPDGs. Based on the Stages of Community Readiness and the Conceptual Framework of Implementation, Pennsylvania has shifted from a state where there were no recognized standards for paraprofessionals' qualifications to state-wide regulations and measurable standards. Concrete products that will be demonstrated include online training modules, the Pennsylvania Credential of Competency, statewide training plan and mini-grants to foster IHE pre-service curriculum alignment to state requirements.

Pennsylvania’s Paraprofessional Training Initiative From Pilots to Statewide Scale-Up and Implementation (PPT, 3/2012)
Pennsylvania Special Education Paraprofessional Initiative Website
Pennsylvania Credential of Competency Checklist
Ten Online Modules: Knowledge and Skill Development for Special Education Paraprofessionals in Pennsylvania
Resources and Training Materials for Professional Development
Pennsylvania Special Education Paraprofessional Publications
If you experience issues with the URLs above, visit the PaTTAN website at www.pattan.net and choose paraprofessionals from the drop down menu under Educational Initiatives on the main page.

PRESENTATION: The Challenges of Building Capacity for Implementation Science in Rural States (PPT, 5/2012) Implementation Conversations, Michael Ferguson, Co-Director, New Hampshire SPDG.

Implementation Conversation PLC Session Recording - Adaptive leadership: Discovering your strengths(January 2012). Dr. Rose Wan-Mui Chu will present an overview of how Minnesota is approaching its evidence-based practices scale up work. Having been a systems engineer, Dr. Chu has a fresh perspective of education and systems change. She’ll share the challenges they are experiencing in realizing their vision, and describe how policy level change impacts the practice level (school, teacher, student, community, business, etc.). We hope you can join us for this meeting.

Related Resources

Gallup Instruments
Now discover your strengths. (2001) Authors, Marcus Buckingham and Donald O. Clifton
"Making Yourself Indispensable" Harvard Business Review, October 2011.
Collective Impact, Standford, By John Kania & Mark Kramer
Complementary Learning - Harvard Family Research, Heather B. Weiss, Directo

Scale Up PLC Discussion: Measuring the effectiveness of professional development Video Recording (Feburary 2012, Facilitated by Jon Dyson) Notes

PRESENTATION: Evaluators' Webinar: Evaluating the Implementation Drivers (PPT,  May 2010)  Gaumer Erickson, A., Ph.D., KS & MO; Morrison, J., Ph.D., OH; Mueller, P., Ed.D., NH & MS.

PRESENTATION: SPDG Implementation Conversations: Show-Me Implementation PPT, Session Recording/ (September 2010) Pam Williams,,Missouri SPDG Director.

Presentation Resources:

Missouri Implementation Audit - Final Report [PDF, 2010] This final report summarizes the findings of Missouri's implementation audit. The purpose of the audit was to assess the range of implementation of educational initiatives & the relationship between the degree of implementation & changes in student achievement. Department staff selected approximately 20 programs/initiatives supported at the state level, & identified districts/buildings implementing the programs/initiatives. It was conducted by an external evaluator, the Leadership & Learning Center.

Missouri Department of Elementary & Secondary Education Implementation Rubric [PDF, 2010] This is an internal review document state staff developed for districts to conduct implementation self assessment (rubric).

Missouri Integrated Model Interview Questions - Building Perspective [PDF, 2010] This file includes example interview questions used with staff at the building level. They looked at four different areas: learning context, professional development, instructional practices & leadership practices.

Missouri Integration Model: Documents to Review School Buildings [PDF, 2010]

PRESENTATION: SPDG Implementation Conversations: Selection of Evidence BasedPractice: Oregon’s Initial Attempts to Derive a Process [PPT, November 2010,  Carrizles, D., Ph.D., Oregon SPDG] How SEAs & LEAs choose innovations to support practices? Dianna Carrizales provides an overview of Oregon's approach to supporting districts & schools in promoting evidence-based practices. She shares a tool they've been using with Districts to evaluate whether an initiative is scale-able.

PRESENTATION:SPDG Directors' Webinar: What Is “It” and How Do We Make “It” Happen? [ PPT, March 2011, Blase, K., PhD; Fixsen, D.L., PhD; Van Dyke, M., LCSW; Duda, M., Ph. In addition, there are notes on various slides from Splean, J., Project Director, Nevada SPDG.]  A pre-requisite for implementing well is clearly understanding what is expected of building staff, District staff, & at the State level.  This means clearly operationalizing the guiding principles of the initiative, the critical components & the activities that make the critical components come alive in classrooms & schools.  This presentation reviews the core features of Practice Profiles, the benefits of developing & using them, and how they interact with the use of the Implementation Drivers to increase the likelihood that innovations will be implemented as intended with good outcomes for students.

PRESENTATION: Scale Up PLC: General Supervision and School Improvement: Utilizing all relevant data to inform statewide efforts for scaling up PPT (June 2011) Presenter: David Guardino, Manager, School Improvement and Accountability, Oregon Department of Education. Webinar Recording

 

Publications and Research Articles

Active Implementation Frameworks for Program Success: How to Use Implementation Science to Improve Outcomes for Children. (PDF, 2012, Metz, A. & Bartley, L., Zero to Three publication). NIRN Associate Director Allison Metz and Implementation Specialist Leah Bartley have co-authored an article for Zero to Three, the journal of Zero to Three: National Center for Infants, Toddlers, and Families. The article outlines how the science of implementation and the use of evidence-based Active Implementation Frameworks can close the research-to-practice gap in early childhood and ensure sustainable program success.

 

Tools and Resources

Consolidated Framework for Implementation Research (CFIR) Wiki. "The goal of this Wiki is to provide an online collaborative space to refine and establish terms and definitions related to implementation and implementation science, promote consistent use of these terms and definitions, and to provide a foundation on which a knowledge-base of findings related to implementation can be developed."

CFIR Constructs with Short Definitions. (PDF) Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. This resource was show-cased at the 2013 Global Implementation Conference. Though developed outside the field of education, their definitions covering intervention characteristics, inner and outter settings, characteristics of individuals, and implementation process elements is applicable to any field. 

Assessing Sites for Model Demonstration: Lessons Learned from OSEP Grantees (PDF, September 2011). Debra Shaver, MaryWagner, Keith Lenz . "The purpose of model demonstration projects (MDPs) is to developnew practice, procedure, or program models on the basis of theory and/or evidence-based research. Each project then implements its model in typical settings, assesses impacts, and, if the model is associated with bene? ts, may go on to disseminate it or scale it up. Since 2005, OSEP has funded four cohorts of MDPs, each focused on a single new and promising (or perhaps poorly understood or implemented) practice, procedure, or program that is deemed to have high potential for improving child outcomes." www.sri.com

North Dakota Capacity Development Tool (DOC, 10/2012)

Oregon's District Capacity Assessment (DRAFT DOC, 7/2012) This assessement is intended to help district leadership teams and state professional development providers assess their readiness with elements related to tiered interventions for academics and behavior and implementation science.

Missouri Integrated Model Implementation (MIM) Implementation Matrix (PDF, 3/2012). This matrix, based on essential features of school improvement, is used to support teams in interpreting multiple sources of data and identifying areas of focus for action planning.

Measuring School-wide Implementation of Academic and Behavior Supports School Implementation Scale (PDF): This evidence-based online survey measures the implementation of key components within multi-tiered models of support school-wide (across all school staff). The measure shows strong validity and reliability and has proven to be beneficial in data-based decision-making for schools, districts, and professional development providers.  Results of the online survey have been used by evaluators and project management to identify professional development/coaching needs, as well as by school leadership teams to target areas for action planning. This cost-effective measure has the potential to provide accurate and reliable data across multiple RTI, PBS and integrated models. 

ICAT Tools ® A Comprehensive Web-Based Management System for IC Teams. ICAT Tools (PDF)The integrated system includes program fidelity and student outcome measures.  IC school, district and state staff have secure access to track and monitor all program activities. School-level and aggregated district reports are available  to inform current progress.  Target audience/level: State, district, school leaders responsible for (IC Team) program implementation and evaluation.

Scaling-Up Within a Statewide Multi-Tiered System of Supports (MTSS) (PPT, 3/2012, Goodman and Harms, SPDG National Meeting) Target Audience/Level: Intermediate school district cabinet-level team and implementation support team and Target SPDG Audience: State-level implementers of MTSS projects, SPDG directors/evaluators

Description: Over the next 5 years, Michigan's Integrated Behavior and Learning Support Initiative (MiBLSi) is working to scale up multi-tiered system of support (MTSS) with 1600 schools and 320 school districts. To achieve this goal, the project will need to partner with Intermediate School Districts (ISDs) across the state who the support local districts and schools. This presentation will highlight the scaling-up process and the tools used at the ISD level to create leadership and implementation support team structures with the focus on fidelity and durability of MTSS implementation. The tools are based on the work of State Implementation & Scaling-up of Evidence-based Practices (SISEP).

Resources: 
Intermediate School District: Cabinet Level Meeting Notes for District Level Implementation (DOC, 3/2012)
Intermediate School District: Implementation Team Meeting Notes for District Level Implementation (DOC, 3/2012)

Innovation Adoption Readiness Model (IARM). Provides a means to identify and track the school through the implementation process. An Innovation Configuration Map (ICM) is a diagnostic tool for measuring the fidelity of implementation of an innovation over time. Both of these tools can be used with the new SPDG Performance Measurement 2 to demonstrate improvement in implementation of SPDG-supported practices over time.  Innovation Adoption Readiness Model (PDF)  Innovation Configuration Map (PDF)

Quality Indicators for Assessing Individualized Services for Preschool with Significant Support Needs (PDF, 7/2012) and Action Plan (EXCEL, 7/2012) Developed by Colorado Department of Education. The Quality Indicators offer guidance to educators and administrators when developing, implementing and evaluating quality programs and services for students with the most significant needs. It's a tool designed to assist those who are educating students with significant support needs or evaluating these programs. 

Ohio Improvement Process (OIP) Implementation Criteria & Rubric (PDF, 3/2012) Ohio Improvement Process (OIP) Graphic (PDF, 3/2012)

Illinois ASPIRE Project Evaluation Tools Overview  [PPT, ASPIRE Director Cox, K.,October 5 & 6, 2009] An overview of two evaluation tools & data collected on levels of implementation for 2007-2008. The primary focus of the ASPIRE project is reading at the K-3 level.

Illinois ASPIRE Project Problem Solving Fidelity Checklist  [PDF, 2009] Allows external evaluation by ASPIRE regional staff of implementation in school sites. The primary focus of the ASPIRE project is reading at the K-3 level.

Illinois ASPIRE. The Self-Assessment of Problem Solving Implementation-School Level (SAPSI-S), 2012-2103, PDF). The Self-Assessment of Problem Solving Implementation-School Level (SAPSI-S) monitors ongoing efforts to establish permanent problem solving procedures, tools, and products and thereby implement a multi-tiered system of supports (MTSS).

Illinois ASPIRE. The Self-Assessment of Problem Solving Implementation at the District Level (SAPSI-D), 2012-2013, PDF. Purpose and Target Participants: The Self-Assessment of Problem Solving Implementation at the District Level (SAPSI-D) monitors ongoing efforts to establish permanent problem solving procedures, structures, tools and products in the implementation of a multi-tiered system of supports (MTSS). The district leadership team should complete the SAPSI-D once each academic year in the spring. The SAPSI-D can, however, be completed more frequently (e.g. once per semester) for the purposes of further district level planning.

National Implementation Research Network (NIRN) Resources

Realizing Excellence for ALL Children in Mississippi (REACH Mississippi) Mississippi's SPDG focusing on supporting school- & district-wide implementation of PBIS at the elementary, middle & high school levels.

Scaling Up Resources [Modules]

ERIA: Effective Reading Interventions Academy A Pathway Towards RtI2 [PDF, December 10, 2010] This 16-page Program Guide explicitly addresses both implementation & intervention practices to guide the design of a site-based program.

ERIA Implementation Rubric [XLS, July 2010, CalSTAT] This 10-item instrument provides a framework for trainers, coaches, site team members & teachers to evaluate & discuss implementation, fidelity & next steps.

BEST PBS Implementation Rubric [XLS, July 2010, CalSTAT] This interactive template is used by school teams to record their implementation level for ten key elements. After entering their data teams can generate reports.

Scaling Up Evidence-Based Practices Exploration Tool: Assessing Usability of Evidence-based Programs, Practices, or Educational Frameworks Evidence-Based Practices [DOC] This tool is to assist LEA staff in determining if practices they are considering are evidence-based. This has a much broader use & has been developed by Bloomington Public Schools as part of its partnership with SISEP & is used for PBIS, RTI, literacy, other curricular areas.

Alignment of Literacy Practices that Positively Impact Student Performance - Evidence-Based High Growth Tool  [DOC, March 28, 2010] This tool is one that was specially developed for the SPDG aligned literacy project which brings together general & special educators in an LEA as they look comprehensively at their literacy curricula & determine whether they offer a complete continuum of curricula to address all of the literacy needs of all learners, including those with severe disabilities. This tool is used with a PLC of LEAs in an aligned literacy project.

Program Mapping Tool At-A-Glance [XLS] This tool is one that was specially developed for the SPDG aligned literacy project which brings together general & special educators in an LEA as they look comprehensively at their literacy curricula & determine whether they offer a complete continuum of curricula to address all of the literacy needs of all learners, including those with severe disabilities. This tool is used with a PLC of LEAs in an aligned literacy project.

Terms of Reference [DOC] A process that has been utilized by the State Readiness Teams to offer a process for clarifying &  documenting communication protocols & core features and functions of a newly established team.

Instructional Consultation Team (ICT) Sustainability Framework [DOC, Matrix developed through the collaborative efforts of: Gravois, T.,  ICAT Resources;  Hall, G., University of Nevada, Las Vegas; Caffarella, E., State University of New York at Cortland; & Cleven, M., Fitzpatrick R. & Splean, J., Nevada Department of Education.] This framework was developed by the IC Leadership Team & implemented with district leadership teams. The Framework was intended to serve as a District Planning Tool, guiding the development of prioritized needs, actions & resources relative to Sustaining Effective Instruction through ICTs. The Implementation Drivers along with the Aligned Services Model were the primary resources utilized & embedded in the overall framework including the components of Sustainability Considerations & Key Indicators.

Here is a link to the coach support page for Michigan's Integrated Behavior and Learning Support Project (MiBLSi): http://miblsi.cenmi.org/Coaching.aspx

Developing Training Capacity for Statewide Implementation of Michigan's Integrated Behavior & Learning Support Initiative (MiBLSi): A Response to Intervention Model for Reading & Behavior Support [DOC]

MiBLSi District READING SUPPORT Coaching Capacity Self-Assessment [DOC, 2011, Draft]

MiBLSi District BEHAVIOR SUPPORT COACHING CAPACITY Self-Assessment [DOC, 2011, Draft]

MiBLSi Identifying Skill Sets Along the Continuum of Support [DOC, 2011]

MiBLSi Building Leadership Team Response to Intervention Practice Profile [DOC, April 2011]

Supporting Administrators in the implementation of Response to Intervention [PPT, April 2011, Goodman, S., Co-Director, MiBLSi] Presentation describing how the Michigan SPDG has formalized their PD approach for administrators.

Evidence-Based High Growth Tool [DOC] This tool is used by LEA staff to determine to different levels of support that are needed to successfully implement an evidence-based practice. This has much broader applicability than literacy.

New York S3TAIR Site Validation Protocol: Review of Current Practice Status  [DOC, February 2011] This document provides an opportunity for S3TAIR Project Replication Schools to review the current status of the practice they will address through the Project. Based on the protocol used by Project site visit teams in determining the validity of nominated practices, the Replication School Protocol presents descriptions of elements of practices that have been determined through research to be correlated with successful implementation & maintenance of evidence based effective practices & in turn in positive outcomes for students with disabilities.

Scaling Up Evidence-Based Practices Exploration Tool: Assessing Usability of Evidence-based Programs, Practices, or Educational Frameworks Evidence-Based Practices [DOC] This tool is to assist LEA staff in determining if practices they are considering are evidence-based. This has a much broader use & has been developed as part of a partnership with SISEP & is used for PBIS, RTI, literacy & other curricular areas.

New York S3TAIR Site Validation Protocol  [DOC, February 2011] The New York SPDG's S3TAIR premise of professional development through partnerships is facilitated by identifying schools/districts with best practices in the areas of literacy, positive behavioral supports & special education & pairing them with schools/districts that have been identified as in need of improvement or assistance. This dynamic tool has been utilized to: 1) support the S3TAIR site validation teams in the identification of effective practice schools & 2) reflect, assess & evaluate the current status of effective practice mentor schools & schools in need of improvement, termed “replication schools,” having been paired with a mentor school.

Training in Instructional Consultation, Assessment & Teaming  [PDF, 2007, Gravois, T., Gicklin, E. & Rosenfield, S.]