Program Monitoring

What Leadership Can Do to Promote Teacher Retention

This infographic was designed to help program leaders understand the data behind who teaches Adult Education and the potential cost of teacher attrition. It also highlights strategies on what program leaders can do to improve teacher retention.

Using Logic Models for Program Planning and Evaluation

Purpose of the Logic Model Tool:

• Provide an overview of what logic models are and how they can be used.
• Provide an explanation of the components of logic models.
• Provide an example illustrating what a logic model is and how it can be used to evaluate adult education program activities.

What Are Logic Models?

Logic models provide an approach for program planning, management, monitoring, and evaluation. They present a concise, graphic depiction of what a process, project, program, or system comprises and what it is intended to do. Logic models map the resources and actions needed to reach the target population and how ultimately desired program outcomes will be achieved. While logic models can serve multiple purposes, their primary advantage is their ability to visually capture essential program components and operations and help foster understanding and consensus among staff and other stakeholders.

Varying formats and terms are used when developing logic models, but the same essential components and concepts are found across them all: (a) inputs, such as the resources needed to drive the program; (b) outputs, such as the activities that serve the participants; and (c) outcomes, such as the results and changes that take place during and after program implementation. The logic model framework provides a succinct, “at a glance” picture of the programming and shows the sequential relationships among all program components and the intended results.

Logic Models in Adult Education

At the state and local program levels, logic models can be useful in planning initiatives, monitoring performance and data management, and evaluating effectiveness. At the state level, for example, a logic model can be used to plan how a new policy will be rolled out or how data from each program will be analyzed. At the local level, programs can chart how they will provide professional development or map the flow of data from collection to management and analysis to state submission.

Regardless of context, the logic model illustrates which resources are needed to provide activities that will produce results that lead, in turn, to desired outcomes. By reviewing these relationships, you can determine if program elements are being implemented as intended and working as designed or if adjustments are needed.

Steps to Creating and Using a Logic Model

You may discover performance issues in your end-of-year data, hear about trends from local program directors and instructors, or uncover issues during a monitoring visit. Some problems may result from how resources are managed, services are delivered, or data are collected. Once you pinpoint an area for further investigation, it is useful to develop a logic model to map the problem you intend to resolve by specifying the outcomes you want, determining the resources and procedures you need to achieve them, and identifying the external factors and assumptions you make that affect the entire process.

By using a logic model, you can formulate targeted questions that will allow you to uncover what is at the heart of the challenges your program is experiencing. You can complete a logic model using the steps below, although you can vary the order of the steps according to your own development process.

1. Identify a topic or problem you want to address and the goal you have for solving it. State your topic or problem related to a process, project, program feature, program, or system and define the goal, which represents the solution to the problem and will dictate the other components of the model.
2. Determine the short-term, intermediate, and long-term outcomes you want to achieve. The outcomes are the desired changes you expect to see as a result of the activities you identify in the logic model over time. You should be able to measure whether you have reached those outcomes. When you reach the desired outcomes, you will have solved the problem and met your goal.
3. Choose the outputs needed to reach the outcomes and the goal. Activities are actions, services, processes, products, and events that are implemented and created to reach the desired goal. You must have the necessary resources to conduct these activities, and the activities must be geared to the target beneficiaries. Think through the participants: Who is affected and who gains when activities are conducted.
4. Identify the inputs needed to reach the outcomes and the goal. Think about all the resources that are needed to implement the activities successfully. These may include people, materials, equipment, space, technology, and funding.
5. Think through assumptions. There are certain existing conditions that support the success of the program or system. Factor these assumptions into your model so you do not spend time developing, creating, or fostering something that is already in place. Also, consider who you can capitalize on these conditions.
6. Think through the external factors. Circumstances, conditions, or events exist outside the purview of your program. There might be local or state policies or priorities that support or conflict with your goals. There may be events that provide or limit resources. It is important to consider outside forces and the impacts they can have on your implementation efforts.
7. Review the logic model to evaluate implementation. Ensure that relationships among the model’s components are clear. Determine if the inputs support the outputs and if the outputs will meet the end goal. You may find gaps in resources, or you may identify additional activities that should be conducted. You can continually update the model as you discover what is happening within the program or system and what might be implemented better.

Below is an example of a logic model developed to address a problem or issue identified through National Reporting System data analysis. This logic model depicts existing program components and desired outcomes, and it addresses a single problem, addressed as Employment outcomes for Integrated Education and Training (IET) participants are below the target. Also included in the example are blue boxes with the headers and definitions for each section of the model.

TOPIC OR PROBLEM:

GOAL:

Conclusion

Logic models can help administrators, practitioners, and other stakeholders understand the structure of a program and how its component parts fit together and operate. The framework can be used for planning, organizing, managing, monitoring, and evaluating the program to ensure that it is implemented to achieve the desired results.

References

• McLaughlin, J. A., & Jordan, G. B. (1999). Logic models: A tool for telling your program’s performance story. Evaluation and Program Planning, 22(1), 65–72. https://doi.org/10.1016/S0149-7189(98)00042-1
• Savaya, R., & Waysman, M. (2005). The logic model: A tool for incorporating theory in development and evaluation of programs. Administration in Social Work, 29(2), 85–103. https://doi.org/10.1300/J147v29n02_06

NRS Tips: Desk Monitoring

Desk monitoring is an effective and efficient way to assist states in meeting their monitoring needs. It provides an approach to reviewing and tracking program performance by using quantitative data that serves as a supplement to on-site monitoring. Local programs provide data regularly (e.g., monthly, quarterly) to the state office where state staff can review them to guide program review activities, inform technical assistance plans, and promote program improvement. Desk monitoring relies primarily on data that programs already collect and report. It is inexpensive to implement and, once established, requires little additional burden to local programs. Based on research and ideas from state and local program staff, the NRS project team created a list of desk monitoring tips.