Project evaluation is an integral part of project management. Often, it is associated with the final stages of a project, but in reality, from the very moment you start thinking and developing a project idea, you also constantly evaluate it.
If you and your team develop excellent project management skills, you might not need to have project evaluation as a separate process, but excelling at project management is only possible if evaluation is enshrined in its every aspect.
Definition
Evaluation is the systematic and objective assessment of an ongoing or completed project, program or policy, its design, implementation, and results.
Why is evaluation important?
Evaluation helps get better project results: Progress review and evaluation tools make deadlocks, overruns, etc. visible, so problems are identified at an early stage. You have time to think through and react accordingly. Thus, you get better control over your project, improve outcomes and enhance the impact.
Evaluation allows you to learn from experience: You understand what has led to successes and failures and how to improve current or future projects based on this knowledge.
Evaluation enhances transparency and improves communication: It shows how the resources were used and increases accountability towards your project donors, supporters, and participants. It can, for example, increase your chance of getting follow-up funding. Moreover, evaluation allows communicating about the results and the effects of your project in a detailed and evidence-based manner.
Evaluation stages
The evaluation process can be divided up into three main stages: Initialization/conception, implementation, and finalization.
Initialization/conception
It is the beginning of the project evaluation. At this stage, the main objective is to determine the viability of your project. There are a few questions you should find a convincing answer to understand whether your project is worthy of your time and effort.
- Why is this project important? – Think about the context of the project: How is it linked to the vision and mission of oikos? How it contributes to the overall strategy of your oikos chapter? To what extent it responds to the needs and requirements of your target group(s)?
- Who are your target groups in this project? – Think about each individual group and what they might expect from the project.
Example: In case of oikos chapter projects, usually, fellow university students are the key target group, though you should not forget about other stakeholders. Think about university administration and faculty, as well as local startups or big companies: How can they support you and what might be their expectations towards you?
3. What do you want to achieve? – Suppose the project is conducted and was a huge success: Think about what you would have achieved and how and by what means you recognize that it was a success. Set clear goals and targets, determine key performance indicators (KPIs) to learn whether you are meeting the targets. Decide on the time frame: When do you want to achieve these goals?
Example: In case of a speaker event on doughnut economics, your goal is to introduce the theory to a greater number of young people and at the same time to ensure that students are happy with the content and organization of the event. During the initiation stage, you should set the target value. It helps to decide whether the event succeeded or not. For example, you can predetermine, that if 100 participants attend the event and 80% are satisfied with the content and organization of the event, only then you say that the event is a success.
4. What is the scope of the project? – Determine the extent and reach of your project, at the same time think about what you want to exclude from the project.
Tip: Learn to say no. Projects that keep on expanding and add an ever-greater number of objectives are not usually successful.
5. What are the costs of the project and the resources you need? Determine your budget and identify what are the necessary skills and roles in the team to deliver the project.
Tip: Do not forget about the time you and your team will need to spend on organizing the event. It is the most valuable resource, and as students, you might not have a lot of it.
6. What are the risks you might face? Determine what the likelihood of the risk is and if its impact outweighs the potential benefits of the project.
Tip: Risks are a constant and you have to learn to think about them before they occur. It helps you manage the situation with less stress as you already have a plan to overcome the problem.
Implementation
At this stage, your project slowly comes to life, and you can observe its first results. Monitoring and evaluation of the project can tell you if you are staying within the project budget and schedule. The better your project management and evaluation during the project is, the less effort is needed for end evaluation. There are a few things you need to do to ensure that the implementation stage goes smoothly.
1. Order and schedule: Decide on priorities and what you should do first, create a schedule for milestones.
Tip: Answering these questions help you decide what ends up in your list of milestones.
– How important is a specific task or event to the execution of the overall project?
– What is the likely impact if this task or event is not implemented on time or as it was expected to?
– Can the execution of this task or event be used as an indicator of the project success?
2. Allocate resources and assign roles: Resources are limited, so you need to decide how to allocate them in the most efficient way possible. It is also crucial to determine team responsibilities and tasks – delegate!
Useful tool: Realtime Board can give your team a good overview of what each individual team member is doing and how much progress they are making.
3. Evaluate success: Decide whether you invite external evaluators or allow team members to take the lead and conduct the internal evaluation.
Main considerations: There are pros and cons to both options. But note, project evaluation is most effective, but also most painful if you choose external evaluators.
If you opt for internal evaluation, you need to decide on qualitative and quantitative data you require to measure your success, how you will gather it and how often you will measure your progress.
To properly evaluate success, you will need to decide on the right evaluation methods and tools.
Tip: Review and study your goals carefully. They will dictate you which evaluation method and tools to choose.
There are two basic evaluation methods:
- Quantitative methods measure things like how many, how much, how long. Data gathered by this method, therefore, are expressed in numerical form.
- Qualitative methods measure things like awareness and attitude. Data gathered by this method, therefore, are descriptive.
Each of these methods provides useful tools for measuring success. They are divided into two categories: Tools for an event evaluation and tools for a continuous process review.
Event evaluation tools:
1.Headcount: Some quantitative data can be gathered through headcount or reviewing project records, like attendance sheets. This very simple tool helps answer questions, such as how many participants attended your project, how many new connections did you make, how many partners did you involve in the project.
2. Questionnaires: Questionnaires can help you measure almost everything. You can learn how satisfied participants were with the project, how useful did they find it, how their knowledge, skills, and attitudes were influenced by the project. Of course, to answer the latter question you might need to send out the questionnaire at least twice. First get baseline data – level of knowledge, skills, and attitudes before the project, then compare it with the responses you receive from the second questionnaire sent out after the project.
If you ask open-ended questions, that require participants to describe their experience, you gain more in-depth insights into the strengths and weaknesses of your project.
3. Interviews: Not as many people might fill in questionnaires as you want or need to. To ensure that you have enough data, you can conduct interviews on the spot after or during your event. You can use the same questions that you would have included in the email questionnaire. But you can also improvise and ask more detailed questions if you find it necessary. That is one great advantage of face-to-face interviews.
Tip: If you do not want to interfere with the flow of the project with official face-to-face interviews, take a different approach. Go out there and talk to participants, ask them the questions you want to have answers to, but do it with ease and as a part of a random conversation. You can reflect on their responses in a learning journal.
Tools for a continuous process review:
1.Learning journals: During the project management you observe dozens of things at the same time, ideas and thoughts come and go, try to record them in learning journals and reflect on your project and progress as you go. It can help you identify useful processes and information and interpret them accordingly.
Tip: Do not just describe what you did or what you saw, instead focus on explaining why and how you did what you did or why and how did you decide something was worthy of attention when you saw it.
2. Milestone and resource tracking: Tracking milestones and KPIs, as well as sources and use of financial and non-financial resources, keeps you always up to date on the proceedings of your project. The key to milestone and resource management is to be informed and prepared, so revisit them and learn how your project calculations are met in a real working environment. Keep your team and stakeholders updated about existing or potential problems. Be frank, honest and flexible when working on the milestone and resource tracking.
3. Risk Management: You must constantly evaluate the actions you take to mitigate risks. Your team members should know that they are responsible to foresee incidents, determine what might go wrong and ensure that everything stays on track.
Example: In a one-day conference, things can go wrong too. What if you planned for a 60-minute lunch break and order food from a nearby diner, how do you ensure that food arrives on time. A simple spreadsheet can help you oversee everything:
4. Interim status workshops with stakeholders: If your project spans across a long period of time and involves multiple important stakeholders, consider inviting them to an interim status workshop. This workshop informs you about how stakeholders respond to your project thus far and what are their expectations going forward.
Example: Suppose that you work on a curriculum change project that involves creating a new module on sustainability. You closely engage with students, professors, lecturers and administration at the university. Each of these stakeholders has a specific role and tasks in the project. Therefore, after a certain time, you might need to bring these people or at least a representative from each group together to learn about how each of them evaluates the progress, what are the shortcomings they seek to improve or achievements they want to capitalize upon.
Finalization
At this stage, your project is already completed. It’s time to look back, interpret your data, identify key findings and lessons learned. It is also crucial that you report to stakeholders and communicate about the project results. Consider taking the following steps to complete your project evaluation successfully.
1. Data analyzes and interpretation: Once you gather all the necessary data, you should categorize and analyze it. This process allows you to discover useful patterns and gain information for improving your projects in the future.
Useful tools: Softwares like tableau, looker, and others, are one of the easiest tools you can use for data analyses, interpretation, and visualization.
2. Review workshops: Project team has insights and knowledge that can improve your projects, so once it is over, gather together and discuss the whole venture at the Team Review Workshop. The team receives feedback from customers, sponsors, partners, and other stakeholders. Use this input to determine main drawbacks, achievements and lessons learned. At the workshop focus on three main questions: What were you doing well? What could you do better? What more could you do?
Useful tool: Dotmocracy is a team review workshop conducted with the help of post-it notes and dots. Organized in three parts, it will help you to find answers to the three main questions posed in the workshop. Learn how to run dotmocracy with your team here.
3. Once you analyze your data and determine the main results of your project, organize your findings and lessons learned into a coherent report or a case study. For youth organizations such as oikos, where leadership always changes it is especially important. The report and case study are crucial for ensuring the transfer of knowledge to the new generations of members.
Tip: If your project was funded by an external donor, they might request you to report back on the project results once it is over. Even if they do not explicitly ask for it, it is still useful to reach out to donors and other stakeholders to report on the process and outcomes of your project. It demonstrates that you hold yourself accountable and informs stakeholders how you managed resources allocated by them and what results they brought. On the other hand, it is useful for organizational learning: It might prove the most important tool for educating the next generation of project managers, as it makes comparison across projects easier and enables sharing of best practices, as well as the collection of dos and don’ts in project management.
Project evaluation requires planning and continuous engagement, but it is a very useful process worth your efforts and time. It allows you to learn about your project outcomes, understand its main achievements and drawbacks, get better results and develop new projects informed by real experience and evidence-based knowledge.
###
This article was prepared as a follow-up to the Project Evaluation Workshop conducted by Dr. Sabine Ruoss of Helvetia at the LEAP Meeting 2018. It summarizes and builds upon the main points raised by the speaker and participants during the workshop.
About LEAP
oikos LEAP is designed to inspire young leaders to become more responsible in their decision making and equip them with insights, knowledge, and tools to do so. oikos LEAP operates in three tracks: Advanced, President and Basic. All tracks are intended to inspire participants to take increasingly responsible actions. Learn more about the program here.
