Effectiveness
Given ADB's annual lending volume, the return on investment in lesson-learning for operational and developmental impact is likely to be high; maximizing it is a legitimate concern. All projects funded by ADB are evaluated to find out what results are being achieved, what improvements should be considered, and what is being learned.
There are two types of evaluation: independent and self-evaluation. Self-evaluation is conducted by the units responsible for designing and implementing country strategies, programs, projects, or technical assistance activities. It comprises several instruments, including project/program performance reports, midterm review reports, technical assistance or project/program completion reports, and country portfolio reviews. All projects are self-evaluated by the relevant units in a project completion report. ADB’s project completion reports are publicly disclosed on ADB’s Internet site. Client governments are required to prepare their own project completion reports.
Independent evaluation is a foundation block of organizational learning: It is essential to transfer increased amounts of relevant and high-quality knowledge from experience into the hands of policy makers, designers, and implementers. ADB’s Independent Evaluation Department (IED) conducts systematic and impartial assessment of policies, strategies, country programs, and projects, including their design, implementation, results, and associated business processes to determine their relevance, effectiveness, efficiency, and sustainability following prescribed methods and guidelines. It also validates self-evaluations. By this process of evaluation, ADB demonstrates three elements of good governance: accountability, by assessing the effectiveness of ADB's operations; transparency, by independently reviewing operations and publicly reporting findings and recommendations; and improved performance, by helping ADB and its clients learn from experience to enhance ongoing and future operations.
Operations evaluation has changed from the beginnings of evaluation in ADB in 1978. Initially, the focus was on assessing after completion the extent to which projects had achieved their expected economic and social benefits. Operations evaluation now shapes decision making throughout the project cycle and in ADB as a whole. Since the establishment of its independence in 2004, IED reports directly to ADB’s Board of Directors through the Board's Development Effectiveness Committee. Behavioral autonomy, avoidance of conflicts of interest, insulation from external influence, and organizational independence have made evaluation a dedicated tool—governed by the principles of usefulness, credibility, transparency, and independence—for greater accountability and making development assistance work better. Independent Evaluation at the Asian Development Bank presents a perspective of evaluation in ADB from the beginnings and looks to a future in which knowledge management plays an increasingly important role.
In recent years, there has been a major shift in the nature of IED's work program from a dominance of evaluations of individual projects to one focusing on broader and more strategic studies. To select priority topics for evaluation studies, IED seeks input from the Development Effectiveness Committee, ADB Management, and the heads of ADB departments and offices. The current thrusts are to improve the quality of evaluations by using more robust methodologies; give priority to country/sector assistance program evaluations; increase the number of joint evaluations; validate self-evaluations to shorten the learning cycle; conduct more rigorous impact evaluations; develop evaluation capacity, both in ADB and in DMCs; promote portfolio performance; evaluate business processes; and disseminate findings and recommendations and ensure their use. IED's work program has also been reinterpreted to emphasize organizational learning in a more clearly defined results architecture and results framework. It entails conducting and disseminating strategic evaluations (in consultation with stakeholders), harmonizing performance indicators and evaluation methodologies, and developing capacity in evaluation and evaluative thinking. All evaluation studies are publicly disclosed on IED's website (some evaluations of private sector operations are redacted to protect commercially confidential information). IED's evaluation resources are displayed by resource type, topic, region and country, and date. Learnings are also gathered in an online Evaluation Information System offering a database of lessons, recommendations, and ADB Management responses to these. Details of ongoing evaluations and updates on their progress are made public too.
Beginning 2006, acting within the knowledge management framework of ADB, IED has applied knowledge management to lesson learning, using knowledge performance metrics.
Learning Lessons in ADB sets the strategic framework for knowledge management in operations evaluation. Improvements have been made that hold promise not only in IED but, more importantly, vis-à-vis its interfaces with other departments and offices in ADB, developing member countries, and the international evaluation community. In the medium term, IED will continue to improve the organizational culture, management system, business processes, information technology solutions, community of practice, and external relations and networking for lesson learning. Among the new knowledge products and services developed, Learning Curves are handy, two-paged quick references designed to feed findings and recommendations from evaluation to a broader range of clients Evaluation News report on events in monitoring and evaluation. Evaluation Presentations offer short photographic or Powerpoint displays on evaluation topics. Auditing the Lessons Architecture highlights the contribution that knowledge audits can make to organizational learning and organizational health.
Of the 1,106 ADB-funded projects evaluated and rated so far (as of December 2007), 65% were assessed as being successful, 27% partly successful and 8% as unsuccessful.
Read more about this topic: Asian Development Bank