Document of The World Bank FOR OFFICIAL USE ONLY Report No: 27704 IMPLEMENTATION COMPLETION REPORT (SCL-41670) ON A LOAN IN THE AMOUNT OF US$33.0 MILLION TO GUATEMALA FOR A BASIC EDUCATION REFORM PROJECT 5/17/2004 This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. CURRENCY EQUIVALENTS (Exchange Rate Effective ) Currency Unit = Quetzal 1.00 Quetzal = US$ 0.13 US$ 1.00 = 7.58 Quetzales FISCAL YEAR 1 January to 31 December ABBREVIATIONS AND ACRONYMS CAS Country Assistance Strategy COEDUCA Parent-run School Committee (Comite Educativo de Autogestión Educativa) DICADE Dirección de Calidad y Desarrollo Educativo DIGEBI General Directorate of Bilingual Education (Dirección General de Educación Bilingüe) EMIS Education Management Information System FIS Social Investment Fund (Fondo de Inversión Social) FONAPAZ National Fund for Peace (Fondo Nacional para la Paz) FSDC Social Fund for Community Development FUNRURAL Foundation for Rural Development GOG Government of Guatemala IDB Inter-American Development Bank INE National Statistics Institute ISEs Instituciones Especializadas en Servicios Educativos MCS Ministry of Culture and Sports MEDIR Evaluation of Intercultural Bilingual Education MFP Ministry of Public Finance MINEDUC Ministry of Education NEUBI Nueva Escuela Unitaria Bilingüe Intercultural PAD Project Appraisal Document PAEB Bilingual Intercultural Education Project in the Quiche area PCU Project Coordination Unit PEMBI Mayan Bilingual Intercultural Education PRONADE National Community-Managed Program for Educational Development (Programa Nacional de Autogestión para el Desarrollo Educativo) SEP Secretaría Ejecutiva de la Presidencia de la República SIAF Integrated System for Financial Administration SIGEPRO Sistema Integrado de Gerencia de Proyectos SIMAC Sistema de Mejoramiento de los Recursos Humanos y Adecuación Curricular SPEB Second Basic Education Project UDI Informatic Unit (Unidad de Informática del MINEDUC) UNDP United Nations Development Program UPE Education Planning Unit URNG National Guatemalan Revolutionary Union ZONAPAZ Geographic area that includes nine departments most affected by the armed conflict Vice President: David de Ferranti Country Director: Jane Armitage Country Sector Leader: Helena Ribe Task Team Leader: Carlos Rojas GUATEMALA BASIC EDUCATION REFORM PROJECT CONTENTS Page No. 1. Project Data 1 2. Principal Performance Ratings 1 3. Assessment of Development Objective and Design, and of Quality at Entry 2 4. Achievement of Objective and Outputs 3 5. Major Factors Affecting Implementation and Outcome 12 6. Sustainability 13 7. Bank and Borrower Performance 15 8. Lessons Learned 17 9. Partner Comments 19 10. Additional Information 31 Annex 1. Key Performance Indicators/Log Frame Matrix 32 Annex 2. Project Costs and Financing 34 Annex 3. Economic Costs and Benefits 36 Annex 4. Bank Inputs 37 Annex 5. Ratings for Achievement of Objectives/Outputs of Components 39 Annex 6. Ratings of Bank and Borrower Performance 40 Annex 7. List of Supporting Documents 41 Project ID: P007223 Project Name: GT BASIC EDUCATION REFORM Team Leader: Carlos A. Rojas TL Unit: LCSHE ICR Type: Core ICR Report Date: May 17, 2004 1. Project Data Name: GT BASIC EDUCATION REFORM L/C/TF Number: SCL-41670 Country/Department: GUATEMALA Region: Latin America and the Caribbean Region Sector/subsector: Primary education (55%); Central government administration (30%); Sub-national government administration (13%); Tertiary education (2%) Theme: Education for all (P); Indigenous peoples (P); Gender (P); Access to urban services for the poor (P); Rural services and infrastructure (P) KEY DATES Original Revised/Actual PCD: 11/24/1993 Effective: 03/19/1998 03/19/1998 Appraisal: 03/31/1997 MTR: 10/10/2000 10/10/2000 Approval: 05/20/1997 Closing: 12/31/2002 06/30/2003 Borrower/Implementing Agency: REPUBLIC OF GUATEMALA/MINISTRY OF EDUCATION Other Partners: STAFF Current At Appraisal Vice President: David de Ferranti S. Javed Burki Country Director: Jane Armitage Donna Dowsett-Coirolo Sector Director: Ana Maria Arriagada Julian Schweitzer Team Leader at ICR: Carlos Rojas Carlos Rojas ICR Primary Author: Angela Demas 2. Principal Performance Ratings (HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HL=Highly Likely, L=Likely, UN=Unlikely, HUN=Highly Unlikely, HU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible) Outcome: S Sustainability: HL Institutional Development Impact: SU Bank Performance: S Borrower Performance: S QAG (if available) ICR Quality at Entry: S Project at Risk at Any Time: No 3. Assessment of Development Objective and Design, and of Quality at Entry 3.1 Original Objective: The project's main objectives are to: a) increase coverage for pre-primary and primary school children in rural areas, especially among indigenous groups and girls, through a community-based school management program (PRONADE) and by introducing afternoon school shifts; b) improve efficiency and quality through the support of bilingual education and multigrade teaching methods; and c) strengthen the organization and management of MINEDUC and decentralize education administration to encourage greater community participation. There is a geographic focus on the areas most affected by decades of internal conflict (ZONAPAZ). 3.2 Revised Objective: No revisions were made to the project objectives. 3.3 Original Components: The total project costs were estimated at US$46.8 million of which US$33.0 million would be financed by the loan and US$13.8 million by the government. The project was comprised of three components: Component I: Expansion of Coverage and Equity (US$22.4 million or 48 percent of total project costs). (a) PRONADE - Programa Nacional de Autogestion para el Desarrollo Educativo. (National Community-Managed Program for Educational Development); (b) Programa de la Niña (Girls' Education Program); (c) Afternoon Shifts Component II: Improvement of Efficiency and Quality (US$9.5 million or 20 percent of total project costs). (a) Curriculum Development; (b) Bilingual Education; (c) Multi-grade Schooling in Rural Guatemala; (d) Fund for Innovations to Support Educational Quality; (e) National Assessment System - NAS Component III: Institutional Strengthening (US$13.6 million or 29 percent of total project costs). (a) Support to ISES to train COEDUCAS; (b) Training to MINEDUC and COEDUCAS in new systems of administration; (c) Strengthening of MINEDUC (planning, human resources, financing, administration); (d) Strengthening of the 22 Departmental Directorates; (e) Development and improvement of the Education Management Information Systems (MIS); and (f) Design and implementation of a system of project evaluation and execution. Component IV: Project Administration (US$1.4 million or 3 percent of total project costs). 3.4 Revised Components: In response to the structural damage and crisis brought about by Hurricane Mitch in 1998, the first - 2 - component of the project, Expansion of Coverage and Equity, was modified to include school rehabilitation and construction. This required a reallocation of resources whereby civil works was increased significantly in relationship to the original foreseen expenditures in this category. The revision required an amendment to the Loan Agreement and implementation plan. The following table shows the reallocation of funds to civil works: Amendment to the Loan Agreement (US$ M) Investment Categories Original Modified Changes 1. Civil Works 1,000 8,000 +7,000 2. Goods 2,700 2,000 -700 3. Consultant Services 8,200 6,500 -1,700 4. Training 2,500 2,000 -500 5. Salaries and Operational Costs 11,500 11,5000 0 6. Educational Innovations 3,300 3,000 -300 7. Unallocated 3,800 0 -3,800 Total 33,000 33,000 0 Source: Legal Agreement and Amendment of December 1998. 3.5 Quality at Entry: ICR Rating: Satisfactory Objectives of the Project were fully consistent with the Guatemala CAS effective at the time of entry. They also were reflective of the objectives as stated in the Government's five year Education Strategy. The project was designed in a participatory manner with the direct participation of the Vice Minister of Education, department heads, and functionaries of the Ministry in every aspect related to the scope and content of the components. This involvement assured that the objectives and activities contemplated were well defined and also minimized the chance of complications during implementation. The input provided by government and other stakeholders fit within the context of the country that had recently ended many years of internal conflict with the signing of the Peace Accords in 1996. The collaborative effort resulted in realistic project development objectives that were built on the foundation of the agreements reached through the Peace Accords. Critical risks that were identified in the PAD as well as lessons learned from previous projects also contributed to management design and execution of the project. A Project Coordination Unit (PCU) rather than an implementation unit was set up with the responsibility to coordinate and monitor activities instead of executing them. Implementation was assigned to the respective operational departments of the Ministry. Additionally, parts of the activities were also contracted out to national entities outside of the Ministry with expertise and abilities better suited for certain assignments, such as the Universities of San Carlos, Rafael Landivar, Francisco Marroquin, and del Valle. 4. Achievement of Objective and Outputs 4.1 Outcome/achievement of objective: Overall ICR Rating: Satisfactory The project objectives established in the Project Appraisal Document (PAD) continue to be as relevant today as they were when the project commenced. They also continue to be consistent with the overall education policies currently being implemented by MINEDUC in order to improve the coverage, equity and - 3 - quality of the education system. The Project was designed to support the compliance of the approved initiatives established by the Peace Accords and were agreed upon by the Government of Guatemala and the National Guatemalan Revolutionary Union (URNG) with the final accords completed on December 29, 1996. The Project's aim to help increase school coverage focused primarily on the nine rural departments that form ZONAPAZ, the area most affected by the armed conflict during the civil war. It is also the geographic area with the highest concentration of the population facing extreme poverty and registers the lowest educational indicators in the country. The project satisfactorily increased coverage for pre-primary and primary school children in the poorest, rural areas; improved the efficiency and quality of the delivery of education through the support of bilingual education and multigrade teaching methods; and strengthened and organized the management of MINEDUC. The achievement of these three main objectives are addressed individually below. a) Increase coverage for pre-primary and primary school children in rural areas, especially among indigenous groups and girls, through a community-based school management program (PRONADE) and by introducing afternoon school shifts; ICR Rating: Highly Satisfactory The community-based school program (PRONADE) far surpassed the expected number of children to be enrolled during the lifetime of the project. Originally the project estimated that coverage would be extended to about 25,000 more students per year, so that by year 2000, about 75,000 more students would be attending PRONADE schools. The actual numbers show that the goal of 75,000 students was easily met and by year 2000 under the project, PRONADE had expanded to accommodate 209,000 children. At project end, more than 3,500 isolated rural, indigenous communities (COEDUCAs) were providing basic education to more than 300,000 students. Teachers appointed and number of schools organized have also surpassed the targets established (see Annex 1). Coverage of PRONADE schools increased from 9.8 percent to 12.8 percent share of the primary school population, and it increased from 11 to 14.8 percent of the official public school population. These achievements have contributed to the growth of the primary net enrollment rate which increased from 69.1 percent in 1996 to 85.1 percent in 2001, according to MINEDUC statistics. At the pre-primary level, net enrollment grew from 20.7 percent in 1996 to 41.3 percent in 2001. Furthermore, gross enrollment for primary students has reached about 103 percent. While the number of boys still outnumbers the amount of girls enrolled, the gap may have narrowed by a few percentage points. For both girls and boys and rural and urban children, significant improvements for both net and gross enrollment have been realized. Net enrollment for girls has increased from 65.7 percent in 1995 to 83.15 percent as reported in 2001 and for boys from 72.6 percent to 87.05 percent. With measurements starting in 1997, gross enrollment of rural children enjoyed an average annual increase of 7.6 percent. Increases ranged from about 4 percent to almost 14 percent, with the largest increases achieved during 1998 and 1999 at a little over 13 percent each. This is in direct correspondence with the large expansion of PRONADE communities supported by the project in those same years. The introduction of afternoon shifts at primary schools in Guatemala also had an impact on the increase in coverage in rural areas. A total of 765 teachers in 439 schools have been appointed to work in the afternoon shifts, thereby contributing to the expansion of coverage to 116,000 more students. With the introduction of school rehabilitation after Hurricane Mitch, a total of 164 schools were reconstructed in areas most critically affected. These facilities provided more children with access to primary education in a safe and suitable environment. - 4 - b) Improve efficiency and quality through the support of bilingual education and multigrade teaching methods; and ICR Rating: Satisfactory Data collected by the Informatics Unit of the Ministry suggests that the efforts to improve the efficiency and quality of Guatemala's education system have had some positive impact. Though it is still early and full effects of activities supported cannot be measured until a few more years, as well as difficulties to disaggregate data for purposes of seeing impact in areas targeted by the project, some initial statistics reveal improvements that could be attributed to the project. Most notable is the decline in dropout rate for rural children from 10.5 percent in 1995 to 8.4 percent in 2001. The average, overall drop out was measured at 7.02 percent for primary level children in 2001. In the same year, primary repetition has been measured at 14.6 percent, only about 2.4 percentage points down from 17 percent in 1996. Girls and boys are close, but girls have a slightly better showing with 14 percent repeating a grade versus boys at 15.2 percent. In a Quasi-Experimental study on performance of PRONADE schools conducted in 2001 and financed by KfW, results comparing retention of PRONADE students with those in traditional schools indicate a significantly better retainment in PRONADE schools. A cohort of PRONADE students from both bilingual and monolingual schools were followed from 1999 to 2001. At the national level it was measured that 44.4 percent of students reached second grade (59.1% urban and 38.2% rural). In the case of the PRONADE cohort, 73 percent advanced to the second grade. Interestingly, the bilingual schools in the sample that represented indigenous communities in Alta Verapaz, El Quiche, and Huehuetenango out performed the Spanish speaking schools in the sample representing Zacapa, Chiquimula, Jutiapa, Jalapa, Santa Rosa and Escuintla. An average of 10 percentage points separated the boys and girls in PRONADE schools with more boys advancing to second grade than girls. By third grade, the sample representing PRONADE schools drops to 56 percent advancing from second grade to third. About 61 percent of boys make it in comparison to 52 percent of girls. c) Strengthen the organization and management of MINEDUC and decentralize education administration to encourage greater community participation. There is a geographic focus on the areas most affected by decades of internal conflict (ZONAPAZ). ICR Rating: Satisfactory On several levels, the project successfully assisted the government with strengthening the management of MINEDUC and decentralizing its administration. Additionally, the underpinnings of several activities of the project support the ideal of greater community participation and the project was designed from the start with much input from a wide group of stakeholders. Support was provided to MINEDUC to redefine the roles and functions at each level of the Ministry's structure. This was successfully completed and put into effect in year 2000, however, while the structure still exists, it has not come to full utilization due to the change in administration and differing priorities. Some other areas of success include the creation of 22 departmental directorates for DIGEBI (JEDEBI) and the creation and fortification of the management information units at the level of the 22 departmental directorates. The regional informatics units have been a successful pilot in the effort to decentralize the education administration and encourage greater local participation. Much of the project focused on extending and enhancing provision of services in the rural areas and those - 5 - most affected by the years of internal conflict. Under the PRONADE program the management of community schools is carried out entirely by the COEDUCAs which are parent run community associations. Through the project some 3,500 more COEDUCAS were formed and have received training from non-governmental organizations in the management and daily operation of their local school. Additionally, the project supported the design of a strategy to assist decentralization called Juntas Escolares and the communications campaign formed to educate the public about the program's function and how they could participate. The Juntas Escolares are small community/parent associations formed at the school level with the authority to manage local aspects of their community's school, such as simple maintenance and repairs, etc. The result of the strategy was the formation of more than 3,500 Juntas Escolares. Similarly, the project provided technical support to create and implement Juntas de Oposición which are bodies at the departmental level with the authority to select and appoint teachers. Unfortunately, during the last half of project implementation, some responsibilities of the Juntas were limited because the new administration preferred to recentralize to the Ministry certain duties including the ability to select teachers. 4.2 Outputs by components: Component I: Expansion of Coverage and Equity (US$22.4 million) ICR Rating: Satisfactory Analysis by subcomponent: (a) PRONADE - Programa Nacional de Autogestion para el Desarrollo Educativo. (National Community-Managed Program for Educational Development) PRONADE went far beyond the original planned increase in coverage supported by the project to meet the evident demand in rural, mostly indigenous communities that were not being served by the education system because no school was available within a reasonable distance. This subcomponent was very successful and quickly surpassed the goal of increasing enrollment by 75,000 students. In 1997, approximately 64,000 students were enrolled in PRONADE schools. By the first semester of year 2000, with assistance from the project, an additional 208,753 students were attending PRONADE schools for a total of 272,753 children in total being served. The enrollment gains as a result of the expansion of PRONADE schools during the life of the project is significant, and it represents about one-fifth of the total matriculation of all rural school children, according to the Annual Education Statistics produced by the Ministry. The principal challenge that PRONADE faces is the lack of assigned budget to cover the cost of the program mostly in relation to teachers' salaries. The economic situation of the country and allocation of budget affects the ability to provide timely payment of recurrent costs, and as teachers in the traditional school system are affected with delay in payment of salaries, so too are the PRONADE teachers. While the project supported the teacher salaries with declining financing, eventually the government is responsible for covering these recurrent costs. The government has been making the payments, but some delays have occurred. (b) Programa de la Niña (Education Program for Girls) The goal of this subcomponent was to enhance the incorporation of girls in the education system and retain them so that they could complete at least a primary education. To achieve this, the project supported the sensibilitization programs both for educators and communities, including activities that would expose the importance of educating girls so participants could see firsthand the benefits. A communications campaign was established to publicize the importance of educating girls and to encourage girls receiving scholarships - 6 - to attend school to stay in school and complete their education. The campaign distributed 10,000 buttons, printed school bags, and publicized the Third National Meeting on Girls' Education in the country. The subcomponent also strengthened the coordination of information and communication among the departmental offices of MINEDUC, as well as with the center regarding this theme. (c) Afternoon Shifts Under the project, the Ministry was able to create 800 teacher placements for afternoon shifts, allowing some 116,000 more students of which 57,000 were first graders to attend primary school. The program contributed to the expansion of coverage, especially for students unable to follow a traditional school schedule or in the case where one-shift schools were over capacity and unable to accept more children. The number of schools offering afternoon shifts reached 439, and 765 teachers were appointed to teach during those shifts. Each shift received furniture, including a desk and a chair for the teacher, a bookcase, and a blackboard. A study analyzing the characteristics and situation of the "Afternoon Shifts" has been produced and is being used by the Planning Unit within MINEDUC. While the government originally had the goal to legally establish 2000 new teaching positions specifically for the afternoon shifts, it only succeeded in creating 800 positions. Economic constraints had an impact on creating more teacher positions due to the recurrent cost factor. If judging the success of the afternoon shifts solely on the number of new teacher places and new afternoon shifts created, it would seem that this subcomponent underperformed, however the goal of the program was to increase coverage so more children could go to school. In this respect, the program reached its goal of enrolling 70,000 students by year 2000, and exceeded this number by another 46,000. From this viewpoint, afternoon shifts are considered satisfactory. (d) Construction and Rehabilitation of School Buildings This subcomponent was not originally planned during the appraisal of the project, instead it was created to address the emergency crisis caused by Hurricane Mitch through an amendment to the Legal Agreement. At the time of the amendment the rehabilitation was thought to support the third Component - Strengthening of MINEDUC, however, it seems to more naturally correspond with the first component for increasing school coverage at the primary level, so we are including it here. To execute this subcomponent, the Ministry entered into a legal agreement with the Social Investment Fund (FIS) and the Executive Secretary of the Presidency of the Republic (SEP), so that these public entities could act as Government Agencies for the contracting, supervision and monitoring of the civil works. The program was executed in two phases and completed reconstruction of 164 educational buildings. In the first phase, FIS constructed 58 official school buildings in the following departments: Chimaltenango (2); El Progreso (3); El Quiché (19); Huehuetenango (4); Izabal (23); Jalapa (6); and Zacapa (1). The SEP reconstructed 16 schools. In the second phase, under the agreement with FIS titled "Classrooms for the New Century," five subprojects covering another 90 educational buildings were completed. For example, one subproject targeted displaced communities in Huehuetenango (7), Peten (6), and Alta Verapaz (2), all communities in the ZONAPAZ areas. Reconstruction was completed during the first months of 2001, at a total of US$5.37 million which was less than originally anticipated. The US$1.63 million in funds remaining under the Civil Works procurement category was reallocated back to the other investment categories. This swift reaction by the Bank, proved to be timely and provided an immediate relief to those communities that had no school infrastructure. - 7 - Component II: Improvement of Efficiency and Quality (US$9.5 million) ICR Rating: Satisfactory While this component's resources were reduced with the reallocation of funds to support rebuilding of schools after Hurricane Mitch, it still successfully improved the quality and efficiency of primary education in the rural areas. To achieve the objective of the component, the project supported development and delivery of training programs specifically targeted to bilingual intercultural education, the use of multigrade methodologies, active teaching methodologies, development of pedagogical materials and their dissemination, curriculum development, and the creation and operation of an assessment system to measure student performance. Analysis by subcomponent follows here: (a) Curriculum Development Through the Departments of SIMAC/DICADE and DIGEBI, the project was able to support efforts to design and develop the curriculum and corresponding educational materials. To further support bilingual education, self-learning guides and 62 textbooks for preprimary and grades 1-3 in mathematics and Mayan culture were completed for several Mayan languages. The project financed the layout, illustration, translation, printing and distribution of these materials. To complement the bilingual training programs in curriculum, many didactic materials were provided to diversify the curriculum materials available and used for instruction and learning in the classroom. They included among others: 8,220 grammar books and 6,365 dictionaries in nine Mayan languages; 7,200 manuals for reading and writing Mayan languages; audio visual aides and instructional materials for a self-learning pilot program in the Kaqchikel language; materials to administer diagnostic exams in the four most commonly spoken Mayan languages; and printing and distribution of 55,000 self-learning guides for third graders in PRONADE schools. Many of the trainings described below in the other subcomponents also include areas of curriculum development. (b) Bilingual Education Guatemala's educational statistics have shown that low levels of completion of first grade and poor quality instruction has been a consistent stumbling block for students, and they have demonstrated much higher drop out and repetition rates in comparison to other grades. This is even more common for students whose native language is not Spanish. While not originally contemplated for financing under the project, the Diploma program for rural teachers for special instruction of first grade curriculum was considered essential, and it was agreed that the loan funds would support the government with this program. Under this activity 352 educators, of which 330 were teachers and 22 were technical pedagogical coordinators (CTPs) earned their diplomas. The program targeted participants from rural areas in 10 departments and was offered by the NGO - INAPRODE. Reading and Writing for Bilingual Teachers. The project successfully trained 3,980 bilingual education teachers at the preprimary and primary level in the process of reading and writing (lecto-escritura ­ L1) in eight Mayan languages, including three of the most widely spoken (Kaqchikel, Mam, and Kekchi), and five others (Tz'utujil, Q'anjob'al, Popti, Chuj, and Poqoman.) The training took place in a week long session offered at the national level in various locations. Participants received a dictionary, a grammar book, and other instructional aides. The program was evaluated by the Linguistics Institute of the Rafael Landivar University, and they found that the experience was widely accepted by teachers who thought the training was positive and beneficial to their performance in the classroom. Recommendations for improvement of - 8 - the program included: (a) lengthening the duration of the training to achieve a greater impact; (b) developing a continued effort to sustain such programs in the medium and long term in order to motivate creation of a policy to integrate the strengthening of bilingual teachers into the mainstream of courses offered; and (c) systematizing these types of training events and elevate their status beyond experimental training sessions. Diplomas in Curriculum, Linguistics and Education Management. The goal of this training was to strengthen the human resources of DIGEBI in the mentioned areas resulting in higher technical skills for bilingual teachers and technical administrators. Through the University Rafael Landivar, the project supported a diploma program that trained 88 teachers in applied linguistics; 45 in education administration; and 59 in curriculum development. The courses were seven months in length and participants graduated after completing 544 hours of study. Further to several training opportunities to strengthen bilingual education, the project supported technical assistance to DIGEBI through consultants that provided skill building for: (a) formation of policies and strategies to develop programs in human resources professionalization; (b) adaptation of educational methods and materials, (c) guidance for girls' education programs; (d) guidance in the NEUBI program, (e) interinstitutional projects, (f) training and strengthening of teachers of bilingual schools. (c) Multi-grade Schooling in Rural Guatemala The main objective of this subcomponent was to increase the number of multi-grade schools in the country that were applying the active teaching/learning methodology combined with content that was pertinent to the cultural customs, traditions, and languages of the communities being served. Approximately, 2,600 rural schools covering all 22 departments in the country participated. Out of the total number of schools, 1,910 were official rural schools and 690 were PRONADE schools. In total 5,500 teachers from rural areas, of which 1,685 were from PRONADE schools, participated in the training process that consisted of 20 eight hour seminars for a total of 160 hours of training spanning five months. The training was delivered by the University of San Carlos through 59 national level networks and the assistance of 213 tutors and 30 specialists in the multigrade methodology. Teachers completing the course satisfactorily, benefited by receiving 15 university level credits equivalent to five university courses and received a diploma in Technical Educational Administration with specialization in active teaching methodology applied in rural, multigrade schools. The training was completed by February 2002. The organizers of the training program also coordinated the updating of the 15 training modules, as well as the updating and editing of the self-learning guides used in multi-grade schooling. In August of 2002, 119,000 second grade self-learning guides in the Q'eqchí language were completed and delivered to schools. The design and revision of self-learning guides in the areas of mathematics, social sciences, natural sciences, and communication were supported by the project for grades one to six. As a complement to the multi-grade methodology training and self-learning guides, the proper teaching tools and educational classroom aides were supplied to 2,200 rural schools. The learning corners consisted of a mini-classroom library with about 180 books and a bookcase, maps, globes, flags, rulers, whiteboards, markers, posters, thermometers, scales, and other didactic materials. In addition to the PRONADE teachers that participated in the training for active teaching methodology, the Instituciones de Servicios Educativos (ISEs) trained 10,091 teachers in (a) community development and self-management; (b) management of intercultural, multi-grade classrooms, and (c) mathematics in the classroom. With the coordination of DIGEBI and PRONADE they also delivered a three day training to - 9 - 764 K'iche and 541 Mam teachers in linguistics. (d) Fund for Innovations to Support Educational Quality Originally this activity was designed to support the implementation of fundamentally innovative education projects for rural areas. To assist the process, the PCU designed an orientation manual for departmental or school level personnel to use as a guide for presenting their proposals. The project financed 24 proposals for a total amount of about US$230,000, however, it was determined that the mechanism for communication about the program was not sufficient enough to generate the demand for the innovation grants that had originally been envisioned. To remedy the situation, MINEDUC decided to refocus the subcomponent to target fulfilling the demand for training in multigrade methodologies and didactic materials for multigrade schools. Approximately, 2500 teachers benefited from the training and learning guides were printed for approximately 1,000 multigrade schools. Similarly, part of the funds for this subcomponent were used to design and execute five innovative experiences related to the use of computers in schools. Computer/information centers in five municipalities were created. The centers offered sensitization sessions to educational communities and seminars directed toward teachers and school communities. Courses in the use of computers and educational software (Micromundos and Mi Isla de la Fantasia) were given to train teachers. The educational communities have demonstrated the capacity and management skill to maintain and promote the educational resources available at the centers. Participants of the trainings continue to use the educational tools available. (e) National Assessment System - NAS With support from the project, the Ministry of Education completed three National Assessments of third and sixth graders (mathematics and reading). The results of the evaluation were presented by the University del Valle to the Ministry of Education. The third assessment was completed in August 2000. Since then, MINEDUC agreed to finance the continuation of the system during the year 2001 using local resources, and during August and September of 2001, Universidad del Valle conducted a new assessment fully financed with local resources. It was decided by the government that while they supported the continuation of National Assessments of third and sixth graders in the two mentioned subjects and would provide their own resources to finance the exams, the next National Assessments to be offered would be postponed. Component III: Institutional Strengthening (US$13.6 million) ICR Rating: Satisfactory The activities completed under the project successfully supported the Informatics Unit (UDI) in building the physical, hardware and software infrastructure; training; technical assistance; and management of the statistical information system. Twenty-two Departmental Directorates were supplied the furniture and computer equipment necessary to establish these regional units. Additionally, the project supported the Ministry to purchase and install communications equipment (servers, etc.) to connect eight Directional Departments which made it possible for them to send and receive information with the network established at the central ministry. Licenses to operate the databases (ORACLE), antivirus software, and automated office programs for the regional and central offices were purchased. Besides the supply of hardware and software, many trainings were delivered to various types of users at the regional and central offices. Additionally, specialized technical assistance was provided to support UDI staff in the programming work, design of the databases, training and statistical analysis. The project also - 10 - supported the activities related to the collection, processing and presentation of the informational statistics which included: design and printing of the statistical bulletins as well as the annual statistical yearbook, training in the generation and use of educational indicators, controls for quality of information reported, and general management of the collection and analysis of the statistical data. The activities executed during project implementation satisfactorily met the planned objectives and the technical assistance provided to UDI significantly strengthened the establishment of the systems and informational services of the Ministry. MINEDUC now has a consolidated information system and has demonstrated that they have the capability to provide better quality reporting of information and statistics. It is worth mentioning that by project end, this component completed development and delivery of various products: (i) the modernization and redesign of the process for collecting and analyzing data (statistical cycle); (ii) more efficient reporting of information; (iii) decentralization of the sector through the deconcentration of collection and preprocessing of data at the level of the Departmental Directorates; (iv) establishing Internet connections and the associated benefits (e-mail, intranet, among others); and (v) the automatization of various processes through the development of informational tools for various departments in the Ministry. The project included support to the department offices. Resources were allocated to equip, construct and/or rehabilitate the departmental offices, however, the second administration decided not to move forward with this initiative. It was decided to include this support under the new on-going education project, Universalization of Basic Education. Component IV: Project Administration (US$1.4 million) ICR Rating: Satisfactory The project supported MINEDUC to develop and improve its project management capacity through the creation of the PCU that worked closely together with the functional units of the Ministry which were directly involved n implementation of the project components. The PCU established was highly successful in contributing to the smooth implementation of most of the activities by providing quality coordination. The unit benefited by having a proactive director, a solid financial team, and a highly qualified procurement team. This contributed to the fast-paced implementation, particularly in the first half of the project's life. While towards the end of the project, some changes in PCU personnel were made, most activities had been completed and the transition did not affect implementation of remaining activities. 4.3 Net Present Value/Economic rate of return: Not applicable. 4.4 Financial rate of return: Not applicable. 4.5 Institutional development impact: ICR Rating: Satisfactory As previously indicated under the Institutional Strengthening Component, resources from the Project contributed to the transformation of the organizational structure of the Ministry. The project supported significant administrative reforms, the strengthening of MINEDUC's management capacity and strategy for planning. These changes were institutionalized through the establishment and implementation of new administrative norms and procedures. - 11 - Another significant impact was the consolidation and modernization of the Informatics Unit within MINEDUC. The capability for data collection and analysis as a tool for better management decisions was deconcentrated to the 22 departmental directorates. Each of the directorates were supplied with computer equipment, software, and professional training. The Informatics Unit played a prominent role in piloting the general decentralization policy for the ministry. During project preparation there was an emphasis on wide consultation at the local level as well as with marginalized groups for their input and shaping of components of the project, making the process more participatory. In line with the Peace Accords, the project continued to support wide participation and community involvement in implementation and decision-making as evidenced in the PRONADE program. The project supported the PRONADE program as an effective management mode to address the challenges associated with the highly disbursed, rural population of the country. As a result, there was a strong deconcentration of educational services, and massive participation and direct involvement of the community and parents of school children. Similarly, support for the organization and operation of parent associations (juntas escolares) opened up an alternative considered effective for improving the efforts to operationalize the deconcentration of services and programs. Further institutional gains are expected under the Universalization of Basic Education Project which also supports continued institutional strengthening. 5. Major Factors Affecting Implementation and Outcome 5.1 Factors outside the control of government or implementing agency: During project execution Hurricane Mitch presented an unforeseen need to include school rehabilitation and construction to parts of the country hit the hardest during this natural disaster. While some resources were shifted from other parts of the project to cover the construction needs (the majority from the unallocated category), it did not alter the development objectives of the project, as the activities still contributed to improvement in coverage and equity in poor, rural areas. The Ministry collaborated with the Social Investment Fund (FIS) to achieve implementation of school rehabilitation with excellent results. One factor most favorable to the project, as well as to the education sector in general, was the implementation of the Peace Accords that supported quantitative compromises that increased the budgetary resources destined to public education for introducing qualitative reforms to the education system. While the project was executed under two distinct administrations, the Peace Accords are considered State agreements and not of a particular government. Therefore, even though the government administration changed mid-way through project implementation and there were expected changes, it did not represent substantial shifts in educational priorities. The project was able to maintain its original orientation. This represents the overreaching beneficial affect the Peace Accords had on the State, and as a consequence to the implementation of the project as its activities were designed to support the Accords. 5.2 Factors generally subject to government control: The project benefited from several factors subject to government control that can be considered favorable. First, the project was prepared and began execution under the same administration which had a positive affect on the understanding and support for the project as well as the pace of implementation. Second, the participatory design of the project and support from the MINEDUC authorities resulted in a project reflecting the appropriate priorities of the Government for the education sector. Midway through project implementation there was a change in government and as it usually happens, there is a period of transition. With the new administration came some issues related to modalities for financing and few changes to - 12 - service delivery of some parts of the project that slowed the pace of implementation. These included among others, limiting the responsibilities of the ISEs in relationship to education training delivery and attempting to by pass some of the standard procedures (RISE) established for execution of the PRONADE program. There were also chronic delays in making payments of the PRONADE teachers' salaries. The new authorities introduced some changes they considered pertinent, but in general, the new administration upheld the initial priorities, albeit decentralization was slowed considerably, and were able to successfully complete the execution of the project. 5.3 Factors generally subject to implementing agency control: As it had been planned, the management and supervision of the Project was the responsibility of an Executive Committee, made up of the Vice Minister of Education, who served as Coordinator, and the directors responsible for the execution of each project component including the Directors of PRONADE, UDI, UPE, DIGEBI, SIMAC/DICADE, and the Director of the Project Coordination Unit. Furthermore, when the topic matter required their input, other staff members were invited to participate in the Committee, including staff from MINEDUC and other entities related to the Project, who served as advisers to help make pertinent decisions. The creation and operation of the Committee was very useful, since it not only made it possible to overcome the problems of coordination faced by previous projects, but it also adequately decentralized Project activities and involved the directors of the executing units, under authority of the ministry. The Project took advantage of the past experience gained in recent years with the implementation of the Second Basic Education Project (SPEB) financed by IBRD, particularly with respect to the creation of an institutional capacity to manage the disbursements of the loan, as well as to comply with the procedures and standards of the Bank in the contracting of works and services and the procurement of goods. It was possible to systematize the majority of processes and procedures managed by the PCU and above all to form and train a team of better skilled professionals. The contracting of the United Nations Development Program (UNDP) as a Procurement Agent was a positive factor that facilitated the implementation of the Project. This judgment was reflected in the praise of the businessmen who participated in the contracting process, who, upon being consulted, commented that the acts of bid opening are more serious and that the presence of the UNDP gave confidence, safety and transparency to the process. The MINEDUC signed an agreement with the UNDP (Project/GUA/98/03) so that it supported the Project and managed some of the procurement processes (international competitive biddings and others), which totaled US$14.8 million of the loan. 5.4 Costs and financing: Total project costs at the time of appraisal were estimated to be US$46.8 million, of which the Bank loan would finance US$33.0 million and the Government would contribute US$13.8 million. The disbursement pace was rapid in correspondence with the speed at which the project was implemented. Much of the activities supported by the project were executed well ahead of schedule. As noted earlier, funds were reallocated in response to the need for rehabilitation of schools after Hurricane Mitch. This shift in resources coincided with the objective to increase coverage, thereby having a positive affect on the project. Funds were mostly reassigned from the unallocated category and from areas that were not having as large an impact or needed less resources to accomplish what was expected, namely from goods and the educational innovations pilot. 6. Sustainability 6.1 Rationale for sustainability rating: ICR Rating: Likely - 13 - It is very likely that the achievements under the project will be sustained. The best indication for sustainability is that the Government now has developed and is implementing the Universalization of Basic Education Project that builds upon the foundations established by the key components and activities of the project. This includes continuing to increase coverage and equity to rural, indigenous communities through PRONADE, provision of textbooks and learning guides, bilingual educational materials, promoting multigrade methodologies and active learning in the classroom, continuation of EMIS, scholarships, and the modernization and strengthening of the MINEDUC. Some of the most important components executed by the Project are sustainable as they utilize the government's own resources and were deliberately designed so that the Government would gradually take over responsibility so that by the end of the project it would provide full financial support. This is the case for the first Component - Expansion of Coverage and Equity, which made it possible to increase educational coverage because the number of rehabilitated schools, PRONADE schools and students were increased, and the school day was lengthened --all of which made it possible for the project to continue to function normally after the support from the Bank was concluded. The test for the Government will be to maintain in a timely manner the payment of PRONADE teachers and ensure the budget allocations correspond with the planned pace of expansion. PRONADE has demonstrated to be an effective management mode to address provision of schooling to highly dispersed, rural communities. In the cases of the other components, some of the activities indeed can continue to be developed with the government's own financing, but others, for example the training programs and the printing of educational materials, will need some external support. This situation has been recognized both by IBRD and by the other international cooperating donors. In terms of institutional strengthening, while the former administration had slowed efforts to decentralize, the project still was successful to assist the transformation of the organizational structure of the Ministry and its managing capacity through establishment of administrative norms and procedures that can be reactivated. The current administration (entered January 2004) has expressed interest in reinvigorating the effort to modernize and decentralize the Ministry. 6.2 Transition arrangement to regular operations: It has not been difficult to integrate the activities under the Expansion of Coverage and Equity Component with the regular operations of the MINEDUC, since the financing granted by the Project was complementary to financing provided by the Ministry to develop as a part of its programs and normal activities. The challenge here is for government to maintain a timely availability of the budget. The activities supported by the project under the Improvement of Efficiency and Quality Component, such as training, design and printing of materials, also do not need greater adjustments or agreements in order to be incorporated into the educational system. These activities were carried out within the system and the system is taking advantage of in-service teachers to integrate these activities. When it is necessary to replenish or to improve the printed materials provided, MINEDUC should request a budgetary increase to assert the priority that Guatemalan society is increasingly assigning to education. The performance of the Project Coordinating Unit during the five years of project execution was satisfactory. Despite some problems in the financial management system detected by an internal audit, it is important to emphasize that the PCU has achieved in recent years advances in gaining experience in the management of procurement and financial procedures. The Procurement personnel now better understand the international standards and the standard documents used by international organizations. They have received training in order to carry out procurement processes that require a certain level of complexity. The PCU has also gained experience in the preparation of annual procurement plans. - 14 - Since year 2000, the PCU has taken steps to improve its management systems, and now is implementing and applying the so-called "Managerial System of Projects"­SIGEPRO. The PCU became the pilot project within the government of Guatemala to adopt the system developed by the Ministry of Finance to be utilized by all the projects financed with external resources, in the framework of the Integrated System of Financial Management (SIAF), which was also supported by Bank financing. However, in 2001 new professionals joining the technical team of SIAF who were not familiar with the process carried out in the pilot project, applied modifications to the program, which induced a significant lack of control in its operation and this prevented system consolidation. Toward the end of 2002 there arose an additional complication with the introduction of a new version of the operating system "Oracle" in the SIAF, which makes it necessary to carry out changes in the SIGEPRO in order to update the system. The system is currently only utilized by the PCU to issue checks. 7. Bank and Borrower Performance Bank 7.1 Lending: Satisfactory. The Bank team engaged the Government in a productive, widely participatory dialogue to jointly define the project objectives and the components. Based on the results and recommendations of several diagnostic sector studies that had been recently conducted, the Bank was in a good position to 1 advise the government in the formation of the components and give guidance on activities that could significantly impact some of the most prominent problems facing the education sector. The three components addressed the critical factors and priorities of the national education system, including low coverage, deficiencies in quality, and the poor institutional capacity of the Ministry. Working together with the MINEDUC personnel, the project design emerged in a way that would help address the obstacles recognized by past project implementation experiences, and paid particular attention to execution capacity. An important factor was the creation of the Executive Committee, as well as the consideration taken in designing the role of the PCU as a coordinating unit and the Ministerial departments as implementers, ultimately building capacity and ensuring ownership beyond preparation. /1 See: "Guatemala An Assessment of Poverty" (1995); "Guatemala: Equity Efficiency in Education" (1995); "Guatemala Building Peace with Rapid and Equitable Growth, Country Economic Memorandum" (1996); y "Guatemala Investing for Peace: Public Investment Review" (1997). 7.2 Supervision: Supervision of the World Bank team can be rated highly satisfactory. No changes in the Task Team Leader from preparation through to project closing provided the government with consistent and high quality supervision over the life of the project. The Bank team was careful to provide quality supervision, adequate and timely missions averaging two or more a year, and regular communication with Ministry officials, the PCU, and other donors also involved in supporting the Ministry. The team was quick to respond to issues resulting from the change in administration and was pro-active in identifying ways to keep the project on track towards achievement of the relevant objectives in the new context presented by the change in administrations. The education team immediately responded to the Minister's proposal to change fundamental aspects of the functioning of the PRONADE program and came to a compromise after providing critical advice and technical assistance, thus preserving the characteristics most important to the success of the program. The Bank team also worked with the counterparts to support the education system demonstrated by its timely and flexible response to reallocate funds for rehabilitation of schools damaged by Hurricane Mitch. - 15 - 7.3 Overall Bank performance: Based on the Bank's thorough and supportive approach to lending and supervision of the Project noted above, performance is rated satisfactory. The Bank encouraged technical exchanges with the Borrower, input and feedback from all levels of the Ministry and local consultation with a variety of interest groups and communities. The professional relationships and open dialogue between the client and Bank team are factors that contributed to the preparation and negotiation of the new education project currently under implementation. The follow-on project built on successes and lessons learned from the Basic Education Reform Project. Borrower 7.4 Preparation: On the part of the Borrower, project preparation can be rated satisfactory. From the beginning, the Ministry was interested in maintaining a participatory approach to designing the project and supporting the spirit of the Peace Accords by broadly employing public consultation. In 1996 and 1997, 18 public forums were conducted to ensure an open discussion and the participation of all who were interested. Additionally, the Ministry encouraged participation of MINEDUC staff at the central and departmental levels to be involved in preparation of the project. It also included consultation with universities and academic institutions 7.5 Government implementation performance: Overall, government implementation was satisfactory, however, some performance differences were experienced between the first half and second half of project execution. At the start of project implementation there was a high commitment from all levels of the Ministry starting with the Minister and continuing to the local levels and the PCU. The Ministry's commitment to the project and achieving the development objectives is evidenced by the project's rapid execution during the first three years of implementation when many of the project's goals were achieved well ahead of schedule and about 75 percent of activities had been completed. The change in administration affected the pace of implementation that had been established in early years of the project as can be common during a transition between governments. While the new administration generally supported the development objectives of the project and continued to work towards their achievement, some procedural changes affected the pace and the established modes for reaching their outcomes. For example, the new administration's desire to introduce significant procedural changes to the PRONADE program that would alter the modalities of delivery of services and financing of those services, led to the resignation of the director, an initial slow down in transfer of resources to COEDUCAs, and a curtailing of the function of the ISEs in the training and establishing of COEDUCAs. Eventually, the administration came to an agreeable working arrangement. Despite the recentralizing of some responsibilities that had already been deconcentrated to the local level and the altering of some aspects of PRONADE, as a whole, the government implementation of the project was satisfactory. 7.6 Implementing Agency: Satisfactory. The Ministry of Education was careful to select well qualified and capable staff for key positions in the PCU, including financial, procurement, technical and managerial staff. Project execution benefited by the application of a decentralized administration of project activities. Unlike previous Bank supported projects, it was maintained that the PCU would play the role of a coordinating unit rather than take on execution responsibilities. Implementation fell to the jurisdiction of the administrative units of MINEDUC and each was responsible for the relevant activities that corresponded to the components. This improved implementation, ownership and relevancy of activities, as well as better integration with the goals - 16 - of the Ministry. Additionally, the PCU's coordinating function helped the ministry to avoid duplication of activities among projects executed with external funds, as it has the responsibility to coordinate execution of both the Bank project and the IDB project. This contributed to the smooth implementation during most of the years of project execution. 7.7 Overall Borrower performance: As noted above the overall Borrower performance can be considered satisfactory. 8. Lessons Learned The following lessons learned can be concluded from the project experience: l Project experience confirmed that local, rural communities can effectively and transparently manage resources transferred by the central level and can hire teachers following established procedures regardless of the participants' educational level. Many of the COEDUCAS consisted of parents that were illiterate, but still had the capacity to effectively manage their local school. l Scaling-up implementation of programs is important, particularly as demonstrated by the multi-grade methodologies model. l Satisfactory results reached by the Project are largely due to a sustained continuity in project coordination. During almost the entire lifetime of the project, the technical team responsible at the PCU that ensured the pursuit, coordination and execution of the activities, was not only maintained but it also received fortification through training. The structure of the PCU included the traditional pedagogical and financial sections, but also included a procurement section, integrated by personnel with the appropriate experience and qualification. l Project execution benefited by the application of a decentralized administration of project activities, with the PCU's main function as a coordinating unit rather than the main implementer of project activities. l So that a decentralized model for execution of project activities can operate effectively, it is necessary to (i) provide timely and relevant training to personnel in the operational units of the Ministry responsible for the administrative and financial areas; and (ii) permanently assign these technical teams so that they can take charge of the implementation of project activities, rather than have staff assigned temporarily to activities but with the risk of constantly losing them to other assignments. l It is beneficial for project execution if the Operations Manual of the Project is adhered to closely. This represented a clear advantage for fostering successful relationships between the PCU and the executing units because all parties were aware of their roles and duties as established in the manual. l The MINEDUC must intensify efforts to take advantage of the favorable atmosphere that exists in the country to increase budgetary allocations destined to the education sector. Since the Peace Accords were signed, the sector has been considered high priority and has generated more attention and raised consciousness in political and social sectors for the need to increase investment at least to levels considered standard in Latin American countries. A generation of greater internal resources also translates to greater opportunities to mobilize external resources. l The MINEDUC has demonstrated that if external conditions are favorable, for example, having - 17 - counterpart resources from the Ministry of Public Finances, it has the capacity to successfully execute projects. l The opening of a special account in the Bank of Guatemala and hiring UNDP as procurement agent, helped to ease the administrative procedures for disbursing resources from the loan and contributed to greater transparency in the procurement and hiring processes. l In order to optimize the use of the resources and to reach the goals and objectives of the MINEDUC, the activities and programs of the Ministry should be systematic and timely. It should capitalize on efforts where national capacity is growing and where government resources have been assigned to compliment loan resources, such as PRONERE and other priority activities. - 18 - 9. Partner Comments (a) Borrower/implementing agency: RESUMEN EJECUTIVO Principales Indicadores de Desempeño 1. Los principales indicadores de desempeño del Proyecto son favorables. Se estima que la sostenibilidad de las actividades es probable (parcialmente), su impacto en el desarrollo institucional del sector educación es substancial, y tanto el desempeño del BIRF como del MINEDUC fueron satisfactorios. Evaluación de los Objetivos de Desarrollo, el Diseño y de la Calidad al Inicio Objetivos originales 2. Los objetivos del Proyecto fueron: i) aumentar la cobertura y equidad de la educación preprimaria y primaria; ii) mejorar la eficiencia y la calidad de la educación primaria rural; y iii) fortalecer la organización y administración del MINEDUC. Estos objetivos forman parte del Programa de Reforma Educativa del país. Objetivos revisados 3. Los objetivos del Proyecto no fueron modificados, aunque sí hubo necesidad de enmendar el Convenio de Préstamo para introducir la actividad de construcción y rehabilitación de escuelas que no había sido prevista. Calidad al inicio del proyecto Calificación ICR: Satisfactoria 4. El Proyecto fue diseñado de manera participativa con la colaboración directa de las autoridades y de los funcionarios del MINEDUC, lo que aseguró que los objetivos y actividades contempladas fueran bien definidas y que se minimizaran los obstáculos para su implementación. Los riesgos críticos que fueron identificados en el PAD y que se habían presentado en los proyectos anteriores fueron superados satisfactoriamente. Evaluación de los Objetivos y Productos Alcanzados 5. El Proyecto formó parte de las iniciativas aprobadas para respaldar el cumplimiento de los Acuerdos de Paz suscritos entre el Gobierno de la República de Guatemala y la Unidad Revolucionaria Nacional Guatemalteca ­URNG-, que culminaron con la suscripción de los últimos acuerdos el 29 de diciembre de 1996. La cobertura del Proyecto comprendió principalmente a la población rural de los nueve departamentos que conforman la denominada ZONAPAZ, que fue el área más afectada por el conflicto armado interno y que coincide también con el área geográfica que concentra los más altos porcentajes de la población en situación de pobreza y la que registra los indicadores educativos más desfavorables del país. - 19 - 6. El Proyecto se benefició de un contexto económico más favorable que el que experimentaron los proyectos educativos anteriores. Ello explica la mejora sustancial en la entrega de los aportes en concepto de contrapartida que correspondió al Gobierno de la República, que había sido uno de los factores críticos en la ejecución de proyectos financiados con recursos externos. No obstante, durante la ejecución del Proyecto se presentaron en forma recurrente algunos problemas relacionados con la asignación insuficiente de recursos en las partidas presupuestarias que correspondían a algunos rubros. Resultados/Logro de los objetivos 7. El Proyecto alcanzó la mayoría de sus objetivos y resultados, principalmente en lo que se refiere a la expansión de la cobertura del sistema escolar y al mejoramiento de la calidad de la educación. En el área de modernización institucional y de desconcentración y descentralización, se debe destaca el nuevo papel que han asumido las direcciones departamentales dentro de la administración del sector educación. A diferencia de los proyectos anteriores ejecutados en el sector educación en Guatemala, el Proyecto cumplió estrictamente con el período de ejecución programado de cinco años sin requerir ampliación alguna. Resultados por componente Componente 1. Cobertura y Equidad a) Construcción y rehabilitación de edificios escolares Calificación ICR: Satisfactoria 8. La construcción y rehabilitación de escuelas fue una actividad que no había sido prevista en el diseño original. Como consecuencia de la destrucción que ocasionó en 1998 el huracán "Mitch" el Gobierno de Guatemala solicitó y obtuvo del BIRF la enmienda del Convenio de Préstamo. Para ejecutar esta actividad el MINEDUC suscribió convenios con el Fondo de Inversión Social (FIS) y con la Secretaría Ejecutiva de la Presidencia de la República (SEP). El programa cubrió en total la construcción, reconstrucción y/o rehabilitación de 164 planteles educativos. Todo el proceso finalizó satisfactoriamente a principios de 2001. b) Expansión del Programa de Autogestión Comunitaria por medio de PRONADE Calificación ICR: Satisfactoria 9. El PRONADE ha tenido un gran éxito para ampliar la cobertura educativa en el sector rural. Este subcomponente mostró un desempeño satisfactorio. Las principales actividades contempladas finalizaron en el año 1999 con un adelanto de más de dos años respecto a la fecha prevista. La meta de aumentar la matrícula en 75,000 estudiantes fue superada ampliamente, El Programa pasó de atender a 64,000 niños matriculados en 1997 a 272,753 en el primer semestre de 2000, es decir un incremento de 208,753 alumnos. c) Ampliación de las jornadas vespertinas Calificación ICR: No satisfactoria - 20 - 10. Este subcomponente no logró cumplir satisfactoriamente las metas previstas y continuó registrando algunos de los problemas que ya se habían presentado durante la ejecución del SPEB financiado también por el BIRF. Entre 1998 y 1999 se crearon 800 plazas de docentes para laborar en jornadas vespertinas de las 2,000 plazas que habían sido contempladas para 2002, de las cuales se ha nombrado a 765 maestros en 439 escuelas. A finales de diciembre de 2002, se les había distribuido 763 juegos del mobiliario consistente en silla y mesa de trabajo para el maestro, librera, rotafolio y pizarrón. Debido a estos problemas, se reevaluó la estrategia de apoyo a los maestros de las jornadas vespertinas. El principal obstáculo para cumplir con las metas radicó en que no se crearon plazas para escuelas existentes o para abrir nuevas aulas en jornadas vespertinas. En consecuencia se decidió que parte de los recursos que tenía asignados se destinaran a jornadas vespertinas de algunas escuelas multigrado que participaron en el programa de inmersión en metodología activa, así como algunas aulas que formaron parte de las actividades del SPEB y cuyos maestros fueron nombrados después de que este proyecto finalizó. d) Expansión del Programa de la Niña Calificación ICR: Satisfactoria 11. El objetivo del Programa de la Niña fue contribuir a la incorporación de las niñas al sistema educativo y retenerlas en las escuelas hasta que logren completar la escuela primaria. Para ello se contempló la sensibilización de las comunidades educativas y además se incluyó actividades de divulgación sobre la importancia de educar a las niñas. Las principales actividades realizadas fueron: el desarrollo de una campaña de divulgación sobre la importancia de educar a las niñas y motivar a las niñas becadas por el Ministerio para que completen la educación primaria; la distribución de 10,000 bolsones o mochilas impresos; la organización y desarrollo del III Encuentro Nacional de Educación de la Niña; y la ejecución de un proyecto cuyo objetivo fue fortalecer la comunicación y la coordinación con los equipos departamentales. Componente 2. Eficiencia y Calidad 12. Por medio de este componente se logró mejorar la eficiencia y calidad de la educación primaria en el área rural, buscando contribuir a la reducción de los índices de deserción escolar y la repitencia. Para alcanzar ese objetivo se desarrollaron programas de capacitación, especialmente en el área bilingüe intercultural y en el uso de metodologías de enseñanza multigrado, se financió asesoría técnica, el desarrollo curricular, la elaboración de materiales pedagógicos, y la creación y operación de un sistema de evaluación del rendimiento escolar. a) Capacitación a docentes y administradores Calificación ICR: Satisfactoria · Diplomado para docentes rurales de primer grado 13. En esta actividad participaron 352 personas de las cuales 330 fueron maestros y 22 coordinadores técnicos pedagógicos (CTPs). Esta actividad cubrió 10 departamentos del país y se ejecutó con la participación de una ONG denominada INAPRODE. · Programa de Inmersión en la Metodología Activa a Docentes en Servicio de Escuelas - 21 - Rurales Multigrado 14. Esta actividad tuvo como objetivo ampliar el número de escuelas en todo el país que aplican la metodología activa y requirió la movilización de una gran capacidad institucional del MINEDUC. Participaron 2,599 escuelas rurales ubicadas en los 22 departamentos del país, de las cuales 1,910 fueron escuelas rurales oficiales y 689 pertenecían al PRONADE ubicadas en el departamento de Alta Verapaz. El número total de maestros participantes fue de alrededor de 5,500. La inmersión tuvo una duración total de 160 horas de formación docente en un periodo de 5 meses. 15. El programa académico implicó la transferencia de la tecnología de la metodología activa, contenida en 15 módulos de formación docente que fueron adaptados y actualizados por los organizadores. Incluyó también como complemento la entrega de material didáctico de apoyo como mini bibliotecas (180 títulos), la adquisición de los elementos de los rincones de aprendizaje y las libreras. Los maestros que completaron satisfactoriamente el programa de capacitación y cumplan con los requisitos exigidos por la USAC podrán beneficiarse de 15 créditos universitarios equivalentes a 5 cursos universitarios y de un diploma de Técnico en Administración Educativa con especialidad en metodología activa aplicada a escuelas rurales multigrado. Las escuelas a las que pertenecen los maestros obtuvieron una mini biblioteca con sus respectivas libreras (2,000), ludoteca, elementos para rincones de aprendizaje, pabellón nacional y casetes del himno nacional, y guías de autoaprendizaje. · Formación en lecto-escritura en idiomas mayas 16. DIGEBI desarrolló un programa de formación de maestros bilingües que implicó un proceso previo de elaboración de materiales pedagógicos y la formación de docentes. Los idiomas que se incluyeron fueron los tres considerados mayoritarios: kaqchikel, mam y q'eqchí y cinco minoritarios: tz'utujil, q'anjob'al, popti', chuj y poqoman. En total se capacitó a 3,980 maestros. Consistió en 5 días de trabajo de 5 horas cada uno, para completar un total de 25 horas. Los maestros que participaron son docentes en servicio en los niveles de preprimaria y primaria. Previamente se capacitó a 120 formadores en la metodología de la lecto-escritura, así como la potenciación del conocimiento del idioma, gramática elemental y nociones generales de la estandarización. Cada maestro recibió como material de aprendizaje una gramática en su idioma y un diccionario. · Diplomado en currículo, lingüística y gerencia educativa 17. Con el propósito de formar recursos humanos calificados de la DIGEBI en las áreas de lingüística, currículum y gerencia educativa, se realizó un programa de Diplomado que tuvo como objetivo elevar el nivel técnico de los equipos de docentes bilingües de base, de los equipos técnico-administrativos y de los docentes bilingües. Finalizaron el programa 168 técnicos. El Diplomado tuvo una duración total de 7 meses acumulando un total de 544 horas efectivas. · Apoyo al Programa de Profesionalización Docente 18. El Proyecto apoyó diversas actividades del Programa de Profesionalización Docente, que consistió en proporcionar material educativo (240,000 textos, 85,000 módulos, 60,000 juegos de la unidad 1 del módulo No. 3 y 4, reproducción de materiales educativos y otros), servicios de alimentación a los asistentes a los talleres, rotafolios y otros. Asimismo, se apoyó a DIGEBI para que en conjunto con DICADE, realizara tres talleres de inducción en el que participaron 1,300 personas de los 22 departamentos del país, que incluyeron a facilitadores, directores departamentales, coordinadores de sede de las universidades, - 22 - coordinadores del programa del MINEDUC y monitores pedagógicos. · Capacitación a docentes de las escuelas de PRONADE 19. Además de la capacitación a los docentes de PRONADE en metodologías activas, por medio de los servicios de las Instituciones de Servicios Educativos ­ISEs-, se capacitó a 10,091 maestros en jornadas de una duración de 3 días en los temas de a) desarrollo comunitario y autogestión, b) manejo del aula multigrado intercultural, y c) matemática en el aula. Adicionalmente, en coordinación con DIGEBI el PRONADE también capacitó en jornadas de 3 días a 764 docentes k'iche's y 541 mames en el área lingüística. · Otras actividades de capacitación 20. Adicionalmente, el Proyecto apoyó un conjunto de actividades orientadas a la capacitación entre las que pueden mencionarse dos encuentros motivacionales formativos y de sensibilización a docentes de preprimaria, talleres de capacitación para madres de familia, talleres para facilitadores bilingües, maestros bilingües y otros. · Asistencia técnica a DIGEBI 21. Durante la ejecución del Proyecto se prestó asistencia técnica a DIGEBI por medio de la contratación de consultores en áreas tales como políticas y estrategias para el desarrollo de programas de profesionalización del recurso humano en servicio, adaptación de métodos y materiales educativos, asesoría de la niña, asesoría en el programa NEUBI, proyectos interinstitucionales y capacitación y formación de docentes de las escuelas bilingües interculturales. b) Desarrollo Curricular y Materiales Pedagógicos Calificación ICR: Satisfactoria 22. Los recursos asignados a este subcomponente se redujeron sustancialmente como consecuencia de la inclusión de actividades no previstas. Sin embargo, se logró apoyar actividades importantes para mejorar la calidad educativa. Desarrollo curricular 23. El Proyecto contribuyó con SIMAC/DICADE, DIGEBI y las dependencias especializadas del Ministerio en el diseño de un nuevo currículum. Para el efecto, se contrató un grupo de especialistas quienes trabajaron juntamente con el equipo de profesionales a cargo del diseño de la profesionalización docente del recurso humano del MINEDUC. Materiales pedagógicos 24. Con recursos del Proyecto se financió la diagramación, ilustración, traducción, impresión y distribución de materiales pedagógicos, entre ellos guías de auto-aprendizaje y textos en los diferentes idiomas para las áreas de matemática y cultura maya. Se adquirieron y entregaron libros de gramática y diccionarios en 9 idiomas mayas, textos de gramática y diccionarios en varios idiomas. Se distribuyeron - 23 - manuales de lectoescritura, grabadoras DVD, reproductoras portátiles, televisores, equipos de sonido y otros equipos y materiales. Programa Nacional de Educación Cívica y Valores 25. El Proyecto apoyó este Programa mediante una serie de actividades que incluyeron la distribución de folletos de divulgación, actividades de sensibilización, reuniones regionales de fortalecimiento, jornadas de inducción con orientadores, talleres, impresión y distribución de manuales, entrega de banderas y otros. c) Sistema Nacional de Evaluación del Rendimiento Escolar Calificación ICR: Satisfactoria 26. El PRONERE inició sus actividades en 1997. Su principal objetivo fue monitorear los logros académicos de los alumnos de tercero y sexto grados de primaria. Con recursos del Proyecto se realizaron tres mediciones: la primera en 1998, la segunda en 1999 y la tercera y última finalizó en 2000. Las mediciones se aplicaron en matemática y lectura y en factores asociados al rendimiento académico. La confiabilidad de las pruebas fue alta y los procedimientos utilizados rigurosos. La ejecución del programa fue contratada con la Universidad del Valle de Guatemala. A finales de 2001 el MINEDUC, con recursos propios, llevó a cabo una cuarta medición, siempre mediante la contratación de la Universidad del Valle. Esta evaluación tuvo en general las mismas características de las tres anteriores. Debido a restricciones presupuestarias, en 2002 el MINEDUC interrumpió las evaluaciones. d) Promoción de Innovaciones Educacionales Calificación ICR: Insatisfactoria 27. El objetivo de este subcomponente era fomentar el desarrollo de programas o proyectos innovadores y novedosos e intervenciones a nivel central, departamental y local buscando mejorar la cobertura escolar y la calidad de la educación básica. Desafortunadamente, su desempeño no fue satisfactorio. En general se considera que las propuestas de innovaciones que fueron aprobadas tendieron a responder a solicitudes de recursos que en principio deberían de haber sido cubiertas por el presupuesto regular del MINEDUC. Componente 3. Fortalecimiento Institucional a) Mejora de la Organización y Administración del MINEDUC i) Reorganización del Nivel Central, Departamental y Local del MINEDUC Calificación ICR: Satisfactoria 28. El Proyecto continuó apoyando el programa de modernización del MINEDUC. La reforma se realizó con base en un estudio de la administración general y de las dependencias del MINEDUC y conllevó un conjunto de cambios que comprendieron, entre otras modificaciones, una propuesta de estructura organizacional y operativa reducida y simplificada de la planta central del Ministerio, el diseño e implementación de las 22 direcciones departamentales, y una reclasificación general de los puestos y - 24 - salarios que se puso en práctica a partir del año 2000. ii) Construcción / Rehabilitación y Equipamiento de las Sedes de las Direcciones Departamentales Calificación ICR: Insatisfactoria 29. Esta actividad formaba parte de los esfuerzos por fortalecer la desconcentración y descentralización de los servicios educativos hacia las 22 cabeceras departamentales. Se pretendía que en cada departamento del país se contara con una sede física que albergara las oficinas del Ministerio. No obstante, el Ministerio al evaluar la conveniencia de continuar con esta actividad decidió posponer la construcción de las sedes departamentales, tomando en cuenta que a su criterio existían actividades calificadas con mayor prioridad a las cuales se debían de asignar los recursos. iii) Entrenamiento en administración del sector público para fortalecer la entrega de servicios en los niveles descentralizados Calificación ICR: Satisfactoria 30. Las actividades que corresponden a este apartado consistieron en un seminario taller de descentralización y desconcentración, la organización de una reunión sobre los diversos componentes de la descentralización educativa, una reunión a las OSCs y UPAFs para que conocieran mejor la base legal de las Juntas Escolares y los mecanismos e instrumentos de control técnico, administrativo y financiero, y colaboración con el Programa de Modernización y Descentralización (PROMODE) para fortalecer el proceso de desconcentración. iv) Diseño, desarrollo e implementación de un sistema de evaluación del desempeño de los funcionarios del MINEDUC Calificación ICR: Insatisfactoria Este sistema no fue implementado. v) Desarrollo y mejoramiento del Sistema de Información del MINEDUC Calificación ICR: Satisfactoria 31. El Proyecto apoyó la gestión de la Unidad de Informática en la creación de una infraestructura computacional básica, en el desarrollo de diversos programas de capacitación, en la prestación de asistencia técnica y en la gestión de las estadísticas educativas. Respecto a la infraestructura se adquirió el mobiliario y los equipos de cómputo para apoyar la sede central y las 22 direcciones departamentales. Se compraron las licencias de bases de datos y diversos programas. Se llevó a cabo un conjunto de programas de capacitación dirigidos a los diferentes niveles del MINEDUC. Cada año se realizaron talleres dedicados a capacitar a los responsables de la recolección procesamiento y uso de la información estadística. Asimismo, se realizó una capacitación continua en el desarrollo de software, lenguajes de programación y sistemas operativos. También se implementó un nodo de Internet, se desarrolló la intranet, y se diseñó, desarrolló y publicó el portal web y el servicio de correo electrónico, correo de voz y videoconferencias del - 25 - MINEDUC. Adicionalmente, se desarrolló un sistema informático que permite el cálculo de los indicadores educativos en forma automatizada. Gracias al Proyecto se ha logrado consolidar a la UDI y establecer un sistema de información moderno y eficiente en el MINEDUC. b) Descentralización del MINEDUC i) Contratar y Asistir a las Instituciones Especializadas en Servicios Educativos (ISES) Calificación ICR: Satisfactoria 32. La expansión de la cobertura de PRONADE se realizó mediante una estrategia que privilegió la descentralización de sus intervenciones, en particular la contratación de instituciones especializadas en servicios educativos ­ISES-. De acuerdo a las evaluaciones que se han practicado, PRONADE ha logrado desarrollar modalidades que han permitido una mejor relación de costo-efectividad en los servicios de seguimiento y capacitación de los COEDUCAs. Las principales políticas que se han impulsado incluyeron: a) focalizar los servicios subcontratados para la capacitación y seguimiento a los COEDUCAS; b) reasumir el pago de los gastos logísticos y de movilización para la capacitación de los COEDUCAS; y c) negociar con las ISEs un mejor servicio con costos sostenibles para generar un mejor servicio. ii) Establecer COEDUCAS y proveer asistencia técnica y materiales Calificación ICR: Satisfactoria 33. Con el apoyo del Proyecto, el PRONADE por intermedio de las Instituciones de Servicios Educativos capacitó a 3,428 integrantes de las juntas directivas de los COEDUCAS en los temas pedagógicos, administrativos y socioculturales. Se contrató a 19 ISEs con el objeto de dar acompañamiento a 2,987 comunidades. Esta actividad consistió en la capacitación durante 6 días de tres miembros de la junta directiva de los COEDUCA y visitas para prestar asesoría y monitoreo. iii) Establecer y capacitar Juntas Escolares Calificación ICR: Satisfactoria 34. El Programa de Juntas Escolares (JE) respaldado por el Proyecto representó una clara alternativa de participación comunitaria. Mediante la contratación de ONGs, se constituyeron e implementaron un total de 8,379 JE, bajo la modalidad de asociaciones civiles con la participación de maestros y padres de familia de cada centro educativo. Las JE también administran a partir del año 2000 los programas de becas. Para mejorar su funcionamiento se implementó un proceso de capacitación que consistió en talleres de capacitación y seguimiento en el que participaron 6,492 personas. Tasa Interna de Retorno 35. Con base en estudios previos, en el PAD se calculó que la tasa interna de retorno para este Proyecto podría alcanzar un 28%. Los beneficios esperados se derivaban de: i) la inscripción adicional de alumnos y terminación de la educación primaria; ii) la reducción de las tasas de repetición, la reducción de las tasas de deserción y el incremento en las calificaciones entre los estudiantes; y iii) una mejora en los costos administrativos. Sin embargo, la elaboración de una línea base que estaba prevista no se llegó a concretar. Esta circunstancia impide una evaluación rigurosa del impacto que pudo tener el Proyecto en los - 26 - indicadores educativos dentro del área de cobertura y estimar los beneficios económicos potenciales que podrían haberse generado. Además, se considera que algunas de las actividades ejecutadas tendrán un impacto efectivo en un período futuro posterior al de ejecución. Otro factor que complica la evaluación radica en la concurrencia de varias intervenciones simultaneas en la misma zona geográfica. A pesar de estas dificultades de orden cuantitativo, se considera que está fuera de toda duda la rentabilidad de la inversión realizada. Impacto en el Desarrollo Institucional 36. Se puede afirmar que el Proyecto tuvo un impacto importante para mejorar la capacidad institucional del MINEDUC. Con recursos del Proyecto se contribuyó a la transformación de la estructura organizativa del Ministerio que implicó significativas reformas administrativas y el fortalecimiento de su capacidad gerencial y de planeamiento estratégico. Estos cambios se materializaron también en el ordenamiento de las funciones normativas y reglamentarias. Asimismo, se logró impulsar y consolidar al PRONADE, como una modalidad de gestión efectiva para resolver los desafíos que plantea el alto grado de dispersión y ruralidad de la población. Además, mediante el apoyo a la organización y operación de las juntas escolares, se abrió una alternativa que se considera eficaz para mejorar el funcionamiento desconcentrado de algunos servicios y programas. Principales Factores que Afectaron la Implementación y los Productos del Proyecto · Factores fuera del control del gobierno o de unidad ejecutora 37. Durante la ejecución del Proyecto no se presentaron factores que estuvieran fuera del control del gobierno o de la unidad coordinadora que afectaran negativamente sus actividades. Por el contrario y a diferencia de proyectos anteriores, el ambiente externo en general fue positivo. · Factores generalmente sujetos al control del gobierno 38. El Proyecto se benefició de varios factores que pueden ser considerados favorables. Fue formulado en el período en que se iniciaba el gobierno y su ejecución empezó a mediados de esa misma administración. Además, fue diseñado con la participación activa y directa de las autoridades del MINEDUC y durante los cuatro años del período de gobierno no se registraron cambios en las autoridades superiores del Ministerio. Con el cambio de gobierno y de las autoridades superiores del Ministerio en el año 2000, se produjeron las reticencias que son naturales en cualquier cambio de administración, pero en general se mantuvieron las prioridades iniciales y se logró completar exitosamente la ejecución del Proyecto. Por otro lado, la ejecución simultanea de dos proyectos con componentes muy relacionados financiados por dos organismos financieros internacionales distintos (BIRF y BID), pudo ser armonizada gracias a que fueron ejecutados por la misma unidad coordinadora. · Factores generalmente sujetos al control de la unidad ejecutora (UCP) 39. La dirección y supervisión del Proyecto estuvo a cargo de un Comité Ejecutivo. La creación y funcionamiento de este Comité resultó muy conveniente, pues no sólo permitió superar los problemas de coordinación que habían enfrentado los proyectos anteriores, sino también desconcentrar de manera adecuada las actividades del Proyecto e involucrar a los directivos de las unidades ejecutoras, bajo la dirección de una autoridad superior del ministerio. Se logró sistematizar la mayoría de los procesos y procedimientos a cargo de la UCP y sobre todo conformar y entrenar un equipo de profesionales mejor - 27 - calificados. La contratación como agente de compras del Programa de las Naciones Unidas para el Desarrollo ­PNUD- fue un factor positivo que facilitó la ejecución del Proyecto y mejoró su transparencia. Costos y financiamiento 40. El costo total del Proyecto fue estimado inicialmente en US$ 46.8 millones, de los cuales el Banco Mundial se comprometió a aportar US$ 33.0 millones y el gobierno los restantes US$ 13.8 millones. Los costos totales del Proyecto se mantuvieron inalterables. Al 31 de diciembre de 2002 el préstamo se había desembolsado totalmente y solamente estaba pendiente de comprometer un saldo remanente de US$ 0.9 millones. De la contrapartida que correspondía aportar al gobierno sólo faltaba desembolsar por el Ministerio de Finanzas Públicas US$ 0.2 millones. Probabilidad de la sostenibilidad y seguimiento Calificación ICR: Probable (parcialmente) 41. Algunos de los componentes más importantes que ejecutó el Proyecto son sostenibles utilizando recursos propios del gobierno. Ese es el caso del componente No.1 que permitió aumentar la cobertura educativa. En los casos de los otros dos componentes la situación no es tan favorable. Algunas de las actividades sí podrán continuar desarrollándose con financiamiento propio, pero otras como por ejemplo los programas de capacitación y la impresión de materiales educativos necesitarán del apoyo externo durante algún tiempo. Acuerdos de transición a las operaciones regulares 42. Las actividades cuyos objetivos eran ampliar la cobertura no han tenido ninguna dificultad para integrarse en las operaciones del MINDEDUC, pues el financiamiento otorgado por el Proyecto era un complemento de programas que se desarrollan como parte de sus programas y actividades regulares. Las mejoras apoyadas por el proyecto en la calidad educativa y para el fortalecimiento institucional tampoco necesitan de mayores ajustes o acuerdos. Desempeño del BIRF y del Gobierno de la República Desempeño del Banco · Preparación 43. La asesoría del Banco contribuyó a identificar adecuadamente los componentes y actividades que deberían conformar el Proyecto. Los tres componentes apuntaron a enfrentar los factores críticos y atender las prioridades del sistema educativo nacional, como son la baja cobertura, las deficiencias de calidad de los servicios y la debilidad institucional del MINEDUC. El Banco juntamente con el personal del MINEDUC diseñó el Proyecto de manera tal que pudieran superarse los obstáculos que se habían registrado en proyectos anteriores. Por aparte, el Proyecto mantuvo un enfoque participativo desde el inicio de la preparación. - 28 - · Supervisión 44. Las tareas de supervisión del Banco pueden ser calificadas de satisfactorias. Desde la misma formulación del Proyecto se previó la necesidad de una estrecha supervisión y apoyo. El Banco prestó asistencia técnica y monitoreo en la implementación de las diferentes actividades del Proyecto, que significó importantes aportes para la ejecución exitosa de cada uno de los componentes. · Desempeño general del Banco 45. El desempeño del Banco desde la fase de identificación del Proyecto hasta su terminación fue satisfactorio. Desempeño del Prestatario · Desempeño de la implementación del gobierno 46. La implementación del Proyecto por parte del gobierno fue satisfactoria. No obstante que el Proyecto fue ejecutado por dos administraciones diferentes, en ambos casos se mantuvo su prioridad y se le proporcionó el respaldo que necesitaba. · Unidad Coordinadora del Proyecto 47. El desempeño de la Unidad Coordinadora del Proyecto durante los cinco años de ejecución fue satisfactorio. · Desempeño general del prestatario 48. El desempeño general del prestatario fue satisfactorio en las diferentes etapas del proceso de implementación del Proyecto. La mayoría de los componentes fueron ejecutados con éxito no obstante que algunos implicaron desafíos considerables a la capacidad instalada del Ministerio. Lecciones Aprendidas a). Los resultados satisfactorios alcanzados por el Proyecto se deben en gran medida a que se logró mantener la continuidad en la coordinación del Proyecto. b). El Proyecto demostró los beneficios que pueden obtenerse por medio de la aplicación de una administración descentralizada. La ejecución se realizó por medio de las unidades administrativas que eran responsables de cada uno de los componentes. c) Es necesaria la capacitación oportuna del personal responsable de las áreas administrativas y financieras de las unidades ejecutoras y además esos equipos técnicos deben ser asignados de manera permanente. d) La estricta aplicación del Manual de Operaciones del Proyecto representó una clara ventaja para la propia UCP y para su relacionamiento exitoso con las unidades ejecutoras. - 29 - e) Se comprobó la necesidad de que los proyectos cuenten con un comité ejecutivo que dirija y coordine las actividades, en el que participe una autoridad del más alto nivel del Ministerio y los directivos de las dependencias que tendrán la responsabilidad de ejecutarlo. f) El conocimiento de las autoridades superiores del Ministerio, en cuanto al alcance y contenido del Proyecto, así como de sus modalidades de ejecución, asegura una respuesta positiva para el cumplimiento de los compromisos y responsabilidades asumidas. g) Durante la ejecución del Proyecto se evidenció las ventajas que representa que una sola entidad ­la UCP- coordine los proyectos que se ejecutan con financiamiento externo provenientes de diferentes agencias. La coordinación de la UCP permitió armonizar apropiadamente las actividades respaldadas por esas entidades, evitar duplicaciones inconvenientes y potenciar el uso de los recursos. h). La apertura de una cuenta especial en el Banco de Guatemala y la contratación del PNUD como agente de compras, agilizaron los procedimientos administrativos para desembolsar los recursos del préstamo y contribuyeron a darle una mayor transparencia a los procesos de adquisición y contrataciones. i). El MINEDUC demostró que si las condiciones externas a la institución son favorables, sí tiene capacidad para ejecutar exitosamente proyectos de cierta envergadura e incluso proyectos que puedan implicar montos de recursos mayores. j) El MINEDUC debe tratar de aprovechar el ambiente favorable que existe en el país para aumentar aún más las asignaciones presupuestarias destinadas a la educación. A partir de la suscripción de los Acuerdos de Paz el sector ha sido considerado de alta prioridad y se ha generado una conciencia creciente en todos los sectores políticos y sociales. k) Para optimizar el uso de los recursos y alcanzar las metas y objetivos del MINEDUC, los programas y actividades del Ministerio debieran ser sistemáticos y sostenidos en el tiempo. l). La continuidad de la gerencia del Proyecto por parte del Banco Mundial también puede considerarse un factor altamente positivo. (b) Cofinanciers: N/A (c) Other partners (NGOs/private sector): N/A - 30 - 10. Additional Information N/A - 31 - Annex 1. Key Performance Indicators/Log Frame Matrix Outcome / Impact Indicators: 1 Indicator/Matrix Projected in last PSR Actual/Latest Estimate Primary Net Enrollment to increase from By 2001, primary gross enrollment rate 68.2% to 85% by End of Project (EOP). reached 102% and net enrollment reached 85%. Universal coverage achieved for the first % of students completing grade six increased three grades of primary school. to 50% by 2000. Gross rural enrollment to increase annually With measurements starting in 1997, gross by 6%. rural enrollment increased annually by an average of 7.6%. Increases ranged from about 4% to 14%, with the largest increases realized during 1998 and 1999 at a little over 13% each. Girls' rural enrollment to grow from 44.4% to 48% by EOP. Drop out rate in rural areas to decrease from Overall the drop out rate for primary school 10.5% to 5% by EOP. children has decrease to an average of 7%. Repetition decreases from 17% in 1996 to In 2001, repetition has been measured at 5% by EOP. 14.6%, only about 2.4 percentage points down from 17% in 1996. Significant increase in mean achievement scores for math and language tests by EOP. Redefinition of roles and functions at each The redefinition of roles and functions at level of MINEDUC by EOP. each level of the Ministry's structure was completed by year 2000, however, it was not fully executed in practice due to change in administrations. The current Ministry is interested in upholding the work accomplished and reinvigorating the effort to decentralize. Output Indicators: 1 Indicator/Matrix Projected in last PSR Actual/Latest Estimate I. Expansion of Coverage and Equity a) Approximately 1,650 new teachers The Coverage goals were surpassed by contracted and 75,000 new students enrolled 2000, with 272,753 students attending in PRONADE schools by 2000 PRONADE schools which represents and increase of about 208,753 new students since 1996. b) 1,000 new afternoon shifts established, 116,000 students of which 57,000 were first 2000 teachers hired and 70,000 new graders benefited from the afternoon shifts students enrolled by EOP created at 439 schools. 765 teachers were appointed to teach at the shifts. c) Construction/rehabilitation of 71 schools A total of 164 schools were rehabilitated. affected by hurricane Mitch. II. Improved efficiency and quality: Project supported MINEDUC's teacher a) Approximately 13,700 teachers professionalization program where 10,091 (PRONADE and other rural teachers) trained PRONADE teachers completed training in by 2002 in multigrade and bilingual diverse course offerings. Additionally, four methodologies and "Programa de la Nina" teacher training programs were designed and issues. delivered specifically to enhance teacher skills in targeted areas: a)352 teachers completed special training for delivering first grade curricula. b) Approx. 5,500 teachers benefited from training in active teaching - 32 - methodology and multigrade teaching methodology. c) 3,980 teachers were trained in reading of Mayan languages. d) 168 teacher graduated with diplomas in curriculum and linguistic educational management. b) 600 packages of teaching materials and Thousands of packages of educational 150,000 packages of pedagogic materials materials were designed, translated (into distributed to schools each year some of the Mayan languages), printed and distributed for use by teachers and by rural children, especially those attending PRONADE schools. c) National Assessment System designed Under contract to the project, the University and baseline tests applied in math and Del Valle administered three assessments, language to students in grades 3 and 6 one each in 1998, 1999, and 2000. The fourth assessment was administered with financing from the government. III. Institutional modernization: a) Yearly training seminars for MINEDUC staff members (90 each year) and dept. personnel (60 each year) on org. development, strategic planning, budgeting and finance, and human resource mgmt. b) Establishment of 22 departmental The 22 Departmental Directorates were directorates established and are operating according to norms and descriptions designed to guide the process. c) Establishment of at least 5,400 education Over 8,600 COEDUCAS now manage community associations and training of some PRONADE schools. 16,200 members by 2000. 1End of project - 33 - Annex 2. Project Costs and Financing Appraisal Actual/Latest Percentage of Estimate Estimate Appraisal Component US$ million US$ million A. Expansion of Coverage and Equity A.1 PRONADE 18.20 21.37 117 A. 2 Program de la Niña 0.80 0.61 77 A.3 Afternoon Shifts 3.40 3.15 93 A.4 Rehabilitations of Hurricane Mitch 0.00 5.97 B. Improvement of Efficiency and Quality B.1 Curriculum Development 3.30 2.53 77 B. 2 Bilingual Education 0.40 0.31 77 B. 3 Multi-grade Schooling in Rural Guatemala 0.40 0.31 77 B. 4.Fund for Innovations to Support Educational Quality 3.50 0.33 10 B. 5 National Assessment System - NAS 1.90 2.47 13 C. Institutional Strengthening C.1 Support ISES to train COEDUCAS 5.80 0.55 10 C.2 Strengthening MINECUD (planning, human resources) 1.00 0.17 17 C.3 Strengthening 22 Departmental Directories 2.20 3.93 179 C.4 Training MINEDUC & COEDUCAS 2.00 1.53 77 C.5 Development and Improvement of MIS 2.00 1.53 77 C.6 Design and Implement a System of Evaluation .50 0.89 179 IV. Project Administration 1.40 1.13 81 Total Cost 46.80 46.80 100 Project Costs by Procurement Arrangements (Actual/Latest Estimate) (US$ million equivalent) 1 Procurement Method Expenditure Category ICB NCB 2 N.B.F. Total Cost Other 1. Works 0.00 0.00 5.97 0.00 5.97 (0.00) (0.00) (5.38) (0.00) (5.38) 2. Goods 1.55 0.00 0.30 0.00 1.85 (1.13) (0.00) (0.27) (0.00) (1.40) 3. Services 0.00 0.00 6.75 0.00 6.75 Consultants' services and (0.00) (0.00) (6.75) (0.00) (6.75) PCU fixed-term cons. 4. Training 0.00 0.00 4.38 0.00 4.38 (0.00) (0.00) (4.38) (0.00) (4.38) 5. Miscellaneous 0.00 0.00 27.46 0.00 27.46 (Info. Campaigns, (0.00) (0.00) (14.77) (0.00) (14.77) Educational Innovations - 34 - Fund, Teachers' Honoraria, Operating Costs 6. Miscellaneous 0.00 0.00 0.34 0.00 0.34 (0.00) (0.00) (0.32) (0.00) (0.32) Total 1.55 0.00 45.20 0.00 46.75 (1.13) (0.00) (31.87) (0.00) (33.00) 1/Figures in parenthesis are the amounts to be financed by the Bank Loan. All costs include contingencies. 2/Includes civil works and goods to be procured through national shopping, consulting services, services of contracted staff of the project management office, training, technical assistance services, and incremental operating costs related to (i) managing the project, and (ii) re-lending project funds to local government units. Project Financing by Category (in US$ million equivalent) Percentage of Appraisal Category Appraisal Estimate Actual/Latest Estimate Bank Govt. CoF. Bank Govt. CoF. Bank Govt. CoF. 1. Civil Works 1.10 0.40 5.40 0.60 490.9 150.0 2. Goods 4.80 1.20 1.30 0.20 27.1 16.7 3. Consultant Services 8.90 0.00 6.30 0.00 70.8 0.0 4. Training 2.70 0.00 4.80 0.00 177.8 0.0 5. Miscellaneous a. Information Campaign 0.70 0.00 0.00 0.00 0.0 0.0 b. Fund for Education 2.80 0.70 0.30 0.00 10.7 0.0 Innovations c. PRONADE teachers' 11.00 10.60 13.50 11.60 122.7 109.4 salaries d. Operating Costs and 1.00 0.90 1.40 1.40 140.0 155.6 PCU salaries TOTAL 33.00 13.80 33.00 13.80 100.0 100.0 - 35 - Annex 3. Economic Costs and Benefits N/A. - 36 - Annex 4. Bank Inputs (a) Missions: Stage of Project Cycle No. of Persons and Specialty Performance Rating (e.g. 2 Economists, 1 FMS, etc.) Implementation Development Month/Year Count Specialty Progress Objective Identification/Preparation 03/11/1996 2 TASK MANAGER (1); S S EDUCATION SPECIALIST (1). Appraisal/Negotiation 1/27/1997 9 TASK MANAGER (1); S S PROCUREMENT (1); EDUCATION SPECIALIST (1); INSTITUTIONAL DEVMT SPEC. (1); OPERATIONS (1); ARCHITECT (1); MIS SPECIALIST (1); FMS (1); ASSESSMENT SPECIALIST (1). 4/7/1997 5 TASK MANAGER (1); S S CONSULTANTS (2); LAWYER (1); RES.REP. (1). Supervision 06/20/1998 5 TASK MANAGER. S S EDUCATOR (1); OPERATIONS (1); FINANCE (1); MIS SPECIALIST (1); ARCHITECT (1). 12/11/1998 9 TASK MANAGER (1); S S PROCUREMENT (1); EDUCATION SPECIALIST (1); INSTITUTIONAL DEVMT SPEC. (1); OPERATIONS (1); ARCHITECT (1); MIS SPECIALIST (1); FMS (1); ASSESSMENT SPECIALIST (1). 3/10/1999 1 ARCHITECT (1) S S 06/11/1999 6 TASK MANAGER (1); S S PROCUREMENT SPECIALIST (1); INSTITUTIONAL DEVLP (1); OPERATIONS (1); MIS SPECIALIST (1); FMS (1). 12/17/1999 3 TASK MANAGER (1); HS HS OPERATION ANALYST (1); MIS SPECIALIST. 07/08/2000 5 TTL. EDUCATION S S SPECIALIST (1); - 37 - OPERATIONS ANALYST (1); PROCUREMENT SPECIALIST (1); COMMUNITY PARTICIPATION SPEC. (1); COUNTRY SECTOR LEADER (1). 02/22/2001 5 MISSION LEADER (1); S S OPERATIONS ANALYST (1); OPERATIONS (1); COMMUNITY PARTICIPATION SPEC. (1); MIS SPECIALIST (1). 9/21/2001 3 TTL - EDUCATION S S SPECIALIST (1); OPERATIONS ANALYST (1); PROCUREMENT SPECIALIST (1). 4/22/2002 4 TTL - EDUC SPECIALIST (1); S S OPERATIONS ANALYST (1); INSTITUTIONAL DEV SPEC. (1); MIS SPECIALIST (1). 10/11/2002 4 TTL - EDUC SPECIALIST (1); S S OPERATIONS ANALYST (1); PROCUREMENT SPEC. (1); FMS (1); MIS SPECIALIST (1). 2/13/2003 5 TTL - EDUC SPECIALIST (1); S S OPERATIONS ANALYST (1); MIS SPECIALIST (1); COMMUNITY DEV. SPEC. (1); ARCHITECT (1). ICR 04/25/2003 MISSION LEADER (1); OPERATIONS ANALYST (1); FINANCIAL MANAGEMENT SPEC. (1). (b) Staff: Stage of Project Cycle Actual/Latest Estimate No. Staff weeks US$ ('000) Identification/Preparation 254,519 Appraisal/Negotiation n/a n/a Supervision 72 221,187 ICR 5 14,252 Total 77 489,958 This table is based on the information currently in SAP. The figures for staff weeks for identification/preparation and part of supervision are only estimated. No data is available for appraisal/negotiation. For data before FY98, the information system provides total amounts, without disagregating by number staff weeks and project cycle stage. - 38 - Annex 5. Ratings for Achievement of Objectives/Outputs of Components (H=High, SU=Substantial, M=Modest, N=Negligible, NA=Not Applicable) Rating Macro policies H SU M N NA Sector Policies H SU M N NA Physical H SU M N NA Financial H SU M N NA Institutional Development H SU M N NA Environmental H SU M N NA Social Poverty Reduction H SU M N NA Gender H SU M N NA Other (Please specify) H SU M N NA Private sector development H SU M N NA Public sector management H SU M N NA Other (Please specify) H SU M N NA - 39 - Annex 6. Ratings of Bank and Borrower Performance (HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HU=Highly Unsatisfactory) 6.1 Bank performance Rating Lending HS S U HU Supervision HS S U HU Overall HS S U HU 6.2 Borrower performance Rating Preparation HS S U HU Government implementation performance HS S U HU Implementation agency performance HS S U HU Overall HS S U HU - 40 - Annex 7. List of Supporting Documents 1. Colbert de Arboleda y Mora, Julia. Evaluación del Programa de Profesionalización "Inmersión en la Metodología Activa a Docentes en Servicio en Escuelas Rurales Multigrado Sujetos al Proyecto". Agosto, 2002. 2. Lizarralde y Lara, CPA. Dictámenes de los Auditores Independientes. Años terminados en diciembre 1998, 1999, 2000, 2001. 3. Ministerio de Educación. Acuerdo Ministerial de fecha 16 de diciembre de 1997 que creó el Comité Ejecutivo del Proyecto. 4. Ministerio de Educación. Manual de Operaciones. 1998. 5. Ministerio de Educación. Memoria Técnica del Proceso de Fortalecimiento y Expansión de la Metodología Activa en Escuelas Rurales Multigrado. Septiembre 2002. 6. Ministerio de Educación. Anuario Estadístico de la Educación 2,000. Unidad de Informática (sin fecha de edición). 7. Ministerio de Educación. Informe de Avance Enero-Diciembre 1999 del Proyecto Reforma a la Educación Básica. 8. Ministerio de Educación. Informe de Avance Semestral Enero-Junio 2000 del Proyecto Reforma a la Educación Básica. 9. Ministerio de Educación. Informe Semestral de Progreso Julio-Diciembre 2000 del Proyecto Reforma a la Educación Básica. 10. Ministerio de Educación. Informe de Avance Semestral Revisado Enero-Junio 2001 del Proyecto Reforma a la Educación Básica. 11. Ministerio de Educación. Informe de Avance Semestral Julio-Diciembre 2001 del Proyecto Reforma a la Educación Básica. 12. Ministerio de Educación. Informe de Avance Semestral Enero-Junio 2002 del Proyecto Reforma a la Educación Básica. 13. Ministerio de Educación. Informe de Avance Semestral Julio-Diciembre 2002 del Proyecto Reforma a la Educación Básica. 14. Ministerio de Educación. Contrato para la Incorporación de la Informática Educativa en Aulas de Escuelas Primarias. 2001. 15. Ministerio de Educación. Unidad Coordinadora de Proyectos de Reforma a la Educación Básica. Informe Final y Evaluación del Segundo Proyecto de Educación Básica. Mayo, 1999. 16. Ministerio de Educación. Diversos convenios suscritos por el Ministerio para el desarrollo de innovaciones educativas (01-98, 01-99, 02-99, 03-99, 08-99, 14-99. 17. Ministerio de Educación/ Banco Mundial. Ayudas Memoria de las misiones de supervisión y seguimiento. Varios documentos: 1998, 1999, 2000, 2001, 2002. 18. Universidad del Valle de Guatemala. Contrato de Servicios de Consultoría No. 001-2000. Programa Nacional del Rendimiento Escolar (PRONERE). Enero 2000. 19. Universidad del Valle de Guatemala. Informe al Ministerio de Educación de Guatemala sobre el Programa Nacional de Evaluación del Rendimiento Escolar. Junio 1998. 20. Universidad Rafael Landívar. Informe Final de la Evaluación del Programa de Lectoescritura en Idioma Maya como L1. Septiembre 2002. 21. World Bank. Amendment to the Loan Agreement. Basic Education Reform Project. December 17, 1998. 22. World Bank. Project Appraisal Document on a Proposed Loan in an Amount of US$ 33.0 million to Guatemala For a Basic Education Reform Project. April 30, 1997. - 41 - - 42 -