Document of The World Bank FOR OFFICIAL USE ONLY Report No: 24912 IMPLEMENTATION COMPLETION REPORT (SCL-40700) ONA LOAN IN THE AMOUNT OF US$28.5 MILLION TO JAMAICA FOR A STUDENT LOAN PROJECT December 31, 2002. This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. CURRENCY EQUIVALENTS (Exchange Rate Effective December 1995) Currency Unit = Jamaican Dollars (J$) J$ 1.00 = US$ 0.025 US$ 1.00 = J$39.7 FISCAL YEAR April 1 March 31 ABBREVIATIONS AND ACRONYMS AFIs Approved Financial Institutions BOJ Bank of Jamaica CAST College of Arts, Science and Technology GOJ Government of Jamaica HEART Human Employment and Resource Training Trust Fund HRDP Human Resource Development Programme IBRD International Bank for Reconstruction and Development ICT Information Communication Technology IDB Inter-American Development Bank JBA Jamaican Bankers' Association JSLP Jamaica Student Loan Program LIBOR London Interbank Offering Rate MIS Management Information System MOEYC Ministry of Education, Youth and Culture MOFP Ministry of Finance and Planning PIOJ Planning Institute of Jamaica ROSE Reform of Secondary Education SESP Social and Economic Support Programme SLB Students' Loan Bureau SLC Student Loan Council STATIN Statistical Institute UTech University of Technology UWI University of the West Indies WITCO West Indies Trust Company TEI Tertiary Educational Institution TL Tuition Loan GIA Grant-In-Aid Vice President: David de Ferranti Country Manager/Director: Orsalia Kalantzopoulos Sector Manager/Director: Marito H. Garcia Task Team Leader/Task Manager: Kin Bing Wu JAMAICA STUDENT LOAN CONTENTS Page No. 1. Project Data 1 2. Principal Performance Ratings 1 3. Assessment of Development Objective and Design, and of Quality at Entry 2 4. Achievement of Objective and Outputs 5 5. Major Factors Affecting Implementation and Outcome 10 6. Sustainability 11 7. Bank and Borrower Performance 12 8. Lessons Learned 14 9. Partner Comments 15 10. Additional Information 17 Annex 1. Key Performance Indicators/Log Frame Matrix 18 Annex 2. Project Costs and Financing 21 Annex 3. Economic Costs and Benefits 23 Annex 4. Bank Inputs 24 Annex 5. Ratings for Achievement of Objectives/Outputs of Components 26 Annex 6. Ratings of Bank and Borrower Performance 27 Annex 7. List of Supporting Documents 28 Annex 8. Borrower's Comments 29 Annex 9. Student Loan Scheme Structure in Student Loan Bureau 36 Project ID: P038700 Project Name: JM- STUDENT LOAN Team Leader: Kin Bing Wu TL Unit: LCSHE ICR Type: Core ICR Report Date: December 24, 2002 1. Project Data Name: JM- STUDENT LOAN L/C/TF Number: SCL-40700 Country/Department: JAMAICA Region: Latin America and Caribbean Region Sector/subsector: Central govermnent administration (5%); Tertiary education (95%) KEY DATES Original Revised/Actual PCD: 11/02/1994 Effective: 10/25/1996 10/25/1996 Appraisal: 02/16/1996 MTR: 05/10/1999 05/24/1999 Approval: 07/25/1996 Closing: 06/30/2002 06/30/2002 Borrower/lmplementing Agency: GOV'T. OF JAMAICA/STUDENT LOAN BUREAU Other Partners: STAFF Current At Appraisal Vice President: David De Ferranti Shahid Javed Burki Country Manager: Orsalia Kalantzopoulos Paul Isenman Sector Manager: Marito H. Garcia Julian F. Schweitzer Team Leader at ICR: Gwang-Jo Kim Jill Arrnstrong ICR Primary Author: Gwang-Jo Kim; Hongyu Yang 2. Principal Performance Ratings (HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HL=Highly Likely, L=Likely, -JN=Unlikely, HUN=Highly Unlikely, HU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible) Outcome: U Sustainability: UN Institutional Development Impact: M Bank Performance: S Borrower Performance: U QAG (if available) ICR Quality at Entry: U Project at Risk at Any Time: Yes No Quality at Entry Assessment at the time of the project was prepared. However, the project was QAGed twice during the implementation. The ratings are as follows: QAG Assessment of Supervision 2 in FY98 (QSA2) Rating: Highly Satisfactory QAG Assessment for Supervision of Risky Project FY01 rating: Satisfactory 3. Assessment of Development Objective and Design, and of Quality at Entry 3.1 Original Objective: The Jamaica Student Loan Program (JSLP) was prepared in the mid-1990s at the urgent request of the Government of Jamaica (GOJ) for assistance in carrying out key policy reforms in the financing of tertiary education. The GOJ reform goals were to increase cost-recovery in higher education in order to liberate public resources to be channeled to basic education, and to ensure that academically qualified poor students would not be denied access to tertiary education because of financial constraints. The GOJ policy reform goals were sound and consistent with Bank policy on tertiary education financing. Thus, the Bank immediately responded by sending a team to Jamaica to work with the Government in preparing the project. The objectives of the project were to: (a) provide adequate financing to low-income students attending eligible Tertiary Education Institutions (TEI); (b) increase equitable access to tertiary education for low-income students by improving targeting of student loans; and (c) improve the financial sustainability and the administrative efficiency of the existing Student Loan Bureau (SLB) student loan scheme. 3.2 Revised Objective: The project objectives were not revised. 3.3 Original Components: The following components were designed to achieve the development objectives of the PSLP: A. Institutional Strengthening ($1.1 million with contingencies) to improve the capacity of the SLB to (a) target student loans and grants to lower income qualified students; (b) increase loan processing efficiency; (c) improve financial management; and (d) carryout out impact evaluation of the program. B. Targeted Student Loan Scheme ($27.0 million with contingencies) in partnership with commercial banks (Approved Financial Institutions - AFIs); C. Grant-in-Aid Fund ($8.5 million with contingencies) to provide more financial assistance to the neediest students in addition to loans; and D. Public Awareness Campaign ($0.4 million with contingencies) to inform students, parents and the general public of the policy reforms regarding cost recovery and financing options for tertiary education, including the Grant-in-Aid Fund. 3.4 Revised Components: The project was restructured in March 2000. The problems leading to project restructuring -2- started to surface shortly before the project's midterm review in May 1999, when the first beneficiaries of the students loan scheme began to graduate and collections were expected to commence. Friction between the SLB and the AFIs became apparent and implementation of the Student Loan Scheme (Component B) suffered a significant setback because the AFIs were not collecting loans repayments from graduates. Loan arrears began to accumulate fast. Several attempts were made to improve project performance during 1999 to no avail. In early 2000, the GOJ and the Bank agreed that the project should be restructured with respect to Component B. The restructuring acknowledged the fact that the private/public partnership envisaged for the Student Loan Scheme was not working and new implementation arrangements were needed. The solution chosen was to eliminate the involvement of the AFIs in the project and transfer the loan portfolio to the SLB, making the Bureau a one-stop-service agency for university student loans and grants. The restructuring also assumed that once the student loan portfolio was cleared up, the SLB would be-in a position to move towards financial self-sufficiency by selling its loan portfolio in the secondary market - although the latter would require a change in the SLB law. In sum, the aim of the project restructuring was to introduce greater market discipline in the operations of the SLB, allowing for better management control over disbursements and collections and for increased accountability (Annex 9 shows diagrams of the original and the redesigned student loan scheme). 3.5 Quality at Entry: A Quality at Entry Assessment was not carried out when the project was prepared. In hindsight, the ICR rates the quality at entry unsatisfactory, in spite of the fact that the project objectives were well selected and were consistent with the GOJ policy reform goals, with Bank sector policy on higher education financing and with the Bank Country Assistance Strategy (CAS). The project components were well designed except in the case of its main component, the Student Loan Scheme (Component B). The design of the other components was based on the analysis of the SLB which clearly indicated the need to strengthen the institution, improve targeting, and mobilize public opinion and promote the use of the new financing instruments for tertiary education. The design of Component B was unsatisfactory for several reasons. First, it made overly optimistic assumptions about the institutional capacity of the SLB and the AFIs to successfully work together in implementing the scheme. Second, it under estimated the administrative costs involved in operating a student loan scheme. Third, it over estimated the demand for loans from poor students. And fourth, it allowed for the continuation of an inadequate governance structure in the project executing agency. These various design flaws led to excessive complexity of implementation arrangements and to unsatisfactory outcomes. Component B was highly innovative in introducing a partnership between the government and the -3- commercial banks for the delivery of student loans, but the procedures set forth to implement this partnership posed several problems. The original idea was that a partnership with the private sector would bring market discipline to the pre-project student loan scheme operated by the SLB, characterized by high default rates and high administrative costs. In this sense it was a correct initiative. However, the instruments used to achieve this goal were not well developed and frustrated the idea. As originally designed, the implementation procedures for the student loan scheme involved the approval of loan applications by the SLB, loan initiation and loan servicing by the AFIs, and the transfer of funds from the SLB to the AFIs to cover approved loan disbursements. The services to be rendered by the AFIs comprised: (a) Interviewing with approved students applicants, to determine whether or not they could provide collateral to guarantee the loan. Based on these interviews, loan risks were divided between the government and the banks: the AFIs assuming the risks for the loans to students who could provide collateral and the SLB the risks of loans to students who could not provide collateral; (b) Disbursing the approved student loans directly to the corresponding Tertiary Education Institution (TEIs); (c) Seeking reimbursement for the loan amount disbursed to the TEIs from the SLB; and (d) Collecting loan repayments from students, starting six months after their graduation, and transferring the repayments back to the SLB. The main design flaws detected in these implementation procedures are the following: (i) The assumptions regarding the participation of the private sector and the coordination between private banks and the SLB were too optimistic; (ii) The fees paid to commercial banks for their services (3.5 % per annum of amounts disbursed) were insufficient to cover the costs of loan processing and collections. The SLB did not recognize this problem and assumed that the banks would be able to cover part of their administrative costs from the float on loan repayments which were expected to occur monthly but were to be sent to the SLB only quarterly; (iii) The estimate of administrative costs for the SLB (1% per annum over amounts disbursed) was also unrealistic, given the actual services to be performed by the Bureau. In Latin America, total administrative costs for student loan schemes have oscillated between 15-20% (Salmi, 2000) compared to 4.5% in the case of the JSLP; (iv) The demand for student loans from students at bottom three consumption quintiles was over estimated at 75-80%. A more realistic estimate would need to have taken into account that this group of students represented only 20-25% of tertiary education -4 - enrollment (Table 1). A more realistic estimate would have set a target of approximately 50% for share for lower-income beneficiaries; (v) The governance structure of the SLB was not appropriate. The Bureau is legally defined as an executive public sector agency governed by a Council comprising representatives of the public sector (Ministry of Education and Ministry of Finance), of tertiary education institutions, of students, and of one merchant bank. The Council had no representatives from private employers or members specialized in financial management. This deficiency led to a lack of professional oversight of the SLB operations and the very limited private sector experience of Council members contributed to a lack of understanding of the AFI's needs on the part of the SLB; (vi) Compounding the problem of inadequate governance structure, the SLB was not made accountable for its financial sustainability. Mechanisms were not designed to ensure performance-based rewards or penalties to the management, and the Ministry of Finance provided the operating budget of the SLB irrespective of its performance; and, (vii) The intended distribution of loan risks between the AFIs and the SLB could not be enforced in practice. Why would AFI assume risks of any loan, if it could classify all loans as "not having collateral" and shift all the risks to the government? Table 1: Enrollment in Tertiary Education by Consumption Quintile, 1996, 1998 and 2000 Ql (poorest) Q2 Q3 Q4 Q5 Total 1996 3% 8% 15% 40% 36% 100% 1998 3% 11% 8% 24% 54% 100% 2000 0% 2% 7% 14% 77% 100% Source: Survey of Living Conditions 1996, 1998, and 2000. 4. Achievement of Objective and Outputs 4.1 Outcome/achievement of objective: The ICR rates as unsatisfactory the achievement of project objectives because the financial sustainability of the student loan scheme is uncertain. Nevertheless, the objectives of providing for more equitable access to tertiary education for low-income students through the provision of adequate financing were addressed and partly successfully achieved, although the share of low-income students at project completion was lower than that originally envisaged (Table 2). The ICR rates the implementation progress (IP) unsatisfactory as well. Credible information on management effectiveness and financial sustainability of the student loan scheme (such as defaults/arrears and loan repayments data) are not available even two years after project restructuring, in spite of the management improvements made by the Bureau. The SLB Council failed to exercise its oversight responsibility by not asking for these statistics when loan repayment collections began. The fact that the - 5- lack-of these statistics was tolerated by the Council and the SLB indicates a serious flow in oversight and management (see 5.2). Table 2. Distribution of Loan Beneficiaries by Consumption Quintfles Year Bottom 3 Quintiles Top 2 Quintiles 1997 22% 78% 1999 50% 50% Original SAR Target 75-80% Source: Project impact evaluations. 4.2 Outputs by components: A. Institutional Strengthening (US$5.2 million or twice the SAR estimate). ICR Rating: Partially successful but overall Unsatisfactory. The SLB made significant progress towards improving its institutional capacity, especially in improving its capacity to handle a larger volume of student loans applications. The number of applications processed per staff almost doubled (from 48 to 90 during the 1995-2002 period). Also, the SLB was reorganized and improved its staffing early on during project implementation. The Bureau prepared and adopted the Implementation Manual on schedule and modernized its Management Information System (MIS) during project implementation. The most recent enhancement to the MIS involved the installation of a Higher Education Loan Management System (HELMS) which enables SLB to do in-house loan disbursement, servicing and collection functions that were previously performed by AFIs. The SLB is planning to setup a web-based online loan application system to reduce loan-processing time and, at the same time, increase access of students to the loan program by lowering applicant's travel cost, effort, time. Although the SLB administrative costs increased two-fold from 1996 to 2001, at 11.3% of the portfolio (2002) it is still significantly below the average for Latin America student loan programs which average 15-20%. The observed increase reflects the assumption by the SLB of the loan servicing functions formerly performed by the AFIs. At project completion, the SLB is aware that it needs to improve further its efficiency by continuing to update the loan and grant programs and by improving its capacity to identify the actual loan demand from lower-income students. It is also necessary for the Bureau to utilize fully its available instruments and technologies. One immediate challenge in this area is to achieve the integration of the MIS loan application and loan servicing systems (HELMS and older MS Access-based JSLS). In spite of the achievements indicated above, the overall outcome of this component is considered unsatisfactory because the partnership between the SLB and AFIs - a central element to the operation of the loan scheme -- was not successful. Corrective measures where not taken by the SLB. For example, with respect to the fees payable to the AFIs, the SLB actually reduced these fees, making it still more difficult for the AFIs to meet their actual administrative costs, and the issue became a key factor leading to the breakdown of the relationship between the SLB and the AFIs. B. Student Loan Scheme (US$27.05 million or virtually the same as originally estimated). IRC Rating: Partially successful but overall Unsatisfactory. The lack of success of the partnership with the private sector for the student loan scheme and the high rates of loan repayment arrears and default indicate -6- that the scheme is not financially sustainable, and this justifies the overall unsatisfactory rate of this component. In some aspects however, the Student Loan Scheme succeeded in achieving development objectives, particularly with respect to improved equity in access to tertiary education. This was the result of better targeting of student loans. The component was also successful in allowing the government to reduce the share of the education budget allocated to tertiary education because the volume of student loans increased significantly through the project. Equity: More equitable access to tertiary education.The targeting of lower-income qualified students, as verified by impact evaluations, improved by approximately 28% percent between 1997 and 1999 (Table 2). Although these figures are much lower than the Staff Appraisal Report (SAR) target of 75-80%, the equity improvement is very significant. Moreover, as mentioned in Section 3.5, the SAR target was unrealistic given the prevailing demand trends for loans by lower-income students. Among the targeting improvements made during project implementation, an important measure was the reduction by 8.1% of cut-off line for first time applicants. At the project mid-term review, the original target for the share of lower-income beneficiaries was reduced to better adjust for the relatively small demand from this group of students compared to the much higher representation of wealthier students enrolled in tertiary education institutions. Equity: More Equitable Allocation of the Public Education Budget. As loans and grants covered an increasing share of the tertiary education expenditures formerly financed by the public sector, the government allocation for tertiary education declined by 3.3 % during project implementation (from 21.5% in 1996/1997 to 18.3% in 2000/2001) permitting a relative increase in the share allocated for basic education. Increased Coverage. During project implementation 30,104 student loans were approved. The number of student loans increased by 12% in five years, from 3,995 loans per year in 1996-1997 to 4,478 in 2001 -2002 period, due to improved loan application processing procedures. The average loan size varied significantly depending on the type of tertiary institution, reflecting differences in average student income in each institution (from US$541 in teachers' colleges to US$ 1,717 the University of the West Indies [UW]). Table 3 shows the breakdown of loan beneficiaries by type of institution. The relative share and number of loans benefitting students attending non-university tertiary education institutions increased over time. Compared to universities, these institutions are mostly community colleges and teacher colleges where enrollment of lower-income students is larger than that of upper-income students. - 7- Table 3. Loans and Grants by Institutions 1996/1997 2001/02* Percentage Tertiary Institution change Enrolment in Total 26,988 29,445 9 tertiary education UWI 7,867 9,442 20 institutions Utech 7,129 8,010 12 Teacher's Colleges 3,065 3,757 23 Community Colleges and other tertiary 8,927 8,236 -8 Number of loan Total 3,995 4,478 12 per year UWI 2,196 1,717 -22 Utech 1,312 1,338 2 Teacher's Colleges 231 541 134 Community Colleges and other tertiary 256 882 245 Number of grants Total 1,274 2,264 78 per year UWI 793 846 7 Utech 481 593 23 Teacher's Colleges 345 Community Colleges and other tertiary 480 Percentage of Total 15 15 0 students with loans UWI 28 18 -10 Utech 18 17 11 Teacher's Colleges 8 14 7 Community Colleges and other tertiary 3 11 8 Percentage of Total 5 8 3 students with UWI 10 18 8 grants Utech 7 17 10 Teacher's Colleges 9 Community Colleges and other tertiary 6 (*) Estimated enrollment data. Efficiency. The SAR target for loan application processing time was 3 weeks, counting from the date of loan application. This target was achieved and at certain times surpassed. Administrative costs remained approximately the same as the original and revised SAR targets, even though they increased between 1996 and 2001 (as indicated in Section 4.2) due to the redefinition of SLB functions following project restructuring. On the other hand, efficient financial management was not achieved. To the extent known from the available data, the financial sustainability of the student loan program is seriously endangered by the high defaultlarrears rates estimated at 30% (default) and 74% (arrears) for 2001. Precise figures for these indicators were not available at the time of the ICR mission in September, 2002 (Annex 1, Table 1). C. Grant-in-Aid (GIA). ($7.12 million or 84 percent of the SAR estimate). ICR Rating: Satisfactory. The GIA component was entirely funded by the GOJ and succeeded in achieving all of its main goals. In Jamaica, tuition fee assistance alone would not ensure attendance to or completion of tertiary education by - 8- lower-income students, unless additional funds for living expenses, transportation and books were made available. The GLA was originally designed to address this need by supporting full-time, lower-income students at UWI and Utech, helping them meet part of the expenses not covered by tuition loans. The grant program started in 1997. The SAR target for grant recipients from the bottom three consumption quintiles was 75%. The actual result from the second project impact evaluation (1999) showed that 79% of the grants benefitted lower-income student, surpassing the original target. During project implementation, a total of 29,613 grants were awarded, covering 32 institutions and over 30 study programs, and 69% of the beneficiaries were female students. The average grant size was about J$26,598 (equivalent to US$647). D. Public Awareness Campaign ($0.42 million or 105 percent of the SAR estimate). ICR Rating: Satisfactory. The public-awareness program has been very effective. It started at the beginning of the project implementation to inform and sensitize all sectors of the society about the GOJ's policy toward tertiary education financing. Activities included: (i) school visits to secondary schools (340 visits) and tertiary education institutions (80 visits), (ii) seminars for students and training sessions for school administrators, (iii) distribution of brochures (1,000 loan informnation packages per year), posters and booklets to schools, and (v) recurrent messages on radio, TV, and newspapers. In general, feedback from the beneficiaries and school administrators were positive. 4.3 Net Present Value/Economic rate of return: Due to the poor record keeping and delays in transferring infornation about the student loan portfolio from the AFIs to the SLB, it is difficult at this time to calculate the net present value of the project benefits and/or its economic rate of return. In many instances, AFIs' records are incomplete or erroneous, and the reconciliation of the sub-loan records between students, AFIs and SLB is a daunting task for SLB. Given this, an accurate measurement of the project costs and benefits will not be possible until SLB finalizes the reconciliation of all records. 4.4 Financial rate of return: Cannot be calculated for the reasons given in 4.3 above. 4.5 Institutional development impact: The project helped strengthen the institutional capacity of SLB through the following activities: (i) financial management and procurement training of staff at project launch; (ii) technical audit done before the mid-term review to provide feedback to the SLB on how to improve the performance of the JSLP; (iii) a study tour to the Colorado Student Loan Program (CSLP) to learn from good management practices of a financially self-sufficient and viable student loan scheme and to obtain ideas for restructuring the JSLP; participation in several international conferences on student loan programs organized by the World Bank and the CSLP; (iv) international and domestic consultancies to facilitate the restructuring of the information technology systems, and to improve technical writing, document storage, work flow, loan servicing, and project management; and (v) information computer technology consultancies to improved the SLB MIS. -9- The SLB shows institutional improvements resulting from these efforts. In particular, the SLB is now more efficient in its capacity to target loans and to process loan application in timely and correct manner and has a technically appropriate and operational MIS system. Notwithstanding these improvements, SLB has yet to demonstrate that it can service the loans effectively and make the scheme financially sustainable. 5. Major Factors Affecting Implementation and Outcome 5.1 Factors outside the control of government or implementing agency: Jamaica's macro-economic situation deteriorated seriously during the implementation of the project. Starting in 1998, there was a general banking crisis which resulted in a massive government bail out of commercial banks. The banking crisis in turn, led to restructuring and merger of some of the AFIs. While the student loan scheme represented only a very small part of the total portfolio of these AFIs and was not seriously affected by the crisis per se, there were some personnel movement during the downsizing and merger which directly impacted the project because it affected the ability of the AFIs to manage their student loan portfolio well. At the same time, the ensuing recession reduced the employment prospects for new labor market entrants including college graduates. After the tragic events of September 11, 2001, the Jamaica's tourist industry (which accounted for about 22% of the GDP) was severely affected, contributing to rising unemployment rates. According to a recent labor force survey, the labor market participation rate of 25-34 year-olds fell significantly; and the rate of return to tertiary education reached a low 8.4%, significantly below the 12.3% regional average in 1999 (Wu, 2002; G. Psacharopoulos & H. Patrinos, 2002). These circumstances affect the ability of higher education graduates to repay student loans and may contribute to further deterioration of loan repayment levels. 5.2 Factors generally subject to government control: The Government could have improved the governance stmcture of the SLB and tighten the monitoring and evaluation of the agency. From project inception to completion, the SLB reported directly to the Minister of Finance and its governing Council had no representatives from private sector employers or industry associations. There was no critical mass of expertise with private sector experience or financial management among Council members. The Executive Director of SLB, as well as the Board's chairperson, were appointed at project inception, remained the same throughout implementation. The SLB had no clear performance-based accountability mechanism (e.g. no requirement of the publication of collection and delinquency statistics), and the MOFP's financial support that was automatically provided to the SLB, gave no incentives for the agency to become self-sustaining financially. The Government policy on cost-recovery for tertiary education states that the GOJ would finance 50 to 80% of the total annual operating budget of a public tertiary education institution (i.e., 80% for UWI; 70-80% for teachers' colleges, and 55-60% for Utech and community colleges). The project contributed to a reduction in the budget share of tertiary education paid by the GOJ from 22.6% of the total education budget in 1996 to 18.3% in 2001, compared to the SAR target of 15% in the year 2006. However, it appears difficult to monitor the progress towards this SAR target because budget targets are only marginally linked to the student loan and grants program. 5.3 Factors generally subject to implementing agency control: Key indicators of the student loan scheme, including default and arrears rates were not prepared and presented by the SLB in a timely manner. As a result, the monitoring of the major elements of the project -10- became difficult or not at all possible. While SLB could not have made the necessary changes to the Participation Agreement and the Operational Manual on its own, it could have highlighted the issues and initiated a process of change. Similarly, disputes between the SLB and the AFIs over interest rates resulted in students not being informed at the time of loan processing about how much they would have to repay after graduation. Similarly, disputes over the administrative fees were not well handled or appropriately resolved by SLB. These management deficiencies significantly contributed to the unsatisfactory outcome of the project. 5.4 Costs andfinancing: Overall, project costs and disbursements were satisfactory. Audit reports of satisfactory quality were completed on time and the opinion of the auditors was unqualified through out project implementation. All legal covenants were complied with during project implementation. At appraisal, total project costs were estimated at US$ 38 million equivalent, of which 74% or US$ 28.5 million were to be financed by the Bank loan; actual costs ascended to US$ 39.79 or 103% of the SAR estimate. Disbursements were made on schedule and the $28.5 million IBRD loan was fully disbursed in May 2002, a month before the closing date. Financing for each component was close to SAR estimates. Counterpart funds were provided in a timely manner by GOJ and entirely financed the GIA component. The actual financing for each component was in line with the original plan (Annex 2). 6. Sustainability 6.1 Rationale for sustainability rating: The restructured project is unlikely to be financially sustainable without continued injection of funds from GOJ/MOFP given the GOJ's weak commitment to require financial viability from the SLB, as evidenced by the agency's loose accountability and governance structure; and the weak SLB management performance. Financial sustainability. Although the SLB has been operational since 1971 and will continue to operate in the future, this does not mean it would become a self-financing agency. As indicated in prior sections, the private sector has no incentives to participate in the student loan scheme and govemment funds are transferred to the SLB irrespective of its management of the portfolio. Weak management. While the student loan portfolio was held by the AFIs, no collection reports were required from them by the SLB, which largely stayed ignorant of the extent of defaults and arrears affecting the portfolio. Because the AFIs kept the student loan repayment figures current by using their own funds (whether or not they received payments from the beneficiaries), the SLB accounting system for the loan scheme reproduced these bogus data and therefore did not portrait the actual conditions of the portfolio. During the ICR mission, the AFIs indicated the arrears were running from 18% to as high as 75% for most of the project implementation period since collections started. At project completion, the SLB is still trying to reconcile beneficiary loan positions and it may take as much as an additional six months before it can establish a meaningful level and trend on arrears. 6.2 Transition arrangement to regular operations: It is understood that GOJ/SLB intends to continue the loan program, although the SLB did not provide information on the transition arrangements for the project's future operation. The ICR mission could not determine what the transition strategy would be with respect to financing, staffing, and the marketing of the - 1 1 - programs. A list of performance indicators for the program was also not available. Future improvements in program outcomes would be dependent, to a large extent, on close monitoring of SLB's performance by the GOJ, especially after the entire loan portfolio transferred from AFIs has been cleared. Ideally, the GOJ would develop performance indicators to monitor the SLB covering the following aspects of the operation: (a) loan/grant targeting, (b) management efficiency, including loan collection, (c) customer/student satisfaction (one-stop-service) and (d) financial viability. However, the desirability of continuing the program needs to be evaluated not only on the basis of the performance by the SLB, but also in the context of an overall policy framework for tertiary education in Jamaica. Future projects supporting tertiary education in Jamaica would need to take into account, inter alia, alternative financing mechanisms (such as a combination of loans, grants and work study programs), country macro-economic conditions, and returns on investment in tertiary education. 7. Bank and Borrower Performance Bank 7.1 Lending: ICR Rating: Unsatisfactory. There was no particular analytic groundwork pertinent to JSLP at the time of project identification. Nor was there any specific estimates of the costs of delivering the previous loan schemes. At the request of GOJ, the Bank team preparing for JSLP looked mainly into the GOJ's education sector strategies, namely: (a) GOJ's priority to basic education; (b) problems associated with uneven resource allocation among sub-sectors of education; and (c) the need for equitable expansion of higher education. The involvement of commercial banks in the student loan program was incorporated in the project design based on lessons learned from other countries. Project identification and preparation was done rapidly, to meet with the GOJ's urgent request and, to that extent, it was clearly client oriented. However, the product was unsatisfactory. This rating reflects the following deficiencies: (a) the project design did not adequately take into account the capacity and incentives of SLB and AFIs to operate the student loan scheme, and key indicators of project performance were not easy to -monitor: (b) sustainability measures did not reflect the real cost of running JSLP because SLB was heavily subsidized by MOFP; (c) although the annual Survey of Living Conditions (SLC) was available, the share of students from the lower consumption quintiles attending tertiary education institutions was not assessed and the target set by the SAR for the share of lower-income students among loan beneficiaries was not grounded in reality; the relatively weak loan demand from eligible lower-income students was not even considered a risk factor; (d) policies guiding the allocation of public expenditure on tertiary institutions were not directly linked to decisions on the amount of student loan funds provided; (d) lessons from the pre-project student loan operations were not properly assessed and incorporated in the project design; and (e) Bank task leaders changed three time during project preparation, making it difficult to efficiently coordinate with partners. 7.2 Supervision: ICR Rating: Satisfactory. Project supervision is considered satisfactory on account of good reporting of the project implementation progress, which was detailed and timely. Performance ratings given in PSRs went from satisfactory to unsatisfactory as the mid-term approached, reflecting the increasing problems related to loan repayment collection and arrears as students began to graduate (Annex 4). There were serious efforts to improve project performance, such as the use of a technical audit to guide improvement efforts, and guidance on how to improve targeting, with demonstrated improved results that led to the - 12- upgrading of the rating on-project implementation. The problems associated with targeting and management of JSLP were detected early on and addressed proactively, leading to a revision of the guarantor system and reviews of audit reports for JSLP. In particular, the independent technical audit provided by Bank supervision proved very effective in addressing key issues such as targeting, AFIs' collateral practice, interest rates and financial viability. The supervision team included senior sector leaders in the Bank, as well as external experts with experience of administering other successful student loan programs, and this team provided the SLB with timely technical assistance through workshops, a study tour to a well-performing loan program (the Colorado Student Loan Program) and competent consultants for the development of the MIS. The Country Director personally went on mission and used the CPPR to address the issues of the projects. The supervision team effectively engaged in the project restructuring process in 2000 and put in substantial efforts to tightly monitor the key indicators including public expenditure and higher education share, administrative cost, and interest rates. The TTL remained the same throughout project supervision ensuring continuity and consistency of advice. 7.3 Overall Bank performance: ICR Rating: Satisfactory. Although uneven considering the whole project cycle, Bank's performance is rated satisfactory considering the substantial efforts put in the implementation stage. Borrower 7.4 Preparation: ICR Rating: Satisfactory. The Borrower's performance was satisfactory at preparation. The JSLP began on account of the GOJ initiative and loan request. The MOEYC sent to the Bank a letter of sector strategy in which it proposed to reduce the budget allocation to tertiary education. In general, project preparation benefitted from good cooperation on the part of the Government. 7.5 Government implementation performance: ICR Rating: Unsatisfactory. The GOJ made a good effort to improve project performance when the conflict between the AFIs and the SLB surfaced and the issues of collection and arrears became a problem. The Minister of Finance designated a special assistant to monitor the project. However, after this special assistant departed, there was a gap in communications between the MOPF and the Bank. Overall, the project could have benefited from better supervision on the part of the Government and for this reason, its performance is rated as less than satisfactory. 7.6 Implementing Agency: ICR Rating: Unsatisfactory. The performance of the SLB is considered unsatisfactory for a number of reasons. The agency developed an adversarial relationship with the AFIs, by not giving credence to their demand for higher fees and by not standing willing to compromise on this and other issues. In general, there was insufficient communication between the SLB and the AFIs, and insufficient guidance from the SLB to help the AFIs systematize accounting practices, which in turn lead to divergence in the way interest rates for student loans were calculated by the various AFIs. Also, there was insufficient pressure on the AFIs to produce statistics on default rates and arrears. The SLB paid a lot of attention to the front end activities of application processing and loan approval, but not enough attention to loan servicing and collections. The SLB also did not develop an instrument to show students, at the time they contracted the loan, how much - 13 - debt they would incur by the time they graduated, and what would be the total amount owned at through the repayment period. As a results, students were unaware of the financial obligation they assumed by taking a loan. The SLB did not monitor the project properly. For these various reasons, the performance of the SLB is rated unsatisfactory. 7.7 Overall Borrowerperforinance: ICR Rating: Unsatisfactory. The overall borrower performance in general and implementation progress (IP) in particular are considered unsatisfactory because the shortcomings of the project implementation agency and the lack of follow through on the part of the GOJ resulted in poor prospects for the financial sustainability of the student loan scheme. The poor prospects result to a large extent from the administrative weakness of the SLB to process student loans and manage the portfolio. This general assessment coincides with most rating given in the PSRs, although those ratings have varied through the project cycle (Annex 4). Furthermore, not all the terms and conditions set out in the Participation Agreement were clear and performance indicators that were established proved insufficient to guide ongoing supervision by SLB to ensure that full compliance. Critical indicators such as defaults and arrears were not prepared and closely examined in the SLB supervision reports. Statistics on default and repayment were still not available at the time of the ICR mission in September, 2002, after six years of project implementation. It is the ICR team's view that a higher degree of effectiveness may have been possible under a stronger administrative direction in program implementation. 8. Lessons Learned Student loans are not easy to administer well because loan amounts tend to be small and it is difficult to track borrowers during the repayment period, generating relatively high administrative costs. However, student loans are an important mechanism to provide equitable financing for higher education. Even though the JSLP is not a successful example, it provides lessons that are instructive for preparing similar projects. The lessons can be broadly grouped as follows. Project Preparation/Appraisal: * Engage all stakeholders in project preparation, including the private sector and employers, not just the public sector and tertiary education representatives. A good program should uphold the public interest and be subjected to market discipline. One of the design problems encountered by the JSLP was that it failed to consider market trends regardomg adequate compensation of the AFIs administrative costs. Design should provide adequate incentives for the private sector (the AFIs) to participate in the program. The contractual arrangements between the implementing agency and the commercial banks should provide equal remedies to both parties. * Ensure transparent management. The JSLP shows that there is a need to disclose the failings of pre-project programs as well as their merits, making the information available to all those inVolved in designing a better system. Without public information on the arrears of previous and on-going schemes, effective public monitoring of a re-vamped program will not materialize, nor will the project improve its efficiency and effectiveness in the long run. The govemance structure of the executing agency needs to include a critical private sector expertise, particularly in financial management, for it to be able to exercise effective and timely oversight. This should also be complemented by an effective, performance-based accountability mechanism (e.g., statutory requirement for the release of information arrears to gauge the health of the loan scheme and the implementing agency's performance) linked to the public funding of the -14- scheme. It is also important to be realistic about the administration costs of student loan programs, by using international benchmarks to estimate the real cost of public subsidy of the loan scheme. The original JSLP estimation of 5% in administrative costs, was much lower than the average recorded by Latin American and Caribbean countries operating such programs (15-20%). * Exercise analytical thoroughness. It is also important to assess realistically the demand for student loans by students of different income levels. An analysis of the socioeconomic profile of students attending higher education institutions is required in order to be able to set realistic targets for the share of lower-income students benefiting from the loan program. Similarly, it is necessary to examine the employment and unemployment rates of university graduates to gauge whether graduates have the means to pay back their student loans. This information will help assess the long-term sustainability of the program. * Diversify the instruments of financial assistance. In addition to loan schemes, the design of projects intended to finance higher education should explore alternative ways of delivering financial assistance which are less dependent on data and the capacity of the implementing agency, and hence more cost-effective. Some of these alternative mechanisms are direct grants/scholarships to students enrolled in teacher-training and other job-oriented programs, and work-study programs. Project Implementation/Supervision/Country Dialogue: Project supervision of the JSPL was very proactive and engaged the Borrower in meaningful policy dialogue. From these efforts, the following lessons can be drawn: * Provide effective technical assistance. The use of technical assistance, including external assessments, helps provide an independent perspective on controversial issues faced by difficult projects. This is so particularly when these issues pertain to the performance of the executing agency, to policies regarding loan eligibility criteria (e.g. reduction of loans awarded to affluent students), to program sustainability (e.g. real project costs estimates, including direct government subsidies to the implementing agency), and overall project performance. Moreover, Borrowers would benefit from the availability at the Bank of a pool of experts on student loan programs in developing countries that would help prepare sector work and projects, and provide training workshops to clients countries. * Use more specific legal covenants. When project elements are critical to project performance they should be supported by appropriate covenants in the Loan Agreement. For example, the provision of critical data on loan repayment, defaults and arrears could have been linked to continued disbursement of the Bank loan for the student loan component, thereby alerting the SLB to monitor these indicators. * Imprint market discipline. Although not achieved in the JSLP, market discipline is an essential component of a successful and sustainable tertiary education financing program, whatever the mix of implementation agencies chosen for the project. 9. Partner Comments (a) Borrower/implementing agency: (Drawn from Borrower contribution and summarized by Bank staff) As described in more detail in Annex 8, the SLB partly achieved its development objectives. The targeting mechanism effectively identified economically disadvantaged applicants. In the five (5) years of the project, 9,767 persons falling below the poverty line were able to access higher education through the - 15 - financial assistance beyond the payment of their tuition fees, by way of the grants (GIA). The public awareness component of the project has increased compliance with Government policy guidelines. The public is now fully aware of the assistance available to them and the responsibilities attached thereto. The appointment of Liaison Officers at each institution involved, was a critical part of the information dissemination; these liaison officers are full-time employees of the respective education institutions and thus are able to play a critical mediation role. The GOJ has maintained its policy of reducing the allocation of funding to tertiary education, notwithstanding an increase in tertiary education enrollment. This is possible due to the fact that more students are now relying on the assistance provided by the SLB. The enrollment of tertiary students whose per capita income falls below the poverty line has increased significantly. Additionally, the transparency of the appeals process has validated the Government's commitment to ensuring that no student in need is denied a tertiary education. From the Project's Experience, the following lessons were learned: * The project design over-estimated the demand of students from poor families. The targeting was predicated on the premise that the demand for tuition loan included significant representation for the very poor. The project assumed that this demand existed prior to its design. * Project design over-estimated the ability and/or willingness of the commercial banks to be partners. The assumptions made were based on the experience of other countries and did not reflect properly the conditions in Jamaica. The inclusion of the AFIs as risk assessors of a scheme targeting the needy created an insurmountable obstacle for many. In some instances, students were asked to provide collateral for substantial amounts of tuition loans although the scheme had identified them as being in the socioeconomic bracket falling below the poverty line. * The provision of the right of appeal has provided much needed assistance for those who are in temporary financial crisis (temporarily poor) and not defined as poor by the project. The appeals process was a great enhancement to the project. * The level of automation in management information systems, provided at the outset of the project, allowed little or no room for growth, and little or no capacity for analysis. Although this was overcome by 1997/98, an appropriate technology infrastructure would have enhanced the implementation experience. Given this fact, the volumes processed exceeded expectation and the capacity of the installed system. The project designers may have overlooked this important aspect of loan financing and grants program. This oversight had financial as well as human resource and management implications. (b) Cofinanciers: Not applicable. (c) Other partners (NGOs/private sector): (Prepared by Bank staff based on interviews with representatives from TEIs, AFIs and student beneficiaries.) - 16 - The ICR mission interviewed representatives from TEIs, all of whom agreed that JSLP contributed to increasing access to tertiary education and also to ameliorating TEIs' financial stance as it helped them mobilize additional resources. Some pointed out that the JSLP was necessary and but they would use other means of helping poor students (e.g. installment payment schedule). All those interviewed appreciated the accessibility/availability of student loans and SLB. The ICR mission also interviewed representatives of the AFIs, who admitted that their involvement in JSLP was not successful due to reasons such as: (a) lack of proper preparation (e.g., manual for computing interest rates); (b) absence of incentives for "due diligence"; and (c) miscommunication and/or confusion as to the key features of JSLP (e.g. interest rates). 10. Additional Information Additional information can be found in the project file. - 17 - Annex 1. Key Performance Indicators/Log Frame Matrix Table 1. Project Impact / Outcome Indicators Policy Goal Outcome / Impact Staff Appraisal Report Actual/Latest Estimation Indicators (SAR) Target Reallocate Share of public recurrent 15% to tertiary; 33% to -1994/95: 25.2% to tertiary resources toward expenditure of the secondary; and 50% primary -1995/96: 22.0% to tertiary basic education MOEYC to tertiary by 2006 (The target was -1996/97: 21.5% to tertiary education vis-a-vis that unreasonable because it did -1997/98: 21.4% to tertiary of other levels not take into account -1998/99: 20.6% to tertiary changing age cohort at -1999/00: 21.0% to tertiary various levels of education -2000/01: 19.6% to tertiary and changing demand for -2000/01: 18.3% to tertiary various levels of education) Original target: 75%-80% First impact evaluation (1997): (without regard to 22% of loan recipients belonged Increase equity of Share of loan recipients distribution of tertiary to the bottom 3 quintiles and access to tertiary in bottom 3 quintiles enrollment by quintile) 62% to top quintile education by poorer students Revised target at the Second impact evaluation mid-term review: 50% of (1999): 50% of loan recipients recipients from the 9th from the bottom 3 quintiles and decile in 1999, 40% in 2000, 28% from the top quintile. and 30% in 2001 to take account of the usual over representation of students from mid-income families in tertiary education. First impact evaluation (1997): Share of grant recipients SAR target: 750/o-80% Few tertiary students from the in bottom 2 quintiles bottom 2 quintiles Second impact evaluation (1999): 79% of grants recipients from the bottom 3 quintiles. Increase efficiency Turn-around time of loan of student loan application in SLB Less than 3weeks 3 weeks program Administrative costs less * Audit 1997: 4.3% Increase Administrative costs than 5%. Revised mid-term * Audit 1998: 8.5% sustainability of target: 16% in 1999; 13% in * Audit 1999: 8.3% student loan 2000; 10% in 2001. * Audit 2000: 9.9% program * Audit 2001: 9.9% * Audit 2002: 11.3% -18 - Default / Arrears rates Less than 10% of all loans Aprill 1999: 69% 2000:28% May 2001: In the absence of new information, the data from each AFI were used: BNS: 30% CIBC: 50% Union Bank: 50% NCB: 74% High level of arrears were results of a lack of commitment of AFIs to the student loan program, lack of incentives for the AFIs to improve its efforts, and weak MIS in AFIs. Real Interest rates SAR target (defined as The real interest rates are inflation minus nominal positive. Since 1999/2000, the rates) > zero interest rates are determined by examining market conditions. Specifically, the interest rates chared to students were: -1996/97: 15% (passbook savings rate +5%) -1997/98: 15% (passbook savings rate +5%) -1998/99: 18% (add-on) -1999/00: 18% (add-on) -2000/01: 16% (add-on) -2001/02: 15% (add-on) -19 - Annex 1. Table 2. Project Performance Indicators Component Project Activity Monitorable Indicators Actual A. Institutional 1. Reorganize SLB New organizational and job Completed August 1996. Strengthening descriptions. Ongoing review to achieve (US$ 2.6 million) optimum efficiency resulting in additional staff 2. Hire staff Hire critical staff Completed July 1996 Hire remainder staff Ongoing 3. Produce Operational Operational Manual Completed October 1996. Manual Reviewed 2000. 4. Install new MIS/FIS System fully functional Completed July 1996 5. Train staff in All relevant SLB staff trained Completed August 1996. MIS/FIS Training ongoing for new staff and in tandem with modernization and upgrades. 6. Conduct Evaluation Evaluation survey - year 1 Completed in 1997 Impact Studies Evaluation survey -year 4 Completed in 1999 7. Produce semi-annual Semi-annual-reports Done for most supervision reports missions, except for May 2001. B. Expanded Assess and extend Done each year. Student Loan loans to eligible No. of loan applications During the project life, total Scheme students No. of loans approved 40,699 received, (US$ 27 million) No. of loans, by institution 29,613 approved, covering No. of loans, by program of study 32 institutions and over 30 No. of loans, by gender programs. 68% of the No. of loans, by bank beneficiaries are female. Average loan size, by institution Average loan size is J$65,653. Financial Indicators Loan approval rates. Done annually Default or delinquency rates Provided since December 1998. C. Grant-in-Aid Assess and award Done each year. Program grants t eligible No. of grant awarded During the project life, total students No. of grants, by institution 14620 grants were awarded, (US$ 8.5 million) No. of grants, by program of covering 32 institutions and study over 30 programs. 69% of No. of grants, by gender the beneficiaries are female. Average size of grants Average grant size is J$26,598 D. Public Done each year. During the Awareness Promotion of GOJ's project life: Campaign policy on tertiary No. secondary institutions visited 340 (some more than once) education financing No. of tertiary institutions visited 80 (more than once) Education of sources of No. of information packages 10,000 per annum financial assistance distributed to institutions Presence in printed media Frequent and consistent. Presence on radio Frequent and consistent Presence on T.V. Frequent and consistent -20- Annex 2. Project Costs and Financing Project Cost by Component (in US$ million equivalent) * - ; - ; . [ , -Appraisal -jActuallLatest':-- Percentage of,: .By~~~~C~~omponent ~~~~Estimfte -, Estmate' - P; roject ost By- p eIl l:S o US$ThIlloon 'ApiusaimaI2 A. Institutional Strengthening 2.60 5.20 200 B. Student Loan Scheme 27.00 27.05 100.2 C. Grant-In-Aid 8.50 7.12 83.8 D. Public Awareness Campaign and Credit Education 0.40 0.42 105 Programi Total Baseline Cost 38.50 39.79 Total Project Costs 38.50 39.79 Total Financing Required 38.50 39.79 Note: For actual column, a weighted August 2002 rate of J$41.116805 = US$ 1.00 was used. Project Costs by Procurement Arr ngements (Appraisal Estimate) (US$ million equivalent) Expenditure Category ICB NCB Consulting Other2 N.B.F. Total Services Cost Training and Seminar ___ _ _ _ _0.1 0.1 Equipment and Software -0. I/ 0.1 ___ _ (0.1) (0.1) Printing Materials 0.01 0.1 0.1 Rental ___ - - 0.1 0.1 Technical Assistance 05. - 0.5 (0.3) (0.3) Subloans and Admin Costs - 27.0 2/ 27.0 (27.0) (27.0) Grants _ _ 8.5 8.5 Recurrent Cost 2.0 2/ 2.0 (0.97) (0.97) Total 0.5 29.2 8.8 38.5 (0.3) (28.1) (28.5) Note: Figures in parenthesis are the respective amounts financed by the World Bank. 1/ LIB for computers and software; International shopping for vehicles. 2/ Not subject to procurement. - 21 - Project Costs by Procurement Arrangements (Actual) (US$ million equivalent) Expenditure Category 1[CB NC¢B Consulting Other2 N..B.F. Total Services Cost Training and Seminar - (0.05) 0.40 0.40 (0.05) Equipment and Software - (0.20).aI - 0.1 (0.20) Printing Materials - 0.33 0.33 Rental - - - 0.00 Technical Assistance - 0.60. - 0.60 (0.20) (0.20) Subloans and Admin Costs - 27.05 2/ - 27.05 (27.0) (27.05) Grants - 7.12 7.12 Recurrent Cost - - 4.2 4.20 (1.00) 2/ (1.00) Total - - 0.6 29.05 12.14 39.79 (0.25) (28.25) (28.5) Note: Figures in parenthesis are the respective amounts financed by the World Bank. 1 Use a weighted August 2002 rate of 41.116805 = US$ 1.00 2/ Not subject to procurement. 3/ LIB for computers and software; International shopping for vehicles. 4 Variance occurred due to exchange rate and recruitment of highly qualified personnel Project Financing by Component (in US$ million equivalent) _ _Percentage of .A.ppraisat - momponent .-- ppaisal Esaimnale .A-ctuaVLaleso Eslimalle k t.. ank G-ovt. CoF. Bankt coml CoF. Bank Go. -Co F. A. Institutional 1.50 1.10 1.49 3.71 99.3 337.3 Strengthening B. Student Loan Scheme 27.00 27.05 100.2 C. Grant-In-Aid 8.50 7.12 83.8 D. Public Awareness 0.40 0.00 0.42 105.0 Campaign and Credit Education Program -22 - Annex 3. Economic Costs and Ben'efits No economic costs and benefits analysis 'was carried out at the time of project appraisal, nor was it done at the time of the ICR due to lack of data. -23 - Annex 4. Bank Inputs () Missions: ,_._. Stage of Project Cycle No. of Persos and Specialty .. Performae Rating- e -(e'. g.2 Econ,omists l-,FMS, etc.)Fa, 4,',141ementation Developneniti' ,.. Month/Year Cou'nt ep,Special' ,i, '., -,-.PObjective Identification/Preparation 11/1994 3 TASK MANAGER (1); ECONOMIST (1); CONSULTANT (1). 01/1995 7 TASK MANAGER (1); BANKING CONSULTANT (1) IMPACT EVALUATION CONSULTANT (1); STUDENT LOAN CONSULTANT (1); ECONOMIST (1); CONSULTANT (1); FINANCIAL ANALYST (1). 05/1995 8 TASK MANAGER (1); BANKING CONSULTANT (1) IMPACT EVALUATION CONSULTANT (1); STUDENT LOAN CONSULTANT (1); FINANCLAL ANALYST (1); HUMAN RESOURCES SPECIALIST (1); SENIOR ECONOMIST (1); SUPPORT STAFF (1). 12/1995 6 TASK MANAGER (1); BANKING CONSULTANT (1); FINANCIAL ANALYST (1); MIS CONSULTANT (1); ECONOMIST (1); BANKING SYSTEM CONSULTANT (1). Appraisal/Negotiation 03/1996 4 TASK MANAGER (1); COUNSEL (1); BANKING CONSULTANT (1); MIS CONSULTANT (1); Supervision 03/1997 4 SENIOR ECONOMIST (1); S S EDUCATION ECONOMIST (1); FlNANCLAL MANAGEMENT (1); BANKING (1) 11/1997 2 PROGRAMME OFFICER (1); S S EDUCATION ECONOMIST (1) 06/1998 4 EDUCATION ECONOMIST (1); U S BACK-UP TASK MANAGER (1); MIS CONSULTANT (1); STUDENT LOAN -24- CONSULTAN (1) 12/1998 4 TASK MANAGER (1); HU U EDUCATION ECONOMIST (1); FINANCIAL SPECIALIST (1); EDUCATION SPECIALIST (1) 05/1999 4 TTL (1); FMS (1); PROJECT U S ADVISOR (1); PROGRAM OFFICER (1) 05/1999 2 TTL (1); FINANCIAL MGMT S S SPEC. (1) 10/2000 4 TEAM LEADER (1); S S PROCUREMENT SPECIALIST (1); FINANCIAL MANAGEMENT (1); CONSULTANT (1) 05/2001 5 ECONOMIST-TM (1); HD U S SECTOR LEADER (1); PROCUREMENT SPECIALIST (1); FINANCIAL MANAGEMENT (1); IT SPECIALIST (1) 05/2001 4 TEAM LEADER (1); U S FINANCIAL MANAGEMENT (1); PROCUREMENT SPECIALIST (1); CONSULTANT (1) 12/2001 5 ECONOMIST-TM (1); U U ROCUREMENT SPECIALIST (1); FINANCIAL MANAGEMENT (1); CONSULTANT (2) 06/2002 1 ECONOMIST-TM (1); U S ICR 09/2002 3 SENIOR EDUCATION U S SPECIALIST (1); OPERATIONS OFFICER (1); STUDENT LOAN CONSULTANT (1) (b) Staff: K -Stage of Project Cycle - Actual/Latest Estimate.- -NcCStaffweeks -S--0-0) 1: Identification/Preparation 40 200 Appraisal/Negotiation 19 95 Supervision 60 349.9 ICR 10 40.9 Total 129 685.8 -25- Annex 5. Ratings for Achievement of Objectives/Outputs of Components (H=High, SU=Substantial, M=Modest, N=Negligible, NA=Not Applicable) Rating O Macro policies O H OSUOM * N O NA O Sector Policies OH OSUOM ON O NA Fl Physical O H OSUOM O N O NA OI Financial O H OSUOM O N O NA I Institutional Development 0 H O SU * M 0 N 0 NA O Environmental O H OSUOM O N * NA Social O1 Poverty Reduction OH OSUOM ON O NA El Gender OH OSUOM ON ONA OI] Other (Please specify) O H OSUOM O N * NA O Private sector development O H OSUOM O N * NA OI Public sector management O H OSUOM O N O NA LI Other (Please specify) O H OSUOM O N * NA - 26- Annex 6. Ratings of Bank and Borrower Performance (HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HU=Highly Unsatisfactory) 6.1 Bank performance Rating O Lending OHS Os *u OHU I Supervision OHS OS OU OHU Ol Overall OHS OS O U 0 HU 6.2 Borrowerperformance Rating OI Preparation OHS OS OU 0 HU OI] Government implementation performan.ce O HS OS 0 U 01 HU O] Implementation agency performance OHS OS 0 U 0 HU O Overall OHS OS 0 u O HU -27 - Annex 7. List of Supporting Documents Second Impact Evaluation of the Student Loan Project, Final Report, Policy Research and Development Institute, Tobago, April 2000. First Impact Evaluation of the Student Loan Project, Evaluation of the Effectiveness of Student Loan Bureau Targeting Mechanisms, Final Report, Lorraine Blank, Independent Consultant, November 1998. Technical Audit on Financial Analysis and Operations Review of the Student Loan Bureau, Price WaterHouse Coopers, 1999 Statistics on Subloan and Grand-in-Aid Application and Approval by income, gender, institution, and study program, Student Loan Bureau, 1996-2002. KPMG Peat Marwick, Kingston, Auditors' Report for Jamaica Student Loan Project, March 31, 2002. Price WaterHouse Coopers, Kingston, Auditors' Report for Jamaica Student Loan Project, March, 2001. KPMG Peat Marwick, Kingston, Auditors' Report for Jamaica Student Loan Project t, March, 2000. KPMG Peat Marwick, Kingston, Auditors' Report for Jamaica Student Loan Project, March, 1999. KPMG Peat Marwick, Kingston, Auditors' Report for Jamaica Student Loan Project, March, 1998. Price WaterHouse Coopers, Kingston, Auditors' Report for Jamaica Student Loan Project, March, 1997. Participation Agreement between AFIs and the SLB, revised version, SLB, 2000. Commitment Letter between AFIs and Students, Revised version, SLB, 2002. Various Project Implementation Progress Reports, Student Loan Bureau 1997-2001 Loan Agreement, Jamaica Student Loan Project, Loan No. 4070-JM, World Bank, August 2, 1996. Project Agreement, Jamaica Student Loan Project, Loan No. 4070-JM, World Bank, August 2, 1996. Staff Appraisal Report, Jamaica Student Loan, Report No. 15594-JM, World Bank, July 1, 1996. Pre-appraisal Mission Terms of Reference, Aide-Memoire, and Back to Office Reports, World Bank, November 1994 - December 1995. Appraisal Mission Terms of Reference, Aide-Memoire, and Back to Office Reports, World Bank, March 1996 Various Project Status Reports, World Bank, 1997-2002. Aide-Memoires of Various Supervision Mission. World Bank, March 1997, December 1997, June 1998, December 1998, May 1999, October 1999, March 2000, May 2001, December 2001. Jamil Salni, "Student Loans in an International Perspective: the World Bank Experience", 2000, LCSHD Paper Series N. 44, Washington D.C. Kin Bing Wu, 2002, "Education and Economic Growth", Draft. George Psacharopoulos and Harry A. Patrinos, 2002, "Returns to Investment in Education, A further Update", Draft. -28- Annex 8. Contribution from the Borrower ORIGINAL OBJECTIVES The specific objectives of the project were to: 1. Provide Jamaican students (especially the neediest) with adequate financing options to pay for their share of higher education on an on-going basis. 2. Increase equitable access to tertiary education through targeting criteria designed to benefit students of lower means. 3. Achieve acceptable levels of financial sustainability and administrative efficiency of the SLB through a consolidated student loan programnme. The project objectives supported the Govermnent of Jamaica's policy of financing education by cost sharing, articulated in letter dated June 10, 1996 from the Ministry of Education, Youth and Culture to the World Bank. This policy necessitated a shift in allocation of Government resources to reflect a prioritization of primary education with receipt of 50% of the available educational resources - 33% to secondary and the remainder to tertiary. In recognition of the reduction in allocation to tertiary, the primary objective of the Project was "to maintain" a targeted students loan and grant programme. Against the background of worldwide experience including Jamaica's past experiences which resulted in low recovery rates, the project took into account certain requisites of a sustainable scheme. These included a) effective collection, b) the maintenance of positive interest rates and c) a competent and capable administration and specifically the importance of measuring need in order to establish an appropriate targeting mechanism. This demanded financial skills, socio-economic analytical skills as well as technical resources and skills. In a very real sense, the project presented significant challenges. By and large, we were embarking on an uncharted course. The challenges and indeed the risks were mitigated by: 1) the commitment of the management of the SLB to the project objectives; 2) the effective project implementation arrangements whose design (as a strategy) was re-evaluated and adjusted where necessary; 3) the adoption of appropriate change management strategies. PROJECT COMPONENTS 1. Institutional Strena-theninm: The project aimed to improve the management efficiency of the SLB through restructuring its organization and staffing. The new structure should reflect the project objectives and mandate. The strengthening of the operational procedures, computerized management of the content of each application and the subsequent means testing and evaluation process, with the relevant documentation were done. A new organizational chart with relevant job descriptions were put in place accompanied by an improved cadre of staff with appropriate skills and experience. Several steps in the processes were automated by a fairly functional MIS system. Relevant training is conducted both in-house and externally on an on-going basis to increase the efficiencies and drive toward best practices. 2. Expanded Student Loan Scheme. This expansion is reflected both in terms of the value of each loan and the number of loans as access to tertiary education has improved. Additionally, the shift in - 29 - allocation by Government (reduction to tertiary) will create commensurate increases in tuition fees. The targeting mechanism was designed to ensure that no needy eligible Jamaican would be denied a tertiary education for reasons of finance. In its initial design, the SLB would assess and approve the loans, while the AFIs would disburse and collect and provide relevant loan documentation. 3. Grant-In-Aid. It was recognized that at the lower levels of need, tuition fee assistance would not ensure attendance to or completion of the programme of study as funds for living expenses, transportation and books were not available. Against that background, the GIA was designed to assist the approved applicants assessed to be at the lower levels of the socio-economic ladder. 4. Public Awareness Campaign. In the period under review, over 350 school visits were made. Some schools were visited at least three times in the period. The SLB undertook several seminars/training sessions for Liaison Officers and Guidance Counselors with a view to ensuring that the student borrowers were facilitated at every step of their tertiary journey. Brochures, posters and booklets were also distributed. All forms of media were used exhaustively. Compliance has consequently increased each year and is now at a satisfactory level. Public support for our policies and practices is now evident. REVISED OBJECTIVES In the 3rd year of the project (1999), it was recognized that the students' interest would be best served by a "one stop facility", i.e. approval documentation, processing and collecting being undertaken at one place. Accordingly, the objectives were changed to transfer the disbursement and collection activities to the SLB. All the accounts affected were also to be returned to the SLB. As at July 2000, all approvals and disbursements were undertaken by the SLB. OUALITY AT ENTRY The project objectives were consistent with the Government's stated policy objectives on education. Government's objectives included reduced allocation to tertiary education, with the accompaniment of careful targeting and support schemes to assist those who could not afford to pay the full fees. Importantly, through targeting the project addressed the critical needs of the poor who obtained access to tertiary education. However, the measurement of poor/needy became particularly controversial as there were persons not included who demonstrated a lack of ability to fund themselves. This necessitated ongoing review. In this regard, impact evaluation exercises were conducted; the first in 1997 and the second in 1999. The inclusion of the AFIs introduced another set of challenges. By and large, they managed the student loan portfolio without the deserved "due diligence" thereby increasing the risks of insustainability. The AFIs appeared to have been co-opted into the project, after the completion of the design. The commitment of their executive seemed apparent but in actuality, the loans were not treated as a bank product. Documentation was scarce and non existent at times, and the students loan accounts were generally not managed like other customer accounts. The Participation Agreement signed by the Executive prior to their actual participation did not appear to form part of the working document for AFI student loan processors. The several seminars hosted by the SLB and the procedures and processes agreed to by the AFIs and SLB were generally not effected in the actual management of the portfolio. There was no operational manual in fact provided for the AFIs at the inception. However, the Particpation Agreement specified the accounting practices to be observed in its Clause 15. In short, whereas the AFIs' executives, -30- for the most part, were committed the demonstrated culture of the AFI staff with respect to student loans "lacked diligence or recognition of the potential customer base the portfolio could provide." Additionally, the volume of loans processed and approved by the SLB exceeded all expectations and raised issues of preparedness and application of resources for such high volumes. The commitment of the GOJ and the management team was unswerving. Working along the lines of the project components and applying the necessary competencies and skills ensured its success. The necessary organizational restructuring included complementary investments in computerization, strengthening of technical areas and institutional reforms. The latter necessitated a work culture re-orientation of staff and stakeholders in order to strengthen the implementation capacity of all involved. Also, there were some assumptions made by the World Bank as a result of other countries' experience about the project expectation and design which did not hold true for Jamaica, the critical one being the ability and/or willingness of the commercial banks to partner in the project. It was also predicated on a much higher enrollment level of the poor in tertiary programmes than actually existed. -ACHIEVEMENTS OF OBJECTIVES AND OUTPUTS Achievements of Obiectives The SLB achieved its development objectives. The targeting mechanism has effectively identified needy applicants. For 1996/1997, 1,353 applicants received Grant-In-Aid while in 2001/2002 (to date) 2,407 applicants received close to Quintile 1. In the five (5) years of the project, 9,767 persons falling below the poverty line have been able to sustain their access to a tertiary programme through financial assistance beyond the payment of their tuition fees, i.e. by way of a GIA. The public awareness component of the project has increased compliance with our policy guidelines. The public is now fully aware of the assistance available to them and the responsibilities attached thereto. The appointment of Liaison Officers at each institution was a critical part of the information dissemination, as being already full time employees of the respective institutions, they played a critical mediation role. The GOJ has maintained its policy of reducing the allocation of funding to tertiary. Notwithstanding, tertiary enrollment has seen a steady increase from year to year with many persons relying on the assistance provided by the SLB. The presence of tertiary students whose per capita falls below the poverty line has increased significantly. Additionally, the transparency of the appeals process has validated Government's commitment to ensuring that no needy student is denied a tertiary educationi. The Management and technical capacity of the SLB have been augmented by the commitment of the implementation team who brought a wide range of relevant experience to the table. The efficiency of the operations and the continuous monitoring and management of the sustained growth in the number of applications required modernization and upgrading of the IT infrastructure. The automated environment provided easy access to the management reports and assessments which informed operational decisions. These decisions became critical to efficiency as the high volumes of loans demanded unanticipated increases in capital and human resource. OUTPUT BY COMPONENTS 1. Institutional StrenLthening. The managerial and technical capacity of the Bureau improved significantly over the period under review. Although the staff complement was significantly reduced in the first year we effectively processed twice the number of applications processed in the prior year. The - 31 - human and financial resources were applied in accordance with the stated objectives of improving the efficiency and increasing the number of loans particularly in favour of the poor. The mandated Operational Manual was produced and approved. The approved loan beneficiaries were able to attend their respective institutions with timely registration. Annual Financial Statements and Budgets were completed and submitted on a timely basis. Perfonnance exceeded expectations with the SLB remaining annually well within its budgetary constraints although expenditure was exceeded on sub-loans and GIA. 2. Expanded Student Loan Scheme. The number of loans processed each year increased from just over 4,000 in Project Year 1 to a steady 6,200+ in the current year. The current programme contrasts sharply with prior programmes particularly in respect of targeting. The following Table shows the steady expansion in the number and value of loans. It should be noted that years 1997/98, 1998/99 and 1999/00 saw negative fluctuations. These were the years of the AFI involvement. In the 2 years since the restructure, the increase has been tremendous. Academic Year # of Approved Loans Value of Loans (J$) 1996/97 4,009 277.4 million 1997/98 4,908 211.3 million 1998/99 4,244 215.6 million 1999/00 3,347 206.9 million 2000/01 4,037 274.7 million 2001/02 4,478 351.3 million An analysis of the per capita available resources of the loan beneficiaries will also reveal an increase in the number of the poor among them. 3. Grant-In-Aid. Loans for tuition are not able to cover living expenses which can be substantially more than tuition fees. Students from lower income levels were often reluctant to access tertiary programmes as the costs were prohibitive to them, notwithstanding tuition assistance. As the project targeted the poor it was inconceivable that maintenance assistance not be given. Against that background, the GIA component materialized. This too has seen a steady increase in payout again with very slight fluctuations. These fluctuations coincided with the involvement of the AFIs and are in keeping with those reflected in the loan distribution. Academic Year _# of GLIA Value of GL4 (J$) 1996/97 1,353 34 miilion 1997/98 2,026 51 million 1998/99 2,523 61.1 million 1999/00 2,167 57 million 2000/01 2,291 68.3 million 4. Public Awareness Campaign. The effect of our public awareness campaign has indeed been felt. Initially, the novelty of cost sharing and targeting were not well received publicly. Our attempts to improve repayment cut against the notion of the funds being "Government money" (not to be repaid). These created significant challenges to the creation of a positive image. Thematically, the campaign embarked on creating a national conscience for repayment and the pursuit of education as an investmient. In pursuit of both, brochures and posters were distributed and several television and radio programmes were aired. The - 32 - printed media was also employed to our advantage. There have been in the project period over 350 school visits (some schools were-visited up to 3 times) bearing the relevant messages. We have now created a standard presentation (PowerPoint) for all such visits or events. Training has also been undertaken. We have had training sessions for guidance counselors which culminated in the realization of liaison officers as frequently financial guidance was provided outside of regular counseling. The Students' Loan Bureau by and large has become a household name with a positive professional and corporate image. FINANCIAL MANAGEMENT - PROJECT COSTS OF FINANCING 1. Project Cost. The total expenditure for the project (to March 2002) amounted to J$1,643,683,309 measured against a budget of J$1,582,997,000, as summarized below and detailed in the Audited Report. A. Institutional Strengthening $ 221,036,306 B. Student Loan $1,112,156,009 C. Grant- in- aid $292,805,465 D. Public Awareness: $17,685,529 Campaign and Credit Education Program Total $1,643,683,870 The variance occurred as a result of the exchange rate difference in the amounts allocated under the 1996 project. The conversion rate used in the initial budget preparation J$40 - US$1 Average rate at which expenditure was incurred J$44.85 - US$1 The exchange rate difference J$ 4.85 2. Financing Plan The actual cost of the project amounted to 1.66 billion Jamaican dollars with a World Bank loan amounting to $1,175,635,870 covering 74% of actual cost. The government of Jamaica was the sole provider of Grant-in-aid and Supplementary Loans. The component breakdown is as follows: Financing Plan By Component GOJ World Bank Amount %Amount/o A. Institutional Strengthening 157.7 71.463.2828.6 B. Student Loan Scheme 1,112.1100 C. Grant-in-aid 292.8 100.0 D. Public Awareness 17.5 99.9020.1 3. Accounting and Audits Throughout the term of the project the Bureau maintained records and accounts for all project components and consolidated them into the project accounts and SOE, which have been reviewed by the World Bank team on a bi-annual basis. The expenditures under sub-loan were disbursed on the basis of statement -33 - of expenditure submitted by the Bureau. The Bureau at all times maintained the supporting documents which are available to the Bank's Missions and the independent auditors as required by the project and as seen in Appendix 4. LOAN RECOVERY AND SUSTAINABILITY The sustainability of the project is increasingly linked to loan recovery and delinquency management. In the five (5) years of the project, the SLB has collected J$426.4 million against an expected amount of J$394.8 million. This amount includes persons who have decided to repay their loan in full in advance of its due date. The level of arrears among the accounts due for repayment are higher than we would want them to be. The actual measurement is difficult at this time as the records from the AFIs in many instances, are incomplete or erroneous. Additionally, frequently students are able to provide us with proof of payment that was never forwarded to the SLB. The reconciliation of AFI records, student records and SLB records is a mammoth task. For this period, we are relying on the actual cash collected to measure our loan recovery status. CONCLUSION Lessons Learnt During implementation and post implementation, there were lessons learnt and observations made which can no doubt be helpful in the future. Particular reference should be drawn to the assumption underlying the original project design. The inclusion of the AFIs as partners in the scheme is of particular importance. The project was designed in 1995 during the existence of another scheme managed by AFIs. Issues relating to sustainability, student convenience and the plight of poor tertiary students were causing considerable concem. Effectively, the inclusion of the AFIs as risk assessors of a scheme targeting the needy created an insurmountable obstacle for many. In some instances, they were asked to provide collateral for substantial amounts of tuition loans although the scheme had identified them as being in the socio-economic bracket falling below the poverty line. The collateral could not be forthcoming. The targeting was predicated on sO the premise that the demand for tuition loan included significant representation for the very poor. The project assumed that this demand existed prior to its design. Whilst there were some poor, the expectations were overstated and whereas all the poor who applied were granted loans and grants, the presence of the vulnerable (those above quintile 1 but lower than 3) was dominant. Whilst the public education campaign and the effects of the management over the years under review have attracted an increase of the poor, this is evidenced by the significantly increased number of approvals now falling below the poverty line. The Survey of Living Conditions has indicated in prior years the levels of the poor present in the tertiary population. Public education strategies could have been employed to attract the desired target prior to project start date. The measurement of need remains a grey area in the minds of many. Our reliance on the Survey of Living Conditions gave every assurance that our measurement is correct. To many however, "need" is relative to a particular set of circumstances at a particular time. In many circles, the discussion continues. We are certain that all the poor who apply have obtained assistance. The provision of the right of appeal has therefore provided much needed assistance for those who are in temporary financial crisis (temporarily poor) and not defined as poor by the project. We believe that the inclusion of the appeals process was of great enhancement to the project. Importantly, some provision should have been made for the "vulnerable" under different borrowing arrangements. We believe that in many such cases, they - 34 - demonstrated the ability to pay while studying. Had we been able to lend these persons, their repayment with interest could have created increased amounts for lending to the poor. In fact, as they must repay to continue borrowing to study, their repayment would be far more "certain". The level of automation provided at the outset had little or no room for growth, and little or no capacity for analysis. Although this was overcome by 1997/98, an appropriate technology infrastructure would have enhanced the implementation experience. Given this fact, the volumes processed exceeded expectation. The project designers may have overlooked the required capacity of the technology provided. This oversight had financial as well as human resource and management implementations. Cost sharing and targeting were new to most persons and were not well received by the public initially. The "Public awareness campaign" had to be "recreated" to work with the existing scenario as the Jamaican experience of coming from free education for all to means testing for a loan was a "real culture" shock. This presented significant challenges for the team as the objection to the present student loan guidelines came from all levels of the society. Whereas the World Bank staff was supportive, there was no -documentation or training that could provide direct relevant assistance or guidance in the initial years. The exposure provided by the Colorado Student Loan Scheme in 1999 provided some exposure subsequent to the decision to take over the loans from the AFIs. A consultancy programme engaged some of their staff for a short time.- The expense incurred and the fact that the operational path was largely different from the path being pursued by the SLB did not warrant a continued relationship; However, with respect to delinquency control, skip tracing and networking, they offered the SLB significant guidance. The appeals process gave to the persons whose loans were declined, a welcomed second chance. It sought to review on humanitarian grounds circumstances negatively impacting the available financial resources of a household. Notwithstanding that the means test indicated that these persons could pay their way for the academic year, they were unable to do so. This was due to the chronic illness of a.member of the household, the recent loss of employment, death of a family member or special education needs/expenses of a "challenged" member of the household. The Appeals Committee is selected from the educational and fmancial community. The interaction of the Committee and the SLB team presented opportunities for SLB's continued public education. The process of evolution from the project design to the present has delivered many lessons of the socio economic and cultural environment of this nation that could only have been recognized and appreciated by those who had shared the "experience"'of the nation. It was this quality that enabled the management team guided by its Council members to make intelligent assumptions for public education adjustments to the project design, such as creating a "one stop SLB" to build the relationships and networking across the educational community, which has facilitated a successful programme for Jamaica. We are of the view that this quality/feature should be relied upon in a project design. Also during the life of this project, every effort should be made to tap on this very real "reservoir" that is the understanding born out of the sharing of culture, experience and traditions of a people to enhance the project experience and ultimately, the project success. It follows therefore, that Task Managers of the World Bank should be inclined not only to recognize the value that may be derived but to incorporate this feature in the project's life and relationship. - 35 - Annex 9. The Student Loanm Scheme Structure n the Stu¢dent Lonm BBureau inm Jzmanae 1nn 0A° R LHh&OU Offg WCR 0u&lm a - -n $ i ,~~WW kMtf%A hM Do Fm ~~~~~~~~~~~~~w fflmmw m9 g~ij~flg1T® on~ L g n = bmm Source Aid Memir Marh,000 - O '¶frO ~ -~ -q3 qe~ - W@g % UX SF s . O Sq2ib~~~~~m~I3k , To v uP _ . ...~~~~~~~~___ . , . . . S s . . .. . . A F8Q D3o°°D(M6v$!44D301[X0)10$e(m ~~~~~~~~~~fr = =. . _. Gb@WCi@4 Source:e Aid Me r March,v 200 Lt --- t -o - - 36 -37- Repore No.: 24912 Type: R