- ma~ - Economic Development Institute of The World Bank Seeing for Yourself Research Handbook i for Girls' Education in Africa h Eileen Kane ~~~: ~ j EDI LEARNING RESOURCES SERIES EDI LEARNiNG RESOURCES SERIES Seeing for Yourself Research Handbookfor Girls' Education in Africa Eileen Kane The World Bank Washington, D. C. C) 1995 The International Bank for Reconstruction and Development / THE WORLD BANK 1818 H Street, N.W. Washington, D.C. 20433, U.S.A. All rights reserved Manufactured in the United States of America First printing December 1995 The Economic Development Institute (EDI) was established by the World Bank in 1955 to train officials concerned with development planning, policymaking, investment analysis, and project implementation in member developing countries. At present the substance of the EDI's work emphasizes macroeconomic and sectoral economic policy analysis. Through a variety of courses, seminars, and workshops. most of which are given overseas in cooperation with local institutions, the EDI seeks to sharpen analytical skills used in policy analysis and to broaden understanding of the experience of individual countries with economic development. Although the EDI's publications are designed to support its training activities. many are of interest to a much broader audience. EDI materials, including any findings, interpretations, and conclusions, are entirelv those of the authors and should not be attributed in any manner to the World Bank, to its affiliated organizations. or to members of its Board of Executive Directors or the countries they represent. Because of the informality of this series and to make the publication available with the least possible delay. the manuscript has not been edited as fullv as would be the case with a more formal document, and the World Bank accept- no responsibility forerrors. Some sources cited in this paper may be informal documents that are not readily available. The material in this publication is copyrighted. Requests for permission to reproduce portions of it should be sent to the Office of the Publisher at the address shown in the copyright notice above. The World Bank encourages dissemination of its work and will normally give permission promptly and, when the reproduction is for noncommercial purposes, without asking a fee. Permission to photocopy portions for classroom use is granted through the Copyright Clearance Center, Inc., Suite 910. 222 Rosewood Drive, Danvers, Massachusetts 01923. U.S.A. The complete backlist of publications from the World Bank is shown in the annual Index of Publicafions, which contains an alphabetical title list (with full ordering information) and indexes of subjects. authors, and countries and regions. The latest edition is available free of charge from the Distribution Unit. Office of the Publisher, The World Bank, 1818 H Street. N.W., Washington, D.C. 20433, U.S.A., or from Publications. Banque mondiale, 66. avenue d'lena, 75116 Paris. France. Eileen Kane is head of the Dcpartment of Anthropology at St. Patrick's College, Maynooth, Ireland. The cover photo is by Andrea Bosch. Library of Congress Cataloging-in-Publication Data Kane, Eileen. Seeing for yourself: research handbook for girls' education in Africa / Eileen Kane. p. cm.-(EDI learning resources series) Includes bibliographical references (p. ). ISBN 0-8213-3453-0 1. Women-Education-Research-Africa-Handbooks, manuals, etc. 2. Education-Research-Africa-Methodology. I. Title. 11. Series LC2417.K35 1995 376'.96-dc2O 95-25244 CIP Contents Foreword vii Preface ix Acknowledgments xi Part I. Before We Begin 1. Introduction 3 Girls' Education 4 Why Africa? 6 Why Do Research? 6 Can You Do the Research? 10 The Handbook 10 How to Use the Handbook I I Further Readings to Help You 13 2. Social Research: The Debate Today 15 How Do We Know About the World? 17 Further Readings to Help You 25 Part II. Planning Your Research 3. What Do You Want to Study and Why? 29 Step 1: Get Your Research Idea 29 Step 2: Clarify the Goals and Purpose of Your Research 31 Step 3: Choose Your Perspective 33 Combining the Approaches 36 Perspective and Purpose 36 Further Readings to Help You 37 4. Planning Research that Describes 39 Etic Approach 39 Step 4: Develop a Rough Research Idea and Refine It 39 iii Contents Step 5: Create a Research Statement or Hypothesis 41 Step 6: Identify the Subtopics for Study 42 Step 7: Make Decisions about Sources of Inforrnation 52 Step 8: Make Decisions About Information Gathering Techniques 53 Emic Approach 54 Our Girls 59 Further Readings to Help You 59 5. Planning Research that Explains and Predicts 61 Cause and Effect Studies 63 Experimental Designs 63 Causation 69 Choosing Your Experimental Design 75 What Experiments Do and Do Not Do 75 Survey Designs 77 Further Readings to Help You 81 6. Whom Will You Study? 83 Probability Sampling 84 NonProbability Sampling 86 Sample Size 88 Sampling and NonSampling Errors 90 There's a Lot More... 91 Further Readings to Help You 91 Part III. Research Techniques: The Basic Tools 7. Selecting Your Techniques and Strategies 95 Your Research Strategy 95 Your Research Techniques 98 Working with Local People and Learning from their Knowledge 104 Developing Key and Proxy Indicators 104 Ethics 106 Further Readings to Help You 107 8. Using What Is Already Available 109 Using the Literature 109 Statistics 111 Some Tips 111 National Statistical Offices 114 National Government Publications 114 International Agencies 114 Bilateral Donors 116 Other Organizations 117 Research Publications and Other Materials 118 "Gray" Literature 120 Other Written Materials 120 Further Readings to Help You 121 iv Contents 9. Surveys 123 Surveys and Questionnaires 123 Mini Surveys 128 Questionnaires 149 Looking Back at Our Interview 150 Further Readings to Help You 151 10. Measures, Scales, and Indices 153 Some Warnings 154 Types and Scales 156 Further Readings to Help You 160 11. Unstructured and Semistructured Interviews 161 Introduction to Qualitative Techniques 161 Unstructured or Informal Interviewing 162 Cultural Bias 170 Further Readings to Help You 174 12. Two Qualitative Strategies: Case Studies and Participant Observation 175 Case Studies 175 Participant Observation 176 Further Readings to Help You 188 13. Other Qualitative Approaches 189 Story Completion or Sentence Completion Devices 189 Pictures 190 Games 191 Traditional Stories 191 Drawings 192 Role Play and Figures 192 Content Analysis 192 Further Readings to Help You 194 14. Rapid Assessment and Participatory Learning Approaches 195 Rapid Rural Appraisal and Participatory Learning and Action 196 Uses of RRA and PLA 199 Comparisons with Conventional Methods 200 Stages in an RRA Project 200 Stages in a PLA Project 204 A Sample PLA Project 205 Advantages, Disadvantages, and Dangers of RRA and PLA 225 The Philosophical and Ideological Foundations of RRA and PLA 226 How You Can Use RRA and PRA as Part of a Larger Study 226 PLA Sources and Contacts 228 Further Readings To Help You 229 15. Recording and Organizing Qualitative Information 231 Written Notes 231 Tape Recording 241 Video Recording 241 v Contents Conclusion 242 Further Readings to Help You 242 Part IV. Working with Your Results 16. Qualitative Analysis 245 Ways to Look at Your Material 246 Stages of Data Analysis 248 Computer Programs for Qualitative Analysis 263 Further Reading to Help You 263 17. Quantitative Analysis 265 Univariate Analysis: Frequency Distributions 267 Bivariate Analysis: Association and Correlation 275 Using Descriptive Statistics 284 Integrating Qualitative and Quantitative Data 285 Further Reading to Help You 286 18. Presenting Your Results 287 Report, Talk, or Workshop? 288 Length 289 Balancing the Emphases in the Report 289 Getting a Plan for Your Report 290 Fornat 292 Circulating the Study 297 Using the Results 299 Further Readings to Help You 300 19. Epilogue: What Next? 301 Appendix I. Sample Research Outline 303 Appendix II. Grid Approach for Assessing a Situation or Evaluating a Project 309 Appendix Ill. Some Measures, Scales, and Indices Used to Study Girls' Education 313 Bibliography 317 Index 323 vi Foreword James Wolfensohn, the new president of the World Bank, has said: Women are absolutely central to sustainable development, economic advancement, and social justice.... Increased emphasis on girls' education [is] the most important contribution the Bank can make to the strengthening of women's role in development. As part of its ongoing contribution to enhancing girls' educational opportunities, the Economic Devel- opment Institute has held seminars for policymakers, developed videos and training manuals for educa- tors, and created targeted handouts for opinion formers. Seeing for Yourself- Research Handbook for Girls' Education in Africa reflects a new dimension in the Bank's efforts to create effective strategies for girls' edu- cation. It recognizes the need to reach deeper into the community and to draw upon local participation to help produce reliable, timely, and culturally sensitive information on issues and priorities. But in many de- veloping countries, research capacity is limited. This handbook is designed for people who do not have pro- fessional training in research, but who understand the issues and have important local knowledge. It provides a "how to" for school-level personnel, concerned parents, ministry officials, nongovernmental or- ganizations active in education, and others who want to join as partners in helping to reap the economic and social benefits that arise from educating girls. Vinod Thomas, Director Economic Development Institute vii Preface Educational systems in most countries have been created for boys-not intentionally, but the effect is the same. They view the student as a person who has time to study because his work at home is not essential to the household; who is not physically, culturally, or spiritually endangered in the school setting; who is not expected to marry early or become pregnant; who functions in an atmosphere of intellectual respect for his abilities; who has appropriate textbooks that reflect his concerns in life; who is taught by people like himself who can act as role models; and whose parents see the relationship between education and advan- tages in later life. That student is a boy. The resources have gone into his education. Our problem is to bring educational systems to the point where they also address the needs of the other 50 percent of the potential school clientele. The introduction to this handbook shows the many economic advantages that a nation can harvest by educating its girls. But as I was writing it was the human side, not the economic, that moved me most. I kept a picture in my mind of two bright little girls, born 100 years apart, one in 1885 and one in 1985. Both had to leave school at nine. The first became my grandmother. "I was going to be a teacher," she said. She spent her life as a servant. The second lives in The Gambia, and her parents have removed her from school in order to send her brother. She was looking in the school window when I saw her. "I was going to be a teacher," she said. She thinks she will spend her life as a servant. It's in our power to do something about the second little girl. This handbook tries to show you how to find her and to work out how to help her. ix Acknowledgments This handbook was written during a six-month period that I spent as Visiting Irish Professor at the Econom- ic Development Institute of the World Bank, Human Resources Division (EDIT IR). The professorship was funded by the Irish government. Time constraints meant that the help of a number of people was not only welcome, but essential. I would like to thank Paud Murphy, EDIHR, who commissioned the work; Armand Van Nimmen, division chief, EDIHR, who supervised it; and the many people who provided invaluable comments. Among these, first and foremost, were the reviewers, Robert Chambers, Institute of Develop- ment Studies, University of Sussex, and James Hoxeng, U.S. Agency for International Development, whose suggestions led to important revisions, as did those of World Bank staff members Carlos Rojas and Maggie Kilo. Adhiambo Odaga was enormously generous with her time and considerable knowledge of girls' ed- ucation in Africa. Elizabeth King kindly provided data for the scattergram in chapter 17. Veena Bhaskar and Hanna Sarkees provided administrative support. The staff of the World Bank Sectoral Library was, as al- ways, cheerful and efficient in helping with materials: Chris Windheuser, Eliza McLeod, Olga Boemeke, Al- cione Amos, and Sophie Hoolboom all helped. I would particularly like to thank Professor Richard Scaglion of the Department of Anthropology, University of Pittsburgh, for enabling me to meet the publication deadline by stepping in and prepar- ing chapter 17. Edith Ortanez processed multiple drafts of the handbook and prepared many of the drawings. The car- toons in chapter 14 were on the spot sketches drawn by researcher Mary O'Reilly de Brun in a project in The Gambia, and, in keeping with the spirit of the book, the "game" in chapter 16 is the work of a twelve- year-old schoolgirl, Deirdre Murphy. John Didier oversaw production, and Alice Dowsett's careful editing put law and order, as well as various felicitous touches, on the final version. Desktop publishing was done by Alex McLellan and proofreading by Kathy Rettinger. The publication of this work coincides with Armand Van Nimmen's retirement from the World Bank after thirty years of service. I extend the traditional Belgian embrassade to him, along with heartfelt appreci- ation for his guidance and support. xi Part I Before We Begin Introduction If we could fit all of human history into one hour, the second half of the twentieth century would take up a tiny fraction of one second. More than 5 billion of us are sharing the earth in this tiny window of eternity; 800 million of us are going hungry, 100 million are homeless, and 14 million of our young children die each year. For many of us our minds are as parched as our bodies: nearly a billion of us cannot read and write and 300 million of our school-aged children are not in school. Two-thirds of those who cannot read and write are women; 60 percent of children not in school are girls (World Bank 1990). GovLrnments, national and international agencies, nongovernmental organizations, and local commu- nities have all focused their attention, in various ways, on these problems, with some success. In the last fifty years average life expectancy in the developing countries as a whole has increased by almost half; un- der-five mortalities have fallen by a third (UNICEF 1989); the lives of 2.5 million children are saved annu- ally by immunization and oral rehydration therapy; the percentage of people able to read and write has almost doubled (UNICEF 1991); and in most places, food production has outpaced population growth in the past twenty-five years (FAO 1985). In the last decade, advances have been even more dramatic: more than 3 million children who would otherwise have been paralyzed by polio are living normal lives because of increases in immunization, infant deaths from neonatal tetanus have been cut by half, and experts foresee an end to mental retardation and blindness caused by nutritional deficiencies (UNICEF 1994). Nevertheless, many problems are stubbornly persistent. In Sub-Saharan Africa, for example, population growth rates are high and food production and life expectancy lag behind other areas. In relation to education, many countries still do not provide primary education for all children. Numer- ous students repeat grades, thereby occupying scarce places; many of those lucky enough to enroll in pri- mary school drop out before completing it; and the level of achievement students attain is often low. These problems affect girls more than boys. In Africa, for example, girls' primary school enrollment accounts for only 57 percent of the school-age population, compared with 75 percent for boys (UNICEF 1991). On aver- age, girls in Africa are more likely than boys to drop out of primary school and to score lower on examina- tions, which in turn limits their enrollment at the postprimary levels. The purpose of this handbook is to help you, as a policymaker, planner, administrator, teacher, or member of a group or association in Sub-Saharan Africa, to understand the research process and to study problems and opportunities associated with girls' education in Africa. This will help you to take girls' education into account when needs are being assessed, policy is being made, and projects are being planned and evaluated. 3 4 Before We Begin Why study girls' education? What about boys? Why Africa? What research is needed? Can you do it? GIRLS' EDUCATION This handbook emphasizes girls' education for two reasons. The first reason is the greater advantages aris- ing from girls' education. From a national point of view, the evidence is now strong enough to support the conclusion that investing in educating females is probably the single most cost-effective investment to im- prove standards of living in developing countries, particularly among the poorest populations. We also know that this investment will not slow economic growth (Schultz 1989). Primary education provides a foundation for helping to alleviate poverty and improve social and economic development. The litany of advantages to developing countries of increased participation of girls in primary school is well documented (see, for example, Benavot 1989; Floro and Wolf 1990; Hert and others 1991; King and Hill 1993; Psacharopoulos 1989; Subbarao and Raney 1992; Tietjen 1991). Some economic benefits include * Faster growth of gross national product (GNP) * Higher rates of return on girls' versus boys' education * Higher family incomes * Improved participation in wage employment and in home and nonmarket production * Higher productivity, a more skilled labor force, better employment opportunities, greater occupa- tional mobility, and improved earnings * The possibility of improved participation in the more capital-intensive areas of self-employment and areas of the informal sector that require literacy and numeracy. Figure 1-1 shows the interrelationships among these benefits. Figure 1-1. The Multidimensional Economic Impact of Girls' Primary Education lProdu(tou y Skills Formation' Emoyment I (Rural and Higher Waig * Numeracy Partiipation Urban) Gain Accesa to r - - - - _ ___ * Literacy 'Labor Force \come s Quali t y of Primary | | . & i Participation toCreditiiual Quality of Primary i I klst efr I Education I Standard eTasks Employment I I ado Primary i * Education and I * Enhanced Ability Informal Sector PrHigh r t I Structure to (more outout) Education I ll l Meel Household of -* Quality ot Teachers - Perceive and Basic N ads Girls I * Access to Material i information \;(mity level) InputsI h Communicate I Eann _ * Curriculum Content I with others I … - - - - - -Evaluateand I More Efficient Hiher adjust to IPerfonnfance on- rdetvt changes Nonmarket * Domestic Worktt rowth -Adopt new land Home _ Child Carr, (cozmmunity and. - Adoptnewhn I Production Choice le national levels) i technology hat -Reduce nutriion) I subjective I LowerFeriliy uncertainty Produclon ot . _ _ _ * ~~~~~~~~~~Goofr for Home _ _ _ _, ~~~~~~~Consumpb,on (e.g.. gardening) Source: Floro and Wolf (1990, p. 17). Reprinted with permission from Creative Associates International. Introduction 5 Even more interesting is that the gender gap itself matters. Other things being equal (capital stock, labor force), countries with a larger gender gap in education will have lower economic production, and in coun- tries with similar per capita incomes and patterns of expenditure, those with smaller gender gaps in edu- cation will have better indicators of social welfare (King 1990, p. 6; King and Hill 1993, pp. 14-21). Some social benefits arising from increased education of girls include * Lower fertility rates * Lower infant mortality rates * Improved nutrition * Increased life expectancy * Better opportunities for their children in the next generation. Figure 1-2 illustrates these relationships. Figure 1-2. Interrelationships among Girls' Primary Education, Culture, and Social Change Independent_ _ Income Cultural> Pattern | Soure:Foroand olf(199,p30)Rerintdwthprmisionfro Crceative AsatuesItrioa In cesed femaleneducat is l asocs e s E iots effetiveway o acievethem Fo exaple,reserch nseent-tw deveiopng Countriesn hw htdu Desire for Fewer Children v l+ f andneded t Desire t Educat ChildrHeanh _9 _ + _ ~~~~~~~~~~~~~Migrabown Source: Floro and Wolf (1990, p. 30). Reprinted with permnission from Creative Associates Internafional. Increased female education is not onlv associated with these benefits-sometimes it is the single most effective way to achieve them. For example, research in seventy-two developing countries shows that dou- bling the number of girls at the secondary level in 1975 would have lowered fertility and reduced the num- ber of infant deaths by a greater amount than doubling the scores for family planning services, and that doubling the number of girls in secondary school would have reduced infant mortality more than cutting the ratio of population to doctors by half or than doubling GDP per capita from US$650 to US$1,300 (Sub- barao and Raney 1992, pp. i-ii).1 The second reason for emphasizing girls' education is the weak base from which girls' education is de- veloping. While most governments claim to be enthusiastic about increasing girls' participation in educa- tion, they have done little to see that it actually comes about. Indeed, despite sensitization programs, seminars, policy statements, and so on, many of the good intentions to improve girls' education remain at the theoretical rather than the implementation phase. 1. The scores reflect "the availability and strength of service-related family planning programs in developing coun- tries (Subbarao and Raney 1992, p. 9). 6 Before We Begin No one is arguing that girls' education should be developed at the expense of boys' education. Indeed, all the research shows that improving educational opportunities for girls generally enhances the life of peo- ple in the community, boys included. WHY AFRICA? In the past three decades, the nations of Sub-Saharan Africa have made considerable gains in educating their children. The number of girls in primary school rose from 51 to every 100 boys in 1960 to 76 to every 100 boys in 1990, and the number of girls in secondary school rose from 51 to every 100 boys in 1960 to 67 to every 100 boys in 1990. In many ways, however, Sub-Saharan Africa has more to gain by increased en- rollment of girls than any other region of the world. In 1986 it had the lowest female gross enrollment ratios (enrollment of students of all ages as a percentage of the population of official school age), in 1985 it had the lowest absolute rate of female persistence to Grade 4 (the percentage of children starting primary school in 1985 who continued to Grade 4 by 1988), and in 1990 it had the second lowest rates of female literacy in the world. On average, in 1990 Sub-Saharan African women had less schooling than women in any other region of the world. (For none of these indicators, however, does Sub-Saharan Africa have the largest gender gap- the discrepancy between the figures for males and females [Hartnett and Heneveld 1993, pp. 7-101-al- though six African countries fall in the top ten countries with the largest gender gap in first-level education [UNICEF 19911). Table 1-1 shows figures for these problems in a number of African countries. WHY DO RESEARCH? All developing countries are at the initial phases of analyzing their situation and developing programs that address the problems of improving girls' education. From an operational perspective, the effectiveness of these programs will depend on the commitment of the government, education staff, and parents; on good coordination among donors; and on the relevance and quality of information from which the programs are devoloped. We need to know more, for example, about the areas addressed in the following sections. We Need to Know More about the Impacts of Girls' Education In the economic sphere we know little about how education affects women's home production of goods and services, partly because statistical data are lacking, partly because those figures that do exist are not grouped by gender, and partly because some effects are not readily quantifiable. Important areas of social information are also missing: most social benefits that have been studied so far relate to women's roles as homemakers and mothers. The benefits to women themselves and to the perfor- mance of their other roles in society have received less attention, again partly because these are not easily quantifiable, but also because until recently they may have been seen as less important. You may want to look at these and other impacts of girls' (and women's) education. Some experts have identified the follow- ing areas as those where research on the impact of education is required (Floro and Wolf 1990, pp. 76-77): * The differences that the structure and content of schooling makes to women's social roles, their eco- nomic activities, and the impact that women have on society. * The impact of women's education - In rural areas - On women of different socioeconomic classes - On women's access to credit, both formal and informal - On women's capacity to engage more widely in economic and social activities - On girls' social skills, self-confidence, and sense of efficiency Introduction 7 - On the rate of return from women's nonmarket or nonmonetary activities - On the status of women. Table 1-1. Girls and Womnen: The State of Deprivation in Africa Adult literacy Enrollme 7t ratios: Pregnant Percentage of Life expectancy: rate: females as a Enrollment ratio. Contracep- women births attended females as a females as a percentage of females as . tive immunized by trained Matemal percentage of percentage of males percentage of males p mvalence against health person- mortalitY males males 1986-91 1986-91 (%) tetanus, nel rate Country 1992 1990 Primary-school Secondary-school 1980-93 1990-92 1983-92 1980-91 Niger 107 43 57 44 4 45 15 700 Angola 107 52 93 - Ix 8 15 - Mozambique 107 47 71 56 4 32 25 300 Sierra Leone 108 35 70 57 4 80 25 450 Guinea-Bissau 108 48 55 44 1 x 35 27 700 x Guinea 102 37 48 33 1 x 70 25 800 Malawi 103 - 83 50 13 66 55 400 Rwanda 107 58 99 67 21 88 29 210 Mali 107 59 57 44 5 8 32 2,000 Liberia 105 58 55 x 39 x 6 20 58 Somalia 107 39 50 x 58 x 1 5 x 2 1,100 Chad 107 43 44 25 1x 5 15 960 Eritrea - - - - - - - - Ethiopia 107 48 x 65 71 2 7 14 560 x Mauritania 107 45 70 45 4 40 40 - Zambia 103 80 92 56 15 20 51 150 Nigeria 107 65 77 77 6 25 37 800 Zaire 106 73 75 50 1x 29 x - 800 Uganda 105 56 83 50 5 16 38 300 Burundi 107 66 81 67 9 56 19 - Central African Republic 110 48 61 35 - 87 66 600 Tanzania 106 - 98 80 10 15 53 340 x Ghana 107 73 82 65 13 9 40 1,000 Madagascar 106 83 96 90 17 2 58 570 Sudan 105 28 71 x 74 x 9 14 69 550 Gabon 106 66 - - - 86 80 190 Lesotho 109 - 116 148 5 x 40 40 - Burkina Faso 107 32 62 56 8 26 42 810 Benin 107 50 51 38 9 83 45 160 Senegal 104 48 73 52 11 26 41 600 Togo 107 55 63 30 34 81 54 420 Cote d'lvoire 106 60 70 44 3 35 50 - Cameroon 106 64 86 68 13 7 64 430 Congo 110 63 - 38 x - 60 - 900 Zimbabwe 105 81 98 85 43 60 70 - Namibia 104 - 111 127 26 52 68 370x Kenya 107 74 96 70 27 37 50 170 x South Africa 110 96x - - 48 - - 84x Botswana 110 77 105 107 33 46 78 250 Mauritius 110 89 x 102 100 75 77 85 99 - Not available. x "Indicates data that refer to years or periods other than those specified in the column heading, differ from the standard definition or refer to only part of a country" (UNICEF 1994, p. 62). Note: Nations are listed in descending order of their under-five mortality rates. Source: UNICEF (1994). 8 Before We Begin We Need to Learn How to Overcome the Obstacles to Girls' Education The reasons for girls' low enrollment, persistence, and achievement in the region are undoubtedly many and complex, and may differ across and within countries. Research from many developing countries sug- gests, for example, that the constraints to girls' education are related to * Delayed rate of return for governments, because the benefits of increased investment in schooling can take many years to become apparent * Legal restrictions * Family costs, including opportunity costs * Sociocultural barriers * Early marriage * Teenage pregnancies * Gender biases in classroom practice and textbooks, * Inaccessibility of schools, which many parents see as a greater obstacle to girls than to boys * Cultural perceptions of boys' superior abilities * Poorer performance of girls on examinations * Lack of employment opportunities for educated girls (see, for example, Bellew and King 1993; Floro and Wolf 1990; Namuddu 1992). The evidence also shows that the relative importance of these factors in parents' schooling decisions dif- fers among communities, often along economic, geographic, and ethnic lines. To what extent do these affect the country, region, or community that concerns you, and why? You may want to look at problems or strengths that have been identified through an examination of sta- tistics for your country or community. For example, Hartnett and Heneveld (1993) have looked at the dif- ferences between boys and girls in forty-six countries in terms of access to school; attainment, or the length of time they remain in school and the level they reach; and accomplishment, or their success after leaving school. By looking at these for your own country, you can prepare a country profile that may highlight areas worth investigating, for example, in Tanzania you might want to examine why so few girls complete sec- ondary school, in Mali you might want to see why girls have problems with access to primary education. You might use Hartnett and Heneveld's tables, or following their guidelines, use your own country's na- tional or subnational statistics to identify questions to study. We Need to Develop Strategies for Improving Girls' Participation We also have information from developing countries about the success or otherwise of a number of inter- ventions designed to boost girls' enrollment, for example, those for getting girls into school (access), keep- ing them there (retention), and helping them to achieve (table 1-2). However, very little of this information comes from African countries. In addition, we do not have enough information to be certain what has made the interventions succeed or fail, and we cannot say with any certainty how effective they will be under oth- er conditions or in other places. What should be done, and what we need to know to do it, may vary widely in the country, or even the community, in which you are working. You may want to explore some of these ideas in your research. Introduction 9 Table 1-2. Summary of Strategies that Work Factors addressed Positive impacts Reten- Achieve- Strategy Interventions Supply Demand Access tion ment Locate schools * Bring schools closer to communities closer to * Create culturally appropriate facilities, including . communities provision for separate toilets and water supply * Establish single-sex schools Promote hiring * Increase the supply of female teachers of female * Provide incentives . . teachers * Provide training locally Lower the costs * Provide scholarships toparents * Provide textbooks and uniforms . . . * Address the opportunity costs of girls' labor Develop * Render the curriculum more relevant relevant * Eliminate math and science gaps . curricula * Account for the future now Increase * Support communities that show interest community * Solicit support of community leaders participation * Involve parents in planning, management, . decisionmaking, and advocacy * Recruit teachers from the local community Promote * Empower communities with responsibility localization/ through local decentralization management mechanisms . * Formulate indicators to monitor progress * Establish greater links among levels of administration Promote * Develop comprehensive strategy advocacy and * Prepare action plan social * Use 'third channel' technologies . mobilization * Allocate sufficient resources for information dissemination Design * Prepare diagnostic studies systems that * Design flexible schedules accommodate * Provide instruction in discrete units the needs of female students Support * Encourage experimental schools multiple * Establish regional educational resource delivery centers systems * Establish stronger links between the different systems * Provide incentives to encourage participation in nonformal or nontraditional altematives Note: Third channel technologies are nonprint communications vehicles, including radio and television as well as traditional forms of communication, such as folk theater, village meetings, and festivals. Source: UNICEF (1992). 10 Before We Begin CAN YOU DO THE RESEARCH? You might wonder why you should look into some of these problems, and whether or not you will be able to. Increasingly, in recent years, experts have come to see the value of local expertise and experience as es- sential in the process of developing appropriate, relevant, and sustainable programs. People who are famil- iar with a country, a culture, a people are far more likely than international experts to be aware of local problems and opportunities, understand the entire network of cultural and societal norms that shape peo- ple's ideas and behavior, ask the right questions, be able to put the answers in context, and assess the mean- ingfulness and practicality of various strategies for accomplishing what people need and want. One of the ways to capitalize on local expertise is to equip people "on the ground" to do research. Eventually, your research may be added to the contributions made by the researchers mentioned in this chapter. A second and equally important consideration is that only through the development of national and lo- cal social research capacities can sufficient work be carried out to provide us with the broad knowledge base that is needed to address the problems we have been discussing. A third is that in most African countries, a lack of resources, the scarcity of trained personnel, and the urgency of the problems mean that nonresearchers often find themselves in the position of having to collect information, whether or not they call it research. They have to do this with little preparation or support, and little time to take from an already crowded work schedule. Finally, there is a larger issue. Historically, decisions about what information was important, who gath- ered it, and how it was used were determined by the powerful for the less powerful. Elites and people of higher social standing studied the poor, men studied women, the industrial world studied the developing world. Learning how to do basic social research enables people who were traditionally left out of this pro- cess to contribute and to bring different perspectives and insights to bear. THE HANDBOOK This handbook has been designed for people with little or no training in social research who are concerned about important issues related to girls' education in Africa. You may be a policymaker or an inspector of schools; someone working in a government department, an international agency, or a voluntary organiza- tion; you may belong to a community group; or you may be a trainer looking for a text that will help your students to do research. What you will learn will help you gather data to identify problems and possible interventions, assess resources for action, plan projects, and understand and evaluate the work of other researchers. The techniques are drawn from a wide variety of disciplines, but the spirit comes from anthropology, sociology, rapid rural appraisal, participatory research, and educational evaluation. The handbook empha- sizes * Understanding the basics of research * Getting valid, relevant information * Meeting relatively short time constraints * Using the principles of optimal ignorance (knowing what is not worth knowing) and proportionate accuracy (recognizing the degree of precision you need) (Chambers 1981, p. 95) * Recognizing that local people are participants in, not subjects of, the research process. At the beginning of most chapters, a box tells you what kinds of questions the approach or technique is good for. A published study that used this approach is given as an example. At the end of most chapters we look at how the chapter applies to a group of girls you will meet in chapter 4. Introduction I I There are many ways of doing research. The techniques and approaches presented in this manual were chosen based on certain assumptions, namely: * That you have expertise or experience in the field of education, administration, planning, or policy- making, but little or no research background * That you are particularly concerned with the cultural and human factors that shape people's behav- ior, beliefs, and attitudes toward girls' education * That while this understanding of cultural and human factors can provide insights beyond the group you are studying, your main aim is understanding at the local level, and that generally, you will not be using your group as a sample to understand a much larger group * That you will be working with relatively small numbers of people and will not be attempting re- gional or national surveys * That you have time and budget constraints and may have little or no assistance * That you may not have, or need, access to a computer. Even if you will never do research, you still have to understand the rationale, value, and limitations of other people's research, or you will forever be the victim of a kind of intellectual neocolonialism, at the whim of people who have power over you because they have a monopoly on skills and jargon. HOW TO USE THE HANDBOOK The chapters in the handbook can be read separately, but if you are carrying out a complete research project, as figure 1-3 shows, some chapters go together. Even though you do not need to understand chapter 2 to do research, it will help you to understand that there are different ways of looking at the world, and that these different ways lead to different kinds of research. Educational research in developing countries is an exciting and rapidly growing field that draws upon good ideas from existing disciplines, cooperation between disciplines, and new developments. It calls for recognition of practical constraints, openness to innovation, and learning from and sharing research expe- riences, good and bad. You are in an excellent position-you have some practical or policy experience of education, and you are coming into a field that values fresh viewpoints. 12 Before We Begin Figure 1-3. Overview of the Handbook's Chapters Read chapter 2 for background Get your ideas clear (chapter 3) Plan your research: Do you want to Describe something Explain something using r an experiment or squrvey (chapter 4) (chapter 5) Decide whom you will study (chapter 6) Choose your techniques (chapter 7) Read uaelut Interviews Pr Sues Measures, scaes, Experiments maatroacaesand indices | (chapter 8) (chapter 1|1) a r ( e 1 ( Do you want to invove local people as partners in research\ (chapter 14) Record and organize (chapter 15) Analyze the resuns Analyze the results (chapter 16) (chapter 17) Present your results (chapter 18) Introduction 13 FURTHER READINGS TO HELP YOU In this chapter and the ones that follow, you will see a number of citations in the text, for example, (Bam- berger 1994). You can find out more about the work by looking in one of two places: at the end of the chap- ter, where you will find a section called "Further Readings to Help You," or at the end of the book under "References." The "Further Readings to Help You" sections contain books and articles of general interest, which will take you one step beyond what is presented in the chapters. The "References" section contains the other books and articles that I used to write the book. Bellew, Rosemary T., and Elizabeth M. King. 1993. "Educating Women: Lessons From Experience." In Eliz- abeth M. King and M. Anne Hill, eds., Women's Education in Developing Countries. Washington, D.C.: World Bank. Floro, Maria, and Joyce M. Wolf. 1990. The Economic and Social Impacts of Girls' Primary Education in Develop- ing Countries. Prepared for the Advancing Basic Education and Literacy (ABEL) Project and the U.S. Agency for International Development (USAID). Washington, D.C.: USAID. Hartnett, Teresa, and Ward Heneveld. 1993. Statistical Indicators of Female Participation in Education in Sub- Saharan Africa. Africa Technical Department, Human Resources Division, Technical Note No. 7. Washington, D.C.: World Bank. Herz, B., K. Subbarao, M. Habib, and L. Raney. 1991. Letting Girls Learn: Promising Approaches in Primary and Secondary Education. Discussion Paper No. 133. Washington, D.C.: World Bank. King, Elizabeth M., and M. Anne Hill, eds. 1993. Women's Education in Developing Countries: Barriers, Benefits and Policies. Baltimore, Maryland: The Johns Hopkins University Press. Namuddu, Katherine. 1992. "Constraints to the Achievement of African Women." Paper prepared for the Conference on Women's Human Capital and Development, sponsored by the Rockefeller Founda- tion, Bellagio, Italy, May 18-22, 1992. Odaga, Adhiambo, and Ward Heneveld. 1995. Girls and Schools in Sub-Saharan Africa: From Analysis to Ac- tion. Africa Technical Discussion Paper. Washington, D.C.: World Bank. Subbarao, Kalanidhi, and Laura Raney. 1992. Social Gainsfrom Female Education: A Cross-National Study. Pol- icy Research Working Papers No. WPS 1045. Washington, D.C.: World Bank. Tietjen, K. 1991. Educating Girls: Strategies to Increase Access, Persistence and Achievement. Washington, D.C. Creative Associates Intemational. UNICEF (United Nations Children's Fund). 1992. Strategies to Promote Girls' Education: Policies and Programs That Work. New York. _.____ 1995. The State of the World's Children. Oxford, U.K.: Oxford University Press. . ..0 a 2 *a Social Research: The Debate Today s u m m a r y Social and culturalfactors can be crucial to understanding a situation, yet many people are reluctant to ex- amine these factors. They are concerned that their research will not be "scientific. " Misconceptions about sci- ence are common. Many people have outdated views of what science is. Paradigms are patterns or modelsfor understanding. Science is based upon one paradigm, postpositivism, which has changed considerably in recent years, but other paradigms also exist. Three explored in this chapter are * Positivismlpostpositivism-the basis of traditional science. There is an objective world. Nature can be understood much as we understand a machine, reality exists, the observer can stand apart from na- ture, the world is orderly and predictable. The aim of research is to discover causes and effects. * Phenomenology-there is no such thing as objectivity, but many, changing perspectives on reality. Getting the whole picture, the context and the meanings, are what is important in doing research, in oth- er words, understanding. The observer is part of the picture, not separate from it. * Critical theory-there is an objective world, the observer can stand apart from it, but whose world is it? Elites have had charge of the research agenda, setting the questions,funding the research, interpret- ing the results. The purpose of research is to empower people to set their own action agendas. Is social science a science? Modern science is moving closer to some of the perspectives of the social sciences. Many of the factors affecting girls' education mentioned in chapter 1 are social factors. The term social is broad, and includes the economic, political, legal, institutional, organizational, community, and cultural considerations that affect girls' participation in education. You may not be as comfortable about collecting information on social factors as you might be about col- lecting other data, perhaps material in your own field, such as "straightforward" figures or technical infor- mation. Perhaps you are suspicious of so-called soft social data. Even if you are convinced that such information is useful, you might be worried about your ability to collect it or to arrange for it to be collected. This handbook will help you deal with these concerns. For now, it is simply worth saying that social factors do matter in development projects. Michael Cernea, a sociologist at the World Bank, has pointed out something that most development practitioners have known in their bones for a long time: in a project, the factors that project designers often overlook are the 15 l6 Before We Begin human and institutional ones: "If these are mishandled the project will fail, no matter what national or in- ternational agency promotes it" (Cernea 1985, p. 6). Conrad Kottak, an anthropologist, studied sixty-eight World Bank projects and found that attention to human issues in projects is not only humanitarian, but has also proven to be cost-effective, leading to "economic rates of return twice as high as those of the socially insensitive and inappropriate projects" (Kottak 1991, p. 432). Project designers, however, often do not take these factors into account. Why? Social scientists, who spe- cialize in collecting social data, do not necessarily direct their studies toward the immediate practical needs arnd constraints of policymakers, project designers, administrators, and the people in the "front lines," those who actually have to live with the problems. Often their research takes too long, produces too great a mass of information, does not spell out a program for action, and is written in highly specialized language. Development expert, Robert Chambers (1983) points out the predicament: "We have technically illiter- ate sociologists, on the one hand, and socially incompetent technical experts, on the other." Similarly, many technical experts working on practical problems of education have narrow, specialized training with little emphasis on people factors. Eventually, hard experience teaches most practitioners that these factors matter. You may already have found this to be true, but how do you take them into account? Is this stuff you are being asked to study "sci- entific?" How many people are there in a sample? How will you know when people are telling you the truth? Aren't numbers more reliable? Will professional researchers scorn your results? Doesn't research take two or three years? How will you hold down your job at the same time? These are all legitimate questions, and faced with them, you may have developed your own system for finding out what people need and want. The most common system is the DeValera model, so-called after a prime mninister of Ireland, who said "When I want to know what the people of Ireland think, I only have to look into my own heart." "Old hands" often use this approach. It certainly saves a lot of time on the road, but has no other useful function. Another system development experts often use is what Chambers (1983) calls development tourism, that is, making brief trips, usually to convenient areas in the dry season, talking to easily accessible people, and producing "quick and dirty" unreliable results. Until recently, the alterna- tive was the "long and dirty" approach, seen in some Ph.D. theses, and huge surveys that take years and may produce material that is not only irrelevant, but costly and badly out of date. There is, however, a middle ground that builds on the fact that you know more about social research than you think you do, and on the fact that local people you are working with can help you more than you may have thought. Another helpful factor is that in recent years, other people facing the same problems as you have been cooperating across a wide variety of fields and disciplines to develop research methods that produce timely, valid, relevant, useful, and cost-effective information. After you have learned some basic research techniques, chapter 14 will look at some of these special approaches. In the rest of this chapter, we will be looking at some idea systems that have had major consequences for how and why research is done. These ideas are included here because they give us some insight into the newer approaches to research and development. They will help those people who are still worried about how "scientific" social research is. They are also important for people who suspect that there are more per- spectives to a situation than Euro-American ones, "official" ones, or those of the most powerful people or nations. Even if you are not worried about any of these issues, it does help to know that today we are in the mid- dle of the Second Scientific Revolution. When caught in a revolution, having some notion of what is going on is always a good idea. One of the nice things about studying a book on your own is that you do not have to follow the author's plan. The rest of this chapter is theoretical; it contains no practical instruction on how to do research. But perhaps you have an urgent practical problem: you cannot get parents to enroll their girls in the first year Social Research: The Debate Today 17 of school, and the school year is about to begin. Or perhaps some hotshot researcher is going to show up in your office next week and you want to be able to ask an intelligent question and understand the answer. ("May I see your research design?" is always a good question, because many people do not have one.) If you find yourself in these circumstances, you are excused from the rest of this chapter, but come back to it when you have a few moments to relax and put your feet up. It will help you to understand why you are doing what you are doing. HOW DO WE KNOW ABOUT THE WORLD? For those who are still with us, here is a A for persevering. Although most experts now recognize the importance of social factors in understanding development- related issues, some people in development and education circles still have niggling suspicions about the social sciences: are they scientific? Their reasoning is something like this: Human factors are too broad and too vague Therefore, they can't be measured Therefore, they aren't quantifiable Therefore, you have no numbers Therefore, you have no hard data Therefore, human factors aten't scientific and you couldn't plan a project based on them Today, however, all thinking about the way we understand the world, including science itself, is undergo- ing a profound change. Traditionally, two techniques have dominated development-related social research: surveys, including censuses, which have been the most commonly used; and participant observation, or watching what people do and asking them about it. Sociologists, economists, educators, agricultural scientists, and health work- ers, among others, are the most frequent users of surveys. Anthropologists are more likely to use participant observation. Surveys are often described as quantitative, that is, they produce results in number form, while participant observation is qualitative, and produces results in words. More recent approaches, such as rap- id assessment techniques and participatory approaches (see chapter 14) often combine both, with a heavier emphasis on the qualitative. But this quantitative-qualitative distinction is a somewhat simplistic, short- hand way of summarizing a much more fundamental debate. 18 Before We Begin The Debate What is reality? Does a single, absolute reality exist, or are there many, changing perspectives on reality? Can reality be known at all? How? Some of these questions are the kind that creep up on people in the mid- dle of a sleepless night. Other people think about these things in the daytime as well, for example, people who are avoiding a par- ticularly boring task, but professionally, it is the work of specialists in the philosophy of knowledge, who ask "What is the nature of reality?" "What is the nature of human knowledge?" "How do we go about finding knowledge?" Philosophers have tried to answer these questions for at least 2,000 years, and the debate con- tinues today. Unfortunately, the literature on these subjects is about as user-friendly as a tangle of wire coat hangers, as you will probably guess from some of the words that come up later in the chapter. It is important for beginning researchers to understand the different answers to these questions, because they lead to different ways of doing research and different ideas about what is acceptable evidence. As you do your research, the choices you will make about what to study and how to study it, even those choices you think are "only common sense" or "natural," will probably be based on one of the approaches we will be discussing. You should be familiar with the features, strengths, and weaknesses of each approach. You should also realize that there is no single way to acquire knowledge. There are many paths to knowing things, and this discussion presents four of them. The answers that philosophers have offered to these questions form sets of assumptions called para- digms. A paradigm is a set of underlying beliefs about the way things are. For example, the way European educators think and have thought about children's natures constitute different paradigms: in the eighteenth century, they often thought of children as small adults, born with a faulty human nature. A proper educa- tion required vigilance and punishment to curb their "naturally" bad tendencies. In the late eighteenth and early nineteenth centuries, the influence of the European Romantic school of philosophers led to a belief that childhood represented a pure, uncorrupted human state, so teaching practice reflected that, and chil- dren's capabilities were allowed to develop "naturally." Each had a different notion of what was "natural." Paradigms are neither rational nor irrational-they are nonrational-they fall outside the boundaries of rationality. They cannot be proven one way or the other, and they tend to be self-confirming. According to the first paradigm, if children did not turn out the way one wanted them to, they had not been molded and pun- ished enough; according to the second, their original natures had been corrupted by an interfering society. The approaches discussed in the rest of this chapter are paradigms about how to understand the world we live in. None can be proved or disproved as right or wrong. PARADIGMS 1 AND 2: POSITIVISM AND POSTPOSITIVISM In research, one paradigm has been dominant for more than 300 hundred years: positivism, and its succes- sor, postpositivism. For many of us it is the only one we know, because it forms the basis of modern science. Among researchers pure positivism-a belief that only observable, objective, provable, hard facts count- was fading even in the 1950s, but generally, the person in the street is still a hide-bound positivist. Are you one of those people? Hold on to your hat. Positivism is based on the assumption that reality exists: it is "out there." Nature is like an orderly machine, a great clock that operates according to unchanging universal laws. We can find out the way things are, what is really happening. We can unlock the secrets of nature if we ask the right questions, just as we can understand the workings of a clock if we take it apart and study it carefully. The "facts" we discover are true if they corre- spond to "objective reality." The relationship between the researcher and nature is dualistic. The researcher does not have to be part of nature. The researcher can stand apart from nature and observe it objectively. Social Research: The Debate Today 19 In this paradigm, the world is stable, consistent, predictable, and orderly. Things occur ina single-line order of cause and effect: A causes B. Causes in both the natural and social worlds can be studied in the same way, through experimental procedures. The researcher singles out and examines the relationships between vari- ables, or separate aspects of reality, just as one might take apart and examine the relationships between parts of a clock. This is called deduction: determining in advance what is important to study and selecting only those elements for examination, for example, only looking at certain parts of the clock. The examina- tion is done through empirical (based on the senses) observation and tests, such as experiments or statistical analysis. Because measurement is standardized, that is, carried out in the same way each time, using the same instruments, such as questionnaires or tests, on all the people in the study, one can repeat the same procedures with other groups and make comparisons. In positivistic research, much time and effort goes into developing the instrument. If it is a questionnaire, the questions are carefully thought out to make sure they address the variables that interest the researcher, the questions are pretested on people to see if they produce the required information, and the people administering the questionnaire are trained in exactly what to say and do. Much depends on the instrument. With other paradigms, as you will see, this is not as important. If our research can identify laws of cause and effect and the conditions under which they apply, we can explain past events and predict future ones provided, of course, that our explanation is correct and the con- ditions are the same. Our study has great breadth. Given that reality is absolute and nature is orderly, our findings will be true anywhere in which the same circumstances exist. However, this can present a problem. Because positivistic research is often carried out under experimen- tal conditions rather than in natural settings, it is unlikely that you will come across the same conditions except in a similar experimental setting. Real life, of course, is not an experimental setting. As a result, we say that our results have high internal validity (are valid in that setting), but not necessarily high external validity (valid in other settings), because the experimental setting is artificial. If you give children a treat before an examination and find they do better than children who did not get a treat, you might conclude that treats improve examination success. You can carefully control the situation so that the children in both groups have exactly the same amount of lighting, air, noise, study conditions, and so on so that you are sat- isfied that it is the treat that made the difference, but in real life will the treat have the same effect on chil- dren whose surroundings will not be as carefully controlled? Will other kinds of treats work too? Your experiment does not answer these questions. The research techniques used in the positivistic approach are usually, but not always, quantitative, be- cause the variables are clearly defined, can be measured, and the results can be converted to numbers. You might define "being a good student" this way: "achieving an average of 80 percent on examinations," "reg- ular attendance at school, that is, not missing more than one day a month," and "frequent participation in school activities, namely, membership in at least two clubs." Using these definitions, measuring the stu- dents and counting the good ones will be easy. The term "quantitative" is often used to refer to the entire paradigm we have been discussing, when, in reality, it is simply a description of the most common way of handling the data. If you can isolate what you consider to be the causes and effects of something, positivistic strategies are good for finding out why something has happened, or for predicting that under certain circumstances something will happen. Perhaps you can begin to see that this kind of prediction is not the same as proph- ecy. Science cannot predict the future except in very limited circumstances. Some of the strengths of the positivistic approach, such as breadth of coverage and the possibility of making comparisons, are also the source of its weaknesses. For example, information in many international agency reports is collected and grouped in such as way that it permits cross-country comparisons, like the information in table 2-1, which is a United Nations Children's Fund (UNICEF) publication. 20 Before We Begin Table 2-1. Social Indicators in Selected African Countries, Various Years Total Per fertility Unde Life Gross enrollment Adult literacy rate, capita rate, five epeca ratio, 1988 Gender gap, 1988 GNP, 1988 mortality at birth, (percent) 1988 (percent) 1987 (per 1,000 rate, 1988 Country Male Female (percent) Male Female (US$) live births) 1988 (years) Lesotho 102 123 -21 62 84 370 5.8 136 56 Togo 124 78 46 53 28 290 6.1 153 53 Chad 73 29 44 40 11 150 5.9 223 46 Central African Republic 83 51 32 53 29 330 5.9 223 46 Source: UNICEF (1992, pp. 76-79). Table 2-1 helps us to look at similarities and differences between countries and to test ideas quickly. In the introduction to this book we said that countries that have a big gender gap between boys' and girls' ed- ucation also have worse social indicators. Tables organized like the one above allow us to make such asso- ciations. Does a country have a big gender gap in education? Does it have higher fertility rates or lower literacy rates? Such tables can allow you to check quickly and to see how your country is doing in compar- ison to other countries. The disadvantage is not only the obvious fact that systems, such as educational levels, may not corre- spond or that records are not kept the same way in each country. For example, in some countries literacy may be defined as the ability to sign a marriage license; in others, it is based on performance on a test. Some- times literacy assessments are based on self-assessment during a national census, but in other cases national authorities simply provide national estimates. It may also be the case that people with no schooling are clas- sified as illiterate, and in other cases, some part of the population, such as nomads, may be excluded. Who is considered an adult also varies. Sometimes the term refers to people as young as nine; in other cases it refers to those fifteen and older (Greaney 1994, p. 222). Reports such as the World Bank's World Development Reports will tell you what is meant by each term used in the report, but that does not necessarily mean that countries supplying information for the reports have always collected the information according to such definitions. A more fundamental disadvantage when you are dealing with social information is that people in different countries or cultural groups may define concepts like "health," or "nutrition," or "employ- ment," or "households" in different ways, making comparisons difficult, and sometimes meaningless. Among your own people, for example, think of what the phrase "a good daughter" means. Now think of another cultural group that you know: what do they think of when they think of a good daughter? Is group- ing the two ideas together easy? Another problem is that the categories chosen for study reflect the variables that the researcher consid- ers important. Other variables and definitions of variables are ruled out because they are not being looked at. For example, you have a theory that women's health and welfare rise with their level of education. Let's say you define health and welfare under the variables "life expectancy at birth," "births attended by health staff," "maternal mortality," and "infant mortality." Access to contraceptive measures, caloric intake, work hours, and fertility may also be relevant, but your definition has ruled them out. You can see, therefore, that you need to know a lot about a situation before choosing the variables to study and defining what you mean by them, and if you are trying to use them on different groups of people, they should have some meaning for those groups. Eventually, however, you must choose so that you can decide what to look at Social Research: The Debate Today 21 and what to ignore, and also so that you can make clear to others, perhaps people who are working with you on your research or those who will read your study, what exactly you were looking at. The paradigm we have been discussing is the one that most people still think of when they think of sci- ence, but for some time now, scientists have recognized a number of problems with it, and a modified ver- sion, postpositivism, has emerged. The postpositivist position is that although a real world does exist, imperfect humans cannot fully comprehend it. Gone also is the idea of the independent, value-212free ob- server, because observers cannot stand apart from nature. In fact, the entire research process is shaped by the interaction between the observer and the observed. How do we get around this problem? Postpositivists say we must not give up the struggle for objectivity. We must still aim for it despite these messy "human" problems. To do this, observers must be honest and forthcoming about their methods and assumptions, and other scholars must carefully scrutinize their research. Also, we should take a more thorough research approach. We should use as many researchers, methods, theories, and sources of information as possible so as to strengthen the findings. We should use natural as well as experimental settings, and qualitative as well as quantitative methods. Most people's misconceptions about science and research arise from the positivistic paradigms. For ex- ample, we tend to believe that when we give a single name or label to what is actually a culturally defined bundle of behaviors or ideas, like "development," or "intelligence," or "personality," it therefore really ex- ists in that form and can be measured. We assume that tests measure what they claim to measure, that the personality test you took in a newspaper really does measure your personality. Once measured, the results can be converted to numbers. Anything in numbers is seen as more scientific than anything in words, and because we also confuse precision (exactness) with validity (soundness), anything in extremely precise numbers, like 8.243, is better yet. We believe that using instruments is more scientific than using a human (perhaps even the human who designed the instrument). Instruments have the added advantage of putting distance between the observed and the observer, thus guaranteeing "objectivity." But as one education ex- pert points out, all that it really guarantees is distance (Patton 1990). A questionnaire, therefore, is more "ob- jective" and "scientific" than an interview in which exactly the same questions are asked. Unconsciously, some people also think that the more a body of material is processed, the more scientific it is: a sample is drawn, a questionnaire given, the answers are checked off in boxes, the results are coded in numbers, the numbers are fed into a computer, and the computer results are grouped and statistically analyzed. Thus, our vague "yes" to a badly-conceived question passes into the system and comes out with the weight of 300 years of science behind it. Finally, the source of material is important: some people think information from officials, such as government ministers or project directors, is more objective; information from local people is impressionistic; and the researcher who argues for the value of the local perspective is accused of going native. PARADIGM 3: PHENOMENOLOGY The phenomenological paradigm argues that positivism may give facts, but it does not give meaning or understanding.' Phenomenology denies the existence of an objective reality: what is important is reality as people perceive, experience, and interpret it. If you have ever been involved in a large family dispute, you may have seen that each person saw things in a different way, and that each person may have been "right." So phenomenologists believe in multiple realities, each of which is a social construc- tion of the human mind. There is no bare, absolute reality. People use models-cultural, historical, group, and individual-to organize, interpret, and reconstruct their perceptions. That is reality. Knowl- 1. The phenomenological paradigm is sometimes called, with some variation in meaning and emphasis, the herme- neutic, relativist, constructivist, interpretivist, or emic approach. 22 Before We Begin edge is the result of an interaction between people or, in the case of the physical sciences, between peo- ple and objects. Facts are created in this process, not discovered. The researcher is part of this construction, not independent of it. The researcher's task is to interpret or make sense of these con- structions rather than to predict. The perspective of the phenomenologist is holistic: the whole of the situation, individual, organization, or project is greater than its parts. Phenomena can only be understood in this total context, not through the neat variables that the positivistic researcher has selected for study. Phenomenologists take a naturalistic approach, rather than an experimental one: researchers study the actual situation as it develops naturally. They do not attempt to identify the variables before the research begins or to manipulate the research setting; the variables emerge or unfold from the research like the fea- tures of an ancient city emerge under the archaeologist's brush. This is called induction. Unlike an archae- ological site, however, the situation is seen as dynamic and changing. Descriptive data, which we might get from participation, from observation, and from interviews, allow us to get an insight into the complex web of interaction, which may not be understandable in a simple, single-line, cause and effect way. People who take this approach argue that theories and facts are not objective, but are value laden from the start. The very act of choosing one variable to study rather than another and the definition of that vari- able reflects values: for example, investigators often study the causes of homosexuality; they do not study the causes of heterosexuality Judd, Smith, and Kidder 1991, p. 5). We also have a lot of research on what to teach children, but very little on what they might teach us. The techniques phenomenologists use are usually, but not always, qualitative, such as interviews and participant observation. The validity of their results is based on the skill of the researcher rather than on the careful development and administration of the instruments. Indeed, the researcher is the instrument. Phe- nomenologists analyze their material through content analysis and case studies. The advantage of the phenomenological approach is that one comes to understand the context of the sit- uation and appreciate the complexity and interdependence of the various aspects of the situation. Studies are thought to have greater depth. For example, in the field of participatory learning approaches (see chap- ter 14), well-being is a commonly used technique: local people, often drawn from a variety of occupa- tions and statuses and from both sexes, are asked, either individually or in groups, to rank all the people in the community by well-being. They sort a pack of cards with the name of each person in the community on a card and create several piles. These piles reflect shared local conceptions of well-being. Perhaps in one community well-being means cash income or ownership of animals, while in another it means the presence of a male in the house, the ownership of technology, the ability to get credit, a high worker-dependents ra- tio, or having a relative working for the government. Obviously, using this method will give you a good picture of local ideas, but another community might have very different ideas about well-being, so you cannot assume that your findings will apply to it. Com- paring people's level of education by their well-being across a number of groups using this approach would be difficult if each has a different idea of well-being. If you used a positivistic approach, however, you could make comparisons by developing one set of common categories such as having metal roofs, telephones, and plenty of daughters. The problem is that they might not mean much to some (or maybe all) of the groups involved. Nomadic herders, for example, would not see well-being this way. Phenomenological studies can take a long time (though in fairness, designing, administering, and ana- lyzing a large survey can also take a long time). If you are looking for funding for your research, you should know that many sponsoring agencies and funding bodies will expect you to provide a clear-cut statement of your research problem, to spell-out the variables to be studied, and to submit a budget and a timetable before the research begins. By now, you should be able to see that phenomenological approaches do not easily fit into this pattern (for advice on this problem, see Dobbert 1990). Social Research: The Debate Today 23 PARADIGM 4: CRmCAL THEORY A final paradigm, critical theory or ideologically-oriented inquiry (Guba 1990, pp. 23-25) shares with post- positivism the idea of an imperfectly perceived, but still quite definite, reality "out there." But the important question asked is: "Whose reality are we talking about?" Because values shape every aspect of the inquiry- the paradigm, the questions selected, the researchers, the methods, the findings, the recommendations, and so on-whose values are being used? Critical theorists say that in most research projects, the values are those of an elite-whether it be males, capitalists, Westerners, or others-who choose the questions to be asked and use the results of the research to support their interests and to oppress others. By contrast, critical theorists interact with participants in the research, rather than manipulate them, to enter into a dialogue and create a "true consciousness" through which the oppressed might then transform their world. The researcher's intent is not simply to collect information and work with people to get an un- derstanding of their current situation (what is); it is to encourage the group to go further and plan action for change. The participants do not just ask "what is?" They ask "what can be?" As two "militant" sociolo- gists, Roseca and Miguel Darcy de Oliveira say, it is the researcher's task to show people that "today's re- ality is not the only possible reality" (1981, p. 47). The strengths of this approach (or rather, these approaches, because feminism, neo-Marxism, and some of the participatory approaches fall under this heading) are that historically unquestioned biases are iden- tified and exposed. Other perspectives and approaches are seen as valid. A weakness of many critical the- orists is that they assume that it is always other people who need consciousness raising, empowering, and liberation. They do not ask the people they study to help them become more liberated. For years, for exam- ple, Westerners have been concerned about the conditions of women in "less developed" societies. They also assumed that in the process of improving women's lives, their status might be raised to that of Western women. But having an easier, healthier, more productive life and status are not necessarily the same thing, as social scientist Martin King White (1980) has shown. In a study of ninety-three societies, he found that the higher the level of complexity-technological, political, economic-in a society, the lower the status of women relative to that of men. Western women do not necessarily have higher status, even though they live in more complex societies. So the lesson is that we all need our consciousness raised. However, critical theorists have managed to make us look at the world from perspectives other than that of the establishment. They have questioned the basis of research ideas and means and ends, arguing that historically they have marginalized the less advantaged. This can be seen clearly in various feminist argu- ments, for example, which show how male-centered research selectively limits or ignores women's roles and experiences (see, for instance, Harding 1987). A critical theory approach in African research on women can be seen in the work of Namuddu (1992) and Longwe (1986). Is Social Science a Science? Probably the only time you will really care about whether social science is a science is when someone who does not like the sound of your project or the results of your research attacks it as unscientific. Some re- searchers in the "hard" sciences, such as biology or chemistry, often dismiss the social sciences as not being sufficiently scientific. (Only relatively recently have social scientists charged the hard sciences with not be- ing sufficiently social.) As you become more familiar with the paradigms on which human inquiry is based, you will probably feel that this issue is not as clear-cut as some think. Earlier in this chapter, we mentioned a Second Scientific Revolution. "Old" science, which has been based on the physics of the sixteenth and seventeenth centuries, argued that there were absolute, certain laws of cause and effect, neatly located in time and space, that the whole was best understood by breaking it up into parts and looking at those parts separately, and that the observer and the subject were separate. This was the objective way: anything else was subjective. Even philosophers, social theorists, and social 24 Before We Begin scientists, from Hobbes to Darwin to Freud, drew inspiration from this model. Auguste Comte (1798-1857) the father of sociology, even called his subject "social physics." However, a "new" science has emerged in the twentieth century that includes relativity theory, quantum mechanics, chaos theory, and complexity theory, all coming from the hard sciences. Although different, each shares a paradigm: that the whole is greater than the sum of its parts; that things are ever-changing, uncertain, and unpredictable, so that new forms are emerging all the time; that the universe is moving toward increasing complexity; and that ob- servers participate in what is being observed (they help to make the reality that is being studied). Does some of this sound familiar after reading this chapter? Some social scientists had been thinking along these lines for quite a while. You may have formed some views on these ideas now, but it is important not to fall into the trap of over- simplistic dualism. Some of the dualisms that you might be tempted to create from the paradigms discussed in this chapter are "Hard" sciences "Soft" sciences Experimental strategies Naturalistic strategies Deductive approaches Inductive approaches Quantitative techniques Qualitative techniques Even though four paradigms are presented here, the most common debate is between postpositivists and phenomenologists: postpositivists think phenomenological approaches are soft or subjective; phenom- enologists think there is no such thing as objectivity. Positivist-oriented purists group together, as natural strategies, the items in the left-hand box, and phenomenological-oriented purists, those in the right. But is it as clear-cut as that? Many researchers argue that while these approaches and techniques are mutually exclusive as ideas, they do not have to be exclusive in use. That is, in a particular research project you can use each of these paradigms and the techniques associated with them, sometimes to support each other, sometimes as sepa- rate stages in a project, getting both explanation (why?) and understanding (how?). Educator Michael Quinn Patton (1990) describes this as a "paradigm of choices." Other researchers reject this. Egon Guba and Yvonna Lincoln (1989, p. 11), supporters of a phenomeno- logical approach, argue that paradigms cannot be mixed or used together because their underlying assump- tions are not just opposite, or even complementary, but fundamentally different. In practice, however, many positivists seem to be warming to qualitative research, and soon will prob- ably be claiming that they invented it. By contrast, few phenomenologically-inclined researchers actually take a completely consistent phenomenological approach in their work. As for critical theorists, those of us who are involved in development-related research are all indebted to them for making us less self-satisfied about some of the perspectives that we once took for granted. But we are not going to solve these problems here. Indeed, as research experts Miles and Huberman (1984, p. 20) have said: "If the debate is unlikely to be resolved during your lifetime, it is probably best to get on with your work, clarifying for yourself and your readers in which camp you are nestled." Certainly, those who just want to know what people in their community think about the need for a preschool do not have to sort the matter out before going down the road with a questionnaire. But it is good to know that it Social Research: The Debate Today 25 was not God who said that questionnaires are the best way to get information. It was a decision that some people made, and that some others dispute. In that spirit, this book presents a variety of strategies for planning and carrying out your research, and shows how you might, in practice, mix them and use appropriate techniques. It is true that the strength of each approach lies its own internal logic and integrity, so mixing them prevents the full realization of their power, but otherwise, as one sociologist says, "Nothing bad happens if you do this" (Denzin 1989, p. 9). Your research won't explode. FURTHER READINGS TO HELP YOU Cernea, Michael M., ed. 1985. Putting People First, 1st ed. New York: Oxford University Press. 1991. Darcy de Oliveira, Rosica, and Miguel Darcy de Oliveira. 1981. "The Militant Observer: A Sociological Al- ternative." In Budd Hall, Arthur Gillette, and Rajesh Tandon, eds., Creating Knowledge: A Monopoly? Participatory Research on Development. Khanpur, India: Society for Participatory Research in Asia. Denzin, Norman K. 1989. Interpretive Interactionism. Newbury Park, California: Sage. Dobbert, Marion Lundy. 1990. "Discussion on Methodology." In Egon Guba, ed., The Paradigm Dialog. New- bury Park, California: Sage. Eisner, E.W. 1991. The Enlightened Eye: Qualitative Inquiry and the Enhancement of Educational Practice. New York: Macmillan. Guba, Egon G., ed. 1990. The Paradigm Dialog. Newbury Park, California: Sage. Guba, Egon G., and Yvonna S. Lincoln. 1989. Fourth Generation Evaluation. Newbury Park, California: Sage. Kottak, Conrad Phillip. 1991. "When People Don't Come First: Some Sociological Lessons from Completed Projects." In Michael M. Cernea, ed., Putting People First. New York: Oxford University Press. Miles, Matthew B., and A. Michael Huberman. 1994. Qualitative Data Analysis, 2nd ed. Thousand Oaks, Cal- ifornia: Sage. Patton, Michael Quinn. 1990. Qualitative Evaluation and Research Methods, 2nd ed. Newbury Park, California: Sage. Pitman, Mary Anne, and Joseph A. Maxwell. 1992. "Qualitative Approaches to Evaluation: Models and Methods." In Margaret D. Le Compte, Wendy L. Mellroy, and Judith Preissle, eds., The Handbook of Qualitative Research in Education. San Diego, Texas: Academic Press. Sechrest, Lee. 1993. "Roots: Back to Our First Generations." Evaluation Practice 13(1):1-7. UNICEF (United Nations Children's Fund). 1992. Strategies to Promote Girls' Education. New York. Part II Planning Your Research . aeOgOg 3 * - - What Do You Want to Study and Why? s u m m a r y All research requires planning. Some dimensions of research to consider are thefollowing: * Is it basic (knowledgefor its own sake) or applied (to be used toward a practical end)? * Is it descriptive (providing a picture), or is it explanatory or predictive (showing relationships, in- cluding cause and effect, between events in the past or thefuture)? * Are you asking "W'hat do I see these people doing?"(research from your perspective) or "What do these people see themselves doing?"(researchfrom the perspective of those participating in the study)? . Will your research be extractive (you determine the issues, choose the approaches, and carry out the re- search), or will it be participatory (local people help to identify the problems and concerns and partic- ipate in gathering, analyzing, and using tihe information)? * What will it cost? Just as you would plan an important journey before setting out, you must plan your research, or you will wander off the track, waste time and money, or abandon the project. Your research design is the overall plan that tells you what information you need to collect, what techniques to use, and where to get the in- formation. Whatever research paradigm, perspective, or approach you take, you must have a plan. It's not unusual for the planning and preparation stage to take up a third of the entire time you have allowed for your research. Box 3-1 shows the stages involved in carrying out your research. Generally, the remaining chapters in this book follow this order, and each will tell you where you are in the sequence. However, as the box shows, chapters 4 and 5 keep popping up. They each alert you to a number of important decisions you will have to make. Before you can make some of the decisions, however, you need to know more about your sources of information (chapters 6 and 8) and how you will get the information (chapters 7-14). Then you can take the decisions necessary to finish the design you started in chapter 4 or chapter 5. STEP 1: GET YOUR RESEARCH IDEA This chapter deals with the first three stages. Where do you get research ideas? Your own interests and ex- periences or those of others may suggest ideas to you. Perhaps you are simply wondering about something, 29 30 Planning Your Research Box 3-1. Research Stages Stage Chapter Step 1: Get your research idea Chapter 3 Step 2: Clarify the goals and purpose of your research Chapter 3 Step 3: Choose your perspective: etic or emic Chapter 3 Step 4: Develop a rough research idea and refine it * Nonexperimental research Chapter 4 * Experimental research Chapter 5 Step 5: Create a research statement or hypothesis * Nonexperimental research Chapter 4 * Experimental research Chapter 5 Step 6: Identify the subtopics for study * Nonexperimental research Chapter 4 * Experimental research Chapter 5 Step 7: Make decisions about sources of information Chapters 6, 8 Step 8: Make decisions about information gathering techniques Chapters 7-14 Step 9: Complete the research design * Nonexperimental research Chapter 4 * Experimental research Chapter 5 Step 10: Obtain the information Chapters 7-14 Step 11: Record and organize the information Chapter 15 Step 12: Analyze the results Chapters 16,17 Step 13: Present the results Chapter 18 for instance, why people in one village are always ready to try a new idea, while people in another are reluctant to make any changes. You may have general questions, say, what would it take to get parents to send their daughters to school? Why are girls doing less well than boys in school? A World Bank (1994) publication, Questions for the Analysis of Female Participation in Education in Sub-Saharan Africa, raises these and other kinds of questions. You might consult it for ideas, but you are more likely to be reading this book because you have come across an immediate problem that needs to be better under- stood or that requires some action: for example, parents in your area are worried about their daughters losing their traditional values if they attend school. What do they mean by this and what can be done about it? You can also get your ideas from research that other people have already done. The introduction to this book has already mentioned some ideas. The professional literature is the best source for getting these kinds of ideas; look at chapter 8 for some useful material. Professional journals are often written in specialized What Do You Want to Study and Why? 31 language and assume some background knowledge of the subject, and maybe of previous research done on the specific topic. Once you become familiar with the material, you might do your own research to see if * A theory being used in a professional article or book can be applied in a new setting, to different people, or to a new problem, or if a different or better theoretical explanation is available for what the author found * The author's findings are valid in your setting * The research techniques used in the study could be applied to a different problem, or other research techniques could usefully be applied to the same problem * Researchers with different characteristics, women as opposed to men, for example, obtain different results. A final possibility is that an organization, a ministry, a village, or a school has asked you to carry out a piece of research. In this case, you will have to work closely, at all stages of the research, with the people who commissioned it. You will not make some of the decisions about the steps at the beginning of this chapter by yourself. Recent research by the United Nations Children's Fund shows that the best predictor of a good study is active partnership by all the participants (Hursh-Cesar 1991, p. 24). Be sure you all agree on the focus, the approach, the methods, the costs, the time, the interim reporting to the sponsors, whether you will be required to make recommendations, who "owns" the raw data (preferably you), and what form the final presentation is to take (a talk? a report?). Make certain you understand the functions and powers of the people who commission the research or you could end up making recommendations that they are powerless to implement. Also watch out for some common problems in commissioned research, namely: * The research is not needs-driven. It is being commissioned because research is fashionable or pres- tigious, or because someone has some money on hand, or both. * The organization or group cannot or does not intend to do anything about the problem. Either it is not in their power, or they have already made up their minds. If you want your research to have an impact, get all of this clear first. STEP 2: CLARIFY THE GOALS AND PURPOSE OF YOUR RESEARCH Why you are doing your research, your aims, your perspective, and the role people who participate in your research will play all have a part in shaping your research. Basic and Applied Research What is your research going to be used for? The aim of basic research is to advance human knowledge with no particular application in mind. Usually basic research, which can be expensive, has no urgent public au- dience, and therefore no one rushing to fund it. It is usually done by academics, who hope to contribute to the development and refinement of theories. They or others may then apply these new insights to specific problems. Applied research, by contrast, usually addresses a practical problem. You may want to find out how to accomplish a goal or why something has happened. An educator might ask, "How can we make science classes more relevant to girls?" or "Why have children stopped using the local libraries?" Applied research can take a number of forms: for example, evaluation research is common in the world of education and in development programs and projects in developing countries. Evaluation research can 32 Planning Your Research be used to assess a problem or need or to select a program or a project, monitor it as it proceeds, and assess it afterwards. The meaning of some of these terms varies from one field to another: for example, in many development projects the term evaluation is reserved for assessing projects after they are completed. In ed- ucation it refers to research on any stage of the project once it is under way. There is a rich literature on eval- uation research. A good handbook for the researcher is Valadez and Bamberger (1994). Another type of applied research is action research, which is done to get enough information to solve a problem. It can be used for a range of activities, from simple troubleshooting within an organization or community to militant mobilization of a group of people. The people involved may be encouraged to de- termine the sources of their own problems or, if the researcher is committed to a particular ideology, such as feminism or Marxism, they may be encouraged to consider their problems within that framework. Ac- tion research often draws upon the people involved in the situation as active participants in the research process. Descriptive, Explanatory, and Predictive Research Research can describe, explain, or predict.1 Descriptive research includes exploratory research to get a picture of a situation, behavior, or attitudes before planning further research: community studies, needs assessment, organizational reviews, and gen- erally any research that presents a picture of a situation, place, activity, behavior, or event. The basic ques- tion you are asking is what: "What is happening?" "What has happened?" "What do people think?" For example, "What is the picture in relation to girls' education in my district?" "What went wrong with the new program for increasing girls' mathematics scores?" You are not trying to prove cause and effect, so your research will not be experimental. Explanatory research is research that shows relationships after the fact. "How did reduction of school fees affect enrollment?" "Why did the introduction of single-sex schools lower girls' mathematics scores?" Predictive research states what will happen or how it will happen before the event. You are asking why or how: "What effect will gender sensitization programs have on villagers' attitudes toward girls' educa- tion?" "Will attendance increase if we subsidize the cost of girls' school uniforms?" There are many ways of explaining something. Draper (1988) shows that explaining can involve making something clearer, giving reasons, showing why a conclusion has been drawn, or making a causal state- ment. For cause and effect studies you will probably test a hypothesis, or a statement of the relationships be- tween two or more things. You can do this in several ways: conducting an experiment, doing what anthro- pologist Russell Bernard calls "thought experiments," or using survey questions. If you do an experiment, your research design will be shaped by the answers to the questions in figure 5-2 in chapter 5. If you are using a survey to test your ideas, you need to look at chapter 4 to plan your questions, chapter 5 to under- stand the fundamentals of experiments, and chapter 9 to understand more about surveys. 1. It is easy to get confused about the use of these words. If you are interested in why people think the reduction of school fees affected enrollment (rather than what effect the reduction actually had on enrollment), you are describing a situation, not explaining what happened. If you want to show how something is being done, like a new gender sensitization program, you are also describing. Only when you are looking for cause and effect are you doing explanatory research. You might think that descriptive research has little value. Why not get the real causes and effects? Why bother with what people think about causes and effects? But people base their actions on what they think: for example, there is no evidence that boys have better cognitive abilities than girls, but many people still think that this, rather than girls' heavy work loads or gender-biased textbooks, is what causes lower achievement among girls. So when resources are scarce they send only their boys to school. Thus people's views of cause and effect have real consequences for girls' lives. What Do You Want to Study and Why? 33 Your research project may contain descriptive, explanatory, or predictive research, or all three. On a "glamour" scale, these three types would appear in reverse order: most researchers would prefer to be able to predict than to explain and to explain than to describe. Except in a few fields such as anthropology, where researchers describe cultures, or in evaluation studies, where researchers describe what is happening in a project compared to what was supposed to have happened, not many researchers are interested in "mere" description. Some people even argue that work that is purely descriptive is not research at all (see, for ex- ample, Leedy 1989, pp. 4-5), but this represents confusion between research and "science" (see chapter 2). Good descriptive research is fundamental to knowledge: you can carry out an elaborate piece of research to explain the causes or effects of something, only to find later that the situation you are explaining does not exist. "Everyone knows" that poorer people in your area send fewer of their children to school. Suppose you were to do a large study explaining the reasons why. The only problem, as someone might later show, is that perhaps poorer people in your area are actually sending more of their children to school. In the mean- time, however, your department has launched a major information campaign directed at poor parents. As a beginning researcher you may be tempted to try to carry out large-scale predictive or explanatory projects that make cause and effect statements, such as "female-headed households are the cause of poor school attendance." You are unlikely to be able to do this for two reasons. First, as you will see in chapter 5, statements at this level are unprovable. Second, you must meet rigid conditions when you are trying to prove cause and effect that can be extremely difficult to meet. But perhaps you can meet them. Consult chapter 5. STEP 3: CHOOSE YOUR PERSPECTIVE What viewpoint will shape your research? Your choice will have a practical impact on how you plan your research and the research techniques you choose. This is a complex subject. In this book for beginners we are going to reduce the questions of perspective to two: do you want to know 'Whaf Do I See These People Doing?" or 'Whaf Do These People See Themselv'es Doing?" For the moment, we are going to call these "my perspective" and "their perspective." In the professional literature these terms are often referred to as etic (g4t ic) and emic (e mic), respectively, and are drawn from anthropology.2 These two approaches are not ordered on a quantitative-qualitative principle. Each can, in theory, pro- vide both types of information, although emic research is less likely to produce material that would profit by quantitative analysis. Each is important, and they can both be used in the same project. "My perspec- tive," or the etic approach, is by far the most common, and probably the most familiar to you. "Their per- spective," or the emic approach, though less common, is a powerful way of understanding a situation, and is becoming increasingly important in development-related research. 2. In recent years emic research has begun to appear in both development research and educational research. It is also called, with some variation, ethnoscience, the New Ethnography, ethnomethodology, and componential analysis. In the mid-1950s, anthropologist Ward Goodenough (1956) borrowed the words etic and emic from linguistics: phoretics is the study of all the sounds used in speech evu;.ywhere; phoremics is the study of the sound categories recognized as meaningful by people who speak a particular language. (Your language may not assign any meaning to the click sound that the Xhosa people use, for example). Goodenough borrowed these concepts and applied them to culture: emic cat- egories are those that mean something to a group of people who share a culture, while etic categories are created by the researcher to get a common basis for comparing groups. A robust literature is available on what emic research can and cannot do (see, for example, Headland, Pike, and Harris 1990). 34 Planning Your Research "My Perspective": The Etic Approach In "my perspective," when you ask "What do I see these people doing?" you determine what is important in the way of information. You have decided that an objective reality exists. You determine the variables and create the questions or, in the case of experimental research, the conditions under which the variables are tested. People's experiences and behavior are forced into your question and answer categories or your experimental conditions. You probably recognize this as a positivistic approach. Questionnaires, censuses, structured interviews, measures and scales, experiments, and the World Bank's development reports are all based on this approach. If you know a lot about a situation, and if youtr respondents see and experience the zvorld the way you do, this approach has many strengths and advantages: you can issue a questionnaire or hold some interviews, put the answers into neat categories, and count the answers or run them through a computer. You can compare one program with another, one school with another, one country with another. Two specialists in girls' ed- ucation, Brock and Cammish (1991), created a questionnaire to find out about primary school children's at- titudes toward the value of education for boys and for girls. They used their questionnaire in six very different countries, and found that everywhere boys and girls had quite different notions about the value of girls getting an education at all, that boys wanted more children than girls, and that boys had a better understanding of how to achieve their career goals. One disadvantage, however, is that you might not ask the right questions. For example, suppose you ask children how often they performed certain activities, such as working in the fields or in gardens, mak- ing things, fetching water, cooking, and sweeping. You have decided on the categories, so the answers you get back have to fall into them. In parts of Africa where children spend a lot of time herding animals or sell- ing goods, those categories will not be reflected. When you use this approach, people will try to fit their knowledge and experiences (truthfully or un- truthfully, accurately or inaccurately) into the choices you offer them. If your respondents do not see the world the way you do, or if you have got the wrong angle, people have to rearrange their experiences and knowledge of life into your categories, and they will probably do it, because most people are pretty oblig- ing. They may even be intimidated by your status, or by something as petty as the fact that the questions are typed, into thinking that your way of looking at things is the "right" way. The trouble is that they then go on living and looking at things their way, while you are busily designing a project or preparing a report based upon your way. To the extent that you are aware of this, you may even conclude that your way is scientific and necessary if the project is to operate properly, and that their way is what caused their problem to begin with. A second disadvantage of this approach is that if you are trying to compare very different groups, you may not be able to develop questions that are equally relevant and meaningful to each group. To solve this, you might create separate questions for each group, but then you cannot compare the results because you have not asked the same questions. "Their Perspective": The Emic Approach In "their perspective," the participants tell you how they see things. You may have decided that no objective reality exists from which to create categories of inquiry, so the participants create the categories. This ap- proach is most useful when you know little about the subject, or when the people you are dealing with have very different idea systems. It is also useful when you are so familiar with a situation that you can't see the forest for the trees. It is particularly important, however, when you are concerned with relatively "invisi- ble" or less powerful groups, such as children, poor people, women, the homeless, and nontraditional fam- ily forms, because, as some critics would point out, many of the "established" categories of information were created, and are used by, the more visible, the more powerful, the richer, and men. What Do You Want to Study and Why? 35 When you use this approach, you are trying to discover the idea systems that shape people's behavior. You are looking for the equivalent of "rules of grammar" that direct their lives and decisionmaking. Because these idea systems are usually somewhat peculiar to a group-children in a Koranic school in rural Gambia; girl street traders in Accra-what you learn will be extremely useful in understanding their situation, but may not be transferable to another group of children in a Koranic school in Sudan, or even in urban Gambia, or to girl street traders in Lagos. Sometimes, however, it is the local situation you need to understand, rather than what you might learn from comparing these people with some other group. Figure 3-1 presents a picture of a particular school system from the point of view of the children in it. Note how different it would be from that of the teachers. Indeed, note how different it would be from your own local school. Figure 3-1. What Teachers Do in a Particular School System from the Children's Viewpoint Beat kids Keep you Smack kids in the face in the Push against wall book Have a paddle [make you Hit ids Hit with books study WHit with yardsticks Slam kids' heads down on desks Hand out yell assig_unent Bitch ~ Send kids to of fice s . Catch kids fighting Send kds to office = Catch kids Catch kids in the halls Snd kids to detention center Cachkds, smoking in the cans Itoiletsl Make whole class stay after [school] Catch ki X Pick kids out who misbehave W Act mean o Keep cigarettes in shirt pocket Pik .Make fun of kids Try to be Dress cool Pick on Pick kids out by ability cOO Crack dumb jokes kibds Won't help kids U Call kids stupid Give detention <: Lean on kids shoulder l Let touch drapes Make kids PUt nose on wall Let readhorally Cut down kids u Let be Let read orally ufl Assume kids are uilty pet e Let write on blackboards (3 Keep kids after school pe Let run errands all the time Z Tie kids to desk F- Let do Let put stuff on the bulletin Embarrass kids something board F- Shake kids 3 special Make kids sit in a certain seat Let tum off lights for movie Give extra assil ments - Let run projector Give setences tco ut toLet off assignments Talkn be nuce to Let run errands Talk a kids Let switch assignments whole lot Let off detention Run A.V Let off eas Let you sleep instead of smacking you laudiovisual] to wake you up equipment Give good grades Give tests Write a note to another teacher telling her you're staying for her Pile on the Don't yell work Call you by your first name Source: Davis (1972, pp. 110-11). In this school, the researcher did not set out to study conflict, but what emerged was a lot of conflict, not only between teachers and students, as you might have guessed from this figure, but also, in another part of the study, between the students themselves. Robert Serpell (1982), in a study of parents and teachers in Zambia, discovered that the aptitudes and skills measured on a standard IQ test completely missed some characteristics that rural Zambians thought essential to intelligence in a child. One was "sendability": an intelligent child could be sent on an errand and trusted to alter the instructions given depending on the circumstances he or she encountered. This is not a "baby" or "folk" word for a skill that has an "official" name-it includes a number of abilities that are examined on the IQ test, and some that are not on it at all. But local people evaluated their ciildren on these and other locally-important abilities, and then made decisions about children's potential for different kinds of education, employment, and so on. 36 Planning Your Research An extensive application of this approach can be comprehensive and time consuming, but it can also be used simply as one part of your research strategy. We will see how to plan such research later. Some of the techniques associated with the "their perspective" approach are emic interviews, cards sorts, and triads, discussed in chapters 4 and 15. COMBINING THE APPROACHES Most studies take an exclusively etic approach: researchers create the categories of inquiry. Few are exclu- sively emic, but some good examples of emic approaches to school studies are those by Davis (1972), Doyle (1972), and Parrott (1972). You may wonder if you can combine the approaches when doing a study. The answer is yes, you can. Here are two examples. You could study a school dav according to etic categories that you had created: teacher-student interaction, reading skills development, recreational activities, civic training, and so on, looking at how much of the day is devoted to each, and describing each. You might then use an emic approach to see how the participants, such as teachers, students, and parents view the day (each group will be different). The teachers might categorize the day as you did (that doesn't make it more "cor- rect"). The students might break the activities up into "work," "fun," and "arithnmetic." The parents, partic- ularly i" they do rot appreve of the curriculum or .he teacher, mightt break it up into "proper lessons," "that new sex education," and "doing errands the teacher should do herself." You could also use the two approaches in a sequence. You could use an etic approach to get a brief de- scription of the community: its geography, its history, and its groups: social, religious, ethnic, economic, and political. When you write your study, this will provide the reader with a background picture. (Remem- ber, the community might not have chosen these dimensions. The most important one to them might be "those who are descended from slaves" versus "those who are descended from free people." A good re- searcher will find this out and include it.) Then you might take an emic approach to the central focus of your study. Perhaps you want to know what girls do during the day. Instead of creating the questions "How much time do you spend in leisure activities? How much in study?" say "Tell me about your day," and let the answers create the next question. Perhaps girls consider "study" as a leisure activity. You will lose that perspective if you ask the questions you created. Chapter 11 shows you how to carry out this kind of research. PERSPECTIVE AND PURPOSE If you are doing descriptive research, you can choose either perspective or combine them. If you are doing explanatory or predictive research, you are likely to ask "What do I see these people doing?" There are two reasons for this. First, in proper cause and effect research you have to be able to spell out what you think is the cause and what is the effect before you begin the research. You cannot let the categories emerge as you go along. Second, because you probably would like your results to apply to other situations or groups and not just the ones you studied, you need to use categories that will apply to many situations or groups, rather than ones that represent only one group's unique perspective. This should not stop you from doing some preliminary research to see whether you can devise categories that not only suit your experiment, but also have locally-appropriate meanings. We will explore that later. Finally, you should consider the stance you want to take in relation to the people involved. Will your research be extractive, that is, will you make all the decisions and simply extract the information from peo- ple? Or will it be participatory, that is, will the participants be partners in the research, deciding the agenda, the issues, and concerns to be looked at; participating in getting the information; and working with you to interpret the results? This is not a question of whose perspective, but rather of the role the participants take in research decisionmaking and activities. You will not be able to answer this particular question until you know more about participatory research, but it is an important one. An entire chapter (14) is devoted to rap- id and participatory approaches. What Do You Want to Study and Why? 37 Once you have answered these questions about purpose and perspective, we can begin to develop the research plan. As you are developing it, remember that an important part of planning involves budgeting. For example, because of mistakes in planning this book, for which the author by custom takes credit, it has cost about three times what it might have in money, time, and irritation. Try to avoid this. Even if you, like me, are working only for the good of humankind, you still may have to pay assistants and cover the costs of transport, food, and accommodation for yourself and others; writing materials; paper and printing for surveys; stationery and postage; books and reports; equipment such as tape recorders; translators; work- shops; entertainment and/or gifts where appropriate; photocopying; computer analysis; and preparing and circulating materials, including your final presentation. You may have to return to a community on a number of occasions, and you may find it a good idea to bring community members with you to a rniristry or nongovernmental organization when presenting final results. Try to estimate these costs as carefully as possible. Research need not be expensive, but many projects have come to a standstill because the researcher ran out of money. Where do you get this money if you need it? Usually you have to prove yourself before an agency or organization will give you any significant assistance. Start with a modest but useful project and see it through. Then you may find that a local organization will give you some form of help, such as a contribu- tion toward transport. "Capacity building" is a popular phrase in development circles these days, and once you can show that you are able to do research, particularly participatory research, at the local level and that you can help others te le.-. bu-v to do it, agencies may be prepared to work with you. In each of these in- stances you need to be able to show clearly what you propose to do, why, and how. You should also be clear, if you look for financial assistance, what obligations are placed on you, what level of control the spon- sor has, and whether this is acceptable to you. FURTHER READINGS TO HELP YOU Bernard, Russell. 1994. Research Metlhods in Cultural Anthropology, revised. Newbury Park, California: Sage. Davis, Janet. 1972. "Teachers, Kids and Conflict: Ethnography of a Junior High School." James P. Spradley and David W. McCurdy, eds., The Ciultural Experience: Ethnography in Complex Society. Chicago: Sci- ence Research Associates. Doyle, Jean. 1972, "Helpers, Officers and Lunchers: Ethnography of a Third Grade Class." James P. Sprad- ley and David W. McCurdy, eds., The Cultural Experience: Ethnography in Complex Society. Chicago: Science Research Associates. Draper, S.W. 1988. "What's Going on in Everyday Explanation?" In C. Antaki, ed., Analyzing Everyday Ex- planiation: A Casebook of Methods. Newbury Park, California: Sage. Hursh-Cesar, Gerald. 1991. "Eight Ways to Make Communication Evaluation More Useful." Development Communication Report 72:24. Parrott, Sue. 1972. "Games Children Play: Ethnography of a Second-Grade Recess." James P. Spradley and David W. McCurdy, eds., The Ctultural Experience: Ethnography in Complex Society. Chicago: Science Research Associates. Serpell, R. 1982. "Measures of Perception, Skills and Intelligence: The Growth of a New Perspective on Chil- dren in a Third World Country." In W.W. Hartrup, ed., Review of Child Development Research, vol. 6. Chicago: University of Chicago Press. World Bank. 1994. Questionsfor the Analysis of Fenmale Participation in Education in Sub-Saharan Africa. Wash- ington, D.C. What a deacriptive approach In good far gettng a picture of a situation Examples: Who's In school? What facilities are available? When did the program stan? Where are the most serdous problems? Sample study: Anderson-Levitt, Kathryn, Mardanne Block, and Aminata Soumare. 1994. Insdd Cbauwno In Guinea: Girs' Experences. Washington, D.C.: World Bank. ...4... .. 4 Planning Research that Describes s u m m a r y There are many ways of planning descriptive research. This chapter looks at three: the research outline and the gridfor etic approaches, and grand tour questions and card sortsfor emic research. It covers stepsfour through eight in box 3-1. Let us look at two ways of planning descriptive research, one asking, "What do I see these people doing?" (the etic perspective, discussed in chapter 3), and one asking, "What do these people see themselves do- ing?" (the emic perspective, also discussed in chapter 3). From now on, we will be using the terms etic and emic in our discussion. ETIC APPROACH When you are in a situation in which you decide what is important to look at, you might think that you could just get out there and start "the real work" of collecting information right away. Unfortunately, the real work starts much earlier: you must figure out, in an organized way, what you want and need to know. Doing this is a bit like cross-examining yourself, so that everything is clear in your mind before you begin. This chapter helps you clarify your ideas (step 4 in box 3-1), narrow them enough so that they can be studied in a manageable way (step 5), work out exactly what you are looking for (step 6), and determine from whom you will get your information (step 7) and how (step 8). STEP 4: DEVELOP A ROUGH RESEARCH IDEA AND REFINE IT You need to avoid two important problems when developing and refining your research idea. One is choos- ing an idea that is too big (development, poverty) and the other is choosing one that sounds fancy, profes- sional, or scientific, but that is really not clearly spelled out ("The cost-effectiveness of measurement and testing in viable curriculum development processes"). If you know exactly what you mean by that, fine. If 1. This distinction between etic and emic has nothing to do with whether you have chosen the topic or an organiza- tion has comunissioned you to study a topic that it chose. It refers to perspectives. You can use either perspective no matter who chooses the topic. 39 40 Planning Your Research you have come up with it because it will sound good at a conference, you are in trouble. Study things that are real to you. Don't be intimidated by jargon. If you follow the process described in this chapter, you should be able to avoid both these problems. If you are working with an organization, use the process to have its staff work out what they want. Let us say that you or they want a good description of the general situation in relation to girls' education. We are already into problem number 1: too big. Look at the questions in box 4-1. We are using the journalist's old trick of simply asking "Who?", "What?", "Where?", "When?", "Why?", and "How?" You could create hundreds of questions this way, which would help you to see how big your idea is. You will have to work out in your own mind what you really want to do, because no matter how much time, money, and assistance you might have, you will never cover the entire topic for all aspects of education, for everyone, everywhere. Box 4-1. Is Your Topic Too Big? Who do you mean? All girls from birth to adulthood? All social, ethnic, tribal groupings? Girls in school and not in school? What do you mean by girls' education? Are you going to cover all levels of education and all forms: formal and nonformal, private and public, single-sex and coeducational, vocational and technical? All media: classroom-based and distance teaching? What aspects of girls' education? Problems? Opportunities? En- rollment? Persistence? Achievement? What are the consequences of girls' education? What is provided? The forms and content of education? Where? Everywhere? The nation? The region? One school? When? Are you interested in past patterns? Possible changes in the future? Why educate girls? Why is education provided in this form, and in this way? What are the aims? What are the policies? How is education provided? The systems, processes, and struc- tures? How it is assessed? Testing and measurement? Begin again. Girls' persistence in school is the concern. This is still too broad, but once you have your topic, you can narrow it easily by * Considering a special aspect of the problem that concems you, for example, girls at a particular level of schooling * Defining the words more specifically, for instance, state schools instead of all schools * Restricting the study to a particular group: poor girls, girls from a specific tribal group, or girls who are in Grade 6 right now * Restricting it to one geographic area, or to a particular institution or school * Restricting it to a particular time (or two or more specific times if, for example, you are comparing a group ten years ago with a group today) * Taking a sample. Planning Research that Describes 41 STEP 5: CREATE A RESEARCH STATEMENT OR HYPOTHESIS Now write out a sentence beginning "I want to study... " or "We want to study... " and fill it in. Using some of the narrowing techniques listed, let us say we have come up with the sentence: "I wani fo sfudy bhe reasons for complef ing educaf ion af Grade 6 in stafe schools among Grade 6 girls in X Disfrictf in 1995." This is your research statement. If X District is very large or if it contains a large number of girls, you may have to study a sample of them. Sampling is covered in part III, but whether you use a sample or not, the process from here on is the same. Literary quality is not important in a research statement. What is im- portant is that all the aspects you want to study are included and stand out clearly. When you have finished the narrowing process make sure that the statement still covers what you want to study. The steps you followed to get this far are: 1. Selecting the topic of interest by asking "Who?", "What?", "Where?", "VWhen?", "Why?", and "How?" and using any other device that helps you to realize how many facets your topic has. 2. Stating what you want to study in one sentence, if possible. If your research has several phases, you may have to use more than one sentence to develop the ideas. Box 4-2 shows some possible situa- tions that might interest you and how each might be converted into a research statement. Box 4-2. Possible Research Situations and Appropriate Research Statements Problem Research statement 1. Excluding nonstate schools may re- Reasons for leaving school among duce the usefulness of your study. Grade 6 girls in X District in 1995. 2. Possibly girls also drop out in Grade Reasons for leaving school among 2, and should be included. Grade 2 and Grade 6 girls in X District in 1995. 3. You may think that there has been a Reasons for leaving school among big change in the reasons why girls Grade 6 girls in X District, 1985 and left school ten years ago and why 1995. they leave now, and that measures to deal with the changes have not caught up because people are not re- ally aware of the fact. (Studying the effectiveness of the measures is an- other project.) 4. Perhaps both boys and girls drop out Reasons for leaving school among and programs have been developed Grade 6 girls and boys in X District in to prevent this, but you think that 1995. they meet boys' needs only, and you need to look at reasons why both girls and boys drop out to assess the program. 3. Defining every major word in your statement so that you have a clear guide as to what you are look- ing at throughout your research. You can define a word any way you like as long as you tell the 42 Planning Your Research reader what you have done: we have used the phrase completing education to mean finishing one's primary schooling and not continuing to the secondary level. You may choose to define some of the words in your statement according to definitions used in the professional literature in your field. Nonformal education, for example, has a generally accepted meaning in the field of education, and unless you are challenging or refining that definition in some way, you will probably use it or ex- plain why you are not. Also remember that words like effectiveness, satisfactory, and school age can mean anything. You have to explain what you mean by them. You may say: "I will define the program as 'effective' if 80 percent of the group that is entitled to participate is using it," or "School age includes anyone be- tween the ages of five and twenty-three." You should have a good reason for your definition, and it should be meaningful to the people involved. If you want to compare the results of your research with that of someone else, or with official statis- tics, you should use the same categories. For example, unless you have a good reason to do so, why group children into age categories four through nine, and ten through fifteen if school records or the official census groups them as five through ten, and eleven through sixteen? 4. Rewriting your sentence, taking all these decisions into account. You can see that by using this procedure you are taking a positivistic approach: you are determining the variables to be looked at and defining what each will mean. In the next step, we take the process even further. STEP 6: IDENTIFY THE SUBTOPICS FOR STUDY Now that you know what general topic you want to study, you have to break it down into subtopics so you will know what specific information you need to collect. This will be your research outline.The easiest way to do this is to identify your variables. A variable is any characteristic or attribute that can take a variety of forms, for example, education, sex, marital status, religion, ethnic group, career aspirations, and type of training can all take more than one form. You can have primary education, secondary education, or tertiary education; you can be male or female; and you can be single, married, widowed, or divorced. Each major word or phrase in your research statement is probably a variable, or else a word or phrase that requires ex- planation to put the situation in context (as in "three African countries" or "Lusaka" in the examples be- low). Here are three sample research statements, with their variables marked with roman numerals: (i (ii) (iii) * Cost-effectiveness of distance education in three African countries. (i (ii) (iii) * Schooling and labor force participation in Lusaka. (i) (ii) (iii) (iv) (v) * Examination results of girls in single-sex schools and mixed-sex schools in Accra. Let us assume we are happy with the statement we worked out in step 5. Our variables are as follows: i. Reasons for completing education at Grade 6 in ii. State schools among iii. Grade 6 girls in Planning Research that Describes 43 iv. X District in v. 1995. This research statement has five major words or phrases about which information must be collected. The easiest way to figure out what information is to take each word or phrase and consider what facets or char- acteristics it could contain, that is, break it down into all its possible parts. (Remember our discussion of pos- itivism in chapter 2, in which we compared reality to a giant clock that can be taken apart for examination?) The word or phrase you begin with does not matter, but "people" words or phrases and "place" words or phrases are easier, for example, people always have certain characteristics: age, sex, height, weight, mar- ital status, social class, residence, ethnic group, education. "Grade 6 girls" is a people phrase and will have these. Because these girls are students, they will have some special characteristics, such as examination re- sults and number of years spent in school. If you were studying teachers, their special characteristics might include type of training, number of years of experience, special courses attended, and so on depending on the focus of the study. So we will begin with the girls. Grade 6 Girls First, make a rough list of all the attributes you can think of for people, such as age, sex, marital status, height, weight, number of children, religion, health, political affiliation, occupation. Keep going. Some may have no bearing whatsoever on your research, but thinking of them and discarding them later is better than forgetting to include them and regretting it later. Now write each on a separate scrap of paper, and see if you can group them in some way that makes sense to you or makes sense in the context of the project. There is no right way to do this. For some projects you might group them into characteristics people are born with versus those they acquire or achieve. For another project you might group them into physical versus social characteristics. You might make more than two groupings or just one long list. If you used physical and social characteristics you might group them this way: Physical characteristics: Social charactetistics: Age Marital stafus Sex Number f children Height Religion Weight Ethnic group Health Political afiliation Occupation Your groupings will make sense only in the context of a particular project, for example, age and sex are physical characteristics, but they can also be the bases of social groupings. And you are born into an ethnic group, so if your sortings were based on groups people were born into versus those they acquired later, ethnic group would end up with age group, sex group, and so on. There is no magic to this. Do what makes sense for your project and the people you are working with. Make another list of the characteristics the girls will have because they are students: subjects studied, examination results, number of years in school, which school they attend. Now put all the attributes together in a list (see box 4-3). Leave out any that you feel certain are irrelevant to your project. For example, collecting information on height is unlikely to be worthwhile for this project. 44 Planning Your Research But be careful about making quick assumptions. Some years ago in western Sudan, where people keep no record of age, children who were unusually tall were refused entrance to secondary school on the grounds that they were probably too old. In a case like that, height could actually be a reason for having to leave school. Box 4-3. III: Grade 6 Girls A. General characteristics 1. Age 2. Area of residence a. District b. Rural/urban 3. Ethnic or tribal group 4. Religion 5. Marital status 6. Occupation(s) of a. Mother b. Father c. Self i. Domestic work a. Hours spent per week b. Activities involved ii. Other employment a. Hours spent per week b. Activities involved B. School-related characteristics 1. School attended 2. Number of years in school 3. Examinations or other assessment a. Subjects b. Results 4. Distance from home 5. School costs a. Fees b. Other expenses i. Books ii. Uniforms iW. Supplies iv. Food v. Transport vi. Other (club memberships, etc.) You will easily be able to think of topics missing from this outline that are important in your area or re- gion, and perhaps you could omit some of the topics above. A few of the topics listed earlier, such as polit- ical affiliation, have been left out of our imaginary project, but in many studies political affiliation might be an important determinant in whether one has the opportunity to go on to secondary school. This is why we said in chapter 2 that in doing positivistic research, in which you determine the variables, as you are doing in this outline, it is important to be very familiar with the situation. This is another reason why local knowl- edge and insights are valuable. X District We can now take a second variable, the place word, X District. Places (and institutions) by their nature have certain characteristics, like a location, a history, a size, and so on. Bear in mind that you are not doing a de- tailed study of the district or the school system. You are getting enough information to put the place in con- Planning Research that Describes 45 text. Box 4-4 sets out the characteristics for X District. As we are studying education, note that we have two sets of characteristics: general characteristics and school-related characteristics. Here is another way that understanding the general context can help. Suppose some girls tell you that the reason they are leaving school is because they are pregnant and the school officials will make them leave. Under the policy point in box 4-4, you may have discovered that it is not school policy to do so. This is worth looking into. Do the girls simply believe that the school officials will make them leave and therefore plan to do so? Or does a particular school make them leave despite the policy? Cross-checking contradicto- ry bits of information is easier and faster if you have a general picture, but if you find something odd and have no other information about it, follow it up anyway, even though it is not in your outline. Suppose that you discover that one reason girls are finishing at Grade 6 and not going on to secondary school is that no religion is taught there, and parents are not happy about that. That can be covered under curriculum. But suppose one reason is that, as was the case in one project, there were lions around the school and some of the girls were afraid to go (so were some of the boys). You cannot think of everything. Unless you know about it in advance, you are not going to put lions in your outline on the off-chance that dangerous animals might be an important feature of the school system. That does not mean that you should not look into it. Your outline isn't cast in cement, nor is it sacred. 46 Planning Your Research Box 4-4. IV. District X A. General characteristics 1. Location 2. Division a. Of larger unit (is it a region, section, etc.) b. Subdivisions within it 3. History 4. Size a. Area b. Population i. By age ii. By sex iii. By ethnic group, tribal group, religion, languages, etc. iv. By school-age populations a. In school b. Not in school v. By literacy a. Male b. Female vi. By economic activities 5. Economy 6. Social and cultural characteristics a. Roles and responsibilities of women and girls b. Beliefs about desirable female characteristics c. Preferred age of marriage d. Decisionmaking abilities of each sex e. Contribution of each sex to the economy f. Beliefs about female nature and abilities g. Attitudes toward education for girls h. Attitudes toward desirable skills and abilities for girls B. School-related characteristics 1. Structuring of school divisions or districts 2. Types of schools a. Levels offered b. Public/private c. Denominational/nondenominational d. Single sex/coeducational e. Fee paying/free f. Nature i. Academic ii. Vocational/technical iii. Formal/nonformal iv. Conventional/distance 3. Administrative characteristics 4. Policies, especially in relation to enrollment, gender-related issues (pregnancy, subjects offered, etc.) 5. Curriculum 6. Textbooks a. Provision b. Content, especially in relation to gender 7. Staff a. Numbers b. Composition by i. Sex ii. Training iii. Qualifications 8. Teaching methods Planning Research that Describes 47 State Schools The next point we might look at is the variable state primary schools, which is also a place word. We already have a lot of information on the school system under the heading X District. We could just go into more detail about the primary state school system under that heading, or set aside a special section here to look into it in greater detail as shown in box 4-5. Box 4-5. lI: State Primary Schools A. Enrollment rates 1. By sex 2. By socioeconomic group, religion, ethnic or tribal group, language group, etc. 3. By rural/urban location B. Repetition rates 1. By sex 2. By socioeconomic group, religion, ethnic or tribal group, language group, etc. 3. By rural/urban location C. Persistence rates 1. By sex 2. By socioeconomic group, religion, ethnic or tribal group, language group, etc. 3. By rural/urban location D. Achievement 1. By sex 2. By socioeconomic grot.p, :ligi;,n. ethnic or tribal group, language group, etc. 3. By rural/urban location E. Percentage of students continuing to secondary school 1. By sex 2. By socioeconomic group, religion, ethnic or tribal group, language group, etc. 3. By rural/urban location Notice point D, achievement. Educators have no simple way of studying achievement. Tests can mea- sure some things, but not others. Later on you are going to have to decide what you are going to take as indicators of achievement, for example, you could use performance on the Primary Learning Certificate. Reasons for Completing Education at Grade 6 Next we come to what we are really interested in: reasons for completing education at Grade 6. Although making a research outline is tedious, it can almost become a game. You can keep thinking up more subtop- ics within a variable, and if you do not mind running around collecting unnecessary information, it is not too serious a mistake. But if you go astray now, that is dangerous, because the variables you select here are going to be the basis of what you ask the girls, their teachers, their parents, and others. If you decide on a list of reasons that have no real bearing on the situation, you might never discover your mistake. People are generally obliging, and may even make the mistake of assuming that you know best. For example, you de- cide the five possible reasons for leaving school are that (a) the grass around the school is too long, (b) the uniforms are unattractive, (c) the students would rather play soccer, (d) the teacher is too short, and (e) oth- er. You would be surprised at how many people will genuinely try to fit their experiences into your cate- gories as best they can. Others will find that your reasons have so little relevance that they will choose "other." Out of 100 people you question, 75 may mark off the box called "other." Everyone, including you, will be baffled. To avoid this you have to do enough preliminary research to break down the variable reasons into meaningful categories. There are many ways to do this: you could interview a variety of people to get pos- sible reasons; you could do a pilot survey in which you ask some broad questions about why girls leave 48 Planning Your Research school, and leave plenty of space for the respondents to write in their own answers; you could look at other studies that have been done; you could look at school records. Doing several of these preliminary activities is a good idea. The school records, for example, may not give pregnancy as a reason, because it might reflect badly on the school or the community. However you have done it, let us now imagine that you have enough information to break down the vari- able "Reasons," and here they are: the family cannot afford the costs, the girl is needed to work at home, she is getting married, she has become pregnant, she thinks she has enough education, the family thinks she has enough education, the family is afraid she will be unmarriageable, the girl or her family or both think the ed- ucation she is getting is irrelevant or that it teaches undesirable characteristics, she is taking a job, she is taking a nonformal course, her marks are not good enough to continue, the secondary school is too far away. Group these if you can. Avoid giving your groupings value labels like "good reasons for leaving" and "bad reasons for leaving." Box 4-6 presents a possible grouping. Box 4-6. 1. Reasonsfor Completing Education at Grade 6 A. Costs 1. Financial costs a. Of fees b. Of transport c. Of books d. Of clothing e. Other 2. Opportunity costs (income or labor lost because the girl is in school) a. Lost income b. Lost labor B. Domestic responsibilities 1. Minding children 2. Housekeeping 3. Producing food for home consumption 4. Working on cash crops 5. Fetching water/fuel 6. Other C. Attitudes toward education: belief in negative consequences 1. Girl will be unmarriageable 2. Girl will learn undesirable ways 3. Secondary education undesirable or irrelevant 4. Primary school education is enough D. Attitudes toward the school 1. School environment not secure (danger of pregnancy, etc.) 2. Not enough female teachers E. Social and cultural beliefs toward 1. The role of girls and women 2. Girls' and women's abilities 3. The value of education for females 4. The desirable characteristics that females should have F. Girls' educational performance 1. Girls' marks too low 2. Girls' don't have required subjects G. Access problems 1. Not enough places in secondary school 2. School too far away H. Other plans 1. Girl going to train/apprentice 2. Girl going to marry/is married 3. Girl is pregnant/has child or children Planning Research that Describes 49 1995 Finally we have the date. You do not have to break down 1995 unless, for example, you are doing a month by month or season by season study. Then you would list the seasons and make sure that you studied the reasons for leaving in each season. If the school system, the economy, or anything else was very different in each season, you would have to study these factors for each season as well. If you were comparing reasons why girls left school in 1985 as compared to why they left in 1995, you would do this entire outline for 1985 and then repeat it for 1995. Putting the Outline in Perspective Now we are finished. Appendix I presents the entire outline. But we need to pause a moment and get a bit of perspective: just because each section of the outline is broken down in considerable detail does not mean the sections are of equal weight in terms of your time and energy. For example, the reason you are looking at X District is to be able to put your study into a geographic, historical, demographic, institutional, and ad- ministrative context. Everything you learn from this part of the outline may occupy only two paragraphs in your final report, if you need to write one. What you learn may help you or others to compare your find- ings with those of other districts: for instance, your district might have coeducational schools while another has only single-sex schools. Or your district may have a high percentage of untrained teachers while anoth- er one has almost none. It may help you to come up with new lines of research to follow later: are girls leav- ing because they are getting a low quality of education? While allowing for the fact that something helpful may emerge from this section of your outline, you could spend all your research time collecting the following information for X District (from box 4-4): 7. Staff a. Numbers b. Composition by i. Sex ii. Training iii. Qualifications Remember, you are not doing a survey of teacher characteristics. You are trying to get a picture of the con- text in which the Grade 6 girls are making their decisions, which is what you are interested in. Try to dia- gram your research statement to see the "heart" of your research and the marginal areas as shown in figure 4-1: solid lines represent the more central parts of the research and thinner lines the information being gath- ered for context, with importance being shown by line thickness. A second caution is that you should not use this particular outline to do a piece of research. There is nothing scientific about it simply because it appears in a book. It was made up for an imaginary country as an example of one way, out of many, to do an outline. Even if you were going to do the same exact project, you should make your own outline using your knowledge of the local situation to make the topics relevant and meaningful. Not every research topic can be handled in this way, but many can. Suppose you have come up with "I want to study girls. Why don't they have more ambition?" All girls? More ambition than what? Boys? More than girls used to have? What do you mean by ambition? You have assumed that they have less. You had better establish your facts first, and if you are correct, then look at the causes. Why not see first what ambi- tions they have compared to boys or compared to girls ten years ago? Then your research statement may become: "Career ambitions among girls and career ambitions among boys in X Middle School." 50 Planning Your Research Figure 4-1. Putting the Variables in Perspective Reasons Girl Dec State Primary Schools X District If you can show that girls have less ambition (defined and measured in some way) than boys, then you are in a position to look at why. You may decide to look at why people (teachers, parents, and others) think they have less ambition. This is a descriptive study, whose research statement may be: "Perceptions of rea- sons for low career ambitions among girls in X Middle School as held by parents and teachers." If you think, in doing these studies, that you got a good idea of what was causing the low ambitions (per- haps no work for adult women in the area or a lack of female role models in the school), you could set up an experiment in which you compared two schools, one that had female role models and one that did not, and see if girls' ambitions were higher in the first school. We wil look at experiments in the next chapter. A Grid Approach No one likes research outlines. People who take a phenomenological approach think they are rubbish. Even some positivists do not like to see the logical outcome of a positivistic approach laid out quite so baldly. The people who dislike research outlines the most, however, are those who are trying to construct one, because the process is about as much fun as scraping your fingernail on a blackboard. Nonetheless, if you are taking an etic approach, failing to prepare a research outline is like wearing a blindfold to look for an item you lost in a forest: you'll waste time, find things you don't need, get off the track, and even when you find some- thing, you won't know whether it's what you lost. Another way of figuring out what you need to know is to use a grid. You can use this technique for any subject, but we are going to use a project-related example here. Suppose that you are going to assess a sit- uation to plan a project. List all the aspects of the situation that you think are important. Convert each idea to one word, or two at most. Write each word in its own block down one side of a large box, and then write the same words across the top (figure 4-2). Then locate any particular box, and reading across and down, create a question out of the two words that "meet" in the box. For example, in box F-5, where "Resources" and "People" meet, you might ask "What resources do the people involved have?" and "What resources do the people involved need?" Not all boxes will produce sensible questions for your project, and because every combination appears twice ("Facts" and "Reasons" is one combination, and "Reasons" and "Facts" is another), you may have questions repeated. This is really only a device to help you cover a lot of angles that you can then include in your research or reject. This grid approach also has other uses. As you will see in chapters 11 and 12, you can use it to get questions for interviews or to choose situations to observe. Planning Research that Describes 51 Figure 4-2 shows part of a grid (the complete grid appears in appendix B. This grid would be suitable for assessing a situation or for evaluating a project. (To assess a situation, leave the grid as is. To evaluate a project, wherever the word situation occurs, change it to project.). For example, box F-4 "How do the people carry out the processes involved?" might help you to look at whether the current staff ("People") in a school are sufficiently trained to carry out a new, more interactive curriculum ("Process"). As you will see, the grid in appendix B is thirteen-by-thirteen, with a total of 169 question cells, but a grid could be much smaller, say two-by-three. The size of your own grid will depend on the problem you are studying and the number of dimensions that seem important. Whatever the size, it is most unlikely that all the questions will be equally important or that you will use all of them. Figure 4-2. Part of a Grid For Choosing Research Questions 1 2 3 4 5 6 7 Reasons Activity Needs Process Resources People Places A l WHY HAS | Why was this Why have Why were Why were Why are these Why this Reasons THIS activity these needs these these people place or these SITUATION chosen? arisen? processes resources involved? places? COME chosen? chosen? ABOUT? l B -hy these WHAT How does the How does the How do the What What Activity activities? ACTIVITES activity meet activity relate activities activities do happens at ARE needs? to the relate to the people carry each place? INVOLVED? processes? available out? resources? C Why these What needs WHAT ARE How do How do What needs What needs Needs particular are these THE NEEDS needs relate to needs relate to do the people are being met needs? activities IN THIS the processes resources? meet? What at each place? meeting? SITUATION? employed? are their needs? D Why are these How are What WHAT What What What Processes processes processes processes are PROCESSES processes are processes do processes are being used? related to being used to ARE BEING used in they employ? being used in activities? meet needs? USED? getting each place? resources? E Why are these How are What What WHAT What What Resources resources resources resources are resources are RESOURCES resources do resources are being used? related to being used to needed to ARE people available in activities? meet needs? carry out the INVOLVED? control/use? each place? processes? F Why are these Who is Who is How do the -What W'HO ARE Whoisineach People people involved in meeting these people carry resources do THE place? involved? the activities? needs? out the people have? PEOPLE processes Who controls INVOLVED? involved? / uses the resources? G Why have Which places Where are the How do the Where are the Where are the WHAT ARE Places these are used for needs being places relate resources people THE (locations, locationsbeen which met? to the being used? involved? PLACES, sites, chosen? activities? processes? SITES, institut- LOCAT- ions) l IONS? 52 Planning Your Research Notice also that this grid asks questions that require judgments that you or your team will make in the process of doing your research. To make the process a participatory one (see chapter 14), the people in- volved in the situation take responsibility for getting the information that will allow them to make these decisions. For a discussion of participatory evaluation see Uphoff (1991). The examples here are instances of descriptive research, but the basic principle involved, that is, making your variables explicit, is important for explanatory and predictive research as well. STEP 7: MAKE DECISIONS ABOUT SOURCES OF INFORMATION Soon you are going to be ready to assemble your entire research plan: the outline, your sources of informa- tion, and your research techniques. When you have finished, you will have a research design. First, for each point in your outline, you are going to have to decide where the information will come from. People? Records? The professional literature? You will probably draw on all of these. In the case of people, you will have to decide what types of people, and maybe even who in particular. In our study of reasons for completing education at Grade 6 in state schools among Grade 6 girls in X District in 1995, we are interested in rmding out why this group of Grade 6 girls is leaving, but do we want only their reasons? Do we also want their parents' reasons and what their teachers think? For some points, such as character- istics of the schools, we may name a category of person (principals), or even a person ("Mrs. N'Dow"). Fi- nally, as we discuss later, but the point cannot be made too often, do not restrict yourself to "important" people, or "experts," or people who are easier to reach. Everyone's voice is important when you are trying to understand something. You will probably also have to consult records and statistics. It often helps to do that before you begin to plan because they may give you some ideas of what needs to be studied. Looking at Hartnett and Hen- eveld's Statistical Indicators on Female Participation in Education in Sub-Saharan Africa (1993), for example, may highlight a particular problem for your country. But even after you have chosen the topic, you will probably need records and statistics to help you get a general picture of, say, the number of girls in school or exami- nation pass rates. Chapter 8 looks at some of the sources available and some of the problems associated with using them. You may want to look at the professional literature to see what else has been written on your subject. Chapter 8 lists some of the major education journals. If you want to read reports or books related to your subject, you may find books or articles in the end of chapter readings or final bibliography that sound use- ful. Getting access to them requires the services of a good library, which may have to order them from other libraries, perhaps abroad, or you may be able to obtain them from someone in a university or research in- stitute in your country who is willing to help you. Or perhaps from the author, who may be willing to send you a copy of an article. You do not necessarily have to look at the professional literature at all. You are probably not trying to make a new contribution to theory or research methods. You may just want to know what is happening in your school or district and do a piece of research that will help you solve a problem. Once you have made these decisions, you can begin to make a research design. For every point in the outline, the design will show the sources of information that you will use and the techniques for getting the information. Box 4-7 shows the part of our outline that deals with people's ideas about the roles and respon- sibilities of girls and women. (This point appeared in our outline under "Social and Cultural Beliefs"). Chapters 6 and 8 will give you more information on choosing sources. Planning Research that Describes 53 Box 4-7. Part of a Research Outline: Point and Sources Outline point Sources Roles and Community members: responsibilities of Elders women and girls Teachers Parents of girls Other: boys, young men Teachers, school officials in X District Community activities Textbooks STEP 8: MAKE DECISIONS ABOUT INFORMATION GATHERING TECHNIQUES For each point in your outline you also have to decide how you will get the information. Will you conduct a survey? Will you interview people? Will you see what the records say? Will you watch to see what hap- pens, for instance, how do girls who say they are leaving because they have no time to study actually spend their time? Part III discusses these and other techniques. For the moment, let us imagine that you have some ideas about your sources and know enough about research techniques to choose the ones you will use (box 4-8). Box 4-8. Part of a Research Outline: Point, Sources, and Techniques Outline point Sources Techniques Roles and responsibilities Community members: Semistructured interviews of women and girls Elders Teachers Parents of girls Other: boys, young men Teachers, school Survey question officials in X District Community activities Observation Textbooks Text analysis This process is carried out for each outline point. Note that for your central topics you should be using as many techniques as possible to get sound information. The use of a questionnaire is mentioned for this point, but of course you would not issue a questionnaire on a single point. You would look through the rest of your research design, see what other points you plan to obtain through questionnaires, see to whom you will put the questions, and decide whether putting them together and giving a questionnaire to that group would be useful. Note also that textbooks are mentioned here, but are also found in point IV. B.6. b. (box 4-4) in the 54 Planning Your Research outline, to be looked at for gender content. This kind of "crossing" will be common, because real life is more complex than an outline. (The particular techniques listed in the third column will be explained in chapters 9, 11, 12, and 13.) Finally, as shown below, you can put all the parts of your research design together: the outline, the sources, the techniques, and two additional columns to help you keep a sense of perspective (box 4-9). Box 4-9. The Complete Research Design for One Point in the Outline What will this How does it contribute Outline Point Sources Techniques tell me? to my research? Roles and Community Semistructured What various These views may affect responsibilities members: interviews community parents' decisionmaking of women and Elders members feel are girls Teachers appropriate roles Parents of girls Other: boys, young men Teachers, school Survey What people in These may affect officials in X the school system classroom behavior, District say counseling Community Observation What I can see What people say and activities happening in what happens may be relation to these different roles Textbooks Text analysis How girls and Texts may influence women are girls' behavior and presented in texts aspirations The fourth column encourages you to think about whether you need a particular piece of information, and whether what you are planning to do will provide it. The last column reminds you that some parts of your research are more central than other parts. Some only provide context, such as population numbers or facts on the local economy. The point in box 4-9, however, relates to the "heart" of the study. EMIC APPROACH The emic approach involves an entirely different way of proceeding. Because you cannot determine the variables to be studied, you cannot prepare a research outline. (This alone may be enough to convert you to the emic approach!) You will be starting with a general topic that interests you-how children categorize the activities in their school or how villagers perceive the characteristics and abilities of boys and girls-but you will be using methods that allow the important dimensions of the topic to emerge, rather than deciding in advance what they are and asking questions about them. For example, in a village in Zambia a group of women gave these causes of poverty: * "God's creation. That is how God created them, hence no matter how hard they struggle nothing will change." Planning Research that Describes 55 * "Laziness. One's heart is not desiring to work." * "Poor planning." * "Lack of initiative. Not using one's initiative and talents." * "Maanu ndibule. Not mixing with friends because they think they know everything." Some of these are not categories that an outsider could guess based on "common sense," but they are mean- ingful to local people. As the investigators in this case point out, straightforward questioning on these is- sues would have tended to produce simplistic results (Norton and Milimo 1993.) Emic research can be quite comprehensive and time consuming. The following discussion presents a general way of using the approach to complement other techniques. Emic research can be carried out in sev- eral different ways. Two examples are the grand tour question and card sorts. The Grand Tour Question The grand tour question approach (Spradley and McCurdy 1972) begins by asking the broadest possible question appropriate to the situation: "What happens here all day?" "Tell me about the children who come to this school." "What kinds of teachers are there?" The aim of the exercise is to learn how people categorize things and why (types of schools, parts of the work day, kinds of villages, characteristics of successful wom- en, tasks of a teachers' union, sources of conflict in a school, and so on). Let us say that you are talking to a group of girls about the students in their school. Question (Q): Will you tell me about the girls in this school? Answer (A): What do you mean? Q. Well, what kinds of girls are in the school here? Tell me something about them. A. Well, there are all kinds of girls here: the poor girls, and the loose ones, and the country girls, and the stuck-up girls, and then, of course, the ordinary girls. Oh, and the handicapped. And the Mandinkas. Q. Tell me about the poor girls. A. The country girls-we don't talk to them. They wear funny clothes and they always look hungry when we're having our lunch. Sometimes they hit us. They're very rough. Q. Tell me about their clothes. A. They can't afford uniforms so they wear whatever their parents can find for them. We shout names at them. Q. Tell me about them hitting you. A. They hit us for no reason at all. We can't help it if they're funny looking. Q. Tell me about the loose girls. A. They don't have enough money to buy books or lunch, so they do favors for the teachers or older stu- dents and get money that way. Q. Tell me about the favors. A. They pretend to talk to them about love and then they get pregnant and have to leave the school. Q. Do the country girls do favors for men? A. No. Men don't like them. They look funny-I told you. Q. Tell me about the rich girls. A. The rich girls always have plenty of money for books and uniforms and lunches. 56 Planning Your Research Q. Where do they get the money? A. They do favors for their boyfriends. Q. Like the loose girls? A. No. The loose girls don't love the men they go with. They only do it to get money. Q. Tell me about the ordinary girls. A. Oh, all the girls from around here are ordinary; nice normal girls like us. We help each other. If one of us is doing a favor for a man because she hasn't got enough money, we'll tell the teacher she got sick. Q. Tell me about the stuck-up girls. A. They think they're wonderful. They don't talk to us. Thev say their parents won't allow them to. We don't care, because they're spoiled. Q. How are they spoiled? A. They never have to worry about money or books or lunches. Their mothers and fathers give it to them, even if they have to work extra hours to get the cash. They go with boys because they want to. They don't need the money. Q. Tell me about the handicapped. A. They're from the country. They need special help or something and so the government makes them come here to the city and they go to school. Most of them are very poor. Sometimes their families have forgotten about them. Q. Do they do favors for men? A.No. Q. Tell me about the Mandinka girls. A. Oh, they're Mandinkas. What can you say? They're all poor people from the country who have moved here. They're always hanging around men. See, there's one over there, the one in a wheelchair. Notice several things about the interview above. First of all, you created only one question of your own, the first one. You might go into more detail about something that interested you in an answer, but basically you are always coming back to the categories the girls gave you in the original answer. Second, it is not your job to set these girls straight. You are trying to find out how they see things-don't point out why the coun- try girls might be hitting them, although you might be sorely tempted-don't mention that your mother was an urban, chaste Mandinka. Finally, notice that these girls have clear-cut categories in their minds that might not seem so clear to someone else, for example, in several of the categories the girls come from the country (including the "country girls"), but in the case of the handicapped, the Mandinkas, and the loose girls, something else puts them in a special group rather than the "country" group. Except for the "rich" girls and the "stuck-up girls" they all seem to be poor. The "ordinary" girls seem to have worked out the same strategy for getting money as the "loose" girls. The only difference is that the "ordinary" girls are all locals. The Mandinka girl in the interview was in a wheelchair-but she was not put in the "handicapped" group. Apparently, in their eyes being a Mandinka is her major characteristic. So far, what we have looks like box 4-10. Card Sorts You can also get the kind of information that we just got from one person or from a small group in the school using a card sort. Put the names of all the girls in the class on cards and ask the girls you are working with to sort them into categories. If they are working in a group, you will be trying to get shared categories. Planning Research that Describes 57 Box 4-10. "The Girls in This School" Enough Look Dofawrs Dofavors Handi- Special Category noney funny for money for love Rural Local capped ethnic group Country X X Loose X Rich X X Stuck-up X X Ordinary X X Handicapped X X Mandinka X X You can test the categories by repeating the process with other individuals or groups in the school. Prob- ably no two groups or individuals will come up with exactly the same categories: some might produce five, some six, and so on, but you may be able to rearrange them into five common categories. You can then go back to the individuals or groups and see if they accept the new categories. When you have the agreed categories, you can reverse the process and present a new group of students in the same school with the categories and the pack of cards with the students' names and ask them to sort them. If the new group finds that some people do not fit, or that one of the categories has no cards in it, you have to refine the categories. You can also test the categories by presenting a person or group with a pack of cards with each person's name on it and ask that they be sorted into piles of "likeness." When the piles are finished, ask what the people in each pile have in common that makes them different from the people in the other piles. You are hoping that their answers will correspond, roughly, to the categories you got from the interview. This card sort or pile sort technique will be explained further in chapters 11 and 15, along with other techniques suit- able for emic research. Some Issues and Problems When you are doing emic research, you will find that although people may categorize information in their heads in the ways shown in our sample interview, they are so unused to being interviewed through this seemingly vague sort of questioning that their first answer will probably be, "What do you mean tell you about the girls in this school? What do you want to know?" Also, they are unlikely to announce the catego- ries as clearly as in our interview, particularly when the subject is more complex. You must train yourself to listen for them. Of course, they may not start out by discussing the dimensions of the situation that inter- est you. They might sort their piles into "people I know versus people I don't know" or "people who owe me money versus people I owe money to." You have to keep at it. If you want social categories, you can 58 Planning Your Research point to someone passing in the school, and say, "Tell me about her. How is she different from you? How is she the same?" and steer the interview in the direction of social categories. Finally, you can ask these questions in a more conversational manner. The interview does not have to sound as rote and mechanical as the sample conversation, which was constructed this way to show the process. What if people do not agree on the categories? Usually the reason you are using this method is to dis- cover shared idea systems, not individual perceptions. If there is no agreement, perhaps the people you are working with do not see things the same way, which is itself interesting. Or possibly the subject means little to them, and no one has really thought much about it. Remember, however, that any area of culture consists of multiple perspectives and interpretations. Peo- ple may agree on categorizations, but attach different meanings to them. Or they may feel that groupings exist, but do not agree on what they are. (We might be safe in guessing that the "loose" girls, even if they agree that such a category exists, may not see themselves in it.) Also, the categorizations occur in a particular context. If you were discussing some other important issue related to girls with exactly the same people, they would come up with different groupings. Groupings also change over time, as do their meanings and the people or items in them. So what is this approach good for? Remember, you are using these techniques for a restricted purpose: to get ideas about tne way people see things in certain circumstances. For example, people tend to treat oth- er people according to how they perceive them. (ThWs is a version of a famous theory put forward by Park in 1928.) If some girls are refusing to attend school because or the way they are treated, they don't have to agree with the category they have been put into, but they'll still stay at home. In the etic research approach, you prepared a research design showing the sources and research tech- niques you hoped to use for each point in your research outline. You have no points in emic research, just general topics you want to explore, so the decisions yoU make about how to get the information are simpler. You will use emic interviews, card sorts, triads, conten4 analysis, and other techniques discussed in part lII. The type of source-people, records, and so on-can be worked out in advance, but you will probably find that as your research progresses, it will lead you on to other sources in a snowball effect. This leads to the interesting question of how you will know when you have finished, because a snowball can keep rolling and growing. You have finished when the group with which you are working agrees that these categories are real to them: when you go to new people and they can offer nothing that you do not already know, and when you can group people or items or events in the same way as the local people do. Then you understand the principles and there is no point in pursuing the categories further. Is this information scientific or objective? By now, perhaps you see that this is not the issue. The question is whether this information is useful in helping you to understand what is going on. The teacher's catego- rizations of "late developers," "gifted children," and "children performing at grade level" are performance categories; the administrators categories of "late fee payers" and "accounts paid" are accounting categories. The ones the girls gave you undoubtedly have more bearing on how they interact with each other. What will you use it for? In and of itself, it provides you with local insights. You can also combine it with etic research. Before you determine your categories in a research outline, you might want to see what local categorizations exist. Grouping people by tribal group is pointless if it is unimportant to people, whereas whether they are descended from slaves or not is. You can leave part of your research outline blank and explore it through an emic approach: for example, reasons for completing education at Grade 6. If you are taking an emic approach, you still may need to outline part of your research. If you want to explain the local setting or the school structure and are not doing it emically, you should outline those parts of the research to keep yourself on track. Chapters 7 and 9-14 will give you more specific help with choosing your information gathering techniques. Planning Research that Describes 59 OUR GIRLS At the end of many chapters, you will see a box called "Our Girls." This will show how you can apply what you learned in that chapter to the problem we have been working on here: "Reasons for completing educa- tion at Grade 6 among girls in X District in 1995." FURTHER READINGS TO HELP YOU Anderson-Levitt, Kathryn, Marianne Block, and Aminata Soumare. 1994. Inside Classrooms in Guinea: Girls' Experiences. Washington, D.C.: World Bank. Hartnett, Teresa, and Ward Heneveld. 1993. Statistical Indicators of Female Participation in Sub-Saharan Africa. Technical Note No. 7. Washington, D.C.: World Bank, Africa Technical Department, Human Re- sources Division. Miles, Matthew, and Michael Huberman. 1994. Qualitative Data Analysis, 2nd ed. Newbury Park, California: Sage. Spradley, James P., and David W. McCurdy. 1972. The Cultural Experience. Chicago: Science Research Asso- ciates. Uphoff, Norman. 1991. "A Field Methodology for Participatory Self-Evaluation." Community Development Journal 26(4):271-85. What this approach Is good for: Showing how two or more things are related Example: What is the effect of dassroom gender bias on girls' achievement? Does femaie genitel mutilation have an Impact on girs' perlormance in school? Are girls safer In religious schools? Sample study: Abrah, Seged, and othes. 1 991, 'What Factors Shape Girls' Perfomne? Evidence from Ethiopia.' Intenalaonal Joumal of EducaJvf Devekpmet 1 1(2):107-1 18. 5 Planning Research that Explains and Predicts summary The experimental approach involves another way of planning research. Experiments do not prove or discover. They are, however, goodfor ruling out possible causes. You may recall that in step 5 (box 3-1) we can use a nonexperimental approach and create a descriptive research statement, or we can use an experimental approach, as we do in this chapter. Ex postfacto research means trying to work backward to establish causes after thefact. Wte use this a lot in daily life. An intervention or treatment is something you expose your group to: it is the independent variable or supposed cause. The dependent variable is the effect. The experimental group is the group tat: eceives the intervention or treatment. The control group is the one that does not. Nonrandomized experiments can be done on one or more groups. Tests can be given to the group or groups before and after, or only after. Their majorfeature is that (a) either they involve only one group; or (b) if two or more groups are involved, people are not assigned to them randomly. Randomized experiments involve two or more groups to which people are assigned randomly. Tests can be given before and after, or only after. Most explanatory social science research is done through analytical surveys using correlation research or quasi-experimental designs. Experimental research is sometimes possible in the social sciences, but meeting the requirements of the most rigorous forms of experiment can be difficult. Table 5-1 shows some of the strategies that have been used in developing countries to try to improve girls' and women's education. Notice the last column, "insufficient evidence to draw a conclusion." Why do the authors consider that we do not know enough about the effectiveness of using gender-r:eutrai school books or information campaigns? The reason is that we have few or no proper experiments to tell us whethe, and under what circumstances these will help girls and women. That is not to say that information campaigns or day care programs for younger children that free girls from baby sitting have not worked well in some places. It is just that we do not have enough information to know why they worked, if they would work elsewhere, or whether something else was happening at the same time that might have been the real cause of improved education for girls. Bellew and King (1993, p. 286) tell us that: "Striving to advance women's education often 61 62 Planning Your Research means proceeding with best guesses, guided by what has worked well under similar circumstances, or by what might theoretically work. Experimentation and careful monitoring are essential." Table 5-1. Summary of the Effectiveness of Strategies to Improve Girls' and Women's Education, Based on Country Experiences Insufficient Effective Ineffective evidence to draw a Objective strategies strategies conclusion Lower the cost of Scholarship Free uniforms Programmed education Culturally instruction appropriate Home production facilities technologies Female teachers Day care Alternative schools Flexible schedules Raise the benefits Vocational Vocational Gender-neutral of education training for training for curricula and growth sectors nongrowth books of the economy sectors of the School feeding when directly economy not programs linked to directly linked to employment and employment and Informaion strong no recruitment campaigns recruitment effort Source: Bellew and King (1993, p.316). In explanatory or predictive research you may be looking for cause and effect explanations, for example, does malnutrition in children cause impaired motor coordination? You may be looking for some other kind of relationship-correlation, for example-in which you are trying to find out if two variables go together, like income and school attendance. In this case, you are not arguing that income causes school attendance (or vice versa), but you may be hypothesizing that they go together in some patterned way, for example, that people with higher incomes tend to send their children to school. The rest of this chapter looks at how to plan either type of research once you have defined your basic terms. It focuses on experiments. Later in this chapter we will look at testing ideas another way, through the use of surveys. Whatever you are looking for, you still have to refine your ideas, as you did in steps 3-5 (chapters 3 and 4), because explanatory research demands that you be crystal clear about what you mean by each of the words and concepts you use. For example, if you are looking at whether literacy projects are more successful if they are run by govermmental or nongovenmmental agencies, you are going to have to define what you mean by literacy, projects, govemmental agencies, and nongovemmental agencies. This will allow you to decide how to categorize each project and agency and whether they qualify to be in your study. So make sure you remem- ber how to carry out step 4: Develop a Rough Research Idea and Refine It, before going any further. Planning Research that Explains and Predicts 63 Remember also that in the social sciences, explanatory and predictive research does not consist of neat, test tube experiments, but is part of a larger piece of research in which you must describe the setting, con- text, background, characteristics of the participants and respondents, and so on to the users of your re- search. The research outline approach in the previous chapter will help you to lay out this descriptive work. As we said in chapter 3, explanatory and predictive research can be used to clarify, to present reasons, to show relationships, and to show cause and effect. In the social sciences, the most common ways of offer- ing an explanation or prediction are through qualitative research using methods such as those presented in chapters 11-13, through surveys (chapter 9), and through so-called thought experiments (Bernard 1988, pp. 72-73). These are nonexperimental methods. Experimental methods are not used as frequently in the social sciences for a variety of reasons, such as the inability to meet the requirements of an experiment, ethical considerations, or a desire to study something under real conditions rather than laboratory-like conditions. These reasons will become more obvious later in this chapter. CAUSE AND EFFECT STUDIES Some social scientists believe that the social sciences should not try to model themselves on the natural sciences (see, for example, May 1993, pp. 86-87). They argue that the rigid requirements of experiments done in the natural sciences cannot be adapted to the social sciences, and if they are adapted, usually through a meticulous battery of compensating and checking measures, people and their responses get crammed into categories determined by the researclhLer. Variation, change, and in particular meaning, are lost in the process. Others argue that the experimental method, whether used on humans or anything else, cannot do what many of its supporters claim it can do anyway (see Cohen 1989, pp. 239-64). Finally, others recognize some of these problems, but believe that all in all, experiments tell us some- thing that cannot be got at in quite the same way by any other method. A basic introduction to experiments will not equip you to contribute to this debate, but it will enable you to understand what it takes to do ex- periments, and if you decide to do one, what kinds are possible given your circumstances. EXPERIMENTAL DESIGNS The most rigorous, and to some minds the only, way of establishing cause and effect relationships is through the experimental method, in particular, through the most exacting type of experiment, the classic experiment or true experiment. Before we look at types of experiments, note that: * Experiments do not prove. * Experiments do not discover. * When people say, "I carried out a little experiment," they probably didn't. If all this is true, what good are experiments? Experimental research does have certain strengths that are more developed than they are in nonexperimental research, namely: * Experimental research can eliminate or rule out more alternative explanations for something than nonexperimental research can. * Experimental research can be repeated in other places or situations or with other people more easily than nonexperimental research because of its c!ear-cut structures and rules. * Experimental research, again because of its structure and rules, can isolate single factors and exam- ine their associations and effects in a way that is difficult in cluttered real life situations. 64 Planning Your Research We will see what these points mean and raise them again at the end of the chapter. Ex Post Facto Research: Experiments in Reverse To begin with let us consider how people usually try to figure out things for themselves. In daily life, people often engage in what is called ex post facto reasoning, that is, concluding that something that occurs before something else must be the cause of it. You put a new roof on the school. Many of the children get sick soon afterward, and people conclude that the roof is making them sick. New, young teachers join the staff, and at the next major examination the children's scores go down. People start saying that older teachers are ob- viously better. Maybe these things are true and maybe they are not. They cannot be proven under these sim- ple "after, therefore because of" conditions. Sometimes, an ex post facto research approach is the only form readily available to a researcher. You visit a school where the children seem unusually lively and participate actively in class. The teacher does not seem to be doing anything different from teachers in other schools. The only thing that is different is that this school has a radio, and children listen to an instructional program for half an hour every day. We form a hypothesis, or a tentative statement, of what causes the phenomenon. We hypothesize that radio programs increase classroom participation. But we are looking at an after-the-fact situation over which we have no control-an experiment in reverse. We did not have a chance to measure the level of classroom ac- tivity before the school acquired the radio, and we do not know if other factors may have been at work. Per- haps before the school got the radio the number of lesson hours had been too high, making the children tired, and taking time out for listening has given them time to rest. Maybe the excitement and novelty of the program has energized them. Maybe the children were even more active before they started listening to the radio program: they do not actually like the program, and this has dampened their spirits. In this case we would have to say that radio instruction programs reduce classroom participation. The point is, we do not know enough to draw any conclusions. Our experience, however, might lead us to design an experiment that would allow us to rule out some of the explanations that occurred to us. We would have to assign children randomly to two groups, perhaps picking their names from a basket and putting them into two piles. By doing it this way, we hope to reduce the chances of all the talkative, active children or all those who had a teacher who encouraged a lot of par- ticipation ending up in one group. Then we expose one group to the radio program, observe them before and after the experience, and compare their before and after results with those children who were not ex- posed to the program. That is a classic or true experiment. In between these two approaches, ex post facto approaches and true experiments, are many other ways of looking at relationships between things. We will divide them into two groups: nonrandomized experi- mental designs and randomized experimental designs. Nonrandomized Experimental Designs Researchers in the social sciences often use nonrandomized experimental designs. The difference between these designs and randomized experimental designs lies, obviously, in the word random. In nonrandom- ized experiments you cannot assign people to different groups randomly. Often, this is because you cannot control what you think is the cause of whatever you are studying. If, for example, you had a theory that thunder decreases children's concentration in school, you could not arrange for thunder (the supposed cause) and you could not assign children randomly to thunder and nonthunder groups. Sometimes, you do not want to control what you think is the cause. If you think that severe malnutrition reduces reading comprehension, you are hardly going to set up a situation in which food is plentiful in one place and scarce in another, and assign the children randomly to the two places. Planning Research that Explains and Predicts 65 Sometimes, all the requirements that you have to meet to perform a randomized experiment are so strict that you end up creating a totally artificial situation, one that is not very useful because you will never find such conditions in the real world. In these circumstances, a nonrandomized experimental approach might be the one to try. Box 5-1. The Symbols Used in Experimental Designs R means random assignment 0 means an observation, a test, a measurement of some sort. 0h means the first observation, 02 the second, and so on X means ar intervention, a treatment, that is, something done to the experimental group To read an experimental design: Experimental group R 01 X 02 Control group R 01 02 Random Pretest Expefimental Post-test assignment group gets intervention (X); control group gets none Let us take one project and follow it through all the experimental types. (You can also use a nonrandom- ized experimental approach in surveys, discussed later in this chapter.) ONE-SHOT CASE STUDY OR SINGLE GROUP POST-TEST ONLY DESIGN Perhaps you have a theory that if girls have the chance to use gender-neutral textbooks that show them par- ticipating actively in all kinds of events, taking decisions, and showing leadership as often as boys, they will get a lot more out of their education. You have managed to get some books like this for a class, and the girls responded positively. "Using gender-neutral textbooks" is your cause, or independent variable, and "in- creased class participation" is your dependent variable, that is, the effect or the phenomenon in which you are interested. Using the standard symbols for experiments, we can show what you have done: One-Shot Case Study x 0 where X stands for exposure to the experimental variable, in this case the new textbooks, and 0 stands for the observation, measurement, or test that you make afterwards. After they get the textbooks, you are hap- py to see that the girls are cheerful and confident, but because you did not test the girls before giving them the textbooks, you cannot be sure what effect they have had. For all you know the girls may have been even 66 Planning Your Research more cheerful and confident before, and are now a bit overwhelmed by all these new books. Maybe they are truly more cheerful and confident because of some other factor, for example, they were delighted to be treated as a special group with new books. Or perhaps something else was happening at the same time: maybe the harvest was good or the village got a new pump that reduced their work load. Your boss thinks this is a silly idea. The ordinary textbooks were used when he was a boy, and look what a fine person he turned out to be. His wife used them as well, and can you see anything wrong with her? So what's the fuss? Anyway, it is too expensive to change them on a whim like this. So the thing to do is to improve your experiment. You are never going to be able to prove that exposure to gender-neutral textbooks makes girls more con- fident. The idea is too vague to test. The words exposure, gender-neutral textbooks, and more confident need to be translated into concrete, clearly defined terms. Instead, create a hypothesis: when only gender- neutral textbooks are used in the classroom, girls will show more classroom participation. You will have to clarify to yourself and the reader exactly what you used as gender-neutral textbooks, what you meant by participation, and what constituted more participation. For example, you might take par- ticipation to mean "raising a hand to answer a question," or "signing up to participate in a project," or what- ever you think is an appropriate measure of participation in the circumstances. If you are going to compare your results with those of a study done elsewhere, you should use the same definition of participation. Now you hope you will be able to show that with your program, under these conditions, the girls participated more. This is more modest than the hypothesis that "exposure to gender-sensitive textbooks causes greater confidence," but at least it is testable. SINGLE GROUP, PRETEST-POST-TEST DESIGN So you start again. You have a group of girls, and this time you are going to test them first, use the new textbooks, and then test them again. The advantage here is that you will know how much the girls partici- pated before and how much they participated afterwards. The disadvantages, however, are considerable. Let us say the girls participate more. You do not know that the exposure to the books is what produced the increased participation. Perhaps other factors intervened in the time between the first test or observation and the second one: the girls are getting an improved diet, or they got a new teacher whom they wanted to impress, or they just got more confident as they got older, or the shy ones left school. Or perhaps the post-test shows that the girls did not participate more. Once again, other things may have intervened. Perhaps the most active girls dropped out because their parents decided they were now mature enough to be helpful at home. Or maybe the pretest had some effect on them. Perhaps they saw you counting their participation rates and decided you were taking down their names to report them to the prin- cipal. What you have done is this: Single Group, Pretest, Post-test 0 X 0 TIME SERIES DESIGN You can tackle some of the problems by extending the process. Instead of observing or pretesting the girls, using the textbooks, and then observing or post-testing them, you could observe them many times, then use Planning Research that Explains and Predicts 67 the books, then observe them many times again. If the real explanation for their improvement was that they were just getting older and more confident, rather than benefiting from your program, the improvement would show as a progression that increased gradually each time you did an observation. If you were wor- ried about whether the test itself was causing an effect, it should show up after each observation, not just the one after the program. Time Series 010 2 03 04 X 05 06 07 08 What about something else that might be happening at the same time? How can you tell whether it or your program caused a change, if one indeed occurred? If the other event was taking place gradually, it would not explain any marked difference you observed right after your program. If it occurred as a single dramatic event at the same time as your program, separating the two would be more difficult. A number of variations on the following approach are possible (including using two groups and observ- ing them over time, but giving only one group the program). Figure 5-1 shows two possible outcomes, one (a) showing that your program had an effect, and one showing that it did not (b). Line (a) indicates that something happened at the point where you introduced your program (X). Line (b) shows that nothing happened at X, but something else was having an impact, because classroom participation rose throughout the period. Perhaps, as we suggested earlier, the girls were gradually getting more confidence as they grew older. Figure 5-1. Levels of Participation Participation (b) High (a) Low 01 02 03 04 05 06 07 08 Temporal intervals when outcome measures were taken, with treatment introduced between 04 and 05. Source: Judd, Smith, and Kidder (1991, p. 115). BEIWEEN SUBJECT DESIGNS Between subject designs refer to experiments in which some of the people involved experience your pro- gram and some do not. Later, we will look at within subject designs, in which the same people are ob- served to see how they perform with and without an experimental treatment. 68 Planning Your Research NONEQUIVALENT CONTROL GROUP, PRETEST-POST-TEST DESIGN One answer to your troubles might be to compare two groups of girls. In that case, you will have to restate your hypothesis, because now you are arguing that girls exposed to the program will participate more in class than girls who are not exposed to the program. You will need to be clear about what you mean by more participation. You are saying that increased participation is a result of, or dependent upon, using the gender-neutral textbooks. Pretest both groups. If the pretest has some effect, it is likely to be similar for both groups. Give the new textbooks to one group (the experimental group), but not to the other (the control group), and test both groups again afterward. This approach is a slight improvement over the last one, because you have been able to manipulate the independent variable-the program-by giving it or not giving it. You also have a better basis for comparison, the control group, but you still have problems: unknown to you, the two groups may have been very different from the beginning. Perhaps one group had a teacher last year who actively encouraged girls to participate. This approach can be represented as follows: Nonequivalent Control Group. Pretest-Post-Test Group I o X 0 Group 2 0 0 STATIC GROUP COMPARISON Now that you are more sophisticated, you may be worried about the effect of that pretest. It may have alert- ed the girls to the fact that something is up, and they may be participating more or less depending on what they think you are doing. Sometimes this is a valid concern, and the static group comparison is the only way to avoid it. However, if you drop the pretest, give the program to one group of girls and not the other, and then give them a post-test, you have another problem: maybe the group who were exposed to the program scored lower, even though the program worked, because the control group consisted of particularly active girls who would have scored much higher both before, had you tested them, and after. A diagram of this approach looks like this: Static Group Comparison Design Group I X OF Group 2 O, Randomized Experimental Designs Your boss is probably fed up by now. You have girls all over the place, new sets of textbooks, old sets of textbooks, tests going on all the time. "What next?" he asks. Now is not the time to give up. Get a big bowl. Get a new group of girls. Find out all the girls' names, put each name on a piece of paper, mix the papers up in the bowl, and draw them out, one by one, Planning Research that Explains and Predicts 69 alternately putting them into two piles. Or you could line up the girls, call them forward, toss a coin for each, and according to the results, assign them to one of two groups, heads or tails. (A simpler method is to use a table of random numbers as described in chapter 6). Gather all the girls whose names are in the first pile of papers into one group, the experimental group, and put the ones whose names are in the second pile into the control group. Use the textbooks with the ex- perimental group and post-test both. You have removed the effect of the pretest, and at the same time you are pretty satisfied that you have two comparable groups. By randomly assigning the girls, you have got around the problem of ending up with one group consisting entirely of teachers' pets or of girls who have taken assertiveness training. (Although, as you can imagine, if you repeated this name drawing process of- ten enough, you could end up with all-of-a-type groups, but this is a chance you are taking.) Here is what you have (remember that R means random assignment): Randomized Control Group, Post-Test Only R Group I X 0 Now that you are into the swing of things, you should consider another plan to ensure that the two groups really are comparable: the classic or true experiment. Randomly assign the girls to the two groups as before, pretest them before you give one group the program, and post-test both. The randomizing helps to ensure that the groups are basically equivalent, and the pretest tells us where each group is on the par- ticipation scale. This can be shown as follows: Randomized Control Group, Pretest and Post-Test R Group I 0 X 0 R Group2 0 0 Finally, if you want to cover all possibilities-the effects of the pretest, the effects of the program, and the use of random assignment-you can combine the last two types of experiments: get your girls, random- ly assign them to four groups, and pretest and give them a program, or not, as follows: Solomon Four-Group Design R Group I ol X 02 R Group 2 ol 02 R Group 3 x O0 R Group 4 0 CAUSATION Now that you have your results, you will want to analyze your findings. Chapter 17 explains how to do simple statistical analyses of data. But before you do that, let us look at some important issues: Does your 70 Planning Your Research experiment show causation? Many people make the mistake of assuming that because two phenomena go together, one causes the other. For example, single parenthood and poverty tend to go together, but does one cause the other, and if so, which one? Or does something else cause both? To make causal inferences, you have to have the following three things: * Co-variation: co-variation means that two phenomena go together, and when one changes, the oth- er does as well. For example, gender and level of income co-vary in most places in the world: women tend to have lower incomes. To prove causation, you must show co-variation, but that in itself is not enough. We know, for example, that gender does not cause low income, nor does low income cause gender. * Nonspuriousness: if two phenomena go together without one causing the other, both may be caused by a third. If you had concluded that one caused the other, you would have made a spurious assumption. The usual example given here is the association between the number of fire fighters at a blaze and the extent of damage caused by the fire. The greater the number of fire fighters, the more extensive the amount of damage. Are the fire fighters the cause of the damage? No, the size of the fire explains both. (In a twist on this, however, in some parts of New York City in the late 1980s and early 1990s, you could have found a relationship between the number of police officers in a few ar- eas and the number of drug deals and associated violence. In this case, however, it was the police officers-now former police officers-who were making the deals and creating the violence.) * Time order: for one thing to cause another, the cause must occur first. You might think this is obvious, as in the case of an electrical shock causing death. In many cases, however, it is not so clear. In devel- oping countries, does increased participation in girls' education lead to greater economic develop- ment, or does greater economic development lead to an increased participation in girls' education? To deal with these kinds of problems, you need to do three things. First, you have to be able to compare the results, either within the same group before and after a treatment, or between two groups, one of which gets a treatment while the other does not. This will help you to see if two things co-vary, in your case, ex- posure to gender-sensitive textbooks and greater participation in the classroom. Second, you have to be able to manipulate the independent variable, that is, because you are arguing that your textbook program improved participation, you have to be able to set up the program and show that the improvement occurred afterwards. In nonrandomized experiments this can be difficult, and some- times impossible, to do. For example, you cannot introduce an earthquake to show that natural disasters encourage greater community solidarity. In true experiments, there are instances where you can manipu- late the variable, but should not, for example, wrongly telling people that their families have just suffered a bad accident to see how they respond to stress. Third, you have to be able to control the research situation, so that you are ruling out other factors that might explain your findings. For example, if you give your experimental group new textbooks and asser- tiveness training, you will not know which one caused the improvement in class participation. If you do not or cannot control the research situation, you will have problems of internal validity: you will not know whether your results are caused by your independent variable (your textbook program, in this case) or by something else. Some threats, or confounds, to internal validity are as follows (Campbell and Stanley 1966): 1. History. This is anything else that happens during the course of the experiment that may affect the results. Let us say that while your girls are using their new textbooks they form a sporting team that wins many prizes. They become more confident, but which, if either, is the cause? 2. Maturation. Developmental changes in the research subjects over time may cause the change, re- gardless of your treatment. People get older and perhaps more conservative, wiser, decrepit, can- Planning Research that Explains and Predicts 71 tankerous, and so on. Let us say a lot of gender-based fighting is going on among children in your local playground, and you decide that ten-year-old boys should be taught to be more aware of the need to cooperate with girls. You set up a five-year training program for them. Don't be surprised if the national press doesn't show up when you announce your results. 3. Mortality or attrition. This doesn't necessarily mean that the participants die, although it can. It means that they drop out of the experimental categories, or out of one experimental category and perhaps into another. If there is any patterning to the dropouts, it may affect your research. For ex- ample, for years researchers have compared the health of drinkers of alcoholic beverages versus nondrinkers and found that moderate drinkers are healthier. But look at the nondrinkers. They might include former heavy drinkers who are now so unhealthy that they have had to give up drinking altogether. 4. Regression to the mean. You may want to include a wide range of students in your study, so you look at their last examination marks and take some who scored very high and some who scored very low. You pretest them. But you may have caught the high performers when they were at their ab- solute best, while the poor ones were at an all-time low. Later, they move toward their mean or more average position. When you post-tested them after your program, their results were certainly different. Is it your program, or were they just reverting to normal? Sometimes, individuals at are such extremes that there is nowhere else for them to go but up or down on the post-test: programs aimed at the poorest of the poor may seem to improve their lot, but perhaps they would have im- proved without the program, because they could not get any poorer. 5. Selection interactions. How you select the participants may interact with some of the other prob- lems listed above and affect your results. For example, selection and history can interact. Let us say you develop a program to encourage people to participate more in community organizations, but during the course of the research, the community from which you have drawn the experimental group loses funding for many of its organizations. When you discover a decrease in participation in the experimental group after your program, you could conclude, perhaps wrongly, that the pro- gram is not very good. Selection and maturation can also interact in that one group of participants may change at a different rate than another. For example, children ten to fifteen years old change more, physically and socially, than adults thirty to thirty-five years old. 6. Testing. People (and other animals) can be affected by tests. They can learn to perform better on them just by doing them, they can figure out what you are looking for, they can get bored, and so on. Changes that you see in the post-test may be a result of these factors. 7. Instrumentation. This is a bit like changing the goal posts during the game. When you pretest and post-test, the tests or measures have to be the same and must be administered the same way under the same conditions. If you had higher standards for "classroom participation" in your pretest and lower ones in your post-test, your program would probably show some improvement in classroom participation, but it would not tell you anything useful about the program's effects. Or if someone is helping you and administers the tests in a different way, that too would confound the results. 8. Diffusion of treatment. This occurs when your program "leaks." People in the control group dis- cover, one way or another, that you are testing to see whether drinking eight glasses of water a day gives one a lovely complexion. They would like lovely complexions too, and they start drinking a lot of water. Now you have two damp groups, all beautifully radiant perhaps, but no experiment. You can probably see by now that pre-experimental approaches are far less successful at handling these problems than true experiments are, largely because they do not have a randomly-assigned control group. For this reason, using pre-experimental approaches to show causation is extremely difficult. Table 5-2 shows the confounds associated with some of the approaches, arranged here from "weakest" to "strongest." 72 Planning Your Research Table 5-2. Experimental Designs and Some Associated Confounds I sinteraction Extraneous effects, involving Testing: variables: selection: Type of e.g., e.g., e.g., experimental approach Selection 6 1,2,3,4 1,2,3,4,5 Pretest-post-test o x o Static group comparison x O + +a _______ __ _ . - 0 Nonequivalent control group + + + o x 0 ++ o Randomized control group R O X O + + + + R O O 0 Note: A minus indicates a weakness in regard to the confound. A plus indicates that the design is resistant to the confound. A blank indicates that the confound is not applica- ble to the design. a. The static group and nonequivalent control group designs are resistant to the effects of extraneous variables to the extent that these variables affect both groups in the same manner. Source: Adapted from Brim and Spain (1974, p. 18). How can you try to get around these confounds? One way that researchers use, particularly when dealing with small numbers, is matching the research subjects, so that for important characteristics, the participants in both groups are more alike. In pair matching, for every teenaged, female, high school dropout in the experimental group, you will have someone with the same characteristics in the control group. However, if you have a lot of factors to control for, you may not find them all neat- ly packaged in pairs of people. An alternative is to ensure that the important characteristics are present in both groups, but separately, at a group rather than at the individual level. Thus, the aver- age age of both groups, level of education, and sex ratio might be the same. An obvious problem with matching is that you might not be able to determine all the relevant characteristics that could have a bearing on your problem. (For an extended discussion of a more complex problem associated with matching when you are not assigning your subjects randomly to the two groups, see Judd, Smith, and Kidder 1991, pp. 118-23.) The other precaution you can take is random assignment, which helps to get around the problem of not being able to identify all the relevant factors in advance. You did this when you assigned your girls randomly to the two groups to avoid having two groups of girls with two very different sets of characteristics. Suppose you found that the experimental group did participate more in class, and then your boss points out that they were all in a single-sex school while the treatment group was in a mixed-sex school. What could you say? (Of course, if you didn't notice something odd about this particular situation to begin with, perhaps you should lie down for a bit.) Planning Research that Explains and Predicts 73 You can also combine pair matching with random assignment. Pair your participants on whatever char- acteristic you think is important-you might pair your girls by ethnic group-then toss a coin for each pair. The girl who gets heads goes into group 1, the other goes into group 2. These are not the only threats to validity. Some arise with the researcher, for example, a researcher could af- fect the results by influencing the subjects in the direction desired, selecting only the experimental results that supported the hypothesis, or interpreting the results in a direction that favored it. Most researchers are ethical, but these threats may occur without the researcher even being aware of it. For example, in a number of studies when researchers doing laboratory experiments were told that their laboratory animals were genetically superi- or or inferior (when in fact both groups of animals were identical), the animals performed well or poorly accord- ing to the researchers' expectations, because unconsciously the researchers treated them differently. Other threats arise with the subjects. People may not behave as they normally do if the conditions of the experiment are different from their ordinary experience. Or they may figure out or think they have figured out what the research is about, and therefore try to do the "right" thing. People unconsciously behaving in special ways because they know they are involved in an experiment is called the Hawthorne Effect from a series of experiments in the 1920s and 1930s in which this happened. When you have dealt with all these problems, you still have one more: external validity. Can your re- sults be said to be true for other girls who were not in your study? Most researchers are not interested just in the groups they studied. They want to be able to generalize the results to a larger setting. In this context you have three issues to deal with: how representative your group is of the larger population (all girls in your country perhaps), whether your study's results will be mirrored in the larger population, and how ar- tificial your experimental conditions were. Representativeness If you were trying to find out whether a vocational training program to provide girls with nontraditional skills leads to an increase in female automobile mechanics, would you be happy to learn later on that all the girls you studied came from a little-known religious group that regards the car as the work of the devil? Or if you wanted to know which of two programs is more likely to encourage people to help the police, and all your participants turn out to be currently active burglars? Your experiment might have high internal validity-the results would be true for the group you studied-but they would not have much general application. Mirroring the Larger Population The issue of whether your results will be mirrored in the larger population is related to representativeness. Let us say that you find that your textbook program increases classroom participation among all but 5 per- cent of your girls. Does that mean it will work on all but 5 percent of girls generally? Most experiments are not drawn on random samples of the entire population, and therefore there is no way to know the extent to which the experiment will work in the larger group. Basing an experiment on a random sample of the entire population is possible (see chapter 6), but as the aim of most experiments is to show cause and effect or some other kind of relationship, this is not always a priority. Artificiality A third problem is that the more carefully you control your study, the more likely you are to be moving away from real life conditions, and therefore the less likely you are to have external validity. Are ordinary teachers going to use a complete set of gender-sensitive textbooks and nothing else as you did in each ex- periment? If not, will they have the same results? 74 Planning Your Research You have to balance internal validity (true for the experiment) and external validity (true for situations outside the experiment). Of course, if your research has no internal validity, there is no point in worrying about generalizing the results to larger situations. If, however, it is impeccably designed, but meaningless in the real world, that is not much use either. In the practical world, there may be a case for sacrificing some internal validity for external relevance. You may not get published in a scientific journal, but you may be able to get some sense of what works in a real situation. Within Subject Designs So far in our experiments involving two groups we have looked at between subject differences, that is, some girls got the textbook program and others did not, and you looked at the differences between the two groups. But you might want to try two different programs on one group. This is within subject differences. Maybe you are a teacher and have only one class of children to work with. Breaking them up into two groups is not practical, because you are trying to teach them. But you are interested in learning incentives, and you want to know whether they do their mathematics homework better if you draw a happy face on their work sheet. One day you give a work sheet with a happy face, the next day you do not. However, other factors might intervene: you might not have a happy face on one of the days, and that might affect the results. Because you cannot assign the students randomly to a treatment and a control group, you could instead give each student a number and assign them all randomly to two groups. On the first day, give one group the happy face papers and the other the plain ones; the next day, reverse the pro- cess with the two groups. That way, even if you looked a bit stern on one of the days, both groups would be equally affected, presumably, and the results of both groups would reflect it. Even if you can manage to control your facial expressions, something else might intervene, like noise or thunderstorms, and this is a good way to ensure that both groups share exposure to the same variables. This approach is also useful when you have small numbers. If you have five or more people in each of the four categories, for a total of twenty people, you can analyze the results statistically. Within subject experiments can be either randomized or nonrandomized experiments. However, the ap- proach has its limitations, just as between subjects approaches do (remember the earthquake variable that you could not manipulate?). Common sense will tell you sometimes: for example, if the treatment has a con- tinuing effect. Let us say you taught your class something one day to see if it had any effect, and did not teach them the next day, then they would still retain the knowledge they acquired the day before. This is called a sequencing effect. Or if you wanted to see whether the happy faces worked better with boys or girls, you would have to do a between subjects experiment, because you could not change the students' sex. In all our examples, we have only used two groups, but we can use more. Suppose, instead of looking at the effect of your textbook program (the independent variable) on classroom participation (the depen- dent variable), you were trying to find out whether the age of the girl (the independent variable) affected the success of the textbook program (now the dependent variable), and you had four age groups to work with: they become Groups 1, 2,3, and 4. (You no longer have textbook and no textbook groups, because you are not examining the effect of the textbooks, but rather the effect of age.) This example raises another point: when the textbook program was your independent variable, you could manipulate it, that is, give it to some girls and not to others. When we start talking about age, how- ever, we cannot manipulate: one is fifteen or one is not. The same is true of gender, for example, or race, or height. These are subject variables, which cannot be manipulated and cannot be used in within subject ex- periments. Also, all our examples have involved only one kind of treatment or independent variable. Experiments that involve two or more independent variables are called factorial designs. Suppose that after all this, your Planning Research that Explains and Predicts 75 boss said that because you had been giving each of the girls in your experiment a vitamin every day, they were generally more alert, and that had a big impact on their classroom participation. You could perform two separate sets of experiments, following one of the designs described above: one in which you did what you have already done, that is, give two groups of girls textbooks/no textbooks, and another experiment in which two groups got vitamins/no vitamins, and compare the results. Another way, however, would be to create one experiment, with all the possibilities included: Factor X: Factor Y: textbooks vitamins Vitamins No vitamins Textbooks No textbooks By assigning your girls randomly to each of the four groups, you can see whether girls with no vitamins and textbooks were as active as girls who had vitamins and textbooks. This is called a two-by-two design, but you can have more levels of the variables, creating, for example, three-by-two designs. The analysis and interpretation of factorial designs is more complicated than that for single variable de- signs: more outcomes are possible and chances for threats to validity to occur increase. CHOOSING YOUR EXPERIMENTAL DESIGN How do you decide which kind of experiment to use? Figure 5-2 summarizes the issues involved. WHAT EXPERIMENTS DO AND DO NOT DO Now we come back to some of the statements made at the beginning of the chapter: experiments do not discover or prove theories or hypotheses. Think of it this way: a boxer cannot prove he is the best in the world. All he can do is refute the claims of others that they are better, and until he is proven wrong, we are more or less content with calling him the world champion. Something of the sort is true for the claims we make for our experiments, but experiments are very good for refuting claims or ruling out alternative ex- planations. Notice that we did not attempt to prove the theory that using gender-sensitive textbooks increases girls' classroom participation, because to do so we would have to examine all girls and all kinds of gender-sensitive textbooks everywhere, past, present, and future. A theory is usually a very general, broad statement. In creating our hypothesis, and therefore making our idea testable, that your new program will increase participationa- mong the girls you studied, you have already limited the scope of your idea considerably. You cannot then leap back to the general theory and argue that you have proved it. (This is called induction-proving the general from the particular.) But we have not even proved our hypothesis. Despite all the precautions we may take, our research may contain errors or be biased. The most our hypothesis can do is to withstand disproof by someone. Experiments don't disprove theories either. Any theory can be converted into a vast number of different hypotheses. You have tested only one quite specific version of the theory. If you say that your experiment has proven the theory, it is like saying that because you have a tiny piece of elephant hide, you have an el- ephant. And if you find that you cannot support your hypothesis, you will not know why; experiments do 76 Planning Your Research not tell you that. All you know is that for some reason, it did not work the way you expected. Any part of the process-the way you defined classroom participation, for example, or the selection of the groups- may have affected the results. The theory may still be true. Figure 5-2. Experimental Design Decision Tree Will you intervene in the research situation? No Yes Are you testing multiple variables? N a- --- Yes Will you have a control group? Hov' many variables? / Ner ~ ~ Yes How many levels for each variable'? I ~~~~~I 12,3et. Will you pretest? Do you want to know (I, 2, 3 etc.) / / \ ~~~~~~differences . No Yes Factorial designs / /tI (2x2, 3x2, etc.) Natural Single Single between groups? -~- experiments group, group, I (ex post facto) post-test pre-test/ Can you assign randomly? within groups? only post-test / No Yes Will the subject(s) Will the subject(s) Can you Can you be the same? be different? pre-test? pre-test? One? More than No Yes No Yes one? Matched subjects Posttes I IRa o design \ | Post-test Rarldom Pre-test/ Are you testing only (not post-test only post-test several levels Single Within \ j commonly (static group of a variable? subject subject used) comparison) design design No Yes Pre-testg Many measures / U|i erse Do youI i post-test (time series J Universe Do you wish to Multilevel measure study) * Nonprobability sample control for design onl (see chapter 6) pretest effects? Y No Y/'es Random Solomon pre-testl four-group post-test design design - Probability sample (see chapter 6) Maybe you do not care about theories, you are only worried about the one situation you are studying. Have you shown that your textbook program increased girls' classroom participation? To answer that you have to satisfy yourself that you have tested all possible alternative explanations for what you found and ruled them out, which, in practice, is impossible. Could it be that just giving girls more attention accounts for the difference? Finally, experiments as such do not discover things. Discovery is an inductive process, in which some- thing is found in one instance that turns out to be true in all cases. Experimental research is deductive, pro- ceeding from the general (the theory) to the particular (the hypothesis). In this respect, we might say that Planning Research that Explains and Predicts 77 experiments are "rigged," from the beginning: certain conditions, certain circumstances, certain procedures are specified, and the outcome we expect happens or does not happen. This is not the same as discovery. Theories deal with cause and effect, but actual research can only show that two or more things go to- gether and that one comes before the other. No matter what precautions you take, an experiment is a simn- plified practical version of the theory. It contains a lot of assumptions and articles of faith, for example, that the threats to validity have been controlled, and that randomization has had the desired effect, that is, you have not ended up with a group of royal princesses who are used to speaking in public and a group of girls who all have laryngitis. The chances of this happening are small, but they exist. So what experiments do is support theories or claims, and through repeated application by other re- searchers, give us more confidence in our line of thinking and suggest new ideas to be examined. SURVEY DESIGNS Surveys, whether in the form of questionnaires or structured interviews (see chapter 9), can take two forms: descriptive surveys, in which the aim is simply to present a picture, and analytical surveys, in which you are trying to show correlation between two things, such as mothers' level of education and the decision to send daughters to school. Correlational research cannot be used to show cause and effect relationships. It shows that two things, A and B, go together. The possibilities for causation seem to be simple: A causes B, B causes A, or something else (C) causes both. Surely common sense should be able to help you. But consider this example from re- search experts Graziano and Raulin (1993). You find that students' reading abilities and mathematics abil- ities go together. Does one cause the other? It is unlikely that high mathematics ability causes high reading ability, but it might be that high reading ability causes high mathematics ability, because reading develop- ment occurs first, and students leam some of their mathematics through reading. You can ask a student: John goes fo ihe markef and buys five tomafoes. Iffhe fomafoes sell for $3 per dozen, how much change should John receive if he gives fhe clerk a $5 bill? This question will measure simple arithmetic, but it also measures something else. Look at this question: Jean va au marche ef achete cinq fomafes. Si les fomafes coufenf $3 la douzalne, comblen de mon- naie Jean dolf-li recevoir s'l donne la vendeuse $5? This is the same question: if you do not know French well, it is more difficult to answer, yet the arithmetic skills involved are the same as in the English question. The lesson is that reading skills are required to per- form well on most tests; therefore, you would find a high correlation between readings skills and most other abilities (Graziano and Raulin 1993, pp. 155-56). A final consideration is that something else causes both A and B, but what? It could be any one of thou- sands of things, none of which may have been examined in your study. Despite these comments, analytical surveys are useful for showing correlations, as the following "exper- iment" shows (adapted from Cole 1980, pp. 31-41). Suppose we did a survey of a sample of 1,000 parents of girls who had just successfully completed the Primary Leaving Certificate. The survey covered urban and rural parents of all ages, some with secondary education and some with less. Half of the parents were Ndebele and half were Shona. 78 Planning Your Research In our survey, one of the things we asked was: "Do you intend to send your daughter to secondary school?" When we looked at the replies, we were surprised to find quite a difference between the Ndebele and the Shona (table 5-3). Table 5-3. Percentage of Parents Who Intend to Send their Daughters to Secondary School by Ethnicity Percentage of respondents in- tending to send daughters to Ethnicity secondary school Number of respondents Shona 72 500 Ndebele 48 500 That is, 72 percent of the 500 Shona and 48 percent of the 500 Ndebele intended to send their daughters to secondary school. Notice the way the table is laid out. You put the independent variable as the first column and the depen- dent variable across the top, so that the table shows the percentage of the independent variable that is re- flected in the dependent variable. In this case, ethnicity is seen as the independent variable: ethnicity "determines" people's intentions to send their daughters to school. It is unlikely that you will try to argue that intentions about sending people's daughters to school deterrnine the ethnic group to which they be- long. This is not just fussiness. The figures will be different if you do the table the other way around: Intention to send daughters to school Percentage who are Shona Number of respondents Yes 60 600 No 35 400 So throughout this exercise, you must be certain that your tables are correctly laid out. Back to our survey. We also found that * More younger parents said yes. * More people who lived within three kilometers of a school said yes. * More parents with a secondary education said yes. We can simply report what we found (description), or we can try to figure out why we found what we did. Why, for example, would more Shona people say yes? Is there something in the Shona tradition that places more value on education? If that is the case, we should try to find out more about the tradition and see if similar ideas can be encouraged in other groups. But let us say we know the Shona and the Ndebele people well, and they both place a high value on education of girls. So it is not "Shona-ness," it is something associated with being a Shona, but what? Maybe more of them have a secondary education, and we know from the literature on girls' education that parents with a secondary education are more likely to send their girls to school. But when we sort our Shonas and Ndebeles by level of education, we find the results shown in table 5-4. Planning Research that Explains and Predicts 79 Table 5-4. Percentage of Parents with a Secondary Education by Ethnicity Percentage of parents with a Ethnicity secondary education Nuinber of respondents Shona 50 500 Ndebele 50 500 Thus we see that it cannot be that more Shona said yes because more had secondary education. So we consider age, thinking that younger parents are more progressive, but when we sort the groups by ethnicity and by age, we find they look more or less like the table above. We can look at the literature on girls' education to come up with possible explanations. One of the strat- egies that has been used to improve girls' participation in education is to locate schools closer to home to safeguard girls' security. Let us form a hypothesis: Shona are more likely to say they are going to send their girls to secondary school because they live in areas where schools are closer to their homes, and people who have schools close to home are more likely to send their daughters to school. We have to find information in our survey to support four statements: 1. Shona are more likely than Ndebele to say they will send their daughters to secondary school. 2. Shona are more likely to have schools closer to their homes. 3. People who live closer to schools are more likely to send their daughters to school. 4. Shona are more likely to say they will send their daughters to school because they live closer to schools. We have already showed in table 5-3 that statement 1 is correct. Now we need to look at statement 2. We sort our two groups by closeness to school (table 5-5), and find that more Shona than Ndebele do indeed live close to a school. Table 5-5. Percentage of Parents Living Close to a School by Ethnicity Percentage living Ethnicity close to a school Number of respondents Shona 85 500 Ndebele 40 500 For statement 3, we need to do a different kind of sorting: of the people who are living close to schools, how many intend to send their daughters (table 5-6)? Table 5-6. Parents Intending to Send Their Daughters to Secondary School by Distance from School Percentage intending to Closeness to school send daughters Number of respondents Less than three kilometers 80.0 625 Three kilometers or more 26.7 375 80 Planning Your Research Thus, of the 625 people who live less than three kilometers from a school, 80.0 percent intended to send their daughters, and of the 375 who lived three kilometers or more from a school, 26.7 percent intended to send their daughters. The last statement, that Shona are more likely to say they will send their daughters to school because they live closer to schools, is not based on any grand theory, although the last proposition in a set of state- ments like this often is. In our case, it may be simply a historical fact that Shona lived closer to main roads to facilitate trading, and that so far the school building program has been concentrated along main roads. We look at our two groups, and pick out the people who are intending to send their daughters to school. How many of these live less than three kilometers from a school and how many live three kilometers or more from a school? Notice that you are now dealing with three variables: ethnicity, intention to send daughters to school, and distance from school, all in one table (table 5-7). Table 5-7. Percentage of Parents Intending to Send Their Daughters to School by Ethnicity and Distancefrom School Percentage intending to send daughters to school Less than Three kilometers Ethnicity three kilometers or more Shona 80 (425) 26.7 (75) Ndebele 80 (200) 26.7 (300) Note: The figures in parentheses show the number of respondents. Thus, 80 percent of the 425 Shona who live less than three kilometers from a school and 80 percent of the 200 Ndebele (340 + 160) who live less than three kilometers from a school intend to send their daughters to school, while 26.7 percent of the 75 Shona who live more than three kilometers from a school and 26.7 per- cent of the 300 Ndebele (20 + 80) who live more than three kilometers from a school intend to send their daughters to school. Clearly the deciding factor is not "Shonaness" or "Ndebeleness," but proximity to a secondary school. This is the process we have followed: We find an independent variable that is associated with the dependent variable; we then find a test factor that might explain the relationship between the dependent and indepen- dent variables. To see if we have found the correct test factor, we examine the relationship between the independent and dependent variables, which remain constant throughout one analysis, separately within each category of the test factor. If the test factor is the right one, the percentage difference should be less in each category than it was in the original two- way table (Cole 1980, p. 41). In our case, in table 5-3, the percentage difference between Shona and Ndebele is twenty points, and in table 5-7 the percentage difference between them in each category of the test factor is zero. In real life for you to get a finding as neat as a zero difference is unlikely. If the difference was large, you would start look- ing for something else as an explanation, but if it was small, although greater than zero, could you be happy with your results? Look at the section on statistical significance in chapter 17. To do research of this sort, your survey has to be comprehensive enough to include questions covering the likely factors, but the possibility still exists that some other factor not covered in your survey explains the findings just as well as distance from school. What you have done here, as you would with an experi- ment, is to rule out factors, not prove. Planning Research that Explains and Predicts 81 FURTHER READINGS TO HELP YOU Abraha, Seged, and others. 1991. "What Factors Shape Girls' Performance? Evidence from Ethiopia." Inter- national Journal of Educational Development 11(2):107-118. Bernard, Russell. 1994. Research Methods in Cultural Anthtropology. Newbury Park, California: Sage. Cole, Stephen. 1980. The Sociological Method: An Introduction to the Science of Sociology, 3rd ed. Chicago: Rand McNally. Graziano, Anthony H., and Michael L. Raulin. 1993. Research Methods: A Process of Inquiry, 2nd ed. New York: Harper Collins. Judd, Charles M., Eliot R. Smith, and Louise H. Kidder. 1991. Research Methods in Social Relations, 6th ed. Fort Worth, Texas: Holt, Rinehart and Winston. ...... *O 6 * - Whom Will You Study? s u m m a r y Some of the words in this chapter have popular as well as statistical meanings: population, universe, sample, random. The statistical meanings are the ones that matter here. You can study the population or universe, that is, all the people or items in a group, or you can take a sam- ple and study some of them. Sampling can be divided into two types. In probability sampling everyone has a chance of being selected, but in nonprobability sampling some people have no opportunity to be included. In a probability sample, in a random sample, everyone has an equal chance of being selected. In a stratified random sample, you can make certain that subgroups you think are important are not left out. You can use cluster sampling when you cannot get a list of people to choosefrom. You use nonprobability samples when you cannot meet the conditions of probability sampling or when you do not need to. Purposeful sampling means deliberately choosing people because they have some character- istic that interests you. Quota sampling, while not a probability sample, is like stratified random sampling in that it allows you to make certain that groups you think are important are not omitted. Sample size is determined by a specificformula rather than by choosing an absolute number or by taking a proportion of the population. This chapter gives you more help with step 7: making decisions about sources of information. Are you go- ing to study everyone within the group that interests you or only some of them? How do you decide? If the answer is only some of them, how do you choose which ones? When you study an entire group, whether the study is of the attitudes of all distance education teachers in Lesotho, of gender sensitivity in all textbooks used in the secondary school curriculum in Mali, or of all schools in The Gambia that have school gardens, you are studying the population or universe, that is, all of whatever there is. So if the number of people or items studied is sufficiently small and accessible that you can question or examine all of them, you are taking a census rather than a sample. When you study the characteristics of only some of the people, situations, or items within the group, you are taking a sample. Sampling has been used to provide some of the major sources of development-related information, such as the World Fertility Survey, the Social Dimensions of Adjustment Survey, and the In- dian National Sample Survey. 83 84 Planning Your Research The words population, universe, census, and sample can be confusing, because they have popular as well as statistical meanings. We think of populations as large and samples as small, but populations can be small, for example, if you want to study only your third grade class, all the children in it form a population, even though the class might only consist of thirty children. A sample, by contrast, can be large. A national sample of third grade children could include hundreds or thousands of children. Samples have a number of advantages. Studying the entire group may take too long and cost too much, both in money and in opportunity time. In an entire region, for example, by the time you asked every wom- an with children under five whether she would use a day care system and what kind of system, those par- ticular children under five might be too old to use it. Obviously, you want your sample results to be similar to those you would have got by studying the mn- tire group, if the entire group is what you are interested in. You cannot ensure that it reflects the population unless you carry out an identical study of the entire group. However, you can develop a sampling plan that enables you to say that your finding will not be different from the universe figures by more than, for exam- ple, 2 percent (this is called the margin of error) more than 99 percent of the time (this is called the proba- bility or confidence level.) Both of these must be set before you can determine your sample size, although these are not the only factors to be taken into account (see Fowler 1988, p. 41 for further discussion). PROBABILITY SAMPLING If you want to be able to specify the margin of error and the confidence level, that is, if you want to be able to say how close your results would be to those obtained from the entire group, you must use a probability sample, which is one in which it is possible to specify for each member of the universe the probability of being selected. This probability need not necessarily be equal. It just needs to be known. Such a sample, if properly drawn and not invalidated by nonsampling errors, allows you to extend or generalize your results to a larger group without studying the entire group. Nonsampling errors are mistakes that have nothing to do with how you took your sample. Some common nonsampling errors occur when a researcher asks ill- conceived questions or records the answers wrongly, or when two researchers administer the same ques- tionnaire in different ways. We will look at this again later. Three basic types of sampling fall into the category of probability sampling: random, stratified, and cluster. Simple Random Sampling Simple random sampling is useful when you have a homogeneous population, that is, the members are similar in terms of the characteristics that interest you, such as level of education. In this kind of sample, each member of the universe has an equal chance of being selected. For example, 200 children are participating in a school feeding program and you want to study the school performance of 50 of them. You can put the 200 names on individual slips of paper, put them in a big bowl or box, and draw out 50. However, a more common approach these days is to give each of the 200 children a number. Then get a table of random numbers, which can be found in many statistics textbooks or which can be com- puter generated, and which will look like table 6-1. Believe it or not, these are not real random numbers. I just made them up for this chapter. Strange as it may seem, however, you cannot make up your own random numbers. You must use a proper table con- structed by people who make their living at this sort of thing. This has nothing to do with union rules. If you make up your own numbers, they will reflect your weakness for certain numbers and your dislike of others. To use a table of random numbers, find a point of entry into the chart using two numbers, say the first two numbers on a piece of paper money, or the last two numbers of the first listing from a telephone directory Whom Will You Study? 85 Table 6-1. Example of Part of a Table of Random Numbers 1 2 3 (etc.) 29 64 73 52 18 11 89 42 67 09 13 49 30 16 54 43 39 80 57 13 1 89 63 27 04 49 02 69 05 99 10 00 89 76 91 02 96 10 49 36 24 16 84 06 19 41 17 83 42 08 83 (chart continues across for a total of 10 blocks) 63 79 41 97 33 62 40 82 59 12 21 45 93 89 40 22 95 80 06 54 2 39 81 66 02 17 ,5 47 03 53 00 46 71 11 50 06 59 27 18 09 35 09 77 30 49 99 14 52 02 40 15 3 (etc.) (chart continues down for a total of ten blocks) opened at random, or the first or last two numbers from an automobile registration number. Let us say the numbers are one and two. Flip a coin to see which will be your first number. Let us say that it is two. Go to the second block across, in the first row (block 2, row 1). Take the first three digits in the first line of block 2.1. (Three because your universe number has three digits. If your universe was 10,000, you would take the first five digits.) The first three digits are 118, so se- lect the person who is numbered 118 on your list. The number below is 433, but you do not have a slip num- bered 433 as yours only go up to 200, so you move on to the first three digits of the next number, 026. In the fourth line you have no nLmber 961, so skip it and move on to the next line for number 178. If you should come across the same number twice, skip it after you have selected the slip the first time. When you get fifty slips, stop. There will be fifty numbers in a column. If you run out, continue upward from your starting block. Then move to the next column if you need more. It is obvious from this that statisticians who specialize in sampling mean business if they are this fussy before you even begin. It is also obvious that you have to get a list of children who are in the school feeding program before you can number them. This list is called a sampling frame. In many instances, no such frame exists, in which case this method cannot be used. Notice that this whole procedure is somewhat dif- ferent from what most people think of as a random sample. Two friends of your mother, a few women from the credit union, and a group of schoolgirls waiting for a bus do not constitute a random sample, or if they do, it would be difficult to say of what. A variation of a simple random sample is a systematic random sample in which you decide your sample size, say 120 in a population of 600. Divide 600 by 120 to get 5, choose any number between 1 and 5 to get the first number in your sample, then choose every fifth number after that until you have selected 120. Problems arise if there is a patterning to the sampling frame. If it contains, for example, a list arranged in order with every fifth person on a list of students being a prefect, you could end up with a sample consisting entirely of prefects. Stratified Random Sampling If your population varies-let us say some have a primary education, some have been to middle school, and some have attended high school-and you want to be sure people from each group appear in your sample, we would divide them into the three groups first. Then, numbering the members of each group 86 Planning Your Researclh separately, choose a random sample of each before combining them again to study them. This way, all three groups will be certain to be in the study. This works if the groups are of roughly equal size. What if they are not of equal size? Suppose you want to find out about attitudes toward girls' education among 1,000 parents in an urban area who come from a variety of ethnic groups. Perhaps you suspect that attitudes vary by ethnicity, and you want to be sure to have people from each group in your sample. You are drawing a sample of 200, and there are only 60 members of the Twi ethnic group and 40 Ga among the 1,000 parents. Your simple random sample might not turn out to be representative of all the ethnic groups in the area, for example, it might contain no Twi or Ga at all. To ensure that this does not happen, divide your sampling frame into ethnic categories, then work out the percentage of the total that each ethnic group forms (6 percent are Twi, 4 percent are Ga), and draw a random sample from each category that reflects its proportion in the population. So your sample of 200 will include 6 percent, or twelve, Twi (6 percent x 200)and 10 percent, or eight, Ga. This is proportionate strat- ified random sampling. Stratified sampling can reduce the costs of sampling considerably. If you wanted to ensure that a simple random sample contained two Ga (or perhaps a disproportionate number, to make a meaningful comparison), you would have to increase the sample size substantially, and would end up with many more people than you wished to have in the other categories just to obtain the desired number of Ga (see Fowler 1988, pp. 24-26, for more discussion of this). Cluster or Area Sampling Even if you had all the money and time in the world, you might not be able to take a probability sam- ple either because a sampling frame does not exist or is flawed. Imagine using the telephone direc- tory for your country as a sampling frame. Who would have no chance to be included in your study? Sometimes, no list of any kind is available: good teachers, market girls in a big city, teenage girls with children. A third kind of random sampling, which helps to deal with this problem, is cluster or area sampling, which is the least expensive type of sampling for large studies, but also tends to be less precise. It is useful when you are attempting to study something for which no sampling frame is available at the highest level from which you wish to sample. For example, perhaps you are studying women taking formal and nonfor- mal technical programs in a region and have no list of their names. in such a case, proceed as follows: * Divide the region into parts such as provinces, territories, or whatever kind of unit is appropriate. Number the provinces and take a random sample of them. * Subdivide the provinces into, for example, educational districts. Number them and take a random sample of these districts. * Find the courses or institutions relevant to your study within the selected districts. If there are many, number them and take a random sample. Go to those that have been selected and study the entire in- stitution, or get the enrollment lists and draw a random sample from them. You may have more or fewer stages than this, but the advantage is that you now have a random sample without ever having had a list of all women taking courses in the region. The disadvantage is that the selected clusters are usually more homogeneous and less representative of their fellow clusters than one might wish. NONPROBABILITY SAMPLING When you use nonprobability sampling, some people will have a higher chance than others of being includ- ed in your study. For example, you decide tc talk to a sample of local schoolgirls at the end of a school day. You put some questions to every tenth girl who comes ou88t the front door. This seems very "scientific" and impartial. Think, however, of all the girls who will have no chance to be included, for example, girls Whom Will You Study? 87 who were kept behind as a punishment; girls who were at home because they were sick, working, or did not have food to bring for lunch; and so on. With a nonprobability sample such as this, you have no way to predict whether the results you got are similar to those you would have obtained if you had used the entire population. Purposeful Sampling Nonprobability samples, such as quota, purposeful, snowball, and convenience samples, are taken when you cannot meet the requirements of probability sampling or are satisfied, for whatever reason, that the group you have chosen meets special requirements that do not involve generalizing your results to the larg- er group. Patton (1990) calls this purposeful sampling: getting "information rich" material from special groups. He lists fifteen different types of purposeful sampling for evaluation studies. Box 6-1 lists some of them. Box 6-1. Examples of Types of Purposeful Sampling Extreme or deviant case sampling the very poor in a village; the very rich; the poor sending all their children to school; the rich sending none. Typical case sampling people studied in greater depth to illustrate what you have found to be "typical." Critical case sampling: selecting a case because it represents a best or worse case scenario: for a program, choosing a village because "if it will work here, it will work anywhere" or because "if it won't work here, it won't work anywhere." Confirming or disconfirming cases: selecting for further examination cases that fit the pattern you have found; cases that do not. Source: Patton (1990, pp. 169-83) For example, you may want to understand the difficulties that handicapped young women face in finding employment, in order to make a film that will educate potential employers. An intensive study of a small number of handicapped women might lead you to a better understanding of their lives than a broad survey of much larger numbers. Sometimes you select a group, such as a community, because it is typical, neither very rich nor very poor. Sometimes you select a group that reflects an extreme, perhaps because that kind of group needs extra help, or because if a program will work there, it will work anywhere. Sometimes you choose a group because a project has worked or has not worked and you want to know why. Quota Sampling One common type of nonprobability sample is the quota sample. You may want to see how poor women's lives are structured, rather than gather extensive survey information to see exactly how many women do what. In such a case identify the types of people from whom you need information, and then find people who represent each of the types. For example, within a region, if you are looking at village women who have never taken courses, you might decide that women with young children, lower caste women, and wid- ows are important to your study. Choose women who fall into each of the types rather than spend time on elaborate sampling procedures. 88 Planning Your Research Other Types of Nonprobability Sampling Both purposeful and quota sampling are nonprobability, that is, not everyone had an equal chance of being included in the study. Other types of nonprobability samples include convenience sampling, in which you just take the handiest people (in development research, however, as Chambers [1983] points out, the danger lies in taking those closest to your office or most like yourself), and snowball sampling, in which one person sends you on to another. Guba and Lincoln (1989) use a snowball sample by asking each interviewee to name someone who would hold very different opinions from those of the interviewee until they have a pic- ture of the concerns, issues, and interests in relation to the groups involved in an evaluation. Sampling often involves a combination of methods, for example, Fluitman and Oudin (1991), in their study of micro-enterprises in Togo, used cluster sampling to get at enterprises, dividing Togo into nine parts, then again into smaller areas, and finally into blocks. Ultimately, the enterprises were chosen by sys- tematic sampling, every nth entrepreneur on a block. However, some enterprises would have suffered in this selection method, for instance, fishsmokers and masons generally worked in marginal areas and might have been missed altogether, so their areas were identified and added. Soapmaking is a "hidden" activity, so the investigators used a snowball method, in which one soapmaker sent the interviewer along to another, and so on. SAMPLE SIZE Most social science research texts do not give you a formula for computing sample size, and this book is no exception. The reason for this is that in the case of probability sampling, choice of sample size is dependent upon having a knowledge of statistics and being able to relate your particular piece of research to the types of statistical tests and procedures that are appropriate to it. This requires an extensive knowledge of all the alternatives, and is particularly important if you are trying to test a hypothesis. Most people, including many researchers, do not have this kind of background. The simplest thing to do, therefore, if you are doing research in which you will eventually have to decide whether to accept or reject a hypothesis, is to be very clear in your mind about what you need to know. Then before you begin to do any research, consult a statistician for advice on the appropriate sample size. (Statisticians tend to get quite annoyed if they are consulted when it's too late.) However, despite this disclaimer, some rough and ready rules of thumb are given later in this chapter. Nonprobability Sampling You will probably be pleased to know that the warning in the previous paragraph does not apply to non- probability sampling. Statisticians have no interest in nonprobability samples, and no rules are laid down for selecting sampling size. Why? If you cannot take a probability sample, you cannot use the statistical tests that allow you to make claims about the representativeness and significance of your findings, so the statis- tical rules for computing sample size are irrelevant. Perhaps, however, the issue is not whether you can or cannot take a probability sample. You may feel that you do not need to. Let us say that you are trying to understand cultural patterns, much as an anthropologist would. Because some ideas and patterns are culturally shared, you can reach the point of diminishing returns, in terms of information, fairly quickly: 10 people may be able to tell you what 10,000 would. Most people, men and women, young and old, in the Ghanaian town of Nsawam will tell you that men do not trade on the roadside and do not do the domestic shopping in the market. That is a cultural pattern or "rule." Asking a carefully selected sample of 250 people would probably not enlighten you much more than asking 20. You can check your information by using other methods, like observing. You will probably see some exceptions. Ask people why. Maybe a subset of rules is Whom Will You Study? 89 involved. Perhaps buying trousers is not domestic shopping or the few men on the roadside are not trading, but "helping out." Probability Sampling Here is where the statistics police will have something to say. The larger your sample, the smaller the difference between the results you will get from the sample and the results you would have got had you studied the entire group or population. This difference is called the sampling error. Obviously, the larger the sample, the smaller the sampling error: if you took a 99 percent sample you could be fairly certain that your sampling error would be very small. But if you are taking a sample, there is little point in taking one that constitutes 99 percent of the entire group. However, you would still like to get a level of precision that makes your findings useful. They will not be perfect, but they will be close enough for what you want. So what is a good figure? Fifty percent? Thirty-five percent? We have asked the wrong question. Many people are not aware that it is not the percentage or propor- tion of the entire group or population that matters. In one of those maddening features of statistical logic that appears to fly in the face of common sense, increasing the sample size proportionate to the entire group does not reduce the sampling error in the way you would expect. Instead, the relationship is based on the square root. This means that to reduce the sampling error by 10 percent you would have to increase the sample size by 100 percent, which in many cases is not worth the extra trouble. Sampling error depends on several things, but not on the proportion that the sample forms of the entire group or population. Instead, it depends on the size of the sample itself. This is a not a particularly difficult concept to grasp once you have a basic understanding of statistics. Most of the books listed at the end of this chapter explain simple statistics, or a statistician can help you, but Casley and Kumar (1988, p. 83) give a surprising example of how this idea works out in practice: "The sample size needed to estimate the birth rate (at a given level of precision) in The Gambia would be the same as that needed to estimate the birth rate in India." In probability sampling, sample size depends on several factors, namely: * How much precision you want between your findings and those you would get by studying the en- tire group. In the formula below, devised by Casley and Lury (1982, p. 76), precision has two ele- ments to be considered, first, the largest difference that you will accept between the value estimated from the sample and the true value in the larger population (D), and second, the level of confidence with which you can say that the result lies within that range of acceptable difference (K). * How much variability (V) exists among the population, for example, in a population where the peo- ple are similar in relation to what you are studying, you will need a smaller sample than one in which variation is greater. You have no automatic way of knowing the variability. You must use your own knowledge of the population, the work of previous researchers, and so on. * The number of variables you are looking at. More variables can require larger samples. Formulas for determining sample size for various kinds of samples are available, but the statistical con- cepts needed to understand them are beyond the scope of this book. You should consult a standard text on statistics or on research methods in sociology or psychology, where samples are commonly used. The follow- ing is a simple formula to show the relationship between the elements outlined above (n means sample size): n = K2 V2 D2 90 Planning Your Research However, table 6-2 is a useful rough guide. Table 6-2. Tablefor Determining Sample Size (s)from a Given Population (P) N s N s N s 10 10 220 140 1,200 291 15 14 230 144 1,300 297 20 19 240 148 1,400 302 25 24 250 152 1,500 306 30 28 260 155 1,600 310 35 32 270 159 1,700 313 40 36 280 162 1,800 317 45 40 290 165 1,900 320 50 44 300 169 2,000 322 55 48 320 175 2,200 327 60 52 340 181 2,400 331 65 56 360 186 2,600 335 70 59 380 191 2,800 338 75 63 400 196 3,000 341 80 66 420 201 3,500 346 85 70 440 205 4,000 351 90 73 460 210 4,500 354 95 76 480 214 5,000 357 100 80 500 217 6,000 361 110 86 550 228 7,000 364 120 92 600 234 8,000 367 130 97 650 242 9,000 368 140 103 700 248 10,000 370 150 106 750 254 15,000 375 160 113 800 260 20,000 377 170 118 850 265 30,000 379 180 123 900 269 40,000 380 190 127 950 274 50,000 381 200 133 1,000 278 75,000 382 210 136 1,100 285 100,000 384 Source: Krejcie and Morgan (1970). SAMPLING AND NONSAMPLING ERRORS Sampling error, as we saw earlier, refers to the difference between the results you got from taking a sample and those you would have obtained if you had studied the entire group or population. For example, per- haps 18 percent of a community of 100 can read, but the results from your sample show that 45 percent can read. This is a serious difference and something is wrong. A difference of 10 percent or more is bad here, and the size of the acceptable difference gets even smaller the larger the sample you use (see Mitton 1982, chapter 11 and appendixes 1 and 2, for clear, step-by-step explanations of these matters for nonstatisticians who need to know more about statistics). Maybe you used an inappropriate sampling design or sample Whom Will You Study? 91 size, both of which are sampling errors. But another possibility exists: nonsampling error. If you used other interviewers to help you, maybe they did not follow your instructions. Perhaps they talked only to the younger people who have had a chance to go to school, and they ignored the elderly, or perhaps people who could not read refused to be interviewed, or maybe the answers were recorded incorrectly. These kinds of sampling errors can and do occur unless you monitor the research closely. But sometimes research in non-Western countries presents additional possibilities for both kinds of errors. In terms of sam- pling error, you may have problems getting an accurate sampling frame or people may be inaccessible, but the possibilities of nonsampling error are even greater, partly because many research techniques are based on assumptions and experiences drawn from Western societies, so that the information you are getting is flawed. Here are some assumptions to watch for: * That the instrument, such as a survey or test, the language, the concepts, and the circumstances of the research are meaningful and acceptable to the person who is responding. * That people are accustomed to making individual assessments or judgments. Doing this as a family or group may be more common. Sometimes people are baffled at the notion that assessments can or should be made. "Are you satisfied with your life?" may be like asking a Westerner, "Are you com- fortable with the notion of gravity?" * That people's answers reflect something meaningful in their lives, and that this corresponds to what the researcher is seeking. Other problematic issues of this sort are discussed in chapters 9 and 10. THERE'S A LOT MORE... Statisticians would not have such a fearsome reputation if this were all there was to the subject of sampling, nor would they be writing books called Statistics Without Tears (Rowntree 1991). For further guidance, con- sult Rowntree or the books listed at the end of the chapter. Once you have the basics, you might find a statistician to advise you on how to apply them to your par- ticular project. Our Girls Remember our girls in X District back in chapter 4? We were looking at the reasons they gave for completing education at Grade 6. How would sampling help us? If X District is large, sampling would be sensible. We could take a sample of the girls, or the school districts, or the schools, or the villages. What type of sample depends on practical questions (can we get a list of all the girls or schools or districts?) On other is- sues, are there very few Muslim schools? If we take a simple random sample of schools, will we be worried if no Muslim schools are in it? FURTHER READINGS TO HELP YOU Bulmer, Martin, and Donald R. Warwick. 1983. Social Research in Developing Countries: Surveys and Censuses in the Third World, chapter 18. Chicester, U.K.: John Wiley and Sons. Casley, Dennis J., and Krishna Kumar. 1988. The Collection, Analysis and Use of Monitoring and Evaluation Da- ta, chapter 6. Baltimore: Johns Hopkins University Press. 92 Planning Your Researchi Casley, Dennis J., and Denis A. Lury. 1982. Data Collection in Developing Countries, part 7, 2nd ed. London: Oxford University Press. Chambers, Robert. 1983. Rural Development: Putting the Last First. London: Longmans. Guba, Egon G., and Yvonna S. Lincoln. 1989. Fourth Generation Evaluation. Newbury Park, California: Sage. Kraemer, Helena Chmura, and Sue Thiemann. 1987. How Many Subjects? Statistical Power Analysis in Re- search. Newbury Park, California: Sage. Mitton, Roger. 1982. Practical Research in Distance Teaching. London: International Extension College Patton, Michael Quinn. 1990. Qualitative Evaluation and Research Methods, 2nd ed. Newbury Park, California: Sage. Rowntree, Derek. 1991. Statistics Without Tears. London: Penguin. Slonim, Morris James. 1960. Sampling. New York: Simon and Schuster. Part III Research Techniques: The Basic Tools e.g... . . 7 *-p, - Selecting Your Techniques and Strategies s u m m a r y The problem or issue you are studying should determine the research technique you use, not the other way around. The information you get will be "stronger" if you use triangulation, that is, as many techniques, researchers, sources of information, methods, and possible explanations as you can. Perfecting your technical research skills is important, but some common biases in getting information can seriously interfere with the validity or soundness of your research. Local people's participation and insights are essential for understanding issues and successfully implementing projects. Key and proxy indicators can help you to do your research more quickly and to do it when what you wish to study is not easily observable. This chapter provides more help with step 8: making decisions about information gathering techniques. YOUR RESEARCH STRATEGY Many people unthinkingly choose their technique before they plan their research. They say, "What we need is a survey on..." or "I'm going to do an experiment to see whether..." Often they choose the technique most commonly associated with their field, for example, participant observation in anthropology, testing and measurement in psychology and education, surveys in sociology, or documentary analysis in history, without considering whether the technique is appropriate for their particular project. Some people choose a technique on the grounds of empathy-they like it, or more likely, they do not like another technique- for example, someone decides to do a study of documents because the office or library is more comfortable than the field. However, picture detectives investigating a murder. Will they be satisfied simply to ask sus- pects whether they had done it? In real life, detectives might ask everyone in the neighborhood to fill out a questionnaire that asked where they were at the time of the murder, if they had seen anything suspicious, and so on. Then they might interview some suspects personally. They would probably watch their move- ments as well and look up old records to see if any of them had committed crimes before. And, of course, they would examine the crime scene carefully to see if it told them anything. 95 96 Research Techniques: The Basic Tools Asking, watching, and looking up are all research techniques, each of which gives you a different angle on the situation. Each research technique is designed to get certain kinds of information and does not get others, for example, questionnaires tell you what people say they think and do, but participant observation can help you to see what people actually do (if they are prepared to let you watch). Using as many approaches as possible is part of a process that many researchers call triangulation, that is, getting the data through a variety of different strategies so as to strengthen and verify the research find- ings (Webb and others 1966, p. 174). Triangulation can be broken down into a number of types. Methodological Triangulation Methodological triangulation is when you use more than one technique to get the same information. Inter- viewing, administering a questionnaire, observing, and examining documents on the same phenomenon provide "stronger" information than using a single technique. Each technique addresses a different aspect of the phenomenon. The use of complementary methods also reveals discrepancies that a single technique might not. For example, an official in the Gezira told Ingrid Palmer (1981, p. 34) that women did not work in the fields, but looking over the official's shoulder, she saw a field full of women working. They were, she was told, "helping out." Palmer was interviewing and observing. Figure 7-1 shows how research techniques complement and support one another. No single technique duplicates exactly the function of the rest. Each technique yields information that only it can obtain, but it also reinforces the other techniques. Figure 7-1. How Different Research Techniques Complement Each Other \ urveys Priiat Each of the petals in the figure is a research technique. At the center they all overlap, and on the sides each overlaps with its neighbors. The clear areas represent the material that this technique particularly ad- dresses; the shaded sections are research areas that can be studied using either, or preferably both, tech- niques; and the center is that part of the research that yields information through use of all the methods. If you had to stake your life on which of these three areas is likely to represent the most accurate, complete research information, you would choose the center, in which you got the information through interviews and questionnaires, reinforced it by observation, and checked it through documentary analysis. In the cen- ter section you are getting not only what people say they do and what you see them doing, but also what Selecting Your Techniques and Strategies 97 they are recorded as doing. The second strongest areas of information are the joint sections, where two re- search techniques can both address the same topic. Finally, in the clear areas you are relying on only one technique, often the only one available for the kind of information you need, but you are getting no verifi- cation from any of the other techniques. Data Triangulation Data triangulation occurs when you * Examine the influence of different times, past and present, on whatever you are studying. * Examine the influence of space, that is, compare the data with that from other places (other villages, other schools, and so on). * Examine the person at different levels: the individual level, the group level, the collective (entire group) level. A father may say he believes that his daughter should get secondary education, but his own parents hold different views, and he can see their point as well. Does he decide at the individ- ual level or the group level in the end? * Examine the situation from different angles: a school from the point of view of the head, the teach- ers, the students, the observer. Researcher Triangulation Researcher triangulation is when you use more than one person to collect the same information to examine the influence of the researcher. One special issue in development-related research is the use of "insider" re- searchers versus "outsider" researchers. An insider can mean someone of the same language group, the sanme ethnic group, or the same profession. The term depends on the circumstances. If you have the chance to work with a team, you will probably see that one composed of both insiders and outsiders gives you a special kind of researcher triangulation. Insiders and outsiders have different ad- vantages and disadvantages. Increasingly, insider researchers, such as local sociologists or anthropologists, are giving us insights that differ from those of outsiders. Insiders are familiar with the situation, can read nuances, and can often be more aware of what people are not saying or doing. They may be trusted more by the participants, and people may feel that because they probably know a lot already, they might as well tell them things they would not tell outsiders. Often, they are more aware of the limitations and constraints of the situation and can make more practical recommendations. The advantages and disadvantages of outsiders are the reverse of those of insiders. They bring a differ- ent perspective to the situation and may see things that insiders take for granted. In addition, local people sometimes see outsiders as an opportunity to compare experiences on a broader scale or talk about things that locals might misread or read too well. Men, in particular, may be prepared to discuss things with out- sider women that would be taboo to discuss with women in their own society. Unfortunately, this is some- times because they see such women either as "loose" or as honorary men. Theory Triangulation Theory triangulation is when you use different or competing theories to try to explain what is happening. Let us say that teachers in a mixed-sex school are treating the girls as though they were less intelligent and important than the boys. The girls are doing poorly on examinations. You might consider two theories, one that argues that the teachers' behavior has no effect (in which case you will look for another explanation), and one that argues that it does. The first theory is called "relative deprivation" theory. People are thought 98 Research Techniques: The Basic Tools to estimate their well-being and satisfaction by comparing themselves to particular peer groups, and not necessarily to people whom they see as being different. The girls, therefore, do not mind this treatment. They would only mind it if some of the girls were treated well and others poorly. You could also look at a theory developed by Robert Park (1928) which, put simply, argues that people behave the way they are treated. In this case, the girls are being treated as if they were stupid, and are performing accordingly. No- tice in this example that you are waiting to get your results before looking for a theoretical explanation. This is an inductive approach. However, you can also start out with a theory or theories, and set up situations in which you can test them. This is a deductive approach (see chapter 2). YOUR RESEARCH TECHNIQUES Table 7-1 compares some research techniques, showing their strengths and weaknesses. Subsequent chap- ters contain practical advice on each technique, but some guidelines and warnings are common to all of them. Table 7-1. The Advantages and Disadvantages of Different Research Techniques Method Advantages Disadvantages Surveys and questionnaires Easy to quantify and summarize re- Hard to obtain data on structure, be- (chapter 8) sults, quickest and least costly way to havior, etc.; little information on con- Schedules of questions with fixed- gather new data, suitable for large texts, situations shaping behavior; choice responses. samples, and useful for repeated not suited for subtle or sensitive is- studies over time, comparisons be- sues; impersonal; risks of nonre- Measures and Scales (chapter 10) tween units, standardized instru- sponse, biased answers, invalid Standardized assessment instru- ments contain pretested items, well- questions; requires skills in construit- ments. suited for studying attitudes. ing instruments and quantitative analysis; danger of overreliance on standard instruments. Interviews (chapter 11) Readily cover wide range of topics Expensive, require skiled interview- Interviewer poses open-ended ques- and features; can be modified to fit ers; respondent and interviewer bias; tions according to fixed list, interview needs before or during interview; can noncomparability of responses in un- guide (list of topics), or on-the-spot convey empathy, build trust; rich da- structured or semistructured inter- judgment. ta; provide understanding of respon- views; difficult to analyze and inter- dent's own viewpoint and pret results. interpretations. Observation (chapter 12) Data on behavior are independent of Constraints on access (timing, dis- Observations of people and their set- people's generalizations, feelings, tance, secrecy, participants' objec- tings. opinions; information on effects of tions); sampling problems; costly, situations; flexible, rich data on range requires trained observers; observer of hard-to-measure topics; generate bias/reliability (getting the same re- insights and new hypotheses. sults on subsequent occasions); may affect behavior of those observed; problems of interpretation, analysis, reporting; may seem unscientific. Secondary Analysis (chapters 8, 13) Unobtrusive, those studied do not Access, retrieval, analysis problems Use of documents, reports, files, un- feel or react to measurement; often can raise costs and time require- obtrusive measurements. quantifiable, repeated measures can ments; validity and believability may show change, participants can help be low and interpretation difficult gather and analyze data, often cheap- when data are not used for original er and faster than collecting new data. purpose; need to interpret in context; limited coverage of many topics. Source: Adapted from Harrison (1984, pp.19-21). Adapted and reprinted with permission of Sage Publications. If you leaf through the remaining chapters in this book, you might think that some of the techniques dis- cussed have been omitted from table 7-1. If you read the literature on educational research and evaluation, you Selecting Your Techniques and Strategies 99 will come across many research approaches, such as project measurement systems, constructivism, and others that do not appear here. The reason is that they are strategies, rather than techniques. Strategies are combinations of traditional techniques, tailored for a special purpose or directed at a special audience. The techniques them- selves are not new. Participant observation, for example, is a long-established strategy. It includes more than simple observation: it involves interviewing, learning from participating, using secondary analysis, and so on to cross-check the results obtained from one technique and get greater depth of understanding. Two strategies that appear to be promising for development-related research on education are benefi- ciary assessment, developed by Lawrence Salmen and used in a number of World Bank projects, and par- ticipatory learning and action (PLA). Chapter 14 is devoted to PLA approaches. Here we will simply show how each strategy meets a need, and accordingly, uses selected techniques. Both beneficiary assessment and PLA try, through relatively quick research, to make development more relevant and sustainable. They recognize the importance of social and cultural factors and the value of local insights and perceptions. The difference lies in their audience: the aim of beneficiary assessment is to pro- duce information that will allow managers and decisionmakers to develop better projects or to adapt exist- ing ones. The aim of PLA is to empower local people to set the research agenda, collect the information, analyze it, and create their own action programs. Each has strengths and limitations: by speaking the language of managers and relating projects to what specific organizations can do, beneficiary assessment ensures that the project is not only meaningful, but deliverable. By contrast, beneficiary assessment does not start from scratch: the organization usually has a project in mind, and the aim is to make it more relevant. It may not be the number one priority of local peo- ple. PLA, however, can discover the priorities of local people, but there may not be an external organization, if one is needed, that can meet some of the needs identified in the way that people want them met. The result is that the two strategies share some common techniques, such as interviewing, observation, and focus groups, but beneficiary assessment is more likely to take an etic perspective ("What do I see these people doing?"), while PLA is more likely to take an emic perspective ("What do these people see them- selves doing?") Beneficiary assessment is also more likely to use surveys, because surveys can cover large numbers, and the proposed project can be extended to a number of sites. Surveys can also produce numer- ical data, and numbers are still valued in the world of large development organizations, because managers want information that is "scientific." PLA uses most of the qualitative techniques described later in this book, often expressed through visual devices, such as matrices and diagramming. These enable local peo- ple, including nonliterate people, to describe and analyze their situation and to work out options. When you come across an approach that sounds like something entirely new, such as constructivism (see Guba and Lincoln 1989), remember that the traditional techniques-asking, listening, watching, read- ing-have not changed. It is how they are combined and for what purpose that makes the strategy new. Before casting your lot in with a new strategy, take care to find out what its purpose is and whether it meets your needs. No strategy meets every need. Style The ideal, when possible, is to get information the way local people do. See if you can adapt any of their methods to the more structured needs and time constraints of your research. People spend a great part of every day gathering enough information about themselves, other people, and their environment to run their lives. They are most comfortable with the methods they use themselves. One of the reasons, for exam- ple, that most people are not particularly happy with questionnaires is that this is not one of the usual ways people anywhere in the world get or give information, whereas interviewing (conversations) and observa- tion are. We still use questionnaires, however, because they have advantages in terms of efficiency, stan- dardization, and cost. 100 Researcih Techniques: The Basic Tools Words and Numbers Some of the techniques in the following chapters are more likely to produce qualitative data-information in words-while others are more likely to produce quantitative data-information in numbers. As should now be clear, one form of information is not more scientific than another. As you will see in chapter 16, ma- terial in words can be converted to material in numbers. It might make it easier to handle, but it does not make it any better. The Instrument The techniques for collecting information presented in part III and the methods for analyzing the informa- tion involve different tools. For example, in informal interviews, the main instrument is the researcher, who develops his or her skill, personality, ability to see patterns, draw conclusions, and so on. With other tech- niques, such as questioinaires, tests, and measures, the main instrument is the device itself. Both involve human judgment, for example, in developing the instrument, knowing how and when to use it, and deter- mining what procedures are best for analyzing the results. The Paradigms In our discussion of the debate in the social sciences todav (chapter 2) we looked at several paradigms, or philosophical approaches to looking at the world and collecting information about it. Most techniques do not fall neatly into one philosophical approach or another. For example, a questionnaire, often thought of as the plaything of positivistically-oriented social scientists, can use categories that have been discovered through phenomenological resea-l-h. By contrast, interviews and participant observation, which may seem to allow much more scope for the individual's perspectives or actions to emerge, may simply be cunningly constructed around the researcher's predetermined categories. Validity, Reliability, Accuracy, and Precision In your research, you need validity (is it sounid?), reliability (would you get the same answers if the re- search were done again?), and to a somewhat lesser extent, accuracy and precision. Think of your watch. It is a valid measure of time, not of height or temperature. It is reliable if it tells you the same time if you look at it exactly twenty-four hours later. It is accurate if the time is right, not 11:30 when it is really 9:45. It is precise if it tells you the exact time down to the smallest fraction of a second. The notion of less accuracy and precision can be upsetting to some people who believe that only absolute precision, down to a decimal point, is scientific. In chapter 14 we look at an idea that might frighten such people: recognizing the princi- ple of "appropriate imprecision"-that is, not being more accurate or precise than necessary, for as Cham- bers (1981, p. 95) has pointed out many times, it is better to be vaguely right than precisely wrong. You probably prefer to be quite right, but you do not necessarily need great precision, for example, finding out how much money, to the penny, people Would be prepared to spend on books, or to the last household, how many children are not in school. General pictures, patterns, and trends and their relationships are like- ly to be more important. Most researchers tend to look for great precision in their own field, but may use fairly rough indi- cators when dealing with material from another field, or when dealing with material that is somewhat marginal to their problem, for example, using great precision to measure reading skills, but paying lit- tle attention to the fact that the measures may contain material that is culturally meaningless to the chil- dren who are being measured. Selecting Your Techniques and Strategies 101 Selection Biases Problems of validity, reliability, accuracy, and precision are aggrevated by some common biases in research. To whom do you talk? Even if you are doing probability sampling, you will encounter many occasions when you want background information, help with questions that have arisen in your research, and in- sights into interpretation, so you will need to choose people to help you. If you are not using a probability sample, you have even more questions about how to select people who will help you. Chambers (1981, 1983, 1991, 1992, 1994a, 1994b, 1994c) has done more than almost anyone to make us aware of the kinds of selection biases most likely to crop up in development-related research. He has ad- dressed most of his cautions to "outsiders," but "insiders" can be equally affected. Some of these biases are presented in box 7-1 and illustrated in figure 7-2. Box 7-1. Causes of Selection Biases Many selection biases arise from the researcher's comfort requirements. It is more comfortable to * Travel shorter distances * Stay on the main road (the tarmac bias) * Talk to people like yourself (the important people bias) * Spend a shorter time * Do your research in the dry season * Avoid the embarrassment of intruding on the poor, or if you are a man, on women * Look at the visible or the easily enumerated (such as school records or min- istry figures) rather than the invisible (such as people's attitudes toward women's intellectual capabilities) * Look at the immediate rather than the trend * Find out what people say as opposed to what they do * Play the important person (the garland bias) The following sections look more closely at some biases. Note that many other kinds of biases are in- volved in research in addition to those mentioned here. Chapters 9 and 11 discuss biases on the part of in- terviewees, and chapter 16 discusses biases that can affect your analysis. NETWORK BIASES Most social research is targeted at people who are less privileged, of lower status, and poorer than the re- searcher. Social researchers usually get to those less privileged through people who have some authority over them, such as employers, school principals, community leaders, union officials, prison wardens, or project directors, in other words, through their own social and professional peers. These people are easy for you as a researcher to get to, and are often more comfortable to talk to. You have a lot in common: profes- sion, social standing, same sex, a common social and professional language. If you are faced with a large, confusing mass of target people and a brief amount of time, letting your peers "translate" the people's con- 102 Research Techniques: The Basic Tools dition, needs and wishes and using their networks is easier. This may give you quicker entry, but you may be narrowing your options in terms of getting representative information, because what you are really do- ing is taking a purposive sample (see chapter 6) without knowing what the selection basis is. As a result, you end up seeing project rather than nonproject situations, adopters rather than nonadopters, successes rather than failures, the exceptional rather than the ordinary. You talk to elites or to the powerful, to men rather than women. This may be all right if you actually need to observe these situations and talk to these people. Placing too much trust in people like yourself is easy, because they speak your language. Unfortunately, they will probably also reinforce your own prejudices and conceptions, accurate or not. You must talk to your peers for many obvious reasons, but their version of reality is not necessarily more accurate than that of the target population. It is just easier to get. It's particularly foolish to talk to men about women's and girls' lives and views unless what you want are men's perceptions. Talk to women and girls. BOTrOM-UP BIASES The bottom-up bias is the reverse of the network bias. It is usually found among people who take a critical theorv approach: they do not want to talk to elites at all, they only want to talk to "the people." They want to work from the bottom up, from the grassroots, rather than the top down. This is usually said with a cer- tain amount of irritating sanctimoniousness, implying that you are doing all your research from a howdah on the back of an elephant. The truth is we have little chance of doing true top-down or bottom-up research in a society of any size. Most researchers are middle class people, and the people at the absolute bottom of the place you are studying are nearly invisible except to the group immediately above them, who are knowledgeable about them because they are desperate not to fall down into this group themselves. You are also unlikely to work from the top down either. True elites are nearly invisible as well except, once again, to the people immediately below them, who are trying to join them. You will, however, be able to talk to people who are in social classes somewhat above you, because your work may be useful to them, because they want a hand in shaping it, because they cannot quite figure out your class if you are an outsider, or because you have come such a long distance on such an odd mission that you must be someone important. Another reason for not working exclusively from the bottom up is that most societies are hierarchical, and their members either respect the hierarchy or fear it. In either case, by bypassing it you are insulting the system, showing that you do not know or do not care how things work, thereby increasing the possibility of your research being sabotaged, reducing the possibility of it ever leading to anything practical, and pos- sibly endangering the people who are participating in your study. Of course, exceptions always exist- working with some hierarchies, such as those who engage in genocide, is ethically impossible-but you have to think your approach out carefully. THE "ONE OF THE PEOPLE" BIAS The one of the people bias is not a selection bias, but a common delusion on the part of researchers. Some people believe that no one understands a particular group of people the way they do. Some anthropologists and "old hands" tend to fall into this way of thinking: "the people" have taken them into their hearts and have made them honorary members. Such researchers then begin to think they are spokespersons for the people and become fanged tigers if any other researcher ventures into "their" territory. It is worth stating here that you will never become one of the people unless you already are. The Irish have a saying about outsiders: "He's more Irish than the Irish themselves." This is not a compliment. Taking this stance affects your ability to be useful to anyone. Selecting Your Techniques and Strategies 103 Figure 7-2. Some Common Biases AtS;~~ '7 7A4 I& '74gm" W&&-74/ O.w,a0 104 Research Techniques: The Basic Tools WORKING WITH LOCAL PEOPLE AND LEARNING FROM THEIR KNOWLEDGE Once researchers realize that their conceptions of organizational structure, motivational systems, incen- tives, and management are often based on Western cultural biases (even if they are not Westerners their professional training is usually embedded in Western concepts), the value of local knowledge, both techni- cal and organizational, becomes clearer. Knowing how groups are structured, how people learn, and how various people fit into the scheme of things are all important in understanding what is going on, and in the case of applied research, are essential for organizing and expediting workable projects. The importance of this topic becomes much clearer in chapter 14. People will usually give you information geared toward your level of understanding, and if you are new to a place and have the cultural knowledge of a four-year-old, they will start from there. Thus, working with local people in determining the problem and gathering information can provide more sophisticated infor- mation in a shorter time. They can help you refine or re-orient the categories of inquiry. Sometimes, researchers require full-time or part-time assistance in carrying out the work, particularly those who are doing surveys. In some cultures, there will be a personal onus on your research assistants to see that you get valid information. However, they may think they should "clean up" their school, village, culture or the respondents' answers to make things look better to you, an outsider. Another possibility is that your choice of local assistants can get you involved, usually without your awareness, in local factions. To achieve anything useful, you have to choose assistants wisely. University students or graduates are not necessarily the best bet. They are often tied up in demonstrating technical expertise, such as doing sur- veys. What you require is someone who is intelligent, astute, and nonthreatening, who may be, say, a mem- ber of a mother's group or a secondary schoolgirl. Whether people of this standing are acceptable to respondents of higher standing is something you will have to find out. Whoever you choose, it is an advan- tage if your assistants understand the larger goal of your research, not just the bit they are doing. DEVELOPING KEY AND PROXY INDICATORS No matter what techniques you use, you cannot look at everything, and anyway, everything may not be readily accessible. So you have to choose what you will ask or look at to give you the best and fastest picture (key indicators) and what you will use to represent things that you cannot observe or ask about directly (proxy indicators). A substantial body of literature on girls' participation in school is now available. As a result, you can assess a particular situation relatively quickly by looking at key indicators such as those researchers Hart- nett and Heneveld (1993) have identified (box 7-2). For other kinds of information you may be able to ask local people to suggest indicators, and then dis- cuss their applicability and exceptions. For example, in one small community all the standard indicators of social class proved irrelevant. The one that worked, devised by local people, was "families who sent their daughters out to work as servants versus families who had other people's daughters as servants." Proxy indicators are substitute measures, in which something observable stands for a more abstract con- cept. For example Hartnett and Heneveld's (1993, p. 7) indicator for accomplishment-success upon leav- ing school-is not as easily measured as the other indicators, so they used employment as a proxy indicator for accomplishment: the percentage of females versus males participating in the labor force, and women teachers as a percentage of the total number of teachers in the school system. Some proxy indicators used to determine "well-being" in Africa over the years have included owner- ship of bicycles and sewing machines in Malawi, possession of tin cans among the Maasai in Kenya and Tanzania, and use of tin roofs in many regions. Table 7-2 shows a set of proxy indicators for well-being in rural communities in Java. As you can see they are relative; they would not mean much in other areas. Selecting Your Techniques and Strategies 105 Box 7-2. Some Key Indicators of Girls' Participation in Schools Access Gross enrollment ratios at primary, secondary, and tertiary level (stu- dents of all ages as a percentage of the populations of official school age) Gross primary admission rate: new entrants (enrollment in the first grade of primary school, less repeaters) as a percentage of the six- year old population Enrollment in sciences at the tertiary level (number of male and female students as a percentage of all male and female students) Attainment Persistence to Grade 4 Completion rates: the proportion of entering students who can be ex- pected to complete the final year of primary and secondary school Continuation rate from the last grade of primary to the first grade of secondary in the following year (and the same from secondary to tertiary) Repetition rates at primary and secondary levels (total number of stu- dents in public institutions who repeat a grade divided by the num- ber of students enrolled) Mean years of schooling: average number of years of schooling per person age twenty-five and above Adult literacy: literate population as a percentage of the population age fifteen and older Accomplishment Labor force participation rate: percentage of the population of each sex that participates in the labor force Percentage of women teachers as a percentage of the total teaching staff at all levels Note: These indicators are available for most countries in Sub-Saharan Africa in Walker (1994). Source: Hartnett and Heneveld (1993. pp.4-7) Proxy indicators only work if they allow you to draw conclusions reliably. They are clues, not causes. Researchers often cite the example of an expert who, seeing that a lot of good schools also had refrigerators, took the presence of a refrigerator in the school as a proxy indicator of a good school. Someone then decided that giving refrigerators to schools was a good way of raising school quality cheaply. Another problem is that proxy indicators tend to be visible assets, which may be convenient for the researcher to observe, but can be highly misleading. A very poor or vulnerable household may inherit a good house, but remain very poor and vulnerable, or a very poor family may be given a radio cassette player by a visiting relative. The assumptions upon which proxy indicators are chosen have to be made very clear, as they are in table 7-3. 106 Research Techniques: The Basic Tools Table 7-2. Measures of Prosperity in Rural Java Prosperity level Indicator Low Medium High House Bamboo Combination Brick and plaster, teak Rooms 1-2, small Many, large Floor Dirt Bricks covered with cement, lime- Polished cement blocks stone blocks Roof Straw, fronds Tiles Windows None Wooden with slats Wooden frames with glass panels Bedding Mats on floor Bamboo slat beds with mats Wooden or iron beds with mattress- es and mosquito nets Lighting Small oil lamps Hanging kerosene lamps Home generator Water source Neighbor's well, river, spring - Own well Toilet Outdoor not enclosed Outdoor enclosed Indoor Transportation None Bicycle, draft cart Motorcycle, scooter, truck, minivan Entertainment None Radio, tape recorder Battery television equipment Refreshment None, tea without sugar Tea with sugar, Tea or coffee with sugar plus snacks served to inter- other sweet drink viewer - No indicator. Source: Soetoro (1979). Table 7-3. Tin Roofs as a Proxyfor Village Welfare Phenomenon Proxy Assumptions Village welfare Tin roofs * Material available * Values and lifestyle support tin roofs * No recent dignitary visit * Development level high enough to allow investment in tin roofs Source: Honadle (1979, p.12). ETHICS When you consider the potential for good or ill of both research and development projects and the fact that you are usually studying people more vulnerable and less fortunate and powerful than yourself, one para- graph on the ethics of your research is inadequate except to guide you to more detailed discussions. The problem is that there is no universal set of ethical principles, or even a core hierarchy of values, that can Selecting Your Techniques and Strategies 107 guide us. Most professional associations have statements of research ethics, which are often debated among their members, and that often are not applicable somewhere or workable in real life. Vexing questions arise. To whom or what are you responsible? The organization for which you work? The sponsors of the research? Your profession or discipline? The people who help you in your research? The answer is all of them, but problems crop a- when any one of these comes in conflict with another. Every circumstance differs. The most vulnerable are probably the participants in your research, and the repercussions of your research on them must always be the first consideration. One cardinal rule is that merely by doing your research, you will raise people's hopes and expectations. Never promise anything you cannot deliver. Wilson (1993) pro- vides a thoughtful view on ethics from the outsider researcher's point of view. But you may be an insider, like Florence Shumba, who, after reading Wilson's article, wrote: The kind of behavior researchers have towards locals tells us that really they just want to exploit them and take from them their ideas and information. It also tells us that they really don't care at all... Not all researchers are exploiters, but most are, and I think it is time up for this now (Wilson 1993, p. 199). What have you learned as a result of being researched? Try to apply the lessons as you begin your own re- search career. For a discussion of these issues and more see works such as Bulmer and Warwick (1983); Cas- ley and Lury (1982); Deyhle, Hess, and LeCompte (1992); Kimmel (1988); or Miles and Huberman (1994). FURTHER READINGS TO HELP YOU Bulmer, Martin, and Donald P. Warwick. 1983. Social Research in Developing Countries: Surveys and Censuses in the Third World. Chichester, U.K.: John Wiley and Sons. Casley, Dennis J., and Denis A. Lury. 1982. Monitoring and Evaluation of Agriculture and Rural Development Projects. Baltimore, Maryland, and London: The Johns Hopkins University Press. Chambers, Robert. 1981. "Rapid Rural Appraisal: Rationale and Repertoire." Public Administration and De- velopment 1:95-106. . 1983. Rural Development: Putting the Last First. London: Longman. . 1991. "Shortcut and Participatory Methods for Gaining Social Information for Projects." In Michael M. Cernea, ed., Putting People First, 2nd ed. New York: Oxford University Press. .1992. RuralAppraisal: Rapid, Relaxed and Participatory. Discussion Paper No. 311. Brighton, U.K.: Uni- versity of Sussex, Institute of Development Studies. .1994a. "The Origins and Practice cf Participatory Rural Appraisal." World Development 22(7): 953-69. . 1994b. "Participatory Rural A1pprtisal (PRA): Analysis of Experience." World Development 22(9): 1253-68. . 1994c. "Participatory Rural Appraisal: Challenges, Potentials and Paradigm." World Developmert 22(10):1437-54. Deyhle, Donna L., G. Alfred Hess, Jr., and Margaret D. LeCompte. 1992. "Approaching Ethical Issues for Qualitative Researchers in Education." Margaret D. LeCompte, Wendy Millroy, and Judith Preissle, eds., in The Handbook of Qualitative Research in Education, pp. 597-641. San Diego, California; London; Tokyo: Acaderiiic Press. Kimmel, Allan J. 1988. Ethics and Values in Applied Social Research. Newbury Park, California: Sage. Park, Robert 1928. "The Bases for Race Prejudice." The Annals 140:11-20. Miles, Matthew B., and A. Michael Huberman. 1994. Qualitative Data Analysis, 2nd ed. Thousand Oaks, Cal- ifornia: Sage. 108 Research Techniques: The Basic Tools Salmen, Lawrence F. 1992. Beneficiary Assessment: An Approacih Described. Washington, D.C.: World Bank, Policy and Social Policy Division, Africa Technical Division. Walker, Eylce M. 1994. "Statistical Indicators of Female Participation in Education: An Update." Washing- ton, D.C.: World Bank, Technical Department Africa Region. Draft. Webb, E.J., D.T. Campbell, R.D. Schwartz. and L. Sechrest. 1966. Unobtrusive Measnres: NTonreactive Research in the Social Sciences. Chicago: Rand McNally. Wilson, Ken. 1993. "Thinking about the Ethics of Fieldwork." In Stephen Devereux and John Hoddinott, eds., Fieldwork in Developing Couintries. Boulder, Colorado: Lvnne Rienner Publishers. What exitng matrials are good for: Getting a background picture Clarifying what you should study Making sure no one has already done what you propose Providing material to be analyzed (entire studies can be done through existing ma- terials) Example: Is there any relationship between family size and children's educatin? Do students who have to repeat a grade drop out rmore frequently than those who do not? Sample study: Uoyd, Cynthia B., and Anastasia J. Gage-Brandon. 1992. Does Sibsize Matter? The Implicaofns of Family Size for Children's Educaion in Ghana. Working Pa- per No. 45. New York: The Populaton Council. 8 Using What Is Already Available s u mm a r y Before collecting your own information, check to see if there are any existing materials that might save you time. Possible sources of data include international agencies, nongovernmental organizations, bilateral do- nors, universities and other research institutions, government organizations, other specialized organizations, and professional journals This chapter will give you additional help with step 7: making decisions about sources of information. Despite the increasing recognition of the need to improve female access to education, female enroll- ments in Sub-Saharan African countries as a percentage of primary and secondary enrollments have in- creased only slightly in the past twenty years. Female wastage or dropout rates are higher and achievement is lower. The factors influencing this have been the subject of much research, but how do you get at it if you need to? USING THE LITERATURE Ignoring the literature can be wasteful of your time and costly. Chambers (1985, p. 405) mentions a failed World Bank rural development project in northern Nigeria that was based on assumptions that three sep- arate studies had already shown to be incorrect. In your case a more likely possibility is that you will end up spending time colecting information that someone else has already collected and analyzed. This chapter is based on the assumption that you may not have easy access to a library that contains jour- nals and books on educational research. University libraries and economic and social research institutions are good sources, but are usually not open to the public. People in university departments of sociology, psy- chology, education, and anthropology are working on the kinds of research described in this book. You will not be ble to match a university researcher in terms of familiarity with the literature or training in research, but if you have a well thought out project, practical experience in your own field, and local knowledge of the place or situation you want to study, you might find that a professional researcher or librarian will find your work interesting enough to give you some time, and perhaps help you to gain access to a library. Also, if you know a researcher at a university is doing work that might help you, write to that person and ask if you can have reprints of articles he or she has written. The reference section of a large general public library 109 110 Research Techniques: The Basic Tools may contain some helpful works as well, such as some of those listed in this chapter. You can use the library as a source of directories and guides to your subject. A friendly librarian is your biggest resource. Although you may have no opportunity to use one now, you should know that the library trend in the future will be toward getting information through computer online databases. These take two forms: online bibliograph- ic databases, which give titles of books and articles, and databases that contain text rather than just titles. These are already established in many large university libraries for academic purposes, but databases will soon be useful for grassroots groups in Africa (see box 8-1). Box 8-1. Online Sources of Information One online database, the Educational Resources Information Center, indexes hundreds of periodicals and hundreds of thousands of research reports on education. You can get access to appropriate materials by entering key words, such as author, title, or subject matter into the computer. The computer then pro- duces a bibliography of articles on your subject. Obviously, choosing sufficiently specific terms is impor- tant: using poverty as a key word would produce thousands of articles and using participatory rural appraisal as a key word would produce dozens. Using the words "and," "or," and "not" can help to limit searches. Usually, a librarian performs the search, but some major libraries permit users to carry out the search. Other online data sources access books, so that by searching several relevant online databases you can get a bibliography of relevant books and articles in, for example, English, French, or both, for a set number of years. Once you have the titles you still have the problem of getting the materials. They may have to be ordered from another library, perhaps a library abroad. Most searches take about fifteen to twenty min- utes. If you do them through a commercial organization the cost can be high, but some university re- searchers who have access to a suitable database may do a search for you. If you have access to the Internet, you can also get at references. One example is a bibliography of par- ticipatory rural appraisal materials (see chapter 14) that can be accessed through the Internet address giv- en in that chapter. Of course, through electronic networks you can also get access to newsletters and can communicate directly with people who share your interests, who may be able to advise you, who can also learn from your experiences. Other databases provide complete documents, rather than titles, abstracts, or raw data. University li- braries, international organizations, governments, and commercial organizations sometimes have this fa- cility. In the last few years, the Women's Research and Documentation Project in Dar es Salaam has been preparing to document information on women's issues and to provide practical resources for grassroots organization by developing online networks and databases. Women in about a dozen other African coun- tries are also organizing to collect computerized information. The Global Fund for Women provides fund- ing for some of these activities. When you are reading someone else's work, ask yourself the following questions (Stewart 1993, pp. 23-24): * What was the purpose of the study and why was the information collected? * Who was responsible for collecting the information? What qualifications were involved? * What possible biases? * What information was collected? How were units and concepts defined? What measures were used? * When was it collected? Was the period unusual in any way? Is the information still timely? * How was the information obtained? * How consistent is the information with other sources? Using What Is Already Available ZZ1 Other sources of information are national statistics departments, government information services, in- ternational organizations, nongovernmental organization documentation centers, and bilateral donors. The rest of the chapter deals with the kinds of information you can get from them and some of its limitations. STATISTICS Statistical information can help you put your own information into context, for example, how is your re- gion, district, or state doing in relation to girls' enrollment at the primary level compared with the rest of the country? They can also help to set targets and to evaluate the outcome of interventions. Statisticians are highly trained people, and enumerators (the people who collect information for them) are carefully instructed, but they are not magicians. Inadequate funding, outdated equipment, questions that have become irrelevant, people who fall through the information net, figures not kept or kept improp- erly, slowness of output, and poor management are some reasons why statistics are not all that we might wish. These problems are not peculiar to developing countries, although some of them may be seriously aggravated in circumstances of little money, poor transport, unreliable postal systems, nomadic people, lit- tle political interest, government manipulation of data, small staffs, and no computerized data processing systems. These problems are compounded by the use of irrelevant concepts as data categories, such as Western kinship units. Even so, industrial countries also have their share of problems, for example, esti- mates suggest that close to 4.5 million people were not reported on the 1990 U.S. census. Also, the researcher who is trying to compare recent records with older ones may find other problems: geographic boundaries may change or categories may be altered. For example, in the U.S. census, the entire coding scheme for oc- cupations changed between 1970 and 1980. Another problem for the researcher in gender-related research is that figures may not be disaggregated (bro- ken down) by sex. In the area of education, you may find that you cannot tell from official figures how many girls versus boys are studying science subjects, or how well each sex has done on examinations. You may have to go back to original records, if you can gain access to them, and if they were collected by sex to begin with. Finally, remember that most official statistics were not collected for the purpose you want to use them for. They are by-products of administration and are shaped by politics, lack of funds, and other consider- ations. As one researcher has said: "Statistics are the poetry of Latin America" (Tannenbaum as cited in Ran- dall 1975, p. 13). Unfortunately, that can be true in other parts of the world too. SOME TIPS If you are collecting figures, try to find out how a major source of information, such as the census or the ministry of education, categorizes information and decide whether comparing it to your results would be useful. If so, categorize your groupings in the same way, for example, age groups should correspond. Col- lecting information on the six to ten age group is pointless if the census groups them in a birth to age ten category, unless you have a good reason (such as children from birth to age six are not of school age, and it is school-age categories that you are looking at). Finally, remember that statistics are your servant, not your master. As Chambers (1981, p. 99) points out, it is important to know what is not worth knowing. Don't collect figures just because they're there. Statistics in Sub-Saharan Africa At the time of their independence, few African countries were collecting information on demographic and social statistics. Although the English-speaking countries had a strong census-taking tradition going back to the end of the nineteenth century, most of it involved simple headcounts. 112 Research Techniques: The Basic Tools The result is that many African countries have not issued basic information, such as annual yearbooks or trade statistics, since 1983. A similarly dismal picture exists for education statistics. As many countries in Sub-Saharan Africa expanded their education programs, especially at primary level, and many private schools were established, the work of compiling reliable statistics on school enrollment by level, age, and sex became more difficult. For some countries, published enrollment figures for primary and secondary lev- els do not accurately reflect the true situation, because corresponding figures from private institutions are incomplete. In 1981 statistical experts Kpedekpo and Arya (1981, p. 74) reported serious gaps: Such statistics as are available are significantly more complete for formal than for informal educa- tion and for maintained than for unassisted institutions.. .Published statistics tend to concentrate too narrowly on enrollments, projected enrollments and pupil-teacher ratios, to the neglect of other vital information. For example, the data available on the costs of education are inadequate, and no solid studies seem to exist on the relative costs of education between primary, secondary and high- er education, or between rural and urban institutions, between comprehensive and general second- ary education, between vocational and academic education, or between vocational on-the-job training and the formal vocational education system. The United Nations Educational, Scientific, and Cultural Organization (UNESCO) is now trying to help countries to improve their statistics through a major technical assistance project for strengthening national statistical information systems for planning and managing education. Censuses Let us now look at some of the statistics that are available. The Handbook of National Population Censuses: Africa and Asia (Domschke and Goyer 1986) provides a background to censuses and some national demographic surveys for each of the countries of Africa. Despite the fact that it is no longer up-to-date, it gives a useful historical account of each census taken in every coun- try, explains how the terms were used, indicates anything unusual about the census, and provides an esti- mate of the quality of the census. Not all the countries have published census information. For example, the Comoros and Congo have no published material, although basic data may be available to researchers. As of 1986, some countries had only one published census (Cameroon, Cote d'Ivoire, Guinea, Mauritius, Niger, Somalia, and Zaire.) Others have had as many as six since the 1940s, for example, Reunion, Tanzania, and Zanzibar. Some countries such as Lesotho have taken censuses every ten years. For the most part, for a va- riety of reasons, census dates have little regularity across Africa or within particular countries. Because most countries follow international guidelines for census definitions, concepts, and data collec- tion methods, the actual data may be internationally comparable to some extent. Among the information that may be found in African censuses is the following: actual or present residence at the time of the census, usual residence, place of birth, duration of residence, place of previous residence, urban/rural, sex, age, re- lation to household head, marital status, children born alive, children living, citizenship, literacy, school en- rollment, educational attainment, educational qualification, ethnic/racial origin, language, religion, household composition, economically active/inactive, occupation, industry, occupational status, income, and housing. Not all country censuses contain information for each of these topics. In relation to education, for exam- ple, Burkina Faso has information on literacy only. Table 8-1 presents two examples from the handbook, showing the types of information collected for two countries: Central African Republic and Kenya. The handbook contains pages like this for every country in Africa. Using What Is Already Available 113 Table 8-1. Sample Census Categories: Central African Republic and Kenya, Selected Years Central African Republic Kenya Year 1961 1975 1962 1969 1979 Actual residence x x x x Usual residence x x x x Place of birth x x x x Duration of residence x x Place of previous residence x x Urban/rural x x x x Sex x x x x x Age x x x x x Relation to household head x x x x Marital status x x x x x Number of children born alive x x x x Number of children living x x x x Citizenship x x x Literacy x x School enrollment x x x Educational attainment x x x x Educational qualification Ethnic/racial origin x x x x Language x Religion x Household composition x x Economically active/inactive x x x Occupation x x x Industry x x Occupational status x x Income Housing x x x Information available in census. Source: Domschke and Goyer (1985). Reprinted and adapted with permission of Greenwood Publishing Group, Inc., Westport, Connecticut. 114 Research Techniques: The Basic Tools NATIONAL STATISTICAL OFFICES Most Sub-Saharan African countries have a national statistical office or department of statistics that has detailed local information about student numbers by sex and age in each grade, in each type of school, and at each level; numbers of repeaters and dropouts; and so on. Such detailed information will probably not appear in the census itself. You might inquire how to get access to this. Is it through the statistical office, the ministry of education, the ministry of planning, or some other source? Perhaps more important, national statistical offices are good sources of recent statistical information other than that gathered in the national censuses. For example, the World Bank's Social Dimensions of Adjustment Surveys and the Living Standards Measurement Surveys contain extensive information on education. Other sources that your national office may have include the Demographic Health Survey and the National Household Survey Capability Program for Africa. NATIONAL GOVERNMENT PUBLICATIONS Many countries publish useful reports and studies, either through the relevant ministry or department, or through a central publications or information office. Working parties may have been convened to look at topics such as sexism in textbooks or how to integrate family life education into the curriculum. Write to the appropriate body to see if anything has been published that might help you in your study. Such reports and studies will often mention other reports, and you can begin to collect useful materials in this way. INTERNATIONAL AGENCIES Many of the major international agencies and organizations are now focusing some of their research efforts on problems of girls' education. This section briefly discusses some institutions that have policies or studies on education in Sub-Saharan Africa related to the problems girls face. The World Bank World Bank initiatives aimed at encouraging girls' attendance at school, although expanding, are still at an early stage. They are usually included as components of general education projects. Most activities are designed to increase educational opportunities for girls. They include a number of pilot programs, including provision of satellite schools to bring primary schools closer to the homes of young girls, thereby making it more likely that they will attend; financing nonformal education programs to reach very poor girls unable to attend formal schools; launching experiments designed to make formal government schools more attractive to girls; hiring more women teachers; continuing scholarship programs for secondary school girls; removing sexist language from textbooks and including positive role models for girls; and increasing the adequacy of physical facilities for girls and women teachers. The World Bank has also worked to develop more community involvement: some programs include the School Development Fund in Senegal and information campaigns in Burkina Faso, Chad, and Senegal. Finally, the Human Resources Division of the World Bank's Economic Development Institute has held regional seminars on girls' education in Africa, which have been followed by national seminars in which individual countries assess their situation and develop action plans. Khoudari (1993) recently reviewed the Bank's experience with assisting girls' education for one of the Africa departments. Box 8-2 lists some World Bank studies and policy papers on female education. Using What Is Already Available 115 Box 8-2. Examples of World Bank Studies and Policy Papers on Education Hartnett, T., and W. Heneveld. 1993. Statistical Indicators of Female Participation in Education in Sub-Sahara Africa. Technical Note No. 7. Washington, D.C.: World Bank, Africa Technical Department, Hu- man Resources Division. (Also available in French.) Herz, B., K. Subbarao, M. Habib, and L. Raney. 1991. Letting Girls Learn: Promising Approaches in Primary and Secondary Education. Discussion Paper No. 133. Washington, D.C.: World Bank. (Also avail- able in French.) Hyde, Karen A.L. 1989. Improving Women's Education in Sub-Sahara Africa: A Review of the Literature. Popu- lation and Human Resources Department, Education and Employment Division Paper Series No. 89/15. Washington, D.C.: World Bank. King, E., and M. Hill. 1991. Women's Education in Developinig Countries: Barriers, Benefits and Policies. Wash- ington, D.C.: World Bank. Lockheed, Marlaine E., Adrian Verspoor, and others. 1991. Improving Primary Educationz in Developing Countries. Washington, D.C.: World Bank. Odaga, Adhiambo, and Ward Heneveld. 1995. Schools and Girls in Sub-Saharan Africa: From Analysis to Ac- tion. Africa Technical Discussion Paper. Washington, D.C.: World Bank. Subbarao, K., and L. Raney. 1993. Social Gainsfrom Female Education: A Cross National Study. Discussion Pa- per 194. Washington, D.C.: World Bank. Summers, Lawrence H. 1992. Investing in All the People: Educating Women in Developing Countries. Working Paper No. 905. Washington, D.C.: World Bank. World Bank. 1994. Questionsfor the Analysis of Female Participation in Education in Sub-Saharan Africa. Wash- ington, D.C.: Africa Technical Department. The address of the World Bank is The World Bank, 1818 H Street N.W., Washington, D.C. 20433, U.S.A. The African Development Bank The African Development Bank sector policy papers include studies of gender equity that will guide it's funding policies. Support for projects that try to remedy inequalities and to ease enrollment in institutions in which women have been underrepresented will be a special priority. Consult the African Development Bank office in your own country, or write to The African Development Bank 2001 Pennsylvania Avenue N.W., Suite 350 Washington, D.C. 20006 U.S.A. The United Nations Education, Scientific, and Cultural Organization Since 1960, most Sub-Saharan African countries have invested a considerable share of their resources in one or more national censuses and surveys and have established a statistical infrastructure to continue a pro- gram of data collection. UNESCO statistical yearbooks provide information and analyze data from these sources on the social and economic characteristics of the population. Box 8-3 provides some examples of UNESCO publications. UNESCO/BREDA, UNESCO's regional office in Dakar, has published a series of documents on the educa- tion of women in Africa. Some titles in the series are shown in box 8-3. 116 Research Techniques: The Basic Tools Box 8-3. Examples of UNESCO Publications UNESCO. 1972. Haute-Volta: projet experimental d'egalite d'acces des femmes a l'education. Series No. 2794/ RMO.RD/MC. Paris. .1975. Report on the Relationship between Educational Opportunities and Employment Opportunities. Par- is. . 1983. Identification of Sex Stereotypes in andfrom School Textbooks. Paris. . 1991. World Education Report. Paris. UNESCO-UNICEF (United Nations Children's Fund) Co-operative Programme. 1988. Women's Education in Africa: A Survey of Field Projects in Five Countries. Paris. UNESCO's address is United Nations Educational, Scientific and Cultural Organization, Place de Fon- tenoy, 75770 Paris, France. Examples of UNESCOIBREDA Publications Bouya, A. 1994. African Girls and the School Science and Techinology Curricula. Kainji, K., and J.N. Chikhunga. 1994. Access of Girls to Primary and Basic Educationi in Malawi. Maiga, F.D. 1993. L 'Acces des Filles a l'Education de Base au Gaboni. Phillott-Almeida, R.A. 1994. A Profile of the Roles of Women as Economic Producers and Family Supporters in The Gambia. Tchombe, T.M. 1994. Access of Girls to Basic Education in Canieroon. The address for UNESCO/BREDA is UNESCO/BREDA, 12 Avenue Roume, BP 3311, Dakar, Senegal. United Nations Children's Fund Research by the United Nations Children's Fund (UNICEF) has identified the general factors affecting fe- male participation in education and the interventions that aim to increase it. Some of the interventions that have been tried around the world include recruiting women teachers, establishing day care centers to allow girls with children to attend school, building appropriate facilities, and providing scholarships to girls. Box 8-4 lists some of UNICEF's studies. Box 8-4. Examples of UINICEF Studies UNICEF. 1991. Country Reports: Sub-Saharani Africa. New York. . 1991. The Girl Child: An Investment in the Future. New York. _.____ 1992. Eduicating Girls and Women: A Moral Imperative. New York. . 1994. Strategies to Promote Girls' Education: Policies and Programs that Work. New York. UNICEF's address is The United Nations Children's Fund, Programme Publications, 3 U.N. Plaza, New York, NY, 10007, U.S.A. BILATERAL DONORS Many national donors provide support for female education activities. Among them are the international development agencies: the Canadian International Development Agency, the Danish International Devel- opment Agency, Irish Aid, the National Advies Raad voor Ontwikkelingssamenwerking (NAR) (Netherlands), Using What Is Already Available 117 the Norwegian Agency for International Development, the Swedish International Development Authority, and the U.S. Agency for International Development (USAID). Two examples of the work of such donors follow. United States Agency for International Development In addition to activities designed to increase access, USAID has designed several projects to increase the physical and social well-being of girls, such as removing sexist language from textbooks and including pos- itive role models for girls; increasing the adequacy of physical facilities for women in primary teacher train- ing institutes; including population, health, and nutrition information in schooling; and training teachers to be more aware of gender-related issues in the classroom. Box 8-5 lists some USAID studies. Box 8-5. Examples of USAID Studies Adams, F.M. 1980. Early Education for Girls: A Priorityjor Literacy. Washington, D.C.: USAID. Anderson, M.B. 1986. The Gender Manual Series: Gender Issues in Basic Education and Vocational Training. Washington, D.C.: USAID, Office of Education and Women in Development. Floro, M., and J. Wolf. 1990. The Economic and Social Impacts of Girls Education in Developing Countries. Wash- ington, D.C.: USAID, Office of Education and Women in Development. Tietjen, K. 1991. Educating Girls: Strategies to Increase Access, Persistence, and Achievement. Washington, D.C.: USAID. Also available from USAID is a useful series of small handbooks on research methods by Krishna Kumar, each on a separate topic, such as interviewing or surveys. USAID's address is United States Agency for International Development, 320 21st Street N.W., Washing- ton, D.C. 20523, U.S.A. The Swedish International Development Authority SIDA provides support for female education activities and cooperates with Sub-Saharan African countries. Box 8-6 presents some sample works sponsored by SIDA. Box 8-6. Examples of SIDA Publications Mbilinyi, M., and P. Mbughuni, eds. 1991. Education in Tanzania with a Gender Perspective: Survey Report. Education Division Documents. Dar es Salaam and Stockholm: SIDA. Palme, M. 1993. The Meaning of School Repetition and Dropout in the Mozambican Primary School. Education Division Documents No. 60. SIDA's address is Swedish International Development Authority, Birger Jarlsgatan No. 61, S-10525, Stock- holm, Sweden. OTHER ORGANIZATIONS The Donors to African Education Working Group on Female Participation, composed of twenty-three multi- and bilateral funding agencies, actively promotes female education. The working group works with African education officials and scholars to develop national education policies that will enhance girls' par- Z18 Research Techniques: The Basic Tools ticipation in education. One objective of the program is to strengthen local capacity for conducting indepen- dent research into strategies for closing the education gender gap in Africa. Currently, fifty-five major projects are being funded under this program. The working group is also carrying out exploratory work on science and mathematics education for girls and on the role of nongovernmental organizations in promot- ing girls' education (Odaga and Heneveld 1995). The Rockefeller Foundation serves as convener and lead agency for the Donors to African Education consortium. For more information, write to the foundation. Dr. Joyce Lewinger Moock The Rockefeller Foundation 420 Fifth Avenue New York, NY 10018 U.S.A. or Dr. Katherine Namuddu The Rockefeller Foundation P.O. Box 47543 Nairobi, Kenya You might also write to the African Academy of Sciences/Donors to African Education Small Grants Program: Program Officer Research Priorities for Education of Girls and Women African Academy of Sciences P.O. Box 14798 Nairobi, Kenya The Forum for African Women Educationalists (FAWE), a pan-African nongovernmental organization, addresses gender disparities in the field of education, influences educational policies, and mobilizes re- sources in support of female education in Africa. It consists of the highest-serving women in ministries of education, including serving female ministers and deputy ministers, as well as vice-chancellors and deputy vice-chancellors, and prominent women educationalists. It currently consists of thirty-two members in twenty-three countries, plus fourteen national chapters. The forum's work program covers five major areas: strategic resource planning, seed grants for country experiments, advocacy and public information, awards for innovators, and targeted capacity building and leadership. Recently, the forum has begun to produce a newsletter. For more information, write to the forum. Forum for African Women Educationalists P.O. Box 53168 Nairobi, Kenya RESEARCH PUBLICATIONS AND OTHER MATERIALS In addition to statistics, you can get useful published material from scholarly books and articles in professional journals; from national sources, such as reports of government departments and Using What Is Already Available 119 nongoverrnental agencies; from international sources, such as United Nations agencies; and finally, from unpublished sources, such as internal reports and working papers. If you are in doubt about the reliability of a publication, such as a magazine or newspaper article, ask someone who is knowledgeable about the field. Above all, remember that just because something has appeared in print does not mean it is valid or accurate. Books At the end of each chapter in this book you will find titles of some important books on research methods or studies that have used these methods. Few libraries in the world will have all of them. To write this book I had to order many of them through an interlibrary loan system. Your public library or university library may be able to get a book for you this way, although a fee may be charged. If you have no other alternative, you might have to buy the book. Books in Print, found in most large public libraries and in very large book shops, contains publishers' addresses. Some Major Research Journals in Education You may also have to order particular issues of journals through an interlibrary loan system or write to the publisher. Some important journals in the field of education are * American Educational Research Journal * Anthropology and Education Quarterly * Child Development - Comparative Education Review * Educational and Psychological Measurement * The International Journal of Qualitative Studies in Education * Journal of Educational Psychology • Journal of Educational Measurement * Journal of Experimental Education * Journal of Research and Development in Education * Review of Educational Research * Zimbabwe Journal of Educational Research Some Free Publications The International Institute for Environment and Development publishes material on rapid and participa- tory approaches (see chapter 14). RRA (Rapid Rural Appraisal) Notes, a series of twenty-one volumes to date, is free to readers in the developing world. In 1995 the series was renamed PLA (Participatory Learning and Action) Notes and will be published three times a year. The address is PLA Notes 3 Endsleigh Street London WC1H ODD England 120 Research Techniques: The Basic Tools The African Research Utilization Network (ARUNET) is a regional network in Kenya, Tanzania, Ugan- da, Zambia, and Zimbabwe that is dedicated to bridging the gap between development research and the implementation of the results of that research. It provides training in participatory approaches and spon- sors a free quarterly newsletter for readers in the region. For more information, contact The Regional Secretariat African Research Utilization Network P.O. Box 43864 Nairobi, Kenya "GRAY" LITERATURE One category of information that does not fit neatly in with known and established archival material (like government statistics) or with published works is the so-called gray literature. This includes working pa- pers, internal documents, house organs, annual reports, and so on. Unpublished reports and studies by ma- jor international agencies are an important source. Some bodies, such as the European Union, have online databases for their gray literature. In other cases, you have to know of a study's existence and write to the organization concerned, which may or may not provide it. OTHER WRITTEN MATERIALS Researchers use many other kinds of written materials, ranging from school records to newspaper articles to minutes of meetings. These can be very helpful, sometimes on their own (particularly in studying the past, when the people involved may not be living) or in combination with other research techniques. Chap- ter 12 looks at some of these. Finally, do not overlook literary sources and materials that local people may have written, such as po- etry or diaries. Remember our debate about facts versus meaning in chapter 2? You can use materials like these to bring your information to life and help people to get a feeling for what your facts mean. Mathabane (1994) is a good example. Also consider the power of these two instances: We shall overcome We shall overcome We shall overcome, some day... and Question 10. Do you think you will ever overcome? Yes No Don't know IF "YES" TO QUESTION 10, ASK RESPONDENT When do you think you will overcome? Now _ Some day While a picture may be worth a thousand words, a poem, short story, or piece of music may convey more meaning than a thousand survey questions. Indeed, the main message that a thousand survey questions conveys is that you have not planned your survey well. Certainly you need to know how Using What Is Already Available 121 representative your information is, but you also need to know, and to show, what it means. Literary and artistic works can help to make your case. Once you are satisfied that you have what you need from existing sources, chapters 9-14 will help you to make decisions about how to choose other information gathering techniques (step 8, box 3-1). FURTHER READINGS TO HELP YOU Colclough, C., with K.M. Lewin. 1993. Educating All the Children: Strategiesfor Primary Schooling in the South. Oxford, U.K.: Clarendon Press. Domschke, Elaine, and Doreen S. Goyer. 1986. The Handbook of National Population Censuses: Africa and Asia. New York and London: Greenwood Press. Khoudari, K. 1993. Approaches to Improve Female Attendance to Primary School: A Review of Recent Bank Expe- riences. Washington, D.C.: World Bank. Kpedekpo, G.M.K., and P.L. Arya. 1981. Social and Economic Statisticsfor Africa. London: George Allen & Un- win. Mathabane, Mark. 1994. African Women: Three Generations. London: Hamish Hamilton. Odaga, Adhiambo, and Ward Heneveld. 1985. Schools and Girls in Sub-Saharan Africa: From Analysis to Ac- tion. Discussion Paper. Washington, D.C.: World Bank, Africa Technical Division. Stewart, David W. 1993. Secondary Research: Information Sources and Methods, 2nd ed. Newbury Park, Cali- fornia: Sage. What the urveys and stnuctured lnarvie ae good for Gettng anawsw to the same quetons fom many people. Relating arwers to one queaton (sex?) to }os of anotder (lee of schooling?) What are tho ducational needs of market girls In a town? Do mother' vws on vsb aspects o girl' ducaton dfer from thoe f the? What do teachers think are toe main reasons why girls are eoving scdool? What do they think would help? Sample stdy: Okumu, Mary I., P. Yourl, and African Medical and Resarch Foundation. 1994. Fe- male Adoescent Health and Sexuaft In Kenyan Secda,y Sdil A Suvey Report Nairobi, Kenya: African Medlcal and Research Foundaton. e.g... 9 *05 Surveys summary Surveys are useful when you want responses to the same questionsfrom a number of people. However, many of the assumptions upon which surveys are based rely on Western concepts. You must adapt the rules and processes to your own society. Surveys take twoforms: structured interviews, which the researcher administers, and questionnaires, which respondentsfill in themselves. Mini surveys (small number of questions, small number of respondents) are useful for short projects and can be analyzed easily without a computer. Work out the information you need by consulting your research outline or using a similar procedure. Decide to whom the survey will be given, and whether you will ask everyone in the group or a sample. Avoid poor questions and poor answer choices: inappropriate questions and answers, double questions, load- ed questions and answers, vague words, overly broad questions, and questions requiring second-hand infor- mation. Answers can be open-ended (blank spaces provided) orforced choice (answer choices provided). A survey may contain a combination of both. When you are ready to design the survey, assemble the instrument: create a statement explaining the research to the respondent (aface sheet); plan the order of your questions; and usefilter questions, which guide peo- ple through the interview. Decide how to prepare answersfor counting. Pre-test the survey. Select and train the interviewers if you are not working alone. Administer the survey. Process the data: count the answers and do any cross-tabulations you need by relating a person's answer to one question to their answer to another. You can do this yourself or use a computer. SURVEYS AND QUESTlONNAIRES One of te bes t ways to Wm a bt uing sueys is to be on te or g reoavmg end of a bad one. Fig 9-1 is an example cia bad suivey. Fowg the suvey you will see how it is made worse by a poor intetvewer. 123 124 Research Techniques: The Basic Tools Figure 9-1. Example of a Bad Survey Greeting and Brief Introduction Surname and Christian Name? Age? (CIRCLE ONE) 20-25 25-30 30-35 35-40 40-45 45-50 50-55 Sex? (CIRCLE ONE) Male Female Do you have any children? How many girls? How many boys? How many of the girls are in school? not in school? ___ How many of the boys are in school? _ __ ___ not in school? What is the main reason why your daughters are not in school? (CHECK ONE) - too expensive - school too far away l_ l - the girls got pregnant - the girls are married - other _ What is the main reason why your sons are not in school? - too expensive - school too far away - needed at home - other Do you send your daughters to school and study sessions and school events regularly and supervise their school work? - Yes -No C Surveys 125 Here's the interviewer (I) and a respondent (R): I. Good morning. I am here from the ministry to ask you some important questions. Surname? R. Good morning. I. Surname? R. What are the questions about? I. The ministry wants to know why people aren't sending their children to school. R. But I am sending all my children to school. Why have you chosen me? I. Because this is a random sample. R. Can't you ask the people in the next house? They don't send all their children to school. Besides, I myself am not an educated person. I can't answer your questions. And look, my animals have broken down the fence, I must save my garden. I. This is more important. And anyone is intelligent enough to answer these simple questions. Name? R. Will the ministry record my name in its books? I. No. Surname? R. Then why do you want my name? 1. Because it says "Surname "here on the form. R. Well... my name is Phiri. 1. What is your Christian name? R. But I am not a Christian. No one in the village is. 1. What is your first name then? R. Joseph. 1. Age? R. Forty-five. 1. Wait a minute. Where do I put that on the form? It says here 40-45 and 45-50. Which is it? R. I can't help you. T. Sex? R. I am sorry, this is unacceptable. I cannot discuss these matters with a lady. I must leave. I. I mean WHAT IS your sex? It asks it here on the form. R. What do you mean what is my sex? What do I look like? I. Number of children? R. Eleven. 1. ELEVEN? That's too many. What about your poor wife? R. I have two wives. I. That's even worse! How many daughters? R. Eleven. I. How many sons? R. None. I already said I have eleven children, and they are all daughters. 126 Research Techniques: The Basic Tools I. Well, it asks it here on the form. This was written by an eminent professor at the university. How many of them are in school? R. All of them. I. How many of your sons are in school? R. I told you. I don't have any sons. I. What is the main reason why your daughters are not in school? - too expensive - school too far away - the girls got pregnant - the girls are married - other R. Oh, the school is certainly too expensive all right. That's a very good reason. I. I don't know where I'm supposed to put that answer. These boxes are all mixed up. Anyway, what is the main reason why your sons are not in school? - too expensive - school too far away - needed at home - other R. I told you-oh well-"other." I. Wait. I see here you have no sons. R. Then why did you ask me the question? I. Do you send your daughters to school and study sessions and school events regularly and su- pervise their school work? R. Oh, yes, regularly sometimes. Other times, not. And my wife supervises the school work. I. "Yes" or "no"? I need to fit the answer into this box. R. "Yes." Some of the problems in the interviewing technique and the survey form may be obvious to you already. By the end of this chapter, you should be able to identify other weaknesses in both. The survey has been the backbone of social science research in developing countries for several decades. International agencies, national governments, and academic researchers are all heavily involved in large surveys such as the Living Standards Measurement Study, the Social Dimensions of Adjustment Survey, to name but two. However, the survey has also been a bone of contention among social scientists. The debate in chapter 2 about approaches to knowledge and methods of analysis really comes alive when social scien- tists start discussing the merits and demerits of the survey. Development researcher Ingrid Palmer has summarized the argument about the empirical, hard ques- tionnaire versus the qualitative, holistic approach: Charges fly that the first gives a historical, snap shot material, is weak on relations and pro- cesses, accumulates excess information which clogs computers and litters the floor of the cut- ting room, and stupefies the mind during the interpretation stage. The other side claims that facts is facts, that qualitative interpretation lacks scientific rigor and is prone to non-represen- tativeness (Palmer 1981, p. 34). For you, an even more serious issue may be the problem of applying surveys to non-Western societies. This is one side of the issue: Surveys 127 A rural Third World Survey is the careful collection, tabulation and analysis of wild guesses, half- truths, and outright lies meticulously recorded by gullible outsiders during interviews with suspi- cious, intimidated but outwardly compliant villagers (Chen and Murray 1976, p. 241). By contrast, another researcher says: Scholars, government officials, and conmmercial interests in the developing countries are increas- ingly recognizing that survey research methods provide the only means by which systematic infor- mation can be collected and analyzed for a wide range of purposes (Mitchell 1983, p. 219). Many misconceptions have led to this impasse: a common assumption on the part of the experienced and inexperienced alike that the survey is the best, most scientific way of getting objective information; that a survey is a standardized, universally applicable tool; and that doing a survey is a simple and quick pro- cess. As a result, many people believe that whatever methods you use, it is best to begin with a survey to get a general picture; that if you can use only one technique, it should be the survey; and that you should verify the results you get from other methods by a survey. In effect, they are saying use a survey at the be- ginning, at the end, or alone, but use at least one. In reality, the survey requires a good deal of experfise and a big investment of time. Like every other re- search technique, it has a specific function: it tells you what people say they do, think, or feel if you ask the right questions, if they understand what you are asking, and if they are able and prepared to give you an- swers. If you want other dimensions of reality, you have to use other techniques as well. As we saw in part II, just because the responses to a survey can be converted into numbers and percentages does not make them more valid or accurate. The survey will provide information on selected variables, but not their context, and the replies, when added up, will not provide a holistic picture, but a set of responses to specific questions. To add another caution, the survey's structure, language, order and content of questions, mode of ad- ministration, and assumptions about how people respond are all based on the cultural beliefs of societies that use questionnaires most frequently, that is, Western societies. Sometimes they can be deeply mystify- ing or alienating elsewhere. Designing or adapting questionnaires for use in other societies, selecting and training appropriate interviewers, and administering the survey all require additional skills that standard reference books may ignore. To read more about some of these problems, look at the works of Bulmer and Warwick (1983), Chen and Murray (1976), or Stone and Campbell (1984). A survey is a specific tool that when used for the right reasons does the job better than anything else, and when used for the wrong reasons is a waste of everyone's time, effort, money, and hopes. Some of the right reasons for using a survey are: - The nature of the problem is such that it is the best approach. * You need a broadly based response to a specific set of questions or items. * You want to compare the results from one group with that of another. Using the same survey for both groups lets you do that. Some of the wrong reasons for using a survey are: * It seems the most "scientific" way to get information. * It seems quicker and easier. * You do not know any other way to get the information. * No one will believe your results unless you use a survey and produce some numbers and percentages. In this chapter we will use the word survey to refer a standardized set of questions put to a number of respondents. Surveys can take two forms: a questionnaire, or something that the respondent fills in, or a structured interview, whether face-to-face or over the telephone, in which the researcher fills in the an- 128 Research Techniques: The Basic Tools swers. There are also other kinds of interviews that are more individually tailored to each respondent or situation, such as semistructured, group, and informal interviews. See chapter 11 for a discussion of these. Of the structured types of surveys you are most likely to use mini surveys in the form of structured inter- views and tightly focused questionnaires. The guidelines for creating both are similar. The differences arise from the extra care needed to construct a questionnaire, because you will not be present to help the respondent. MINI SURVEYS Regular surveys, which are large in terms of numbers of respondents, numbers of questions, or both, are labor intensive to design, administer, and process. In many countries serious problems may arise in con- nection with sampling. If such a survey is going to be a central feature of your research, you should try to get help from a spe- cialist in survey design, sampling, coding, and analysis to avoid some of the pitfalls that can and do happen even to experts. Several good sources on surveys and questionnaire, are Casley and Kumar (1988) and Cas- ley and Lury (1987). More likely you will be carrying out what development research experts Kurt Finsterbusch (1976) and Krishna Kumar (1990) have called a mini survey. Mini surveys are carefully focused on a specific topic; con- tain only fifteen to thirty questions; are given to a small sample of twenty-five to seventy people; and usu- ally use more closed than open-ended questions; that is, they use questions that force the respondent to choose from a small set of alternative answers, rather than inviting a freely expanded comment. Some uses of the r nini survey are * To get a picture that will help you to design the next stages of your research * To assess the feasibility of a project * To get reactions from beneficiaries * To evaluate projects. A mini survey can be completed in three to seven weeks, whereas many large surveys can take a year, and often much longer, for the whole process to be completed and the results analyzed. Technically, mini surveys for development research are usually structured interviews rather than questionnaires, because questionnaires exclude people who cannot read. Interviews have the added advantage of allowing you to help people through a process that may be culturally alien, confusing, or intimidating. A mini survey has some disadvantages, which are probably obvious from the above. The main one aris- es from the size: if you want to claim that your results can be applied to a much larger population, you will have problems, particularly if you use nonprobability sampling (see chapter 6). Also, some statistical tests do not work with the small numbers you will find in your study. But while a mini survey may not give you great precision, it may be good enough to give you a general picture of the situation, of trends, and of pat- terns. If you need this in a format that can be easily converted to numbers and percentages, a mini survey may be the answer, either on its own or as a complement to other research techniques. Survey design, administration, and analysis have to be carried out carefully, or the results will be of little or no use. For the best results follow the nine steps explained in the following sections. Step 1: Clarify Your Objectives As you would in any piece of research, ask yourself: "What do I want to find out?" "Why?" "Is this technique the way to get this kind of information?" "When I get the answers to these questions, will they meet my needs?" Surveys 129 Step 2: Find Out What Else Has Been Done Someone may already have done a survey that is good enough for your purposes, that will provide you with some useful questions, or that will allow you to build on existing work and go a step fur- ther, but do not automatically use someone else's questions unless you are convinced they will work for you. Of course, you face a predicament if you want to compare your results with someone else's, because you will have to use the same questions as the other researcher did. In the process, you may sacrifice comparability for local meaningfulness: some of the questions may not mean the same thing to the people you are studying as they did to the other researcher's study group. However, if you adapt the questions for the country you are working in, then your respondents are answering differ- ent questions. Another possibility is that a standard scale or measure has been developed for your topic (see chapter 10). These can present the same problems described in the last paragraph. Step 3: Choose the Respondents First, you must decide whether you are going to ask your questions of the entire group, whatever it might be (for example, all the girls who completed primary school in a district, if the district is the unit you are studying) or just some of them. The first is a population or universe. The second is a sample. If you are using a sample, you must choose the type of sample. Reviewing chapter 6 will help you decide. Remember the warnings about nonsampling errors in that chapter. They have often been a more serious source of error in non-Western surveys than sampling errors (see Bulmer and Warwick 1983 or Stone and Campbell 1984 for a discussion). Step 4a: Develop the Questions There are no universal, all-purpose questions. You cannot even assume that you should start by asking re- spondents their name, age, and sex because you may not need to know those things. So where do you get your questions? If you are using a research outline approach (see chapter 4), look at the points in the outline to see which of them might be studied through a survey. Look at these points, taken from the outline we developed in chapter 4. Which of these could be studied through a survey? Grade 6 girls: general characteristics 1. Age 2. Area of residence 3. Ethnic or tribal group 4. Religion 5. Marital status 6. Occupation(s) X District 1. Location 2. Division 3. History 4. Size You could get individual responses to the first group of points, but a survey would not be useful for the second. You do not want individuals' opinions of the size of the district; you want to know its actual size. 130 Research Techniques: The Basic Tools When you have selected the points for study, write each one on its own card or piece of paper, and on the back write the question that you will use for the point. Then you can rearrange the cards to put them in the best order (we will return to this later). If you do not have an outline, think about the topic you need to study. Let us say you are trying to develop an evening course for market girls. Consider what you need to know. To help yourself ask "who?", "what?", "where?", "when?", "why?", and "how?" and you will begin to get your questions as in the box below. First the girls: Who are they? What are their ages? Where do they live? When did they leave school? Why? How are they making their living now? Then the course: Who would attend the courses? What courses do they want? Where could they attend them? When would they be free to go? Why do they want particular courses? How would they get there? For each of these points, you can ask more detailed questions using the same method. For "when would they be free to go?" you might want to know "how often?" and "whose permission do they need?" Of course, the approaches mentioned so far will lead to questions that you think are important. However, you will almost certainly have overlooked important areas in your planning or emphasized certain issues more than the situation warrants. Open-ended investigation will help you here. Use some of the approaches discussed in chapters 11-14 to get people to talk about their concerns and perspectives. This will make your survey focus, questions, answer choices, and language more relevant. Further questions will occur to you when you think of the possible answers people can give you. Plan the direction your survey will take by making a flow chart. If you ask "Did you complete primary school?", you want to plan what to ask next depending on the possible responses (figure 9-2). You can use the flow chart to decide how many of the lines of inquiry are worth pursuing. As you are planning the questions, try to picture the people who will be answering them and their circumstances. For example, some of the girls will be older, with children; some will never have been to school; some will have other employment in addition to marketing; some will already have taken some courses; some may only be out of primary school temporarily. Knowing these facts will help you to include other important questions. Will the girl need child care facilities? If a girl has taken a course and dropped out, why? Surveys 131 You learn to write good questions by thinking things through, knowing about the people who will an- swer them, and making mistakes. You will usually discover these after the survey is over, and you will be astonished at how stupid they are. Nonetheless, you can learn in advance from other people's stupid mis- takes. Figure 9-2. An Example of a Flow Chart Did you complete primary school? I No Yes Did you attend Do you intend to go on to primary school? secondary education? No Yes No Yes Why not? For how long? Why not? What kind? Why did What will Where? you stop? you do? Would you attend primary school now if you had a chance? |No Yes I Why not? What do you need to help you? I What work will your education prepare you for? In addition to those described here, many other mistakes are possible, some of which you will probably in- vent on your own. A good rule of thumb to remember is that the average IQ of any human being seems to drop about forty points when designing a survey, so don't be disheartened if things go a bit wrong. ASKING INAPPROPRIATE QUESTIONS Wording can be equally meaningful to both parties without the meaning being shared. Among the Baganda of Uganda, "intelligence" includes wisdom, slow thoughtfulness, and saying the right thing. Obviously, a Europe- an and a Bagandan would recognize the concept of intelligence, but their meanings would not entirely overlap. Lack of familiarity with the situation, the particular group, or the culture can lead to asking the wrong questions, giving poor answer choices, or constructing an awkward survey form. You can guard against the first two by doing your homework, but the last is more subtle: many of the pieces of advice given in this chapter are inappropriate somewhere in the world. In some places, talking about the future is presumptu- ous, because the future is in God's hands. In other places, hypothetical questions ("If you had household 132 Research Techniques: The Basic Tools help, would you be able to send your daughter to school?") are meaningless. Even a "simple" question can be misunderstood: Stone and Campbell (1984, p. 31) asked people in Nepal: "Have you heard of abortion?" Some people took this to mean did they know of anyone who had one or who had done one rather than did they know about the concept. Using inappropriate language is another source of poor questions. Nervous novice researchers often feel that using pretentious language ("Do you regard unilateral school board interventions in curriculum as- sessment as being productive?") will make their respondents more respectful and cooperative, but all it will do is annoy and confuse them. However, if you are dealing with people who use a specialized language, such as technical words, you need to use them or the respondents will assume that they are wasting their time on an amateurish effort. Remember that your survey may have to be translated into a local language. Have it translated by a bilingual person whose first language is the one in which you will finally present the questions to your respondents. Then as a check, try to have the questions translated back to the original by a bilingual person whose first language is the one in which the questions were first drafted. If it says what you want, use the first translation as your survey. It may not be easy to find people with these linguistic skills. In that case, work closely with the person who is translating from your language into his or her own so that your ideas and intentions are clear. Surprising things can happen in the translation process. Roger Mitton (1982, p. 64), a specialist in dis- tance education, wanted to ask "What do you do on Sundays that is different from other days?", meaning "What do you do on Sundays that is different from what you do on other days?" This was translated into Sesotho as "What do you do on Sunday, which is the Lord's day?" A lot of people said they went to church. This seemed to be the "right" answer. Do not be surprised if the translation is fine, but still does not get you what you want. The words may now be correctly translated, but they may not evoke the same ideas or feelings among the people in your study: you want conceptual equivalence, not simply lexical (word) equivalence. A substantial body of literature is available on back-translation (see, for example, Brislin, Lonner, and Thorndike 1973 for good advice). Using inappropriate or irrelevant concepts is even more serious. These can occur even when no transla- tion is involved. Words like "income," "household," or "employment" mean something entirely different from one place to another. A body of literature exists on what constitutes the household in different societ- ies. When you ask what employment a person has had in the past five years, what do you really want to know? Ask that instead. ASKING Two QUESTIONS IN ONE Sometimes one question is actually two. "As part of its gender sensitization program, the Ministry of Edu- cation is now taking on more women teacher trainees than men. What do you think about this program?" is two questions: "What do you think about the gender sensitization program?" and "What do think about the fact that more women than men are being taken on?" If someone says, "I disagree," is it with the pro- gram, the extra numbers of women, or both? ASKING LOADED QUESTIONS (QUESTIONS THAT BIAS THE ANSWER) The very fact that you ask a question in the first place tells the respondent that you attach some importance to the subject. If a major part of your survey is on gender-sensitive textbooks or on the expansion of techni- cal subjects for girls, your respondents realize that an "educated" or "official" person sees this as important. Their replies may be affected, even though the subject may not be important to them. So questions are al- ready loaded, but you can take care to reduce other ways of introducing bias. Surveys 133 If you begin a question with "Don't you think...?" or "Wouldn't you say that...?" you are setting up the respondent to agree. If you invoke authorities or use pejorative words-"The imam wel- comes the president's call for radical feminists to control themselves. What do you think your- self?"-don't expect much disagreement. Calling initiation ceremonies "female genital mutilation" when local people call it a "circumcision" is going to get you off on the wrong foot, too. Often, loaded questions presuppose something that may not be true: "Why do you like your school?" "Why do you think it is important for girls to get an education?" "How do you think we should expand the curriculum?" Sometimes, researchers deliberately construct a question that appears to presuppose something. If you are trying to find out what people do not like about your program or project, you can ask "What do you like about the project?" and then "What don't you like about it?", and people may feel easier about making criticisms. Asking a question that appears to have a "right" answer is pointless: "School pregnancies appear to be in- creasing at a very alarming rate. Is this a cause for concern?" Or, "Did you enjoy the course you took with us?" Answer choices can also bias the replies, for example, consider the following question: If a male teacher has had sexual intercourse with a female student a. The punishment should be more severe than it is now ............1 b. The punishment should be less severe than it is now ...............2 c. The punishment should fit the crime ...........................................3 Most people will choose "c." Some will think it is the same as "a" and others will think it is the same as "b." USING AMBIGUOUS OR VAGUE WORDS OR PHRASES One of the major assumptions of a survey is that all respondents are answering the same questions. How- ever, if the questions mean different things to different people, this is no longer the case. Ambiguous or vague words and phrases are one cause. "Frequently" can mean daily to one person and annually to anoth- er. Thus, "Do you bathe frequently?" can truthfully be answered "yes" by two people, but you would prob- ably prefer not to share a seat in the bus with one of them. "Do you think secondary-educated girls are more successful?" "Do you revise your lessons often?" "All things considered, would you like to live nearer to the school?" "Do you attend the parent-teacher meetings regularly?" ASKING QUESTIONS THAT ARE TOO BROAD When you are planning your survey, you might ask people some broad questions such as "What do you think of the school system?" to see what dimensions of the situation are most important to them. Their answers will help you to focus the questions you ask on the final version of your survey and to omit issues that appear to be irrelevant. Asking such a broad question on the survey itself, how- ever, is not very useful. Some people will criticize the ministry for raising fees; others will say they like the new curriculum better than the old; and others will say that when they were young and the world was run correctly, the teachers were at least sixty-five years old, whereas now they appear to be about eighteen. Decide instead what aspect of the school system you want people to comment on and phrase the question more precisely: "What do you think about the new plan for paying fees after the harvest?" 134 Research Techniques: The Basic Tools ASKING FOR INFORMATION PEOPLE DO NOT HAVE OR WILL Nor BE ABLE TO REMEMBER ACCURATELY Even if people do not have the information you seek or cannot recall it well, they will often give you an answer anyway, and you have no way of judging its accuracy. One researcher (White 1984, p. 20) looking at the ability of people in rural Java to recall hours of income-producing work found that asking people twenty-four hours after finishing the work produced one set of figures; thirty days later they recalled only about two-thirds of the hours; and one year later, between 43 and 60 percent of the hours. ASKING FOR SECOND-HAND INFORMATION OR OPINIONS WHEN You NEED DIRECT INFORMATION OR FACTS "Why don't your neighbors send their children to school?" should be asked of the neighbors themselves. "What percentage of parents would send their girls to middle school if one existed?" will tell you people's opinion about the percentage, not what it is. QUESTION CHECKS Sometimes intentionally, and sometimes because of poor questions, people will answer questions incorrect- ly. If people are intent on deceiving you, it is pretty difficult to catch, but you can insert question checks for some items: you can ask, for example, "Do you attend the parent-teacher meetings?" Let us say that 75 percent of the respondents say "yes." Later in the survey you can ask: "On what day is the parent-teacher meeting held?" If only 23 percent of respondents know the answer, you know something is wrong. Step 4b: Develop the Answers Once you have got the questions, you have to think about the answers. It is often said that good lawyers, when cross-examining a witness in court, never ask a question unless they already know the answer. In a survey, you do not have to know the answer, but you have to know the possible answers so that you can decide how to handle them. OPEN-ENDED OR FORCED-CHOICE QUESTIONS Your questions will either be open-ended (blank spaces are left for answers) or forced-choice or closed (you provide the answer categories). Open-ended questions allow more freedom for the person replying, but are more difficult for you to process when the survey is over. You have to read all the answers, decide what categories they include, go back and sort the answers into the categories, and then count them. Open-ended questions are suitable when * You do not know enough to provide the categories * You want to see what people choose to say themselves * The material is intimidating or sensitive (research-in Westem societies-shows that people will re- port socially unacceptable behavior more readily when asked in an open-ended question). Forced-choice questions require that you know enough about the situation to give relevant choices. This takes a lot of preliminary work, but once the survey is administered, you can process forced-choice answers quickly simply by counting the answers in each category, either manually or with a computer. Make sure your answers are exhaustive (cover all the possibilities) and exclusive (do not overlap). For example, Sex of teacher: Female 1 Male 2 Surveys 135 "Female" and "male" are examples of exhaustive and mutually exclusive answers. There are no more possibilities, and everyone will fall into one or the other. (In a face-to-face interview, you do not ask this question, of course; you figure it out for yourself.) An example of not being inclusive enough would be What work do you do after school? Sweep E] Work in the fields Z Look after children E] Cook C] Such activities as fetching water or fuelwood, washing dishes, marketing, and carrying food to people have been left out. You can put in a category called "other," but you will not know what it includes. A good idea is to add Other | (Explain ) Some questions can have as a final category, "Don't know" or "Anything else?" Be sure you are not us- ing these categories to save time when designing the survey, because they will add time when analyzing the results. A very common mistake of not being exclusive enough is overlapping number categories: Age: (CIRCLE ONE) 10-15 15-20 20-25 Let's say you are fifteen. Which one do you circle? Forced-choice questions can include sliding scales. How would you rate the presentation of women as leaders in the Primary Six social studies textbook? very poor W poor l fair l good very good C3 You can use words like always, most of the time, sometimes, very rarely, never as categories to describe the frequency of something. Another common set of categories is various stages of agreement: categories is strongly agree, agree, disagree, strongly disagree. More extensive scales are also common. The example presented in figure 9-3 is from Dawson's (1967) Traditional-Modern Scale, which he used in West Africa. An "x" is inserted at the appropriate intersection of lines. 136 Research Techniques: The Basic Tools Figure 9-3. Dawson's Traditional-Modem Scale Agree Disagree strongly Agree Not sure Disagree strongly 1. Men are better than women in every way, _ and it is proper for a man to assert his authority over a woman. 2. It is true that women are inferior beings, but l l l l l a man should never look down at a woman. 3. Men are better than women, although some _ women may be as good as men. 4. Women are as good as men in every way, and women's social position should be the equal of men. Source: Dawson (1967). Another way of showing degrees of response is to use an adaptation of the Semantic Differential, devel- oped by Osgood, Suci, and Tannenbaum (1957) (the original purpose of this scale is more sophisticated than the one described here). Concepts, presented in polar opposite terms, are placed at the left and right of a seven-point scale, and the respondent is asked to rate people, objects, places, and so on along the scale for each concept (figure 9-4). For example: Figure 9-4. Example of Osgood's Semantic Differential-Type Scale Girls with Secondary Education Respectful IJI X I I JIJI IJNot respectful Intelligent IJX I Not intelligent Marriageable I I I I X I I I I Not marriageable Other pairs of choices are presented on the same page, with the scale redrawn for each pair. You might use the same pairs for girls who left school before secondary level or girls versus boys, and so on. Schensul (1969) adapted this as a board game for people in Uganda (figure 9-5). The use of a board like this allows use of the Semantic Differential with nonliterate people. Surveys 137 Figure 9-5. Schensul's Adaptation of the Semantic Differential The city The town My (Kampala) (Mbarara) Myselt village -The Best Ufe- (Inhospitable( (hospitable) Source: Schensul (1969). CATEGORING 1HE ANSWERS When thinking about the answers, you should also think about making the completed forms easier to deal with. If you use open-ended questions, they might look like this: What form did you complete in school? What are the responsibilities of a daughter to her parents? For forced-choice questions, you have probably noticed that the examples in this chapter have been laid out in several ways. You can ask people to put a mark in the box beside the right answer, circle the answer, or choose the number of their answer. A forced-choice question might look like this: What form did you complete in school? None Z First L Second W Third But you can make counting the answers easier if you give each choice a number. If you are going to use a computer to analyze the results, you will have to do this, but even if you are not, it will help. 138 Research Techniques: The Basic Tools What form did you complete in school? (CIRCLE ONE) None - 1 First Form- 2 Second Form 3 Third Form- 4 CIRCLE ONE is written in upper case letters as an easily visible instruction to the interviewer. WRITING THE QUESTIONS Now write out your questions in such a way that when the researcher reads them out, they sound natural, like spoken rather than written language. This, for example, is written English in many countries: "I wish to address a number of questions to you. However, I shall be pleased to answer any queries you might have first. " Spoken colloquial English would be more like: "I'd like to ask you some questions. Is there anything you'd like to ask me first?" Read your questions aloud to see how they sound. The following guidelines for writing questions were adapted from work by cross-cultural research ex- perts Brislin, Lonner, and Thomdike (1973), who created them to help in translating questions from one lan- guage to another. But they are useful even when you do not have to translate. * Use short, simple sentences of less than sixteen words. However, sensitive questions may require a softener. ("Some girls find that for one reason or another, they need to get money from older boys or men. Have you ever found yourself doing this?") * Use the active rather than the passive voice: "Should the teachers discipline the students?" rather than "should discipline be carried out by the teachers?" * Repeat nouns instead of using pronouns: "When the teacher saw Modu, he was terrified." Who was terrified? * Avoid metaphors and colloquialisms: "Abdullahi and Ahmed agreed, but Mariama thought that was a horse of a different color." * Avoid the subjective mode, such as verbs with could and would: "If the school could improve its security system, would people send more girls?" * Avoid vague words such as "nearer," "often," and "frequent." "Would you like to live nearer to Ka- mpala?" * Avoid possessive forms where possible: "Khadija's sister took her request to her teacher." Whose re- quest, whose teacher? * Use specific rather than general terms: the chief, the imam, rather than the authorities, the soccer club, the debating team, rather than extracurricular activities. * Avoid words with two different verbs if the verbs suggest two different actions: "Should villagers attend and challenge the teachers at the parent-teacher meetings?" Many more could be added to this: for example, do not write in double negatives: "Would you find it im- possible to believe that your daughter wasn't giving enough time to her studies?" A vast literature on writing questions is available, much of it very helpful because it is based on ques- tion variations that have been tried on large numbers of people. Some of the pointers, however, may not apply in your society. Surveys 139 Finally, you can use a guide like the following to get some perspective on the value of each question to your research (adapted from Leedy 1989): Answer form What will How does this Do I need to Howv does this Outline (open, forced the answer relate to my use a question at is the I provide a point Question tell me? research aims? check? q slion? check? When you pre-test the questions (discussed later in the chapter) you will learn more about how potential respondents evaluate your questions. Step 5: Assemble the Instrument You are now ready to assemble the various parts of your survey. THE FACE SHEET Begin by making a face sheet. This will contain the following kinds of information, adapted to meet your circumstances. Title of Survey Serial Number Interviewer's Name Date Reason for no interview [if no interview can be obtained] Introduction: My name is (name). I work for World Aid, which helps to set up courses for people who have left school. We would like to make our courses more useful to people. Chief (name) has given us permission to visit the village and ask people what courses they need. I would like to get your opinions. I do not need your name, and I will not discuss your an- swers with anyone else. Is there anything you would like to ask me before we begin? The serial number helps you to keep track of the respondent's survey form. If seventy-five people are being interviewed, the first person will be numbered "01." You can keep the respondents' names on a mas- ter sheet, along with their numbers, in case you need to follow up the interview. You can also enter it into the computer if you use one. Keep your introduction short. FILTER QUESTIONS Sometimes your study will contain people who have very different characteristics or experiences, and not all your questions should be answered by each person. You may have to use filter questions to guide people. For example: 140 Research Techniques: The Basic Tools 1. Did you ever attend primary school? Yes E Question 2 No k Question 8 2. Did you complete primary school? Yes E Question 4 No E Question 3 3. Would you like to complete primary school? Yes E No E This keeps people who did not go to primary school from going on to tell you they completed it, and people who completed it from being asked whether they would like to complete it. If this format is not clear enough for your interviewers, try other methods: for example, after question 1, you can write: "IF RESPON- DENT NEVER ATTENDED PRIMARY SCHOOL, GO TO QUESTION 8." If you do not plan your filter questions properly, you may be sure that at least one boy will answer ques- tions on his pregnancies, or one girl will say she is still in school and in the next question will give you the rea- son why she left. Even worse, your respondents may simply give up. If you know that the groups you are interviewing are so different in their experiences that you will have to include large numbers of filter questions, making the survey difficult to administer, you might consider developing a separate survey for each group. ORDERING THE QUESTIONS Order your questions. On the principle that once people invest some time in a survey, they are less likely to balk at unattractive questions, ask general, open-ended, easier, more interesting, less personal, and less sensitive questions first. Sensitive questions differ from one culture to the next: many Westerners are used to the idea of giving details of age, sex, number of children, and so on while bristling at questions about income, but people in a study in Ghana, for example, were prepared to discuss income and assets, but not numbers of children (apparently fearing a tax) (Devereaux 1989). There is not a lot of point in trying to get around this by assuring people that the information they give is confidential. You must assure them, but often they do not believe you and think you are amazingly naive for believing it yourself, or else they start having second thoughts about the wisdom of discussing the matter if you, an "expert," think it is such a confidential matter. (Of course, it still must be kept confidential, whatever people believe.) Keep questions on related items together, and when you change the subject, use a bridge: "Now I'd like to talk to you about something else" or "Now I'd like to ask you a little bit about the village." You do not automatically need names (or numbers) on surveys, but you do if you want to follow people up later. For example, after a questionnaire you might write to thank people who replied, and remind peo- ple who have not that you still need their replies. Or you may want to compare survey results with other materials from the same respondent, or you might be doing a long-term study on the same people and need to keep track of them. Sometimes people want to be anonymous, but sometimes they feel their responses are meaningless when separated from their identity. When you need names, you can create a final page. Say "Now I'd like to ask you a few questions about yourself," and get the respondent's name, age, and any- thing else you need and that has not already been asked. Surveys 141 CLOSING THE SURVEY On this final page, close your survey by asking once again if the respondent has any questions to ask or anything to add. Thank the respondent. If there is going to be an occasion when people can hear the general results of your study, tell the person when and where you will do this. Leave a space on this page for the interviewer to make a note of anything that happened during the inter- view that might have affected the results: "This woman would not answer questions about school fees be- cause she said it was her husband's responsibility to pay them," or "Respondent was unhappy because she said a previous interviewer had paid her, and she stopped halfway through when told she wouldn't be paid." Step 6: Pre-Test the Survey Pre-test the survey on people who will not be taking the final version. One way is to ask five to ten people to sit down with you, answer the questions, and then give you comments about each question. A pre-test should help you to find out whether the survey * Flows properly, is arranged in a workable way for the interviewer (boxes in the right places, enough space for answers, manageable filter questions) and asks the questions in an order that seems rea- sonable to the interviewee. * Sounds good when read aloud. * Does not take too long. For standard surveys, research in Western societies indicates that people are prepared to be interviewed for anything up to an hour, but mini surveys will require much less time. You will have to determine the time people are able and prepared to give in your own society. Ex- perienced interviewers in your country should be able to help you. * Is interesting to the respondent. If the questions are all "yes-no," or all involve ranking on a scale from one to seven, people will probably find even the mini survey a bit dull, but the biggest cause of disinterest will be if the questions have little or no bearing on the respondent's life. Sometimes you have to ask things that are not central to the respondent's interests, and in those cases you need to make the survey itself lively by wording the questions in a conversational way, varying the tasks the person is asked to do, and so on. After the pre-test you need to discover the following: • Were the questions clear to everyone? * Did they mean the same thing to each person? * Were people able/willing to answer them? * Were the answer choices suitable? * Did the questions/answers give you the information you need? * How long did the interview take? Did people get tired or bored? * Can you code the answers if you need to? * If you are using assistants, did your instructions to them work? Have they any suggestions for improvement? * When you look at the results, is there any information you will need that is missing? Step 7: Train the Interviewers The following sections cover the things you need to remember if you are using other people to help you. 142 Research Techniques: The Basic Tools CHOOSING INTERVIEWERS When choosing interviewers, their personal characteristics-personality, intelligence, honesty, and ability to work with people-are far more important than their formal educational qualifications. A young woman who has completed middle school, or perhaps Form 3 of secondary school, could well be a better interview- er than someone with a degree from a teacher training college. Relate your selection of interviewers to the group you are trying to study, and the subject: teenage girls are not going to discuss female circumcision with a senior male interviewer, and men have sometimes refused to discuss agricultural or economic mat- ters with a woman. By the same token, try not to use people who are very involved with or seen to be associated with the topic. If you send known teachers out to ask about the value of education, people will hardly "insult" them by saying that they have more important priorities. The teachers themselves may have a stake in the find- ings, as well: for example, recording more people as wanting a new school may lead to getting one. TRAINING INTERVIEWERS You can help your interviewers by pre-testing your survey, writing out the questions exactly as they are to be presented, and writing all instructions throughout the survey very clearly in upper-case letters: for example, IF RESPONDENT SAYS "YES" ASK "WHY?" Use role playing for practice. You play the respondent, and ask the interviewer to administer the survey to you. Both you and the interviewers can also create some bad interviews and identify the mistakes. Create a difficult interview, one which is interrupted frequently, a household where no one is at home. If you are not familiar with the area and the interviewers are, this kind of role playing may help you to deal with things you had not thought of. Decide whether the questions must be asked word for word, identically, by all the interviewers, or whether you will allow them to take a more informal approach, tailoring it to the situation. If you use an informal approach, practice that as well, so that the interviewers have a common understanding of how to do this while remaining faithful to your information requirements. Interviewers should understand that their job is to make the respondent feel respected and comfortable. The interviewers should accept all answers neutrally, without displays of disapproval or shock. Lecturing the respondent is unacceptable. This is a survey, not a sensitization program. Practice recording the answers. Pay particular attention to recording open-ended answers. The inter- viewer should record the answer verbatim, not in summary form. No two interviewers will summarize in the same way, and two answers summarized as "No" may be very different. For example: "If the new tech- nical school is built, will you send your daughters?" Answer 1: "No, because I won't have the money." Answer 2: "No. Form 4 is enough for any girl. She should be learning how to look after a husband and children, not wasting time on things she's no good at anyway." Prepare for all the circumstances you can anticipate. If people refuse to be interviewed or are never around, the interviewers should note this on the face sheet. If you are doing random sampling, tell the in- terviewers exactly what procedure to follow. Make sure they do not simply substitute one household for another. Box 9-1 presents a sample instruction from Mitton (1982). Plan the human elements as well. Work with the interviewers on the best way to dress, how to approach the local leader(s), the courtesies expected, how to explain the work to interested people, how not to raise expectations that cannot be met, how to behave when not working, what to do if asked for money or gifts, and how to leave the area when finished. A team of Gambian interviewers working on problems of girls' education in The Gambia came up with a list of do's and don'ts for themselves (box 9-2). Surveys 143 Box 9-1. Example of an Instmction to Interviewers Take the first house that you come to as you approach the village. Toss a coin. If it is heads, take that as your first house. If it is tails, take the next house as your first house. When you have visited your first house, decide which two houses are nearest to that house. Choose the further of the two. When you have visited that house, decide which two houses are nearest to it, not counting any of the houses you have considered already. Choose the further of the two. Carry on like this. Example. (A) is the first house you come to. You toss a coin. It's tails, so you take no \\ (B) as your first house to visit. (C) and (D) are the nearest houses to (B), so you take the further one - (D). The nearest to (D), not counting (A), (B), or / A (C) are (E) and (F). You take the (ijj further one - (F). The nearest to (F), not counting (E), are (G) LnA (Gh and (H). You take (H). Source: Mitton (1982, p. 69). Box 9-2. Example of Do's and Don'ts for Interviewers See the village authoritfis on your arrival Present cola nuts to chief Greet people as you meet them Ladies should not give hands to men unless they are welcomed to do so In appraching elderly persons, take offyour shoes Don'f look at elders direcfly when falking to them For ladies: don't fail to cover your head, don't put on tempting dresses Don't put on cap/hat when appoaching elders Don'f send away a child who comes near you Don't belittle their village by saying: (a) you don'f have hospital/market/school (b th village is dirty/small/etc. Never smoke in fiont ofelderly persons Never hiss within the villagers' hearing Never point at elderly people when talking/explaining Don't ever make a promise to their problem Don't be too tribalistic Don't put food on the ground (as markers in making maps and diagrams outdoors) Accept tod grateWally Thank people 144 Research Techniques: The Basic Tools Unless you are a Gambian, some of these guidelines will be unfamiliar to you. Every place has its own cus- toms, and it is your job to learn them. MAKING SURE THE CONDIrIONS OF WORK ARE CLEAR Do you pay by the hour, the day, or the interview? What about arrangements for accommodation and trans- port? Will the team work together in one place and move on together or work separately? What happens in an emergency? Your interviewers need to know the answers to such questions before starting work. Step 8: Administer the Survey Conduct the interviews at times that are convenient for respondents. Introduce yourself using the material on the face sheet. If people have questions, try to reassure them and put them at ease. Box 9-3 presents some advice on what to say in response to common questions. Box 9-3. Typical Questions and Retnarks Exchanged during the Initial Stage of an Interview Interviewers frequently encounter a set of common questions at the beginning of an interview. Some typical examples are given below. 1. QUESTION: "Why did you pick me?" ANSWER: "The purpose of this survey is to find out the views of peo- ple on _ _. Since it is not possible for us to talk with every one, we have selected a few persons like you to help us out." (Interviewer can then add a sentence or two about the sampling process.) 2. REMARK: "I am too busy. Why don't you leave me out?" RESPONSE: "It will not take more than 15 to 20 minutes of your time. However, if you are very busy now, please tell me the time when I can visit you again. It is extremely important that I have the benefit of your ideas and experience on this important subject." 3. REMARK: "I really don't know anything about this." RESPONSE: "We are interested in your opinions and experiences and not in what information you may or may not have. I am sure that you will find the questions interesting and will be able to answer them easily. We have asked the same questions from many people who did not have any problem in answering them. I must mention that in a study of this type, there are no right or wrong answers to questions." 4. QUESTION: "Who's behind this?" "This study is sponsored by the - project/program. Its purpose is to get ideas and information that will be helpful in improving its ac- tivities." Source: Kumar (1990, p. 44). Surveys 145 Then ask the questions clearly and slowly. If people have trouble answering * Repeat the question. * Allow an expectant pause. * Repeat the person's reply so far. * Offer neutral comments, such as, "Why do you feel that way?" or "Anything else?" * Ask for clarification: "I'm not quite sure what you meant when you said that 'girls are for marrying.' Can you tell me a little more?" Check the completed survey forms periodically to catch problems. Are the interviewers filling them out correctly? Are one interviewer's survey results very different from those of others? Why? You can begin coding the survey forms now, if necessary. Step 9. Process the Data You have several choices in relation to processing your data. Counting the answers straight from the forms yourself requires no coding. Everything else does. This includes methods such as recording on large sheets and computerizing the results. The approach you choose will depend on how much data you have, what you want to do with it, and whether you have access to and are comfortable with a computer. The best rule is "simplest is best." Use the least complicated method that still produces what you want. If you have done something as small as a mini survey, you should be able to count directly. Before you do anything, figure out what information you want. You want the figures or percentages for each of your questions. These are called frequency counts. But what else do you need? Do you want to know how many of those who are female (question A) went to secondary school (question B)? This is called a cross-tabulation, or cross-tab for short. Think of your report and picture any tables you will need. Make up the blank forms or dummy tables as shown in table 9-1. This way you will not be counting things that you do not need. Table 9-1. Example of a Dummy Table Level of schooling Sex Primary Middle Second level Third level Female Male If you have a large number of survey forms, a large number of questions, or want to do extremely com- plex analysis, you should use a computer. Programs such as the Statistical Package for the Social Sciences (SPSS), SPSSx, or the Statistical Package for the Personal Computer (SPPC) will do the work very quickly once you have the material in proper form. However, putting the material in proper form is time consum- ing, and for small surveys may take more time than simply doing all the processing yourself. Because all approaches evolved from the practice of doing your own counting, we will look at that first. If question B is 146 Research Techniques: The Basic Tools "What is the highest level of education you have attended?" Primary ........1 Middle ........2 Second Level ........ 3 Third Level ......... 4 you can sort the forms into four piles and count the answers. You can then keep them in these piles to look at what is recorded for question A: "Sex of respondent." How many of those in the primary pile are male? How many female? You can then fill in the dummy table. You might want to look at other questions as they relate to level of education: how many primary-educated males said "agree" to question 12? 12. Education is more useful for boys than for girls Agree ........ . I Disagree .........2 Don't know ......... 3 If you are not doing cross-tabulations for questions, you may simply want to keep the forms all in one pile and record the answers on forms like this: Yes JR JRtj Jt III No JIH Question 4 Don't know I Not applicable JII J 1 Incomplete Some people will not have answered the question because a filter question will have directed them else- where. They are counted as "not applicable." For others (one in this case) the survey form was not clear: two answers were recorded, the question was skipped, or something else happened that prevents you from reading the reply. This is "incomplete." What do you do if people can choose more than one answer? For example: Which of the following do you do every day after school? (CIRCLE AS MANY AS APPROPRIATE) Sweep .01 Fetch water .02 Fetch fuel wood .03 Mind children .04 Cook .05 Wash dishes .06 Work in the garden .07 Sell in the market .08 Pound grain .09 Other (please explain) .10 Surveys 147 For such questions count the answers. When you give your totals in a chart, you will have numbers that are higher than the number of girls in your study. Put a note at the bottom of the chart: "Respondents could choose more than one answer." Notice that in this example, double numbers are used for answers, because "Other" takes us up to a two-digit number, 10. If you are counting by hand, this does not matter, but if you are using a computer, your answer category numbers all have to have the same number of digits. If you ask people to rank their answers, you need to show which ones were considered more important. For example: Here are some reasons why girls don't go to school. In your opinion, which are the most serious? Please rank them from I to 5, giving a "1" to the most serious and a "5" to the least serious. * Financial costs z * Girls are needed to work at home m * Parents are worried about girls' security * Parents believe that education is more important for boys W * The school is too far away you can add up the scores for each answer. In this case, the answer with the lowest score is considered the most important. You cannot read a long list to people. Even for a relatively short one, you may be better off giving people cards with pictures or symbols on them and asking them to arrange them in order. Note that someone familiar with the culture should draw the pictures or symbols. Pictures can be culturally biased in their content, and even in the way they are drawn: for example, something drawn very small may not be interpreted as far away in some societies. When you have finished, you can record the answers on a blank survey form. All other methods of recording involve coding. For the question Would you like to go to secondary school? Yes .........I No .........2 You have already pre-coded the choices. Writing all your codes down in one place and creating a coding frame is a good idea. These are instructions to a coder about what to write on a recording sheet or enter into a computer as each answer comes up. Question 1: Sex Female ............1 Male ............2 Question 2: State Yobe ............1 Borno ............2 Jigawa ............3 148 Researcfr Techniques: The Basic Tools Question 3: Attended Evening Classes Yes ........1 No ........2 Don't know ........ 8 Incomplete ........9 (Note: the code numbers to the answers in question 3 are the ones commonly used by coders for these an- swers, as is "0" in question 4.) Question 4: Participated in the Advanced Program Yes ........1 No ........2 Don't know ........ 8 Incomplete ......... 9 Not applicable ......... 0 Let us say the first respondent (serial number 01) is a female: for question 1 she would be given a "1." If she lives in Jigawa, she will get a "3" on question 2. WVhere this information is recorded depends on the method you use. You can enter the codes into a comput- er program, but if you are doing a mini survey, writing them on a large sheet is probably sufficient (table 9-2). Table 9-2. Counting Answers for a Mini Survey Question number Respondent number 1 2 3 4 Respondent 01 1 3 2 0 Respondent 02 2 1 1 9 Thus respondent 02 is a male who lives in Yobe and attended evening classes, but you cannot read the an- swer to his question 4. Continue tabulating for all respondents and all questions. You can do a cross-tab of sex by attendance at evening classes. Look at each female ("1" in question 1) and see what she said on question 3. Do the same for each male (table 9-3). If you had fifteen respondents, these would be the codings for the cross-tab of questions 1 and 3. Doing this on your large sheet is feasible if you do not want more than thirty cross-tabs. Beyond that, a computer is a better choice. You can also do cross-tabs for three or more questions: of the females (question 1) who attended evening classes (question 3), how many had children (question 6)? You can see that this will take a lot of work. If you use a computer, this takes very little work. Once the codes and instructions are in the computer and you are using an appropriate program, such as SPSS or SPSSx, you can cross-tabulate everything by everything else if you like, and end up with a lot of meaningless stuff: for example, of men who do not know whether they attended the advanced program (question 3), how many have four children? So choosing what to cross-tabulate is important, or you will end up with kilos of computer paper, and everything on it will look very "scientific" and worth putting in your report. Surveys 149 Table 9-3. Cross Tabulation of Two Questions Question 3 Code I Code 2 Code 8 Code 9 Code A I Ct Code 2 A An appropriate computer program will not only count and cross-tabulate, it will perform statistical tests of significance, association, and others (see chapter 17) on your material, even when the tests do not apply to what you have collected. Putting all that in the report is also very tempting. If you intend to use a computer program, you should get help before and during the survey design pro- cess, so that the answers will be in a form that is easy to enter. You might consult someone who has used the program, or read a manual or guide, such as one of those listed in box 9-4. Box 9-4. Computer Guides and Manuals Brownell, Blaine A. 1985. Using Microcomputers: A Guidebookfor Writers, Teachers and Researchers in the Social Sciences. Beverly Hill, California: Sage. Bryman, Alan, and Duncan Cramer. 1990. Quantitative Data Analysis for Social Scientists. London and New York: Routledge. Foster, Jeremy J. 1992. Starting SPSS/PC+: A Beginner's Guide to Data Analysis. Wilmslow, U.K: Sigma Press. Kleiger, Douglas M. 1984. Computer Usagefor Social Scientists. Boston: Allyn and Bacon. Madron, Thomas W., and C. Neal Tate. 1985. Using Microcomputers in Research. Newbury Park, California: Sage. Nie, Norman H., and others. 1988. SCSS: A User's Guide to the SCSS Conversational System. New York McGraw Hill. Rattenbury, Judith, Paula Pelletier, and Laura Klem. 1984. Co0nputer Processing of Social Science Data Using OSIRIS IV. Ann Arbor, Michigan: University of Michigan, Institute for Social Research. Schrodt, Philip A. 1987. Microcomputer Metlodsfor Social Scientists. Newbury Park, California: Sage. SPSS, Inc. 1986. SPSSX Users' Guide, 2nd ed. Chicago. QUESTIONNAIRES The procedure for developing a questionnaire, a form the respondent fills out, is more demanding, be- cause you will not be there to work with each respondent, clarify questions, make sure that the answers are in the right places, and encourage the person to complete it. Everything has to be there on the paper in clear, relevant, and attractive form. How many questionnaires have you ignored because they lacked these characteristics? A questionnaire is useful only if the respondents are literate and you are likely to get a minimum of 60 per- cent back. In places with poor communications systems, you will have to deliver and collect them yourself. Much of what we know about making questionnaires accessible and motivating people to return them comes from the Western world, and involves cultural assumptions that may not apply and practical 150 Research Techniques: The Basic Tools considerations that are different. Much research has been done on the days of the week that are best for mailing, what kind of incentives might be offered to get people to cooperate, the appearance of the questionnaire, its length, how much you can increase the rate of response by sending "waves" of follow-up letters asking people to return the forms, and so on. Investigators maintain, for example, that postal questionnaires usually have a response rate of less than 50 percent (70 percent or higher with follow-up waves), but that figure assumes, among other things, that an area has a convenient and trustworthy postal service and that people stay in the same place. The advantages of a postal questionnaire are lower costs (you do not need a team of interviewers); ev- eryone can be reached at about the same time instead of interviews stretching out over days or weeks; the respondent might feel a greater sense of anonymity; respondents have more time, especially if they have to consult records; and so on. You also do not have to worry about how the interviewer might bias the results. The disadvantages are a lower response rate, and related to this, the fact that you have no idea what kind of people did not respond. Were the nonrespondents people who disapprove of education for girls? If so, you might conclude that 90 percent of people support girls' education, when this would not be the case. The biggest disadvantages, however, relate to confusion on the part of the respondents and a lack of motivation when you do not have interviewers. Postal questionnaires are not the only type of questionnaire. You can also administer questionnaires to "captive" groups, such as schoolchildren or teachers attending a seminar. The return rate for these is much higher, and the potential for confusion lower if you are present to go through the questions with the group. But sending them to a school, for example, and allowing individual teachers to administer them is not a good idea. Each will have a different idea of how to do it, some will think the good name of the school de- pends on the children giving the "right" answers, and so on. A major disadvantage of this approach is that although you might have large numbers, they cannot be said to represent anything except that group. If that is all you want, however, this can be a good method. Questionnaire forms, whether mailed or not, require a cover letter or statement explaining the purpose of the research, who is sponsoring it, who is being asked to participate, an appeal for cooperation, an assur- ance of confidentiality, and clear directions for completing the questionnaire. It may tell people that there are no right or wrong answers. Finally, it may invite people to tick a box at the end of the questionnaire if they would like to receive a summary of the findings. (Do this, of course, only if you are able to get such a summary to them.) LOOKING BACK AT OUR INTERVIEW Now that you have read this chapter, look back at the interview at the beginning of the chapter. Can you identify all the problems with it? Our Girls Much of what we want to know about the girls in X District can be obtained through a survey: you can ask girls about their characteristics and their reasons for leaving school, for exam- ple. You can ask parents about their attitudes toward girls' ed- ucation and about why their girls left school. You can ask teachers the same questions. Surveys 151 FURTHER READINGS TO HELP YOU Brislin, Richard W., Walter J. Lonner, and Robert M. Thomdike. 1973. Cross-Cultural Research Methods. New York: J. Wiley. Bulmer, M., and D.P. Warwick. 1983. Social Research in Developing Countries. New York and London: John Wiley and Sons. Casley, Dennis J., and Krishna Kumar. 1988. The Collection, Analysis and Use of Monitoring and Evaluation Da- ta. Baltimore: The Johns Hopkins University Press. Casley, Dennis J., and Dennis A. Lury. 1987. Data Collection in Developing Countries. Oxford, U.K.: Clarendon Press. Chen, Kwan-Hwa, and Gerald F. Murray. 1976. "Truths and Untruths in Village Haiti: An Experiment in Third World Survey Research." In John F. Marshall and Steven Polgar, eds., Culture, Natality and Family Planning. Chapel Hill, North Carolina: University of North Carolina, Carolina Population Center. Dawson, J.L.M. 1967. "Traditional versus Western Attitudes in West Africa." British Journal of Social and Clinical Psychology. 6(2):81-96. Finsterbusch, Kurt. 1976. "Mini-Surveys: An Underemployed Research Tool." Social Science Research 5 (1): 81-93. Kumar, Krishna. 1990. Conducting Mini Surveys in Developing Countries. A.I.D. Program Design and Evalu- ation Methodology Report No. 15. Washington, D.C.: U.S. Agency for International Development. Mitchell, Robert Edward. 1983. "Survey Materials Collected in the Developing Countries." In M. Bulmer and D.P. Warwick, eds., Social Research in Developing Countries. New York and London: John Wiley and Sons. Stone, Linda, and J. Gabriel Campbell. 1984. "The Use and Misuse of Surveys in International Development: An Experiment from Nepal." Human Organization. 43(1):27-37. White, Benjamin. 1984. "Measuring Time Allocation, Decision-Making and Agrarian Changes Affecting Rural Women: Examples from Recent Research in Indonesia." IDS Bulletin 15(1):18-32. What mnwaarae, mctes, and hndie are good or. Mesurhg or asseaeeng p¢eo on aIdia, apitdea, aftuide, or daisom c. Examphe: Aseserng girt.' mathemadts saIls Raing vilage on cadewacs such asamodemeWn Meaurhig the degree to which people aree with stblnaf about boW and girW' ab1M Two Sample studies: Roe, Kenneth N., and T. Neville Posletwslte. 1991. indiowbe of9w QuaNy yof EdLcatlon: A A tional Sthe of Stoos In ahbabwo, 2 vole. Pads: Intemetoul Instltute for Educatonal Plannhig (etc vlew. Serpell. R. 1 9Mez of Peepti, Sldb and Irta : The Growlh a NOw Pswedve an Chidren In a Third World CourfnW. In W.W. HNa, ed., Ae- Wo ofO ChiWd DOmantesd Wol. 6. Chcago: Unhveally d Chcao Pre (kIC viw). 10 Measures, Scales, and Indices summary Scales and indices are quantitative measures. Many standardized scales and indices are available to the researcher, particularly in thefield of education. If they meet your information requirements and are culturally relevant and valid, when properly adminis- tered scales and measures can be very useful. Some possible problems arise because many standardized scales and indices are culturally biased in their as- sumptions, concepts, language, andform. A common scale used in the social sciences is a Likert scale. Scales can be nominal (the items are named differences, like red or green); ordinal (the items are ordered, for example, cool, warn, hot); interval (the differences between items can be given in numbers, say, ten de- grees, twenty degrees); and ratio (the numbers are based on something that has a true zero,for instance, ten years, twenty years). Perhaps you have an idea that people's unwillingness to send their daughters to school is related to their conservatism, perhaps even more than to a lack of money, the distance from the school, or any of the other possible reasons. You feel that if this is true, you might be better able to address people's worries by chang- ing the school curriculum, the teaching arrangements, and so on. But how can you find out if the people who are objecting to girls' education are "conservative?" Will you just ask them if they are? Or will you put a set of questions to them, the answers to which, when assembled, might tell you? Social researchers often use scales and indices as measures or tests to show degrees of difference among people in relation to an attribute or characteristic, such as attitudes, behavior, mental abilities, or psycho- logical characteristics. A scale or index is standardized, that is, given in the same form to everyone, and is intended to be objective, that is, not dependent on the researcher's biases or personal opinions. Thousands of standard scales and indexes have been designed to assess people on characteristics such as intelligence, achievement, aptitudes, prejudice, authoritarianism, modernity/conservatism, social com- petence, leadership, alienation, and many others. Perhaps you or your children have been given a scale or measure, such as an IQ aptitude, or personality test. Such tests may be part of the testing and assessment procedures used in your education system. They are being discussed in this chapter because they are com- 153 154 Research Techniques: The Basic Tools monly used in the social sciences and in the fields of educational research and testing, so readers may be familiar with them and want to use them, especially because they are ready-made. None of the preparatory work involved in a survey is necessary: experts have already designed the questions or items and worked out the systems for scoring them. Even though they may seem simple, tests and assessments using scales and indices are complex instru- ments based upon sophisticated theoretical assumptions. Simply applying them in a "cookbook" ap- proach-asking the questions and scoring the answers from a book-is not enough. There's a second problem, as well: in their standard forms they may be culturally biased and inappropriate except in the so- cieties for which they were designed. First, let us distinguish between an index and a scale. An index is a self-contained measure: "How would you rate your school? Very good, good, fair, poor, very poor." You might assemble a number of such ques- tions. When the questions are interrelated in such a way that the answers form a pattern, you have a scale. A scale might show, for example, that people who say yes to the question "Would you support a woman can- didate for prime minister?" are also likely to say yes to "Do you think women should be allowed to vote?" and "Should girls be allowed to attend school?" If you have devised a valid scale, people who say no to the last question are probably going to say no to the other two. Those who say yes to the last but no to the sec- ond are likely to say no to the first, as well. One type of pattern-the Guttman scale-is discussed later. Indices and scales can be used in two ways: you can ask a person to make a judgment about her own characteristics or about those of others: Myself Traditional I IxI I I I I I Modern The Next Village Traditional I I I I I X I I I Modern A second possibility is to give a set of multiple items, for example, a number of questions, each of which looks at some aspect of the characteristic, as in Dawson's (1967) Traditional-Modern scale (see chapter 9, figure 9-3). When all the questions on a scale have been answered, each response is given a value, and the values are added up to produce a score. In the first example you are relying on the person's judgment. In the second, you are assuming that your items are valid measures of whatever you are studying, such as "modernism." If you create the items your- self, you have no guarantee of that. What you think of as modernism and what someone else thinks may be very different. If you simply want to compare your own respondents, one against another, this does not matter too much, as you are giving them the same scale, whatever it measures. You just cannot prove that it measures what you say it does, and you cannot compare your results with those obtained by using some- one else's scale, such as Dawson's. SOME WARNINGS Even if you use a standardized index or scale, you can still have problems. There is no absolute guarantee that a standardized form measures what it says it does. What does the Raven Matrices or the Wechsler Measures, Scales, and Indices 155 intelligence test measure, for example? Some experts have argued that intelligence is a universal, mea- surable characteristic. Other have argued that it is culturally relative, that is, what is a crucial feature of intelligence in one society may not be valued or recognized in another. New theories of intelligence are emerging (see, for example, Gardner 1983; Sternberg 1985), but in the meantime, we are left with the ex- isting tests, which measure something, but what? The same question applies to personality tests. One of the most commonly used tests around the world, the Minnesota Multiphasic Personality Inventory (MMPI) and its successor, MMPI-2, contain Western cul- tural biases in the assumptions about mental illness, in the items presented, in what constitutes "normal," and in how the results are interpreted. Defenders of standardized tests point to years of testing. They can show that people who score a partic- ular way on one kind of personality test, for example, tend to score the same way on some other personality tests (in other words, the test is highly correlated), so something is being consistently measured. It is almost certainly being measured in a better way than something you put together yourself. But you still face a ma- jor question: is the scale valid in your culture, that is, does it measure what you want it to measure? Many standardized tests are culturally biased in several ways: * In the language or kinds of concepts used. For example, in the Western world people commonly think of the very intelligent as impractical or unworldly: the mad scientist, the absent-minded pro- fessor. They almost consider it surprising when a genius carries out ordinary tasks, like remember- ing to get a haircut or planning a party. In many societies, such practical sense is a fundamental attribute of intelligence, along with many others that are not measured on standard IQ tests. * In any pictures or drawings used. They too can be culturally biased, showing items that are unfa- miliar or novel to people or that mean something else in that society. An old woman dressed in a black shawl can be a witch in one society, a respected grandmother in another, or a member of a religious order, a widow, or any number of other interpretations. Sometimes psychological tests that are based on pictures, such as the Thematic Apperception Test (a psychological test) have to be redrawn to be meaningful. Once they are redrawn, of course, the results cannot be compared with those obtained from people who took the original test. * In the conventions of representation. Although people's eyes are the same everywhere, we are taught to "read" pictures in different ways. Part of this is because while the world appears three- dimensional to the eye, pictures are two-dimensional. One convention used in many places is to show objects that are meant to be far away as smaller, but some people may read this as a small, rather than a distant object. Some research has also suggested that people who are accustomed to certain kinds of environments or living conditions (housing with no right angles, hunting that in- volves scanning the horizon, and so on) read pictures differently from people not accustomed to those conditions. * In the importance of the concepts in your society. An instrument called the Myers-Briggs Type In- dicator (MBTI) (Myers and Meyers 1980) is very popular in the United States. It measures individual personality type on four dimensions: - (E) Extraversion-Introversion (I) - (S) Sensing-INtuition (N) - (T) Thinking-Feeling (F) - U) Judgment-Perception (P) Thus, of the sixteen possible combinations, a person might be classified as "INTJ": taking an intro- verted, intuitive, thoughtful, and perceptive stance in relation to situations. Employees of the World Bank tend to think that most of the people working there fall into one category. What that is, and 156 Research Techniques: The Basic Tools whether there is any substance to the rumor, does not concern us here. What matters is that even if the MBTI is a valid indicator, what it is measuring may not be important in your society, or not as important as other dimensions that are not measured. Suppose that the preeminent distinction in a society is between those who are optimistic and pessimistic or competitive and noncompetitive no matter how else they approach life? That will not show up on the indicator, which was not designed to accommodate dimensions other than those mentioned. Or suppose that your society recognizes something more sophisticated than these bi-polar concepts, something that combines aspects of both? The danger is that because the standardized indicator exists and has been widely used, you might be tempted to use it, even if it does not tap the distinctions that are meaningful in your cul- ture. In the assumptions on which testing is based. People in many cultures are not accustomed to being tested. Some are unaccustomed to individual work: their culture values a team approach, and they want to consult others about their responses. The order of presentation of items (going from the gen- eral to the particular or vice versa, going backward in time, or starting in the past and moving for- ward); the use of examples and abstract concepts; and the length of the test can all affect the results. Trimble, Lonner, and Boucher (1983), experts in using measures cross-culturally, say that three as- sumptions underlie measurement: - People can order or rank the ideas or items that you present to them along a single line: for ex- ample, people may be asked to rate the school system on a scale of one to seven, one being very good and seven being very bad. However, there is no evidence that people in all cultures think this way. - The items in the test reflect all people's real-life way of thinking: "Which two of these three items are more alike, and which is different?" This may be alien to some cultures. - People are able and prepared to be self-reflective and to share their thoughts with the research- er: "How satisfied are you with spouse?" is too personal for some people, and simply baffling to others. "Satisfied" may not be a concept to be applied to spouses, or it may be irrelevant. One can do nothing about one's spouse, so what is the point of thinking about whether one is satis- fied or not? For an excellent review in cross-cultural testing and assessment see Lonner (1990). TYPES AND SCALES Scales in the social sciences, whether standardized or not, tend to fall into one of three types: Likert scales, Thurstone scales, and Guttman scales. Each is based upon a different set of assumptions about the way peo- ple respond to things. For example, in the hypothetical Guttman scale shown in table 10-1, you have asked eight parents whether they pay for extra studies for their children, provide all their textbooks, give them lunch money, and buy their exercise books. You have recorded the answers, and as you can see, they form a pattern. In a perfect Guttman scale, anyone who pays for extra studies also does all the other things; anyone who does not pay for extra studies but pays for books, pays for lunch and exercise books as well; anyone who pays for lunch also pays for exercise books; and anyone who pays only for one thing pays for exercise books. This would tell you that paying for exercise books is the easiest to achieve and paying for extra stud- ies is the most difficult, and you could plan a program accordingly. Notice that our scale is almost perfect: only respondent number 1 breaks the expected pattern.1 1. To calculate the degree to which such a scale approaches a perfect scale (or the coefficient of reproducibility), the formula is 1 - (number of "errors"/number of entries). Our example has thirty-two entries (eight respondents times four items) and one "error." Calculated, this give us a figure of 0.97. Anything above 0.90 is considered significantly close to a perfect scale. Measures, Scales, and Inidices 157 Likert and Likert-like scales are the most commonly used today for a variety of theoretical reasons (see Judd, Smith, and Kidder 1991 for a discussion of this). In a Likert scale people are invited to respond to a set of items. Let's say you want to look at people's progressiveness or some other meaningful concept in your society and you have tried to work out all the dimensions of this concept as they apply to girls' edu- cation. Now you ask people whether they strongly agree, agree, don't know, disagree, or strongly disagree with these statements (figure 10-1). Table 10-1. Example of a Guttman Scale Respondent Extra studies Books Lunch rnoneey Exercise books 7 + + + + 5 + + + + 8 - + + + 4 - + + + 6 - - + + 2 - - + + 1 + - + + 3 - - - + Figure 10-1. Example of a Likert-Like Scale Strongly Strongly agree Agree Not sure Disagree disagree It would be better to tackle the problems of boys' education first before worrying about girls' education Too much time is spent teaching girls things _ | they will never use l Today, girls need as much education as boys Both boys and girls should be taught to memorize the president's speeches The assumption is that people with a strongly favorable view toward something (girls' edu- cation, for example) will agree with favorable items and disagree with unfavorable ones, while people opposed to it will do the reverse, and people who are neutral or undecided will fall some- where in the middle. The person's score is obtained by subtracting the numbers of responses to items that are considered negative from those that are considered positive. On a standardized Likert scale, the positive and negative items have already been chosen, so the scoring instruc- tions are clear. Many researchers put some Likert-like questions in their survey. These may be taken from one of the many standardized scales or created by the researcher, as we did in chapter 9. Let us say you show six pic- tures of girls dressed in a range of clothing from very traditional to ultra-modem. You ask people to rate them on goodness, responsibility, honesty, virtuousness, marriageability, and so on. You use a word scale like very good, good, and so on, or a drawn scale with good on the left, bad on the right, and five or seven 158 Research Techniques: The Basic Tools spaces to choose from in between. Perhaps you have a good idea as a result that many people think girls in ultra-modem clothes are bad and irresponsible. This is a modest effort: you are simply using the scale as a form of forced-choice question. You may even find that the answers form a pattern, as on a Guttman scale. However, experts warn against assuming that you can easily create an acceptable scale yourself: re- search expert Delbert C. Miller calls it "an activity of last resort" when you cannot find a scale that suits your needs, or the scale you want to use is poorly constructed. But, as he points out (Miller 1991, p. 579), "putting some items together and assigning arbitrary weights to them" is not the answer. Look back at figure 10-1. How do we know, for example, that people who agree with the statement "girls need as much education as boys" are progressive? Maybe they mean girls should have as much ed- ucation, but it should be restricted to home economics. If we change it to "girls should have exactly the same education as boys," is this necessarily a progressive idea? Is "both boys and girls should memorize the pres- ident's speeches" progressive or nonprogressive? No distinction is made by sex, but is memorizing the president's speeches progressive, nonprogressive, or something else entirely? Items that do not "work" should be discarded. This is usually done by asking an independent panel to rank items for progressive- ness/nonprogressiveness. Items with high agreement among the panel are more likely to be useful. The fi- nal results of the scale are then compared with other results and tests: for example, on your scale, are the people who scored as progressive the same ones who recently voted to close all schools to girls? The measures and scales discussed in this chapter are almost always etic: you, the researcher, decide the categories, the mode of ranking, and so on. In chapter 14, we will look at participatory measures and scales, which are usually emic. They enable people who are participating in the research to develop and rank con- cepts, problems, and solutions as they see fit. The participants decide what constitute the important dimen- sions of a situation and what weights to assign them. As a result, each scale or measure is unique to the group concerned. Whatever the scale, it can take four different forms: - Nominal: one object is different from another: men or women, red or green. White (1984, p. 27) used a nominal scale when classifying whether husbands, wives, or both participated in decisionmaking. (See chapter 16 for the scale itself.) * Ordinal: One object is bigger or better or more of something: never, sometimes, frequently; primary, secondary, tertiary education. I Interval: one object is so many known units more than another, and there is no true zero: ten degrees Fahrenheit, twenty degrees Fahrenheit; an IQ of 75 and one of 150. * Ratio: one object is so many times more, in multiples, and a true zero exists: ten inches, twenty inch- es. Note the difference between interval and ratio: twenty degrees is not twice as hot as ten degrees, but twenty inches is twice as long as ten; a thermometer has an artificial zero, a physical object mea- sured by a yardstick has a true zero. The scales most commonly used in the social sciences are ordinal and interval. Table 10-2 shows these and the statistics that can be used with them. For some of these statistics see chapter 17. If you decide that a scale is what you need and you are choosing a standardized scale, try to get as much information as possible on these questions: * Is the scale valid: does it measure what it says it measures? What is the evidence (for example, are its results consistent with those of other, similar measures)? * Is it reliable: does it produce the same results each time if the circumstances are the same? * Does the scale fit your problem? * Will the scale be relatively easy to administer and score? Measures, Scales, and Indices 159 * Do you understand the statistics that are used to interpret the scale? * Is the scale appropriate for your culture? Table 10-2. A Summary of Measurement Scales, Their Characteristics, and Their Statistical Implications Measurement Characteristics Statistical scale of the scale possibilities of the scale Nominal A scale that measures in Can be used for determining the terms of names or mode, the percentage values, Noninterval scale designations of discrete units or the chi square. scales Ordinal or categories. scale A scale that measures in Can be used for determining the terms of such values as mode, percentage, chi square, more or less, larger or median, percentile rank, or smaller, but without rank correlation. Interval sspecifying the size of the Inerale intervals. Interval A scale that measures in terms Can be used for determining the scales Ratio of equal intervals or degrees of mode, the mean, the standard scale difference, but whose zero deviation, the t-test, the F-test, point or point of beginning is and the product moment arbitrarily established. correlation. A scale that measures in terms Can be used for determining the of equal intervals and an geometric mean, the harmonic absolute zero point of origin. mean, the percent variation, and all other statistical determinations. Source: Leedy (1989, p. 26). Reprinted by permission of Prentice-Hall, Upper Saddle River, New Jersey. How do you find scales and measures? Hyde (1989) has compiled a guide to scales, measures, tests, and other instruments, such as inventories, surveys, and interview and observation forms that have been or might be used to study girls' attendance and achievement in developing countries. Of the thirty-one instru- ments in the guide, twelve have been used in Africa and seven others outside Canada, the United Kingdom, and the United States. Appendix 2 contains a list of them, along with addresses to which you can write to get a particular item. For an extensive list of scales and measures, generally, see Miller (1991). Our Girls Back to chapter 4 and our girls in X District. Would measures or scales help here? Look at box 4-3. Under "B. School-related characteristics" we have "ex- aminations or other assessment." Here we might look at the results each girl got on standard assessments for reading, mathematics, and other subjects. In box 4-5, we could look at overall performance in the school system by gender, socioeconomic group, ethnicity, language, and so on. Do girls of a particular group do worse? Is that one reason why they leave at Grade 6? 160 Research Techniques: 7Te Basic Tools FURTHER READINGS TO HELP YOU Brislin, R.W. 1990. Applied Cross-Cultural Psychology. Newbury Park, Califomia: Sage. Brislin, R.W., W.J. Lonner, and R.M. Thomdike. 1973. Cross-Cultural Research Methods. New York: John Wiley. Irvine, S., and J.W. Berry, eds. 1983. Human Assessment and Cultural Factors. New York: Plenum. Judd, Charles M., Eliot R. Smith, and Louise H. Kidder. 1991. Research Methods in Social Relations, 6th ed. Fort Worth, Texas; London: Holt, Rinehart and Winston. Lonner, W.J. 1990. "An Overview of Cross-Cultural Testing and Assessment." In Richard W. Brislin, ed., Applied Cross-Cultural Psychology. Newbury Park, California: Sage. Lonner, W., and V. Tyler. 1988. Cultural and Ethnic Factors in Learning and Motivation: Implicationsfor Educa- tion: The Twelfth Western Symposium on Learning. Bellingham, Washington: Western Washington University. Miller, Delbert C. 1991. Handbook of Research Design and Social Measurement, 5th ed. Newbury Park, Califor- nia: Sage. Stemberg, R.J. 1985. Beyond I.Q.: A Triarchic Theory of Intelligence. New York: Cambridge University Press. What the intrvIew Is good fr: Greater tailoring of quealons to Indivdual people or situaIons More Interaction between Interviewer and responrdent More fedbillty In that the interview can be redirected as It procmse depending on what you leam Example: You want to explore wlth a emall group of parents their fears about girls' securIty. What are their experiences? You are considering a new Wdrnd of projert for girls. You hear that various nongoven- mental organiatons have trled similar Ideaa. What happened? intervIewing the staff of each nongovemmental organization might help to refine your project Sampie study: Bledsoe, Caroline. 1990. 'School Fees and the Marriage Prooess for Mende Gilr in Sierra Leone.' In P.G. Sunday and R.G. Goodenough, eds., Beyond to Second Sex. Philadelphia: University of Pennaylvanh Preaa. ...... 11 ersees Unstructured and Semistructured Interviews summary This chapter and the next two look at qualitative techniques, or techniques that produce material in words rather than in numbers. When you are using qualitative techniques, your main instrument is yourself. Your personal qualities will really come into play now. Many of the underlying principles of structured interviews also apply to semistructured and unstructured interviews. This chapter explores a particular type of unstructured interview, the emic approach, using interviews, card sorts, and triads. A checklist is also providedfor semistructured interviews. You can cary out interviews with individuals, such as key infortnants, or with groups. Some special types of group interviews are community interviews and focus groups. The techniques in this and the next two chapters are often called qualitative: they produce material in words rather than in numbers, although it can later be converted to numbers, if appropriate. INTRODUCTION TO QUALITATIVE TECHNIQUES Qualitative techniques usually, but not always, resemble the everyday methods that people use for getting information and take place in the same kinds of settings: in people's houses, in a village, in the classroom, that is, the places where what you are interested in actually happens. Because of this, you get the context in which the action, behavior, or process occurs, which gives you a more holistic picture as you can see the background in which your material is embedded. Qualitative research can be used for * Getting a preliminary picture so that you have enough information to refine your strategies and questions * Interpreting the meaning of material collected through quantitative techniques * Illustrating and fleshing out findings from quantitative research * Ruling out hypotheses. In some cases, qualitative research is the best way of getting certain kinds of information. 161 162 Research Techniques: The Basic Tools In quantitative techniques the instrument is the questionnaire, test, or measure. In qualitative methods, the instrument is the researcher. Thus part of the reason why rules for qualitative research have never been set forth as clearly as those for surveys is probably because a lot more depends on the researcher's good sense, experience, and personality. Used properly, qualitative techniques can give you a richness and depth that you are unlikely to obtain through other methods. Used carelessly, they produce material with no more value than a tourist's snatched impressions. As always, using a variety of methods to strengthen your in- formation is a good idea. Some of the techniques in this section are time-honored methods and others are less common. You should adapt all of them to your own cultural situation and requirements. Three comprehensive books that focus specifically on qualitative research in education are Dobbert (1984); LeCompte, Millroy, and Preissle (1992); and Wolcott (1994). UNSTRUCTURED OR INFORMAL INTERVIEWING Unstructured or informal interviewing is often used at the beginning of a piece of research to get a broad pic- ture, or because you do not know what is important to ask. "Tell me about..." is a good way to start. Another, which can produce useful informnation, but has to be used in a way that shows you are trying to learn rather than being frivolous, is "What questions should I be asking you about this?" Of course, people may then won- der why you are the researcher and they are the interviewees, but being humbled is not the worst thing that can happen to a researcher. You can also use informal interviewing to cross-check pieces of information, to fill in details, to explore new areas as they arise, and to take advantage of unexpected opportunities. Informal interviews have the appearance of conversations, and follow the social rules appropriate to the people you are working with. The difference is that if it seems acceptable and practical, you will be taking notes (if not, you have to take them later). When analyzing and presenting what people tell you, you must, of course, be careful about their integrity and uphold the trust they placed in you. Emic Techniques for Interviewing In chapter 3 we discussed etic and emic research. In etic research you determine the agenda and the cate- gories in which people give information back to you. In emic research you try to see from within, to discover perspectives, interests, and problems as seen by the insider. In emic research the respondents rather than you decide what information is important, and they create the categories for the interview. This is particu- larly important in relation to girls' and women's concerns, such as labor force activities, because many es- tablished categories of information, such as certain census categories, were created for male activities. Some applications of emic research require considerable sophistication and are quite time consuming. The following discussion presents a more general way of using the approach to complement other tech- niques, as a rather unusual type of unstructured interviewing. Nevertheless, however unstructured the technique may appear to the observer, its purpose is to discover the structures, categories, and groupings that the people you are working with use to organize the way they see things. EMIC INTERVIEWS Chapter 4 already provided an example of emic questioning using the grand tour approach developed by anthropologist James Spradley (1979). Here is another example. Remember, the aim is to learn how people categorize things, ideas, concepts, processes, and so on. The only question that you create is the first one. The rest arise from what the respondent tells you. Q.1 What kinds of courses are offered in this center? A.1 There are skills courses and crafts courses and technical courses. Unstructured and Semistructured Intervietws 163 Q.2 What kinds of skills courses are there? A.2 Life skills and catering. Q.3 What kinds of life skills courses are there? A.3 Childcare, cookery. Q.4 What kinds of childcare courses are there? A.4 There are no kinds, just one course. Q.5 What kinds of catering courses are there? A.5 Cookery. Q.6 What is the difference between life skills cookery and catering cookery? A.6 Girls take life skills, boys take catering. Q.7 What kinds of craft courses are there? A.7 Food preparation, mechanics, and general woodworking. Q.8 What kinds of food preparation courses are there? A.8 Cookery and food technology. Q.9 What is the difference between this cookery course and the other two? A.9 You need to have finished primary school to do this cookery course. Q.10 Do you need to have finished primary school to do the other craft courses? A.10 For mechanics, yes. Not for general woodworking or food technology, they aren't taught in pri- mary school. Next year they will be. Then you'll have to have finished primary school to take it. Q.11 What kind of technical courses are there? A.11 Cabinet making and mechanics. Q.12 How does this differ from general woodworking and mechanics? A.12 You can get a certificate in the technical courses. The others don't give you a certificate. And so on. When your understanding is complete, you could construct a diagram-a taxonomy or a tree-showing types of courses and distinguishing features, including the fact that for crafts courses, if there is a primary school base course, it is a prerequisite to entry (table 11-1 and figure 11-1). Table 11-1. Courses Offered by the Center Presented in Tabular Format Nonceri&aJe 1 Cerafe Tecchnicl Skills courses Cra courses courses I Primary schol I Primary school Cabinet making Boys lolsculicat uceriNleale not required required Mechanics Catering tifi skills General Cookery woodworking Mechanics Cookery Ch'Ilcare Food fechnology Meais 164 Research Techniques: The Basic Tools Figure 11-1. Course Offered by the Center in Tree Form Courses in center Noncertificate Certificate Skills courses Crafts courses Technical courses Boys Girls Primary certificate Primary certificate not required required Catering Lifeskills General Food Cookery Mechanics Cabinet making Mechanics \ woodworking technology Cookery Childcare You might then move on to try to discover the characteristics of the groups you have identified by look- ing for similarities and differences among them. Suppose you asked girls in a primary school what kinds of vocational and technical training programs they knew about. You could then try to discover what distin- guishes them in the girls' minds, and might come up with a chart like that in table 11-2. Table 11-2. Knowledge of Center Courses by Girls at Primary School Religious Qualification Program requiremnent ? Cost ? Prerequisite? wvhen finished ? St. John's Yes High Secondary Diploma education Automotive training No Medium Primary Certifica te Business college No High Secondary Diploma Mrs. Ali No Low Middle school None; finds you a job YWCA Yes Low Middle None Policelady No None Secondary Gives you a job school plus pull (you need to know someone) Table 11-2 is actually an example of a componential analysis, which we look at as a method of analysis in chapter 16. CARD SORTS OR PILE SORTS We saw examples of card sorts in chapters 4 and will come across them again in chapter 15. In this proce- dure you ask people to sort cards, photographs, drawings, or objects into categories of their own choosing. For example, give people a set of cards with the names of people, types of education, jobs, or activities and ask them to sort them into groups. Then ask what the categories are, what their characteristics are, what Unstructured and Semistructured Intervieuws 165 makes the people or items in one pile more similar to each other than they are to the items in another pile. Keep repeating the process. As you are looking for shared cultural knowledge, not personal idiosyncrasies, you need to ask a number of people. You will know when to stop when the categories become repetitive. Everyone's categories will not necessarily be identical; see if you can find the common ones. You can then check your perception of the categories by doing the process the other way round: give someone the pack of cards, list the categories, and ask that person to sort the cards into the categories. If the sorter has some people or items left over or cannot fit the cards into the categories, they may not be meaningful, and you might have to start over. You might give someone a pack of cards with names and ask her to sort them into who is likely to remain in school and who is not likely to remain in school. Then ask why. Nonliterate people can simply go through their community, mentally, house by house, and categorize the girls in each. TRIADS To get more refinement on the card sort method, you can present people with groups of three cards or items and say "Choose the one that doesn't fit" or "Choose the two that seem to go together," and then ask "Why?" Regroup the items within other sets of three cards and repeat. This allows you to understand the grounds on which some things share characteristics, for example, "Which two are more alike in this group, mother, daughter, niece?" tells you whether the direct line of family or the generation is more relevant in this particular context. "Which two are more alike, trained teachers, untrained teachers, trainee teachers?" may tell you something about experience if the first two are grouped together, something about status if the last two are, and something about the importance of professional qualifications if the first and third are. You can also use triads as a major research technique using large numbers of people and many items. These are still presented to people in groups of three in all possible combinations. Then you need to work out how often each possible pair is selected. See Bernard (1994, pp. 231-34) for a discussion of how to work with the combinations that arise when using large numbers of respondents. You cannot read anything other than cultural information into your sortings when using either cards or triads. For example, in the Western world some experts think that categorizations based on perception (col- or, size, shape, number) reflect a simpler level of analysis than sortings made by function (things that are used together or have a similar purpose). Sortings based on taxonomies are thought to be the most sophis- ticated of all (putting certain plants or animals together because they belong to the same species or family). Although schooling can create a tendency to classify by taxonomy, people are more influenced by their cul- ture when grouping things together. This is useful to know, because what you are trying to get is cultural information, not individual intellectual or personality assessments. Semistructured Interviews In semistructured interviews you do not have a standard interview form. You have an agenda that you use as a reminder to ensure that you eventually cover the basic points, but your questions are tailored to the individual or category of person and to the circumstances. For example, you may want to know what kinds of skills training programs should be developed for young women. Some of the people you may be ap- proaching will be government officials, nongovernmental organizations heads, employers, old people in a village, young women, and girls, each requiring a different approach to the interview, including different wording, order, and length. The issue is not simply a matter of people not understanding your questions if you do not adapt them. If you fail to use appropriately professional language to experts, they may feel you do not know enough to warrant their expenditure of energy on you; if you use professional or official sounding language on people who are wary of officials, they may not cooperate; if your language is not suf- ficiently respectful to the elderly or sufficiently sensitized for strongly committed people, you may get a lec- ture rather than your answer. 166 Research Techniques: The Basic Tools Good books on actual interviewing rather than on how to approach people are difficult to find, because sense, style, empathy, and on-the-spot thinking and responsiveness are more important than sets of guide- lines. However, some general pointers follow. Some of these are also useful for participant observation (see the discussion later in this chapter) because observation usually involves asking questions as well. Before the interview * Develop a brief list of points, perhaps ten at most, or a mental map of what you want to know, but be prepared to be flexible. Because these kinds of interviews are ideal for getting more "in-depth" information, you will lose this advantage by asking too many questions. * Choose interviewers or decide whether you yourself can do a particular interview, remembering that all else being equal, people are usually more comfortable with people like themselves in terms of age, ethnic group, gender, social standing, and so on, although sometimes they see an "expert" as more broad-minded, unshockable, or impartial. * Choose the people you are going to interview. You can select them on a probability sampling basis, but you are more likely to have some other basis on which to choose them: some people have spe- cialized knowledge, some people have undergone an experience that you need to understand, some people represent a good example of a pattern you have found, some people fly in the face of every- thing you thought would be the case. During the interview * Behavior is important. Observe local courtesies: for example, not sitting on a higher seat than a se- nior person is important in some cultures or not pointing a finger at anything in others. Try to use local conversational patterns and colloquial language. Introduce yourself, explain briefly the pur- pose of your study, tell how the person was selected, say about how long the interview will take, and ask if it is convenient for the person to talk to you now. Fit your interviews in with people's timetables. Offer to help people with what they are doing, if appropriate. Do not lecture, do not show surprise or shock or distaste. When you ask a question, let people talk. Do not interrupt them or finish their sentences for them. Listen to what they actually say, not what you expect them to say. * Choose the questions. Where do you get your questions? If you are using the research outline ap- proach (chapter 4), "interviews" will be listed beside some of your points: Outline Technique Source Parents' attitudes toward Interviews Parents of girls in - girls' abilities Grade 6 Convert the outline point into a question or questions. If you are using an emic approach (chapter 4), you have a grand tour question: "What happens in this science class?", and perhaps a card sort or a triad: "Of these three (things, people, processes) which two are more alike and why?" "Why is the other one different?" These will lead you to more questions for clarification. If you are using the grid approach shown in chapter 4, you can use the questions in the boldly outlined boxes to give you a basic picture and any others that may be relevant. * Ask the questions. In organizing your questions, try to ask for neutral facts first, eventually moving on to the interviewee's opinions and to any sensitive issues. Take the trouble to find out from some- one who is sympathetic to your research what the sensitive issues might be, such as girls' circumci- sion or local feuds. Do not try to discuss them until the interviewee is comfortable with you. This may take several visits. People may not discuss some topics at all. If you really need the information, Unstructured and Semistructured Interviews 167 see if you can get it in other ways, for example, can you use a proxy indicator (see the discussion in chapter 7)? Sometimes you can deal with sensitive issues by showing that you already know something about the subject: "As I understand it, some people think the teacher should not have been transferred for having sexual relations with a girl of twelve because she was old enough to be married. Others feel it was rape and it was right that he had to leave. What is your own feeling?" If you simply asked, "Should a teacher be punished for have sexual relations with a pupil?", people might say "I don't know. We don't have problems like that here." An interview can give greater depth than a questionnaire, because you can probe-encourage peo- ple to expand on their answers-and cross-check information. Chapter 9 mentioned some probing strategies that are also appropriate for interviews: - Ask questions that allow people to develop their answers, not questions that can simply be an- swered by "yes" or "no." - Pursue useful information further by asking questions that will tell you "Who?", "What?", "Where?", "When?", "Why?", and "How?" as appropriate. - Encourage people to expand on an answer by pausing after the reply, and perhaps giving some sign of encouragement, and using phrases like "Can you tell me more about that?" "Can you give me an example?" - Encourage people to clarify their answers: "Let me see if I understand this correctly. You say all your children are now at Oxford?" - Cross-check the answers by phrasing the question slightly differently: for example, to someone who wants to send a daughter to school but cannot: "What are the main reasons why you can't send your daughter to school?", and later, "What would help you to send your daughter to school?" Observe: people communicate both verbally and nonverbally (through actions). The better you know a culture, the better you will be able to "read" its people. Noticing interviewees' gestures, body movements, hesitations, pauses, tones, and other signs may help to give you a better under- standing of what they are saying. But do not jump to conclusions. In your notes record both what they said and what they did, rather than your interpretation. After the interview * Thank the interviewee. If you are not going to see this person again, tell her or him what to expect next. Are you eventually going to hold a village meeting to discuss your findings? Is a report going to come back to the chief? Will a delegation come to the village? Will a departmental meeting be held on the issue? Although you can use what you learned to develop new questions, and in subsequent interviews you can test the ideas you gained, never carry information from one interviewee to an- other. * Evaluate the answers. For many researchers this seems to be a more important issue in infor- mal and semistructured interviews than it is in surveys. However, the problem is the same, no matter what the technique, and no foolproof way exists to ensure that what someone tells you corresponds to what the person actually thinks or experiences, no matter how many cross-checks you insert. However, this is a good opportunity to make a few points that people often forget: - Often you are not really interested in the truthfulness of individuals. You are studying social and cultural factors, and unless an entire group is deliberating deceiving you by every means 168 Research Techniques: The Basic Tools available to them, this is not an important issue. The more important issue is your own ability as a researcher, which is a far more common source of major error. - If you are using triangulation (multiple methods, researchers, sources, and so on as discussed in chapter 7), serious errors based on deception should emerge. You will find discrepancies be- tween what people say, do, and may be recorded as having done. Also, experience and theory in your own field may tell you that something cannot be the case. For example, an anthropol- ogist who is told that a group of people who are herders trace their descent through women and observe the practice of giving a bride price when a girl is married to her mother's sister's son would hear a warning bell go off. These customs do not go together anywhere, and the combination does not make sense. Your own field probably has "bells" like this. - What people say and what happens are often two different things. When a culture is changing, for example, people often fail to notice that the ideal and the practice no longer correspond. "Daughters go to live with their husband's people when they marry," they say, and yet you discover that almost no household reflects this pattern, perhaps because times have got worse and young couples go to live wherever their best chances lie, but people still hold to the ideal. You may recall the experience of researcher Ingrid Palmer (1981, p. 34) as noted in chapter 7: an official in the Gezira told her that women did not work in the fields, but at the same time she saw a field of women working. When she asked the official about it, he said they were "helping out." See box 11-1 for a discussion of why people "lie." Box 11-1. Why Do People "Lie"? In the case of the Gezira example, several explanations are possible: . The official was lying, or was just wrong. . It is a cultural myth and the facts have never supported it. "Be- cause of their lesser physical strength, women are protected from hard labor" is a myth in the Western world, easily contradicted by watching a mother wrestle a pram, children, and shopping onto a bus. Women are protected from hard paid labor. . The cultural pattern was changing rapidly and people still thought these instances of women working in the field were exceptions. . What people in Gezira consider to be working in the fields may re- fer to some other part of the agricultural cycle or another agricul- tural activity. . This was an odd event. Women don't work in the fields. . This is a man's domain and anything women do is considered to be "helping out." * They were working in the field, but shouldn't have been for legal, cultural, or other reasons. When people say one thing and do something else, you can often learn much more about what is going on than when everything matches. Of course, first you have to know enough to know that things do not match. This can be difficult when the subject is sensitive and not easily checked. Wolf Bleek (1987), who studied subjects such as famnily quarrels, suspicions of witchcraft, sexual relationships, and birth control practices, including induced abortions, in a Unstructured and Semistructured Interviews 169 Ghanaian town, provides one of the most striking examples. He lived with and studied one lineage (or large family group) that included nineteen women of childbearing age. He concluded after some time that induced abortion was common among the women. However, at the time of his research, he notes that "statistical data enjoyed higher esteem than at present," and therefore he arranged for 179 women to be interviewed during their visits to a child welfare clinic. The women reported few induced abortions and little use of birth control. The nurse-interviewers believed that many of the women were lying. Only 4 percent of the 179 women admitted to an induced abortion, while more than half of the 19 women in the com- pound did; onlyl4 percent admitted to ever practicing birth control, while 63 percent of the women in the compound did. By accident, Bleek discovered that six of the lineage women had taken part in the survey. Table 11-3 compares the answers given by two of them on the survey, versus what Bleek discovered as a result of coming to know them in the compound. Table 11-3. Comparison of Two Women's Answers Information obtainedfrom survey interview Information based on participant observation Name: Opokua Married both according to custom and in Roman Cath- olic church Unmarried Lives with husband Has no husband Eats with husband Has no husband Has been pregnant once Has been pregnant twice Has induced an abortion with a tablet (her husband Has induced an abortion at the request of her boyfriend; helped her) she used a mixture of herbs, milk, and sugar Name: Nyamekye 24 years old 31 years old Divorced once Divorced twice Given birth to four children Given birth to six children (two have died) Has been pregnant four times Has been pregnant at least nine times Lives with husband in Accra Lives without husband in Kwahu town A female servant prepares her meals She has no servants Has never used any method of birth control Has experience with many methods of birth control Never had an induced abortion Had at least three induced abortions Source: Bleek (1987, pp. 314, 317-20). In retrospect, Bleek concludes that the women had reason to lie to the survey researchers, because abortion was not only very private, but also a criminal act. He also realized that his "detective-like" approach with the nineteen women, while revealing hidden facts, prevented him from recognizing their serious social and psychological importance. Had the women had a chance to conceal their be- havior, he would have seen how serious a subject they considered it to be, and more of what it meant to have an abortion in Ghanaian society. Remember as well that people's beliefs and perceptions are as important as what they may actually be doing at the moment, even though what they are doing seems to conflict with what they say. What they say may be the general pattern, all other things being equal; what they are doing at the moment represents an attempt to accommodate the pattem to the circumstances. Both are impor- 170 Research Techniques: The Basic Tools tant. Beliefs are facts just as much as activities are. Do not fall into the trap of thinking that beliefs are subjective and what people do is objective. Both are, or neither are, depending on your philo- sophical approach (see chapter 2). Because many cultural patterns are shared, do not discount anyone on the grounds that he or she could not know anything useful. Everyone has a sufficient command of his or her own cultural rep- ertoire to be able to function. Some people are better at communicating this than others; predicting who is difficult. Do not persist with people who cannot or will not participate. You do not have the time. * Write up your notes immediately. Record - The circumstances in which the interview took place - What you were told, in direct quotes where possible - What you saw - Any judgments of your own that you think might be useful ("I think Mr. Okoro was more at ease before his wife joined us"), and record them separately. See chapter 15 for advice on recording techniques. CULTURAL BIAS You might think that because people converse everywhere in the world, the interview would be less liable to cultural bias than, say, questionnaires and measures. Certainly, the interview can appear more natural, particularly when you observe proper local protocol. Still, even an informal interview is not the same as a conversation because the expectations of both the interviewer and the interviewee are different. Two researchers, Slim and Mitchell, have discussed some other problems. One is the researcher's expectation that people have an answer to or an opinion on every question. The second is "nutshell- ing," the Western notion that people should give a brief, to-the-point summation of their experience or knowledge in response to a question, whereas in many cultures talking through and considering all angles is the most judicious way of presenting one's ideas. Both these notions are contrary to the experience of many people in Africa and other parts of the world: they know that "one cannot know everything, and that the little one knows cannot be uttered in a moment" Slim and Mitchell (1992, p. 69). A third problem, which applies to surveys as well, is that most people prefer a two-way ex- change, or even more. People may want to know what you think, how many children you have, and may want to call in a few neighbors or family members as well to ponder some of your questions in a more companionable way. Often, this does not matter. You are trying to understand what is hap- pening, how, and why, so if this helps you, fine. If it interferes with your learning what you need to know, you will have to develop strategies to withhold your own opinions without seeming rude, and figure out how to talk to people privately. Other useful books on this topic are McCracken (1988) and Spradley (1979). Key Informants A special kind of interviewee is the key informant. Key informants may be people with a particular spe- cialization, such as school planners, or they may represent a particular viewpoint or experience. People want a primary school in their area. How can it meet the needs of various groups? Someone with a handi- capped child, someone who leads a movement to exclude girls from all but religious schools, a member of a nomadic group who can tell you the problems nomads have in sending children to schools can each tell you something about the needs that may have to be taken into account. Unstructured and Semistructured Interviews 171 There is another kind of key informant who is much more difficult to find: the unusually insightful per- son who can provide analytical information, context, or insights in relation to what you are studying. The ability to make valid generalizations beyond one's own immediate situation and experience is in part a function of personal sagacity and in part an exposure to a range of experiences. You may be fortunate enough to find such a person. You still must use all the other techniques at your disposal, of course. No one person can provide a total picture. The reliance of some anthropologists in the past on a few male key in- formants alone has led to very peculiar interpretations of some societies. Key informants can help to explain a technical process, put parts of a pattern together for you, or show how facts and experience come together. You can also use their services in a more organized way. For in- stance, you have identified four categories of households in a village: some send all their children to the state school, some send all their children to the religious school, some send boys to one and girls to another, and some send none. You have learned something about these families through other methods, a short sur- vey, for example, or a card sort. Now you want to know more about their circumstances and attitudes. You could interview based on a probability sample, but you could also choose some people deliberately for their ability to provide valid insights beyond their own experience, that is, key informants. Selecting key informants is a form of purposeful sampling (see chapter 6). You may choose them because they are typical, that is, they represent a pattern or attitude or have experienced something that you have identified as common, or you may choose them because they are unusual, that is, they hold a different view, their experience has been different, they have more expertise. Key informants are not necessarily representative or "important" or "official" people. They can be min- istry experts or teachers, but they can also be an elderly low caste person, an astute observer of the local political situation, or a mother who has helped in the school for many years. Sometimes "marginal" people are good key informants. These are people who, for whatever reason, have been forced to watch the situa- tion at a slight remove, or have had to study and accommodate to the ways of others in order to function, such as members of minority groups. Whoever you choose, you must understand the advantages and lim- itations of their viewpoints. Sometimes, of course, people see others as key not because they are able to give a broader picture, but because they or their positions are due respect, or because they have a vested interest, or because they can influence events and decisions. You may be wise to consult them, while recognizing that they are pivotal people rather than key informants. Interviewing key informants requires careful preparation. Each probably requires a specially designed set of questions. Once again, flexibility is important. For further information see Johnson (1990) and Kumar (1989). Group Interviews So far, we have been discussing interviews with individuals, or perhaps with several people who are sharing a household or working together, but you can also use semistructured interviews with larger groups. Two common types are community interviews and focus group interviews. These are not cheap substitutes for surveys or indi- vidual interviews: they give people the opportunity to put aside individual considerations and discuss something in the larger community context. As researcher Budd Hall ((1981, p. 16) has pointed out: "Responses to problems offered by groups of people are not necessarily the same as the sum of individual responses of people speaking alone." Group interviews are useful for getting a general picture of a place and its needs; developing tentative hy- potheses; getting a better understanding of material you have collected; or finding out what people think about the current situation, a planned project, or a completed project. You put a small number of carefully selected questions or topics to the group, for example, to parents who are worried about girls' security in school: * What are the most worrisome situations? * Which girls are most vulnerable? 172 Research Techniques: The Basic Tools * What is being done about the problems now? * What might be done? * What would this involve? * What would it accomplish? COMMUNITY INTERVIEWS In community interviews you are seeking information from a "natural" group, one that comes together at other times for a common purpose, such as a village meeting or an organizational discussion. Many people may be present, although groups larger than thirty are probably too big and may have to be broken up. In- dividual views are not sought. The aim is to put a small number of questions to the group for general dis- cussion: "How can the security of girls be ensured at school?" "Would a credit union help with the problem of school fees?" Sometimes a research project is based almost entirely on community interviews: for exam- ple, certain categories of beneficiaries of a project may need to be consulted in depth. If you are using com- munity interviews for this purpose, you have to consider how you are going to select the communities. You are most likely to use some form of purposeful sampling (see chapter 6). One problem with community interviews is that you do not necessarily know what, if anything, the group represents. Who didn't come? Why? You cannot automatically take the results of a community in- terview as "the voice of the people." Also, even though you are working with a group, you may still be get- ting individual views. Certain categories of people-elders, men, the dominant ethnic group, the wealthier, the people already involved, a faction, people who agree or disagree with what they think your position might be-may dominate the meeting. If you anticipate these kinds of problems, you can consult members of the various groups in advance, and at the meeting, say "I've already discussed this with the [elders, teachers, staff of nongovernmental organizations] and would like to hear what others here think." Others may not be accustomed to putting their views forward or may fear repercussions, such as com- munity ridicule, loss of business, or a host of other possible dangers. For people likely to be left out, you could say "The women who are selling food at the school may have some concerns about this. For example, is - a problem?" You must use this approach carefully. If people are still afraid to speak, it may sound to others as if they have no problems. If you are not really familiar with their problems, your example may direct the discussion away from their real concerns, and if you reveal something they thought they were saying in confidence, they will probably never trust you again. Focus GROUPS Focus groups differ from community discussions in a major way: in community discussions the major in- teraction is between the researcher and the group, whereas in focus groups the interaction is among the members, who work through an idea, issue, or problem that the researcher has selected. Focus groups consist of people you have specially selected for their experience in relation to whatever you are studying. Usually, the group consists of six to twelve people of similar background in terms of age, sex, class, and so on who are brought together to discuss a small number of questions or issues-no more than ten-for about an hour or two. For some purposes meetings are longer, and may occur over a period of days. Sometimes a more complex strategy is used: a researcher starts out very generally with one group, allowing it to map out the discussion, and later, using what was learned from the first group, creates new types of focus groups to explore points identified in the first session. In focus groups the group interaction, rather than answers to questions, produce the insights. People may argue points, correct one another, give exceptions, and support their points with examples from their own experiences. An approach like this can save time if planned properly. Unstructured and Semistructured Interviews 173 Let us say you are trying to find out how to meet the needs of families who have to take their girls out of school to work at home. You have discovered that the girls mind younger children, take food to their mothers in their gardens, and scare birds away from the fields. What ideas do people have about how these tasks might be carried out so the girls could go back to school? This is a kind of brainstorming. Parents who are in this position know their own problems best. Feedback from them will help you to assess your ideas. Or perhaps you have worked out some ideas for providing day care, establishing a system for sending food to mothers each day, and for deterring birds. Will these work? What else are girls doing that might not have been identified? What else do they need to go to school? You will probably need other focus groups as well. Perhaps various nongovernmental organizations have tried some of these ideas before. Their rep- resentatives may be able to tell you what happened and how you should amend your plans. Another possibility is that your scheme has already been implemented and does not work. People are still keeping their daughters at home. Why? A focus group may help here too. For some topics you might use a number of separate groups. For example, if some kind of further edu- cation is being considered for girls who have left school because of early marriage, you might hold a series of discussions with groups with different viewpoints: girls with children, girls without children, husbands with different levels of education and no education, and so on. HOLDING A GROUP INTERVIEW Controlling a community interview or a focus group can be difficult for a novice researcher. Feelings may be high, some people may overwhelm the meeting, or someone may hijack the meeting for another pur- pose. Kumar (1987) has outlined some techniques for preventing this. They are presented in box 11-2. Box 11-2. Techniques for Maintaining Control of a Group Interview * Give nonverbal cues to the respondent to stop, such as looking in another direction, showing a lack of interest, and stopping note taking. . Politely intervene, saying that you have somehow missed the point and would like to sumrmarize what the respondent was saying, then refocus the discussions. • Take advantage of a pause and say that the issues raised are of vital significance and should be discussed in a separate session. Source: (Kumar 1987). You probably called a community interview meeting or a focus group to get more information faster. The temptation is to speed up the process even further by drawing hasty conclusions from your sessions; hearing what you want to hear; and accepting the ideas of the "more important" people, the more articulate, the more potentially troublesome, the people most like yourself. Try to ensure that people with other views are heard. Did anyone have a different experience? Does someone think an idea will not work? Is it true for the women? Is it true for people who live in different parts of the area? Drawing conclusions that rely mainly on community interviews or focus groups requires considerable experience and skill. Beginning researchers should use a variety of techniques. A group interview approach may help you to complement information you obtained by interviewing people on their own, by observing, or by looking at records and documents. 174 Research Techniques: The Basic Tools Our Girls In our study of X District girls (chapter 4), unstructured and sem- istructured interviewing would help us before and during the re- search. At the beginning of our study, we might interview knowledge- able people about the social and cultural characteristics of the group and about the form of the school system (both in box 4-4). Before we plan box 4-6 "Reasons for Completing Education at Grade 6," we should talk to individual parents, teachers, and girls to explore the range of possible reasons. We might also hold some group discussions. During the research, if we find some unusual cases, for example, girls who are not leaving despite every obstacle, we might inter- view them to see what motivates them. FURTHER READINGS TO HELP YOU Bleek, Wolf. 1987. "Lying Informants: A Fieldwork Experience from Ghana." Population and Development Re- view 13(2):314-22. Dobbert, Marion Lundy. 1984. Ethnographic Research: Theory and Application for Modern Schools and Societies. New York: Praeger. Grandstaff, Sornluckrat B., and Terry B. Grandstaff. 1987. "Semi-Structured Interviewing by Multidisci- plinary Teams in RRA." Proceedings of the 1985 International Conference on Rapid Rural Appraisal. Khon Kaen, Thailand: University of Khon Kaen. Johnson, Jeffrey C. 1990. Selecting Ethnographic Infortnants. Newbury Park, California: Sage. Kreuger, Richard A. 1988. Focus Group Interviezvs: A Practical Guidefor Applied Research. Newbury Park, Cal- ifornia; London; New Delhi: Sage. Kumar, Krishna. 1987. Conducting Group Interviezws in Developing Countries. Washington, D.C.: U.S. Agency for International Development. .1989. Conducting Key Informant Interviews in Developing Countries. Washington, D.C.: U.S. Agency for International Development. LeCompte, Margaret L., Wendy L. Millroy, and Judith Preissle, eds. 1992. The Handbook of Qualitative Re- search in Education. New York: Academic Press. McCracken, Grant. 1988. The Long Interviezv. Newbury Park, California: Sage. Morgan, David L. 1988. Focus Groups as Qualitative Research. Newbury Park, California: Sage. Spradley, James P. 1979. The Ethnographic Interviezv. New York: Holt, Rinehart and Winston. Wolcott, Harry F. 1994. Transforming Qualitative Data. Thousand Oaks, California: Sage. What case studies and participant observaUon are good for: Examples: Getting an in-depth picture of a situation, a process, or an experience How has a particular principal managed to make schooling so attractive to local peo- ple? A case study of her approach would teach us something. Teenage schoolgirls limit their future options when they become pregnant. A case study of one well-chosen girl's experiences would illustrate this. Participant observation: Seeing what happens, and perhaps participating. Examples: What Ls invohved, in temns of lime, effort, and siLs, in girls' domesti work civies? Do girls behave differently in single-sex versus mixed-sex schools? Sample studies: Case studies-Wai, Nyakwea. 1994. Images of Women in African History: Wangu Wa Makeri: A Pioneenng African Feminist Profile No. 2. Nairobi: Kenya Forum for Af- rican Women Educationalists. Participant observation-Schuster, llsa. 1979. New ...... ... 12 2 a - Two Qualitative Strategies: Case Studies and Participant Observation =_ -~ ~ smmr s u m m a r y Case studies and participant observation are not usually grouped together. Here they are, because both are strategies, that is, combinations of techniques, rather than single ones. Case studies allow you to collect and present information in a way that provides more context. They are goodfor showing how something happens or works in a real life situation. Participant observation is a strategyfor interviewing, observing, and sometimes participating. It can be used to get preliminary information, to check other information, and to understand something in its larger context. You may remember that in chapter 7 we emphasized the importance of triangulation of techniques, that is, us- ing several techniques to get the same information. Both case studies and participation have built-in triangula- tion: each involves using a variety of approaches to get a better understanding of what you are studying. CASE STUDIES Case studies prov