Using qualitative assessment methods rather than purely data-based information is crucial to understanding many community issues and needs. Numbers work well to show comparisons, progress, an statistics of community efforts, but they cannot express motives, opinions, feelings, or relationships. This section discusses how to use qualitative assessment methods and when to implement them into communtiy planning. Show
What are qualitative methods of assessment?Qualitative methods of assessment are ways of gathering information that yield results that can’t easily be measured by or translated into numbers. They are often used when you need the subtleties behind the numbers – the feelings, small actions, or pieces of community history that affect the current situation. They acknowledge the fact that experience is subjective – that it is filtered through the perceptions and world views of the people undergoing it – and that it’s important to understand those perceptions and world views. There are two major scientific ways of gathering information: quantitative methods and qualitative methods. Quantitative methods are those that express their results in numbers. They tend to answer questions like “How many?” or “How much?” or “How often?” When they’re used to compare things – the results of community programs, the effects of an economic development effort, or attitudes about a community issue – they do it by subjecting all of the things or people they’re comparing to exactly the same tests or to the same questions whose answers can be translated into numbers. That way, they can compare apples to apples – everything or everyone is measured by the same standard. Quantitative measures are often demanded by policy makers; they are considered trustworthy because their results can be measured against one another, and because they leave less room for bias. Qualitative methods don’t yield numerical results in themselves. They may involve asking people for “essay” answers about often-complex issues, or observing interactions in complex situations. When you ask a lot of people for their reactions to or explanations of a community issue, you’re likely to get a lot of different answers. When you observe a complex situation, you may see a number of different aspects of it, and a number of ways in which it could be interpreted. You’re not only not comparing apples to apples, you may be comparing apples to bulldozers or waterfalls. As a result, researchers and policymakers sometimes see qualitative methods as less accurate and less legitimate than quantitative ones. That can be true, but, as we’ll see, if qualitative methods are used with care, they can also yield reliable information.
There are a number of qualitative methods that can be used in assessment of issues or community needs. We’ll list the major ones here, and look at them in more detail later in the section. They include:
Many types of qualitative information are turned into numerical results, although not always accurately. The transformation may miss important details, or the information may simply be too complex to fit easily into numerical constraints, unless you can create a computer model or similar number-based framework that has the capacity to take in an enormous amount of variety. There are many software programs – NVivo and Atlas.ti are fairly well-known, but there are many others, including some freeware – that are intended expressly for analyzing qualitative data. Since qualitative methods give you results that are not always easy to compare, or even to check for accuracy, people who want hard and fast evidence often see them as suspect. In fact, both quantitative and qualitative measures are important and necessary, depending on the situation. When you’re assessing community issues, as we’ve discussed, you’ll often get closest to the complete picture by using both. The problem is convincing those who need to be convinced – policymakers, funders, etc. – that your qualitative measures are reliable. There is a debate in the research community about how to judge qualitative methods. Some say they should be evaluated by the same standards as quantitative methods. Others maintain that, because they are intrinsically different from quantitative methods, qualitative methods need a set of standards that take into account their philosophical base and the kind of information they yield.
Guidelines that can help you argue for the reliability of your qualitative assessment include:
Why use qualitative methods of assessment?The basic reason to use qualitative methods is that there are some kinds of questions and some dimensions of community assessment that can be better addressed by them than by quantitative methods. The methods you use should be determined by the questions you’re asking. Since it may be hard to convince policymakers and others that qualitative methods are useful, however, why bother to use them at all? Some of the major reasons:
When would you use qualitative methods of assessment?Clearly, there are times when quantitative research will give you the information you need. So when do you use qualitative methods? It depends to a great extent on the question you’re asking. (The first four situations below are based on a USAID guide to using rapid appraisal methods, Performance Monitoring and Evaluation Tips.)
How do you use qualitative methods of assessment?Now that you’re convinced of the importance of using qualitative methods of assessment, how are you going to do it? There’s seldom one right way to do anything, but we’ll offer some steps to take in using qualitative methods, including some guidelines for doing interviews and observations, the two most common methods. (Most of these guidelines hold equally for using quantitative methods as well.) Start by deciding what it is you want to know.You may remember that this is also one of the guidelines for qualitative reliability. It may seem elementary, but it doesn’t happen anywhere near as often as it ought to. The importance of deciding what you want to know is that it determines the character of your assessment – what kinds of questions you ask, whom you ask them of, how you’ll go about it, etc. Without that minimal amount of structure, you’re likely to wind up with a confused and unorganized mass of information. There are many ways to approach a community assessment, and, consequently, many questions you might choose to start your assessment with. You might even use more than one, but it’s important to be clear about exactly what you’re looking for. Some possibilities:
Choose the method best suited to finding the information you’re looking for.If you want to learn about people’s public behavior, you would probably use direct observation. Observing mothers and children in a clinic waiting room, for example, might give you information about the mothers’ anxiety levels or child-rearing practices. If you want to know people’s opinions or how they feel about issues, some type of interview would be appropriate.
Choose the people who will gather the information, and, if necessary, train them.With qualitative methods, where contact is often personal, the question of who carries them out can be very important. Academics or others who are perceived by community members as “the other,” whether because of their behavior, their speech, or simply because they’re outsiders, may find it hard to gather accurate and complete information from a population that’s very conscious of class or cultural differences. Often, it makes more sense to train members of the population or others who are known and trusted by – or at least familiar to, in their behavior, dress, and speech –those who are being asked to contribute their opinions and observations.
If you recruit members of the community or of a specific population to do qualitative information gathering – because they relate to the population better, because they speak the language, because you’re engaged in a participatory effort, or simply because you think they’ll be good at it – you should provide them with training to make sure that the results they come up with are reliable. Depending on what kinds of methods they’ll be using, some of the elements of a training might be:
Determine from whom or from where you need to gather information.It may be that you want to hear from all sectors of the community, but some issues or circumstances demand more specific informants. Some possible interview subjects may be public officials, members of a specific population or cultural group, people from a particular geographic area, or people with certain characteristics (parents of young children, individuals with disabilities, males 18-24, people with high blood pressure).
Observation may or may not involve people. If it does, the question may not be whom you want to observe, but rather what activity or situation you want to observe. If it’s general – what kinds of street activity take place in various neighborhoods, how people use a public park – it’s not necessary to focus on a particular population, but rather on the place. If it’s more specific – back to commercial activity in that Hispanic neighborhood – you’ll need to be in the right place at the right time. Gather the information.Now it’s time for you or the people you’ve chosen to go out and collect the qualitative information you need. InterviewsAs mentioned above, interviews can be structured or unstructured. In a strictly structured interview, the same questions in the same order are asked of everyone, with relatively little room for wandering off the specific topic. Semi-structured interviews may also be based on a list of specific questions, but – while trying to make sure that the interviewee answers all of them – the interviewer may pursue interesting avenues, or encourage the interviewee to talk about other related issues. An unstructured interview is likely to be more relaxed – more like a conversation than a formal interview. There are advantages and disadvantages to each approach. A structured interview may make the interviewee focus in on the questions and the interview process, take it more seriously, and thus provide excellent information. Because everyone is interviewed in the same way, a structured interview may be – or at least may look – reliable. It may also make an interviewee nervous, emphasize the differences between him and the interviewer, and lead to incomplete or less-than-truthful answers. A semi- or unstructured interview may allow the interviewee to be more relaxed, and thus more forthcoming. It also leaves room for pursuing a topic that’s not directly related to the formal list of questions, but that might be important or even crucial. At the same time, because it can be far-ranging, a semi- or unstructured interview – particularly one that doesn’t start with a list of questions – is, or appears, less reliable than a structured one. It also, in the hands of an inexperienced or indecisive interviewer, may allow an interviewee to get sidetracked and never get back to the original questions. What kind of interview you use depends on the nature of the information you’re looking for, the needs of the people you’re interviewing (e.g., whether comfort is more important than structure), and your own comfort. The author has conducted all three types of interviews, and has found that semi-structured interviews – having clear questions and goals for the interview, but conducting it in an informal way, with room for pursuing tangents and some simple friendly conversation – is generally productive. The following guidelines for interviewing reflect that view.
ObservationWhat do we mean by “observation?” For our purposes, there are essentially two kinds: direct and participant observation. Direct observation is the practice of examining or watching places, people, or activity without interfering or taking part in what’s going on. The observer is the proverbial fly on the wall, often unidentified, who does nothing but watch and record what she sees and/or hears. A direct observation to see how people use a public park, for instance, might consist of one or more observers simply sitting in one place or walking around the park for several hours, or even several days. Observers might come back at different times of day, on different days, or at different times of year, in order to understand as much as possible of what goes on in the park. They might occasionally ask questions of people using the park, but in as low-key and unobtrusive a way as possible, not identifying themselves as researchers.
Participant observation involves becoming to some extent part of the life of the people you’re observing – learning and taking part in their culture, their celebrations and rituals, and their everyday activities. A participant observer in the park above might introduce himself into the activities he observes – a regular volleyball game, winter cross-country skiing, dog walking, in-line skating – and get to know well the people who engage in those activities. He would also monitor his own feelings and reactions to using the park, in order to better understand how its users feel about it. He would probably ask lots of questions, and might well identify himself as a researcher.
Both direct and participant observation can be useful in community assessment. A participant observer in that situation is likely to be a member of the group being observed, because of the length of time it can take to establish an outsider as a participant observer. Direct observation is probably more common as an assessment tool. Regardless of its type, your observation should be conducted so as to be reliable. Some guidelines for reaching that goal:
What should you observe and record? That depends on the questions you’re trying to answer, but some basics include:
How do you record observations? That depends on the nature of the observation and on your resources. Video recording, unless it’s done from a concealed spot, or in a situation where such recording is expected (a tourist site, or that street festival, for example), can change people’s behavior or put the observer under some suspicion. Audio recording is much less obvious, but also provides less information, unless it’s specifically sound information that you’re seeking. In most cases, recording would be done with a notebook and pencil or with a laptop computer. If recording during the observation would be disruptive or out of place, you’d probably wait till after you had left the situation – but as soon after as possible, so as not to forget or confuse details. Analyze the information.Once you’ve gathered information by whatever qualitative method, you have to figure out what it tells you. Some of that will be obvious: if you’ve been interested in who uses that public park we were talking about earlier, and your observation tells you that it’s mostly young people, you have an answer to your initial question. Your next questions may be why other groups don’t use the park as much, and whether the fact that it’s largely used by young people keeps others away. When you’ve answered those questions, you may have generated others, or you may have a basis for planning a campaign to get more people using the park. Make and carry out a plan to address the issue or problem you’ve identified or were concerned with.The final step here is to use the information and analysis that came from your use of qualitative methods to change the community for the better. All the assessment in the world is useless if it doesn’t lead to some action that’s meant to create positive change. In SummaryQualitative methods of gathering information – methods such as interviews, observation, focus groups, and community meetings that don’t always yield results that can be reduced to numbers, or that are used to capture a level of information difficult to get with quantitative methods – are often extremely useful in community assessment, especially when used together with quantitative methods, which do give numerical results. Qualitative methods can get at the things that numbers don’t, such as the reasons for people’s actions, or community history. They can help to identify community issues and needs, and provide a basis for planning community efforts that lead to long-term change. |