So far, you have learnt how to ask a RQ. In this chapter, you will study the details of how to design a method for collecting the data needed to answer the RQ. You will learn to:
From the RQ, we know what data must be collected from the individuals in the study (the response and explanatory variables)... but how do we design a study to obtain this data? After all, data are important: they are the means by which the RQ is answered. Three broad methods for obtaining data are to use:
The type of study depends on the type of RQ.
Example 3.1 (Research design) Suppose we wish to compare the effects of echinacea on the symptoms of the common cold (based on Barrett et al.89). How could we design such a study to collect the necessary data? What decisions would you need to make?
Descriptive studies are used to answer descriptive RQs (Fig. 3.1).
Definition 3.1 (Descriptive study) In a descriptive study, researchers only focus on collecting, measuring, assessing or describing an outcome in the population.
FIGURE 3.1: A descriptive study, used to answer a descriptive RQ
Example 3.2 (Descriptive study) Consider this RQ:
The outcome is the average increase in heart rate. The response variable is the increase in heart rate for the individual men. The increase in heart rate would need to be found by measuring each man's heart rate before the walk, then their heart rate after the walk, and finding the difference between them. The increase in heart rate would be computed as the after heart rate minus the before heart rate. Some of these differences might be positive numbers (heart rate went up), and some may be negative numbers (heart rate went down). No comparison being made: every man in the study is treated in the same way. This is a descriptive RQ, which can be answered by a descriptive study.
Observational studies (Fig. 3.2) are used to answer relational RQs. They are a commonly-used study design, and sometimes are the only possible study design that can be used.
Definition 3.2 (Observational study) In an observational study, researchers do not impose, and cannot manipulate, the comparison or connection upon those in the study to (potentially) change the response of the participants.
FIGURE 3.2: An observational study, used to answer a relational RQ
Definition 3.3 (Condition) Conditions: The conditions of interest that those in the observational study are exposed to.
Example 3.3 (Observational study) Consider again this RQ:90
This would be a relational RQ if the researchers do not impose the echinacea (that is, the individuals make this decision themselves). For this RQ, the conditions would be taking echinacea, or not taking echinacea (Fig. 3.3).
FIGURE 3.3: Observational studies Broadly speaking, three types of observational studies exist (Table 3.1):
These differ in when the response and explanatory variables are observed. Many specific types of observational studies exist (case-control studies; cohort studies; etc.), but we will not delve into these.
In retrospective studies, the response variable is observed now, and the researchers look back to see the value of the explanatory variable in the past (e.g., case-control studies).
Example 3.4 (Retrospective studies) An Australian study91 examined patients with and without sporadic motor neurone disease (SMND), and asked about past exposure to metals. The response variable (whether or not the respondent had SMND) is assessed now, and whether or not they had exposure to metals (explanatory variable) is assessed from the past. This is a retrospective observational study.
In prospective studies, the explanatory variable is determined now, and researchers look ahead to assess or measure the response variable (e.g., prospective cohort studies).
Example 3.5 (Prospective studies) A study92 measured the softdrink consumption of men, and determined who experienced gout over the following 12 years. The response (whether or not the individuals experience gout) is determined in the future. The explanatory variable (the amount of softdrink consumed) is measured now. This is a prospective observational study.
In cross-sectional studies, both the response and explanatory variables are gathered now.
Example 3.6 (Cross-sectional studies) A study93 asked older Australian their opinions of their own food security, and recorded their living arrangements. Individuals' responses to both both the response variable and explanatory variable are gathered now. This is a cross-sectional observational study.
In South Australia in 1988--1989, 25 cases of legionella infections (an unusually high number) were investigated.94 All 25 cases were gardeners, with hanging baskets of ferns. Researchers compared 25 cases with legionella infections with 75 non-cases, matching on the basis of age (within 5 years), sex, post codes. The use of potting mix in the previous four weeks was associated with an increase in the risk of contracting illness of about 4.7 times. What type of observational study is this?
Retrospective: people were identified with an infection, and then the researchers looked back at past activities.
Experimental studies (Fig. 3.4), or experiments, are commonly-used study designs. Well-designed experimental studies can establish a cause-and-effect relationship between the response and explantory variables. However, using experimental studies is not always possible. Experimental studies have an intervention, and so experimental studies are used to answer interventional RQs. The researchers impose and can manipulate the values of the explanatory variable: they create changes in the explanatory variable, and record the changes in the response variable.
Definition 3.4 (Experiment) An experimental study (or an experiment), has an intervention: the researchers impose and can manipulate the values of the explanatory variable. The researchers allocate treatments (i.e., apply the intervention).
FIGURE 3.4: An experimental study, used to answer interventional RQs
Definition 3.5 (Treatments) Treatments are the conditions of interest that those in the study can be exposed to (as the explanatory variable). In experiments, treatments are imposed by researchers. Two types of experimental studies (Table 3.2) are:
True experiments are commonly used, but conducting a true experiment is not always possible. An example of a true experiment is a randomised controlled trial, often used in drug trials.
Definition 3.6 (True experiment) In a true experiment, the researchers:
While these may not actually happen explicitly, they can happen conceptually.
Example 3.7 (True experiment) The echinacea study95 (Sect. 2.7) could be designed as a true experiment. The researchers would allocate individuals to one of two groups, and then decide which group took echinacea and which group did not (Fig. 3.5).
FIGURE 3.5: True experimental studies
A researcher wants to examine the effect of an alcohol awareness program96 on the amount of alcohol consumed in O-Week. She runs the program at UQ only, then compared the average amount of drinking per person at two universities (A and B). What type of study is this: observational or true experimental? Answer these questions to help:
It is neither. The researcher did not determine the groups: the students (not the researcher) would have chosen University A or University B for many reasons. The researcher did decide how to allocate the program to University A or University B.
Quasi-experiments are similar to true experiments, but treatments are allocated to groups that already exist.
Definition 3.7 (Quasi-experiment) In a quasi-experiment, the researchers:
Example 3.8 (Quasi-experiments) The echinacea study (based on Barrett et al.97) (Sect. 2.7) could be designed as a quasi-experiment. The researchers would need to find (not create) two existing groups of people (say, from two different suburbs) then decide which group took echinacea and which group did not (Fig. 3.6).
FIGURE 3.6: Quasi-experimental studies
In experimental studies, researchers create differences in the explanatory variable through allocation, and note the effect this has on the response variable. In observational studies, researchers observe differences in the explanatory variable, and observe the values in the response variable. Different RQs require different study designs (Table 3.3).
Importantly, only well-designed true experiments can show cause-and-effect. In general, well-designed true experiments provide stronger evidence than quasi-experiments, which produce stronger evidence than observational studies.
Example 3.9 (Cause and effect) Many studies have reported that the bacteria living in the gut of people on the autism spectrum is different than the bacteria in the gut of people not on the autism spectrum (Dae-Wook Kang et al.98, Lucius Kang Hua Ho et al.99). However, these studies have been observational, so the suggestion of a cause-and-effect relationship may be inaccurate. Other studies100 propose that the relationship works the other way: people on the autism spectrum are more likely to be "picky eaters", which contributes to the differences in their gut bacteria. Although only experimental studies can show cause-and-effect, experimental studies are often not possible for ethical, financial, practical or logistical reasons. The animation below compares observational, quasi-experimental and true experimental designs. In addition, experimental studies may suffer from lack of ecological validity and the influence of the Hawthorne effect, which some observational studies may manage better. Well-designed quasi-experiments and observational studies can still produce strong conclusions, but cannot be used by themselves to establish cause-and-effect conclusions.
FIGURE 3.7: The three main study designs
As far as possible, all studies should be designed to be externally valid (Chap. 5) and internally valid (Chaps. 7 and 8) Internally validity refers to how reasonable and logical it is to conclude that changes in the value of the response variable can be attributed to changes in the value of the explanatory variable; that is, it refers to the strength of the inferences made from the study. Studies with high internal validity show that changes in the response variable can confidently be related to changes in the explanatory variable in the group that was studied; the possibility of other explanations has been minimised. In contrast, studies with low internal validity leave open other possibilities, apart from the explanatory variable, to explain changes in the value of the response variable.
Definition 3.8 (Internal validity) Internally validity refers to how reasonable and logical it is to conclude that changes in the value of the response variable can be attributed to changes in the values of the explanatory variable; that is, the strength of the inferences made from the study. A study with high internal validity shows that the changes in the response variable can be attributed to changes in the explanatory variables; other explanations have been ruled out.
Example 3.10 (Low internal validity) A study of programs that used double-fortified salt programs to manage iodine and iron deficiencies examined numerous existing studies. The authors found that
One of many threats to internal validity might be that the groups being compared are different to begin with (for example, if the group receiving echinacea is younger (on average) than the group receiving no medication). This is a form of confounding. To check this, the baseline characteristics of the individuals in the groups can be compared: the groups being compared should be as similar as possible, so that any differences in the outcome cannot be attributed to pre-existing difference in the two groups being compared.
Example 3.11 (Baseline characteristics) In a study of treating depression in adults,102 three treatments were compared: exercise, basic body awareness therapy, or advice. If any differences between the treatments were found, the researchers need to be confident that the differences were due to the treatment. For this reason, the three groups were compared to ensure the groups were similar in terms of average ages, percentage of women, taking of anti-depressants, and many other aspects. An internally valid study requires studies to be carefully designed; this is discussed at length later (Chaps. 7 and 8). In general, well-designed experimental studies are more likely to be internally valid than observational studies (Fig. 3.8).
FIGURE 3.8: Well-designed true experiments are more likely to have high internal validity A study is externally valid if the results of the study are likely to generalised to other groups in the population, apart from those studied in the sample. For a study to be externally valid it first needs to be internally valid, since the results must at least be internally valid for the group under study before being extended to other members of the population. Using a random sample helps ensure external validity. In addition, the use of inclusion and exclusion criteria (Sect. 2.3.1) helps clarify to whom or what the results may apply outside of the sample being studied.
Definition 3.9 (External validity) Externally validity refers to the ability to generalise the results to other groups in the population, apart from the sample studied. For a study to be truly externally valid, the sample must be a random sample from the population. A study is externally valid if the results from the sample studied are likely to apply to the intended population. It does not mean that the results apply more widely than the intended population.
Example 3.12 (External validity) Suppose the population in a study is Queensland university students. The sample would be the students studied. The study is externally valid if the sample is a random sample from the population of Queensland university students. The results will not necessarily apply to all Queensland residents, or university students outside of Queensland. However, this has nothing to do with externally validity. External validity concerns how the sample represents the intended population in the RQ, which is Queensland university students. The study is not concerned with all Queensland residents, or with non-Queensland university students.
Choosing the type of study is only a small part of research design. Planning the data collection process, and actually collecting the data, is still required. Data may be obtained by:
Either way, knowing how the data are obtained is important. The design phase is concerned with planning the best way to obtaining the data to ensure the study is internally and externally valid, as far as possible. Internal validity considerations include:
External validity considerations include:
Ethical issues must also be considered (Chap. 4), and the limitations of the study understood when the results are interpreted (Chap. 9). The following short (humourous) video demonstrates the importance of understanding the design!
Studies may be observational or experimental. Observational studies can usually be classified as retrospective, prospective, or cross-sectional. Experimental studies can usually be classified as true experiments or quasi-experiments. Cause-and-effect conclusions can only be made from well-designed true experiments. Ideally studies should be designed to be internally and externally valid. The following short videos may help explain some of these concepts:
Which of the following are true?
Selected answers are available in Sect. D.3.
Exercise 3.1 In a study on the shear strength of recycled concrete beams,104 beams were divided into three groups. Different loads were then applied to each group, and the shear strength needed to fracture the beams was measured. Is this a quasi-experiment or a true experiment? Answering these questions may help:
Exercise 3.2 A study had this aim:
Patients were provided with either alternating pressure air overlays (in 2001) or alternating pressure air mattresses (in 2006). The number of pressure ulcers were recorded. This study experimental, because the researchers provided the mattresses. Is this a true experiment or quasi-experiment? Explain.
Exercise 3.3 Consider this initial RQ (based on Erika Friedmann and Sue Thomas106), that clearly requires a lot of refining:
To answer this RQ:
Exercise 3.4 Consider this journal extract:
"Scientific Research and Methodology: An introduction to quantitative research and statistics in science, engineering and health" was written by Peter K. Dunn. It was last built on Last updated: 2022-08-03 14:22:03.
This book was built by the bookdown R package. |
What is the 3 types of research design?
Postagens relacionadas
direito autoral © 2024 mesadeestudo Inc.
