Understanding Research Methodologies and Ethical Considerations

Posted on May 1, 2025 in Psychology and Sociology

Describe Scientific Ways of Thinking in Research

  • Empiricism, testing theories, working in a community, tackling basic and applied problems, making their work public, and communicating with the world.
  • Make systematic observations, operational definitions, reliability and validity, appropriate research methods, and evaluation of theory.
  • Aim to be systematic and rigorous, allowing their work to be independently verifiable.

Describe the Theory-Data Cycle

  • Theory leads researchers to pose specific research questions, which lead to appropriate research designs to test hypotheses, which are preregistered before they collect data.

Distinguish Between Different Sources of Knowledge

  • Intuition: thinking about something logically or naturally.
    • Good story, accepting a conclusion because it seems believable.
    • Availability heuristic: things that are cognitively readily available tend to guide our thinking.
    • Present/present bias: ignoring what is not there.
    • Confirmation bias: actively seeking out what matches your thinking.
    • Bias blind spot: the belief that we are unlikely to fall victim to biases.
  • Authority: authoritative opinion, doctor, teacher, or even parent.
  • Use of reason: deductive reasoning.
  • Anecdotal experience: personal experience.

Explain the Advantages of Research Over Intuition and Experience

  • Research has comparison groups, can control for confounds, and is probabilistic.

Identify Main Variables and Distinguish a Variable from Its Levels

  • A variable is something that varies and must have at least two values.
    • Can be measured (levels are observed or recorded) or manipulated (controlled by the researcher).
    • Or constant, stays the same and helps to control for confounds.
  • Conceptual variable: the construct itself.
  • Conceptual definition: the theoretical definition.
  • Operational definition: when you turn a conceptual variable into a measured or manipulated variable.

Differentiating the Three Types of Claims

  • Frequency: describes a particular level or degree of a single variable, only one measured variable.
  • Association: argues that one level of a variable is likely to be associated with the level of another variable; at least two measured variables.
    • Soft verb indicating something is going on, but the specifics are unknown.
  • Causal: argues that one level of a variable causes change in another; at least one measured and one manipulated.
    • Strong verbs indicating causation and direction.

Ask Appropriate Questions to Interrogate Each of the Four Big Validities

  • Construct: how well the variables of the study are operationalized.
  • External: do they generalize to a larger population or a different context?
  • Statistical: how well do the numbers support the claim?
  • Internal: how well the design of the study supports the conclusion that one variable caused change in another.

Describe Examples of Research That Raised Ethical Questions

  • WWII Nazi studies led to the establishment of the Nuremberg Code (voluntary consent, protection from suffering, risks vs. benefits).
  • Declaration of Helsinki: research must be based in science and undergo independent reviews.
  • Tuskegee Study: withheld treatment, targeted group, lied to families, causing deaths.

Define and Apply the Three Ethical Principles of the Belmont Report

  • Respect for persons: informed consent, considering vulnerable populations. “Are participants fully informed and free to make their own choices about participation?”
    • Children and people with disabilities may be unable to give informed consent.
  • Beneficence: assess potential harms and take precautions. “Are we maximizing benefits while minimizing risks?”
    • Privacy comes into play.
  • Justice: who bears the burden? “Are benefits and burdens of research fairly distributed?”

Describe Procedures in Place to Protect Human Participants in Research

  • IRB approval, informed consent, confidentiality/anonymity, minimizing risk and ensuring beneficence, debriefing, right to withdraw.

Describe Three Ways to Operationalize Variables

  • Self-report measures: recording people’s answers to questions about themselves in a questionnaire or interview.
  • Observational measures: recording observable behaviors and quantifying them. Interviews can count here if done for depression or behavioral studies.
  • Physiological measures: recording biological data.

Classify Measurement Scales as Categorical or Quantitative

  • Categorical: levels are categories (nominal).
  • Quantitative: levels are coded with numbers.
    • Ordinal: rank order, no true zero, no equal intervals.
    • Interval: no true zero, equal intervals (e.g., IQ score).
    • Ratio: true zero, equal intervals (e.g., height in cm).

Differentiating Validity and Reliability of a Measure

  • Reliability = consistency.
  • Validity = accuracy.
    • A measure can be reliable but not valid.
    • Both are necessary for construct validity.

Identify Different Types of Reliability

  • Test-retest: correlation of scores between time point 1 and time point 2.
  • Interrater: the extent to which two raters agree with each other.
  • Internal: consistent pattern by similarly worded items, measured by Cronbach’s alpha.

Identify Different Types of Validity

  • Empirical:
    • Criterion: the measure correlates with a relevant outcome (predictor).
    • Convergent: correlates with another measure on a similar topic.
    • Discriminant: does not correlate with another measure on a different topic.
      • Both convergent and discriminant are necessary for construct validity.
  • Subjective:
    • Face: the extent to which a test appears to measure what it claims to measure at first glance.
    • Content: the measure captures what you want to capture from the conceptual definition, as determined by experts.

Describe Different Ways Questions Can Be Worded

  • Open-ended: you can answer any way you like; provides rich information but is difficult to code and takes time.
  • Forced choice: picking the best of two or more options; quick to code but less rich information.
  • Likert: 1-5 scale of agree or disagree.
  • Semantic differential: 1-5 scale, but there are contrasting adjectives at each pole.

Describe Challenges to the Construct Validity of a Survey

  • Leading question: encourages a specific response; to solve, word neutrally.
  • Double-barreled: asks two questions in one; break into two questions.
  • Negatively worded: contains too many negative words, confusing; to solve, word neutrally or ask questions in both positive and negative manners and look at Cronbach’s alpha.
  • Question order: previous questions affect subsequent answers; to solve, prepare different versions with different orders.
  • Response set/nondifferentiation: answering questions in the same way, happens in longer questionnaires, participant is not being measured on the construct.
    • Acquiescence: tendency to agree; include reverse-worded items or extreme questions.
    • Fence sitting: answering in the middle suggests they do not have an opinion; take away the neutral option.
    • Social desirability: making yourself look better than you really are; to solve, ensure anonymity.

Describe Challenges to the Construct Validity of Observations

  • Observer bias: expectations influencing interpretation; use codebooks, multiple observers, and blind studies.
  • Observer effect: changes in behavior caused by the observer’s expectations; to solve, use a masked design.
  • Reactivity: change in behavior when participants know they’re being watched; to solve, wait it out, use unobtrusive observations, or unobtrusive data.

Identify and Apply Random Sampling Techniques

  • Probability:
    • Simple random sampling: sample chosen at random.
    • Systematic sampling: using a randomly chosen N and counting every Nth person (both are difficult and time-consuming).
    • Cluster sampling: creating arbitrary groups, randomly selecting the group, and using everyone in the group.
    • Multistage sampling: random sample of cluster, then randomly selecting from the cluster.
    • Stratified sampling: identifying demographics then randomly selecting from within.
      • Oversampling: intentionally overrepresenting, adjusts final results to be weighted to proportion.
  • Nonprobability:
    • Purposive: only certain kinds of people are recruited.
    • Snowball: recommending friends.
    • Quota: identifies a subset of interest, sets a target, then selects (not randomly).
    • Convenience: recruiting those easy to contact.

Explain Why a Random Sample is More Likely to be a Representative Sample

  • Because every individual in the population has an equal chance of being selected.

Identify an Experiment’s Variables

  • Independent Variable (IV): manipulated, conditions.
    • Manipulated: experimenter can fully manipulate.
    • Subject: comes with the individuals, can be used to categorize but cannot be manipulated.
  • Dependent Variable (DV): measured, outcome.
  • Control: held constant to control for confounds.

Apply Three Criteria for Establishing Causal Claims

  • Temporal precedence: establishing that A comes before B.
  • Covariance: that the two variables correlate.
    • Method of agreement: if A then B (sufficient).
    • Method of disagreement: if no A then no B (necessary).
  • Internal validity: whether you can establish a causal conclusion, or if there is anything else involved.

Identify and Distinguish Between Experimental Designs

  • Between subjects: different groups of participants placed in different levels of IV.
    • Looks at differences between subjects, best for long-term or irreversible conditions, requires more participants and more variability between people. Prevents carryover; needs larger n and equivalent groups.
  • Within subjects: participants presented with all levels of IV.
    • Looks at within-group variability, compares a person to themselves across different conditions. Best for small differences, fewer participants; allows for equivalent groups, IVs are kept constant, and provides more precise estimations.

Differentiate Confounds and Extraneous Variables

  • Extraneous: any variable you’re not intentionally studying but could influence the DV; e.g., in a memory study, room temperature may be an extraneous variable; if held constant or randomized, it is not a problem.
  • Confounding: a type of extraneous variable that varies systematically with the IV; e.g., exercise and mood improvement, where the exercise group meets in the morning and the other group meets at night, making time of day a confound because it is linked to the IV and can also affect mood. This is always a threat.

Evaluate Potential Threats to Internal Validity in an Experiment

  • Between subjects only:
    • Selection effects: when one group of participants in an IV condition systematically varies from another.
      • To avoid: matched groups with a variable that may matter; random assignment.
  • Within subjects only:
    • Order effects: a condition changes how participants will react to a later condition; three types:
      • Practice (fatigue), carryover (contamination), testing (improvement).
      • Avoid with counterbalancing, two ways: reverse (reverse order, better for two conditions) and block randomization (randomize order of conditions in no particular way).
  • All studies:
    • Maturation threat: a change in behavior that emerges spontaneously; to solve, use comparison groups. E.g., student scores pretest/post-test with a teaching method, improving scores just because they have gone through a semester of school or grown older.
    • History threats: result from historical or external factors affecting most members; selection history threat only affects one group; comparison group.
    • Regression to the mean: when the average is unusually high or low, the next time it will be less extreme; comparison group.
    • Attrition threat: in pretest/post-test, an issue if it systematically varies; selection attrition, when one group systematically drops.
    • Testing threats: improving over time; prevent by doing post-test only or using a comparison group (repeated testing).
    • Instrumentation threat: measuring instruments change over time; prevent with post-test only, codebook, counterbalancing.
    • Observer bias: researcher expectations influence interpretations of results; use blind studies.
    • Demand characteristics: participants guess what the hypothesis is and adjust their behavior accordingly; use double-blind methods.
    • Placebo effect: improving only because they believe they are; to prevent, use a double-blind placebo control study where nobody knows who is in what group, with one receiving the real treatment and the other receiving a placebo.
    • Null effects: IV has no effect or it wasn’t detected, not enough between-group differences or too much within-group variability (noise).
      • Not enough between: weak manipulation of IV or insensitive measure on DV (use a measure with more than just two or three levels).
      • Ceiling/floor effects: result of problematic IV or poorly designed DV.
    • Too much within: individual differences, measurement error, situation noise.

Differentiating Correlational and Experimental Methods

  • Correlational methods do not experimentally manipulate variables; instead, they measure variables, describe associations, and make predictions.
  • Medium amount of control.
  • Used when variables cannot be manipulated or participants cannot be assigned to conditions for ethical or practical reasons.
  • Research questions are broad and exploratory, or focus on what it is like to have a particular experience.

Differentiating Correlations of Different Directions and Strengths

  • Direction is indicated by the + or – sign.
  • Strength is how close the correlation coefficient is to 1.

Recognizing Simple Regression

  • Y (DV) = A (y-intercept) + b (slope) x (IV); the goal is to use values of x to predict y.
  • If b = 0, there is no slope, just a horizontal line; if b does not equal zero, then you most likely have a statistically significant relationship.

Describing Caveats in Interpreting Correlations

  • How strong is the relationship? What is your correlation coefficient? (.05 = weak; .1 = weak; .2 = moderate; .3 = fairly powerful; .4 = unusually large).
  • Is it really linear? Is it curvilinear (starts positive then goes negative) or spurious (original association is not present in subgroups)?
  • Limited range or outliers?
    • Restriction of range: if the variables do not span their full possible range, correlation can appear stronger or weaker than it really is.
    • Outliers: especially detrimental in small populations; extreme scores can inflate a pattern.
  • Correlation does not imply causation: consider third variables and directionality.

Identifying and Explaining Directionality and Third Variable Problems

  • Directionality: the problem of not being able to determine the direction of causality.
    • To solve: longitudinal studies with cross-panel design and cross-lagged correlations will provide this answer.
  • Third variable: not being able to control for extraneous variables in correlational research.
    • Partial correlation: is x still related to y if we remove the influence of z? This removes the part that overlaps and sees if they are still correlated. It provides a correlation coefficient, usually with two main variables and one or two controls.
    • Multiple regression: predict the value of an outcome based on values of two or more predictors.
      • b coefficients tell you how much each predictor uniquely affected y (DV).
      • b coefficients show the strength of each predictor in the same units.
      • Outcome = criterion = DV; predictor = IV.

Differentiating Mediators and Third Variables

  • Mediators are the mechanism; how or why; they transmit the effect of IV to DV.
  • Third variables are an outside reason for the original relationship.

Differentiating Large-N and Small-N Designs

  • Large N: grouped participants, patterns across participants; data shown as group averages; enables group averages to be estimated more precisely; research typically involves 10 or more participants.
  • Small N: each participant is treated separately, pattern within a participant; data for each individual is presented; compares each individual during treatment and control periods; research and therapeutic settings.
    • Small N can be between or within; it just depends on whether the participant experiences one or all conditions.

Identify Small-N Designs in Applied Settings

  • Stable baseline: researcher observes behavior of interest over a period of time to ensure it remains stable before introducing an intervention. This allows us to rule out maturation or regression.
  • Multiple baseline: applies intervention at different points in time across multiple baselines (people, behaviors, or settings); the goal is to see if changes in behavior happen only when the intervention is introduced. This allows us to rule out history threats.
  • Reversal: introducing then withdrawing the intervention to assess if changes in behavior are indeed due to the intervention.
    • Appropriate for non-long-lasting situations.

Explain Why Researchers Conduct Replication Studies

Researchers conduct replication studies to confirm or extend effects, assess generalizability, identify limitations, and evaluate the impact of context changes.

Describe Different Ways Research Conducts Replication Studies

  • Direct: reproduce the original study as closely as possible; helps determine if original results were a fluke.
  • Conceptual: same research questions but different methods or measures; the goal is to see whether the original concept or relationship holds under different conditions.
  • Replication-plus-extension: original study plus new elements; done to expand on findings to explore additional questions.
  • Multi-lab/site replication: tests generalizability and helps ensure that results are not specific to one lab or sample.