Introduction to Research Design: Types, Validity, and Methods

Posted on May 15, 2024 in Psychology and Sociology

ITEM 3: INTRODUCTION TO RESEARCH DESIGN

I. INTRODUCTION TO RESEARCH DESIGN

1. DESIGN CONCEPT

Design refers to the form of a test in concrete scientific research. It involves the provision and connection of elements, serving as a plan for obtaining and processing data necessary for verification.

Research design should not be confused with the project, although the terms are sometimes used interchangeably.

2. IMPORTANCE AND DESIGN REQUIREMENTS

The design refers to:

The essence of social research, testing, and application of the scientific method.
The essence of scientific work and validity.

Requirements:

In-depth knowledge of research, scientific method, and factors (validity).
Realism to adapt to changing circumstances of each case.
Discernment of factors affecting validity to find effective solutions.
Flexibility (adaptability to new design requirements).

End of design: Achieving maximum possible validity, i.e., more accurate correspondence of study results with reality.

3. VALIDITY AND RATES

Aspects to consider:

Take steps to ensure research results are valid.
Check, after research, the degree of accuracy with which results represent the reality they refer to.

3.1. Types of validity

Internal: Consistency of results obtained with the reality under investigation.
External: Concordance with the reality of other non-investigated similar realities.
Construct: The degree to which the observed extension in the empirical reality of variables can be considered representative of the extension of the construct of reality.
Statistical: Arises when samples are used to represent the population.

3.2. Concepts linked to validity

3.2.1. Error

Definition: Wrong knowledge that does not fit properly with the reality to which it refers. Different from ignorance (lack of knowledge of something).

Feature: Difficulty of measurement, as this implies exact knowledge of the reality in question or its actual value, which often does not exist.

Sierra Bravo points out different types of errors:

Reasoning: Lead to mistaken knowledge arising from improper deduction of logical consequences.
Perception: Caused by deception of the senses or misuse of observation and measurement instruments.
Assessment: Derived from a subjective intellectual vision of reality that does not consider relevant objectives.
Planning: Due to a misconception of research operations.
Based: Either on personal characteristics of observers or alterations in the normal response of subjects or phenomena investigated.
Action: Due to a poor understanding of the circumstances of the action.

Classification of these errors according to their character:

Systematic: Consistent with a given system in direction and magnitude.
Random: Variable in direction and magnitude.
Sampling: Random error of a statistical, probable, and generic nature, occurring when samples are taken and based on random sampling fluctuations.

3.2.2. Extraneous Variables

Also called external variables, these are strongly related to the types of errors noted in the previous section.

These variables are factors that can influence research results, and their effects mix with them. Paul C. Stern states that external variables allow alternative explanations for a set of observations: either the observed relationships are due to the variables of the scenarios, or they are derived, at least in part, from external variables.

3.2.3. Control

Definition: Attempt to eliminate the effect of external variables to ensure that the effects found are due only to the independent variable being tested.

Ways to control:

Disposal: Eliminating extraneous variables from the investigation. Feasible only in laboratory experiments.
Maintain constant: Extraneous variables remain constant throughout the process.
Randomization: Making something random. Performed whenever the allocation of subjects to the experimental group is done by drawing lots. Based on the laws of chance.
Equalization: Ensuring extraneous variables are equally distributed in the groups formed.
Counterbalancing: Used in investigations applying different treatments. Involves changing the order of treatments applied so that the order is different in different groups and subjects.
Repetition: Can be done through various treatments to the same individuals or groups or using the same treatment.
Control group: Involves investigating a control group with similar characteristics to the experimental group, but not subject to the independent variable.

Control takes place in experimental investigations. Non-experimental investigations, by definition, exclude such manipulation and, therefore, direct control of extraneous variables. This does not mean they cannot be controlled.

II. TYPES OF DESIGNS

(See schedule on page 155)

The following elements are considered when performing different research designs:

Objects, subjects, or groups investigated and the shape of their choice.
The number of observations, the nature and order of execution of such operations.
The form of assignment of subjects or groups to treatments.
The nature of the investigations.
The nature and number of independent variables investigated and the number of levels or categories.

1. EXPERIMENTAL DESIGNS IN METHODOLOGY

Each method has variations that allow adaptation to the demands of specific research problems.

According to Latorre (1996), strict experimental studies are characterized by:

The researcher manipulates the independent variable and assigns levels.
Two or more levels of the dependent variable are applied.
The sample is selected randomly. This group will be consistent and comparable.

1.1. Completely randomized design

Here, groups are formed by randomly assigning each subject to one group of the experiment and then randomly assigning each level of treatment to different groups. The designs used are:

Bivalent designs: Used when applying two levels of the independent variable. They consist of an experimental group and a control group.
Multivariable multivalent designs: Used when applying three or more levels of the independent variable. Both bivalent and multivalent designs (pretest-posttest), by demonstrating the initial equivalence (homogeneity) of the groups, simply compare the mean posttest to test the relative effect of the independent variable. Whenever possible, it is better to use multivalent designs.
Factorial designs: The researcher is only interested in the activities of an independent variable (or simple unifactorial designs). The aim is to keep everything constant except the independent variable to assign observed changes in the dependent variable to that variable. This has two difficulties:
- It is nearly impossible to keep all variables constant except one.
- In educational reality, it is not appropriate to speak of simple accusation.

In multifactorial or factorial design, the researcher manipulates two or more independent variables, each with two or more levels of treatment, to study the changes in the dependent variable for such manipulation.

1.2. Intragroup or repeated measures designs

The efficiency and sensitivity of an experiment increase with homogeneity.

If each patient was applied at all levels of the independent variable and compared with each other, showing the effects, we would have taken the conditions of uniformity or individual differences to the extreme. The order of application can influence the results of the dependent variable. To avoid this, it is feasible to randomize the order of application.

1.3. Types of experiments

To achieve the greatest degree of control in an experiment, there is a tendency towards isolation from the natural context where the phenomenon occurs. To the extent that we control extraneous variables, we guarantee internal validity, but the conditions under which the experiment develops become more artificial. We differentiate:

Laboratory experiment: Research study where isolation is achieved from the natural context in which the phenomenon occurs to eliminate the many extraneous influences that may affect the dependent variable.
Field experiment: Research study in a real situation where one or more independent variables are manipulated by the experimenter under controlled conditions to the maximum extent possible under the situation.

1.4. Possibilities and limits

Latorre (1996) presents the following possibilities and limits:

a) Potential: The experimental method is the only means to strictly observe causal relationships.

Basic Features:

Discover relationships in pure and uncontaminated conditions.
Test predictions derived from theory or another category.
Develop theories and hypotheses to develop theoretical systems.

These functions are possible due to the high degree of control exercised over independent variables, extraneous variables, and measurement.

The advantages and possibilities of this methodology can be summarized in precision and economy.

+ Laboratory experiments are suitable to meet investigation objectives such as:

Note relationships in pure and uncontaminated conditions.
Test predictions derived from theory (hypothesis).
Generate new research.
Refine theories and hypotheses. Help construct theoretical systems.

Advantages:

Relatively complete control.
Allows random assignment and manipulation of one or more independent variables.
High degree of specificity in the operational definitions of variables.
Precision. Well-defined and unambiguous. Small error variance.
High internal validity.
Inflexibility, there are many experimental possibilities.
Replicability, possibility to repeat the experiment by introducing slight variations.

+ Field experiments. Advantages:

Widely used in education. They are held in classrooms, schools, etc.
Allow manipulation of the independent variable and subjects and treatments assigned randomly.
The more realistic the situation, the more powerful the variables are.
Suited for studying complex social influences (processes and changes).
Can be used to test theories and solve practical problems.
Flexibility and applicability to a variety of problems.

b) Limits:

Limitations of laboratory experiments:

Weak independent variable
Artificiality of the experiment
Low external validity, little generalization.

Limitations of field experiments:

Lack of rigorous control
Objections to the manipulation of variables
Lack of precision
To some extent, require an expert in social work

2. QUASI-EXPERIMENTAL METHODOLOGY

In these investigations, the researcher deliberately varies the values of the independent variable to see the effects that such variation causes in the dependent variable. Many extraneous variables are uncontrolled. The methodology takes place in a real situation or field.

2.1. Types of designs

Nonequivalent group designs
Interrupted time series designs
Single-subject designs (N = 1 or within-subject)

2.1.1. Nonequivalent group design

This methodological option is pursued when the researcher intends to analyze causal relationships and can manipulate the independent variable but is forced to use groups already formed naturally, such as a college class.

2.1.2. Interrupted time series designs

Indicated when the problem to be investigated requires taking a series of measurements of the dependent variable over a certain period, breaking the series with the implementation of any treatment. Types:

Simple: Involving a single group of subjects.
Two groups are not equivalent: Involving a different group, usually a control group.
With withdrawal of treatment: Compares three periods. It is more reliable.
Multiple replications: Occurs when the previous situation is repeated.

2.1.3. Single-subject designs (N = 1 or within-subject)

They can be useful when:

Ideographic studies are required (intensive studies of individuals).
It is reasonable to assume that the studied process is general.
There is only an opportunity to observe and study a single subject.
If we want to study the problem in depth, it may help define variables, identify issues, and suggest ways of tackling it.

2.2. Possibilities and limits

a) Possibilities: Offers many advantages for its proximity to educational reality, which often does not allow for an experimental category because it is not feasible to alter the structure or configuration of personal needs. This gives a real dimension to the variables that can exert a more powerful influence, so it is suitable for studying complex social influences, educational processes, and changes in real situations. It allows for testing theories and solving practical problems.

b) Limits: Many cause-effect relationships have taken place prior to the researcher’s performance and thus fall outside its scope. We cannot know for sure if the manipulated levels of the independent variable are solely responsible for the changes observed by measuring the dependent variable in the different groups.

3. NON-EXPERIMENTAL DESIGN (EX POST FACTO)

Kerlinger: Empirical systematic search in which the scientist has no direct control over the independent variable because its manifestations have already happened or cannot be manipulated.

Latorre presents the following comparative scheme:

Experimental Methodology | Non-experimental Methodology
— | —
We induce (manipulate) the independent variable and observe changes in the dependent variable. Future orientation. Groups scrambling effects. | Effects have already occurred. The independent variable is unchanged, just selected and observed. Orientation toward the past. Naturally formed groups.

Causal comparative method:

Used when the researcher attempts to explain causal relationships by comparing groups of data, but the variable that the researcher studies cannot be manipulated and only supports one level of selection. The strength lies in finding the influence of the independent variable on the dependent variable in the context, trying to determine the relationship between them.

Descriptive methods

Explore relationships. They try to relate and compare data sets. Exploratory end.

Development Studies

Describes the evolution of variables over a period. They focus on differences related to age. Modalities:

Longitudinal: Analyzes the characteristics of individuals at different time or age levels through repeated observations.
Transversal: At one time, different developmental periods are studied. We compare different age groups observed at one time.
Incidence: Combines features of the above. Can be made from data collected in other prior investigations.

3.2.2. Survey Studies

The survey consists of direct questions to a representative sample of subjects from a previously developed protocol or script. It is very useful when the research requires descriptive data that subjects can provide from personal experience. The investigation method is based on a series of questions to subjects who may constitute a representative sample of a population to describe and/or relate personal characteristics and certain areas of information needed to answer the category problem.

Presents two features:

Data collection is based on asking questions to people who have the information and are able to communicate through personal interviews, mail, or telephone.
The survey method seeks to estimate population conclusions from the results of a sample.

3.2.3. Observational studies

Appropriate in educational settings that have features like:

The subjects are unable to give verbal information.
The subjects do not present an explicit desire to inform.
In some situations, it is likely that retrospective accounts of individuals may suffer temporal distortion.

They are one of the basic methods for the discovery of hypotheses, identifying relevant phenomena, suggesting variables that cause action, etc.

Interrelational Studies

For some authors, the correlation method has its own identity and is included as a form of the descriptive category or experimental method. However, many authors consider it sufficient to set its own entity as a specific category. As with descriptive research, the correlational method is integrated into the so-called model-based science education. However, you can also test hypotheses and seek explanations by studying relationships between variables.

Predictive studies

Prediction involves estimating possible values of a dependent variable or criterion variable from which another independent variable or predictor is taken. The predictive method can be based on:

Descriptive studies: Highlighting Pedro Rosello’s”patter” method, Tusquets’s projectionist method, Escalona’s behavior analysis, etc.
Regression techniques: Involve approximating or”returnin” points on a scatter plot to a straight line to predict values from the equation of that line or regression equation.
Techniques for prediction: A guess is made about the value that a variable will take based on its relationship with another variable.
Multiple regression: Used if it provides more than one independent variable.
Canonical correlation: Used to predict when there is more than one dependent variable.
Discriminant analysis: Offers the prediction of group membership.

Possibilities and limits

The ex-post-facto methodology provides expertise to the development of education as a scientific discipline and is essential in many educational settings. However, by itself, it is not sufficient to develop scientific knowledge. The ideas of this method must be complemented by the experimental method.

Descriptive and correlational methods are more associated with the theoretical method of induction, and empirical generalizations can be reached through the establishment of regularities and relationships between the observed data, while the experimental method operates with the deductive method, trying to test hypotheses derived deductively from a theory.

4. DESIGNS IN ETHNOGRAPHIC RESEARCH

To understand the lifestyle of primitive cultures, anthropologists had to”go and liv” with the natives for a period. The work is done descriptively, using a systematic technique, combining participant observation with interviews.

Concept of ethnography

It is a form of category embedded in the family of qualitative methodology, constructivist for others, which stands as an alternative model to the category traditionally used by social scientists to study social reality.

For Malinowski, the goal of ethnography is to learn the native’s point of view in relation to life, to realize their vision of the world.

Educational ethnography: Ethnography applied to the study of social reality in education.

For Latorre, some features of ethnography are:

Its holistic character: Describes global phenomena in their natural context.
Its natural condition: Studying people in their natural habitat.
Use via inductive: Relies on evidence, empathy, and skill.
Phenomenological or emic character
Data are contextualized
Free of value judgments
Your thoughtfulness: The researcher is part of the world they study and affects it.

Knapp emphasizes the following aspects of the ethnographic category:

Access to the scene initially exploratory, open to the contingencies of the problem.
Intense involvement of the researcher who studies the social environment.
Keep a careful record of what happens and record all kinds of evidence.
An explicit attempt to understand events in terms of meaning.
Marked interpretive underlining the important role of context.
Processing of the results and the category descriptively.

The ethnographic research process

The ethnographer must ensure access and permission to enter the scene, and once there:

Establish the reason for being there.
Develop a paper to let people know who the researcher is, what they are doing there, etc.

Hitchcock and Hughes, stages in the completion of fieldwork in seven steps:

Identify a point of study
Find a place and manage to enter the stage
Choose key informants
Develop field relations
Gathering data in the field
Collecting data off the field
Analyze data

Techniques for collecting information

Participant observation: Observing patterns of conduct and participating in the culture.
Informal interview: The participant keeps talking about things of interest.
Written materials (documents): Official documents, personal documents, and questionnaires.

Difficulties in ethnography plans

Access to the stage: Accompanied by a first stage of contact, called vagrancy (recognizing the phenomenon and becoming familiar with the participants). Shapes:
- Through reciprocal relations, contributions, grants, etc.
- Introduced if participants are not open to observation and do not cooperate.
- Go to a recognized endorser or authority.
Informants or actors: Possess knowledge, status, or communication skills of great interest to the researcher.
The role of the ethnographer: The core instrument of the category. They are data collectors, observers, narrators, and writers. The ability to communicate with other cultural groups and to adapt to their natural habitat is called border crossing.

Some methodological guidelines

Be descriptive when taking field notes.
Collect a wide variety of information from different perspectives.
Triangulate and perform validations and data collecting different types.
Use literal quotes and stories in the language used by participants.
Carefully select key informants.
Be aware of the different stages of fieldwork.
Be involved as much as possible to assess the educational reality.
Clearly differentiate descriptions, interpretations, and value judgments.
Provide formative feedback and control. Observe its impact.
In the notes and the report, include experiences, thoughts, and impressions of your own.

5. DESIGNS IN THE CASE STUDY

This study is a detailed description and analysis of social units or unique educational entities. Included within the idiographic approach, which aims at a profound understanding of reality. Useful for a short period.

Concept of case study

Properties: (Merriam)

Particularist: Focuses on a situation, event, program, or particular phenomenon.
Descriptive: Attempts to produce a rich description of the phenomenon.
Heuristic: The study illuminates the reader’s understanding.
Inductive: Comes to generalizations, concepts, or hypotheses by inductive processes.

Objectives: (Ary, Jacobs, and Razavieh)

Describe and analyze unique situations (e.g., gifted child).
Generate hypotheses and then compare them to other more rigorous studies.
Acquire knowledge.
Diagnose a situation to guide and implement advice.
Complete information provided by strictly quantitative investigations.

Case Study Design

Conditions: (Kenny and Grotenlensh)

When the desired objectives focus on humanistic or different cultural results.
When the information obtained is not subject to truth or falsity, but credibility.
The uniqueness of the situation leads us deeper into the case.
Develop a better understanding of the dynamics and a program.
If the problem involves a new line of category that needs further conceptualization and factors or functions.

Phases:

Exploration and appreciation: We analyze the contexts and subjects that can be a source of information and potential significance to the aims and objectives of the category.
Selecting the subjects or units to explore, people to interview, what strategies will be used, the duration of the study, etc.
Collection, analysis, and interpretation of the information, ending with the report and decision making.

Types of studies

Merriam, depending on the nature of the report, groups them under:

Descriptive: Detailed report of the case. They are descriptive, not guided by previous hypotheses. Provide basic information. Studies programs and innovative practices.
Interpretive: Information about a case to interpret or theorize about the case.
Evaluative: Involves description, explanation, and opinion. Study of school programs.

Advantages and difficulties of the case study

Advantages: (Latorre)
-can be a way to deepen a process inv. from data analysis.
invs-appropriate. On a small scale, within a limited time, space and resources.
-is an open method to resume other personal or different institutions.
-is very useful for the teachers involved in the cat.
-leads to decision making, to get involved, make decisions, …

Advantages: (Stake)
-is more specific. Linked with our experience.
-this contextualized. For the reader’s interpretation.
-is based on reference populations close to the reader. Involvement easier.
Challenges:
-as to make generalizations from a singular reality.

Challenges: (Kratochwill)
-lack of attention to validity, both internally and externally.
-its limited design options.
-the difficulty of generalizing the findings.