Study guide: Observational and Self-Report Methods

Course Outline

  1. Experimental Methods, Variables and Bias
  2. Experimental Designs
  3. Types of Experiment and Population
  4. Ethical Issues in Research
  5. Study Procedures and Controls
  6. Naturalistic Observations
  7. Observational Design and Sampling
  8. Reliability and Study Design
  9. Questionnaires, Open and Closed Questions
  10. Self-Report Design and Questions
  11. Correlations and Their Limits
  12. Data Types and Sources
  13. Descriptive Statistics and Graphs
  14. Mathematical Skills and Statistical Testing
  15. Peer Review in Research

1. Experimental Methods, Variables and Bias

Key Concepts & Definitions

  • Experimental method: Manipulation of an independent variable (IV) to measure its effect on a dependent variable (DV).
  • Hypothesis: A clear, precise, testable statement predicting the relationship between variables and the expected study outcome.
  • Independent variable: The independent variable (IV) is the factor changed or manipulated, naturally or by the researcher, so its effect on the dependent variable can be measured.
  • Extraneous variables: Extraneous variables (EVs) are variables other than the IV that may affect the DV if uncontrolled; examples include lighting, participants’ age, and participant reactivity.
  • Confounding variables: A variable that changes systematically with the IV and may affect the DV, making the true source of changes in the DV harder to establish.
  • Demand characteristics: Demand characteristics are cues from the researcher or setting that participants may interpret as revealing the study’s purpose, leading them to change their behaviour.
  • Investigator effects: Investigator effects are unwanted influences of the investigator on research outcomes, including unconscious cues, leading questions, or control over participant selection, materials, and instructions.
  • Aim: A general statement of what the researcher intends to investigate and the purpose of the investigation.
  • Dependent variable: The dependent variable (DV) is the measured outcome that should be affected by a change in the independent variable.
  • Operationalisation: The clear definition of investigation variables so that observable and measurable features are specified.
  • Randomisation: Use of chance to reduce bias when designing materials and deciding the order of conditions, thereby reducing the researcher’s influence on study design.
  • Standardisation: Standardisation gives all participants the same environment, information, and experience; standard instructions help ensure that non-standardised differences do not act as EVs.

★ Must-know

📌 A directional hypothesis predicts the direction of a difference or relationship, whereas a non-directional hypothesis predicts a difference or relationship without specifying its direction.

Further detail

  • Uncontrolled EVs weaken the establishment of an IV–DV cause-and-effect relationship and can reduce study reliability.

  • Participants may overperform to please the experimenter (the please-you effect) or deliberately underperform to sabotage results (the screw-you effect).

Memory Hook

Uncontrolled influences blur the IV–DV relationship

2. Experimental Designs

Key Concepts & Definitions

  • Independent groups design: In an independent groups design, two separate groups experience different levels of the IV, with one in the experimental condition and the other in a control condition where the IV differs or is absent.
  • Repeated measures: In a repeated measures design, all participants experience both conditions, and the two sets of data from each participant are compared and analysed.
  • Matched pairs: In a matched-pairs design, participants are paired on relevant variables, then one participant from each pair is assigned to each of two conditions; the study proceeds like an independent groups design.
  • Experimental design: The way participants are organised into experimental conditions for testing.
  • Counterbalancing: Counterbalancing controls order effects in repeated measures by having half the participants experience conditions in one order and the other half experience them in the opposite order.

Essential Points

📌 Independent groups avoid order effects but may have participant-variable differences between groups, which can weaken cause-and-effect conclusions; random allocation is used to address this imbalance.

  • Independent groups designs are less economical than repeated measures because each participant contributes once and twice as many participants are needed to produce the same amount of data.

  • Repeated measures control participant variables and require fewer participants, but order effects such as boredom or fatigue may worsen performance on the second task, and participants may infer the study’s aim.

📌 Matched pairs reduce participant-variable differences but cannot match participants exactly, and matching is time-consuming and expensive; because participants take part in only one condition, order effects and demand characteristics are less problematic.

3. Types of Experiment and Population

Key Concepts & Definitions

  • Lab experiment: An experiment conducted in a highly controlled environment, which need not be an actual laboratory.
  • Field experiment: An experiment that manipulates the IV in a more natural, everyday setting.
  • Natural experiment: A natural experiment uses a pre-existing IV that would have changed even if the researcher had not been interested in it, and measures its effect on the DV; participants may still be tested in a laboratory.
  • Quasi experiment: A quasi experiment uses an existing difference between people, such as age or gender, as the IV; the researcher determines rather than manipulates this variable.
  • Population: The group of people who are the focus of a researcher’s interest, from which a smaller sample is drawn.
  • Sample: The group of people who take part in a study, drawn from a target population and presumed to represent it.
  • Generalisation: The extent to which findings can be applied to the whole population, which is possible when the sample is representative.
  • Random sampling: Gives every member of the target population an equal chance of selection, using a complete population list, numbered names and a lottery method to generate the sample.
  • Systematic sampling: Selects every nth member of a target population from an ordered sampling frame, using a nominated or randomly determined interval until the sample is complete.
  • Stratified sampling: Identifies population subgroups, calculates the proportions needed for a representative sample, and randomly selects participants from each subgroup.

★ Must-know

📌 Opportunity sampling selects people who are willing and available, whereas volunteer sampling uses people who select themselves to participate.

Further detail

  • Laboratory experiments provide control over EVs, support cause-and-effect conclusions, and allow replication, but may have low generalisability, external validity, and mundane realism because the setting or tasks can be artificial and behaviour may be unnatural.

  • Field experiments can produce more authentic behaviour and higher mundane realism and external validity, but reduced control over EVs makes cause and effect harder to establish and precise replication difficult.

  • Field experiments may raise ethical issues because participants may not give informed consent or know they are being studied, creating a risk of privacy invasion.

  • Natural experiments can enable research that is impractical or unethical to conduct otherwise and may have high external validity, but rare events limit research opportunities and generalisability, while non-random allocation can weaken cause-and-effect conclusions.

  • Quasi experiments can be conducted under controlled conditions, allow replication, and may have high external validity, but participants cannot be randomly allocated to conditions, so participant differences may confound results.

Memory Hook

In natural experiments the IV occurs naturally; in quasi experiments it is an existing difference between people

4. Ethical Issues in Research

Key Concepts & Definitions

  • Deception: Deliberately misleading participants or withholding information during an investigation; it prevents fully informed consent but may be justified if it does not cause distress.
  • Informed consent: Requires telling participants the study aims, procedures, their rights and how their data will be used, providing relevant information in a signed consent letter or form; participants under 16 require parental consent.

★ Must-know

  • Participants have the right to withdraw from a study at any time if they wish.

  • Researchers must protect participants from physical or psychological harm and must not expose them to greater risk than they would face in daily life, including embarrassment, inadequacy, stress or pressure.

  • Privacy is participants’ right to control information about themselves, whereas confidentiality requires protecting personal data if privacy is invaded; anonymity can be maintained by recording no personal details and using numbers or initials.

Further detail

  • At the end of a study, debriefing reveals the true aims and withheld details, explains the use of participants’ data, offers the right to withhold data, reassures them about their behaviour, and may include counselling after extreme stress or embarrassment.

  • The BPS code of ethics provides guidelines researchers must observe, and ethics committees use a cost-benefit approach to assess whether research ideas are ethically acceptable.

5. Study Procedures and Controls

Key Concepts & Definitions

  • Pilot study: A small-scale trial run with a small group of participants before the main investigation, used to identify problems and modify the procedure or design, saving time and money in the long term.
  • Single-blind procedure: Participants are not told the research aim, their condition or whether another condition exists, while the researcher knows the aim and conditions; this controls demand characteristics.
  • Double-blind procedure: Neither participants nor the researcher interacting with them know which condition each participant is in, and a third party may conduct it without knowing the condition assignments.

Essential Points

  • In drug trials, patients and doctors may not know who receives the real drug and who receives the placebo.

📌 An investigation usually compares an experimental group or condition with a control group or condition, which provides a baseline for comparison; a greater behavioural change in the experimental group supports a cause-and-effect link between the IV and DV if confounding variables remain constant.

Memory Hook

Single blind hides information from participants; double blind hides the aims from both participants and researcher.

6. Naturalistic Observations

Key Concepts & Definitions

  • Naturalistic observation: Watching and recording behaviour where it would usually occur, with all aspects of the environment free to vary so behaviour is more natural than in a laboratory setting.
  • Naturalistic observation: Watching and recording behaviour where it usually occurs, with environmental aspects free to vary.
  • Controlled observation: Watching and recording behaviour in a structured, controlled environment where the researcher manipulates variables.

★ Must-know

  • Naturalistic observations have high external validity because naturally occurring behaviour can be generalised to everyday life.

📌 Covert observation records people without their knowledge or consent, whereas overt observation records them with their knowledge and consent.

📌 Participant observation involves the researcher joining the group and producing a first-hand account, whereas non-participant observation keeps the researcher outside the group.

Further detail

  • Naturalistic observations are difficult to replicate because the researcher has little control over variables.

  • Uncontrolled extraneous variables make patterns of behaviour and cause-and-effect relationships difficult to establish in naturalistic observations.

  • Controlled observations are easier to replicate because extraneous variables may have less influence.

  • Controlled observations may be difficult to generalise to real life because an unnatural setting can produce demand characteristics.

📌 Covert observation must take place in public to be ethical.

  • Covert observation avoids participant reactivity because participants do not know they are being watched, so behaviour is natural and validity may increase.

  • Covert observation raises informed-consent concerns because people in public may not want their behaviour observed.

  • Participant observation can increase validity by giving researchers insight into the situation as participants experience it.

  • Participant observation risks researchers identifying too strongly with the group and losing objectivity, a process called going native.

  • Non-participant observation helps researchers maintain objective psychological distance and avoid going native, but may sacrifice insight gained through participation.

Memory Hook

Naturalistic observation follows everyday settings; controlled observation structures the setting.

7. Observational Design and Sampling

Key Concepts & Definitions

  • Structured observation: Simplifying target behaviours into clearly defined acts so researchers can quantify observations using predetermined behavioural categories and sampling methods.
  • Unstructured observation: Writing down everything observed, producing detailed accounts suited to small-scale studies with few participants.
  • Behavioural categories: Observable, measurable subdivisions of a target behaviour used to create a structured observation record or behaviour checklist.
  • Event sampling: Recording a target behaviour every time it occurs in a target individual or group.
  • Time sampling: Recording a target individual’s or group’s behaviour within a fixed, pre-established time frame using a behavioural checklist.

Essential Points

  • Structured observations use behavioural categories to make recording easier and more systematic, and produce quantitative data that is easier to analyse and compare.

  • Structured observations can produce reliable, replicable data.

  • Structured observations may lack contextual depth and can be susceptible to participant reactivity and demand characteristics.

  • Unstructured observations produce rich, detailed data but risk observer bias because objective behavioural categories are absent and researchers may record only conspicuous behaviours.

📌 Behavioural categories should be precisely defined, cover all ways the target behaviour may occur, avoid inference, and be mutually exclusive.

  • Clear behavioural categories make data collection more structured and objective and require no further interpretation, but ambiguous boundaries such as smiling versus grinning can reduce validity.

  • Event sampling is useful for infrequent behaviours that could be missed by time sampling, but complex events may lead observers to overlook important details.

  • Time sampling can reduce the chance of sampling observations that are unrepresentative of the observation as a whole, but may miss infrequent behaviours that occur outside the specified time frame.

Memory Hook

Event sampling counts occurrences; time sampling observes within fixed intervals.

8. Reliability and Study Design

Key Concepts & Definitions

  • Inter-observer reliability: The consistency with which observers record and judge the same behaviour.

★ Must-know

  • Researchers establish inter-observer reliability by familiarising observers with behavioural categories, having them observe the same behaviour during a small-scale pilot, comparing and discussing their records, and correlating pairs of observations to calculate an overall figure.

Further detail

  • At least two researchers should conduct observations because a single observer may miss details or notice only events that confirm their opinions or hypothesis, introducing bias.

  • Designing an observational study involves deciding its aims and hypothesis, setting, observer status and involvement, sampling method, dependent-variable recording, observation structure, behavioural categories and checklist, ethical issues, and results presentation.

Memory Hook

Train observers, pilot together, compare judgements, then calculate reliability.

9. Questionnaires, Open and Closed Questions

Key Concepts & Definitions

  • Questionnaire: A preset list of written questions or items answered by participants to assess their thoughts and feelings, using open or closed questions according to the desired depth of response.
  • Open questions: Open questions do not offer a fixed range of answers, allowing respondents to answer freely and producing in-depth qualitative data that may be difficult to analyse.
  • Closed questions: Closed questions offer a fixed number of responses, such as yes/no or a rating from 1 to 10, producing numerical quantitative data that are easy to analyse but lack depth and detail.
  • Interview: An interview is a live, face-to-face or telephone encounter in which an interviewer asks questions to assess an interviewee’s thoughts or experiences; questions may be preset or develop during the conversation.
  • Structured interview: A structured interview consists of a predetermined set of questions asked in a fixed order.
  • Unstructured interview: An unstructured interview is a free-flowing conversation with no set questions but a general topic, in which the interviewer prompts the interviewee to expand and elaborate on answers.
  • Semi-structured interview: A semi-structured interview uses questions written in advance while allowing the interviewer to ask follow-up questions when appropriate.

★ Must-know

📌 Structured interviews are easy to replicate because their standardised format reduces differences between interviewers and the effect of interviewer bias, but their fixed questions limit elaboration and the detail obtained.

📌 Unstructured interviews allow follow-up questions and free elaboration, but their data are difficult to analyse, conclusions may be hard to establish, and respondents may give socially desirable answers; a skilled interviewer may build rapport to encourage truthful discussion of sensitive topics.

Further detail

  • Questionnaires are cost-effective because they can quickly gather data from large numbers of people.

  • Questionnaires reduce researcher bias and effort because the researcher need not be present while participants complete them.

  • Fixed-choice closed questions can produce quantitative data that is easy to analyse and can support statistical analysis and comparisons between groups using graphs and charts.

  • Questionnaire responses may be untruthful when respondents try to present themselves positively, producing social desirability bias as a demand characteristic.

  • Acquiescence bias occurs when respondents tend to answer in the same way, such as always ticking yes, possibly because they completed a questionnaire too quickly to read the questions properly.

Memory Hook

Structured interviews standardise questions; unstructured interviews let the conversation flow.

10. Self-Report Design and Questions

Key Concepts & Definitions

  • Likert scale: A Likert scale asks respondents to indicate agreement or disagreement with a statement on a usually five-point scale ranging from strongly agree to strongly disagree.
  • Rating scale: A rating scale asks respondents to select a value representing the strength of their feeling about a particular topic.
  • Fixed choice option: A fixed choice option presents a list of possible answers and requires respondents to indicate all options that apply to them.
  • Leading question: A leading question is phrased so that the questioner’s attitude toward a topic makes the respondent feel obliged to answer in a particular way.
  • Double-barrelled question: A double-barrelled question contains two questions in one, so respondents may agree with one part but not the other.

★ Must-know

  • An interview schedule is a list of questions the interviewer intends to cover, and standardising it for each participant helps reduce interviewer bias.

Further detail

  • Interviewers may take notes during an interview or record it for later analysis.

  • Interviews commonly involve one interviewer and one participant in a quiet room away from other people, while neutral opening questions can help the participant feel comfortable and build rapport, especially when topics are sensitive or personal.

  • Good interview questions avoid jargon unfamiliar to people outside a specific subject area.

  • Good interview questions replace emotive wording with neutral alternatives so the wording does not lead the respondent toward a particular answer.

11. Correlations and Their Limits

Key Concepts & Definitions

  • Correlation: A correlation is a mathematical technique for investigating the association between two measured co-variables, which are plotted on a scattergram.
  • Co-variables: Co-variables are two variables that show a correlation rather than causation and are not called independent and dependent variables because they do not represent a cause-and-effect relationship.

★ Must-know

📌 A positive correlation means that as one co-variable increases, the other also increases; a negative correlation means that as one increases, the other decreases; and a zero correlation means there is no relationship between them.

📌 Experiments manipulate or control the IV to measure its effect on the DV, whereas correlations do not manipulate either co-variable and therefore cannot establish cause and effect, even when the correlation is strong.

Further detail

  • Correlations provide a precise, quantifiable measure of the strength and direction of a relationship, which may suggest further research when the variables are closely related.

  • Correlations can be quick and economical because they require neither a controlled environment nor variable manipulation, and use of data collected by others can make them less time-consuming than experiments.

  • Correlations cannot show why variables are related or which co-variable causes the other to change, and an untested third variable may explain their relationship.

Memory Hook

Correlation shows association; an experiment manipulates an IV to test its effect on a DV.

12. Data Types and Sources

Key Concepts & Definitions

  • Quantitative data: Quantitative data are expressed numerically, usually as individual participant scores, and can be statistically analysed or presented in graphs and charts.
  • Qualitative data: Qualitative data are expressed in words rather than numbers or statistics and may describe participants’ thoughts, feelings, and opinions; they can be converted into numbers for analysis.
  • Primary data: Primary data are original data collected first-hand by the researcher specifically for the investigation, using methods such as experiments, questionnaires, interviews, or observations.
  • Secondary data: Information collected by someone other than the researcher that already exists before the research begins and may or may not have already been subjected to statistical testing.
  • Meta-analysis: Research about research that combines results from multiple studies on a topic to provide an overall view, using qualitative reviews of conclusions and/or quantitative analyses that produce an effect size.

★ Must-know

  • Secondary data can vary substantially in quality and accuracy, be outdated or incomplete, or fail to match the researcher's needs because the original investigation had different aims.

Further detail

📌 Quantitative data are simple to analyse, allow easy comparisons between groups, and are more objective and less open to bias, but their narrower scope may reduce ecological validity and make real-life generalisation difficult.

📌 Qualitative data provide rich detail and broader scope because participants can develop their views, and may offer greater external validity through meaningful insight, but they are difficult to analyse, cannot be summarised statistically, and rely on subjective interpretation that may be affected by researcher bias.

  • Primary data is authentic information obtained from participants specifically for an investigation, with questionnaires and interviews designed to target the researcher's required information.

  • Collecting primary data requires time and effort, and producing it through an experiment requires considerable planning, preparation and resources.

  • Secondary data can be found in:

    • journal articles
    • books
    • websites
    • government statistics such as the Census
    • population records
    • an organisation's employee absence records
  • Secondary data is quick, cheap and easy to access, requires minimal effort, and removes the need to conduct an experiment to collect primary data.

Memory Hook

Quantitative data use numbers; qualitative data use words.

13. Descriptive Statistics and Graphs

Key Concepts & Definitions

  • Descriptive statistics: Use graphs, tables and summary statistics to identify trends and analyse data sets.
  • Measures of dispersion: Measures of the spread or variation in a set of scores.
  • Normal distribution: A symmetrical spread of frequency forming a bell-shaped curve, with the mean, median and mode at its highest peak.
  • Measures of central tendency: Measures of the average value in a data set.
  • Median: The central value in a data set when its values are arranged from lowest to highest.
  • Mode: The most frequently occurring value in a data set.
  • Scattergram: Represents the strength and direction of the relationship between co-variables in a correlational analysis.
  • Skewed distribution: An asymmetrical spread of frequency in which data clusters at one end and appears to lean to one side.

Essential Points

  • The mean is calculated by adding all values in a data set and dividing by the number of values.

📐 Formula - The range is calculated as the highest score minus the lowest score, plus 1 as a mathematical correction: Range=highest score−lowest score+1\text{Range} = \text{highest score} - \text{lowest score} + 1.

  • Standard deviation measures how much scores deviate from the mean; the variance is calculated from the differences between the mean and each score, and the standard deviation is the square root of the variance.

  • A bar chart represents the frequency of each variable by bar height and can display mean values across categories of discrete data.

📌 In a positive skew, the long tail is on the positive (right) side of the peak and most of the distribution is concentrated on the left; in a negative skew, the long tail is on the negative (left) side of the peak and most of the distribution is concentrated on the right.

14. Mathematical Skills and Statistical Testing

Key Concepts & Definitions

  • Statistical testing: Determines whether hypotheses should be accepted or rejected by assessing whether differences or relationships between variables are statistically significant or have occurred by chance.
  • Probability: Probability is the likelihood that a result occurred by chance rather than through the effect or relationship being studied; a result is significant when probability is less than 0.05, allowing the researcher to accept the experimental hypothesis and reject the null hypothesis.
  • Significance: Indicates that an observed difference is unlikely to have occurred by chance, assuming the null hypothesis is true; this does not establish the experiment's hypothesis as true.
  • Critical value: A benchmark used to determine whether a statistical test result is significant by comparing the calculated value with a critical-value table selected using the sample size and whether the hypothesis is one-tailed or two-tailed.

★ Must-know

  • 🔄 To carry out the Sign Test:
    1. Calculate each difference and mark it positive or negative.
    2. Count the positive and negative signs.
    3. Take the least frequent sign as S, the calculated value.
    4. Compare S with the critical value using N after excluding ties, probability 0.05 and the hypothesis direction.

Further detail

📐 Formula - The percentage of participants is calculated as the number of participants divided by the total number of participants, multiplied by 100: Percentage=number of participantstotal number of participants×100\text{Percentage} = \frac{\text{number of participants}}{\text{total number of participants}} \times 100.

  • To convert a percentage to a decimal, move the decimal point two places to the left, which is equivalent to dividing by 100.

  • Results can be estimated by calculating the mean or range of a data set or by rounding to a specified significant figure.

📌 In the Sign Test, if S is smaller than the critical value, the difference is significant, so the null hypothesis is rejected and the hypothesis is accepted.

15. Peer Review in Research

Key Concepts & Definitions

  • Peer review: The evaluation of research by experts before publication to check its accuracy, methodology and relevance and help ensure its quality and credibility.

★ Must-know

  • Peer review helps decide which research should receive funding, assess research quality and validity of methods, check that conclusions rest on solid evidence, recommend improvements and, in extreme cases, advise withdrawing unsuitable work from publication.

  • Publication bias occurs when journal editors favour significant, headline-grabbing positive findings, causing research without such results to be ignored and creating a misleading impression of findings.

  • Peer review can suppress research that challenges mainstream theories because reviewers may favour work supporting their own views, allowing established findings to pass more readily than innovative opposition and slowing change in a discipline.

Further detail

  • Peer review is described as supporting the validity and accuracy of research.

  • Reviewers remain anonymous to encourage honesty, but anonymity can be used to criticise rival researchers, particularly when researchers compete for limited funding.

Synthesis Tables

Correlations and Experiments

FeatureCorrelationExperiment
Variable handlingMeasures co-variables without manipulationManipulates or controls the IV
InferenceShows association, not causationMeasures the IV’s effect on the DV
Setting and effortNo controlled environment required; may use others’ dataRequires manipulation and control

Test yourself

Take a 51-question practice quiz on Observational and Self-Report Methods, with every answer explained.

1. A researcher varies the amount of background noise and records participants’ reading accuracy. Which feature makes this an experimental method?

2. Which statement is a hypothesis for a study comparing conversation levels after caffeine or water?

Take the quiz →

Study with flashcards

Memorize the key concepts of Observational and Self-Report Methods with 98 flashcards.

What does the experimental method manipulate to measure an effect?

The independent variable.

What does a hypothesis predict about variables and study outcomes?

A testable relationship between variables and the expected outcome.

What does a directional hypothesis specify that a non-directional hypothesis does not?

The direction of a difference or relationship.

See the flashcards →

Related classes

Make your own study guides

Upload your class notes and Revizly writes a study guide, a quiz and flashcards in about a minute.

Study guide maker