Exploratory Field Study Instructions: in Naturalistic Observation
Goal– Age Difference in Cultural Engagement Research Paper Help
The purpose of this assignment is to complete the training in the SP BA program through carrying out a field study where psychological principles are applied to messy, real-world settings. Students will carry out an exploratory field study using naturalistic observation, which means unobtrusively recording naturally occurring behaviours. A confirmatory field study that tests a hypothesis is also allowed. We are leaving the research topic open for students to have the freedom to explore a topic of their own interest. However, we also provide guidance and suggestions.
A group of 2 of students will:
(1) choose a topic in social psychology of interest to their group
(2) generate a research question and design a field study involving naturalistic observation
(3) search the literature for relevant articles
(4) conduct the study
(5) compute inter-rater reliability measures on the observations
(6) analyse and interpret the data
(7) individually submit a 2000-word-max (±10% okay) APA-style paper. The word count includes in-text citations and excludes abstract, tables, figures, and references.
Groups will collaborate on the design, coding scheme, data collection, and data cleaning. Each student will independently perform the analyses and write independently.
The final paper will include the following sections: title page, abstract, introduction, method, results with at least one table or figure, discussion, and references. See the assignment materials on Canvas, and also various pages on topics such as literature search, data analysis, writing, etc. at www.cameronbrick.com under ‘Resources’.Like in a research paper, lengthy methodological detail (e.g., the coding scheme) or robustness tests that are inappropriate for a concise narrative can be put in an Appendix (no word limit, after References) and any such pieces must be referenced in the main text. Optionally, any additional materials for reproducibility and transparency could be uploaded to the Open Science Framework and linked in the main text.
We encourage groups to select their own behaviour, context, and predictors. However, we have some pre-built suggestions if you want assistance. Any of the suggested topics are sufficient to pass the assignment. High grades generally require creative and novel approaches that demonstrate more independent work than pre-built materials provided here.
Pre-built topics (or develop your own)
Predictors could include features of the context (location, time of day, day of week, distance between people, presence or absence of something), of the people interacting (gender, ethnicity, age, formality of dress, group size, alcohol, other demographics), or other variables. Be creative.
Searching the literature– Age Difference in Cultural Engagement
Students will search the psychological literature using PsycInfo/Google Scholar/Web of Science to build an argument for the relevance of the research question using psychological literature. We suggest a minimum of three empirical research articles that were not assigned as readings. Use both narrow and broad search terms. E.g., about whether people hold the door for others, one might search for door holding but also the broader area of prosocial behavior. Students are welcome to include literature reviews or book chapters, but must focus on empirical articles. Provide the complete citation and reference for each article in APA style. Students are unlikely to find many recent articles describing studies involving naturalistic observation, given their scarcity. That’s OK. The assignment is to find articles that provide support for the study, regardless of what specific method or question earlier researchers used. Part of the intended learning outcome is to identify these evidence gaps and build chains of logic from evidence of other behaviours.
Proposal
A proposal (in the form of a short presentation) will be due in one of the tutorial sessions describing (1) whom you will observe and where you will observe them, (2) exactly what behaviours you will observe and how you will code them (operational definitions of the behaviours involved), and (3) your hypothesis (what do you expect and why?). Whenever you make a claim, like “Men are taller than women on average,” you must cite the empirical evidence. You will observe naturally occurring behaviour in a public location without the expectation of privacy, there is usually no manipulation, and we are not intending to publish these data in a journal. Therefore, there is no need for ethics approval, information sheets, or consent forms.
Suggested (Mostly Social) Psychology Journals
These journals primarily publish empirical articles. Other journals are welcome.
| Basic and Applied Social Psychology European Journal of Social Psychology Journal of Applied Social Psychology Journal of Experimental Social Psychology Journal of Nonverbal Behaviour Journal of Personality and Social Psychology Journal of Social and Clinical Psychology Journal of Social and Personal Relationships Collabra: Psychology | Motivation and Emotion Personal Relationships Personality and Social Psychology Bulletin Psychological Science Self and Identity Social Cognition Social Influence Group Processes & Intergroup Relations Journal of Environmental Psychology |
Other sources, often containing review articles instead of empirical results
Advances in Experimental Social Psychology
Annual Review of Psychology
Personality and Social Psychology Review
Psychological Bulletin
Psychological Inquiry
Psychological Review
Stages of Conducting the Study
1. Generating a hypothesis and providing a rationale
The first step is to think of some frequent, observable behaviour in public that you find interesting. See our pre-built suggestions above, but do not be limited by them. Many interesting hypotheses can be generated about observable behaviours including movement or speech. For example, you might consider examining gender differences in some behaviour. Or, you might test a hypothesis that involves examining interactions between people. Other interesting hypotheses could be generated from the social influence materials in previous courses.
Writing the introduction
During the tutorials, explanation will be provided on what elements are important to provide in an introduction for a practice-oriented research paper. This differs from other courses in the Social Psychology BA because the learning objectives in this project are not primarily about articulating logic from previous theory. It is not required to situate the study within a single theoretical framework such as the Theory of Planned Behaviour. It is ideal to base your research question within existing literature and models, but the main focus in this assignment is on methods, results, and interpretation. Therefore, the Introduction section will be relatively short compared to assignments in other courses in the major.
Experiments with manipulation
Manipulation is not recommended because of the complexity. However, it is possible to set up a situation in a public setting and observe individuals’ reactions. For example, you could place a piece of litter on the ground and see whether people are more likely to pick it up if they are in groups or by themselves. You could also consider whether people’s behaviour differs depending on whether or not they are being watched. For example, you could observe drivers’ behaviours at a zebra crossing (crosswalk) in a non-busy area (1) when they do not know that they are being watched, and (2) when they are aware that they are being watched. In this case, you’d want to keep certain things constant across conditions (always use the same stop sign, observe in the same kind of weather at approximately the same time of day). Most groups will not use such a manipulation. Any proposal for manipulation needs to be approved by your tutor. It is allowed to work together with another team on the same topic, for example by choosing to observe the same behaviour but in different situations, so a potentially interesting comparison could be made. Data collection should happen together with your partner: you both will observe the same behaviours at the same time in the same place.
Please do not confine yourself to these brief examples. Be creative, but also be ethical: for example, only observe behaviour in places where people do not have a reasonable expectation of privacy (e.g., do not collect data in bathrooms).
2. Operationally define your variables
Next, operationally define your variables. For example, in the litter example above, you need to define what counts as a “group”. You could simply record the number of people that pass by the piece of litter at the same time, but do they actually constitute a “group” (i.e., are they talking to each other and so forth)? If someone else is walking close behind them, is that person part of the group? How much distance until you code them as not within the same group? In addition, you need to define the outcome variable, e.g., (not) picking up the litter, or finer measurements (e.g., seconds until picking up the litter)? Other variables to measure might be related behaviours: looking at the litter, walking directly over the litter, walking around the litter, etc.
At this stage, it is highly recommended to practice the data collection by observing the intended behaviour for an hour or two. How frequent is it? Will that work for your data collection plan? Are there unanticipated challenges about how you expect it to go? It would be surprising if there weren’t. Field studies are challenging and messy. Adjust your data collection plans to match what is feasible and leads to the best measurement validity.
3. Creating an observational recording instrument
Once you’ve created your observational definitions, the next step is to create the coding sheet for the observations. For example for the stop sign study:
Date: _______ Day: _______ Time: _______ Weather: ___________ (Other:___?)
| Observation | Condition 1 = researchers are visible 2 = researchers are not visible | Gender of driver 1 = male 2 = female | Type of stop 1 = full stop 2 = rolling stop 3 = no stop |
| 1. | |||
| 2. | |||
| (etc.) |
A key choice is determining what constitutes a single observation: how many people, how close by, what behaviours by the cars, etc.
Or, if you’re observing an interaction over time (e.g., 10 minutes), you could record the frequency counts of certain behaviours (e.g., people tossing their hair or checking their phones during a conversation). All key variables should be represented on the coding sheet.
4. Location
Next, figure out where it makes the most sense to observe people. In the litter example, in order to ensure enough participants, you’d need observations in a high-traffic area because littering is relatively rare, but not such a busy area that you couldn’t keep up with recording the data. If you are trying to be unobtrusive and not let people know they’re being watched, how will you accomplish that? Figure out where you could position yourselves. Are you going to observe every person that comes by a specific point? Every third person? Only people in groups? If you are manipulating something (e.g., litter/no litter), make sure you flip a coin or otherwise randomly determine what condition you’re using each time. Going outside and practicing for 10 minutes with a draft observation sheet is great preparation at this stage.
In addition to filling in your sheet, keep a separate paper for writing down general observations. For example, did people notice you? Did anyone say anything to you? Did you notice any other interesting behaviours? Was it raining? Did you have unexpected problems? This will help you write the final report.
5. Sample size and reliability
The best-practices approach would include a justification for the sample size and/or a post hoc sensitivity analysis to describe the observed power for each hypothesis. However, this is impractical for many of the custom designs that will be run, so it is not required to run a power analysis for this assignment. However, if you’d like to learn more, see www.cameronbrick.com/cleaning-and-analysis.
Minimum: each student should spend at least 3-4 hours observing and coding. To ensure sufficient sample size for data analysis, keep in mind that each observer’s data should have at least n = 30 in each major cell of the design (e.g., 30 observations of mask on AND 30 of mask off, or 30 each of car stopped AND 30 of car didn’t stop).
Multiple observers and inter-rater reliability (IRR). Different observers generate different data even watching the same events, which is one source of measurement error. Ideally, studies are run with at least two people making observations together, and then checking inter-rater reliability to test the similarity of the ratings. For instance, one could record whether cars come to a full stop, rolling stop, or no stop. Train together and discuss, and then at the data collection times, each observer records what they see without consulting one another. Afterwards, they compare notes and quantitatively check the rate of agreement (see separate instructions about IRR). If the agreement is too low, you should usually redefine the coding and/or do more training together and then do the observations again. The IRR results are reported in the Results section. This is best practice and a goal for all projects in the course, but is not strictly required to pass because of potential design and data collection errors. All projects can reflect on different aspects of inter-rater reliability and their implications for the inferences, whether or not IRR can be calculated.
6. Analysis
For design and analysis help, please see the Field textbook from previous courses (Discovering Statistics using IBM SPSS Statistics), course handouts, and www.cameronbrick.com/cleaning-and-analysis. The below is just a guide. Please use the analysis that is most appropriate for your design and measurements.
Categorical data (data that fall into discrete categories)
Example 1: whether a person (1) picks up or (2) does not pick up litter
Example 2: whether a car (1) stops completely, (2) comes to a rolling stop, or (3) doesn’t stop.
With categorical data, you report percentages (e.g., % of people who pick up litter and % who don’t). You could conduct a chi-square statistic to determine if the percentages of people in the different categories differ from what you’d expect by chance alone.
Continuous data (specifically, interval/ratio) data (data that fall along a continuum)
Example 1: distance in centimetres that men vs. women stand apart from each other when talking
Example 2: number of times a person checks his/her phone during a 10-minute period
With continuous data, you could report means and standard deviations for the distance (cm) between conversational partners for male-male dyads and female-female dyads. You could compute an independent groups t-test if you were comparing two groups (e.g., do male-male and female-female conversational distances differ?), an ANOVA if you were comparing 3 or more groups, or a regression if your predictor and criterion variables were both continuous.
Evaluation of group members
Each group member will evaluate their own and the other group members’ participation on various aspects of the project, and these evaluations can affect the final grade. This is designed to increase transparency of your group process to the lecturers, to improve fairness, and to reduce free riding.
Suggested Schedule (see main Canvas page)
This is a fast-paced course. Students are expected to work on the course full-time (40 hours/week) for four weeks and there is little room to fall behind. Here is a suggested schedule for independent work outside of the tutorials and lectures. Preparatory Assignments (PAs) are due 12 hours BEFORE the start of the tutorial. Please make your own custom schedule of due dates based on your tutorial day.
Further Help
Post on the Discussion Forum on Canvas! And please help each other there, too.
6. Assignment grading criteria
| Intro The introduction is clearly, coherently and convincingly argued why the chosen topic is relevant to study, the particular study is relevant/important, and how the study will contribute to new knowledge and understanding.Relevant empirical articles are well-integrated into the argument and their relevance for the RQ is clear. This depends on the use, integration, and explanation of the literature.Examples are provided when appropriate to clarify theoretical pointsThe hypothesis(/es) is stated clearly (maybe visually) and follows logically from the introduction Whether the study is a replication (not novel) or not, convincing justification is provided for why the study is worthwhile |
| Method All relevant participant info is includedThe method is appropriate to answer the research question, given unavoidable constraintsThe coding scheme is clearly explained in the method section and included in the AppendixThe coding scheme is high quality, based on: a) Clear focus (who and what is observed) in line with the RQ b) Categories are mutually exclusive c) Easy to operate d) Need for interpretation is minimised e) Key observable factors are all included Transparently explains the design and execution of the field study from start to finish, including a short description of the unstructured observation that led to the construction of the coding scheme and collection and what cleaning of the data was necessary.Validity and reliability are addressed in some fashion, ideally with quantitative checks. The point is to reflect on the vulnerabilities of the operationalisation for inference and assess/mitigate these risks where possible. Depending on the project, these issues might be dealt with mostly in the Discussion. |
| Results Descriptives and analyses are clearly and accurately reported (including inter-rater reliability)Appropriate statistical tests are usedTables and figures are properly described and interpreted in the textFigures are drawn appropriately (e.g., large, readable axis labels, legend, appropriate scale of y-axis, etc.) |
| Discussion Appropriate and clear summary and interpretations of the findings. This should be short.The findings are discussed in relation to previous research, and possible explanations are given for discrepancies. If the main results are null (non-significant), relatively more time can be devoted to non-trivial suggestions for future research (e.g., suggesting collecting more data would be a shallow suggestion).Perhaps within the previous point, correct and clear reflection on the specific features of the validity and reliability of the method, whether or not these two words are used.Discussion of relevant and creative suggestions for research, and optionally, practical or policy implications |
| General APA 7 format requiredHigh grades will require excellent organisation (e.g., smooth transitions between levels of abstraction), concision, lack of redundancy, and clarity at all levels (structural, paragraph, and sentence). This would require many rounds of editing.1 bonus point is available for superior efforts, including especially effortful and novel designs, coding schemes, other unusually independent features of the project such as correct and complete power analysis, or exceptional narrative logic and argumentation, integration with previous literature, and superb concision. |