Wikiversity:Fellow-Programm Freies Wissen/Einreichungen/Effects of the Generic Masculine or its Alternatives? A Multi-lab Replication and Extension of Stahlberg, Sczesny, and Braun, 2001
In languages such as German, French, or Hindi, plural forms of job occupations and societal roles are often in a generic-masculine form instead of a gender-neutral form. Although meant as “generic”, this generic-masculine form excludes women from everyday language and might even entail the cognitive effect that listeners and readers will less likely think of women. Several studies have demonstrated this and related cognitive effects in the past. Due to the societal relevance of gender-neutral language, I and my collaborators propose a direct replication and extension of a classic study by Stahlberg, Sczesny, and Braun (2001, Experiment 2) in a multi-lab setting. In this study, German speaking participants were asked to name celebrities from several domains, such as politics, music and sports. When they were asked in a gender-neutral form (e.g., “Bitte nennen Sie Politikerinnen und Politiker”; English: “Please name male and female politicians”), participants came up with more women than when the generic-masculine form was used (“Bitte nennen Sie Politiker”; English: “Please name politicians”). However, the original study did not control for potentially relevant control variables and moderators, such as the participants’ political orientation. Hence, to achieve sufficient statistical power, the proposed multi-lab study will be conducted among 12 labs, collecting data from approximately N = 2,000 participants.
The manuscript for the mulitlab study was finalized and submitted at the International Review of Social Psychology (https://www.rips-irsp.com/)
The editors of this journal welcomed the submission and created a new OSF project (https://osf.io/fdrn6/)
In to bring the manuscript in a standardized format, I had to split it up into four parts: Theoretical Rationale and Hypotheses, Pilot Data, Methods and Procedures, and Planned Analyses
By the end of the month, one reviewer had submitted his review.
Overall, this first review was positive about the multi-lab study and only had minor ideas how to improve both the manuscript and the research design of study
These changes will be implemented as soon as the second reviewer has submitted his review
The second reviewer was positive about the overall aim of the project and appreciated the multi-lab approach
He also suggested several changes and improvements, which will be summarized here (the full review can be accessed after completion of the project: https://osf.io/c3rvw/)
Adding some theoretical information on the "perceived base rate" in the theory section
Reorganization of the manuscript: putting the pre-studies into the supplementary materials and focussing on the description of the main study
Clarification of the power analysis (i.e., how we justify our sample size) across all labs
Adding multi-level analyses (i.e., controlling for lab effects and participant effects directly in the statistical model)
The letter and the revised version of the manuscript was sent to the group of co-authors from the collaborating labs
Suggestions from the co-authors were implemented
New version of the manuscript was finalized and re-submitted to the journal.
Short Interim Report for Wikimedia Closing Event[Bearbeiten]
While the topic of my research is important as it contributes to an ongoing public debate (see here and here), I could not conduct my research as expected thus far. The original plan was to finish the data collection in late 2020, but this date has passed.
However, at this point it is important to note that despite this delay being somewhat annoying for my research team and myself, it is the best for the scientific endeavor: I am conducting a so-called registered report, which is simply a reserach article that is peer-reviewed before the data collection. Hence, the reviewers and the editors evaluated my (planned) research solely on the basis of my theory and methods, whereas in conventional articles they would only see the finalized research. Now this could be misunderstood as a ineffective way to do science: After all, why should I submit something in a prior form to a journal?
The reason why a registered report is a superior format to the conventional articles lies in the publication system itself: conventional articles suffer from the fact that reviewers look at whether the results are in line with the theory and methods and therefore "make sense". The authors are then inclined (and partially motivated by the reviewers) to present their data in the most favorable way, which leads to a excess of positive results in the literature (see Fanelli, 2010; Scheel et al., 2021). In the long run, this results in a distorted depiction of reality in the the published literature (also known as publication bias, see Kühberger et al., 2014). On the contrary, registered reports will provide feedback only before the data collection starts and come with a contract: no matter the results of this study, it will be published anyhow as long as you do, what you planned to do.
The reviews that I received were comprehensive, but constructive. Importantly, they came before I collected a single piece of data, which motivated me to change my analysis strategy or make slight changes in my materials. These changes were always well considered as I had to think twice and thrice about what I wanted to achieve with the study. In other words, it made it clearer for me how to do the best possible science: open and transparent science.
