Supporting quality in the humanities – chances of open sources implementation in research and training[Bearbeiten]
The goal of this project is to establish awareness and explore potential solutions for the automatic reproduction of incorrect data in Art History in the German landscape, acting exemplary to illustrate the as of yet largely unrealized potential for open science in the humanities.
Open science offers new possibilities in comparing data, verifying research and in using more democratic participation and feedback to help identify and correct incorrect data. However, in the existing system the correction of unclean data is often only a by-product of good research.
Within the current shift, where open science practices are being implemented, and shared open practices guidelines are being formulated, it is instrumental to analyze the systematic origins and replicating modes that create and recreate incorrect data in order to establish better practices.
With a set of three specific case studies the proposal plans to realize a publication examining the status of openly available sources, digital application and critical reflection of data quality in Art History. By documenting clear examples tracing the citation and re-citation of misinformation in historical, and now digital, form, the proposal hopes to help establish source criticism and source correction as its own topic worthy of becoming a complementary science.
To change attitudes and practices within the field, it is considered essential to first establish the
problem on the basis of clear evidence. Therefore, the first step and the foundation of a better
process of quality control has to be the collection of thoroughly dissected examples and conclusive
In the framework of the Wikiversity Fellow Program the plan is to focus on the publication of a paper
analysing three clear examples that are originated in the applicant’s previous research. This ensures a
realistic timeline and allows to do additional examination of discourse and to contextualize the
findings within the current developments of open science and open data.
The paper will provide three concrete examples of incorrect data being cited and reproduced in
digital form – including Wikipedia - as well as examine teaching materials to determine the status of
both the use of open science applications and the critical reflection of sources in academic training.
For each of the three case studies it can be traced how the misinformation came to be, that it has
created further false derivatives and that it is currently being reproduced in digital form.
The proposed paper will consist of four parts:
- Three case studies demonstrating in detail the problematic reproduction of incorrect data
- A review of current teaching materials and study guidebooks with regards to source criticism
in academic research
- A review of the current status of discussion on quality control in academic publications,
including institutional guideline for good scientific opens source practice
- An outlook of possible solutions on the training, researcher and institutional data managing
level with examples and practices from other disciplines
The findings of the study will be published in form of a concise paper in an academic journal. All
related material and data sets will be made available through Wikiversity or other Wikimedia open
source channels and referenced to in the published paper.
The study can in part be based on materials collected for the applicants 2016 Magister exam,
including documentation of academic teaching materials and the then status of discourse on critical
source examination. This allows for a 5-year comparison of the development of source criticism in Art
History, and especially for tracing how implementations of open access have affected questions of
good scientific practice.
The three case studies are based in previous research. This ensures good knowledge of the
problematic data and it’s contextualization. The project flips the focus from the original research
question to investigate origins and modes of replication of misinformation. Each examples highlight a
different aspect of these objectives and a different method in examination.
review and additional research:
- Submit paper proposal to art history or semantic web journals
- 2-day research trip to Wolfenbüttel (cancelled due to Covid)
- review of and adding to material gathered
- requesting teaching materials from German Art History study programs
- collection of additional literature for case studies online and in Berlin libraries
- collection and review of study guidebooks online and in Berlin libraries
- building Citavi and Zotero bibliographical database
- review of the literature and first outline draft
- incorporation of project feedback from Wikiversity workshop
case study writing:
- review of additional literature for case study
- Writing case study draft
conducting additional research to fill gaps, state of discourse chapter draft
- research trip to Heidelberg to review additional literature not available in Berlin (in contact with University Library; visit for non-students still not possible as of March 2021)
- finalising case study drafts with feedback from Wikimedia program
- review of discourse literature and draft
- final report outline
final writing and preparation of material
- final report writing and review
- preparation of material for future open access publication
- final presentation of results and future outlook
|Plan to finalize by
|Identify journal for publication
|wishlist collected, reach out to open access papers, ask advice of colleagues
|OA journals new / rare in Art History
|completed January 2021
|Research trip Wolfenbüttel
|postponed / cancelled
|organize, fill gaps
|completed March 2021
|building Citavi bibliographical database (extended to Zotero)
|currently 300+ titles in 10 main categories with subcategories (continuous process of organizing, ordering books where necessary). Extended to include Zotero based on open access sharing principles and contact with Zotero Cita plug-in developer
|completed May 2021 / ongoing extension where needed
|incorporation of project feedback from Wikiversity workshop
|reach out to additional sources
|information research aspects relevant
|collect teaching materials from German Art History study programs
|completed collection and initial review of circa 35 institutional "good research practice" PDF guidelines
|completed December 2020
|WikiData case study linking
|collaboration with fellow Fellow Florian Thiery
|testing out functions for automated visualization, incorporating data in open access system for tracing
|end of March 2021 / ongoing extension
|collaborations Wiki community
|established weekly digital "coffee" meet-up within fellows community, main collaborations with fellow fellows Sophie Schmidt and Florian Thiery, reached out to and created input collaboration with Zotero Cita plug-in creator and developer Diego de la Hera (supported by a grant from the Wikimedia Foundation), joined German kuwiki (Arbeitsgemeinschaft Kunstwissenschaften + Wikipedia) and attended Wikipedia and art historians collaborative meetings
|introduced project and led discussion group Arbeitskreis Digitale Kunstgeschichte, contact NFDI4culture, online conference discussion Open Bar Camp etc.
|discuss ideas and approach, gain additional insights, promote quality control aspects
|collaborative report written with fellow fellow Florian Thiery "How to increase data quality in the Digital Humanities? Two Linked Open Data community approaches"
|based on WikiData focused collaboration established during fellows program
Due to Covid several aspects of the early project phase had to be adjusted. Visits to libraries that would have been only restricted during the summer are now not possible at all.
Instead, wide initial research online led to some positive results, but also soon clarified the need to set project and research boundaries in order not to get too lost.
In a change to only a few years ago, many Art History university departments (not all!) now have a PDF on "academic writing guidelines" or "good research practice" available online. This allows for comparing instruction material from a much larger selection of German institutes (planned are more quantitative analysis of 10-20 and a more qualitative of circa 3-5). This of course is only one aspect of educational materials and need to be examined in comparison and context to both tutorial materials and published standard study guideline books. However, it is proposed that the digital approach makes teaching material more easily adaptable to changes and that students will likely rely on the most easily accessible source.
The meeting via zoom with the mentor and fellow mentee Andreas was very helpful in exchanging ideas (both on each project and in about more general, but relevant, topics) and especially important to establish more of a timeline and set clearer boundaries for the project development. Explaining research to others is also found to be extremely important in realizing where the project needs clearer argumentation (and sources) and which aspects work well or are interesting to people within the field.
Joined Arbeitskreis Digitale Kunstgeschichte [ workgroup Digital Art History: https://digitale-kunstgeschichte.de/]. Continued building Citavi bibliographic database and widely reading for outlook contextualization chapter. Gained relevant insights on for example reputation economy and bidirectional linked data. Looked into methods for analysis and documentation such as Webspider, Web Harvey and HTTrack.
Visit to Technical University / University of Arts library Berlin (one of the few allowing to get books oneself from open stacks and thus allowing for quick cross-checking for a large number of titles before loaning out relevant ones).
Call with co-fellow Andreas to discuss the project and challenges in sorting out project aspects, which was extremely helpful.
Proposed and led discussion group on quality control ´Qualitätssicherung – Replikation von inkorrekten Informationen, kritischer Umgang mit Daten und Möglichkeiten der Korrektion bereits etablierter Fehler´ at the Digital Art History Open Space online meeting. Wrote documentation, shared document with other participants from the discussion gopu and wrote blog contribution which was published at arthistoricum.de blog .
Collection and review of printed introduction to art historical studies and academic writing titles, as available and loanabel from the Berlin libraries within Covid restrictions.
Initiated weekly Fellow Coffee Meet-up sessions to further exchange with other fellows. Due to Covid restrictions the 2020/2021 cohort only ever could interact via mail and online meeting, which changes the Fellowship experience quite a bit. To partly offset the lack of personal interaction, the weekly digital hang-outs were proposed. With help from the Wikimedia team, the free open-source video conferencing software Jitsi was chosen. After doodling the weekly hang-outs were set for Friday afternoons.
The very first Fellow hang-out was already extremely productive and inspiring, and longer than the initially set time, and led to some collaboration ideas with co-fellows Florian Thiery and Sophie Schmidt. In the following weeks, a number of direct online meetings with the two Fellows was set up, to discuss my projects aspect of quality control in context of the other Fellows Ogham Stones project. Collaboration is ongoing.
Fellowship Programm Winter School February 26th and 27th. Call with mentor and co-mentee Andreas. Discussed planned paper structure and decided for meeting to present case studies to allow for better discussion. The talk was very helpful to re-focus on the central aspect of the case studies. Wide reading in the months prior had led to considering a large number of angles and aspects which are inspiring but can be overwhelming.
Participation in Barcamp Open Science February 16th, promoted aspect of quality control investigation.
Continued planning paper outline and started writing draft.
Appointment with mentor and co-mentee to present and discuss case studies. Writing on paper draft. The introduction draft was completed and shared with mentor and co-mentee.
Bibliometry discussion meeting March 19th initiated by co-mentee Andreas Pacher with both further and former Fellows and external contributors. Bibliometry is especially interesting as systems used for citation indexing can likely also be utilized for citation line tracing to support better quality control.
Requested additional relevant background metadata information for case study II from Bodleian Library in Oxford. The Bodleian team was very helpful and open to sharing information. In turn this led to a mutual exchange of bibliographic lists and additional information regarding the object which is discussed in the literature at the centre of the case study. The exchange also showed how open non-researcher professionals in the field, who do not have to rely on reputation economy, are to sharing data.
The weekly Fellow Coffee meet-up sessions were continued and proved to be continuously encouraging. The established network, especially with Sophie Schmidt and Florian Thiery, has led to ongoing support for example in the form of proofreading or paper draft discussions.
Together with Fellow Florian Thiery a system to model citation lines in WikiData was devised. This extremely helpful project aspect has led to a new approach and continuous active collaboration with several evening sessions of digital co-working. Florian's support and technical guidance is extremely valuable and the work together mutually encouraging.
To model the citation genealogy in WikiData all bibliographic entities have to be inserted or - where already existing - have to be linked. Every author, every citation, every book, every article and every journal have to be created as an individual item in order to be then connected in the semantic WikiData network. In the context of the case study II for example this means that more than 50 authors have to be added. Some titles have several authors, inflating the work. Other authors already had an existing WikiData page, reducing work. While very time-intensive the work is considered worthwhile. Not only will the data be integrated into an open access system and is made accessible for future use; the work with WikiData itself is a valuable learning experience.
Ongoing work on WikiData insertion including work meetings with Florian Thiery. Ongoing weekly Wiki Fellows Coffe Meet-ups.
Read and gave feedback for possible editing on co-mentee Andreas Pacher's paper. (And his very impressive webpage!)
Discussion with Florian Thiery if collaboration and example of WikiData use for humanities can be presented in a conference or similar.
Participation in vDHd 2021 workshop (Digital Humanities Germany conference section) on April 9th.
Contact with the NFDI4Culture Task Area 2 "Standards, Data Quality and Curation" was established to discuss possible future presentations or thematic discussions in workgroup meetings.
Ongoing work on WikiData insertion including work meetings with Florian Thiery. Ongoing weekly Wiki Fellows Coffe Meet-ups. Co-working sessions with Fellow Sophie Schmidt.
Started conceptualization for co-writing LOD / WikiData focused article with Florian Thiery.
Contact with Zotero Cita plug-in creator Diego de la Hera (supported by a grant from the Wikimedia Foundation). The collaborative exchange between led to concrete revisions of the Cita plug-in and helped to accelerate the Wiki project through the partial mirroring of bibliographical data from Zotero into WikiData.
Final presentation at Fellows Programm concluding session.
Writing, community collaboration and outreach in art history and GLAM institutions communities ongoing.
- Name: Charlotte Oertel
- Institution: IZT – Institut für Zukunftsstudien und Technologiebewertung, Berlin; sowie The Digital Cicognara Library Project, Princeton University und National Gallery of Art, Washington D.C.