1 Overview

Repository location

10.5281/zenodo.4315208

Context

“d-Prose 1870–1920” was created as part of the project “Gender and Illness” in the cooperation project “Automated modelling of hermeneutic processes – The use of annotation in social research and the humanities for analyses on health”. The focus of the project was on the description of illness from a gender perspective. The research focused on how the presentation, perception, and dealing with illness differ among characters depending on their gender.

2 Method

Steps

The text files were taken from the KOLIMO corpus (), which in turn is based on the repositories TextGrid, Deutsches Textarchiv, and Gutenberg-de. The KOLIMOtoText tool () served as the basis for the corpus creation, extracting all texts in the KOLIMO corpus published between 1870 and 1920. After this, a manual check was performed in order to exclude all non-prose texts as well as all texts that have not been originally published in German (i.e., translations). Since the received text collection contained duplicate texts due to the occurrence of different editions with different titles, duplicate texts were removed in a further step using the author-name-title comparison program ANTComp () and the full-text comparison program BatchSED (). All texts were manually cleaned of paratexts (i.e., author names, dedications, prefaces, etc.) and supplemented by metadata: author’s name and pseudonyms, author’s gender, author’s date of birth and death, title of the work, repository source, file name, number of words and types, as well as publication date (extracted from the metadata of the original repositories, checked and if necessary corrected or extended by data from the literary encyclopedias Killy () and Kindler ()).

Sampling strategy

In order to cover a broad variety of phenomena in literary prose texts, a certain degree of heterogeneity in form and content was aimed for in the sampling of the corpus. Beyond that, there were no content-related restrictions of the text selection. Thus, criteria for text selection were only date of first publication, text language, genre, and text length (for a more detailed discussion of sampling criteria, see ). As a result, the corpus includes the works of 334 authors from at least three different literary movements (i.e., naturalism, realism, and modernism). It contains approximately equal proportions of long texts (novels) as well as shorter prose forms (cf. Table 1).

Table 1

The average text size (in words and types) was calculated by Voyant Tools (). The statistical data refer to the content of version 2.0 of the corpus “d-Prose 1870–1920”.


number of texts2511

number of texts written by female authors346

number of texts written by male authors2165

number of authors334

number of female authors72

number of male authors262

text size average31146 words; 4753 types

standard deviation58117 words; 5323 types

shortest text1006 words

longest text990351 words

number of texts per decade1870–1879226

1880–1889327

1890–1899542

1900–1909623

1910–1920793

Quality control

The steps of automatic processing described above (, , ) were evaluated by manual control of random samples of the output. This lead to an iterative improvement of the results. For the de-duplication process () a manual evaluation of all duplicate pairs identified by the process (i.e., about 1,000 text pairs) was performed (). After the automated cleaning, a manual cleaning was performed. All texts were manually cleaned of paratexts such as author name, author biography, dedication, preface, remarks, etc. in a collaborative approach. In this approach, the data curators worked in a review process so that each text was double-checked. The same procedure was used for enrichment with metadata. A first data curator added the respective meta information and a second data curator reviewed the entered information.

3 Dataset Description

Object name

d-Prose 1870-1920

Format names and versions

plain txt-files (UTF-8); spread sheet (xslx) with metadata. Version 2.0

Creation dates

2017-05-01 – 2021-06-22

Dataset creators

Adelmann, Benedikt (Developer, University of Hamburg); Gius, Evelyn (Conceptualization, Project administrator, Supervisor, Technical University of Darmstadt); Guhr, Svenja (Data curator, Supervisor, Technical University of Darmstadt); Kurz, Laura (Data curator, Technical University of Darmstadt); Otte, Felicitas (Data curator, University of Hamburg); Schlesiger, Nicole (Data curator, Technical University of Darmstadt); Schreiber, Annekea (Data curator, Technical University of Darmstadt); Sökefeld, Carla (Data curator, University of Hamburg); Krüger, Katharina (Project member, University of Hamburg); Murawska, Anna Aline (Project member, University of Hamburg); Uglanova, Inna (Validation, Application, Technical University of Darmstadt).

Language

German

License

Creative Commons Attribution Non Commercial Share Alike 4.0 International

Repository name

Zenodo.org

Publication date

2020-12-15 (2021-06-22 (V.2.0))

4 Reuse Potential

“d-Prose 1870–1920” is primarily of interest to literary scholars and linguists, as well as to scholars involved in the modelling of textual and cultural phenomena. The corpus can be used to test literary, literary-historical, cultural and linguistic hypotheses. It may be especially helpful for the study of the literary developments and artistic movements of the represented period of time. The corpus has sufficient volume to allow for the application of machine learning techniques like clustering, classification, or topic modelling (see , for an example based on topic modelling). It can be used as analysis material for didactic purposes too, such as developing core knowledge and skills for working with a literary corpus (studying patterns of usage, historical dynamics, typical and unique contexts of language or literary phenomena, author’s neologisms, collocates across metadata, etc.). The dataset can be extended and included as a sub-corpus for more global tasks. Due to the fact that the corpus is heterogeneous in its structure, it can be divided into smaller subcorpora tailored to specific research objectives. In particular, subcorpora can be created on the basis of publication year, gender of the author, size (short stories, novellas, novels), or, with some additional work on/with the metadata by genre (historical novel, adventure novel, social novel, etc.), literary movements (realism, modernism, naturalism), or other criteria. For more special tasks, the corpus can be pre-processed with a linguistic analysis pipeline, specifically developed for this dataset by Adelmann (). The pipeline is optimized for the corpus and provides tokenization, part-of-speech and morphological tagging, lemmatization, and dependency parsing. It is accompanied by detailed instructions that can easily be used by users with minimal technical knowledge. The data format makes it easy to use linguistic, web-based technology without further extraction and conversion. Additionally it can be used with popular GUI-based open-source software for data mining and analysis such as Voyant Tools (), the manual annotation tool CATMA () the concordance program AntConc (), and others.