(2.1) The Corpus: Portuguese Parish Memories – Alentejo

The Parish Memories-Alentejo (PM-A) corpus is an essential source for obtaining Portugal’s description in 1758–1761. Digitised copies of the original documents are available at Arquivo Nacional da Torre do Tombo.²Figure 1 presents an example of the original manuscript.

The collection corresponds to a survey sent to those responsible for all country dioceses at the beginning of 1758, and then distributed to the parish priests. The Portuguese crown sent the survey to better know the country’s real situation less than three years after the big earthquake of 1755. At that time, it was usual for the sovereign states and some academies to conduct surveys to improve their knowledge about certain territories or topics. Portugal is no exception, and as such, the coverage and great detail of the Parish Memories make it one of the most important surveys of its kind (Chorão, 1987). It contains detailed descriptions of almost each parish of which there were about 4,232 of these religious units in Portugal, in 1798 (Santos, 1995). The parish was the smallest organisational unit in the country at that time.

The survey was organised into three major parts: land, mountain, and river. It included 27 questions about the land, 13 about the mountains, and 20 about the rivers. The parish priests only responded to what suited their territory, and their texts were handwritten. Regarding the land section, the questions included historical, administrative, jurisdictional (ecclesiastical and secular), geographical, and economic topics. It also asked about the impact of the 1755 earthquake in each locality and requested people to describe the reconstruction’s status. Questions about the mountains were asked to obtain descriptions of orography, fountains, medicinal herbs, mines, lagoons, villages, and monasteries. Concerning the river section, the detail was also significant: size and intensity of flow, navigability, the direction of the stream, fish and fishery-related activities, bridges, mills, cultivation of the banks, and many other subjects were asked about. Each of the three parts ended with an invitation to describe what was specific and relevant about each place and was not analysed in the previous topics (Santos et al., 2020a). Some parish priests accepted this last invitation, and others had nothing special to point out.

Transcribed digital versions of these reports are available through CIDEHUS website.³ These transcribed versions constitute a sub-corpus with texts from 366 parishes of today’s Alentejo region, Southern Portugal’s largest administrative area. The PM-A represents 90% of the total of Alentejo’s parishes in 1758.

In these originally handwritten texts, the orthography variations are remarkable, as there was no standardisation at that time, in the sense that there was no uniform spelling that everyone followed (Mateus & Cardeira, 2007). This classical Portuguese language period is, in fact, characterised by significant variation in orthography (Gonçalves, 2003). Nasal diphthongs have many spellings, and the use of double consonants and pseudo-etymological spelling were frequent. The sibilants register is also an excellent example of variation, among others (Kemmler, 2001).

In the PM-A corpus, these phenomena are pervasive and illustrate the orthographic variation: ‘tãobém/também’ [also], ‘administrassam/administração’ [administration], ‘officiais/oficiaes’ [oficial], ‘parocho/paroco’ [priest], ‘oitosentos/oitocentos’ [800], and ‘freguesia/freguezia’ [parish]. The variation is sometimes difficult to predict, as we can find many spelling variants of a single word, like in ‘noticcia/noticia/notiçia/notticia’ [news]. In this period of the history of the Portuguese language, there was a remarkable renewal in the lexicon; many new words entered the written language (Verdelho, 1987), enlarging the possibilities of variation in words.

Due to the large number of texts (one for each of the 366 parishes), the original documents were transcribed by a group of people, including undergraduate students and research fellows, over a period of 12 years (2008–2020). It also comprises transcriptions previously published by other authors outside our research group. The transcriptions were collected as MS Word documents. Although not having a significant impact on the historical research objectives, these different origins reflect distinct, and sometimes contrasting, transcription criteria. The resulting variation is a challenge for the automatic processing of classical Portuguese as most existing tools operate in the contemporary Portuguese language (Cameron et al., 2020). In addition, uppercase and lowercase letters were used randomly in the original documents, with some transcribers keeping the randomness and others interpreting and updating it to current Portuguese. This constitutes a particular challenge for NER, since uppercase is usually a strong indicator. Also, the syntax and punctuation marks do not precisely follow the contemporary usage and conventions, which constitutes another challenge.

(2.1.1) Annotated Entities in the Parish Memories

Named entities recognised by most NER systems are generally of the types PER, LOC, and ORG. These entities are also essential topics for historians as they are for NLP in general. They structure a necessary part of the historian’s inquiry, answering basic questions: Who? Where? Which institution? In the past, everything happened in a specific place, was developed or executed almost always by someone, even if it is not known by whom, and organisations/institutions often framed the actions. Figure 2 shows examples of each category occurring in the PM-A.

Figure 2 

Examples of named entities in the PM-A.

(2.2) The NER Systems

There are many software options regarding the NER task (Schmitt et al., 2019). We evaluated two NER systems: the Portuguese NER model available at the spaCy framework and a Portuguese NER model, based on Flair contextual embeddings assessed in several domains (Santos et al., 2019a). The former was developed to consider industrial needs, and the latter is an academic product that has shown the best results in several evaluations. The intention was to choose the NER system with the best performance for these classical Portuguese texts. Both spaCy and Flair frameworks provide their own tokenisation.

(2.3) Manual Annotation

We performed manual annotation in a part of the PM-A corpus. We chose one of the large texts, aiming for a good number of examples for each class. Our choice of text was considered adequate, since the smallest number of examples in a class is 368 (as presented in Table 1). We followed the guidelines proposed by the HAREM project (Santos & Cardoso, 2007). We considered only three main categories, as previously explained. We made no distinctions between different sub-types of mentions. For instance, in HAREM, the PER category was also marked for sub-types regarding its kind as being an individual or an occupation, as in Carlos I or Sua Majestade (The King), respectively. For performing the manual annotation, two annotators, historians, used the INCEpTION open-source tool (Klie et al., 2018).⁷

Table 1 presents the number of manually annotated instances for each class. The two annotators agreement was measured based on Kappa statistics, usually employed for measuring annotation quality (McHugh, 2012), using the script provided by Python’s sklearn (Pedregosa et al., 2011) library.⁸ The resulting Kappa was 0.71. Although higher agreement has been achieved for this type of annotation task, for this work we considered it satisfactory since our main goal was not to deliver a gold standard, but instead, to provide the automatic annotation of the collection. Another reason for accepting this agreement level is that the range 0.61–0.80 is, in general, considered as substantial agreement.

(4.1) Description of the dataset

The dataset, Parish Memories with named entities,¹² consists of 366 transcribed texts from the original handwritten collection, where each text contains the description of a parish. It amounts to approximately 650,000 word tokens and 35,000 word types. These texts are annotated with entities PER, LOC, ORG. The dataset also contains the list of the extracted entities for each text and a global list with all entities from the collection. All these lists have frequency counts of terms in each category, number of mentions in each text. The transcribed texts with no annotations are also included. We performed the annotation automatically, and the agreement of the system with the manual annotation was around 0.65, whereas the agreement between human annotators was 0.71. The manually annotated text used in the evaluation and the corresponding list of entities are provided in the dataset.

In total, the collection has 13,600 person mentions, 23,511 location mentions, and 1,321 organisation mentions. The total number of automatically identified mentions is 38,432. The annotation is given in two IOB formats: (i) CoNLL, which lists each word/token with both positive and negative tags, and (ii) the in-text tags in which positive tags follow the respective words. In both formats (B-CAT) identifies the first word of an entity belonging to category CAT and (I-CAT) identifies the continuation of an entity, as exemplified in Table 6. The dataset provides the manually annotated texts, used for evaluation purposes, in the CoNLL format. A better visualisation interface of the annotated texts can be obtained with the INCEpTION tool, as shown in Figure 3.

edificada O
na O
Estremadura B-LOC
, O
e O
vizinhancas O
de O
Castella B-LOC
, O
em O
distancia O
de O
huma O
legoa O
, O
na O
ProvinciaB-LOC
do I-LOC
Alenteio I-LOC

… edificada na Estremadura <B-LOC> , e
vizinhancas de Castella<B-LOC> , em distancia
de huma legoa , na Provincia <B-LOC> do <I-LOC>
Alenteio <I-LOC>

Table 7 shows examples of automatically extracted entities for each category, while Table 8 illustrates some of the most frequent expressions referring to entities. For counting the global frequencies, we normalised words to lowercase and removed diacritics. However, there are other spelling variant issues to deal with for further statistical analysis (e.g., matris/matriz).

In terms of using this type of resulting information, we can say that these frequencies confirm the impact of the two most prominent characters in Ancien Régime society: God and the King. Regarding LOC entities, the domination of “villa” is very expressive and the main towns of the South are also present as well as Lisbon, since the distance from each village to the Court is mentioned in every text. Addressing the organisations, “villa” is also dominant. The disambiguation among these two uses of the term is indeed a challenging factor for the annotations.

(4.2) Applications

Available in an open-access format, this corpus provides both the list of extracted entities and the entities annotated in-text, along with its original textual context. Keeping context is essential for historians to decode the precise meaning of a word and to assess the relevance of historical data, and this approach allows us to preserve it.

Entities, once identified, can be linked to other information, and texts can be reused by other researchers and projects. When we tag historical sources, enabling better document retrieval and information extraction, we improve the capacity of analysis. It is possible to find correlations between entities faster than manually or even to find new ones that would not be found otherwise. In this way, when we extract named entities from this kind of corpora, we can cross this information with other datasets from the same period, working with a larger number of sources. For instance, the PER category can be linked to prosopographical data, making it easier to identify biographical aspects, like birth or burial places, among others. This type of approach allows us to extract data to compare and carry out different analyses.

Regarding the connection with geographic information systems, although we are not aware of the exact borders of all the freguesias in the corpus, we know the exact location of the most important villages and the locations of some points of interest referred to in the texts (e.g., archaeological ruins). By mapping this data with artificial grids (lattices) (Birch et al., 2007) for representing the regions geographically, we can compare the parishes with each other to verify the institutions’ homogeneity in the various geographical areas. We can also find the most valued institutions of that time through the frequency of citations in certain regions, clarifying the profile: ecclesiastic, secular, or mixed. Inspired by central-place theory (Christaller & Baskin, 1966; Romão, 2019) that measures the influence area of a geographical point, we can also verify the area of influence of a parish, organisation, or person by measuring the distance over where it is mentioned in other Parish Memories. These are only a few of the many possibilities of research that can be developed through this corpus.

We acknowledge that the data, being entirely automatically produced, is far from perfect, especially for the class ORG, which is rather incomplete. The idea is to provide some input, even if there is missing information; the findings brought about by the extraction may serve as partial evidence when seen through the lens of the historians. For instance, even for the case of ORG, which is low in recall, by looking at the data we can observe the prevalence of churches, councils, counties, brotherhoods, hospitals, and parishes. The material made available can be further improved by other research groups interested in either the material themselves or in the technical challenges for improving it. This first release serves as silver data to train other NER systems that may be more apt for texts from the classical period.