2.1 Summary

Below, we provide a brief summary of metadata for CuneiML:

Object name CuneiML_v1.0.tar.gz, tran.

Format names and versions JPEG and JSON.

Creation dates 2023-09-01

Dataset creators Danlu Chen, Aditi Agarwal, Taylor Berg-Kirkpatrick, Jacobo Myerston.

Language Sumerian and Akkadian.

License CC BY-NC 4.0.

Repository name https://doi.org/10.5281/zenodo.8307503

Publication date 2023-09-01

3.1 Downloading the metadata from CDLI

We used the CDLI Github repository¹ to get the public catalog data containing a list of P-numbers for all tablets. “P-number” is a 6 digit unique identifier prefixed with the letter P, used to uniquely identify a tablet by the CDLI initiative. Using the P-number for every tablet we then crawl 2D images,² lineart images³ and metadata⁴ including transliteration from CDLI. We gathered a total of 133,923 tablets from CDLI, of which 56,694 come with a 2D scan image and 52,637 come with a lineart image.

3.2 Data sampling and filtering

Most 2D photographs from CDLI are high-resolution color images showing six faces of a single tablet. Since our goal is to create a dataset for training and evaluating machine learning systems, consistency is paramount: if non-standard examples are included (e.g. black and white images when the majority of the dataset is in full color), machine learning systems may learn to leverage these features as false predictors (e.g. if most black and white images tend to come from the same period due to how they were collected, a classifier will learn to depend on this spurious correlation). Thus, we filtered out non-standard and low-quality and images according to several conditions:

• The image is black-and-white.
• The resolution of image is lower than 100*100 px.
• The tablet is in poor condition, e.g. too many fragments or barely readable cuneiform.
• The image does not contain a well-defined major face.

We also filtered out artifact type such as cone, cylinder and prism and only keep entries whose artifact type is tablet. After this processing, we eventually have 38, 947 tablets with high quality 2D images.

3.3 Cutting out the major faces

An additional issue for machine learning development arises from the variability of the imaging setup used to capture the raw composite photographs from CDLI. While the majority of composite photographs consist of six tablet faces, arranged in a fixed layout from a consistent camera angle, these properties vary to some extent based on when and where imaging was performed. In order to increase consistency, and therefore reduce the danger of overfitting to false correlations that are dependent on the imaging process itself, we systematically extract cutouts of the major tablet face in each composite and include this as an additional, more consistent, photographic representation for machine learning systems. Specifically, we build and test three different computer vision methods to segment and obtain individual faces of each tablet. As a way of validating and producing final extractions, we reconcile differences between the bounding boxes each system produces by computing the area of their overlap. When the area is large, the methods are in agreement and our output is reliably high-quality. The three methods are described briefly as follows.

1. Connected component segmentation. We first convert the images into black and white where the background is black. This is a classical rule-based segmentation that clusters the adjacent pixels with the same color. TWe use OpenCV’s implementation cv.connenctedComponents().
2. Watershed segmentation. We first convert the images to grayscale. The watershed algorithm views a greyscale image as a topographic surface where high intensities denote hills while low intensities denote a valleys. Each valley is labelled with a different color of water. As the water rises, unknown pixels will be colored and therefore clustered. We use OpenCV’s implementation cv.watershed().
3. SegmentAnything. This is the state-of-the-art general segmentation algorithm using neural networks. We uses the official toolkit⁵ (Kirillov et al., 2023) with default model weights to obtain cutouts.

Quality checking We automatically cut the images using three methods and only keep tablet cutouts whose overlapping area is larger than 90%. We sampled 100 images randomly to validate the cutouts; 97% met our quality requirements. Figure 3 shows a sample of 20 major cutouts produced by our algorithm.

3.4 Converting transliteration to cuneiform Unicode

CDLI and ORACC offer transliterations of cuneiform tablets, which are sometimes but not always accompanied by their photographs and linearts. The transliteration standard used in both projects is called ATF and is explained in detail in the ORACC’s website.⁶ Transliteration is the process of transcribing cuneiform signs into the Latin alphabet, using conventions which have varied over time and can take particular shapes according to various Assyriological projects. But, despite its possible inconsistencies, transliteration has played a crucial role in making Akkadian and Sumerian more accessible to the non-specialist; it has also facilitated the creation of dictionaries and critical editions. From the point of view of data processing, transliteration is also important because it reveals how modern scholars read certain signs that allow multiple interpretations. In this sense, transliteration is a form of disambiguation. Take for example the sign that can read as “sky” or “god” but can also be interpreted as the syllables an or il. This issue of interpretation occurs with many cuneiform signs that need to be disambiguated so that modern editors can stabilize what seems to them the most plausible reading of a text. A final example may serve to further illustrate this point. In the well-known Epic of Creation or Enūma eliš, the mother of the gods’ name is often spelled with the signs TI and GÉME, a combination of signs that is usually transliterated as Ti-amat and rendered into English as Tiamat. Now, this transliteration somewhat conceals that the goddess is the Sea, something that can be expressed more directly if one transliterates TI GÉME as ti-amtu, the “sea” in Akkadian. Thus, transliteration implies a reduction of possible choices which were present for an ancient audience, but which are concealed to modern readers that use latinized editions of cuneiform texts.

One possible issue if we use the transliteration directly for machine learning, is circular reasoning. Given that the transliteration itself might exhibit bias towards specific time periods and other attributes – e.g., an expert’s approach to transliterating a tablet is already influenced by preconceived notions about its time period. Thus, as an additional layer in our dataset, we produce and provide cuneiform Unicode conversions of the original Latin transliterations. Specifically, we follow the ATF convention to remove some of the editorial marks, tokenize, and map the transliteration into machine-readable cuneiform Unicode format. We use cuneifyplus⁷ to map the Latin transliteration to cuneiform Unicode. If a latinized sign is not processed, we then query eBL’s sign list⁸ to obtain the cuneiform Unicode. We briefly describe the rules here.

1. Uncertainty. The query (?) placed after a grapheme indicates uncertainty and the asterisk (*) indicates a collated reading. We remove the marks but keep the grapheme by default.
2. Breakage. The $ sign represents breakage, sometimes also indicating how many lines are broken. If the number is recorded, we insert the same number of <LB>. E.g. 2 lines broken → <BREAK><LB><BREAK><LB>. Moreover, the annotation […] indicates missing signs. We also insert a special token <BREAK> to indicate the missing content.
3. Compound words. We remove the markers of compound words. E.g. |SU.KUR| → su-kur.
4. Reading. sudx(|SU.KUR|) means the reading is sudx, while the signs are su-kur; we remove the reading and only keep the actual signs for tokenization.

Unimodal tasks

• Language modeling. One of the most popular and broadly useful machine learning applications is to train a language model on text in a given domain. Language models trained on our dataset could be used to encode cuneiform Unicode for further processing and analysis, or as a generative prior in related downstream tasks like transcription and restoration (Assael et al., 2022; Lazar et al., 2021).
• Transliteration. As mentioned above, there are multiple ways to transliterate the same cuneiform sign sequence. Gordin et al. (2020) proposed several models, inculding HMMs and LSTMs, to automatically transliterate and segment Unicode cuneiform glyphs. Our dataset could be used as further training or validation data for this task.
• Lineart generation. Analogously, in the image modality, there are potential use cases for automatically “translating” photographic representations into lineart, which potentially increases the readability of tablets to scholars. This task is structurally similar to image generation tasks in the broader field of computer vision. Following similar techniques, CuneiML could be used to train neural models (Isola, Zhu, Zhou, & Efros, 2017; Rombach, Blattmann, Lorenz, Esser, & Ommer, 2022) capable of accurate lineart generation conditioned on a tablet image.

Multi-modal tasks

• Attribute prediction. Our dataset supports training and evaluating classifiers for predicting metadata based on images or lineart. The attributes in the metadata include geographical, genre, and chronological attribution (Bogacz & Mara, 2020).
• Sign identification / automatic transcription. A useful, but particularly challenging task that our data supports is automatic transcription of tablet images or lineart into cuneiform Unicode text. There is very little text line annotation data for cuneiform tablets. Even with line-level annotations, the task is much harder than documents written on paper. Recently, new page-level end-to-end OCR systems (Coquenet Chatelain, & Paquet, 2023) have been developed that are capable of high-accuracy transcription of more modern languages without line-level annotation. The lineart-cuneiform Unicode parallel data presented in our dataset is an ideal testbed for extending these techniques to more ancient languages.

In the following section, we present preliminary results on attribute prediction tasks using major face images, cuneiform Unicode and Latin transliteration in order to demonstrate a specific use case of our dataset for machine learning.

Result and analysis

Table 2 shows the result of attribute prediction using three different type of input as features. We can see that the image baseline model achieves reasonable accuracy on all three types of attributes. The preliminary results demonstrate the effectiveness of our data for training machine learning systems that make predictions based in tablet images and further underscore the difficulty of provenience and genre attribution. Note that the major face cutout features seem to have the overall best performance, but it is possible that lighting and camera configurations influence the classification of a tablet. Furthermore, label imbalance and the distribution shift between the training and testing sets remain significant challenges in cuneiML.

Further research and analysis are necessary to assess the reliability of the predicted results.