Enriching Historic Photography with Structured Data using Image Region Segmentation
Abstract
Cultural institutions such as galleries, libraries, archives and museums continue to make commitments to large scale digitization of collections. An ongoing challenge is how to increase discovery and access through structured data and the semantic web. In this paper we describe a method for using computer vision algorithms that automatically detect regions of “stuff”—such as the sky, water, and roads—to produce rich and accurate structured data triples for describing the content of historic photography. We apply our method to a collection of 1610 documentary photographs produced in the 1930s and 1940 by the FSA-OWI division of the U.S. federal government. Manual verification of the extracted annotations yields an accuracy rate of 97.5%, compared to 70.7% for relations extracted from object detection and 31.5% for automatically generated captions. Our method also produces a rich set of features, providing more unique labels (1170) than either the captions (1040) or object detection (178) methods. We conclude by describing directions for a linguistically-focused ontology of region categories that can better enrich historical image data. Open source code and the extracted metadata from our corpus are made available as external resources.
Document Type
Article
Publication Date
5-2020
Publisher Statement
Copyright © 2020, European Language Resources Association (ELRA), licensed under CC-BY-NC
Proceedings of the 1st International Workshop on Artificial Intelligence for Historical Image Enrichment and Access (AI4HI-2020), pages 1–10 Language Resources and Evaluation Conference (LREC 2020), Marseille, 11–16 May 2020.
Recommended Citation
Arnold, Taylor. “Enriching Historic Photography with Structured Data Using Image Region Segmentation.” Proceedings of the First Artificial Intelligence for Historical Image Enrichment and Access (AI4HI), 2020. https://www.aclweb.org/anthology/2020.ai4hi-1.1.pdf.