Digitization projects in the public sector: Building future-ready content with open tools

Public-sector digitization projects often begin under challenging conditions: budgets are limited, and customers do not always have in-house expertise in digitization, archiving, or scholarly data processing. At the same time, large collections of information must be made accessible, structured, and usable over the long term.

How can a project like this succeed? We spoke with our colleague Frank Ralf, who supported a project to digitize historical cabinet meeting minutes. We discussed working within a clearly defined budget, selecting the right tools, and enabling organizations to continue the work independently once the initial implementation is complete.

The project's technical implementation and results are described in our case study on the digitization of cabinet meeting minutes. This article focuses on the practical experience and challenges encountered throughout the project.

How archive digitization projects begin – and what is often unclear

Frank, how did the project come about, and what was your first impression?

The project began when the State Archive was faced with the challenge of not only digitizing the cabinet meeting minutes but also transforming them into a resource that would remain usable in the long term. The goal was not simply to create scans, but to produce structured, searchable content.

My first thought was that this would be a fascinating project—but also one that could become highly complex very quickly. With the digitization of historical documents, you never know exactly what challenges will emerge until you start working with the material.

What was already clear at the beginning, and what remained uncertain?

The overall vision was clear from the outset: to create a digital solution that offered significantly more value than the existing printed volumes, including improved search capabilities, cross-references, and structured content.

What remained uncertain was how well the existing material could actually be structured. We didn't yet know how consistent the records were or how much manual work would ultimately be required.

Between planning and reality: Typical challenges in digitization projects

Under what conditions was the project carried out?

The project had to be completed within a clearly defined budget. The archive team contributed extensive knowledge of the collection and its scholarly indexing. In addition, specialized expertise in XML, digital editing environments, and publishing workflows was required.

Our task was therefore to combine the archive team's subject-matter expertise with an appropriate technical solution. This included selecting suitable tools, designing efficient workflows, and providing targeted knowledge transfer so that the archive team could continue maintaining the digital edition independently after the project had ended.

What were the biggest practical challenges when working with the material?

The collection had evolved over many decades and naturally contained numerous inconsistencies. There were different formats, varying writing conventions, and occasional structural discontinuities.

This meant we had to examine the material very carefully rather than simply automating every step. Quality assurance and traceability were essential throughout the project.

How did you deal with the project's budget constraints?

We first looked together at the resources that were actually available and determined what could realistically be achieved within those limits. The key was to use the available tools as efficiently as possible.

At the same time, we designed the solution so that the archive team would be able to carry out as many tasks as possible independently going forward.

How did you choose the tools for the project?

The tools had to fit the available budget, the project's requirements, and the users' workflows. For that reason, we deliberately chose open-source software and open standards. This avoided licensing costs and prevented long-term dependence on a single vendor.

We used the open-source editing environment ediarum, which was specifically developed for scholarly digital editions. Content was structured using TEI XML. We also configured a project-specific authoring environment in Oxygen XML Editor. From Oxygen, the content could be exported as HTML and then processed further and published in WordPress.

How did you approach knowledge transfer?

Providing a technical solution was only part of the project. Knowledge transfer was equally important.

Together with the archive team, we developed transparent workflows and provided the technical knowledge needed to continue the work independently. The training covered XML and TEI, Oxygen, ediarum, and the export and publishing processes. This allowed the team's existing archival expertise to be complemented by the technical skills required for maintaining the digital edition.

For us, open source, open tooling, and knowledge transfer were inseparable parts of the overall solution.

What was particularly important for successful collaboration?

Above all, the project required close cooperation and a shared understanding of the overall goal. It was not enough for everyone to complete only their own tasks independently. The team needed to communicate regularly and make decisions together.

Flexibility was equally important. In projects like this, requirements and assumptions inevitably evolve over time, and the team must be able to respond without losing momentum.

Challenges in digitizing historical content

Which aspects of the project proved to be the most demanding?

The analysis and structuring of the material required the greatest effort. Many people initially focus on the technical implementation, but in reality the biggest challenge is often understanding the content and preparing it in a meaningful way. With historical documents in particular, this is a critical part of the work.

What does digitization actually mean in this project?

Digitization is about much more than transferring existing documents into a digital format. The content is captured in a structured way and enriched with semantic markup. This makes it possible to standardize the information, reference it consistently, search it efficiently, and reuse it in other contexts.

What opportunities do structured content provide?

Structured content makes it possible to identify and relate people, places, institutions, and sources consistently. This creates opportunities to enrich the content over time—for example by linking it to photographs, authority records, or external resources such as Wikipedia.

The result is not simply a digital copy of historical documents but a sustainable foundation for future research and reuse.

What did you personally take away from this project?

The project reinforced how important it is to develop technical solutions in close alignment with the content itself. Even the best technology has limited value if it does not match the specific characteristics and requirements of the material.

It also demonstrated that projects like this are most successful when subject-matter experts and technical specialists work closely together and jointly develop practical solutions.

Learn more about the project's implementation and results in our case study on the digitization of cabinet meeting minutes.

Explore the online portal for the cabinet meeting minutes (in German).

Are you planning a digitization project or looking to make historical collections sustainably accessible in digital form? Get in touch with us to discuss suitable strategies, tools, and long-term workflows.