Publications – NDSA Blog

2023 NDSA Storage Survey Report Published

The NDSA is pleased to announce the release of the 2023 Storage Infrastructure Survey Report, available at https://doi.org/10.17605/OSF.IO/9QP4W

From October 24 to November 22, 2023, the 2023 NDSA Storage Infrastructure Survey Working Group conducted a 51-question survey designed to gather information on the technologies and practices used in preservation storage infrastructure.

This effort builds upon three previous surveys, conducted in 2011, 2013, and 2019. The survey encouraged responses from NDSA and non-DSA members to gain a broader understanding of storage practices within the digital preservation community. The survey received 138 complete responses, with most coming from the United States, but it did have a global reach. The 2023 survey also incorporated two new questions on storage and environmental impact.

Some major takeaways from the report include:

The amount of preservation storage required for all managed copies appeared to stabilize relative to previous surveys. Fewer organizations reported higher allocations of storage, but the anticipated need for storage over the next three years remains elevated.
Only 28% of respondents currently participate in a cooperative system – down from 45% in 2019 – and 63% indicate they are not considering a distributed storage cooperative. The use of commercial cloud storage providers rose from 46% in 2019 to 55% in 2023.
Heavy use of an onsite storage element was reported by academic institutions (91%), archives (88%), and government agencies (71%). It also shows that use of onsite storage is most often combined with use of either independently managed offsite storage or commercial cloud storage managed by the organization.
The leading offsite storage provider used by 56% of the responding academic institutions is Amazon Web Services. For responding archives, Amazon Web Services (36%) and Preservica (21%) are the most prevalent. Non-profits, museums, historical societies and public libraries use Amazon Web Services 45% of the time.
52% of respondents said their organization is considering their environmental impact during storage planning.

The proposed schedule for the Storage Infrastructure Survey to be conducted is every three years, allowing for ongoing tracking and analysis of approaches to preservation storage over time. The next planned Storage Infrastructure goup is scheduled to kick off in 2026. Interested in participating in the next Storage Infrastructure Working group? A call for group members will go out in late 2025 or early 2026.

~ NDSA 2023 Storage Infrastructure Survey Working Group

Results of the 2022 NDSA Web Archiving Survey Report Now Available

The Web Archiving Survey Working Group is excited to announce the publication of the results of the 2022 Web Archiving Survey. The 2022 survey builds upon surveys previously conducted in 2017, 2016, 2013, and 2011 — and though previous surveys were focused on the United States, the 2022 survey was open to international audiences as well. The 26-question survey was completed by 190 respondents over a 10-day period. This report details the outcomes of the web archiving survey that was distributed to various local, national, and international professional organizations and topical groups in October 2022. Topics discussed included archiving policies, tools and services, and access and discovery.

Some major takeaways from the report include:

Most respondents indicated that “staff capacity” was the biggest barrier to web archiving. Few organizations dedicate more than one, full-time employee to web archiving and very rarely is someone’s entire position dedicated to web archiving.
Since the first Web Archiving Survey in 2011, the landscape of web archiving has dramatically changed, and the 2022 survey results show an increase in web archiving practices by grassroots organizations, volunteer groups, and non-academic communities who seek to document their lived experiences.
An overwhelming majority of respondents indicated concerns about their ability to collect social media—in particular, Twitter, Instagram, Facebook, and Reddit. Content housed within social networks has always been difficult to capture for a myriad of reasons and recent changes to numerous social platforms have made this task harder.

Thank you to all NDSA members and others who participated in the survey. We appreciate your time and effort spent in providing the information to us.

Thank you to the members of the Web Archiving Survey Working Group who worked over the last year and half to make the report possible.

Good Migrations: A Checklist for Migrating Your Digital Preservation Infrastructure

My, how time flies. About half a decade ago, in 2015, I became one of the NDSA Infrastructure Interest Group co-chairs. This was part of the NDSA transition from being hosted by the Library of Congress to a new host, which would eventually be CLIR/DLF. Earlier that year, the Infrastructure Working Group (as it was known back then) drafted a document designed to be a checklist for system migrations. The draft was put out to the community for comments, but then, as the influence of the Library of Congress faded into the background during the move from one host to another, so did the main impetus for publishing the checklist. Ah, the sweet irony of a migration checklist being forgotten during a migration!

However, the checklist was not completely forgotten: it existed as a potential project in my head, one that I was interested in completing once other projects wrapped up. Yet the years passed and there were always other projects. Finally, in the summer of 2020, two events occurred: several significant projects finally did conclude and Nathan Tallman, an NDSA Coordinating Committee member and former co-chair of the Infrastructure Interest Group, reached out to me asking about the status of the Migration Checklist. “It’s time,” I thought. And it was.

With that preface, I now introduce to you, “Good Migrations: A Checklist for Migrating Your Digital Preservation Infrastructure.” This checklist has changed significantly from its 2015 version due to the incredible amount of community feedback received in 2020. The 2020 Good Migrations Working Group; which consisted of myself, Nathan Tallman, Paige Walker, Linda Tadic and Matt Schultz; not only incorporated the community response, but also placed the checklist into a Levels of Preservation framework in order to better highlight how migration steps match preservation areas. Over the past six months, we have worked diligently to clarify scope and provide a comprehensive yet still succinct checklist that will hopefully aid those who are tackling a migration process.

NDSA is interested in feedback from anyone who is using the Good Migrations checklist. Based on the level of community interest, NDSA may consider a followup working group to gather illustrative use cases to accompany the document and document methods for accomplishing checklist items. Please let us know if you use the Checklist and have found it to be of assistance.

~Sibyl Schaefer, Chair of the Good Migrations Working Group