Next Phase of Archiving: Data Together

EDGI has been working diligently to plan for the future based on overwhelming and ongoing interest in data archiving and engagement with government data!

Thanks to everyone who participated in the DataRescue Townhall on April 1, and also to the event organizers, attendees, and members of librarian, archival, and environmental advocacy communities who provided feedback over the past two months! It’s been inspirational to travel this path together, and your insights are invaluable. Now that we’ve made it past the initial period of potential immediate data loss, EDGI members have used your reflections to thoroughly review the last six months of archiving processes and events, in order to look ahead to what’s next.

A more comprehensive EDGI archiving reflection document is being written, and we’re excited to share it as soon as it’s wrapped, for now Here’s the latest:

  • The End of Term Harvest has concluded! We recommend phasing out seeding activities
    As we shift into longer-term archiving, the existing seeding workflow no longer meets the current needs for effectively preserving and interacting with archived data. The tools and documentation for the previous event workflow are Creative Commons licensed and will remain available. however, they will no longer be maintained and updated after July 5, 2017.
  • What is needed now is for people to provide context about datasets, we are introducing a new approach for describing metadata
    One common challenge with the data archiving events was a high bar for entry because of the comfort and familiarity with tools and technical skills that scraping and downloading required. To address this, we’re working on a new approach that provides opportunities for engaging with data & metadata rather than solely emphasizing the technical details of harvesting data.
  • We are partnering on building “Data Together” with Protocol Labs and
    This new approach to archiving is undergoing active development. We expect to prototype, test, and present on it throughout the summer in hopes for a September “launch”! We will need help!
  • We see this operating through local working groups that meet regularly
  • One-off events are not sufficient to address large-scale archiving, or to allow for unpacking the complex and ongoing considerations that go into this form of preservation. We’re really excited to help people set up working groups and provide support for the first meetings to get everything started, but ultimately we hope for these efforts to turn into activities driven by local communities.

Join #data-together in the Archivers slack to follow the conversation, or email at (learn more about sending secure email to us) if you are interested in working with us to test out this new approach to archiving.