Skip to content

Versioning

Proper data versioning matters in the context of our commitment to open science and adherence to the FAIR Data Principles. Versioning is important for reproducibility, citability, and helps improve transparency, which further builds trust in the science that we do. It is recommended practice to assign a version to your dataset or research output, even when you do not anticipate making changes to the resource or releasing additional versions, or when you intend to keep the output private.

Include a version number when it is possible that changes are made to the underlying dataset or output, and when either the older version of the dataset or research output remains accessible to users, or changes are documented in a Changelog file. We recommend defaulting to version 1.0 for the initial publication of a resource.

Version number components

Version number components are usually defined as MAJOR.MINOR.PATCH. We suggest using Semantic Versioning (e.g., 1.2.0), where 1 is the major version element, 2 is the minor version element, and 0 is the patch element. Major version element increments are made when changes affect the whole dataset or output, such as data reprocessing or updates to the sampling methodology. Minor or patch version element increments are made when smaller updates are made to the dataset, such as minor error updates, documentation updates, or in time series updates.

When versions are incremented, it is critical that previous versions of the dataset remain accessible, or that users can understand what changes were made between different versions.

Note

Following the DataCite recommendations, we recommend generating a new metadata record and assigning a new DOI when there is a new major version of the resource. You can link versions of a resource as 'Related Works' (e.g., IsNewVersionOf / IsPreviousVersionOf). See for example: Kelp Canopy Extent 2012 - NW Calvert Island.

Versioning is specific to the data product, not the metadata record. The metadata record simply reports the current version of the underlying data product; it does not carry its own independent version number. Changes to the metadata do not require an update on the version.

The recommended citation in the Hakai Catalogue can be updated and should always reflect the recommended citation for the latest version.

Versioning Provisional, Continuous, and Research-ready Data

The Hakai Institute makes a wide range of datasets available, including provisional, continuous, and research-ready data. Provisional data is data that has not undergone full quality control, and is often added incrementally. Continuous, or near real-time, data is updated much more frequently, but has often also not undergone much quality control. Research-ready datasets on the other hand are often published to support manuscripts, or have been thoroughly QC'd and are considered of high scientific quality. Some considerations when versioning these datasets are outlined below.

Provisional and Continuous Datasets

  • Provisional and continuous datasets are often updated much more frequently, and are made available with certain caveats and limitations. As such, a version number is not required. Instead, when citing the resource, users are encouraged to include a Date Accessed: statement in the citation.
  • A DOI is only recommended when the full range of data is made available. If the provisional or near real-time dataset does not do this (e.g., only the last 3 months of data are made available), a DOI is not recommended.
  • If these datasets are no longer updated, the publication year should reflect the latest year that data was added.
  • If a DOI is minted, downstream users should be encouraged to include a Data Accessed: element in their citation.

Research-ready Data

  • Research-ready datasets should be properly accessible, reproducible, and citable. This means that all research-ready datasets should have a DOI.
  • Research-ready datasets are mostly made available in data package on e.g. GitHub, or through platforms such as ERDDAP.

ERDDAP:

  • For research-ready datasets that are made available through ERDDAP, for example the CTD Dataset, the mechanism for publication and data release is different compared to GitHub Releases. Rather than creating a new ERDDAP dataset for each new version, the research-ready dataset in ERDDAP is updated. We encourage users to include a Date Accessed: when citing the resource.
  • Time series updates to research-ready datasets in ERDDAP, e.g., additional QCd data points being added, do not require the version number to be updated in the metadata record, because older versions are not archived. However:
    • The revision date in the metadata record should reflect the latest data point in the ERDDAP record;
    • Whenever the research-ready ERDDAP dataset is updated, changes can be documented in a Changelog to help users understand what changes were made and when;
    • The metadata record should always describe the underlying resource. Therefore, whenever the research-ready dataset is updated, it is worthwhile to review the associated metadata record;
    • Modifications to existing, published ERDDAP data (e.g., fixing erroneous data) should result in a new version number, and be documented in a Changelog.

If a subset of a larger research-ready ERDDAP dataset is included in a data package or in a publication, the versioning guidelines as outlined below apply.

GitHub:

  • GitHub can easily make the different versions available through GitHub Releases.
  • Whenever a research-ready dataset or data package on GitHub is updated, this should result in a new release, and in an incremental minor version element change. GitHub releases allow archived versions to remain accessible to users.
  • The tag of the latest Release should match the version number recorded in the metadata record.
  • The date of the latest GitHub Release should match the 'revision date' field of the record in the Metadata Entry Form. Updating this field when you publish a new release ensures that the citation is accurate for the latest release.
  • Briefly review the metadata and author list whenever the revision date field is updated to ensure that the metadata record still accurately describes the underlying resource.

CHANGELOG.txt

For changes in minor versions, it is important to include a log of what changes were made using a Changelog.txt file. This Changelog file should be stored within your data package in the GitHub repository, or linked through the metadata record. For data products not stored in a Hakai GitHub repository, ensure that your data product has an associated Changelog that indicates what the most recent version is, in addition to a folder containing archived versions if applicable.

Note

Oftentimes, the primary user of your data package in the future will be yourself. As such, a changelog can be useful to indicate what the differences are between any raw collected data, and the processed data made available, along with the reasoning (e.g., are certain error flags omitted from processed data).

GitHub Releases

GitHub simplifies the ability to generate a new release of your dataset. Once you're happy with the latest version of your repository main branch, create a new release by clicking on the Create a new release link within the release section of the repository main page. When filling in the form, add an appropriate new version tag and click Publish release.

This will package your latest repository version into a zipped package and add it to all the releases made available on your repository releases page. Through the metadata record, you can ensure that you always point to the latest release: https://github.com/HakaiInstitute/YOUR_REPOSITORY_NAME/releases/latest