Using other standards with DDI 4
Jay Greenfield, Booz Allen HamiltonElizabeth Hostetter, Statistics Canada
June 6, 2014
| Anchor | ||||
|---|---|---|---|---|
|
During the development of DDI 4, an international standard for the description of statistical and social science data, a need was identified for guidance on implementing DDI 4 with existing data and metadata standards. The 4th version of DDI is designed to complement existing data and metadata standards and should preclude the recreation of objects, elements, and other types of information from existing standards.
Purpose
This document contains guidelines for content modellers and others interested in referencing and integrating standards with DDI 4 to support metadata and data interoperability, and add flexibility to implementations of DDI.
| Anchor | ||||
|---|---|---|---|---|
|
Description The guidance in this document consists of general information on the use of other standards with DDI and instructions on how to develop profiles, as well as several examples of how standards can be used with DDI. The guidance is based on ISO/IEC TR 10000:1998(E) Framework and Taxonomy of International Standardized Profiles.
DDI 4 and other standards
DDI 4 can be used with existing standards to support the research data lifecycle/statistical business process. See Appendix A for descriptions of standards that can be used with DDI.
When combining standards, it is recommended that modellers create "profiles", which are specifications for the use of one or more standards. A profile can contain one or more profiles, for example a profile for data discovery can have profiles for DDI + SDMX/Data Cube and DDI + Dublin Core. The result is a master profile that can be implemented in whole or in part.
The ISO Technical Report refers to the International Standardized Profile (ISP), which is "[a]n internationally agreed-to, harmonized document which describes one or more profiles" (p. 1.) This guidance will not deal with ISPs specifically.
Approaches to combining standards
...
*Including locally-defined elements.
Interoperability and DDI
Standards identified for possible use with DDI are assessed to determine the extent to which use of the standards supports interoperability.
The two levels of interoperability for DDI 4 are:
...
- Go to source model and evaluate objects, definitions, and relationships for conformance. See "Conformance to DDI" for more information.
- Ensure that objects with same label or definition are not duplicated within a profile.
Element mapping
- Evaluate the elements in the standards to be used with DDI, in particular the element labels and definitions.
- Look for element groupings and splits, where an element in one standard is expressed as two elements in the other.
- Look for element hierarchy and nesting issues, as well as dependencies and constraints.
- Ensure that elements with same label or definition are not duplicated within a profile.
- Beware of "False friends": elements that have the same or similar labels or definitions, but are conceptually different.
...
| Anchor | ||||
|---|---|---|---|---|
|
Conformance is the extent to which an implementation of a standard complies with the requirements of a specification of that standard. The degrees of conformance are: full, partial, and non-conformant. The standard of conformance is declared in the statement of intent of a profile.
| Anchor | ||||
|---|---|---|---|---|
|
Implementation Conformance Statements
To evaluate the conformance of a particular implementation, it is necessary to have a statement of the capabilities that have been implemented in support of one or more specifications, specifically including the relevant optional capabilities and limits, so that the implementation can be tested for conformance to the relevant requirements, and only to those requirements. Such a statement is called an Implementation Conformance Statement (ICS).
...
Profiles should be maintained to support common implementations of standards with DDI. In the context of a metadata registry, this can be accomplished through registry governance structures; however in cases in which a profile is jointly developed, one party should be responsible for managing the profile and handling inquiries. Profiles are updated based on feedback from users, based on full profile implementations.
Versioning should be handled by the profile authors, based on broader parameters defined by the DDI project manager.
Contact information
DDI Alliance...[TBD]
| Anchor | ||||
|---|---|---|---|---|
|
- ISO/IEC TR 10000-1: Information technology – Framework and taxonomy of International Standardized Profiles
- Guidelines for Dublin Core Application Profiles
- Library of Congress: About profiles http://www.loc.gov/z3950/agency/profiles/about.html
- Government of Canada: Viewpoint to Developing a Metadata Application Profile – GCPEDIA. Last modified March 11, 2013. [Accessed June 2, 2014]
Appendix A: Standards by phase - Research data lifecycle
Specify | Collect | Process | Disseminate | Preserve | Discover |
|---|---|---|---|---|---|
DCMI (Dublin Core) citations can be leveraged by a software agent to assess the research value of candidate constructs and measure | Protocol Specification is a functional view using GSIM and OWL-S together with DDI. This view is applicable to longitudinal studies as well as multi-mode contact procedures. | PROV can be used to tell a story about entities, activities, and people involved in producinga piece of data or thing over time | DCMI (Dublin Core) citations can point to publications that can assist researchers in the analysis of dissemination datasets | DISCO is one of several functional views. | |
GSIM is borrowed from in the DDI conceptual model and its views. |
| GSIM, OWL-S and DDI are combined in a functional view that supports the description of data processing pipelines. Provenance chains are a special case of a data processing pipeline. | DCAT (Data Catalog Vocabulary) annotates microdata datasets with their themes, publication information and access information. | Vis a vis DISCO, the NIH Data Discover Index is a more minimal approach to discovery | |
ADMS is a framework for identifying and annotating semantic assets such as code lists |
|
|
| PROV stories can figure into OAIS AIPs and DIPs | Vis a vis DISCO, DwB (Data Without Boundaries) is a more comprehensive approach to data discovery |
|
| Data Cube is used alongside DDI in the description of ncubes |
| OAIS-Data Lake is a specialization of the OAIS reference model that can take the form of metadata tags in a data lake. OAIS-Data Lake is a work in progress. |
|
Appendix B: Profile for DDI 4 + ?
Appendix B: Profile for DDI 4 + ?
Appendix C: Profile template
...
Section | Description |
Title page | Prepared using the documentation format defined for the DDI 4 project |
Contents | Optional. Provides an overall view of the profile and facilitates consultation of the document. Should normally list only the clauses and annexes in the profile. |
Foreword | Mandatory. Consists of information relating to the organization(s) responsible for the profile, as well as (optionally) a statement about whether the profile cancels or replaces other profiles or documents, a statement of major technical changes since the last version of the profile, and a statement on which parts of the profile are normative and which are informative. |
Introduction | Mandatory. Provides information about the process used to draft the profile and the degree of harmonization between the standards described in the profile. |
| Mandatory. |
| Mandatory. List of normative documents referenced in the profile, in bibliography form. Cannot include documents that are not publicly available. Note errors or corrections issued for the documents. |
| Optional. Provide definitions necessary for understanding the profile. Use the statement: "For the purposes of this DDI profile, the following definitions apply." |
| Optional. Provide a list of symbols and abbreviations, along with information needed to understand them. |
5. Requirements | Mandatory. |
6. Testing methods | Optional. Description of testing methods used to determine interoperability and conformance levels, as well as the conformance (ICS) and testing statements (PTS). |
7. Supplementary elements | Optional. Informative annexes, footnotes, and editorial and layout information. |
Glossary
Implementation Conformance Statement (ICS): A statement made by the supplier of an implementation or IT system claimed to conform to one or more specifications, stating which capabilities have been implemented, specifically including the relevant optional capabilities and limits. (Source: ISO/IEC TR 10000-1:1998(E))
Interoperability: The ability of two or more IT systems to exchange information and to make mutual use of the information that has been exchanged. (Source: ISO/IEC TR 10000-1:1998(E))
Profile: A set of one or more base standards and/or ISPs, and, where applicable, the identification of chosen classes, conforming subsets, options and parameters of those base standards, or ISPs necessary to accomplish a particular function. (Source: ISO/IEC TR 10000-1:1998(E))
Profile Test Specification (PTS): A statement describing the testing methods of the conformance tests carried out on the profile, as well as the test results and general conclusion.