In attendance: Arofan, Wendy, Slava, Dan G., Deirdre, Hilde, Benjamin and Flavio

AGENDA

1. Administrative (if any)

2. WG reports

3. Dagstuhl preparation

a.      Reminder about prep materials/presentations

b.      Any questions

4. Slides for training group

5. Existing issues

Quick recap

The meeting began with administrative discussions and agenda setting, including updates on various working groups and their ongoing projects. The team reviewed progress on several technical initiatives, including documentation efforts, tool demonstrations, and data integration work between different standards. The group concluded by discussing model transformations, variable statistics, and data format handling, while also addressing document organization challenges and planning for upcoming workshops and presentations.

Next steps

• Flavio: Update older meeting minutes before going to Dagstuhl

• Dan: Work on primer at Dagstuhl if time allows, potentially get it to a point for review by people unfamiliar with CDI

• Tools group : Meet next week before Dagstuhl

• Deirdre: Provide Achim with updated Maven version of Acceleo 4 for review

• Flavio: Compile documents for the mapping work and put together a presentation with introduction to DDI CDI and use cases for Dagstuhl

• Flavio: Complete qualitative model by the weekend or early Monday before Dagstuhl, definitely before the workshop

• Hilde: Review Arofan's paper on variables tomorrow

• Wendy: Set up Google Space for DDI Alliance within the next few weeks

• Flavio: Gather introductory materials and use case slides for Dagstuhl, to be used as basis for submission to Training Group (Arofan volunteers to help with this over the next 6 months)

• To Follow Up with Pascal: Get variable statistics requirements related to Croissant

Summary

Working Group Reports

The working group discussed progress on their primer, noting slow word-smithing progress but potential acceleration at Dagstuhl. The importance of documentation was highlighted, and it was recognized that the qualitative work is less pressing than documentation at this point. An update was provided on the Tools Group, mentioning a meeting next week. There was a demonstration of a tool for AI-driven concept linking to external sources (such as Wikidata), with auto-translation capabilities. There will soon be a meeting with Simon Hodson of CODATA to discuss applying Acceleo to CDIF generation, including the DDI-CDI profile.

Dagstuhl  Topics

Documentation (as above)

Tools:

The team discussed progress on several projects, including Deirdre's work on updating Occam with an Acceleo 4 Maven version.

Mapping:

There are plans for a focus on data integration and interoperability between SDMX and DDI, with the goal of developing practical recommendations and implementations using the UNSD Data Commons use case. The team also discussed the need for further work on mappings across different standards and the potential to leverage existing Python code for these efforts.

DDI and SDMX Integration Discussion

Flavio – as the chair of the UN/ECE Modern Stats effort - discussed the integration of DDI-to-DDI mappings and the challenges of expressing mappings between DDI and SDMX schemas. He mentioned working on a presentation about DDI and its use cases, aiming for a 30-minute duration to keep it engaging. Arofan offered a real-world SDMX-to-DDI use case from the Malawi Disasters project, which Flavio expressed interest in exploring for commonalities with their current work. Flavio is compiling materials for the introductory presentation at Dagstuhl and has reached out to various team members for use case examples.

Codebook Mapping and Qualitative Data

The team discussed mapping between codebooks, lifecycles, and CDI, with Pascal Heus having previously implemented this in his Python library. Extracting the mapping from Pascal's code was considered. Arofan mentioned sending a paper on variables to the documentation loop, which Flavio had reviewed and found clear, and Hilde agreed to review it the following day.

Qualitative Data:

Noemi – the chair of the Qualitative group at Dagstuhl – was not present, but Flavio committed to providing a concrete proposal for qualitative data recommendations by the upcoming workshop. He noted it might be ready by early Monday.

DDI Alliance Document Management Update

The group discussed document organization challenges and a new Google Space for the DDI Alliance that is being setting up. The new space will provide structured access to documents, allowing for editing and preservation, with complete access expected in the coming weeks. The team agreed to use the existing CODATA setup for the current workshop while waiting for the new DDI Alliance space to be fully operational.

Training Slides

A request from the Training Group for DDI/CDI slides was addressed, with Arofan volunteering to adapt the Dagstuhl deck. It was suggested that use cases be included to make the materials more practical for training purposes.

DDI-CDI Issues

Current filed issues were discussed, with some being prioritized, notably addressing the description of binary data formats, in line with the CDIF4XAS project feedback, and the modelling of variable statistics. The idea is that the model used in DDI Lifecycle should serve as a basis, but that additional requirements should be gathered. Other issues were seen as less pressing.

The team discussed model transformation approaches, with a suggestion that a model-to-model framework was preferable to direct syntax transformations. The potential use of OGC Building Blocks for creating modular profiles using SHACL was noted, which could complement their existing model-to-model approach. The group acknowledged the need to address PSM versus PIM conformance issues, though there was some concern about getting bogged down in theoretical discussions without clear requirements.

Next meeting: Wednesday November 26th

In attendance: Achim, Arofan, Wendy, Dan G., Deirdre, Hilde, Benjamin, Noemi, John Kunze

Agenda and minutes

Qualitative Recommendation Implementation Status

The group discussed the qualitative recommendation implementations, which was needed before Dagstuhl. Arofan to follow up with Flavio on this. Flavio and Dan had conversations about using an annotation component (as suggested by the Quali sub-group), but concluded other options would be preferable.

XAS work/binary formats

Some bugs were found related to handling binary formats like HDF5, NetCDF, and Parquet, which need addressing. The team agreed that while the XAS topic is a first use case for this model, for similar formats, other formats will need to be investigated. The draft of the model for HDF5 will be part of the XAS project deliverables, and will be ready before the first week of Dagstuhl (Steve Richard did most of the work on this). Some initial documentation of issues has been done by Flavio.

CDI Primer Development Update

The documentation subgroup is developing a primer for CDI, aimed at new users with minimal assumptions about their prior knowledge. The guide will include sections on CDI's features, such as the variable cascade. The team expects to make progress during the upcoming Dagstuhl workshop.

Possible new user

PSDI in the UK, expressed interest in using DDI-CDI (PSDI is part of STFC). PSDI maintains a lot of reference data bases using a standard called OPTIMADE (a chemoinformatics standards group) but they see value in being able to describe their resources using CDIF, including the DDI-CDI profile for datasets. They have identified this as a future activity, but may require support.

CDI Documentation Alignment Discussion

The group discussed aligning documentation and descriptions of DDI-CDI's variable cascade to avoid confusion, particularly after George's experience with SDTH. Dan emphasized the need to clearly distinguish between things and descriptions of things in DDI-CDI documentation, as users might otherwise get confused about the meaning of the variable concept in DDI-CDI. Arofan will circulate a draft of the document he is working on with a similar theme, related to data structures. Hilde suggested having an exchange at the next DDI-CDI meeting to ensure consistency in how concepts are expressed.

RDF Class Generator Development

The team discussed the development of a tool for generating Pydantic classes and serializers/deserializers for RDF. Pascal had been working on this, creating a two-line process that generates SHACL and Pydantic classes. The output had been tested thoroughly and was working well. Deirdre is working on a stand-alone approach aiming for a directional goal to generate Pydantic classes and test cases. Deirdre to send a JSON-LD sample to ensure alignment with expectations.

Nectar Tool Implementation Planning

Olaf will not be attending Dagstuhl but is willing to communicate about Nectar and potentially give a presentation. The team discussed the importance of involving Olof the ongoing work on Nectar. Benjamin to invite Olof to the next meeting in the tools sub-group.

Enhancing Documentation and Collaboration

The team discussed the importance of having proper tools and documentation to move forward with their work, in particular information on CDI representation. The JSON-LD style used in DDI-L v4.0 needs to be aligned with other DDI products; Deirdre to look into. Joachim is working on finalizing the documentation and formal definition of UCMIS, aiming to show progress before the next workshop. The team agreed to collaborate on meeting minutes, with Hilde offering to draft them and Arofan offering to edit.

Next meeting: Wednesday October 29th

In attendance: Deirdre Lungley, Noemi Betancort, Filipi Miranda Soares, Hilde Orten, Joachim Wackerow, Wendy Thomas, Benjamin Beuster, Dan Gillman, Darren Bell, Slava Tykhonov, Flavio Rizzolo

Agenda and minutes

ISO proposal: template content needs to be removed before publication in the DDI Alliance site. Wendy to produce a cleaned-up version to be reviewed by Dan.

Qualitative data: the team discussed part of the qualitative data proposal, specially the need of using a proposed annotation component instead of a standard attribute component. Annotations need to be linked to specific data points at both the record level and variable level. Flavio to write down the current understanding and to discuss with others once that’s done. Hilde shared examples from UK data showing how annotations can be applied to different types of data, including text and numerical values. Filipi suggested the use of annotations for AI model training, highlighting the need to understand how these fit into the overall model structure. The group will continue exploring these ideas, potentially at the upcoming Dagstuhl meeting. Once the qualitative proposal is implemented in the model, it will be passed to the new DDI Alliance Qualitative Working Group so that similar implementations can be done in Lifecycle and Codebook.

Team to review the updated field level documentation.

Darren announced the upcoming launch of a social sciences portal next week and discussed the need to finalize the memorandum of understanding and draft charter for the data extension working group.

Other subgroup updates: Slava presentation on converting data files using AI tool (answering questions about DDI-CDI retrieving documents and providing responses, http://mcp.dev.codata.org ), Deirdre's work on Acceleo 4, and Dan's progress on drafting a high-level CDI description document. Filipi highlighted the potential for using DDI-CDI in various data spaces, particularly regarding high-value datasets and mandatory variables.

This report was completely produced by AI based on DDI-CDI sample https://github.com/codata/cdi-xas/blob/ai/docs/CDIF_overview_missing.md

Next meeting: Wednesday September 17th

In attendance: Deirdre Lungley, Noemi Betancort, Hilde Orten, Joachim Wackerow, Wendy Thomas, Benjamin Beuster, Dan Gillman, Darren Bell, Slava Tykhonov, Flavio Rizzolo

Agenda and minutes

W3C collaboration: Approved by the Executive Board. Darren will setup a meeting with Pierre Antoine and others to draft the charter for the joint working group. This is important for the Dagstuhl meeting. It’s important to keep the W3C work in sync with the ISO document, although they can cover some different aspects.

ISO document: to be published on the DDI site. Wendy to work on preparing the ISO document for publication on the DDI site by stripping out ISO-specific content.

Field level documentation tool: now supports mobile devices, new syntax representations, and allows users to enlarge diagrams as overlays and download them as SVG or PNG. It’s important to note the tool is now independent from DDI-CDI and can generate documentation for any UML model that complies with UCMIS.

Qualitative data: proposal is a priority, together with the Format Description cleanup. Flavio to work on the format description cleanup and modeling the qualitative proposal. Further developments on qualitative will be done in the new DDI Alliance Qualitative Working Group chaired by Noemi.  

issue #20: content negotiation is available on all classes, and the group discusses whether it should also be implemented at the root URL level. It should also be implemented at the root level for consistency and completeness.

Not much progress has been done with the Acceleo 4 migration (Achim has been working on the documentation tool update. Deirdre to help Achim with parts of the migration. The group agrees to focus first on syntax representations rather than documentation, as this is less complex programmatically and would benefit users like Pascal who need access to RDF and XST.

Slave to work on the integration of Nectar publisher with Dataverse to support DDI-CDI. Benjamin, Slava, Deirdre, and Pascal to meet next week to discuss further development of the nectar tool.

Documentation subgroup to continue working on the CDI primer draft.

Next meeting: Wednesday September the 3rd

In attendance: Deirdre Lungley, Noemi Betancort, Hilde Orten, Joachim Wackerow, Wendy Thomas, Benjamin Beuster, Dan Gillman, Darren Bell, Slava Tykhonov, Flavio Rizzolo

Agenda and minutes

Flavio will send a recurrent meeting invitation (freq: every other week)

The use of tags to organize the work in Github was discussed. Team will review tags and provide feedback.

Subgroups will need to be created for tasks that involved reviewing new requirements, coming up with a model update proposal and finally update the model.

Issue #15: I-ADOPT can be used for organizing and qualifying concepts in data, particularly useful for natural sciences but applicable to socioeconomic data as well, which can be useful for describing data transformations and relationships between variables without information loss. A subgroup will look at further exploring this framework in relation to CDI. Dan volunteered to participate. Other potential member could be George and possibly Kathy (Barbara is the key expert in I-ADOPT, but might not be able to be part of this group, although validating the output of this group with her will be useful).

Issue #16: there is potential for information loss when transforming data between formats. We need to make sure we can capture all types of variable relationship to ensure lossless data transformation. Related to issue #15 since I-ADOPT could be part of the solution.

Issue #18: Two types of non-traditional data formats need to be captured: (i) graphs and binary formats encoding quantitative data, and (ii) formats for qualitative data. The latter is being investigated in the qualitative CDI subgroup, soon to morph into a DDI qualitative WG. This group needs to focus in the former, probably creating a subgroup.

Issue #22: Slava reported on a prototype service that can convert XAS files to CDIF format using knowledge graphs, which is now operational and being extended to work with Nectar service. The team discussed the complexity of handling non-traditional data formats and agreed to organize a broader discussion about this topic.

Issue #25: challenges of not having a separation between a platform-independent model (PIM) and a platform-specific model (PSM) remain. Potential solutions: creating a final PSM or making the conceptual model more in line with the syntax representations. Probably a sub-group needs to look into this?

Issue #26: Some issues related to the Classification and Codelist models are being discussed in a subgroup (Dan and Flavio). The core of the issue is the relationship between structural hierarchies of classification items and semantic hierarchies of categories (these hierarchies must be consistent while allowing for some flexibility in relationships).

Note: With so many small (and some not so small) updates to the model in the works, the question of how new versions will be published need to be addressed. As a reference, see https://zenodo.org/records/7765072 . The team needs to start discussing in the next meeting the prioritization of model updates and timelines for CDI v1.1

• Documentation: the documentation subgroup has identified gaps in the current documentation and is working on creating a structured guide that ranges from a high-level overview to more technical details, catering to different user needs

• W3C: the document will be signed off at the EB meeting on Monday after finalizing a few comments. The new RDF variable description vocabulary will align with existing W3C standards like DCAT and Data Cube. Once approved, the team will draft a charter as an extension to the Data Exchange Group and likely convene a working group. Pierre Antoine would need to be informed of the W3C approval process and might have to take action on the W3C side.

• Qualitative: first meeting would be on August 15^th

• SDMX-DDI mapping: to tackle some roadblocks in the mapping work, a new approach was proposed focusing on mapping groups of classes and attributes rather than individual classes. They plan is to test this approach with two use cases: a UN case being worked on by Louise and a codebook metadata mapping project led by Edgardo.

Next meeting: Wednesday Aug 6th

In attendance: Deirdre Lungley, Hilde Orten, Wendy Thomas, Benjamin Beuster, Dan Gillman, John Kunze, Flavio Rizzolo

Issue #12: The automation of the DDI-CDI generation of the field-level documentation and encodings is working and will be integrated in the Github environment as TC plans.

Issue #13: the preliminary way statistics are captured in the model requires a redesign to determine the best way of associating statistics to data sets. A sub-group will be formed to address this and provide concrete requirements to the modeling team, with Dan and Wendy expressing interest.

Issue #14: The variable requires harmonization (or at the very least alignment) across DDI products, focusing on achieving consistency while acknowledging differences due to backward compatibility and application-specific needs. The ISO submission will be used as the basis for a detailed description of the variable cascade, independent of specific specifications. Balancing standardization with flexibility, particularly in application-specific contexts, is essential. The connection with the SDTL/H Variable Instance will be moved to the TC (in a separate TC issue). Characterizing Variable Instance with a precise definition and examples is the first step, since part of the problem seem to be agreeing on terminology/vocabulary. A new working group under the scientific board should be created to look into how SDTL/H integrates with other products, specially the Variable Cascade.

Discussions focused on data structure organization, the formation of a new qualitative data group under DDI, and ongoing work on documentation resources. The team agreed to address modeling changes and integration with the W3C web annotation model in future discussions with relevant team members. There is a considerable overlap between the W3C and ISO submissions, which means both works need to be aligned in some form or shape. (Wendy will present the mapping work within DDI products to the UNECE SDMX-DDI interoperability group on Thursday.)

• A prioritization tagging system is required, Deirdre offered to investigate available tags and labels and come back to the group with a proposal.

• A new CDIF subgroup will be formed to study the use of key-value structure for Mass Spectrometry Data, with Flavio, Joachim, Benjamin, Deirdre, and Dan expressing interest. Deirdre is creating a key-value store example and a wide data set to demonstrate how they can be combined, suggesting this would be less complex than implementing the XGCI format. Flavio will send an example from a slide deck to Heike as a reference. Deidre will create a new issue in Github to document the current state and further developments. Deirdre and Benjamin will work on the initial steps. A meeting will be setup asap to discuss with the broader sub-group.

Next meeting: Wednesday July 23rd

In attendance: Deirdre Lungley, Hilde Orten, Wendy Thomas, Benjamin Beuster, Dan Gillman, Darren Bell, Flavio Rizzolo

Agenda and minutes

Second discussion of the backlog to determine how much effort is required. There will be a prioritization later on.

Issue #21: simple technical fix, to be assigned to Darren to be updated.

Issue #20: it’s about RDF namespace content negotiation. Darren to update issue when implemented.

Issue #9: Can CDI support data reorganization in the context of household and person-level data? Yes, the standard supports different types of records and how to move content from one to another. However, it’s necessary to provide some guidance document explaining how to do it given that users may not always understand the implications. Flavio to update issue.

Issue #10: Having all attributes with default values as required have become a pain point. Decision: make attributes with “false” default value as optional, while keeping those with “true” default value as required. Flavio to update issue.

Issue #11: Signification pattern is not the issue. Rather, the issue is that the UML model is conceptual and platform-independent, which means that implementing all classes in the same way in the syntax representations is sub-optimal and makes the representation unnecessary complicated (e.g. Notation or Position shouldn’t be represented like Code or List just because they are all classes in the conceptual model). As long as we use the current platform-independent model (PIM) as if it were a platform-specific model (PSM) we’ll see these issues. Flavio to close and create new issue about the need for platform-specific models for syntax representations.

Note: no need to add issue for long association names.

Discussion will continue in the next meeting.

Documentation: Reviewed existing CDI documentation and found most materials are advanced and technical, leading to a decision to create introductory-level documentation. Existing training materials and webinars can be supplementary resources. Look at how to use existing training material. Gaps for the training group.

Tools: Discussed the Nectar tool's implementation of CDIF, focusing on code list concepts and variable classification. The tool needs further development to validate against the current SHACL DDI CDIF standard. Acceleo development will continue.

Qualitative: getting closer to establish a DDI-wide qualitative working group (it currently focuses on CDI). Need to compile CDI requirements to be included in the model as soon as possible.

W3C: Darren to complete a W3C collaboration justification document to be discussed in the Scientific Board.

Next meeting: Wednesday June 25^th

In attendance: Deirdre Lungley, Noemi Betancort, Hilde Orten, Joachim Wackerow, Wendy Thomas, Benjamin Beuster, Dan Gillman, Darren Bell, Flavio Rizzolo

Agenda and minutes

First attempt at trying to organize the work in the repo backlog.

Just going over the issues for quick discussion to determine how much effort is required. There will be a prioritization later on.

Issue #2: just a technical task related to the upgrade to the latest EA Sparx. No decision yet on when the upgrade will be in place.

Issue #3: pertains the duplication that occurs when an association with the same name is defined both in a class and its super-class. More problematic when dealing with two inheritance hierarchies connected by associations because it requires constraints, either as notes in the model or in a formal language, to indicate which of all combinations are valid (see variable and unit cascade as an example). Related to issue #4

Issue #6: the issue has likely been solved, Flavio will check the UML model.

Issue #7: requires a separate discussion on how to better remodel some parts of the process model to capture both prospective and retrospective provenance. The main concern is how parameters are instantiated at runtime. Flavio to update issue with more details.

Issue #8: pertains to the complexity of identification and reference. There are dozens of attributes, with the large majority targeting corner cases. Developers are not pleased. The (small) set of most common attributes need to be identified to separate from the rest. Unclear yet which is the best way to handle the separation of concerns (inheritance vs composition vs something else). Flavio to update issue with more details.

Discussion will continue in the next meeting.

Documentation: initial meeting. Discussion of the template brought by Noemi and initial review of the existing documentation to get a sense of what’s the extent of the work.

W3C: IP & governance need to be discussed. It started (and will continue to be for the time being) as a CDI project, but it will eventually include profiles of other products. Darren to draft 2-pager on current state and goals.

W3C and ISO activities need to be aligned/coordinated. Ideally the same CDI profile/fragments will be the core of both standards. A discussion between both groups will be scheduled.

Qualitative: presentation at the DDI scientific meeting

Tools: knowledge sharing between Benjamin and Deirdre (Bergen meeting). How to handle microdata at data point granularity was discussed (billons of triples in some cases). Most use cases don’t need to include all data points in CDI format, just those that require annotations and/or traceability individually. In most cases record-level, or even variable-level, would suffice. Will parquet and other physical formats be supported?

DDI-SDMX: presentation at the DDI Scientific meeting. Submission to the SDMX Global Conference is being prepared.

Next meeting: Wednesday June 11^th

In attendance: Deirdre Lungley, Noemi Betancort, Benjamin Beuster, Dan Gillman, John Kunze, Flavio Rizzolo

Agenda and minutes

Different options were discussed, e.g. shorter meetings every other week, monthly, etc. The consensus was that once a month, with the option of adding another meeting when necessary, best fits the way sub-groups are currently working.

Decision: try the monthly option for a few months and see how it goes. Due to conflicts the next meeting will be in three weeks (May 28th).

Tools: No meetings. Next meeting in person in Bergen next week. Deirdre will be working with Pascal, who wants to make his DDI toolkit available (he is currently off for a few weeks). The Tools sub-group wants to meet with the documentation sub-group to coordinate how documentation about tools is produced.

Syntax: will start migration from Acceleo 3 to 4.

Qualitative: keep meeting twice a month.

Documentation: initial meeting. Discussion on whether changes need to be made. First documents to tackle are the implementation guide and the variable cascade. The Documentation sub-group will meet with the Tools sub-group, as requested.

DDI-SDMX: no meeting and nothing to report. Brief summary of the group activity so far was discussed as a recap.

Noemi provided an overview of the https://docs.google.com/document/d/1q1nCUvDTTm_aBjb37Vl4nCfUEuPxYD-_n--pOb8jdgc .

The template was put together based on templates used by RDA WGs, FAIR mappings WG, and other groups working with repositories. No standard, but many commonalities among them.

The main goal is to describe data for implementation purposes, i.e. what the implementation is about and how CDI can help.

It contains a description of what data looks like (sample), characteristics of the data, and requirements (disclosure limitations, terms of use, linkage, metadata requirements)

It includes an output section covering the benefits of this implementation (provenance, transparency, reproducibility, etc.)

Next steps would be to test it by the sub-groups to determine how to adapt/improve

It might too much information to fill upfront. However, it’s a tool to clearly communicate requirements and as such many details are necessary in some cases to properly implement a solution. It all depends on who is the audience and who is filling out the template.

Where do we go from here? It aligns with the SB strategic goals, we need to follow up with the TC.

In attendance: Hilde Orten, Achim Wackerow, Noemi Betancort, Dan Gillman, Wendy Thomas, Flavio Rizzolo, Deirdre Lungley, John Kunze

Agenda and minutes

(Noemi and Flavio - doc at https://docs.google.com/document/d/1RFOvs7ciHQAS0P_EG-NF3lSJFWTu_-eKeBvLKPNwyN8/edit?usp=sharing
spreadsheet: https://docs.google.com/spreadsheets/d/1qIAw6zK6KIDBNSUQTFJfIjqRdkyMkj9oFTyQnNdttpI/edit?gid=6601827#gid=6601827
Work is ongoing.

(See the email from Jon/Jared regarding funding opportunities below). There seems this year to be two options: 2000 US Dollars for each WG, + open call for 10 000 USD. Deadline May 7th.
SB clarifies with EB on next Friday how the situation is with the funding for the working groups.
Some money for improving the tooling for the generation of the field-level documentation and encodings would be good.

Deadline May 1st.
Two reports will be done, tooling, and Chur meeting. Hilde and Achim take care for the reports. An interim vice-chair (see below) should send the reports in.

Syntax rep. group: CDI-98 (Deirdre)
The issue is in the current issue tracker at github, see: https://github.com/ddi-cdi/ddi-cdi/issues/10
It is re-opened. Additional information will be added.

o   Libby is not available as vice-chair. There is a need for a vice-chair at least for an interim period.

o   Two issue trackers for CDI: Jira https://ddi-alliance.atlassian.net/jira/software/c/projects/CDI/issues/ vs. GitHub https://github.com/ddi-cdi/ddi-cdi/issues
JIRA write access will be closed

In attendance: Deirdre Lungley, Joachim Wackerow, Noemi Betancort, Bejamin Beuster, Hilde Orten, Dan Gillman, Flavio Rizzolo, John Kunze

Agenda and minutes

WG members review of ISO draft (Wendy and Dan)
see DDI ISO Standard 2025-03-01.docx, https://groups.google.com/group/ddi-cdi/attach/a0a136ad202b9/DDI%20ISO%20Standard%202025-03-01.docx?part=0.1
The comments should be done accordingly the ISO template, see: https://share.ansi.org/ISOT/ISOT%20Voting%20Resources/ISO%20Commenting%20Template.doc
Hints from Dan:
The general, editorial, and technical classification of comments can be defined this way:

• Technical – comments addressing requirements expressed in the document

  • Example – if I have described value domain incorrectly, then that is a technical issue

• Editorial – comments about wording that is not about requirements

  • Misspelling of a word

  • Grammatical errors

• General – comments pertaining to the document in its entirety

  • Use of the wrong font or style

  • The same error cropping up throughout

(Noemi and Flavio are working on it), see: https://docs.google.com/document/d/1RFOvs7ciHQAS0P_EG-NF3lSJFWTu_-eKeBvLKPNwyN8/edit?usp=sharing

Syntax representation group: work is planned regarding Python Pydantic classes (data validation and manipulation)

Noemi mentioned a funding option for software development in the German National Research Data Infrastructure (NFDI), https://www.konsortswd.de/en/flexfunds/

The production framework for generating the field-level documentation and encodings can now be run in batch mode. A related Git repository is provided to the TC, see: https://bitbucket.org/wackerow/ddi-cdi_production/ . TC plans to integrate this into Github automation.

In attendance: Deirdre Lungley, Joachim Wackerow, Noemi Betancort, Bejamin Beuster, Hilde Orten, Dan Gillman, Flavio Rizzolo

Agenda and minutes

1. Admin

• CDI Webinar
  parked

• Review and publication of CDI article – status
  The paper “DDI-CDI and Other Standards” is published in the DDI Working Paper Series, see: DDI-CDI and Other Standards (DDI Working Paper Series, No. 35)

• Production tool – status
  Good progress is made
  Annual meeting of the Scientific Community (May 22) prep

• Plan and CDI WG

2. The Scientific Board has asked all WG’s to provide a two year plan (from summer 20204 through December 2026) in line with the Scientific Work Plan, before the Annual meeting of the Scientific Community on May 22nd. The CDI plan is, however, missing. It would be really good if the CDI group can get a plan for 2025-2026 in place before the annual meeting. Plans from the Working Groups are available here. Scientific Work Plan summer 2024 through December 2026.

• CDI reporting (group will be contacted by SB before Easter)

• CDI presentation for the Annual meeting of the SC (group will be contacted by SB before Easter).

We need somebody to help with the CDI WG plan and to collect items from the CDI sub-groups. It will be good if other people than the SB chairs do this. Flavio and Noemi have been asked by email today. Let’s see if they would be willing/have time for this. Otherwise somebody else?

Noemi and Flavio are willing to put together the report and plan for the CD WG. The subgroups should send related material.

 Input from the sub-groups
Benjamin (Examples and Tools [ET]) reported on a meeting with Deirdre and Pascal Heus. A further meeting is planned with Oliver and Olof.

3. Qualitative data [QD] has interest in a documentation subset for their purpose.

4. AOB

In attendance: Deirdre Lungley, Wendy Thomas, Joachim Wackerow, Noemi Betancort, Bejamin Beuster, Hilde Orten, Filipi Miranda Soares

Agenda and minutes

1. Admin

   1. Google folder structure
      TC is preparing a proposal for a generic structure. It will be sent to EB because of funding.

2. CDI Webinar
   Postponed.

3. Review and publication of CDI article
   Noemi reviewed it, Wendy and Filipi will look at it.

4. Production tool
   Work is going on. It is complicated because Acceleo is designed as an interactive tool

5. 2 year plan and reporting CDI WG

   The Scientific Board has asked all WG’s to provide a two year plan in line with the Scientific Work Plan before the Annual meeting of the Scientific Community on May 22nd. The CDI plan is, however, missing. It would be really good if the CDI group can get a plan for 2025-2026 in place before the annual meeting. Also, we will be asked to do some reporting before meeting and should be prepared for that.
   Hilde will send SB work plan and expectations how a CDI work plan could relate to the SB work plan. Subgroups might have input into this.

6. Input from the sub-groups
   Tools subgroup: see report from Benjamin.

7. AOB

In attendance: Darren Bell, Deirdre Lungley, Wendy Thomas, Joachim Wackerow, Noemi Betancort, Flavio Rizzolo, Bejamin Beuster

Agenda and minutes

1. Admin

   • Feedback on CDI after the release

   • Continuation with subgroups and main group (only reminder)

   • Email list, subject prefixes, email archive (public) (only reminder)

   • Confluence pages: purpose, updates (only reminder)

   • Possible use of Google docs

     • Who could create this?

     • Folders per subgroup should be then located below a main folder, (relevant info linked from the Confluence site).
       Google docs structure for DDI working groups also for DDI-CDI subgroups. Flavio will file an issue. TC will look into it. A generic solution would be great.

2. CDI Webinar
   Darren will set up a Google document where ideas can be collected. The topic will be again discussed in two weeks.

3. Review and publication of CDI article
   Paper on “DDI-CDI and other Standards” can be reviewed by the group until Friday, March 7, 2025. It can be published as DDI Alliance paper (where it will get a DOI). It should be listed in the related page and linked from the DDI-CDI page.

4. Questions from the sub-WGs

5. AOB
   Consistency of associations names in RDF and other encodings should be discussed. This will be addressed by the syntax representations subgroup. Information on the issue will be sent to the group.

DDI-CDI WG Meeting – 12 February 2025

In attendance: Darren Bell, Deirdre Lungley, Hilde Orten, Wendy Thomas, Joachim Wackerow, Arofan Gregory, John Kunze, Flavio Rizzolo

(1) Admin, if any 

(2) Finalisation for the public release of 1.0

(3) Organising sub-groups and meetings

(4) AOB

1. Deirdre to meet with Benjamin regarding joint meeting between Implementation and Tools groups, will e-mail if further action needed (joint meeting, issues etc.)

2. Achim proposes to have a dedicated list for DDI-CDI, with subgroup prefixes. Use e-mail as first step, then escalate issues to the generic group, then meet if needed.

From an e-mail exchange (Achim):

With the background that subgroups are starting now, it would be really great if we can use soon the CDI Google group email list for the communication. The advantage would be that an email group archive develops then. All subgroups should use this email list. Each subgroup should have an acronym which can be the prefix in the email subject (like: [mod] subject). This way the subgroup can be identified easily and everybody is informed what is going on. Suggested acronyms:

·       nq  Quali

·       mod Modelling

·       doc Documentation/Community Implementation Guide

·       map SDMX-DDI Mapping

·       sr  Syntax Representation

·       et  Example and Tools

·       w3c W3C Alignment

• Will need public access to the e-mail archive. Link from Confluence. Only members can post to it - access would be read only.

3. The DDI-CDI 1.0 release is just waiting on the announcement in the newsletter. (Jared says there will be both a general announcement and the newsletter, released later today.) Should we do a webinar? We could do a written announcement for e-mail lists, and then do a webinar of it can be organised. Darren will pursue the webinar.

4. Will rely on Hilde/Achim/Libby to organise meetings during Arofan's medical leave. Next meeting will be in two weeks. Hillde will send e-mail announci9ng the next meeting.

5. Final open questions from release discussions. These are minor points in e-mails between Darren, Achim, Wendy and Arofan. Address at next meeting. Darren will follow up with Achim.

DDI-CDI WG Meeting – 5 February 2025

In attendance: Darren Bell, Deirdre Lungley, Hilde Orten, Benjamin Beuster, Dan Gillman, Wendy Thomas, Joachim Wackerow, Arofan Gregory, John Kunze, Flavio Rizzolo

(1)   Admin

Arofan to go on Medical leave as of February 16. Achim, Hilde, and Libby to fill in chair temporarily.

(2) Finalisation for the public release of 1.0 - Achim's questions

·       HTML request for RDF namespace needs a new page, Darren to create

·       Add links to field-level documentation (without “RDF” in the path) – not a replacement for the RDF links. This does not need content negotiation, as it is essentially a link to the model.

·       Remove RDF XML version until we follow up with Pierre Antoine (Fuseki auto-conversion)

·       Redirects needed to support relative links in field-level doc will be added

·       We will distribute the “unique names” version of the model, and add a comment that additional versions are available in the download package

All changes to be made by this coming Friday.

(3) Hilde asked some additional questions:

·       Testing for/model additions for new data structures new (e.g. NetCDF)? (Could be important of CLIMATE-ADAPT)

Dan Gillman and Flavio Rizzolo are both interested in this issue, which we should discuss with the CDIF effort covering the same thing. This may not end up in CDI beyond the logical description, but we should have one solution.

·       Proposals/model snippets for changes to or new things for CDI, processes.

Raise specific issue in the CDI WG and we can deal with them.

·       Status of SHACL/ ShEx – are they sharable (even if not published)?

CLIMATE-ADAPT ontologist wanted access to this. Deirdre has decent SHACL, but the ShEx is not working as well. We could share the SHACL as a work-in-progress. Not very time-urgent. Do in the next couple of weeks. Achim, Hilde, and Deirdre to sort out as they work on this., Also share with Doug Fils for CDIF – he could use the CDI SHACL.

(4) Organisation and schedule moving forward

We will meet next week, and then shift to meeting every other week. We will try to get Libby to join. There will be further e-mail communications.

(5) AOB

[Hilde is in Athens with free ice cream. Darren says “Beware of Greeks bearing gifts…”]

None.

DDI-CDI WG Meeting – 29 January 2025

In attendance: Darren Bell, Deirdre Lungley, Hilde Orten, Benjamin Beuster, Dan Gillman, Wendy Thomas, Joachim Wackerow, Arofan Gregory, John Kunze

1. Admin, if any 

·       Report to SB still TBD

·       Wendy to send Arofan minutes in collapsable Minutes on Confluence

2. Finalisation for the public release of 1.0

Everything is now in the repository and is correct.

Process: we send TC an e-mail with final locations. Need to establish a full download package location (with everything in the release). Currently the full download package is in GitHub. Wendy will ask people in TC (Olof, Dan Smith) and get an answer as to how to produce the needed ZIP from GitHub.

Achim will transmit the download package to TC once he knows how to do it. There may also be a partial download package just for the field-level documentation.

We now have a product page on the new Alliance website, as well as a page per version for each spec. Once done, we can review (links are not yet final.)

3. Hilde's credit issue

From Hilde:

The list is here: https://ddi-alliance.atlassian.net/wiki/spaces/DDI4/pages/2681241718

In the short version of credits at the field level doc. These are mentioned:

Contributors to DDI-CDI Development

Members of the Cross Domain Integration (CDI) Working Group shepherded the standard into its final form and produced the final documentation. Listed in alphabetical order they are:

• Arofan Gregory (chair)

• Dan Gillman

• Flavio Rizzolo

• Hilde Orten

• Jay Greenfield

• Joachim Wackerow

• Larry Hoyle

• Oliver Hopt

• Wendy Lee Thomas (Technical Committee contact)

I would suggest to add the following members to this short list (alphabetical):

• Benjamin Beuster

• Darren Bell

• Deirdre Lungley

• John Kunze

Proposed fix:

Add heading before Spreadsheet paragraph on https://github.com/ddi-cdi/ddi-cdi/blob/main/CREDITS.md

“Members of the Cross Domain Integration (CDI) Working Group since 2023”

• Benjamin Beuster