Virtual MRT meeting 2021-04-14

Agenda (as of email from Arofan of 2021-04-14

1. Updates on Annotations subgroup
2. Issues and status
3. Organization for next steps

MRT meeting minutes 2020-04-14

Attendees

Achim, Arofan, Flavio, Hilde, Jay, Larry

Apologies

Dan, Oliver, Wendy

1.      Updates on Annotations subgroup

The group will stop meeting until the modularization issues including packaging and layering discussions are done.

Achim is currently working on a mapping of annotation content to Dublin Core.

2.      Issues and status

Larry’s group has been categorizing issues as to whether they can be closed or need to be done or discussed before the public release.

As a next step Arofan and Hilde will go through the issues for discussion and ask people to prepare proposals with pros and cons for those that are important for the release.

3.      Organization for next steps

The goal is to have a draft ready by the end of June.

The preparatory work will be done in two rounds. The things that are likely to impact the model most will be handled first. Work in three areas will start now:

• Identification, linking, deep linking: Achim has provided a proposal regarding identification. He will widen his current proposal to include all content touching on identification and reference, and will provide widened draft by the end of April.
• Modularity: Flavio, Achim and Larry will look into this and provide a draft with pros and cons. The work is estimated to take place in May.
• Formats: Flavio and Larry will look into this, and make a statement regarding what CDI will support and what it will not support. An analyses of use will be provided in 2-3 weeks. Some guidance will also be provided. Possible changes can be thought of in the second round.

In addition to this, Hilde will start to look into updates the field level documentation, with help of Dan and Wendy. Larry will help with analyses of completeness and structure of the field level documentation.

Other topics mentioned for the second step were:

• High level examples for mapping to other specifications of importance to the release.
  • DDI-C, DDI-L
  • SDMX, DCAT
• Use-cases originating from Arofan, Achim and Simon’s work on the DDI-CDI for EOSC report.
• Process (Jay and Flavio). This needs more documentation. Flavio pointed additionally at the need to compare the way CDI solves things, to how other projects think, for example the UNECE process discussions. This could for example be included in an implementation guide to be made available post-release.
• How to connect concepts to vocabularies. Look into how Lifecycle deals with this (Jay, Dan). A question is whether this is doable for the first release. This needs to be checked out.
• Look into if the collection pattern can be amended to point to external resources (Larry).
• Outcome of proposals for solving Jira issues of importance for the release.

Next meeting will focus on the timeline for the release work

Virtual MRT meeting 2021-04-07

Agenda (as of email from Arofan of 2021-04-06)

1. Updates on Annotations subgroup
2. Issues and status
3. Documentation and examples (brainstorm)

Virtual MRT meeting minutes 2021-04-07

Attendees

Achim, Arofan, Dan, Flavio, Hilde, Jay, Larry, Wendy

Apologies

Oliver

General updates

Arofan reported that DDI will be presented at Dublin Core meeting.

Topics and presenters:

• Overview: Jared
• Codebook: Jane
• Lifecycle: Barry
• CDI: Arofan

1.      Updates on Annotations subgroup

This group has not met since the last MRT meeting, as the date fell on Easter Monday.

2.      Issues and status

The group led by Larry had made good progress on categorisation of Jira issues for discussion since the former MRT meeting. A draft has been available to the group post meeting, for discussion at the MRT meeting on April 14^th.

3.      Documentation and examples (brainstorm)

Topics that will be in focus in the coming time:

• Task list and work processes
• Field level documentation
• Comments on the spec
• Formalise standards alignment
• Examples (from webinars; worked examples)

A brainstorm regarding examples followed, and the following ideas were mentioned:

• DCAT AP/CDI example (Arofan, Achim, DCAT).
• Mapping to other standards (SDMX, Lifecycle, Codebook).
• How to use CDI together with vocabularies from other standards, for example SKOS (Jay, Flavio).
• FIP example.
• Process examples (Jay). Make these more valuable to people. What can be done without steps for example. Parameterized vs. non parameterized processes, etc.
• Transformation between data structures (Larry, Flavio). Include SDTL?
• Example from the CDI-EOSC report on cross-domain CDI usage (Arofan, Achim)?
• Possible BLS usecase (Dan).

Arofan pointed out that people now want small standard components for use together, rather than monolithic standards, for example DCAT + CDI Variable Cascade. This could be viewed as a design principle. Flavio commented to this that the model needs to be looked into to find ‘bridge classes’ to other standards. ResourceIdentifier as a possible linking point was mentioned. Achim pointed to the need to look into a UML way to refer to other specifications or model parts,  and explore what is best at each level of abstraction etc. This topic will require a separate discussion.

Arofan also pointed to the importance of user guides to inform the understanding and implementation of the examples.

Forthcoming work

• Organizing the work will be a topic next time.
• Packaging and formatting has already been agreed as topics to be discussed in separate groups.
• Larry will present the outcome of the categorization of Jira issues for discussion.
• The TC will receive a list of planned changes to the model when those have been agreed. This is needed to decide whether a second review of CDI is needed before publication.

Virtual MRT meeting 2021-03-31

Agenda (as of email from Arofan of 2021-03-31)

1. Updates on Annotations subgroup
2. Issues and status
3. IASSIST and other outreach topics
4. Possible subgroups for further work (e.g., Packaging subgroup? Others?)

Meeting minutes, virtual MRT meeting 2021-03-31

Attendees

Achim, Arofan, Dan, Flavio, Hilde, Larry, Wendy

Apologies

Jay, Oliver

0.      General info

Arofan, Achim and Simon are working to finalize the DDI-CDI for EOSC.

Flavio has started to look into relationships between DDI-CDI and SDMX.

1.      Updates on Annotations subgroup

The latest discussions have been centered around packaging and modularity, as well as mapping to citation standards (Dublin Core/DCMI).

The group has also been looking into how date can be simplified for Annotation, and, possibly, for other aspects of the model where interchange with other standards is important. The broader topic of rules for interchange has also been touched on.

2.      Issues and status

A group is currently working to classify DMT and CDI issues in Jira into topic groups for discussion. The group is continuing to work regularly on this.

3. IASSIST and other outreach topics

Larry and Hilde got an abstract about Data Description in the DDI-CDI accepted for the virtual IASSIST conference 2021. They are interested in input from the group regarding the content of the presentation. Arofan pointed to the importance of explaining how CDI can be used to handle new types of data. Examples from EOSC project that Arofan and Achim is working on can be used for this. Flavio pointed at provenance and lineage relationships between datasets as important topics. Dan mentioned that a description of the process model could be useful to include.

As this is just a presentation of normal length it was agreed that the main focus will be on data description and examples, and that process could be covered by one slide. Arofan and Achim will provide the EOSC examples for the presentation.

3.      Possible subgroups for further work (e.g., Packaging subgroup? Others?)

Packaging will require a separate sub-group. Based on interest from DCAT and other, Achim suggested the Variable Cascade to be worked on as a first example. The logical rather than the physical aspects will be in focus of the work.

Flavio is pointing out that packaging is one among other important criteria, and an approach to the complexity is needed. Achim agrees with this, and suggests that the model is looked into, as to which parts of the model can be made modular without large efforts, and of what needs to be done. The goal is to make improvements regarding modularity for the first release and continue to work on this. Larry and Achim are currently carrying out analyses on the interconnectivity of the model.

Formatting is a different topic that Flavio proposed to focus on. Does CDI covers the necessary formats? A separate group will be convened to work on this on the immediate term, while the packaging work will start later.

Focus will also need to be put on mappings, to SDMX (ongoing), and to Codebook and Lifecycle, prior to the release.

Virtual MRT meeting 2021-03-24

Agenda (as of email from Arofan 2021-03-23)

1. Updates regarding SDTL, Dataverse, and EOSC (plus any other short news)
2. Annotations proposal from sub-group & next steps
3. Combined issues list/updates
4. Model development collaboration: "sandbox" ideas, etc.

Meeting minutes Virtual MRT 2021-03-24

Attendees

Achim, Arofan, Dan, Hilde, Larry, Wendy

Apologies

Flavio, Jay, Oliver

1.      Updates regarding SDTL, Dataverse, and EOSC (plus any other short news)

Arofan reported that good contacts for possible CDI implementation have been made through the CDI webinar and symposium work, as well as through the EOSC/DDI-CDI project that Simon, Arofan and Achim are running. One of those collaborations are Dataverse, where how to express codebook information and variable level metadata in CDI is currently explored.

Another topic of interest that has arisen through the contact with SDTL is to use the CDI Process model to drive business processes, in addition to documenting them.

2.      Annotations proposal from sub-group & next steps

Larry has sent out a proposal for changes to Annotation prior to the meeting, which includes a proposal to break up the Annotation class. The proposal contains, among others, the idea to include a Note class, that would allow to attach a note of controlled vocabulary type, or any information element with a URI, to any CDI element. In the proposal Note can use the data type ResourceIdentifier can be used to point to things without object type constraints. The strength of this approach is the freedom it gives to reference anything. However, to use associations would however be required when control over the type of the referenced object is needed.

Arofan suggests that the proposal to break up the Annotation class would not represent huge, but not insignificant changes to the model. To make the class more detailed would on the other hand require less work.

The proposal from the Annotation group received overall positive response from the participants at the meeting, but will need to be reviewed by more group members and in relation to mapping to other standards use cases before approval.

A new sub-group will look into the model packaging and what can be decoupled. This group will collaborate with the Annotation group.

3.      Combined issues list/updates

A separate sub-group is looking into this. It is currently working on grouping DMT and CDI Jira issues that need discussion, and will get back to the full MRT group with a proposal.

4.      Model development collaboration: "sandbox" ideas, etc.

How to bring in changes or new content to the model was discussed. While collaborative modelling platforms were mentioned, Achim pointed out that it is again important to have controlled procedures for implementing changes and new content in the model. People can work on model parts in sandboxes that can be copied into the model by a model master.

Flavio will again be asked by Arofan if he would be willing to take this role.

Regarding the XMI creation from EA, Achim explains that this is currently done manually in 5 steps. Achim will provide a description of the procedures he uses for this. He is also exploring how Eclipse can be used for XMI creation purposes.

Virtual MRT meeting 2021-03-17

Agenda (as of email from Arofan of 2021-03-16)

The focus this week will be on the bug disposition/discussion plans from the meeting Larry organized today (Tuesday 16^th) - he sent an email around earlier for people to look at.

Meeting minutes virtual MRT meeting 2021-03-17

Attendees

Achim, Arofan, Dan, Hilde, Larry

Apologies

Flavio, Jay, Oliver, Wendy

1)      Report from the Annotation group

The current CDI packaging is more conceptually based. Work is ongoing to explore how the model can be modularized for usage of separate modules in different contexts.

Larry had sent out a couple of Jupyter dendrograms to the groups, based on the associations in the model. These show how the classes in the model group together based on common associations. The dendograms provides input to the discussion on how future the future packaging could look like, from one specific perspective.

A different perspective would be the one of use-cases. A common use-case in the EOSC work of Achim, Arofan and Simon is how  DDI-C examples would look like in CDI, and look at clusters this perspective. This relates to the work of Larry and Achim about DDI2 in R.

Related topics were discussed, among those:

• Leave out inheritance super-classes from implementation guidelines
• Flattening of the level of the model
• Quality of associations: What is necessary and what could be solved differently?
• Deep linking
• Size of profiles: How small can they be?
• How can the model be more easy to adapt, for example for JSON developers?

The Annotation group meets again on 2021-03-22 to discuss further.

2)      DMT Jira issues discussion

A smaller group is looking into this. On a meeting on 2021-03-16 a set of pre-Berlin Sprint DMT Jira issues were classified according to if they could be ‘closed’, ‘postponed’ or ‘needs further discussion’. The classification was based on comments from the creators of each of the issues, as well as a review performed by Larry. The classification of the pre-Berlin Sprint DMT Jira issues made was agreed at the MRT meeting.

The next step for Larry and the smaller group will be to go through the DMT issues that were fielded during the MRT period, as well as the CDI post the review release issues, and classify them in the same way as the pre-Berlin Sprint ones.

When all of the open issues have been reviewed, a groping of them will be made, and finally a priority list for issues solving will be made by the full MRT.

Provided by Larry for the next meeting in the smaller group, which will take place on 2021-03-23: Updates on classification of DMT issues fielded in the MRT period, but before the review release.

Virtual MRT meeting 2021-03-10

Agenda as of 2021-03-10

(1) Annotations sub-group update (and related)

(2) DCAT mapping update

(3) SDMX mapping planning

(4) Modular design/packaging

MRT meeting minutes 2021-03-10

Attendees

Achim, Arofan, Dan, Flavio, Hilde, Jay, Larry, Wendy

Apologies

Oliver

(1)   Annotations sub-group update (and related)

Achim reported from discussions in the Annotation group.

• How to make the module more modular was discussed. Larry is performing cluster analyses to check connections between clusters. Achim will analyse associations to find out where deep/light connections are.

It is an agreement in the group that it would be nice to have some parts of the model as a separate pack, for use in different contexts. The Variable Cascade was mentioned as one possible example.

Documentation should be developed that reflects CDI usages for specific purposes. Currently an experienced user must understand the full model in order to use a part.

Achim suggests to use Package in UML for this.

Larry suggested to use the collection pattern to create profiles. This would allow users to create own collections.

Annotation as an overlay has been discussed, where a connection could be made that would not require remodeling. How to link to other standards also formed part of the discussion. Larry pointed out that Annotation currently has an issue in that a relationship to an agent requires Agent to be a class. An ExternalControlledVocabulary could be used alternatively, to refer to a DDI or PROV class, for example. Annotation will be remodeled when issues around it are sorted out.

• Wendy asked how much content is allready in the published review version of the model, compared to what is expected for the first version.

Arofan then replied that there will not be big changes in the specification, but that forthcoming changes in the packaging is expected that regards grouping more than content, and that alignment with other specifications needs to be addressed.

(2)   DCAT mapping update

Achim and Arofan is currently working on a project with Simon that regards what DDI-CDI can do for EOSC, focusing on EOSC requirements. One of the focuses is to find out how DDI-CDI can be used in connection with DCAT. A DCAT expert is working on this in collaboration with the group. The outcome of the DDI-CDI EOSC project will be a report that can feed into DDI-CDI.

Achim explained that the DCAT/CDI collaboration project is about how the standards can be used together, rather than about mapping.

Flavio suggested that DCAT AP also is looked into in this connection, as this is used content of importance to statistical agencies.

A discussion about linking to other standards, as well as of the notion of attachment and attachment points then followed.

Achim mentioned that in relation to DCAT there are two perspectives: The CDI perspective and the DCAT perspective. Each perspective could contribute something. It would be good if DCAT could refer to a rich metadata instance, that could be CDI. Flavio and Dan pointed out that what DCAT has done and what it needs would be useful to explore. The collaboration with the DCAT expert continues, and a report is expected end of March. Discussions around the topic involving MRT and possibly external experts will then follow.

(3)   SDMX mapping planning

Flavio is looking into possibilities for this, and will check out how other standards deal with it.

Arofan points out that the importance to have an official recognition  between the institutions of the purpose first, and then to develop a timeframe for the work.

(4)   Modular design/packaging

Achim points out that ongoing analyses will help to understand the extent of the issue, how it can be fixed, and how much work it will take to do the fixes. Results will form basis for deciding what can be done prior to- and after the first publication. Possible remaining work could be announced as a task for a second CDI release. Achim recommends a pragmatic approach to this, as to what can be done without large efforts, and what would require more time. Improvement of the packaging for a more flexible usage should be accompanied by implementation guidelines for main purpose usages, as well as for usages in other contexts. The connections between packages should be minimized and the UML Package structure will be used for this.

Larry pointed again to the idea of machine actionable packaging. This could be considered as a second, post release, step.

The group will look into how modularization is done in specifications that has solved modularisation well, for example schema.org.

The suggested approach was agreed by the group.

Any other business

• Larry suggested that the group discuss content for his IASSIST presentation in a specific MRT meeting. 31^st of March was agreed as a date to discuss this and other possibilities for presenting DDI, possibly in collaboration with the Training Opportunities group.
• The next MRT meeting on March 18^th will focus on Jira issues.

Virtual MRT meeting 2021-02-10

Agenda

1. Report from Dimensions group and possible follow-on activities with SDMX mapping
2. Report from the Annotations sub-group meeting 
3. Issues overview from Larry/Hilde (review and process for dealing with these)

Minutes virtual MRT meeting 2021-02-10

 Attendees

Achim, Arofan, Dan, Flavio, Hilde, Jay, Larry

Apologies

Wendy, Oliver

Administrative

Achim provided a short report from the first meeting of the Scientific Board. Minutes from that meeting is found here .

As the new SB will call for an update from the current work and future plans of each of the working groups of the Alliance, the MRT will arrange a separate meeting for this on Wednesday February 17^th. Larry created an issue in Jira (CDI-33) where group members can enter ideas for this.

1.      Report from Dimensions group and possible follow-on activities with SDMX mapping

Dan reported from this group: The CDI spec has more classes than GSIM. This is however expected for a logical model closer to the implementation level. It would also be possible to implement CDI without using all of the classes. Examples for this are needed.

Flavio will look into possibilities for a mapping between DDI-CDI and SDMX (that is currently moving into microdata). This could be a project for the future (after the first publication of DDI-CDI?).

2.      Report from the Annotations sub-group meeting 

Larry reported from this group: AnnotatedIdentifiable expands content of Identifiable to include an optional annotation, thus currently uses Identification by inheritance. This information could alternatively be a set of properties. Annotation has some semantics in the name relating to the term ‘Note’ that may be confusing as the core object of this class is to cover discovery information.

AnnotatedIdentifiable could be broken up in persistent and changeable parts and should be able to point to things. Achim explained that the W3C annotation framework is using “selectors” to address parts of an identified object. See CDI-4 by Achim for more on this discussion. Achim added to this that if a similar approach is used this could be used for every specification (CDI, Codebook, Lifecycle).

3.      Issues overview from Larry/Hilde (review and process for dealing with these)

Larry has been reviewing all DMT issues marked as ‘resolved’. He has classified these issues as to whether he feels they should be re-opened or could remain closed. He sent out a spreadsheet post-meeting including his proposals for the group to look at. This will be adressed at the MRT meeting on February 24^th. Many of these issues are MRT tasks created in connection with Sprints etc.. A question is how to handle such process related tasks in the future.

Hilde has send out a workbook that contains a set of older pre-Berlin Sprint issues and asked each of the issue creators to review them.

Virtual MRT meeting 2021-01-27

Agenda (as of email from Arofan of 2021.01.27)

1. Quick update on discussion with Training Outreach (following on from Webinars)
2. Report from side-meeting on issues resolution (Larry?)
3. Organization of sub-groups
   1. Sub-groups
   2. Process
4. Examples and high-level documentation (and other documentation?)
5. Timing and game plan
6. Other areas (if not touched on in 1 above, from the list of areas we summarized last week)

Minutes, virtual MRT meeting 2021.01.27

Attendees

Achim, Arofan, Dan, Hilde, Larry, Wendy

Apologies

Flavio, Jay, Oliver

Admin

Arofan informed about the following:

• There will be regular MRT meetings at the usual time (Wednesdays at 10:00 Eastern/16:00 CET). Next week’s MRT meeting will be cancelled, as the first meeting of the new SB is scheduled for the usual MRT time. Achim and Hilde will make sure that a different timing will be chosen for forthcoming SB meetings.
• Jared is organizing a training session on DDI for the Dublin Core Metadata Initiative. Instructors will be Jared (introduction), Barry (Lifecycle), Jane (Codebook) and Arofan (CDI). Achim presented DDI at a DC conference in 2008, which Arofan will reference in his talk.

1.      Quick update on discussion with Training Outreach (following on from Webinars)

Based on the experiences from the CDI webinars held in collaboration with CODATA and the training sessions held at EDDI and the FAIR Convergence Symposium in December, a series of web-based training events is under planning by a collaboration between the CDI webinar team and the Training Opportunities group. CODATA has offered to use their webinar platform and to help with administrative tasks, which we view as a highly appreciated contribution from our new member of the DDI Alliance. A program that includes a series of webinar on different topics is under planning. The plan is to have this sent out to the MRT group, the new SB, the Marketing team and the Training group, before the next meeting of the Traing group, that will take place on February 2^nd.

2.      Report from side-meeting on issues resolution (Larry)

A small group consisting of Achim, Dan, George, Hilde, Jay and Larry met on Tuesday January 26^th to discuss resolution of Jira issues. Larry sent minutes to the srg list after the meeting.

a. Organization of sub-groups

Larry reported that small groups have been formed at the meeting on the 26^th to find solutions some of the important issues:

• Identification (CDI-3): Achim will ask Flavio to work together on this. There is agreement about a solution, but an efficient implementation needs to be discussed.
• Annotation (CDI-4): Achim, Dan, Jay and Larry signed up for this group. Achim has provided a proposal for solution. Arofan has been asked to convene the group. Jay will contribute in relation to schema.org.
• Dimensions (CDI-16): Dan is responsible for this. Arofan will be in the group and Flavio will be asked to contribute as well.
• Issues resolution follow up group: Larry and Hilde will follow up on this by categorizing issues and propose priorities. Wendy volunteered to help regarding older issues that are not closed, dating before the MRT period.

b. Process

At the meeting on 26th, George proposed a consensus approach for making agreements regarding issues resolution: Each sub-group prepares proposals to which the members of the larger group is asked to indicate their consent, and where no comment is interpreted as an agreement. This procedure was tentatively agreed at the MRT meeting.

3.      Examples and high-level documentation (and other documentation?)

a. Examples

Arofan mentioned the importance of having worked examples that explores areas of the spec, as well as the syntax representations. This could work as a sanity check of the model.

Possible examples that mentioned were:

• Australian Election Study
• Units of measure
• org/DCAT mapping (person outside MRT)
• JSON-LD (person(s) outside MRT)
• Usage of CDI together with Lifecycle and Codebook
• Possible examples emerging from the EOSC DDI-CDI project run by Simon, Achim and Arofan

Examples could also be used as a basis for future training material.

A question is which of the examples could be automated rather than being made by hand.

b. Documentation

The high level documentation (specification) can be finalized when agreed changes on the model have been implemented. Larry explained that new diagrams needs to be produced when the changes the model have been made. Arofan pointed out that the naming/use of terms in the model might need a review. This has to do with the way we communicate about the model. Dan means, on the other hand, that the current model uses precise terms.

Arofan will be responsible for the high level documentation, and will provide an overview of which chapters of the spec that is likely to change most.

Hilde will look into the field level documentation together with others in the MRT group, prior to the release.

4.       Timing and game plan

The sub-groups as specified under point 2 will meet in the coming days to start their work and will report back to the larger MRT group at the next meeting.

Larry will start to look into the Australian Election Study examples as well as the blood pressure example. A goal is to produce an R package for DDI-CDI.

At the meeting on the 26^th Larry explained about an agreed association that had disappeared from the model. To be able to detect similar deviances before the release, Achim will look into possibilities to produce a diff programme that could document changes to the model between versions.

Virtual MRT meeting 2020.01.21

Agenda (as of email from Arofan of 2021.01.19)

1. Summary of review inputs

    - Issue tracking

    - Webinar inputs

2. Topics for revision work

a) Mappings to other DDI versions/worked example

b) Process model and impact of recent discussions (Convergence Symposium)

c) Discussions with GO FAIR (FIPs, FAIR Data Objects/Points)

d) Representation (XML? RDF?)

e) DCAT mapping

f) Annotation and Citation (including access, licensing, notes, etc.)

g) Other open issues (IDs, string clean-up/rationalization, etc.)

h) Documentation work

i) Input from EOSC project

3. Scope and Timeline

4. Additional people

Minutes, virtual MRT meeting 2020.01.20

Attendees

Achim, Arofan, Dan, Flavio, Hilde, Jay, Larry, Wendy

Apologies

Oliver, Simon

1.      Summary of review inputs

The main topic of the meeting was to look into the range of work, as well as the scope and timeline for the publication of the CDI.

    - Issue tracking

See point 2 below.

    - Webinar inputs

People from the MRT group held last autumn a series of CDI focused webinars in collaboration with CODATA that received lots of attention. We took part in activities of the FAIR convergence Symposium and taught DDI-CDI at EDDI. This resulted in interest in DDI/CDI and possibilities for further collaboration.

Achim raised the question about what to do about CDI webinars that were planned but have not yet taken place. We need to decide which of those to follow up on. Larry pointed out we need to distinguish between which ones would be needed for the first publication (areas where we need input) and which ones can wait till after.

A discussion followed about the Upper model (Jay) about a possible extension to handle infrastructures around processing. Jay will pursue this, and follow up with George Alter to discuss the possibility of taking this on board. That is however not likely to happen for the first release of the CDI.

2. Topics for revision work

a)      Mappings to other DDI versions/worked example

A worked example about how CDI and Lifecycle can work together should be planned for.

b)      Process model and impact of recent discussions (Convergence Symposium)

Possible bugs need to be addressed.

c)       Discussions with GO FAIR (FIPs, FAIR Data Objects/Points)

Arofan, Achim and Hilde have been in contact with Erik Schultes and Barbara Magagna from the FAIR team regarding how FIPs etc. technically fits with CDI, and how CDI possibly could be used in their ecosystem. A meeting has been agreed but has not yet taken place.

d)      Representation (XML? RDF?)

Representations need more focus. Would a webinar (as planned for earlier) be a good way to move forward with this?

e)       DCAT mapping

Alejandra Gonzales Beltran is working on a mapping between CDI and the latest version of DCAT. Flavio who has experience of this from similar work at StatCan will look into this when final.

f)        Annotation and Citation (including access, licensing, notes, etc.)

Needs to be looked further into.

g)      Other open issues (IDs, string clean-up/rationalization, etc.)

Jira issues fielded while the review process so far need to be looked into. (Available at https://ddi-alliance.atlassian.net/wiki/spaces/DDI4/pages/897155138/DDI-CDI+Comments).

h)      Documentation work

Cleanup of the model documentation needs to be performed prior to the publication.

i)        Input from EOSC project

CODATA is responsible. Achim and Arofan are contributing from the MRT/CDI side. The project is looking into how DDI-CDI can be used in a data sharing environment. Output from the project will be formalized as requirements that will be fed back to the MRT team. The project plans to deliver in April 2021.

4.      Scope and Timeline

• Wendy laid out the timeline needed for the publication process. The voting requires one month. Then the TC will need a week or two to prepare for the publication. If substantial changes made are an additional review of those changes will need four to six weeks.
• The question is then whether to publish in the shorter term (Q2), and get back with an update not so long after, or wait a little longer (Q3) and possibly bring in more content. This will needed to be agreed in the short term.
• Communication with the TC regarding decisions about how to proceed is important.

5.      Additional people

This point was not addressed.

Action points

• Weekly MRT meetings will again be held.
• Discuss which CDI webinars to follow up on.
• Decide regarding scope of work and publication.
• Larry will convene a group that will look into how the issues raised can be fixed and the process to implement the fixes and improvements (Achim, Flavio, Hilde, Larry, possibly Dan, other interested?).

Virtual MRT meeting 2020-04-29

Agenda

Identify target groups for review webinars on basis of spreadsheets and input from Simon Hodson of CODATA regarding break-down of the scientific unions and relevant standards.

MRT meeting minutes 2020-04-29

Attendees

Achim, Arofan, Dan G., Hilde, Jay, Larry, Oliver

Apologies

Flavio, Wendy

Discussions with CODATA

Arofan reported about discussions between Achim, Simon H. and himself regarding review of the DDI-CDI. Simon H. is very supportive of the DDI-CDI, and is willing to help promoting the review. He has provided an email list of engaged people involved in standards work in different domains within the RDA, and is willing to help further.

Webinar planning

The next question is how to group interested reviewers in topic groups for the webinars under planning. We are considering 6 to max. 12 webinars to be held in connection with the review.

Arofan proposed for the weekly MRT meeting in the forthcoming time to replace the weekly MRT meetings. All MRT members would be welcome to attend the webinars to be organised for each topic group of possible reviewers, and after each there will be a 15 minutes MRT discussion.

This was agreed by the group.

A discussion then followed regarding which topic groups could be split up or collapsed, looking at the MRT spreadsheet of contacts by topic, and Simon H.’s list.

Geoscience data was considered as a candidate for a stand-alone webinar, while life sciences for example, could be a candidate to merge? Larry introduced the idea to organize webinars according to topics that could be interesting for people from different domains, like for example handling of binary formats/streaming.

Arofan then proposed to start the webinar series with some clearly defined topic groups, like official statistics and health sciences and learn from that to plan forward.

Next MRT meeting and follow-up work

Arofan proposed to hold the MRT meeting next week as usual.

Side meetings will be organized in order to reorder the topic groups and assign people to relevant webinars, as well as to start to prepare material for the presentations.

Virtual MRT meeting 2020-04-22

Agenda (as of email to srg list of 2020-04-22)

1. Administrative 
   1. Presentation to EB
   2. Discussion w/CODATA
2. Reviewer spreadsheet/recruitment
3. Dagstuhl planning (in light of the pandemic)
4. Webinars for DDI-CDI review

MRT meeting minutes 2020-04-22

Attendees

Achim, Arofan, Dan G., Flavio, Hilde, Jay, Larry, Oliver, Wendy

Apologies

Jon

1. Administrative 

a. Presentation to EB

Arofan will report about the DDI-CDI release at the next EB meeting on May 4th.

b. Discussion w/CODATA

Achim and Arofan are in the process of discussing recruitment of reviewers with CODATA. The MRT group sees this as a great opportunity for promoting the DDI-CDI.

CODATA has put information about the DDI-CDI Public Review Release on their website https://codata.org/.

Dan G. proposed to look into possibilities for collaboration with other bodies as well and will look into options.

2. Reviewer spreadsheet/recruitment

Achim has provided a set of spreadsheets for organizing the review work.

• An email-list spreadsheet, containing email addresses to send the release announcement to.
• A feedback spreadsheet capturing received feedback from possible reviewers.
• A reviewers by topic spreadsheet where group members can fill in proposals for people and organizations to contact for the review.

3. Dagstuhl planning (in light of the pandemic)

Achim, Arofan, Steve and Simon H. are planning two Dagstuhl workshops scheduled to October 2020. The goal of the first workshop is technical testing of the DDI-CDI. The MRT group and external experts and experts from different domains will be invited. The second is a follow-up of last year’s workshop on interoperability of metadata standards in cross-domain science.

The plan for DDI-CDI is to organise the public review first, and then to update the public review version based on comments from the review. An intermediate release is scheduled to August. The aim oft he first Dagstuhl workshop would then be to have an extensive review of the interim version.

In light oft he pandemic it is still not sure if the Dagstuhl workshops can be organized as face-to-face sprint meetings as planned, and virtual alternatives needs to be considered. A question is how the review groups then best could be formed.

Flavio shared his experiences from the GSIM review, where sprint like groups were convened and worked remotely for 6-8 hours a day for a 4 day’s meeting. According to Flavio this worked, but was less efficient than a face-to-face sprint meeting would have been. Jay also reported similar experiences from meetings he has been involved in.

Jay asked if a project related to the RDA-COVID 19 he is involved with could be used as an example at the first or the second workshop. When Jay explained his example further,  Arofan recommended the examples to be used in both workshops.

4. Webinars for DDI-CDI review

Webinars by topic/audiences will be organized to introduce interested reviewers to the DDI-CDI. Adapted slide decks will need to be developed. People from the MRT group were asked by Arofan to present at the webinars, as well as about experiences from using different webinar platforms. Achim and Arofan will follow up on this.

Training material developed and used for the webinars could usefully be forwarded to the Training group. Dan G. will be the contact for this. A question is whether related material should be provided to the training group before or after the DDI-CDI has been published.

MRT meeting minutes 2020-04-15

Agenda (as of email to srg list of 2020-04-15)

1. TC review finalization:

- Text for website (review)

- Text for DDI Community e-mail/list announcement (review)

- Text for non-DDI Community e-mail/list announcement (review)

2- Planning for reviewer recruitment -  DDI-CDI reviewers spreadsheet

3- Planning for Webinars

Attendees

Achim, Arofan, Dan G., Flavio, Hilde, Jay, Larry, Oliver, Wendy

Apologies

Jon

1. TC review finalization:

- Text for website (review)

Arofan, Wendy and Jared have been corresponding about the website text. A new page for developing products of the DDI Alliance has been created, where the text related to the DDI-CDI will be found. The draft text was discussed at the meeting, and two minor suggestions for amendments were agreed. The revised text was sent by Arofan to Jared post-meeting, and the web site was updated accordingly.  

- Text for DDI Community e-mail/list announcement (review)

Wendy had drafted a text for this, where the only thing to change was the end date for the review period, that was changed from the 30^th of June to the 31^st July. Wendy will send the text to Barry and Jared for sending to the DDI Community e-mail lists.

- Text for non-DDI Community e-mail/list announcement (review)

Achim has drafted an email for non-community groups, based on Wendy’s proposal. This will be sent to the groups by MRT, and will also be sent to Jared and Barry for their information.

2. Planning for reviewer recruitment -  DDI-CDI reviewers spreadsheet

The spreadsheet (on google drive) has been updated since the last meeting. Potential reviewers are grouped by topics. The plan is to organize webinars for the identified topic groups. See below.

3- Planning for Webinars

Arofan suggested the following regarding webinars for potential reviewers from the different audiences:

• Webinars to start one to two weeks after the public review release has happened.
• A simple tool could be used to conduct the webinars.
• The webinars should be organized for smaller groups of potential reviewers, content being adapted to each group.
• 2-3 MRT members are expected to take part in each webinar.
• Content should consist of a 20 minutes introductory DDI-CDI presentation, plus discussions.
• Name and email addresses of interested reviewers to be collected.

This was agreed by the MRT group.

Flavio and Hilde were asked to distribute DDI-CDI recently held, to the group.

Virtual MRT meeting 2020-04-08

Agenda (as of email from Achim to srg list of 2020-04-08)

• DDI-CDI: decision on ready to roll, announcement
• DDI Alliance website: see proposal in Arofan’s email from yesterday
• DDI-CDI reviewers spreadsheet

Attendees

Achim, Hilde, Jay, Larry, Oliver, Wendy

 Apologies

Arofan, Flavio, Jon

MRT meeting minutes 2020-04-08

Achim was moderating as Arofan had a time conflict.

DDI-CDI: decision on ready to roll, announcement

The DDI-CDI public review package is ready to be published, the comments from Wendy and the TC taken on board.

Wendy har prepared a general announcement email for the DDI Alliance lists and IASSIST. A question is whether the same email will be sent to other groups as well or if amendments would be needed. Achim will talk to Arofan about this.

DDI Alliance website: see proposal in Arofan’s email from yesterday

A text proposal for a new section on the DDI Alliance web site provided by Arofan 2020-04-07 was discussed. Larry had some minor suggestions for additions. Larry wrote back to Arofan post-meeting, and after input from the marketing group and the TC the text can now be regarded as final.

DDI-CDI reviewers spreadsheet

A new reviewer’s spreadsheet by topic has been developed by Achim with input from Arofan and Hilde.  Potential reviewers to address was discussed. Group members were asked to add ideas for reviewers to the spreadsheet (in google drive).

So far the following topics are listed:

• Official Statistics
• RDA
• W3C Semantic Web
• UML Modeling
• DDI Community
• Semantic Interoperability and
  Conceptual Framework
• Policy Monitoring Indicators
• Infectious Diseases
• Resilient Cities

The plan is to arrange webinars for each topic, or combination of topics. Each topic group will get an MRT contact person for the review.

Achim asked the group about useful tools for webinar usage Larry then explained about PowerPoint audio narration that he uses when teaching classes online, and about Keltura that can be used to make videos.

Zoom will be used for the webinars, unless a better suited platform will be available.

Other

NADDI will this year be organized as a virtual event, and Wendy informed that a call for items to the program will be sent out. Options for possible virtual DDI-CDI presentations for NADDI was discussed. Arofan had proposed that an introductory DDI-CDI tutorial could be useful as well.  

Virtual MRT meeting 2020-04-01

Agenda (as of email to srg list of 2020-04-01)

1. TC review package corrections/progress
2. Planning for reviewer recruitment - Hilde's spreadsheet
3. DDI Alliance Website text and presentation

Attendees

Achim, Arofan, Dan G., Flavio, Hilde, Jay, Larry, Oliver, Wendy

Apologies

Jon

MRT meeting minutes 2020-04-01

Note on scheduled Dagstuhl workshop, October 4 – 9

The DDI-CDI will be out for review soon. After the review period amendments based on outcomes from the review will be made, and a new intermediate version of the specification will be provided.

A Dagstuhl workshop with the topic ‘Review from multiple domains’ is scheduled to October 4 - 9. Achim and Arofan have started to plan for this regarding what to review and who to invite. In addition to people from different domains, the MRT group will be invited.

Some ideas for topics/perspectives are:

• Integration
• DataDescription
• Interplay with other specifications
• Computational application processes
• UML review/interplay with other domain agnostic specifications (W3C)

Arofan asked the MRT group to get back with ideas on topics and people to invite for the planned workshop.

1.      TC review package corrections/progress

Achim received a list of things to correct from Wendy. This has been fixed post-meeting and Achim has sent an update of the package to Wendy.

Topics were related to the following:

• Mix in diagrams in PDF and EA file.
• Verification of links in the package Overview.
• Diagram notation to be introduced before the diagrams.
• XML schema root. This should be CDI rather than DDI.
• Shorten pathnames for the XSD documentation. 
• Add a note on the review page (already existent in the package) regarding the Javascript issue when using the Filed Level documentation locally.
• Add a note related to the examples regarding possible limitations.
• Namespace: Achim has proposed to use URL for this as W3C clearly states that namespace should be an URI, while DDI-C and DDI-L uses a string for this. It was agreed to make the namespace an URL in line with the W3C recommendations, and to make a notification that this needs to be adapted in DDI-C and DDI-L too, so that all the products are in line with W3C.

2.      Planning for reviewer recruitment

At the last MRT meeting people were encouraged to come up with ideas for people/projects to contact regarding the review. Larry and Dan G. provided some input, and Achim sent a comprehensive list of projects and people to contact to the srg email list prior to the meeting.

It was agreed to arrange introductory webinars for possible reviewers/teams organised by topic.

Ideas for topic groups are:

1. Official Statistics
2. RDA
3. W3C/Semantic Web
4. UML Modeling
5. DDI Community
6. CODATA Semantic Interoperability and Conceptual Framework
7. CODATA Policy Monitoring Indicators
8. CODATA Infectious Disease
9. CODATA Resilient Cities
10. Others (to be assigned): CESSDA, SSHOC, DataOne, Machine Learning, Computational Replication

Achim, Arofan and Hilde will do follow up work on this.

3. DDI Alliance Website text and presentation

Arofan is looking into this.

Virtual MRT meeting 2020-03-25

Agenda (as of email to srg list of 2020-03-25)

1. TC issues with review package and managing their resolution
2. Planning for reviewer recruitment - Hilde
3. DDI Alliance Website text and presentation

 Attendees

Achim, Arofan, Dan G., Hilde, Jay, Larry, Oliver, Wendy

Apologies

Flavio, Jon

MRT meeting minutes 2020-03-25

1.      TC issues with review package and managing their resolution

Achim will be the gatekeeper of the package and will liaise with Wendy regarding this.

Wendy and the TC is looking out for information related things, specific XML things, obvious broken things and errors.

The review is scheduled to last till end of June. If there would be a delay of the release with a couple of days the period could according Wendy be expanded till the end of July.

One of the issues detected was related to the unzipping issues with long path names for the XSD documentation. Achim will provide a new package with shorter path names, and will add a related note on the review page. In the meantime, it works to unzip the package in a path with a short pathname like “c:\ddi”.

Wendy also commented that in the schema included in the first version of the delivery package an URL was used for the namespace, whereas xmlns for DDI products usually has the form ‘ddi:name_of_xsd:version number’. Achim and Larry suggested it would be useful to have something that could resolve directly for review purposes if a related web page is put up. Arofan suggested that common practice is used for now. Wendy will send an example of this to Achim.

The name of the xsd was then discussed. It was agreed to use CDI for this purpose (plus something to indicate that this is a review version?).

Another issue that has been detected pre-release regards differences between the diagrams in the PDF and the EA documentation. Achim will look into this as well.

Wendy will get back to Achim with further issues detected by the TC while this week. The release by end of March is still a goal.

2. Planning for reviewer recruitment – Hilde

Recruitment of reviewers for the DDI-CDI was discussed. MRT are encouraged to contribute ideas for possible reviewers.

Virtual MRT meeting 2020-03-18

Agenda

Finalizing the review package and looking forward to getting reviewers.

Virtual MRT meeting minutes 2020-03-18

Attendees

Achim, Arofan, Dan G., Hilde, Jay, Larry, Oliver, Wendy

Apologies

Flavio

Finalizing the review package

Arofan and Achim are preparing the package for download and online access. Achim will prepare a bitbucket store for online access and will liaise with Oliver regarding that. This task has been completed post meeting.

Wendy and the TC will need at least a week to prepare the package for the review. End of March is still a still a goal for the DDI-CDI to be out for the public release.

Looking forward to getting reviewers

A plan for approaching interested reviewers and teams is under development.

Virtual MRT meeting 2020-03-11

Agenda (as of email to srg list) of 2020-03-11)

1. Finalizing the review package revision and preparing for handling comments.
2. List of known issues/dispositions
3. FAQ and DDI Alliance Site
4. Planning for recruiting external reviewers

MRT minutes 2020-03-11

Attendees

Achim, Arofan, Dan G., Hilde, Jay, Larry, Wendy

Apologies

Flavio, Jon, Oliver

1.      Finalizing the review package revision and preparing for handling comments.

Arofan, Achim and Hilde had discussed the release preparations in advance of the meeting.

Packaging of deliverables: Arofan is currently working on the packaging, and will send the final package to Wendy accompanied by a list of changes.

Known issues: The package includes a list of known major issues. These will also be available as Jira issues under the DDI-CDI project. The issues listed relates to handling of identification, annotation, as well as the simplification of strings. It was agreed at the meeting to include these major issues as part of the public review deliverable for information purposes.

XSD and examples: The XSD has been updated by Jay to support examples. All examples now validate.

Diagrams in the HTML documentation: Achim will include diagrams from the EA file in the ralted UML HTML documentation.

Architecture document: Sequence of content could usefully be changed. This will however not happen until the intermediate release scheduled to July.

Credits: Arofan will prepare the credits document for the deliverable.

BitBucket storage: Achim will make the zip file available directly from BitBucket, and will provide a link to the repository.

Follow-up of reviewers: A discussion followed regarding follow-up of reviewers. Hilde will be the main contact for reviewers entering comments or issues by email, and will write back to them to let them know that that their comments have been received.

Wendy recommended that it would be useful to email commenters by email in where comments are entered as issues in Jira. This was agreed at the meeting. Hilde will follow up on this.

Further follow-up on comments and issues will depend on perceived needs related to specific issues.

Wendy proposed that no DDI-CDI issues in Jira are closed before the end of the review period. This was agreed.

2.      List of known issues/dispositions

A list of open MRT related issues exist in Jira under the DMT project. These will be handled after the publication of the deliverable for the public review release. Hilde will send a link to these issues to the group for comments. A small group will then review the issues and the related comments and close solved or outdated issues.

3.      FAQ and DDI Alliance Site

The web-site group will be contacted regarding this. Possibly also the marketing group.

4. Planning for recruiting external reviewers

Options for recruiting potential reviewers were discussed. Plans will be developed in the coming time.

Virtual MRT Meeting 2020-03-04

Agenda (as of email to srg list as of 2020-03-04)

1. Finalizing the review package and preparing for handling comments. (TC has identified a couple of issues, and there is a short list of known ones.)
2. Organizing for the future:
   1. Timeline (as discussed)
   2. Plans for engaging reviewers

MRT meeting minutes 2020-03-04

Attendees

Achim, Arofan, Dan G., Hilde, Jay, Larry, Oliver, Wendy

Apologies

Flavio, Jon

1.      Finalizing the review package and preparing for handling comments. (TC has identified a couple of issues, and there is a short list of known ones.)

Delivery package:

The public review delivery package for the DDI-CDI has been sent to the TC for internal review and publication.

Wendy has provided a set of comments where some have already been dealt with, and the some will be dealt with prior to the publication. She has also provided a draft of the review page for discussion (see discussion below), as well as a DDI-CDI review issues tracker in Jira.

The DDI-CDI delivery package will be made available from the review page when agreed updates have been made to the package and the page.

At the meeting Dan G. commented that it would be a good idea to include diagrams in the UML field level documentation to be made available. Diagrams could be included in a separate folder and made available as HTML picture element, for example. Arofan will look into this.

Arofan presented a to-do list of things that remains to be looked into before the package can be published. The list was discussed with Achim and Hilde in advance.

• Credits and licensing must be improved with wording. Arofan will take on this. Achim has sent the necessary information.
• Achim will update the EA file with EA diagrams used in the specification documents.
• Achim is working on a list of open issues that he will deliver to Arofan for editing. The list will make up part of the delivery package.

• Achim is providing a bitbucket repository for storage and retrieval.
• Improved Canonical XMI files need to be integrated in the package. They are already available in Goggle drive.

• Improved documentation sets for UML and XSD is updated and available in Bitbucket and links from there are provided. The online documentation will be made available from a HTML release note page included in the delivery package.

• Arofan will write a note on UML documentation that there could be Javascript security issues with local use.

• Jay will fix diagrams on Architecture document – Arofan will include them in the specification document.
• Arofan will send XML examples plus schema updates to Achim

Jay mentioned at the meeting that the schema location needs to be changed in his examples. A (model related) tweak on the XSD would also needed in order to make some of the examples work. Jay, Achim and Arofan will meet to address the implementation of the change.

 Review page:

Arofan, Achim and Hilde had reviewed Wendy’s proposal for the page in advance, and provided a proposal for changes to the draft page. The proposal included a.o. incorporating  information about known open issues in the delivery package, as well to provide a sub-page for providing review comments in Jira or by email. The sub-page to be accessible from the main page.  It was also proposed that links to the online filed level documentation are included in the delivery package itself. The proposal for changes to the draft review page was agreed at the meeting.

It was also agreed that the full delivery package will be provided from the page, and that the content Overview of the delivery package will be duplicated on the review page.

In addition Larry proposed to link up specification documents (Introduction, Architecture, Detailed Model, Examples) directly on the page. This was agreed as well.

Wendy will make the edits to the review page with input from Hilde and the group.

Other:

Achim will create a Google Group email address for capturing review comments sent by email. Hilde agreed to coordinate review feedback entered by email.

Hilde will liaise with Achim regarding the Google Group email address and the review follow-up, and send the email address to Wendy for publication on the review page.

2. Organizing for the future:

a. Timeline (as discussed)

According to Wendy the TC will take on the DDI-CDI publication for review while the week of March 16^th. This gives the MRT group a week to perform the final tweaks. Wendy stresses that the importance of having a clear approach for reviewers regarding what to do and how to go about to do it.

The next step will be to prepare for an interim July release where important comments from the review are taken on board. A working area will be set up in Confluence where a copy of the public review package will be worked on.

Virtual MRT meeting 2020-02-26

Agenda as of email to srg list as of 2020-02-06

1. Administrative items
2. Packaging Proposal for Review
   1. - Outline (see attached)
   2. - Credits and licensing
3. Review page
4. Open issues list

MRT meeting minutes 2020-02-16

Attendees

Achim, Arofan, Dan G., Flavio, Hilde, Jay, Larry, Oliver

Apologies

Jon, Wendy

1.      Administrative items

Deliverables for the release: Arofan is happy with the material for the deliverable received from the group. Things go well together and the specification looks more coherent than earlier DDI specs. The final editing of the spec. for the public review release is currently ongoing.

Dagstuhl workshops: Achim informed there will be two DDI related Dagstuhl workshops this autumn.

• The first is a DDI-CDI workshop, focusing on DDI-CDI review from multiple domains. This workshop will take place from October 4^th through 9^th.
• The second workshop will have focus on interoperability of metadata standards in cross-domain science, health and social science application. This workshop is organized in collaboration with CODATA.

The MRT group will be invited to take part in the first workshop. Attendance for the second workshop is still under discussion.

2. Packaging Proposal for Review

a. - Outline (see attached)

Arofan sent out an outline of the DDI-CDI Download Package by email to the srg group on February 26^th. The content of this email is included in the Appendix.

It was agreed in the group to distribute the complete delivery package in a single download. This will be made accessible from a HTML landing web page that will be referenced from the TC review site. An overview of the deliverables explaining the content of the package will also be made available from the landing page. Arofan will be writing the text for this.

b. - Credits and licensing

Credits: The following was agreed:

• To list MRT members and their institutions.
• To list all DDI Moving Forward, DDI4 and MRT Sprint workshops since 2012 and the organizations contributing to their funding.
• To list all previous (DDI Moving Forward/DDI4) working groups.
• To mention Dagsuhl, that hosted nine DDI4 related workshops, subsidized costs and provided free seminar rooms.

Achim will draft a proposal for the crediting and send to Arofam.

Licensing: The DDI-CDI will be licensed under aCreative Commons Attribution 4.0 International License.

Achim has provided Arofan with further details.

3. Review page

Wendy is developing a draft of the TC review page in collaboration with Arofan and Hilde.

Wendy has also suggested to provide a CDI project in Jira for the purpose of capturing the comments from the review filed as issues. This idea of Wendy’s was agreed in the group. Hilde will ask Wendy to set up the CDI issue tracker in Jira.

Arofan and Hilde will further get back to Wendy regarding the TC review page when the work on the delivery package is completed. Hilde and Wendy will then coordinate the publication and send out info to the group.

Jay raised the question whether we will know who downloads the package for the review. There was a discussion regarding this, and an agreement was made that it would be better to be able to download the package anonymously, without having to provide an email address.

On the other hand it would be good to know the email addresses of people providing review comments in Jira. Hilde will ask Wendy if this will be possible.

4. Open issues list

Achim is working on an open issues list that will be provided to Arofan on Friday 28^th.

Other

Normative/non conformative references: Which references are normative and which are not has not been discussed so far. Dan G. was asked to do a job on this for the final release, starting after this release. This is particularly important for the sections of the DDI-CDI specification that regards Standards alignment and the Foundational part.

Upper model: The Upper model will not be made available in the Canonical XMI and for consistency purposes not in EA either. It will however be described in a related chapter of the specification document. Related diagrams will be added to the document.

Appendix

Email from Arofan to the srg list as of 2020-02-26:

Outline of the DDI-CDI Download Package

Directory: DDI-CDI Public Review 1

• File: Overview.htm describing and linking (locally) to package contents
• Directory: Specification Documents
  • Files: Document I – IV of the specification, in PDF format
  • Directory: Supporting Documents
    • UML Subset document
    • Others as needed
• Directory: Model
  • Files: 2 XMI versions, one with unique relationship names, one without
  • File: Enterprise Architect file
  • File: Release notes (if needed) in PDF
  • Directory: Field-Level Doc
    • Files: HTML Field-Level doc for the model
• Directory: Syntax Representation
  • File: XSD
  • File: Release notes (if needed) in PDF
  • Directory: XML Schema Doc
    • Files: HTML XML Schema documentation
  • Directory: Examples
    • Files: XML examples
• Directory: Examples (if needed)
  • File: 00ReadMe.txt
  • Files: Any examples other than the XML examples (which would be referenced from the ReadMe.txt file)

Virtual MRT meeting 2020-02-19

Agenda as of email to srg list of 2020-02-19

1. Administrative items
2. Documentation status

- Field level

- High level

1. Model freeze/final modifications
2. Process for updating diagrams, round 2

Attendees

Achim, Arofan, Flavio, Hilde, Jay, Larry, Oliver, Wendy

Apologies

Dan G., Jon

MRT meeting minutes 2020-02-19

1.      Administrative items

Product name

The new product name DDI Cross Domain Integration, or DDI-CDI has now been approved by the EB.

Barry has asked Arofan for an update of the on the DDI product page and the related FAQ to be provided.

The working name DDI4 Core will be changed to DDI Cross Domain Integration/DDI-CDI in all deliverables for the public review release.

Review site preparation

Wendy is preparing a standard TC public review site as a placeholder for the public review delivery package. She will liaise with Hilde regarding this. A link to the site was provided post meeting. A list of deliverables will be provided.

2. Documentation status

- Field level

Hilde has been updating the field level documentation of the ClassLibrary of the frozen EA file (of 2020-02-16), based on input from MRT colleagues. Detected issues will be flagged as known in a related release document.

Post meeting notes: The documented EA file version available was on Friday 21. Based on the file Achim has produced and made available related deliverables (Canonical XMI, XSD, EA documentation etc.).

- High level

Roughly all needed documentation sections are provided, but need editing. As the release date is rapidly approaching a thorough pre-release MRT review will not be possible timewise.

Achim is working on a document explaining some of the aspects of UML, for example what the arrows mean.

The specification will have a form closer to that of other specifications (e.g. W3C) compared to earlier DDI releases.

2.      Model freeze/final modifications

The frozen model need some final modifications before publication for the public review release that regards which packages to keep in the file. It was agreed at the meeting to remove the ModelManagement section and the SupportingDiagrams from the public review version of the file. Regarding diagrams, see 4. below.

RDF will not be available for the public review release, but Eric Prud’hommeaux will be contacted regarding this after the release.

4. Process for updating diagrams, round 2

Larry has created diagrams for the DataDescription package from the frozen EA file. The diagrams are in EMF (vector) format. They will be integrated in the frozen EA model if possible.

A discussion about the correspondence between the diagrams in the deliverable documents and those of the EA model took place. It was agreed that a correspondence between the diagrams in the model and those of the documents is needed to avoid confusion. Larry suggested that identical diagrams in the EA file and the documents carry the same name, which was agreed.  

Jay pointed out that currently different color codes are used for diagrams from different sections. Color codes will need to be standardized in the future. Larry proposed that for now each document section could include information about the color codes used as a reusable footnote. This approach was agreed by the group.

The format of pictures was also discussed. EMF vector files are useful, in particular when larger diagrams are displayed. A question that arose is whether there would be a MAC problem related to EMF or if relevant documentation formats to be used supports EMF. This needs clarification and Larry will liaise with Arofan regarding this.

Achim will look into the diagram sections of the frozen model with the aim to rationalize and get the diagrams better organized. It was agreed that all current and used diagrams should live in a single DDI-CDI Library diagrams folder in the EA file.

Arofan, Achim and Jay will liaise further regarding diagrams for the public review release.

Other

There was a discussion about outreach, and ideas that came up were to offer webinars/conference calls for interested parties, and as a post public review release activity, to contact organizations and groups that have shown interest in this new product of the DDI Alliance.

Virtual MRT meeting 2020-02-12

Agenda (as of email to srg list of 2020-02-12)

1. Administrative items
2. Documentation status

- Field level

- High level

3. Model freeze/XML representation

4. Process for updating diagrams

MRT meeting minutes 2020-02-12

Attendees

Achim, Arofan, Dan G., Hilde, Jay, Larry, Oliver, Wendy

Apologies

Flavio, Jon

1.      Administrative items

Product name

Arofan has sent a mini-report to Barry in the Marketing team that reflects the naming discussion in the MRT group. A new name for the DDI4 Core is needed in all the deliverables for the public review release, therefore a swift decision is required.

2. Documentation status

- Field level

Hilde has now received revised rsd files from most of packages for the release from her colleagues. Feedback on Long (Flavio) and StructuredDataTypes (Oliver) is expected in the near future.

When the frozen version of the Master EA file from Achim (scheduled to Friday 14^th evening) is available she will start the field level documentation upgrade on the file.

- High level

Data Description document

Hilde and Arofan have been working to reorganise and update the Dagstuhl version of the document. The document needs further refreshment of diagrams and their related content.

How to read the model?

A new document chapter about how to read the model will be developed. This will include explanations of what the arrows mean, as well as other content aiming at facilitating the understanding of how to read the model for new users.

Foundational document

This is fine for now. It will be revised when the other things are in place in order to make it consistent with the rest.

Architecture document

Achim will provide more content for this. Flavio is working on the CollectionPattern. Jay has provided the standards alignment part of the document, but will check this for updates.

Use-cases

Use-cases will be an ongoing activity. Jay has started this and 3 – 4 examples are now available in XML.

Dan G.’s Time Series example will also be included – including XML if time allows, otherwise it will be published as a conceptual use-case.

The examples will be made public when a storage has been provided for them.

2.      Model freeze/XML representation

Master EA model and examples

Jay and Achim has tested the model and the examples. Now things work!

The tested version includes Process updates from Jay as well as agreed model clean-ups from Achim. Main changes to Process regards the Scripting representation. The changes allow Process to do a lot of things, for example to handle processes like that of C2Metadata.

Process now integrates nicely with DataDescription and can act on it’s classes, as well as on the more foundational classes like variables, codelists etc.

The InformationObject class from the UpperModel will now be moved to DataDescription (Achim).

The Upper Model

The UpperModel is not part of the deliverable other than as an explanatory diagram. This needs to be verified to make sure the connection to the ongoing work is kept. After the public review release the upper model needs to be documented.

Model freeze

Achim will provide an EA Master model version for freeze on the evening of Friday 14^th. Some minor clean-up changes will be implemented before the freeze.

3.      Process for updating diagrams

Diagram presentations and updates

Achim explained about a feature in the EA that allows export of all diagrams in various formats including .png and .jpg. Larry suggested vector formats could be used. It was agreed to use .png format for the public review release and consider vector format later.

Larry pointed out that not all diagrams in the documents stems from real EA diagrams. Jay’s diagrams are snapshots of the model copied to clipboard to make them available in each section of the document. They will, however, be turned into EA diagrams after the public review release.

EA diagram updates

Larry agreed to update the diagrams for the Data Description document. The procedure will be as follows:

• Larry receives the frozen version of the Master EA that will be sent out on the evening of Friday 14^th. This will have been updated with a folder named ‘Data Description diagrams’ for entering of the updated diagrams.
• Larry makes a copy of the file where the suffix ‘has diagrams’ is added to the file name.
• He deletes outdated diagrams available in the ‘LH’ environment of the Sandbox.
• Updated diagrams for Data Description (with a specific name describing what is in it) provided by Larry will be stored in the ‘Data Description diagrams’ folder.
• Achim will include the updated DataDescription diagrams in the frozen Master EA file if possible to merge in. This would happen (if possible) after the documentation upgrade has been done.

Larry suggested that it would be good to have a connection between the diagrams in the EA model and the representation of the diagrams in the documents. Achim suggested that each diagram is given a specific name describing what is in it, and that the same name is used in the Word document. This was agreed by the group.

The work of the MRT group for the public review release continues in good progress.

Virtual MRT meeting 2020-02-06

Agenda

Follow-up from meeting 2020-02-05

MRT meeting minutes 2020-02-06

Attendees

Achim, Arofan, Hilde, Jay, Larry, Wendy

Apologies

Dan G., Flavio, Jon, Oliver

Arofan opened the meeting by saying that what would be needed for the prototype release is a model and a schema that people can understand and work and play around with. Lots of good work has been done on DataDescription, Process and the VariableCascade and it is important to get this out to people. Syntax representations can come later on. The Review process will bring useful input to the final product. What the model (PIM or PSM) is called is not important at this stage. In addition to the deliverable a test space could be provided. The model and the schema needs to be frozen for the release now.

Latest modeling updates: Achim received an updated version of the model that included needed changes for the Process model part that facilitates handling of components from DataDescription and Foundational. Achim has produced a Canonical XMI and an XSD (not valid) on basis of this. The Process model needs some further upgrade. In this take identifiable information will be added in on relevant classes, in order to be able to make examples. Jay will fix things that led to the invalid XMI and perform further changes and will liaise with Achim on Monday 10^th regarding the model. A new version of the model including updates to Process will be available on Tuesday 11^th . Testing can continue in a separate sandbox.

Model clean-up: When Achim receives the file from Jay on Tuesday he can continue with the model clean-up as far as time permits. On Friday 14^th he will hand the file over to Hilde who will start to edit the field level documentation (until Saturday 22^nd).

The following agreement was reached regarding what to edit, based on Achim’s email to the srg list as of 2020-02-05:

1. Leaf classes without attributes and associations to other classes
1. ControlStep, ScriptingAgent and RulesEngine - Drop
2. Sequence and Service - Leave
3. Designation - Drop
4. What is the purpose of UnitSegmentLayout? – Leave
5. PhysicalSegmentLocation has no attributes, only one association to ValueMapping, and only one sub class, SegmentByText. Currently it doesn’t make much sense. Should this be maintained for future expansion? – Drop for now
2. The current approach of string types seems to be too complicated. The structured string types could be folded into the regular ones. Furthermore, there are some inconsistencies.
   Diagrams on the current approach and the suggested approach are attached. – Merge existing string types if time allows.
3. Missing identifiers for classes in DataDescription, Process, and value? of a Code , see Jay’s email on “new model version 2020-02-04 / generated files” from 2020-02-04.
   Attached is a list of classes in DataDescription and Process. The classes which inherit from Identifable are highlighted. – Jay will add identifiers to DataDescription and Process. - Achim will liaise with Flavio on whether something needs to be changed in the other packages.
4. Identification and Annotation. Identification could be an attribute not a super class (otherwise “, annotation could be a separate class. – Implement identification as super classes for now and change to attributes later on. - No change regarding handling of Annotations.

Branding: A name for the product will be needed for next week. Arofan will follow up on this and send the MRT naming discussion to Barry.

 Virtual MRT meeting 2020-02-05

Agenda (as of email to srg list 2020-02-05)

1. Administrative items

- Branding

- FAQ/Site

1. Field-level doc status
2. High-level doc status
3. Model fixes/freeze (including XML Representation)

Attendees

Achim (last part of the meeting), Arofan, Dan G., Flavio, Hilde, Jay, Larry, Oliver, Wendy

Apologies

Dan G., Jon

MRT minutes 2020-02-05

1.      Administrative items

- Branding

   Arofan will send the MRT email discussion thread on branding to Barry. Then the Marketing group will then decide for a name on the consent of the EB.

- FAQ/Site

   Barry in the Marketing group has also asked the MRT group to update the Work Products of the DDI Alliance page with new content on the DDI4 Core (which name will change before the public review release), as well as the integrated FAQ. This task will be taken on after the     public review release has happened.

2.      Field-level doc status

Hilde has received Agent and Identification package rst files from Wendy, Process from Jay, Conceptual, Representations and FormatDescription from Dan G., as well as Components, Dimensional and the root classes of DataDescription (Datum etc.) from Flavio.

Flavio has also agreed to do Wide and Key value. Oliver agreed to look into StructuredDataTypes.

Hilde would need the remaining updated content packages in hand by the 12^th be able to integrate the content in the field-level documentation for the deliverable.

3.      High-level doc status

DataDescription part: Arofan and Hilde are looking into this.

Process: Jay is responsible for this.

Foundational: The document is good but needs some positioning. Arofan will contact Dan G. regarding this.

Architecture: Achim and Jay have provided content for this. Flavio’s chapter on design patterns to follow. A draft needs to be available by February 22nd.

4.      Model fixes/freeze (including XML Representation)

In advance of the meeting Achim had sent a proposal for alternatives for model fixes with attachments by email to the srg list (see the Appendix). This includes among others a proposal to treat Identifiable as attributes rather than super-classes. A decision is needed regarding what of this to implement now, and how.

Jay argued that when testing Process issues were revealed that would require more testing to make this part of the model workable, and wanted some timespace for this.

Arofan pointed out that the model needs to be frozen on the 14^th for the field level documentation to be done, and that Jay needs to liaise with Achim regarding possible changes to the model.

Arofan proposed that a Wiki page will be made available where examples can be updated along with the review, this way separating the normative from the non-normative parts of the deliverable. This was agreed.

A discussion about how to handle identifiers followed. Options for how to fix inheritance for now discussed at the meeting were:

• Add identification as attributes for classes that miss identification, and change identification by inheritance to identification as attributes for those classes where this applies.
• Add identification as attributes for classes that miss identification and leave classes with identification by inheritance as they are.
• As a temporary solution, add identification by inheritance to classes that miss identification (Larry).

Larry argued annotations should be handled needs a further discussion. Arofan then proposed to decouple annotation from identification.

Jay expressed no strong opinions of this, while Arofan argued for the value of Achim’s proposal to make identifiable attributes to avoid class hierarchies in the model. Simple IDs could be implemented for all instances.

The approach was discussed in relation to time. Arofan argued Achim had indicated willingness to do the job and that this should be discussed further with Achim on his arrival at the meeting.

Jay argued then that Process would needs some further modeling and wanted space while the coming week for testing and brainstorming with Flavio till next Wednesday.

This would leave Achim only two days for the model clean-up fixes. On his arrival at the meeting Achim indicated that this would not leave him enough time to implement identification consistently and do other fixes.

Time did not allow the group to reach agreement on whether Process modeling can continue before the release, how to deal with identification, or on other of  Achim’s proposals for clean ups.

Therefore, a follow-up meeting on Thursday 6^th at 11 Ottawa & New Hampshire/17 CET was scheduled.

Appendix

Email from Achim to the srg list 2020-02-05:

Here is a list of open model issues.

The goal would be that we either find today an agreement or we describe the issue and possible solutions and park the issue.

I’ll then make changes to the model according to the agreed decisions.

1. Leaf classes without attributes and associations to other classes
   1. ControlStep, ScriptingAgent and RulesEngine
      1. Jay: they can be removed
   2. Sequence and Service
      1. Jay: need to remain as is in line with idea that this is a conceptual/logical model that some but not all will use for implementation too (Smile).
         Sequence is a container for other ControlConstructs and because it inherits from ControlLogic which is reflexive maybe Sequence can host other ControlLogic and itself.
         Service is an idea we would like to finish at another time.
   3. Designation
      1. Ja: Designation is another touchpoint with Signifier, Sign and Signified
      2. Flavio: We are not using Designation anymore

• Dan: Designation isn’t really needed since we have the signification pattern. As a result, each application of designation, whether it be a code, term, or other, will be modeled in the realization. Designation, being a generic term, probably is not needed.

1. What is the purpose of UnitSegmentLayout? Does it make sense for future expansion.

1. PhysicalSegmentLocation has no attributes, only one association to ValueMapping, and only one sub class, SegmentByText. Currently it doesn’t make much sense. Should this be maintained for future expansion?
2. The current approach of string types seems to be too complicated. The structured string types could be folded into the regular ones. Furthermore, there are some inconsistencies.
   Diagrams on the current approach and the suggested approach are attached.
3. Missing identifiers for classes in DataDescription, Process, and value? of a Code , see Jay’s email on “new model version 2020-02-04 / generated files” from 2020-02-04.
   Attached is a list of classes in DataDescription and Process. The classes which inherit from Identifable are highlighted.
4. Identification and Annotation. Identification could be an attribute not a super class (otherwise “, annotation could be a separate class.
   The current model uses Identifiable and AnnotatedIdentifiable as super classes. The Identifable is just “Data-Based Inheritance”. There are currently 16 classes which inherit directly from Identifiable and 23 classes which inherit directly from AnnotatedIdentifiable. There are overall 110 classes which inherit from other classes and finally from Identifiable.
   The information which is captured by Identifiable can be modeled as a data type which could be used as attribute with the name “identifier” for each class. Even the information which is captured by AnnotatedIdentifiable can be modeled as a data type. The exception is the association to AgentAssociation. There are four m:n associations from AnnotatedIdentifiable to AgentAssociation, each with some kind of role (publisher, contributor, versioning agent, creator). The weird thing is that AgentAssociation itself has again a role item (external CV).
   I see here some requirement for discussion how this could be simplified. Diagrams on the current approach and one suggested approach are attached.
   1. The information of Identifiable and AnnotatedIdentifiable could be collapsed into one structured data type with the exception of AgentAssociation.
      Annotation could be an attribute of Identifier. This would be more limited than the current solution in terms of the annotation relationship.
   2. Another approach would be to have associations from classes which need Annotation. One annotation could be then used for many classes. A related super class could be constructed. The latter approach would make distinction between identification (which would be an attribute) and annotation (to which an association would exist).
   3. A third approach would be to include the identification information in the super class described in b). This is similar to the current approach but Annotation is not tied to a single class. This seems to be a flexible solution.

Achim

Virtual MRT meeting 2019-01-29

Agenda (as of email to the srg list as of 2019.01.29)

1. Administrative items
2. Tasks for review finalization
3. Field documentation status
4. Document status
5. Model clean-up (Achim's proposal)

MRT meeting minutes 2019-01-29

Attendees

Achim, Arofan, Dan G., Flavio, Hilde, Jay, Larry, Oliver, Wendy

Apologies

Jon

1.      Administrative items

Branding: Arofan and Wendy met with the Marketing Team that proposes to have a new name for the DDI4 Core. Arofan will get back regarding what is decided. (Lively discussions by group members regarding the name followed post meeting).

2.      Tasks for review finalization

The delivery date for the TC is set to February 28^th. Arofan will need to have the frozen model with field level documentation included on the 19^th by the very latest.

Achim will have the navigation cleanup done by the end of the week and will provide an updated EA file, Canonical XMI, XSD and rst files.

When Hilde receives updates field level documentation files (rst) from the group members she will include them in the updated rst files. Achim will continue with model clean-ups till this is done and content is ready to be copied into EA.

For documents, see the documents section.

3.      Field documentation status

Hilde had received updated Agent and Identification packages from Wendy, Process package from Jay before meeting. Dan delivered updated Conceptual, Representations and FormatDescription files  post-meeting . Input from Flavio on the DataDescription packages and StructuredDataTypes is expected in the coming days.

4.      Document status

DataDescription: This needs restructuring and an introduction. Work is expected to happen in the coming days. Hilde and Arofan will liaise on this.

Foundational: A cross-walk from the UpperModel to this is needed. The document needs to get a little more easily accessible. Dan G. will look into this and finalise within the next two weeks.

Architecture documents: Jay has responsibility for the standards alignment. Achim takes care of the UML sub-set used as well as PIM/PSM. Flavio on the CollectionsPatterns piece.

Examples: Jay and Chifundo are currently working on examples on data management (HDDS), standards alignment, and data transformations between formats. Some of the examples are dependent on a valid XSD, and some possible iterations may be required.

Flavio mentioned that a ‘fake object model’ also could be useful, to have something on the model level.

It remains to be decided which of the examples to include in the model.

Documents final for review by the MRT group will be made available on the new Public Review Preparation pages in Confluence.

5. Model clean-up (Achim's proposal)

Achim has a proposal for further model clean-ups, among those the role of Identifiable and AnnotatedIdentifiable as super classes. Identification Instead could be made an attribute. And how to handle Annotation best? Regarding Annotation Larry commented that it is important that several object can have the same annotation. Achim will lay out his proposal by email to the group later.

Virtual MRT meeting 2020-01-22

Agenda (as of email to srg list of 2020-01-22)

1. Administrative items/feedback from EB
2. Status of the specification/overview of current docs
3. Plan for finalizing the model and completing inline doc review/incorporation
4. Plan for finalizing specification docs and deliverables
5. RDF syntax representation discussion

MRT meeting minutes 2020-01-22

Attendees

Achim, Arofan, Dan G., Hilde, Jay, Larry, Oliver, Wendy

Apologies

Flavio, Jon

1.      Administrative items/feedback from EB

EB meeting: The report provided by Arofan for the EB meeting on January 21^st was well received.

Achim informed there was a naming discussion and it was decided that the Marketing group will liaise with Arofan to find a new name for the DDI4 Core.

Timeline: The public review deliverable will be sent to the TC on February 28^th for release. The work on content of the MRT group should be final by the 14^th. The TC will need some time for the publication after the review package has been received.

A longer period is expected for the public review. Wendy informed that four weeks is the minimum requirement of the TC for a review period, while three months is an average.

The MRT group is working on a timeline to prepare for the final public review release. The timeline will be made public at a later stage.

2.      Status of the specification/overview of current docs

The documents sent to the EB need upgrade for the release.

The introduction document is now quite good, but group members are still welcome to feedback comments or proposal for amendments.

The other documents need further amendments – see point 4 below.

3.      Plan for finalizing the model and completing inline doc review/incorporation

An open issue under discussion is whether to have a merely conceptual PIM would impact on how to handle directionality in EA.

For the public release, however, a functional and tested XSD with a corresponding variant of the model would be needed.

Arofan proposed the following approach for the public review:

• The latest version of the model in EA (as of 2019-12-19) will be used as a starting point. A variant of this will be created that implements the changes needed to produce a Canonical XMI that will serve as basis for a workable XSD. The changes to the current version of the EA file includes fixing the directionality of aggregations and compositions, as well as other changes that may be needed to be able to produce a workable XSD representation.

  Achim was assigned the task to change the directionality of aggregations and compositions as well as to liaise with Oliver and Jay regarding other possible changes. Iterative XSD testing and other checks may bring up issues that needs to be fixed in the model.

  A discussion on whether the amended variant of the model will be called a PIM or a PSM will depend on the changes made and will be taken up again later on, providing pros and cons of each.

  Diagrams based on the 2019-12-19 version of the file could possibly be provided that documents how a pure conceptual PIM would look like. Then people can input on what they would like to have while the review. This needs a decision.

• The documentation upgrade of class, property and attribute level documentation, will be performed on the updated variant of the model, as it would not be enough time to implement upgrade the documentation in two different variants of the model for the public release.

  Hilde will enter the revised documentation based on input as provided by the group members (rst files by package). So far rst files for Identification, Agent and Process has been provided. How to handle Enumerations, RegularExpressions and XMLSchemaDataTypes       needs  to be looked into. Achim will provide some advice.

The proposal of Arofan above was accepted by the group. No one objected strenuously against this proposal.

4.      Plan for finalizing specification docs and deliverables

The documentation upgrade work with the rst continues. Flavio will be asked to look at StructuredDataTypes in addition to the packages already been assigned to him.

The following tasks regarding upgrade of documents were assigned at the meeting:

Foundational: Dan G. was asked to provide a text piece bridging from the upper level documentation to the detailed Foundational document created at Dagstuhl.

DataDescription: Hilde was asked to look into the DataDescription part and make suggestions for ordering of content and needed updates of diagrams and content to be compliant with the latest version of the model. Larry already started this work.

The transformation between data structures might be included as usecase example in the Appendix (suggestion by Arofan).

Process: Jay and Arofan will look into this. A data management usecase example would be useful to include if possible.

Standards alignment: Jay has some new content developed by Cifundo and himself that regards mapping of DDI4 Core to DDI Codebook/NESSTAR, as well as to schema.org. Jay will liaise with Arofan regarding inclusion of this content.

Release timeline:  A release timeline, including steps towards the final public review release, is under planning by the group. This will be made public made when final.

5.      RDF syntax representation discussion

Eric has come back to Achim regarding transfer of the model transfer to RDF. It looks like it would now be possible to produce an RDF OWL representation based on the Canonical XMI. Achim will come back with more next week.

Virtual MRT meeting 2020-01-15

Agenda (as of email to srg list as of 2020-01-15)

1. Administrative items (if any)
2. Versioning issue update
3. In-line documentation
4. Model status & updating (directionality, etc.)
5. Document deliverables: outline/intro update

MRT meeting minutes 2020-01-15

Attendees

Achim, Arofan, Dan G., Flavio, Hilde, Jay, Larry, Wendy

Apologies

Jon, Oliver

1.      Administrative items (if any)

Tuesday January 21^st is the deliverable deadline for the progress report for the EB. The deliverable will contain an introduction document with a list of deliverables, high level documents on the Core detailed model, architecture, detailed examples and use-cases, XMI and XSD files, as well as field level documentation.

Achim will provide XMI, XSD, documentation and rst files for the EB deliverable. He will tentatively use the December 19^th version of the model for this. If immediate agreement is reached regarding how to handle navigability in the UML spec, and the decision involves to change the directionality of aggregations and compositions manually in EA, Achim has offered to take a stab on this if time allows, based on a frozen version of the model.  

The delivery of the review package will come later. We’ll need to have the work on the content for the package ready by mid-February. Then some days will be needed for the packaging and sending to the TC. The review package needs to be done with by end of February. Together with the deliverable a set of known issues will be published.

2.      Versioning issue update

The latest published version of the master model is that of December 19^th. Flavio will check if further amendments have happened since then.

Flavio explained that ‘Manage versions’ in Google Disk has been used to manage versions of the Master EA file. See how in the appendix.

Arofan asked Wendy for TC recommendations on where to handle version control of delivered files for the public review. Wendy recommended to use Bitbucket for this purpose.

3.      In-line documentation

Work is ongoing and Wendy has provided proposals for the Agents and Identification packages. Dan G. will work on FormatDescription, Conceptual and Representations.

The rst files provided for the work by Achim is based on a version of the model available by December 15^th. On some of the packages, for example Process have undergone important changes since then and the forthcoming updated rst files for those would helpful to complete the work.

Hilde is coordinating the class, attribute and association level documentation. She has also been asked by Arofan to do the copy/paste of revised documentation into the model in EA for the review version. The class, attribute and association level documentation will be further reviewed during the public review process of the deliverable.

Arofan pointed out that the UpperModel is meant as an introductory diagram for the review package. Class level documentation is thus not important for this purpose. However, at some point Jay provided some class level documentation for the UpperModel, which is not available in the current version of the master model. Jay and Flavio will liaise regarding this at a later point in time after the review package delivery.

4. Model status & updating (directionality, etc.)

An issue relating to navigability and directionality in the model that needs to be solved prior to the review deliverable was discussed at the meeting. Achim offered to change direction of aggregations and compositions manually in EA if there would be agreement in the group for that. Larry agreed to write up a proposal for how the navigability issue could be solved. This was sent by email to the srg list after the meeting. Achim responded positively to Larry’s proposal, while follow-up from Flavio is expected in the near future.

5.      Document deliverables: outline/intro update

Prior to the meeting, Arofan sent an updated version of the introduction document for the EB report to the group for comments, which includes main updates from Jay on the Upper model and Process.

In an email to the group prior to the meeting, Achim proposed some changes to the first introductory pages of the document, which in his view should convey the main messages to a broader audience.

Key features could be described as:

• domain-independent
• flexible data description (based on the description of a single datum/cell, also for new data type)
• provenance/process description
• foundational metadata (Variable Cascade, Format Description, Classifications)
• UML for interoperability (other standards) and sustainability (other representations)

These and other ideas of Achim's were approved in the group, and Arofan will take a stab to update the introduction document accordingly.

Appendix

Versions of the Master file on Google Disk can be found by clicking on the newest version of the file, and then on the ‘timer’ on the column to the right. See figure below.

Virtual MRT meeting 2020-01-08

Agenda (as of email to srg list of 2020-01-08)

1. Administrative items (if any)
2. Status and timeline
3. In-line documentation
4. Model status & updating
5. Document deliverables: outline/intro

        (1) The exact outline of the delivered spec (which is described in this document)

(2) Marketing and branding - there has been some discussion of this. We need to make a decision for the review at least. Also, I wanted to come up with a slogan for the header.

(3) Process of updating the diagrams. These (and consequently their explanation) is, I think, out of data. What will the process for finalization be?

(4) How do we credit members of the group? What licensing text do we use?

MRT meeting minutes 2020-01-08

Attendees

Achim, Arofan, Dan G., Flavio, Hilde, Jay, Larry, Oliver, Wendy

Apologies

Jon

1. Administrative items (if any)

There will be a meeting in the EB on January 21^st. Jared has asked for an update by then on where we are with the review deliverable. The update will be provided by Arofan in due time.

The next step is to finalise the review deliverable. Hilde agreed to prepare a new milestone page for the work in Confluence.

2. Status and timeline

Everybody is now back from vacation.  21^st of January will be too early for finalising the review delivery package, but work is gradually approaching this goal. February is a rough estimate for the deliverable.

3. In-line documentation

Work to manually upgrade the Class, Association and Properties documentation has started. Procedures are described in DMT-318. reStructuredText files for each model package provided by Achim on December 15th is used as basis for the work, as described in issue DMT-318. Possible issues affecting the current work is that the model has changed since the rst were produced (latest version 2019-12-19), and that some content from Lion might not have been transferred successfully to the current model.

Achim, Flavio and Hilde will look into how to best follow up on this.

Achim proposed additionally to separate definitions in reStructuredText from examples, synonyms, mappings etc. and provide a hierarchy of documentation files for different purposes and audiences. This idea is important and needs follow-up.

4. Model status & updating

The latest version of the Master eap file for the DDI4 Core was provided by Flavio on January 19^th. Decisions need to be made regarding directionality in EA, that needs to be set in order to get the navigability for aggregations and compositions right in the export to XMI and XSD. This is specific to EA.

In his emails to the srg list of 2019-12-17 Achim proposed to change the directionality of aggregations and compositions manually in the eap version of the PIM. Flavio agrees to do the manual update of the directionality in EA, but would prefer to make the model variant with set directions a PSM. Achim, however, would prefer not to make the PIM dependent on specific EA solutions. Achim and Flavio will look at this together, based on Achim’s findings on the topic from December and two examples from Flavio, and get back with a recommendation to the group.

It also needs to be decided whether Identifiable and AnnotatedIdentifiable sould be defined as structured data types and used as properties in the related classes, as suggested in issue DMT-298 . Achim and Flavio will look into this as well.

5. Document deliverables: outline/intro

(1) The exact outline of the delivered spec (which is described in this document)

Arofan sent out a revised version of the DDI4 Core Model introduction document to the group on the meeting day, that contains a proposal for an outline of the review version of the specification.

The group was asked to go through the outline till next time and provide feedback.

(2) Marketing and branding - there has been some discussion of this. We need to make a decision for the review at least. Also, I wanted to come up with a slogan for the header.

Comments have been raised regarding use of numbers in the names of DDI versions (DDI2.5, DDI 3.2, DDI4 and so on). To find a new product title for the ‘DDI4 Core’ needs to be considered.

(3) Process of updating the diagrams. These (and consequently their explanation) is, I think, out of data. What will the process for finalization be?

Some changes have been made to the model since Dagstuhl that affects the diagrams provided and the related documents. Arofan will get back to the group regarding contribution to the upgrade of diagrams and related documentation.

(4) How do we credit members of the group? What licensing text do we use?

Wendy sent an email to the group before the meeting where she reports about an existing agreement for licencing under CC BY 4 for all DDI products. In the email she also refers to the XKOS and DDI Lifecycle Readme as two examples of acknowledgement of contributors.

At the meeting the group was asked by Arofan to provide ideas on how to best acknowledge contributors for the DDI4 Core review deliverable.

Virtual MRT meeting 2019-12-18

Agenda (as of email to srg list of 2019-12-18)

(1) Timelines and deliverables reporting to EB

(2) Modelling issues and impact on XML representation

(3) Documentation clean-up

MRT meeting minutes 2019-12-18

Attendees

Arofan, Dan G., Flavio, Hilde, Larry, Wendy

Apologies

Achim, Jay, Oliver

(1) Timelines and deliverables reporting to EB

Reporting is due January 21^st. Arofan will report about the state of the art of the DDI4 Core in due time.

(2) Modelling issues and impact on XML representation

Flavio has been making good progress on the model clean-up. The new CollectionPattern is now in place and DataTypes have also been looked into. See email from Flavio to srg list of 2019-12-19 for a description of the post-meeting (2019-12-19 ) version of the EA Master file.

Navigability in aggregations and compositions when transformed to XMI is an open issue. Achim suggested in email to srg list of 2019-12-17 a way to handle this that consists in manually changing the ‘Direction’ in EA before export. At the meeting Flavio proposed an alternative approach that would be to programmatically change all aggregations and compositions in EA to associations. In a post-meeting email to the srg list of 2019-12-19 Flavio however expresses support of Achim’s approach.

To have the navigability fixed is important in order to provide XSD files for testing by Jay and possibly others. 

Another open issue regards the handling of Identifiable and AnnotatedIdentifiable that currently adds 1-2 additional levels in the inheritance chain. They could be defined as structured data types and used as properties in the related classes, as suggested in issue DMT-298. It has been agreed earlier to implement this.

Decisions on approaches to handle the navigability and inheritance related issues will be made when everybody is back from vacation.

(3) Documentation clean-up

The documentation per class, attribute and association needs review and update. The work needs to be carried out externally to the EA, for manually inclusion in the EA Master file later on.

In his email to the srg list of 2019-12-15 Achim provided documentation templates per class to be used for the documentation clean-up. Each template contains one package. The template indicates where necessary editing of the documentation per class, attribute, or association should be made. Edited and agreed sections will later be updated manually by copy/paste to the EA model.

The syntax of the templates is restructuredText (like in the prototype documentation). Achim suggests to use the online reStructuredText editor tool http://rst.ninjs.org/ for this work. This was agreed at the meeting.

Hilde agreed to coordinate the documentation per class, attribute and association update work, and will contact MRT members individually to assign tasks. It was agreed at the meeting that each package will be updated by one MRT person and reviewed by a different one. Alternative proposals for changes in the class, attribute and association documentation will be discussed in the group when needed. Procedures for the work is further specified in issue DMT-318 (created post meeting). The work is expected to start after the vacation.

Virtual MRT meeting 2019-12-11

Agenda (as of email to srg list of 2019-12-10)

1. EDDI report
2. Modelling issues from the list/model status
3. XML Representation testing
4. Open issues from list

Attendees

Achim, Arofan, Dan G., Flavio, Jay, Larry, Hilde, Oliver

Apologies

Jon, Wendy

1. EDDI report (Achim)

DDI4 at EDDI 2019

Achim chaired the session on the DDI4 Core at EDDI.

Flavio presented remotely in the session that went very well. In his presentation Flavio gave a high-level overview of the DDI4 Core, and provided examples of potential use of the DDI4 Core at StatCan (see Flavio's abstract).

After Flavio’s presentation Hilde presented data description with the DDI4 with examples. Her presentation was written with Larry (see Hilde and Larry's abstract).

For both presentations we got positive feedback from the audience. Our impression was that simple slides work best, and that new architectonical things take time to present. For a presentation of 20 minutes, 20 to max 25 slides would be appropriate. Further supplementary material can usefully be added to the end of the presentation for questions and later reference. The topics of the two presentations work well together but for a next time an idea could be to integrate them tighter. Another question is how to best present the capabilities of the DDI4 Core.

In addition to chairing the DDI4 Core session, Achim held a tutorial on generating syntax representations from the DDI4 Core with the Eclipse tools (see Achim's abstract). The slide deck for the tutorial contains more than 100 slides, and got positive feedback from the participants. Achim will get back to the group later regarding ideas related to this.

Achim also held a 10 minutes presentation of the DDI4 Core at Conference Plenary of EDDI.

IASSIST 2020

For IASSIST abstracts for similar presentations to those held at EDDI have been submitted by Flavio and Hilde. An idea is to include an example where process is used to convert e.g. wide format to aggregate in the data description presentation for IASSIST.

In addition Dan G. has submitted a proposal to present usage of the DDI4 Core data description in the modernization processes of the BLS.

2.      Modelling issues from the list/model status

Flavio updated the master model DDI4CorePIM EA file before the meeting 2019-12-11. Updates include:

• Work on the new CollectionPattern, moving all realizations into the new one. 2-3 classes remain to be updated.
• The DataStructures package has been renamed to DataDescription.
• Along with comments by Flavio in Jira issue DMT-315, LogicalDataDescription have been moved to the Legacy folder, some used classes like Datum and DataPoint has been moved to the DataDescription package root. LogicalRecord (abstract) is now a collection of InstanceVariables. This has now been moved to FormatsDescriptions.

By the meeting on Wednesday 11^th, cleanup of data types still remains to do on the model (see issue DMT-301). Wendy (not present at the meeting) offered earlier to check which are used based on XMI.

This could be done on the basis of the latest Canonical XMI provided by Achim.

Flavio will continue to work on the CollectionPattern and the StructuredDataTypes in the coming days. He will also fix some errors detected by Achim (see of email to srg list 2012-12-11 or the Appendix).

The ProcessPattern also needs to change location and be moved to a package as this is no longer a pattern. Larry commented that we need to make sure that abstracts are tagged.

To do

Flavio will continue to correct errors and provide two variants of the master PIM model before the weekend, one with Process moved to the ClassLibrary as a package and one without.

It should also be looked into how UML can be adjusted to be able to map more directly to schema.

3.      XML Representation testing

Achim provided an update of the XMI prior to the meeting, as well as an XSD file. The is not valid due to detected errors (see the  Appendix).

An important issue regards usage of non-unique association names.

When Flavio has provided an updated EA file (see 2 above), Achim will create a new Canonical XMI file with unique association names on the level of the model on the form subjectName-predicateName-objectName, possibly using machine generated acronyms for subjectName and objectName.

Achim, in collaboration Oliver, will then develop a new version of the XSD file based on the updated Canonical XMI. Non-XML data types needs to be fixed on the level of the schema.

When available Jay can then use this for test purposes.

4.      Open issues from list

Documentation structure

In email to the srg list of 2019-12-04, Dan G. provided a proposal regarding which documentation to include on the level of the class, property and association (see Appendix).

Some further discussion regarding this will be needed, as class name for example equals ID in Canonical XMI.

To do  

An XSLT transform of Canonical XMI content to markdown, applying the (possibly amended) proposal regarding which documentation to include on the level of the class, property and association from Dan G., could be used as a structure to add and edit content.

Edited content needs to be flagged.

Achim will look into this, after the Canonical XMI/XDS work is done. Discussions with Dan G., Larry and possibly others will be needed to complete this work.

Appendix

XSD issues detected by Achim; Extract of email to srg list 2012-12-11

Below are a couple of issues (visible in the XSD) which should be fixed in the model.

Achim

textContent (of StructuredDataTypes::DynamicText) is defined as DynamicTextContent

name (of DataDescription::Dimensional::DimensionalDataSet) is defined as objectName

name (of DataDescription::Dimensional::DimensionalDataStructure) is defined as ObjectName

overview (of DataDescription::Dimensional::Revision) is defined as InternationalStructuredString – maybe in wrong package defined?

frequency in DataDescription::Dimensional::ScopedMeasure is defined as char

name in DataDescription::Dimensional::ScopedMeasure is defined as ObjectName

changed to xs:string in XSD as workaround

data types in XMLSchemaDatatypes are not recognized as xs data types

type="anyURIType" changed to type="xs:anyURI"

type="languageType" changed to type="xs:language"

should be fixed in the transform to XSD

has (and others) are ambiguous as association names

Approaches for fixes must be discussed.

Email Dan G. to srg list 2012-12-04

MRT,

Here is a follow up to the email I sent last week about documenting the model. I propose the following template:

Class

ID

Name

Definition

Properties

                Name

Definition

Cardinality

Optionality

Datatype

Relationships

                Pointer to Relationship table

Example

Relationship

ID

Source Class ID

Cardinality

Optionality

Target Class ID

Cardinality

Optionality

Semantic

Name

Definition

Relationship type (relation: association, subtype, composition, aggregation)

Maybe we can consider this via email before the next meeting.

By optionality I mean whether something is required, optional, or conditional.

Dan

Virtual MRT meeting 2019-11-27

Agenda (as of email to srg list of 2019-11-27)

1. Administration (EDDI, etc.)
2. Status of drafts
3. Status of model
4. Status of code deliverables (XMI, XSD, etc.)

MRT meeting minutes 2019-11-27

Attendees

Achim, Arofan, Dan G., Flavio, Hilde, Jay, Larry, Oliver, Wendy

Apologies
Jon

1. Administration (EDDI, etc.)

EDDI 2019: Achim will chair the DDI4 session at EDDI.

Flavio will give the presentation ‘DDI 4 Core: describing and managing data for traditional and modern data platforms’ remotely. Hilde will subsequently present ‘Data description with the DDI 4 Core‘.

Achim will additionally give the tutorial titled ‘Generating DDI 4 in the Language of your Choice’at EDDI.

The presentations and the tutorial can later be used as supplementary material to the DDI4 Core deliverable.

IASSIST 2020: The deadline for abstracts for IASSIST is December 6th. Possible contributions from the MRT group were discussed. Dan G. has already sent in a proposal.

Flavio agreed to submit an abstract based on the same topic as that of the EDDI presentation.  Hilde agreed, depending on travel possibilities, to send in a proposal for a tutorial on data description in the DDI4 Core (hands on approach with actual instances).

2. Status of drafts

Arofan continues to work on the chapters for the DDI4 Core specification.

Achim will start to work on the architecture document after EDDI. He will also update and complete the part of the modeling guidelines that was created in Ottawa.  

Furthermore the examples need to be looked at.

3. Status of Model

Model update: Flavio will start the modeling upgrade work  again this week. The first things he will look into is the new collection pattern and the data types (see issue DMT-301).

Wendy offered to help to check where they are used. Flavio will send Wendy a check list of what he needs in advance.

Class/properties/association documentation:

A discussion about documentation on the class/properties/association level resulted in the following agreements:

Agreements

• Definitions, examples and explanatory notes will be added for classes.
• Synonyms, mappings to DDI 3.2, GSIM and RDF that was available for classes in Lion will be removed to a separate document. The ‘Trace’ relationships in the model will be used to identify mappings between DDI4 Core classes and classes belonging to other standards.
• For associations and properties only descriptions are needed.
• Larry will look into possibilities for generating UTF text files as structured as markup for all content (one folder pr. package). Then documentation on the class/properties/association level will be easy to review, see what is missing etc. Larry will use the same software as used for the DDI4 R package. If the work of Larry turns out to be complicated Achim and Oliver will look into other possibilities.
• New and updated content could be added in the text file and should be flagged with an asterisk, for example. New/revised content can then be entered in EA. Jay agreed to take on that.
• Dan G. together with Larry and Hilde will provide guidelines for forming definitions.

4. Status of code deliverables (XMI, XSD, etc.)

Flavio will let Achim know when the next version of the model has been published. Achim will then create a new Canonical XMI and send to Oliver. Oliver will then produce a new XSD file.

If the update of Flavio takes time Achim and Oliver will make the updates based on the current version of the master model (of November 17).

Sphinx will continue to be used for production purposes. Oliver is almost finished to bring Sphinx to use Canonical XMI.

Later Jay will start to test  the model on real examples.

Other

Correspondence between the DDI4 Core and DDI-C on the variable level

In Dagstuhl correspondence with DDI-C on the level of the variable was discussed, and it was agreed to add a defined set of classes with descriptive text to the model for the January release. These classes will be deprecated and removesd  along with further DDI4 developments.

How to relate to DDI-C on the study level is a bigger discussion that will happen post-release.

Meetings

The next meeting is December 11th. This will be the last MRT meeting of this year.

Virtual MRT meeting 2019-11-20

Agenda (as of email to srg list of 2019-11-19)

1. Administrative items
2. Progress on the model (Flavio)
3. XMI/Syntax representation issues
4. Update on issues (Hilde)
5. Upper model discussion (Jay)

Attendees

Achim, Arofan, Dan G., Flavio, Hilde, Jay, Larry, Wendy

Apologies

Jon, Oliver

MRT meeting minutes 2019-11-20

1. Administrative items

After this autumn’s Dagstuhl workshops the DDI4 Core is on its way to become an additional product of interest to several organisations and projects. One example is the ALPHA 2.

A related product, the DDI4 libraries R, has also become a product of interest.

The increasing interest from different parties makes it even more important that the MRT group delivers the specification within a due timeframe.

2. Progress on the model (Flavio)

Flavio has updated the ‘DDI4 Core-Master.eap’ file (last update 17th November).

He reports that the structural changes have been implemented. He has also be working on bug fixing and clean up related to the latest issues, as well as on updates on diagrams.

The class/association level documentation remains to be looked into.

Data types: In issue DMT-301 point 4. Achim reports that some classes are indeed only data types (no associations).They should be defined as structured data types not as classes.

We know which data types are wrong, but not where they have been used. An XMI scan can help to sort this out.

Documentation:

Classes/associations: Dan G. looked into definitions for  the Conceptual, Representations and LogicalDataDescription packages at the Dagstuhl workshop.  There are still parts of the model that lack content. In addition everything has to be reviewed.

Word or a spreadsheet can be used for updates and content can then be then copied to EA.

HTML documentation from EA will serve as basis to find out where documentation is lacking as well as for the review.

Defintion development rules need to be decided,  and a template for class/association level documentation upgrade and review needs to be provided.

The field level documentation in Lion containes more documentation on the level of the class than just the definition, for example practical examples. A similar template could be used for the DDI4 Core class/association level documentation. The template can be retreived  from markdown from Lion.

Agreements

• Data types; Issue DMT-301: Flavio will write to Achim and specify what he needs for the XMI scan. Achim will then provide Flavio with lists of things that needs to be fixed based on the scan. Wendy also offered to help with this. Flavio will make the necessary amendments based on the feedback from Achim.
• DMT-303, DMT-315: Flavio will provide lists for what needs to be checked.
• The next step for Flavio will then be to finalise the implementation of the new collection pattern. This is likely to happen in December.
• Dan G. and Hilde agreed to look into class/association level documentation development rules and to provide a template for the documentation upgrade and review
• Jay has offered to provide a document about the new collection pattern documentation.
• Achim agreed earlier to write the model architecture piece of the spec.

4. Update on issues (Hilde)

The below issues were discussed. Comments made at the meeting for each issue are found in the table of the appendix. It was decided to set DMT-302 and DMT-300 (both in green in the table below) to ‘resolved’.DMT-300: We need a clear description how aggregation is used in contrast to regular associations.

Virtual MRT meeting 2019-11-06

Agenda (as of email to srg list as of 2019-11-05)

1. Update on the DDI 4 Core Summary document
2. Modelling issues discussed at the MRT meeting 2019-10-30, issues updates from Achim
3. Other

MRT meeting minutes 2019-11-06

Attendees

Achim, Arofan, Flavio, Jay, Hilde, Larry, Oliver, Wendy

Apologies

Dan G., Jon

1. Update on the DDI 4 Core Summary document

Arofan has sent the ‘Looking forward to DDI4 Core’ summary document to Jared for forwarding to the EB and Barry in the Marketing group.

The document will be revised for an external public later on.

2. Modelling issues discussed at the MRT meeting 2019-10-30, issues updates from Achim

XSD schema:

Oliver has provided a more or less validating schema base don the latest Canonical XMI from Achim.

Detected issues related to the XML schema are recorded by Achim in issue DMT-303 ‘Model issues in transform to XSD: multiple defined properties and associations in super class/concrete class, missing data type etc.’.

According to Oliver these issues do not require much work to fix.

Oliver asked if document information is required to describe the context of the XML instance. It was agreed in the group to wait with this till January, and feed back to Oliver when this is ready for inclusion.

Model issues checks by Achim:

Achim performed several checks on the model since the last meeting . The following was discussed at this meeting.

Overall impression: No major issues

The checks reveal several minor things that need to be cleaned up, but no severe errors in the model were detected. Flavio will start on the tidy up work and make some structural changes to the current file regarding Design Patterns and packaging.

Association names

It was agreed to stick with the agreement of last MRTmeeting  to provide two variants of PIM, one with unique- and one with non-unique association names.

Quoting from Achim’s comment at issue DMT-299:

MRT meeting of Oct 30, 2019: same association names in one package are allowed for the purpose of readability.

There will be a related general purpose version of the Canonical XMI.

A second Canonical XMI version could have unique association names based on the existing association name combined with the related class names. This version can be consumed by UML tools which don’t allow non-unique association names in one package.

The agreement made at MRT meeting of Oct 30, 2019 was  confirmed at this meeting.

Unique association names should have the form ‘subject class name + association name + object class name.

Achim will try to implement programmatically Hilde’s idea to use acronyms for the subject- and object class names. This has second priority, though.

Navigability of aggregations and compositions

At the MRT meeting of Oct 30, 2019 it was tentatively agreed to remove navigability from associations in the model. Based on analyses by Achim regarding how unnavigability of aggregations and compositions perform in different UML tools and in the export to XMI, the group decided at this meeting to keep the navigability of aggregations and associations, but to create excisting aggregations and compositions anew.

We currently use the navigable end (arrow) to indicate the direction of associations. In the case of associations or composition navigability should be on the level of the ‘part’, that is at the opposite end of the ‘whole’ (indicated by the diamond). The opposite end is usually ‘unspecified’ in the current model. Some UML tools can have issues with that. Achim reports that in EA there is sometimes no distinction to see between unspecified and navigable or between unspecified and navigable. In Magicdraw unspecified navigability in an imported XMI results in a non-navigable state,which is an issue.

According to Achim a clear way to avoid the problem would be to first create a regular association in EA with an arrow on the ‘part’ side and then set the aggregate side to ‘shared’ using theassociation menue (role). The aggregation end should be set to aggregsation=shared.

There are currently 21 aggregations and 34 compositions in the model.

It was agreed that Flavio creates the aggregations and compositions anew in EA to get things standardised and correct.

PIM/PSM distinction

Moving towards January the role of the PSMs in relation ot the PIM will be more pronounced. Actual implementations will make the distinction between PIMs and PSMs clearer. Achim reports that a document regarding the PIM/PSM difference is in the process. This document will make up an important part of the specification.

3. Other

Flavio’s  EDDI presentation:

Flavio has submitted an abstract for a DDI4 Core related  presentation at EDDI that has been accepted by the programme committee .

Unfortunately it turns out that it will not be possible for Flavio to attend the EDDI conference in person this year . Flavio and the MRT group mean that the best alternative would be that Flavio gives the presentation remotely, with assistance from the session chair, and with backup possibilities.

Achim agreed to convey the view of the MRT group to the EDDI Programme Committee.

Dagstuhl week 1 delivery package for the EB:

Arofan has been preparing the delivery package for the EB from the DDI4 Core Sprint at Dagstuhl to be made available for the EB for their upcoming meeting.

MRT meeting 2019-10-30

Agenda (as of email to srg list of 2019-10-29) :

1. Administrativia
2. DDI 4 Core overview draft
3. Status on specification documentation
4. Status on the model
5. Status on syntax representations 
6. Modeling guidelines - planning for changes

Meeting minutes virtual MRT meeting 2019-10-30

Attendees

Achim, Arofan, Dan G., Flavio, Hilde, Jay, Larry, Oliver, Wendy

Apologies

Jon

2. DDI 4 Core overview draft

Arofan has sent an extended revision of the DDI4 Core summary document for review by the group. The document is scoped for the DDI community. He is expecting comments through Friday November 1st. The document will be sent to the EB in good time for their next meeting on November 7th. The MRT group is author of the document. The names of the group members will be included in a footnote.

3. Status on specification documentation

Larry has added a document about ‘Streams’ to the Dagstuhl Wiki page in Confluence. Arofan will include this as an example for the DDI4 Core report from Dagstuhl. Experts in the field will be asked to review later.

4. Status on the model/5. Status on syntax representations 

Achim has provided  a Canonical XMI file for the group to look at.

Oliver will use this to produce XSD files while this week. As Views are no longer in the XMI he will need to adapt the transformation to the new structure.

Both the XMI file and the XSD are planned to be included in the Dagstuhl delivery package for the EB.

6. Modeling guidelines - planning for changes

In advance of the meeting Achim had posted three isues related to modeling rules:

DMT-298 Inheritance chain - transform Identifiable to property defined as structured data type.

DMT-299 Association names must be unique in one package. Association names should be mandatory.

DMT-300 Aggregations don't add semantics. They should be removed. Some compositions have issues.

DMT-299 and 300 were discussed at the meeting.

DMT-299 Association names

 In Canonical XMI identifiers are based on association names. If they are non-unique within packages this is a real issue. Quoting from DMT-299:

• Associations without names could be considered by some UML tools as associations with same names which is not allowed.
• Many syntax representations need a name for an association. It doesn't make sense that each transformation for these syntax representations invent names. This can be error prone if different names are created.

According the modeling rules from Ottawa: each association should be understood as triple where the association name is the predicate and the object is navigable (arrow) in UML. The triple should be read as English sentence. This supports the understanding of the model and syntax representations like OWL/RDF.

        Many association names don't fulfill this requirement.

The status of the current association names is provided in the spreadsheet ‘Associations.xlsx’ attached to the issue.

In advance of the meeting Achim made a proposal for to solve this (sent by email to the srg list prior to the meeting). Requirements and the proposal is quoted below:

       There are multiple (possibly competing) requirements regarding association names:

1. Readability of associations in diagrams and documentation, i.e. no long names with class names included
2. Support of reuse of associations in some syntax representations like OWL/RDF (i.e. same associations should be recognized programmatically while the transformation)
3. The identifiers in Canonical XMI must be unique. These identifiers have a semantic structure based on the names of items like associations.
4. UML doesn’t allow non-unique names of items (like associations) in the same package. Some UML tools complain about this, some ignore it.

       Proposal which tries to address all requirements

1. The requirement one can be understood as the guidance because this has first priority. There is already the rule that associations - with the connected classes – should be understood as a simple sentence with subject (class one), association (predicate), and object (class two, navigavable, i.e. line has an arrow). The used association names should be reviewed according to this.
2. Same association names should be allowed in one package if they have really the same meaning. They could be then reused in a syntax representation. The reuse could be across packages.
3. The unique XMI ids can be generated while the transformation to Canonical XMI according to the rule that the ids are structured like “class1+assocation+class2”.
4. A second Canonical XMI version could be generated with unique association names according to the structure mentioned above with the ids. Unique association end names (roles) could be here addressed in a similar manner.
   This version can be offered for consumption by UML tools where the other Canonical XMI version produces errors.

Agreements

The full proposal was agreed by the group. Consistency needs to be handeled on the     level of the PIM.

Default names for missing association names for aggregations and compositions  should be set to ’has’.   

Task

Flavio will add non-excisting association names (‘has’) in EA for aggregation and composition , and upgrade other association names to comply with English grammar rules. He will also remove references to class names in the association names where this occurs in the EA PIM. The spreadsheet  ‘Associations.xlsx’ in DMT-299 serves as basis for the work.    

The upgraded EA file will serve as basis for the XMI related implementations in 3. and 4. of the proposal.

General comment regarding consitency checks

The reason for naming aggregations and compositions is to ensure consistency across implementations of the PIM.

Flavio comments that it is important to note that the DDI 4 Core spec goes beyond the PIM and that similar issues involving potential implementation inconsistencies arise elsewhere, e.g. constraints that are not machine-actionable, extensions of UML datatypes, etc. These could be interpreted differently by different developers creating interoperability problems similar to those of unnamed associations. We need to look at all the places in which there are non-machine-actionable constructs and find ways to ensure implementation consistency.

DMT-300 Aggregation/composition

Aggregation

The discussion regarding aggregation started with that it is not clear in our model what is the difference between aggregation and multiplicity >1 at one end. Either the difference should be explained or aggregation should be dropped. For software this would be the same thing.

Flavio explained that aggregation is just for users who look at part/whole relationships. This is only a visual help and has no programmatic impact.

Jay pointed out that aggregation is part of the UML specification and can be used for this reason.

Achim suggested that the type of model should impact on whether aggregation is used or not. If the model is explorative modellers recommend to include aggregations. If the model is an implementation model they recommend not to use them. Flavio proposed to view the DDI4 Core PIM as explorative as it is a conceptual model, while the PSMs would be closer to implementation models. On this basis it was agreed to keep aggregations in the PIM.

Aggregation, composition and navigability

Larry raised a possible issue regarding navigability of aggregation and composition that there might be differences in the model between things imported from Lion and drawn in the EA. As normal XMI parsing of associations is based on navigability. Flavio suggested to drop navigability of aggregations and compositions. This was tentatively was agreed by the group at the meeting.

Larry was asked to follow up on this and provide Achim with examples. Post meeting Larry found out that there was no real issue with this in the model (quoting from Larry’s  post meeting emai to the srg list):

                The role properties for these associations are consistent between imports from Lion and aggregations or compositions drawn in EA. Only the “Source” and “Destination” are  swapped.  In both cases the Navigability property is set to “navigable” on the end opposite                the one where the Aggregation property is set to “composite” or “shared” (the diamond end).

Post meeting Achim wrote (in a post-meeting email to the srg list):

We use the navigability of associations in the implementation of representations for the purpose of identifying (and versioning) an object. What does this then mean in the case of a composition without navigability? We might want to establish a rule how the navigability should be realized in the representation for a composition, i.e. a composition has a composite (which is navigable).

Agreements

Based on the fact that some group members find aggregation useful and that the DDI4 Core model can be perceived as explorative it was decided at the meeting to keep aggregations in the model.

Tentatively agreed to remove navigability in aggregations and compositions (see also the comments from Larry and Achim post-meeting above).

Task

Flavio and Achim will work together to document the use of aggregations and composition in the model.

MRT meeting 2019-10-23

 Agenda (as of email to srg list 2019-10-22):

1. Update on Sprint documents/overview
2. Report from TC meeting (Jay/Larry/Dan) - ?
3. Status on Model
4. Status on specification documents
5. Use of GANTT charts (Hilde)

Meeting minutes virtual MRT meeting 2019-10-23

Attendees

Achim, Arofan, Flavio, Hilde, Jay, Larry

Apologies

Dan G., Jon, Oliver, Wendy

1. Update on Sprint documents/overview

Arofan will provide a summary report early the coming week at latest. This will represent a snapshot of where we are. He will include an intro to the DataDocumentation part. The Patterns document will be included when the model is stable.

2. Report from TC meeting (Jay/Larry/)

DDI variants: There is currently consensus in the TC to continue to have multiple DDI variants in the future.

DDI4 Core – DDI-Lifecycle : Wendy started to work on a high level mapping.

COGS: COGS in its current state would not yet be suitable for adaption by the group for model  based production and management puropses, as some current model content would not be supported. It would for example not be possible to impose restriction on ‘source’ cardinalities for associations.

3. Status on Model

Larry proposes that newly developed structural components are included in EA to facilitate the production of the Canonical XMI. This was agreed by the group. It was also agreed to implement the new package structure agreed at Dagstuhl.

Achim will produce Canonical XMI based on the next stable version of the model (which includes the new structural components and has implemented the agreed package structure from Dagstuhl). He will then talk to Oliver regarding the implementation of syntax representations.

Achim and Larry will do analyses on the current model version in Canonical XMI and provide a list of things that need to be updated for the January release (like critical associations, aggregations, compositions etc.) or classes that need to be moved back from the Legacy folder of the current version of the model . The list will be distributed to everybody in the group and discussed by Achim, Flavio and Larry in a separate meeting.  Agreed changes need to be implemented and followed up to check  that everything has been done.

Another issue regards the direction of diamonds in compositions and aggregations. When drawing diagrams in EA, EA assumes a direction. However, Achim explains, ‘source’ and ‘target’ does not excist in UML.  Arrows have navigability that should be based on the predicate as well as the  versioning direction. New developments and amendments to the model since the prototype need to be looked into to check out usages of compositions/aggregations in relation to navigability.

Flavio will look into the Modeling guidelines to check out agreements regarding usage (or not) of aggregation, and check out the needs to update the model and/or guidelines.

Achim proposes that more guidelines on how to use EA is needed. This was agreed by the group.

4. Status on specification documents

The introduction document and the DataDescriptopn pieces have been reviewed by different group members.

Larry will write a part on Conceptual Value for January.

Dan G. s Time Series document will be included as an example. A short version of the Foundational document will be provided  for January.

5. Use of GANTT charts (Hilde)

Hilde tried out a free Gantt project tool for follow-up on the MRT tasks in the Core delivery phase. She will look further into tasks reporting possibilities in Jira and get back to the group with options.

MTR meeting 2019-10-16

Agenda as of email to srg list 2019-10-15

1. Taking Lion offline?
2. DDI 4 Core - DDI Lifecycle mappings: planning for TC Minneapolis meeting
3.  Review of Sprint and status report (including 2nd Dagstuhl week presentation)
4. Plan for moving forward on deliverables

MTR meeting minutes 2019-10-16

Attendees

Achim, Arofan, Dan G., Flavio, Hilde, Jay, Larry, Wendy

Apologies

Oliver, Jon

This was the first MRT using Zoom, link https://zoom.us/j/641763395

1. Taking Lion offline?

It was agreed to keep Lion up for a couple of weeks still as this could be useful for variuos purposes (review, complex data types, referencing for example).

According to Wendy it is no problem to keep Lion up for some more time.

2. DDI 4 Core - DDI Lifecycle mappings: planning for TC Minneapolis meeting

Mapping to earlier versions of DDI will be topic at the forthcoming TC meeting in Minneapolis. According to Wendy mapping will then be looked into with a broad focus. The discussion could feed into the work of the MRT group. A goal would be to be able to map DataDescription and  Process/Provenance  of the DDI4 Core to earlier versions of DDI (DDI-Codebook; DDI-Lifecycle), and  to have mappings available for the January release.

3. Review of Sprint and status report (including 2nd Dagstuhl week presentation)

First Dagstuhl week

Arofan summed up shortly from the 1st Dagstuhl week . All  goals of the meeting agenda were achieved.  The RDF representation, classified as ‘Nice to have’ in the agenda, remains open.

Second Dagstuhl week

Achim reported about great interest of the DDI4 Core as an integration model at the second dagstuhl week. The work started at week 2 will continue, with the goal for the DDI4 Core to be used in a larger context.

Arofan, Larry and Dan G. explained  about synergies with CODATA work, interest from other research domains in how the DDI4 Core deals with data types,  concordances with ontologies etc.    Interest was also shown by representatives from schema.org.

4. Plan for moving forward on deliverables

Hilde sent the following documents by email to the srg list prior to the meeting:

1) Dagstuhl Sprint overview of deliverables and tasks.docx - This document contains an overview of published deliverable related content, provides a summary of forthcoming tasks and lists Jira issues filed since the Ottawa Sprint.

2) Dagstuhl discussion regarding DDI4 Core compatibility with DDI Codebook_2.docx - This document describes topics addressed at the meeting with Larry regarding Codebook compatibility for the DDI4 Core. It also gives an overview of classes and packages moved to the Legacy folder of the latest version of the model in EA.

To follow up the work she will look into possible to use Gantt diagrams tools to track dependencies.

Follow-up tasks

Working documents for internal review by the Sprint team developed at the Dagstuhl Sprint (week 1) are available from the Dagstuhl Sprint Task pages. Arofan is currently preparing the deliverables from the Sprint for presentation for the EB.

The following agreements regarding  follow-up tasks (1 - 11 below) were made at the meeting:

1. Overview and Purpose (document)

2. The Upper Model for DDI 4 Core (diagram and document)                                                                                             

           Arofan’s draft document is published for review by the group. Larry has already commented. Others to comment.

       3. Foundational Classes - concepts, variables, categories, classifications, etc. as presented in the Prototype, but updated to agree with the current modeling guidelines (diagrams and document)

           Dan G. has drafted a detailed document. Arofan will use this as a basis to prepare the Dagstuhl deliverable. Group to comment. Dan G. will provide a high level document with examples will be available for the January release.

       4. Design Patterns in DDI 4 Core – an introduction to the design patterns used (diagrams and document)

           This task presupposes integration of the new/revised patterns in the model. The document should be based on the final version of Patterns for the model. Some post-Sprint discussions may be required for the finalization. See for example DMT-294.

        5. The DDI 4 Core Process Model (diagrams and document)

             Jay is currently working on two different documents. One of the documents represents a description. The other document is about how to use the Process model (Example View with basis in the work of the Alpha network).

             He will liaise with Flavio while the week and get back to the group.

         6. The DDI 4 Core Data Description Model (diagrams and document)

a. Core classes

b. Unit-record data

c. Tall/skinny data

d. Cube structures for dimensional data

e. Key-value/No SQL data

               Documents for all a. – e. has been provided by Larry. Arofan has reviewed the documents with approval. Other group members are asked to review as well.

          7. DDI 4 Core and External Standards (GSIM, GSBPM, DCAT, DDI-C, DDI-L, etc.) [NICE TO HAVE]

                a. Use of external standards for DDI users

                b. Use of DDI 4 Core for external users

                c. Mapping to DDI Codebook

                Arofan has produced a document that will be published on the Wiki when refined. Arofan will liaise with Jay regarding input to this from the second Dagstuhl week. A goal is to publish this document before the TC meeting in Minnesota next week.

          8. Functional Views in DDI 4 Core (exemplary views) (diagrams and document) [NICE TO HAVE]

                a. Data Management View, encompassing process/provenance and data description.

                b. Data Description View, covering the straight data descriptions case(s)

                c. External Standards View, showing how DDI can be used in conjunction with an external domain standard [specific example TBD]

                 Views are likely to be a little different than described in the agenda. Views are currently viewed as diagrams and related examples.

                 Proposal for Views:

-Data Management View (including basics of DataDescription) – An idea is to base this on the work of the ALPHA network. Include a collection of diagrams rather than a single diagram.

-Process/Provenance

-External Standards View – this is now likely to be more about mapping.

-Time series - Would it be better to include this as a View rather than as a separate data structure?

           9. Canonical XMI for DDI 4 Core (valid XML XMI describing the DDI 4 Core Model with documentation).

               A draft is available. Needs update along with forthcoming model updates and some clarifications regarding associations, names and roles. Achim is in dialogue with developers of the XMI specification and will feed outcomes of the discussions back to the group.

          10. XML Schema Definition (XSD) for the DDI 4 Core Model (XSD schemas with documentation)

               Achim will work with Oliver  on the generated documentation and the XML schema. Will feed back in a two weeks time.

           11. OWL/RDF Syntax Binding for the DDI 4 Core Model  (RDF vocabulary definition and documentation) [NICE TO HAVE]

                 RDF will be drafted for the January deliverable and the public review. Choices regarding RDF will not impact on the PIM but rather on the RDF PSM. Achim reminds that the usages of ‘trace’, ‘refine’ and possible ‘derive’ in the model should be expressed.

                 This is useful for the transformation of PIM to RDF PSM as well as for for external standards mapping.

Model freeze mid November

It was agreed by the group to freeze the structure of the model by mid November, as much of the work including the Canonical XMI, the syntax representations, the EA diagrams and examples as well as the documentation and reporting work is depending on a stable model. All classes, names, associations, properties and cardinalities should then  be in place.

Core Scope

While Dagstuhl a set of classes were moved from the Class library to a Legacy folder in EA (Table 2 in the ‘Dagstuhl discussion regarding DDI4 Core compatibility with DDI Codebook_2.docx’ document). The trimmed down version of the DDI4 Core will form basis for the January deliverable. A complete view of what will be in the model  should be available by mid November.

Virtual MRT meeting 2019-09-04

Agenda (as of email of to srg list 2019-09-03)

1. Admin
2. Dagstuhl Work Agenda/Deliverables - agree final draft
3. Prouction framework status
4. Modeling update (Flavio)
5. Aligning fundamental structures
6. Cube data features to include - general data model integration
7. High-Level Model Update (Jay)
8. Status of process model

Minutes DDI4 MRT Virtual meeting 2019-09-04

Attendees

Arofan, Dan G., Flavio, Jay, Hilde, Oliver (part of the meeting), Wendy

Apologies

Achim, Jon, Larry

2. Dagstuhl Work Agenda/Deliverables - agree final draft

Arofan presented an updated version of the agenda for the DDI4 Core workshop in Dagstuhl. Comments from group members at the meeting and after will be addressed for the version to be sent to the AG and further to the EB.

4. Modeling update (Flavio)

Flavio has been working to compile the Cube/multidimensional part of the model. A draft is available post-meeting (2019.09.05). The next step is to integrate Long (Tall/Skinny) and thereafter KeyValue. The integration of Key Value is scheduled to before Dagstuhl (one to two weeks of work). The integration of the Process model will take place after KeyValue.

5. High-Level Model Update (Jay)

A new version of the high-level is available. The high-model will need updates along with changes to the integrated model. Arofan points out that it is important to express clearly what the entry points are to the model. When people come from the SDMX or from RAIRD for example, it is important that people are able to recognize themselves in the landscape of the high-level model. Dan G. points out the need of mapping between vocabularies. Naming rules are also important (how to deal with classes with identical semantics but different names in DDI4 and other standards, or with identical names but deviating semantics).

Jay proposed to provide a ‘source’ class for each class for each model diagram. The source could class be a DDI class from earlier versions, a GSIM class etc.

6. Status of process model

Jay has updated the process model. His proposal is less deterministic that that of the prototype, and unnecessary inheritance has been removed. Arofan and Jay will have a take on the process model before handing over to Flavio for integration.

Forthcoming tasks

Arofan follows up with the group regarding the agenda for the Dagstuhl Sprint.

He will also contact Achim and Oliver regarding the production framework and the Canonical XMI.

The modeling and integration work continues with the aim to have as much as possible ready before the Dagstuhl Sprint.

Virtual MRT meeting 2019-08-14

Agenda (as of email of to srg list 2019-08-13)

1. Admin: (if any)
2. Marketing discussion (branding) - report (Arofan)
3. Overview of proposed high-level model (Jay)
4. Status update on modelling (Flavio)
5. Status update on data doc (Larry)

Minutes DDI4 MRT Virtual meeting 2019-08-14

Attendees

Arofan, Dan G., Flavio, Hilde, Jay, Larry, Wendy

Apologies

Achim, Jon, Oliver

2. Marketing discussion (branding) - report (Arofan)

Arofan and Wendy took part in the meeting of the Marketing and Partnerships group. DDI 4 Core branding was discussed. Arofan will set up a list of proposals for a new name based on suggestions from the MRT group. He will send around the list to the MRT group for agreement before sending it to the marketing group. The name needs to be decided before the DDI4 Core is published for review. This is planned as the first release of the DDI4 prototype, not of the whole of it. The name should preferably reflect this.

3. Overview of proposed high-level model (Jay)

Jay presented his ideas of a high level model of the DDI4 (developed in EA v14, see post-meeting email from Jay to the srg list of 2019-08-14). This includes process and data documentation that is currently within the scope and timeframe of the Core, as well as parts of the prototype to be addressed by the MRT post-core.

The background for Jay’s ideas for a high level model for DDI4 is based on follow-up work from last year’s interoperability workshop in Dagstuhl, with aims to create data in the context of the Sendai Accord. This led the team to take an inventory of data pipelines regionally, nationally and internationally on the one hand, and a description of the data products associated with those data pipelines on the other. The team was also asked to think in terms of data science and algorithms to be used in order to get the most out of the resources and the data that was available.

The result represents a way to put together constructs that people in the ALPHA network use in a high level model that provides flexibility using non-deterministic control strategies. The prototype model is also used as library to produce mash up diagrams for different purposes and audiences.

Flavio and Hilde commented for example that ControlConstruct elements like Sequences IfThenElse are now enumerations rather than classes and thus platform specific. Hilde asks what this would mean in terms of inter-operability and reuse of construct related components – for example Sequences.

Arofan proposed that the high level model in some form and level of detail could be of help in order to describe the scope of the technical specification of the DDI4 Core in relation to the full scope of the DDI4, and to prepare a roadmap for forthcoming MRT work. What to include and what is for the future in relation to the Core? What to do with what is in the prototype etc?

Arofan also stressed the importance to scope the DDI4 Core specification in relation to the other DDI products like Codebook, Lifecycle and the DDI4 prototype.

Larry asks how much of Jay’s model that will be covered by the Core deliverable. A question is how to deal in the short and long term with content that is currently not scoped for the Core. In this context Larry referred to two Jira issues he recently posted that regard correspondence with DDI4 and Codebook: DMT-283 and DMT-284.

Flavio argued that it is important not to broaden the scope of the Core much due to the anticipated workload within the scheduled timeframe.

Status: Jay sendt around related EA 14 files (can be viewed in free EA Viewer) plus a zip archive with related pictures of the diagrams for the MRT to look at and evaluate.

Ongoing work/next steps

Work on modeling and data documentation continues and regular meetings are held

 A high level draft of how the Core spec is organised is to be outlined

Virtual MRT meeting 2019-08-07

Agenda (as of email of to srg list 2019-08-06)

1. Admin: work agenda plans
2. Marketing update (Arofan)
3. Model integration (Jay/Flavio)
4. Data document update (Larry)

Attendees

Arofan, Dan G., Flavio, Hilde, Jay, Larry, Wendy

Apologies

Achim, Jon, Oliver

Minutes DDI4 MRT Virtual meeting 2019-08-07

1. Admin: work agenda plans

Arofan will draft an agenda for Dagstuhl while the week and send to the MRT group for comments and agreement. The proposed agenda should then be sent to the Advisory Project Management Group (AG) for comments before forwarding to the EB.

2. Marketing update (Arofan)

Arofan has received an invitation from Barry in the DDI Marketing and Partnership Group to take part in a meeting about branding of the DDI4 Core on Monday August 12^th.

He pointed out that a possible new name for the DDI4 should not be too ambitious and not exclude extensions to the DDI4 (for example of Data description and Qualitative) after the Core has been finalized.

Alternatives for new names for the DDI4 Core were discussed. Some of the alternatives mentioned were ‘DDI Interoperability’ to indicate the connection with the FAIR principles, ‘DDI Cross-Domain’ to indicate that the DDI4 Core is not just for social science, or ‘DDI Data Hub’ (see https://en.wikipedia.org/wiki/Data_hub)  focusing on the power of DDI4 Core to handle various types of data from multiple sources. Interoperability was turned down by the group as the earlier versions of DDI facilitate interoperability as well. It was also discussed if a possible new name should be followed by a Core (for example ‘DDI Cross-Domain Core’) to indicate that the Core represents the first step in finalizing DDI4.

Status: Arofan and Wendy will discuss DDI4 Core branding further with the Marketing and Partnership Group at their meeting on Monday 12^th.

3. Model integration (Jay/Flavio)

Jay has been working on Process modeling and a paper for Modern Stats. The ‘process model’ is split in two. Data Management is intended to support data flow descriptions. Instrument Management supports the description of questionnaire flows. So far only the Data Management model has been developed, relying on a series of design patterns. Datum is turned into a pattern in this new proposal. The Signification pattern is left out and a new way to model transformations between tall/skinny has been proposed.

A side comment from Hilde was, as Instrument Management process modeling was mentioned, that the Data Capture package of the prototype is not yet final and would need a take - a suggested post-Core activity.

Status: Jay will prepare his work on Process for review by the MRT group as a whole. The process model(s) will subsequently be tied to Flavio’s work on the Core.

4. Data document update (Larry)

Larry has received content from Dan G. regarding multidimensional/cube data for inclusion in the document. This task has a dependency challenge as the EA diagrams for inclusion in the document are time consuming to make and work of importance to the integrated model is still ongoing.

Flavio pointed out the importance to focus on the Datum in the data description of the DDI4 Core.

Other

A modeling meeting will be held on Thursday August 8^th, directly after the TC meeting. Arofan Dan G., Flavio, Jay, Larry and possibly others will take part in the meeting. Flavio will put up an agenda in advance of the meeting (included in email to srg list of 2019-08-08).

Some possible topics for the meeting were mentioned:

• Ongoing modeling tasks/integration of work.
• Maintenance of EA files, how to keep track of different variants and versions, model integration (Where, how, who).
• Model tidy up. Make classes and relationships distinct and avoid duplication of properties.

Other proposal for upcoming tasks:

• Documentation tidy up. Make sure that terms and definitions are clear and understandable, and consistently used (How, who, when).
• A glossary of terms ((How, who, when).

Virtual MRT meeting 2019-07-31

Agenda (as of email of to srg list 2019-07-30)

1. Admin: Work agenda for Dagstuhl (for EB)
2. Production framework status update, new requirements, issues (Oliver)
3. Modelling update: Proposal for Patterns and Diagrams (Arofan/Jay/Flavio)
4. Process model update (Jay)
5. Data model document update (Larry)
6. Standards alignment update (Jay)

Minutes DDI4 MRT Virtual meeting 2019-07-31

Attendees

Achim, Arofan, Dan G., Flavio, Hilde, Jay, Larry, Oliver, Wendy

Apologies

Jon

1. Admin: Work agenda for Dagstuhl (for EB)

Work agenda for Dagstuhl:  Work on the agenda will start and need to be completed in good time before the workshop.

Vacations: Achim and Oliver will be on vacation from August till the beginning of September. Hilde will be off last half of August.

Video conferencing tool: The TC will test out Zoom as a replacement for GoToMeeting as video conference tool. This will not affect the meetings of the MRT who continue with GoToMeeting for now, till the TC has made a final decision on which tool to use in future.

2. Production framework status update, new requirements, issues (Oliver)

Background info: Email to the srg list of Oliver via Wendy, of 2019-07-31 (included in appendix point 1).

Oliver has transformed the latest canonical XMI from the Core selection from the last Sprint meeting into XSD. After correcting some uniqueness violations he had a validating schema file.

One example is ‘takesPlatformSpecificSentinelValues’ object relationship of InstanceVariable that additionally is inherited from RepresentedVariable. The object relationship is duplicated, and the information from the classes inherited from is overridden.

The violations detected by Oliver should be solved on the level of the model, rather than on the level of the level of the schema. The issues need to be looked further into.

Flavio will look into this and identify situations where duplication exists in connection with inheritance vs. situations where duplication not related to inheritance exists.

Larry suggests to have a separate modelling meeting to solve this and other outstanding modelling issues (see below).

Status: Agreed

3. Modelling update: Proposal for Patterns and Diagrams (Arofan/Jay/Flavio)

Arofan pointed out that progress on the modeling is dependent on how to deal with patterns.

The proposal includes the following:

A simple heart of the Core deliverable should include the model in Canonical XMI and documentation with diagrams:

• One main variant of the model in Canonical XMI without the patterns, accompanied by descriptive diagrams where the patterns are not shown
• Another variant of the Canonical XMI for developers and advanced users where the patterns are kept in, accompanied by descriptive diagrams where patterns are kept in.

The background for this proposal is that some user groups find the patterns confusing.

Views should be documentary, each sub-set (View) relating to a particular use-case.

A discussion followed regarding the role of the diagrams, notes in diagrams, packaging and how to deal with the ‘realize’ association in the prototype on the short and the long term.

• Code reuse: Dan G. pointed out that design patterns supports code reuse and that it should be conveyed that code reuse is part of the design.
• Diagrams/crosswalk between variants: Flavio proposed that diagrams documenting the main variant of the model could be produced by hiding pattern related classes and associations in EA. The crosswalk between the two variants of the model could be done by removing the patterns in the ‘main’ model XMI variant.
• Packaging: Achim pointed out that it can be risky to do transformations between the two variants with Enterprise Architect, and will look into if a way of packaging to avoid this would be possible. (See explanation and results of first tests at point 2 and 3 in the Appendix).
• Notes in diagrams and Canonical XMI: Larry asked if notes in diagrams could be represented in Canonical XMI. There was a discussion and Achim agreed to look into this. See explanation and the results at point 2 of the Appendix.
• The ‘realize’ association: The ‘realize’ association for incorporating patterns from the prototype needs to be dealt with. Arofan proposes to keep the ‘realize’ association in for the first release of the Core due to time constraints. Achim points out the need to take on this, preferably on the model level, or at a minimum to post-process the XMI (see agreement below and the Appendix).

Agreement regarding release products: The group agreed that the products to be released would be:

(1)  A "user" model expressed in Canonical XMI, accompanied by diagrams and explanatory text, which would not include the design patterns

(2)  A "developer" model which would be the model expressed in Canonical XMI, accompanied by diagrams and explanatory text with the patterns in place

(3) Functional Views expressed as diagrams and explanatory text for a specific functional purpose, not accompanied by Canonical XMI or other machine-actionable representation at this time

Further, the current style of incorporating patterns - the "realizes" associations need to be removed in the XMI.  Either by another approach in the model (preferred) or by post-processing of the XMI.

Please note that questions regarding how packaging and transformation to generate these products remain open, but that some practical ways of doing this from the EA working file have been identified. More exploration here is being undertaken. The idea here is to not restrict additional functionality/products in the future, but to come up with a useful set of outputs for the 2019 release. Additional machine-actionable products may be identified and produced going forward, etc.. 

Follow-up work

Flavio, Jay and Larry will organize modelling meetings in the coming weeks. Dan G. will take part in some of the meetings.

Appendix

1) Email to the srg list of Oliver via Wendy, of 2019-07-31

Hi,

 I transformed the latest canonical XMI from the core selection into XSD. After correcting some uniqueness violations, I had a validating schema file.

 The violations are:

CategorySet -> Contains, IsStructuredBy

InstanceVariable -> BasedOnConceptualVariable, TakesPlatformSpecificSentinelValues

ConceptualInstrument -> Name, Usage

InstanceQuestion -> Name

GeographicUnitClassification -> References

GeographicUnitTypeClassification -> References

ClassificationItem -> Denotes

Code -> Denotes

SamplingDesign -> ExpressesAlgorithm, SpecifiesGoal, ImplementedBy

SamplingProcedure -> HasDesign, IsExpressedBy, ComponentMethodology

ConditionalControlStep -> Executes

WorkflowStepSequence -> Name, Purpose

 Best,

Oliver

2) Email from Achim to srg list 2019-07-31 (post-meeting) :

I made a couple of tests. The results support the ideas we were discussing in today’s meeting.

I use the latest version of Enterprise Architect, 15.

Notes in diagrams:

Notes can be independent note boxes in diagrams. They can have a “usage” relationship to a class or an association (it is not possible to connect the relationship to a property). A note can have “usage” relationships to multiple items.

The notes appear in the regular section (not EA extensions) of XMI as owned comments which have one or more annotated elements. If the note has not relationship in the diagram, the comment in the XMI would have the package as annotated element where the diagram lives in.

This means the notes in the diagrams would be maintained in the XMI and the related Canonical XMI. This is independent of the diagrams which are expressed in the EA extensions of XMI.

Design patterns

The design patterns are organized in packages (like in the prototype). These packages could be all in one master package “DesignPatterns”. This package can be exported in EA as separate XMI file. EA will then put then the package in a model “EA_Model”.

All other packages could be all in a master package like “PIMforUsers”. Again, this package can be exported in EA as XMI to a separate file. This file can be transformed into Canonical XMI.

This means that the idea of separation of concerns would be supported. The design patterns for the model developers and the advanced users on one side and on the other side the model.

The EA file (EA model) would comprehend everything. The two master packages make the separation. The two exported XMI files would be separate models.

The open question is still what could be done with the “realize” association.

Achim

3) Email from Flavio to srg list 2019-07-31 (post-meeting), response to Achim’s email:

Achim,

I've just tested the XMI by package you suggested and it seems to take care of the "realizes association" removal. I will confirm when we have the full package structure in place, but it looks promising.

Flavio

Virtual MRT meeting 2019-07-24

Agenda:

1. Alliance admin update: Dagstuhl attendance discussion
2. Production framework status update (Oliver)
3. Modelling update (Flavio)
4. Data model document (Larry)
5. Process model update (Jay)
6. Standards alignment update (Jay)

MRT meeting minutes 2019-07-24

Attendees

Achim, Arofan, Dan G., Flavio, Hilde, Jay, Larry, Oliver, Wendy

Apologies

Jon

1. Alliance admin update: Dagstuhl attendance discussion

The group that will attend the Dagstuhl workshop in person is Achim, Arofan, Dan G., Flavio, Hilde, Jay and Larry. Oliver and Wendy will attend virtually. Oliver will be available to meet virtually and take on MRT tasks on most days, possibly except Thursday. Wendy would be available on afternoons Dagstuhl time on Monday through Wednesday.

This is a small group of people of similar size as when DDI3 was finalized in 2007.

The work plan for the meeting needs to be ready good time in advance of the meeting, like in the first or second week of September. Forthcoming documents on DataDescription (Larry), Process (Jay) and Foundational will help in the understanding of what remains to do. Arofan also pointed at the need to review the field level documentation.

Delivery of a DDI4 Core specification will happen immediately after the Sprint. The model in Canonical XMI needs to be final by the Thursday of the Dagstuhl week. Arofan raised the question if the finalization of actual bindings/representations would need to be ready by Dagstuhl as well, or if this could wait till the week after if fixes on the model need to be made at the Sprint. Wendy expressed that it should not be an issue to delay the finalization of the representations/bindings for a week as Oliver who will do the push will not be present at the Sprint.

Wendy asked if the group wanted to have (read only) access to Lion for the Sprint, including listings of what is in the views (read only). The group wanted to have this, and Wendy will make sure that Lion will be available for the Sprint. Wendy also has copies of DDI4 Drupal content of the Prototype, the Core components, active development work and the dump from DDI 3.2.

EA v 14 will be used for modelling work at the Sprint. Most participants have this or will upgrade their EA versions before the meeting. The Canonical XMI will be based on this.

Status: All agreed

2. Production framework status update (Oliver)

The implementation of agreements made at the last MRT meeting has been made. This means that no flattening is done and inheritance by extension to abstract classes is used.

Oliver experienced some smaller issues with ‘realize association’ from the prototype resulting in duplicates. This is different from UML realization.

Further validation will take place when the model or parts of the model is ready to test.

3. Modelling update (Flavio)

Flavio will start to prepare the patterns first and aims to have something ready for next week. Jon will set up an environment in Bickbucket where the files from the Core can be maintained.

Patterns are useful to keep the model consistent. Arofan asked Flavio and Achim to explore and discuss different options regarding how to deal with patterns and get back to the group with a recommendation. Jay will input to this work.

Achim explains that in the prototype copies of properties and associations are made by hand and cannot be understood by UML tools. This results in multiple inheritance (mostly from identifiable).

Alternatively UML inheritance could be used, but there could be issues with that as well. Properties may for example be renamed in an applied sense and it is not clear how tools deal with this.

UML realization (see https://www.uml-diagrams.org/realization.html) might be a third alternative.

Flavio and Achim will liaise regarding how to deal with the patterns and will get back to the group with a proposal.

4. Data model document (Larry)

Larry has made some updates to the Data model document, mainly by breaking up the larger diagram.

5. Process model update (Jay)

Jay is working on the process model. He will liaise with Larry and send something around.

6. Standards alignment update (Jay)

Arofan asked Jay to put standards alignment on wait for the moment.

Virtual MRT meeting 2019.07.17

Agenda (as in email to the srg list of 2019.07.16)

1. Alliance admin news update, if any
2. Production framework status update (Oliver)
3. XML Binding Issues update (Achim)
4. Modelling Issues (Flavio):
   • Update on data model integratio
   • Patterns issue
   • UML data types issue
5. Standards alignment update (Jay)

Minutes DDI4 MRT Virtual meeting 2019-07-17

Attendees

Achim, Arofan, Dan G., Flavio, Hilde, Jay, Larry, Oliver, Wendy

Apologies

Jon

1. Alliance admin news update

The invitations for the Dagstuhl workshop have been sent out. The deadline to respond to the invitations is July 23^rd. Work to prepare the meeting agenda will start soon.

2. Production framework status update (Oliver)

Oliver will add the abstract classes in the PSM and extend from that pairing. Abstract types can be used in the related schema (see also minutes from MRT meeting 2019-07-10).

It was discussed whether the PSM is needed as the flattening of the model (which will no longer happen)  according to Oliver was the only thing that happened at that step. Arofan comments that the distinction between PIM and PSM might be useful in the future and suggests to keep both for this reason. Achim points out that the PSM conceptually and structurally should reflect both the syntax of the representation, so that any changes to the syntax should be reflected on the level of the PSM.

Oliver will make the needed changes to trivialize the PIM into PSM that will form basis for schema with abstract classes in. This work will be done in the next week or two.

Possible overriding (for example when it comes to cardinalties) in the model and the corresponding representations  should be looked into and removed. Inheritance should occur by extension.

               Status: Agreed

Jay asked what the rationale for the proposal to flatten the model on the level of the PSM. Arofan replied then that it was because of the long inheritance chains, but that this has been improved so that flattening would not be needed any more. Some patterns implementations may potentially complicate this and would need to be looked into.

3.  XML Binding Issues update (Achim)

Achim has started to produce a generic mapping table. This is complicated but useful work. He will have something to show to the group possibly next week.

4. Walk-through of revised Data Description document

Larry is currently revising the Data Description document from the NADDI sprint by adding content and including snippets from the model related to the different data types described. At the meeting he walked the group through changes as of the meeting day. Work on the document will continue and will incorporate changes in the model. The multidimensional/cube proposals from Dan G. has not yet been included.

It was discussed whether the description of the Classification piece, process and Patterns and Provenance should be included in the same or in a different document for the DDI4 Core deliverable. A goal could be to have a draft of all available by the end of the Dagstuhl week.

4. Modelling Issues (Flavio):

Flavio pointed out that the following needs to be addressed. Some of those are:

• Finalising the modeling guidelines
• Clean up of the documentation
• Revise the definition of Datum
• Decide if the Signification pattern should be kept or not
• Update class documentation

Achim and Wendy proposed to use an editable markdown template to prepare the documentation of classes, properties and associations. In the end the markdown can be copied into the relevant comments fields of the EA. Flavio will coordinate this work.

A sandbox is needed to store different work products. Flavio will ask Jon about which platform to use for this.

Virtual MRT meeting 2019-07-10

Agenda

1. Alliance admin news update
2. Production framework status update (Oliver)
3. XML Binding Issues (Achim)
4. Modelling update (Flavio)
5. Standards alignment update (Jay)

Minutes DDI4 MRT Virtual meeting 2019-07-10

Attendees

Achim, Arofan, Dan G., Hilde, Jay, Wendy

Apologies

Flavio, Jon, Larry, Oliver

1. Alliance admin news update

Achim informs that invitations to the Dagstuhl workshop will be sent soon.

3.  XML Binding Issues (Achim)

Achim and Oliver have been meeting regarding how to approach XSD schema design.

In the follow-up Achim, Oliver and Arofan will discuss the lower level xml schema issues.

Higher level issues like use of abstract classes and inheritance in the PSM and the corresponding representations were discussed at the meeting, see below.

3.1) Can abstract classes be used in the PSM and the representations?

• Achim proposes inheritance by extension to abstract classes in the XSD. The model has chains of inheritance and this should be reflected in the XSD schema. An important step to shorten inheritance chains has been done on this when Identifiable has been moved from an abstract root class to a structured property. Further steps to shorten inheritance chains must also be solved on the level of the model.

 According to Wendy,  use of abstract classes in bindings have not created problems earlier.

• Multiple inheritance/Inheritance of the same classes from different sources: According to Jay there is a problem when the Process and Collection patterns are used in the same workflow. Achim explains that his needs to be solved on the level of the model. An idea can be to use single patterns.

Status: Achim’s proposal to use XSD abstract types for UML abstract classes was agreed. Oliver will be asked to implement the proposal for the XSD, keeping in mind that inheritance chains still can be long.

3.2) Cardinalities

Differences between cardinalities between UML and the XML schema have been detected. Wendy confirms this is an error.

Status: Achim will file an issue, see DMT-281.

3.3) Enumerations

This is used in connection with realization of concrete classes from abstract classes. This is often used, but some only allows a single value. Arofan and Wendy think this can be a workaround from abstract classes.

Status: Enumerations with a single value may be removed when XSD abstract types will be used.

3.4) PSM XMI

Canonical XMI can now be imported to Enterprise Architect but not yet in Magic Draw.

Status: Achim will follow up on this and make further analyses.

3.5) Model and representations mappings

A mapping table of model components to OWL and XSD for a restricted set of classes would be a good starting point for understanding the relationships.

The work can also inform the work on Standards alignment (see point 5. below).

Status: Achim will start the work on a mapping table for model and representation components.

Wendy points out that work is ongoing within the TC to look at XML bindings/representations for the DDI across versions. The current discussions in the MRT group could inform that work. The current discussion is going on on GitHub.

5. Standards alignment update (Jay)

On July 4^th Jay sent an update of the ‘Relationship to Other Standards/Interoperability’ document from the NADDI Sprint by email to the srg list. No comments were received by email before the meeting.

Jay’s approach regards to use the NADDI Sprint document on standards alignment as a basis and add content that regards DDI from a producer perspective. How does it look like when other standards uses parts of the DDI. DISCO would be one use-case.

At the meeting Hilde commented that a mapping similar to what Achim suggest for the representations would be a good first step to get an overview of possible relationships between the DDI and a selection of other standards. In her view focus should be put on the standards the MRT ‘committed to’ (DCAT, GSIM, GSBPM and PROV-O).

Wendy commented to this that other standards that deals with geography usefully couls be addressed as well.

Achim pointed out that a limited set of classes need to be explored for the different standards.

Virtual MRT meeting 2019-07-03

Agenda

1. Alliance admin news update
2. Confluence & Issues resolution update (Hilde)
3. Production framework status update (Oliver)
4. Modelling update (Collections, etc.) (Flavio)
5. Data description update 
6. Final package contents/milestones - brainstorm
7. Standards alignment - Jay to propose a paper

Minutes DDI4 MRT Virtual meeting 2019-07-03

Attendees

Achim, Arofan, Dan G., Flavio, Hilde, Jay, Larry, Oliver, Wendy

Apologies

Jon

1. Alliance admin news update

Achim reminded the group about the EB’s approval of the Dagstuhl workshop of 29^th September through 4^th October. He informed about the request of the EB that the group informs about expected deliverables before the Dagstuhl Sprint and report back directly after the Sprint about the outcomes and factual deliverables. Jared will send out invitations to the Dagstuhl Sprint.

2. Confluence & Issues resolution update (Hilde)

Hilde and Arofan will discuss the take on the Jira issues provided by Wendy. A meeting regarding this  needs to be scheduled.

3. Production framework status update (Oliver)

Achim and Oliver will meet in Cologne on Friday 5^th to discuss the production framework.

Oliver asks people to look at the schema from the last production run, sent by email to the srg list on June 5^th. Changes include things in attributes being put into child nodes.

4. Modelling update (Collections, etc.) (Flavio)

Flavio is waiting for an update to an EA programme version that can import Canonical XMI of the Core directly. Meanwhile he will work on EA file exported from Larry’s newer EA version (14.5).  

5. Data description update 

Arofan point out that the work on the Cube/Multidimensional and Key Value pairs should be put in as part of Larry’s work to consolidate things.

Dan G. will build the cube/multidimensional model in Visio and send to Larry. Dan G., Larry and Flavio will discuss and the model will be implemented in EA.

6. Final package contents/milestones – brainstorm

Arofan asks the group to think about what the DDI4 Core deliverable will look like, and asks if the DDI4 Prototype way of packaging a good way, or if things need to be changed for the deliverable of the Core?

-Achim pointed out that a good understanding of the deliverables is important both for the work of the MRT group itself, for communication with the EB before and after the Dagstuhl Sprint, as well as for users later on.

-Flavio expressed that the set of Views for the deliverable needs to be defined. Which Views should be included? Examples are description of concept systems, data management (as part of process) etc. This needs to be further explored.

-Arofan pointed at the importance of good documentation for users both high level and more detailed. The data description document is already in progress. Technical and field level documentation needs to be looked into.

-Larry mentioned that it would be useful to describe what the DDI4 Core will allow users to do, in comparison with earlier versions of the standard.

-Hilde focused of on the over-all understandability of the model. Understandable definitions and examples should be provided for components where this is currently. Naming rules used need to be made public etc.

The discussion regarding the DDI4 Core deliverables will continue in the forthcoming meetings.

7. Standards alignment - Jay to propose a paper

Jay will use Wendy’s paper on standards alignment from the Ottawa Sprint as a starting point for a proposed paper on standards alignment. According to Jay, what has been done till now is good but DDI centric.

It would also be possible to see things the other way around, like how to make DDI more accessible for other standards to use. Jay proposed that the DDI4 Core can be packaged in a way to make it accessible to other groups.

Arofan pointed out that the standards we ‘committed to’ are DCAT, GSIM, GSBPM and PROV-O. As Achim indicated in the last MRT meeting, this does not mean that everything from these standards needs to be brought into the Core.

Standards are expressed in many different ways. Larry expressed that alignment to other standards can be made without necessarily adopting the semantics of the relevant standards.

Jay will continue to work on the document produced at the Ottawa Sprint, make edits and add new content.

Standards alignment could, additionally, be proposed as one topic for the Dagstuhl interoperability workshop scheduled to October.

Virtual MRT meeting 2019-06-05

Agenda

1. News from IASSIST members meeting/Alliance Admin (EB, SB, etc.) - Achim
2. Confluence & Issues resolution update (Hilde)
3. Production framework status update (Oliver)
4. Modelling update (Collections, etc.) (Flavio)
5. Data description discussion/update (Various)
6. Process model presentation (Jay/Flavio)

Minutes DDI4 MRT Virtual meeting 2019-06-05

Attendees

Achim, Arofan, Dan G., Flavio, Hilde, Jay, Larry, Oliver,

Apologies

Wendy, Jon

1. News from IASSIST members meeting/Alliance Admin (EB, SB, etc.) – Achim

Achim was not in Sydney and is awaiting feedback from Ingo and Jared. We hope next week some information will be available. An EB meeting is foreseen for mid-June. Achim points out that the agenda with linked meeting documents have been sent to different DDI email-lists. This way they are accessible for the MRT group members and others to read.

2. Confluence & Issues resolution update (Hilde)

Hilde asked Flavio to look into issues related to the Collection Pattern, as proposed by Arofan. While Flavio recommends to  close some of the related  issues, some issues in particular DMT-238, DMT-203 and DMT-189 will require further discussions.

Hilde is currently working to link Prototype_Review issues, Post-prototype issues and issues transferred from TC to DMT, to MRT tasks. She will liaise with Arofan next week regarding this work.

3. Production framework status update (Oliver)

Oliver sent a DDI4 build by email to the srg list on the meeting day. This includes a validation schema file produced on basis of the canonical XMI from the prototype release. A proposal for PSM was also included. Back-tripping is not yet possible.

Achim has looked into the former version of the build that Oliver provided and has some questions regarding the realization of the PSM. Achim and Oliver will liaise regarding this and will get back to the larger group with input from their discussions.

4. Modelling update (Collections, etc.) (Flavio)

In an email to the srg list of 06.01 Larry questions Flavio’s suggestion that aggregations would not need naming. Larry sees this as a problem, for example when describing XPaths where there are two aggregations with the same target type. Larry and Flavio will liaise to frame up the discussion and will get back to the bigger group with input.

5. Data description discussion/update (Various)

Multidimensional/Cube:

Dan G. and Flavio will get back with modeling input as quickly as they can.

Key/value pairs:

Flavio is preparing an example from streaming. He and Jay will get back with feedback regarding key/value pairs in one to two weeks.

Data Description with examples document:

Hilde will take on this again after the meeting with Arofan about the issues.

6. Process model presentation (Jay/Flavio)

Jay sent an updated version of his presentation of Process by email to the srg list right after the meeting. This proposal covers both white and black boxes.

The audience for the presentation is both the MRT group and ModernStats.

Jay asks people to look into this till next time and get back with comments.

He points out that there would be several ways to deal with this task, and that terms probably needs to be addressed.

Virtual MRT meeting 2019-05-29

Agenda

1. Any news regarding the EB or upcoming members/SB meeting at IASSIST
2. Issues resolution update (Hilde)
3. Production framework (including binding issue for XSD) (Oliver)
4. Applying the Collections model (Flavio)
5. Process model presentation (Jay)
6. Data description discussion/update

Minutes DDI4 MRT virtual meeting 2019-05-29

Attendees

Arofan, Dan G., Flavio, Hilde, Jay, Jon, Larry, Oliver

Apologies

Achim, Wendy

1. Any news regarding the EB or upcoming members/SB meeting at IASSIST

No news are available yet, but are expected after the Annual DDI meetings.

2. Issues resolution update (Hilde)

Hilde is looking into Prototype_Review and Post-Prototype DMT Jira issues, as well as issues transferred from TC to DMT, and assigning them to MRT tasks. Flavio will look into the issue related to Collections linked to the MRT task DMT-232 and will get back to Hilde regarding their status.

3. Production framework (including binding issue for XSD) (Oliver)

Oliver raises a question originally posed by Larry regarding some structured data types properties becoming attributes and some child nodes in the xml representations (bindings).

Larry points out that it is not a good idea to arbitrarily making things different when parsing into xml, and that child nodes in general are much easier to deal with.

Arofan expresses that this needs to be looked further into.

Oliver will, in the course of one to two weeks, try to draft a simplified set of rules for this, and at the same time produce a schema that implements the proposed for rules - all for review by the group.

               Status: Agreed

4. Applying the Collections model (Flavio)

Flavio sent an update of the collection pattern including an XMI and an EA file to the group on the meeting day (05.29). He expresses that the current proposal should not deviate too much from the original.

Some implementation examples are needed to check if things need to be fixed.

Arofan means that the current proposal could be applied as things are modeled. Revisions of the over-all the model can be done when the pattern is tested through application on different scenarios.

5. Process model presentation (Jay)

At the meeting Jay went through the slide deck on Process sent by email to the srg list on the meeting day (2019.05.29).

Jay’s ideas include a proposal for simplification of the Process model as in the prototype, focusing on pre- and post-conditions and supports data management.

Dan G. pointed out that with Jays proposal for Process, the Methodology pattern of the prototype might be redundant, to which Jay agrees.

Jon points out the importance of transparency of processes.

Arofan expresses that data lineage is the primary scope of Process for the DDI4 Core.

Jay will continue to work on the Process model. GSIM, GSBPM and PROV-O will be looked into.

Jay, Flavio and Jon will provide different pipeline examples to inform the further work.

Other

Dan and Flavio continue to work on multidimensional data.