TC Meeting Minutes

ATTENDEES: Wendy, Oliver, Dan, Jeremy
REGRETS: Flavio, Jon

AGENDA

Update from Scientific Board meeting

Scientific Board agreed with the suggestion from TC that there should be a cross-product group on qualitative content. Darren (who is on the CDI subgroup) will draft content for the creation of a formal working group.

http://docs.ddialliance.org discussion

• Darren and Jon were unavailable today

• Confirmed the basic uses of doc.ddialliance.org which should contain:

  • generated content from production pipelines of published content including:

  • schema and specification content

  • documentation about the standard (not all pipelines generate the same mix of documents but those that are generated in other ways should also be located here.)

• Note that AWS provides us with accessible cloud storage

• Resolution systems: Controlled Vocabulary (current link is https://d2jdhhz9hcqxia.cloudfront.net/) and RDF content resolution systems (http://rdf-vocabulary.ddialliance.org )

• PDFs etc. accessible through web pages are managed within HubSpot system

MISCELLANEOUS

• What happens with contact form information (issue was filed that there is no confirmation)

  • there seems to be no autogenerated response but a statement that DDI staff will contact you

  • Jon's is the email address that contact info is sent to

• DDI-L v4.0

  • add the content to codeValue type to reflect full field of objects added to Codebook (Dan will create pull request)

• DDI-C v2.6

  • Dan has been working on the validation tool to generate and retain neat and consistent indenture. This will be switched on and content validated just before publication.

• DDI-Codebook and DDI-Lifecycle working groups should be established at point of publication. Be ready with the working group proposals by then.

• Oliver added information on the use of the Blog feature for News items to the Web Editing page

ATTENDEES: Wendy, Oliver, Dan, Darren, Jeremy, Flavio
REGRETS: Jon

AGENDA:

• Website updates

• Roadmap status

• Qualitative content group

• Profiles

Quick recap

The team discussed updates and plans for the website, including the formation of a web management group and the incorporation of qualitative data representations into DDI Lifecycle. They also explored the potential of using a new format for profiles, which would allow for more machine processable and future-proof data, and the transition from the current DDI-based profiles to DCTAP. The next steps for the team were outlined, including sending information on the interoperability working group, packaging examples for TC-drafts, and ensuring the project stays on the agenda.

Next steps

• Darren to add qualitative data representation discussion to the Scientific Board agenda for Tuesday.

• Wendy to review and attempt to express existing XPath-based profiles using the new Dublin Core Tabular Application Profile approach.

• Dan to package up examples of the Dublin Core Tabular Application Profile and put them in the TC-drafts folder.

• Darren to present the Dublin Core Tabular Application Profile approach to the Interoperability Working Group.

• Wendy to ensure profiles discussion is added to a future TC meeting agenda.

• Darren to make a list of documents and their current locations for future disposition discussion (http://docs.ddialliance.org ).

• Oliver to write up directions for creating news items on the website.

• Oliver to document what features (like blogs) are being used on the website and for what purposes in the Atlassian documentation.

Summary

Website Updates

Wendy discussed several updates and plans for the website and the scientific board. She mentioned that the website update will be announced in the newsletter and that the review request has been discussed with Jared, who expressed concerns about the lack of response. Wendy also suggested moving the 'getting started' items to a top level and proposed moving resources into the 'getting started' section. She also mentioned that the web management group will be formed, with Jared's advisory group being considered.

Roadmap

https://docs.google.com/spreadsheets/d/1qIAw6zK6KIDBNSUQTFJfIjqRdkyMkj9oFTyQnNdttpI/edit?usp=sharing (see "WG Plan - items" sheet)
Wendy then shared a roadmap from the scientific board, outlining the plan for the next few years, including completing the shift to Github, revising the website, and starting overall modeling work. No major issues or problems were raised by the team. We are on plan and will address Short-Medium Term items over the spring and focus on Medium term items following that. Lifecycle and Codebook working groups will also be proposed this year.

Qualitative Data Representation in DDI

Darren discusses the potential incorporation of qualitative data representations into DDI Lifecycle. He mentions that Shauna suggested 15-20 attributes for a possible qualitative module or package. The group has been considering this idea for several years, but previous attempts were hindered by other changes and lack of activity. Flavio inquires if this would include discussions on describing images, as previously discussed in DASH. Darren confirms that it would involve referencing file packages and qualitative collections.

• Qualitative Data Subgroup

• Super narrow adding annotations in DDI-CDI.

• This needs to be a broader group

• Recommend to SB to make this a content working group (cross-product)

In the meeting, Darren and Flavio discussed the need for a more comprehensive approach to content management, rather than focusing solely on annotations. They agreed that a cross-product working group could be beneficial for integrating common design elements across different products. Wendy suggested that this should be a content working group under the scientific board, to ensure consistency and coordination across products. They also discussed the need for a common design for common things, such as classification models. The team agreed to bring this topic to the scientific board agenda for further discussion. Additionally, Wendy raised concerns about the management of broken links and outdated information on the website, suggesting a need for a more efficient system for updating and maintaining content.

Alternative Profiling Methods Discussion

Dan presented a status report on alternative profiling methods for the X path based profiles available in lifecycle. He discussed the use of Dublin Core tabular application profiles, specifically the Cesda profiles for the Question Bank, and how they could be applied to work with the Ddi models. Dan also mentioned that these profiles could be converted into checks for creating shapes for RDF serializations. He sought feedback and asked if there were any other requirements they should consider as they continue to investigate different ways to make these profiles. Darren expressed his appreciation for the work done so far.

Darren and Dan discussed the complexities of tying different elements together in XML, specifically the issue of Xpath. They agreed that the current system was confusing and that Matthew Morris had to write hacky code to overcome the Xpath problem. Dan explained that the grid they were discussing was a way to reference different elements, such as a study unit, and how it could be linked directly to a data collection. Wendy added that the data collection could live anywhere, but it needed that connection. They also discussed the future of version 4 of lifecycle, which will have XML, JSON, or RDF representation to convey content.

The team discussed the potential of using a new format for profiles, which would allow for more machine processable and future-proof data. They considered the possibility of using relative paths and value constraints, which would provide more control over the data. The team also discussed the need for a more robust tool, like the metadata validator, to handle the new format. They agreed to explore this format further and possibly publish it as an example. The team also discussed the need for clear documentation and the potential for the new format to be adopted by other groups.

The team discussed the transition from the current DDI-based profiles to DCAT, which they found to be a better solution. Dan suggested that parsing CSVs is easier and could be automated. Wendy agreed, noting that the new approach is clearer and more machine processable. Darren mentioned that he would take the discussion to the Interoperability working group and that Alan was sympathetic to the transition. Dan offered to package up the new approach as an example and put it on the shared drive for others to review. The team also discussed the potential for using the new approach for RDF shapes and the need to decouple from XML for future profiles.

• DC Tabular Application Profiles (DC TAP) - Primer

• It was much clearer as documentation than the xpath profiles

• Issues of relationships from package/resource/fragment to fragment only in DDI-L v4.0 that is a problem with xpath based approach

Next Steps and Document Management

Wendy outlined the next steps for the team, including Darren sending information on the interoperability working group, Dan packaging examples for TC drafts, and Wendy ensuring the project stays on the agenda. Wendy also discussed the need for a clear understanding of document locations and rules for future disposition. Oliver confirmed that Jared now has editing rights and will be creating news items. Wendy emphasized the importance of documenting processes and features for future reference. The team agreed to meet again next week to discuss the http://docs.ddialliance.org discussion and document locations.

ATTENDEES: Wendy, Oliver, Dan, Jon, Darren, Jeremy
REGRETS: Flavio

Content and organization of http://docs.ddialliance.org
GitHub - ddialliance/docs.ddialliance.org: Static content for docs.ddialliance.org website

• Field level documentation is currently on docs

• To what extent do we want to put schema (XML, JSON, ...) on docs. CDI is currently on HubSpot file location -- should this be redirected

• In principle we should be holding things like specifications on spaces that we control AWS or http://docs.ddialliance.org rather than HubSpot

  • This makes sense - any maintainable technical artifact held on

  • Currently this would involve migrating a few dozen documents

DDI-CDI publication status

Last few changes requested by CDI are being implemented by mid-day Friday

Web review announcement approved
https://docs.google.com/document/d/1ByEfdJDmSC1rSCdjJ7gNWBzar9DNShTiQ6q0-a2L-oc/edit?usp=sharing

ACTION ITEMS:

• Oliver will put up instructions for adding to the News items on the web

• Wendy will notify Jared regarding DDI-CDI status and announcement

• Wendy will notify Jared about Web Review notice

• Darren will make a list of what documents are where (docs.ddialliance.org and HubSpot, etc.) so we can make lists and rules for future disposition

AGENDA for February 13

• DDI-Lifecycle Profile task - Dublin Core application profile for CESSDA (Dan will provide some examples)

• What do we want to suggest for profiles - content groups are interested

• Determine scope of profile

• Validation of content against profile

• DDI-Lifecycle Profile task - Dublin Core application profile for CESSDA (Dan will provide some examples)

• Tested out using the Dublin core applications files that went pretty well - multiple serializations based on the content of the model level so it could be applied to all the serializations

• Functionalities:
  If this is there that has to be there - is there a possibility of tooling that can check that at a higher level
  Specifies at a structural level -
  Provide a best practice for Tool defining tools
  Schematron - XML
  ShEx and SHACL - RDF

• Members should think about use cases and functionality for profiles, as well as overall approach (profiles based on model so that it can be applied across specifications, they best practices for creating profiles for specific specifications using available tools)

AGENDA for February 20
http://docs.ddialliance.org discussion

AGENDA for February 27
Codebook (high level documentation)

ATTENDEES: Wendy, Oliver, Dan, Jon
REGRETS: Darren, Flavio

Quick recap

Wendy, Dan, and others discussed the status of website updates, the publication process for CDI, and the potential shift of published packages from Bitbucket to Github. They also reviewed the roles and responsibilities of the management group, the progress of the technical committee's work, and the need for clearer application profiles. The team agreed to revisit these issues in future meetings and explore alternative methods for XML usage.

Next steps

• Dan to contact Achim regarding the issue with creating the package for CDI publication and provide instructions.

• Wendy to have Achim write to Dan (copying Oliver) explaining the specific issue he's having with creating the package.

• TC to review and finalize the website review and management documents shared by Wendy.
  Review process
  https://docs.google.com/document/d/1ByEfdJDmSC1rSCdjJ7gNWBzar9DNShTiQ6q0-a2L-oc/edit?usp=sharing

• TC to work with Jared on forming the management group for the website (this would be under the EB).
  Web management
  https://docs.google.com/document/d/1q3c0Gzfqys5waAnwWFJORY99n6fxoQ2SLenIGx0bEPg/edit?usp=sharing

• Technical Committee members to review and provide feedback on the new technical infrastructure page on the website.
  https://ddi-alliance.atlassian.net/l/cp/P4Cx6ct2

• Dan to present examples of profile tasks using Dublin Core application profiles at the next meeting.

• Jon to prepare notes and thoughts on questionnaire profile validation for discussion at the next meeting.

• Wendy to inform Jared about using tomorrow as the official publication date for the website and start negotiations for the announcement.

Summary

Website Update and Package Publication

Wendy discusses the status of the website update and the publication process for CDI. She mentions that some concerns raised by Jared have been addressed, including the maintenance of the tools list.

CDI Publication

CDI issues for TC:

• Publication of package - how to produce the download package in github

• Where are publication packages being placed

• Use GitHub releases - upload assests as a release and includes a source as part of a release as well

• Still need a zip file with a full package

• This is the final part of the full pipeline - a deliverable asset to be stored

• Produce a complete download package, partial download package for the field-level documentation folder and sub-folders

• ACTION: Have Achim provide details on the problem he is having to Dan cc Oliver and Wendy

• CDI link updates later today

• Arofan will go through the pages

• Dan and Wendy discuss the issue of creating a package for publication, as Achim is unsure how to produce it. Wendy asks Dan to follow up with Achim to clarify the problem and provide instructions. They also discuss shifting the storage of published packages from Bitbucket to Github, with Dan suggesting the use of Github releases for this purpose.

Website Management

Wendy shared a document for review and management of the website, outlining the roles and responsibilities of the management group, including a technical representative, user interface specialist, content specialist, and archivist. There was some confusion about the file management responsibilities, with Dan confirming he set it up and Oliver expressing unfamiliarity with it. The next steps include getting the management group together and working with Jared and Jon to formally organize the group.

Technical Committee Progress and Feedback

Wendy discussed the progress of the technical committee's work, including the creation of a technical infrastructure page and the ongoing editing of guidelines. She emphasized the need for feedback on the new site and the importance of understanding the technical infrastructure. Wendy also mentioned the upcoming discussion on the contents and organization of the http://docs.ddialliance.org . Jon agreed with Wendy's points, highlighting the need for more control over content and the ability to make changes. Dan suggested discussing the profile task again to get a status update.

Clarifying Application Profiles and Validation

Dan and Wendy discussed the need for clearer application profiles, particularly for the Dublin Core and CESDA. They agreed that the current XML usage was difficult to use and suggested exploring alternative methods. They also discussed the importance of having a list of available profiles for different implementations. Jon added that the profile should include conditional elements and a validation process, especially for content validation. The team agreed to revisit these issues in future meetings.

NEXT WEEK'S AGENDA

• http://docs.ddialliance.org discussion

• DDI-Lifecycle Profile task - Dublin Core application profile for CESSDA (Dan will provide some examples)

  • What do we want to suggest for profiles - content groups are interested

  • Determine scope of profile

  • Validation of content against profile

ATTENDEES: Wendy, Oliver, Darren, Dan, Jeremy, Flavio

Agenda:

• Website

• DDI-CDI publication

• DDI-SDMX mapping

• Variable cascade

Quick recap

The team discussed the progress of their website project, focusing on addressing issues raised by Jared and the need for a Javascript developer. They also discussed the mapping between DDI and SDMX standards, the need for better coordination across different communities, and the establishment of a lifecycle committee and a codebook committee for content iterations. Lastly, they discussed agenda items for February: http://docs.ddialliance.org , the upcoming establishment of a lifecycle and codebook committees, and the potential for future changes to their web page project.

Next steps

• Darren to send a status update on the website to Wendy and Jared by the end of tomorrow.

• Darren to work with a JavaScript developer to implement filtering functionality for publications and tools pages early next week.

• Wendy to change the stubs for lifecycle and CDI to follow the same naming convention as other versions.

• Wendy to set up HTML pages for CDI version 1.0 within an hour after the meeting.

• Darren to set up routing for CDI URLs after the meeting.

• Wendy to send a reminder to the CDI group about their commitment to the production pipeline automation.

• Flavio to check on existing mappings between Lifecycle and SDMX.

• Technical Committee to consider creating a temporary working group under the Scientific Board for major mappings across DDI products.

• Technical Committee to discuss the variable cascade issue and determine who should address it.

• Technical Committee to prepare an agenda item for the first meeting in February to plan next steps for mapping activities.

• Wendy to focus on Codebook after the website and CDI are completed.

• Technical Committee to review and update the technical infrastructure documentation in Confluence.

• John to provide clarity on what will be on docs.ddialiance.org and the timelines for implementation.

• Scientific Board to prepare for upcoming elections, including at least one open position.

Summary

Website Progress and Future Improvements

Wendy led a discussion about the progress of the website, with a focus on addressing issues raised by Jared. Darren confirmed that he had made temporary fixes to the membership page, but acknowledged the need for a more permanent solution. The team also discussed the need for a Javascript developer to work on the publications and tools page. Wendy suggested that someone with Javascript knowledge would be a valuable addition to the team. Additionally, Wendy noted a discrepancy in the naming convention for the lifecycle version pages and decided to change it to align with the rest of the website.

Web Page Project Progress and Testing

Wendy, Darren, Flavio, and Oliver discussed the progress of their web page project, focusing on the redirection of pages, the need for a status update by the end of the next day, and the potential for future changes such as separating the 'getting started' section and moving the glossary to the product section. They also discussed the need for testing URLs before publication and the need for field-level documentation. Flavio mentioned the need for an update on the pipeline commitment.

DDI and SDMX Mapping Discussion

Wendy and Flavio discuss mapping between DDI and SDMX standards. Wendy mentions there is already some existing mapping between lifecycle and SDMX that needs updating. She will locate that previous work. Wendy also notes that codebook has an n-cube, so that should be considered. The mapping activity originated in the UNECE's Modernstats Supporting Standards group, where it remains an ongoing effort for this year.

Improving Data Integration and Coordination

Flavio and Wendy discussed the need for better coordination and alignment across different communities, particularly in terms of data integration and mapping. They agreed on the importance of understanding existing mappings and lifecycles, and the need for a group to oversee these processes. Wendy suggested the creation of a temporary working group under the scientific board to handle these tasks. They also discussed the variable cascade and the need for consistency across different products. Dan mentioned that the RDF Equivalence group would be working on similar issues. The team agreed to plot out their next focus areas and ensure that important tasks do not get dropped.

Transition to Docs and Committee Roles

Wendy and Darren discussed the need for more clarity on the transition to http://docs.ddialliance.org , with Darren expressing a desire for specific timelines. Wendy agreed to prioritize this issue and mentioned that Jon would be more available for the discussion in the near future. Wendy also brought up the upcoming establishment of a lifecycle committee and a codebook committee for content iterations. Furthermore, Wendy mentioned that there would be at least one open position on the scientific board, suggesting a competitive election process. Ingo was noted to be stepping down from the Scientific Board to focus on tools and hackathons.

Future meeting agenda items:

• http://docs.ddialliance.org organization and content

• Codebook v2.6 publication

• Lifecycle v4.0 publication

• Codebook WG

• Lifecycle WG

• Mapping work