TC Meeting Minutes

ATTENDEES: Wendy, Darren, Dan, Jeremy, Flavio

Quick recap

Darren, Wendy, and Dan discussed the readiness of the postman collection and the need for a written implementation plan. They also discussed the need to update the sequence property in their system and the progress of their project, with a focus on the potential for an RDF representation of the DDI codebook. The conversation ended with plans for a hackathon challenge.

Next steps

• Darren to email Sanda about the postman collection and HTML page revisions being ready and discuss making it available next week.

• Darren will remind Arofan to provide the implementation plan in writing for CDI-100 as requested by the Technical Committee.

• Dan to file an issue regarding the RDF code URIs and version stability.

Summary

Postman Collection Readiness and Implementation Plan

Darren and Wendy discussed the readiness of the postman collection and the need for a written implementation plan from Arofan. Darren agreed to send an email to Sanda and Jared to update the CV page and set up a webinar on this new system. Wendy will initiate the vote process with Jared once the written implementation plan is received. Dan and Darren also discussed the possibility of implementing a version-independent URI for the CVs, which Dan suggested could be a future enhancement. The team agreed to proceed with the current plan and to address the version-independent URI issue later.

Updating Sequence Property and Cardinalities

Wendy and Dan discussed the need to update the sequence property in their system. They noted that the property was already ordered in the Beta version, but other properties were not. Dan has made revisions addressing comments Wendy made on the Pull Request (#48). Dan mentioned that he had updated the cardinalities and the order, but there were some choices that didn't match their normal naming convention. Wendy agreed to go through the choices to ensure consistency. Dan will make changes and include in this pull request.

Project Progress, Documentation, and XML Linter

Dan and Wendy discussed the progress of Codebook work, with Wendy committing to complete certain tasks by the end of the day. They also discussed the need to re-engage Adrian regarding the documentation. Dan will implement the format validation using XML Linter and suggested the addition of instructions on how to use the XML Linter to reformat it before making changes. Wendy agreed to look at Adrian's previous work and compare it with the current documentation to ensure consistency. They also discussed the need to check each individual element in the documentation to ensure proper formatting and content.

DDI Codebook RDF Representation and Ontology

Dan and Wendy discussed the potential for an RDF representation of the DDI codebook, given the status of DDI Lifecycle as a superset of DDI Codebook. They considered creating a DDI codebook ontology, reusing URIs from the lifecycle model. Wendy emphasized the need for a clear production process and the progress towards automated production for their products. Dan agreed to write up a challenge for the upcoming hackathon and to forward it to Wendy for additional ideas.

CDI pre-vote webinar

ATTENDEES: Wendy, Oliver, Jon, Dan, Jeremy
REGRETS: Flavio

AGENDA:
DDI-CDI submission
Timeline and steps needed to initiate Vote
Webinar Oct 3, (need to reschedule TC)
CDI-101
CDI-100
Next Steps
Getting 4.0 BETA2 out

Quick recap
Wendy discussed the upcoming webinar and the subsequent vote, as well as the implementation of an automation system by the CDI group. The team also discussed their schedules, the upcoming webinar, and the resolution of the two issues Jeremy filed in CDI Jira. Lastly, Wendy discussed the completion of the build scripts for DDI-CDI script for the DDI space, which is expected to be completed by the end of 2024, and the need for a build script that can be run in Github actions.

Next steps

• Wendy to communicate with CDI group about the need for a build script that can be run locally or in Github actions, clarifying expectations and timelines.

• Achim to develop the build script for CDI implementation, aiming for completion by the end of 2024.

• Oliver to assist with integrating the CDI build script into Github actions once it's developed.

Summary
Webinar Discussion and Automation System Update

Wendy discussed the upcoming webinar and the subsequent vote, which is expected to take place after the event and last for a month. She also mentioned that the CDI group has asked for clarification on providing a timeline for implementing their automation system. Wendy noted that Jared had sent out a notification for the webinar and that Flavio was reviewing 97 issues in the Jira issue tracker to determine which have been resolved. This should be completed prior to the webinar. There were no objections or concerns raised about the proposed timeline or steps.

Schedules, Webinar, and Lifecycle Updates

Wendy, Oliver, Jon, Dan, and Jeremy discussed their schedules and the upcoming webinar. Jon mentioned he would be away for the next three weeks, and Dan suggested cancelling the meeting next week unless there was a significant agenda item. Dan also planned to complete an ordered elements in Lifecycle task soon and intended to discuss the BETA2 of Lifecycle before Jon's return. Wendy agreed to these plans and proposed to discuss on email and only meet on this if necessary if necessary.

Project Implementation and Process Details Discussion

CDI-101 has been resolved as Jeremy suggested. Wendy will inform Arofan so they can close this issue. Regarding CDI-100 she also mentioned a request for more information on the implementation of a process, which was initially unclear. Wendy clarified that the team wanted more details on the implementation, not just the process itself. The team had previously attached a document outlining the process, but Wendy suggested they needed more information on its implementation.

Script Implementation for DDI Space Automation

Wendy discussed the implementation of a build script for the DDI space, which is expected to be completed by the end of 2024. The script, to be developed by Achim, will be used to automate the system once data is extracted from EA. Wendy clarified that the long-term goal is to have the build script run in the DDI space, not in a separate pipeline. Jeremy and Dan suggested that the script should be run either locally by developers or in a continuous integration system like Github actions. They also emphasized the need for a build script that can be run in Github actions, which can be set up after the build script is developed. Oliver suggested that the script should be runnable inside Github actions, and that the team should maintain the Github actions. Wendy agreed to clarify these points to Achim and to seek specific timelines and responsibilities for the project.

Background information on CDI-100

See Section I: Production Flows in process document (page 2)

Achim is starting work on implementation of an automated production line (jar files, etc.) following Dagstuhl (area 1 of funding request) Goal is to complete by end of 2024

Working in AW space - contact with Oliver on getting the pipelines into DDI space (eventually get these on GitHub Actions)

Read issue CDI-100 comments

What specifically do we need from CDI - prior to vote

We asked for a clarification of what needed to be done, who was working on it, what was the timeline. Do we need this from them in writing? If so do we summarize our understanding and then have them sign off on it? This could be fastest as much of it has simply been discussed.

TO DO DURING MEETING:

CDI-101 can we tell them this is acceptable. Basically, Achim agreed with TC and they retained only the canonical XML schema in the package (other will be provided as a convenience schema).

CDI-100 Review comments
What specifically do we need from them. I have notes from the CDI meeting about planned steps and asked Arofan to clarify the statement that a document is in the works from the issue comment.

Note that there is step-by-step documentation being drafted, but the overall production flows are described in the syntax representation document. This will be made final in the near term.

Is the movement to Git workflows a current imperative or a goal?

FROM AROFAN
That comment needs updating, as per our discussion today. I am happy to write up whatever is needed, but at this point it seemed like that section of the syntax representation document was good enough.

I would like to see specifics - with links to the correct versions of each piece of code - added when we have the pipeline set up. At this point, though, I am not sure that makes sense. The idea would be to have a common code repository and full documentation in reference to that repository, presumably for all of the TC products. I can work with Achim as this develops further.

Website work update:
Management document
https://docs.google.com/document/d/1TgC-KEng2cccXU_ugvvhHIjrx4SC9HU5xj0XmZ3eTcY/edit?usp=sharing Darren to email Sanda about the postman collection and HTML page revisions being ready and discuss making it available next week.

Darren will remind Arofan to provide the implementation plan in writing for CDI-100 as requested by the Technical Committee.

Dan to file an issue regarding the RDF code URIs and version stability.

ATTENDEES: Wendy, Oliver, Jeremy, Dan, Darren, Flavio

Quick recap

Wendy provided an update on the CDI status and discussed two issues raised by Jeremy, one related to the automated pipeline setup (CDI_100) and the other concerning XML schemas (CDI-101). The team also discussed the challenges of publishing multiple incompatible XML schemas and the status of issues related to the package. Lastly, they discussed the TC management of the Alliance website, the synchronization protocol, and the ongoing project, emphasizing the need for updates on assignments and testing email templates and mail shot functionality.

Next steps

• Flavio to update the CDI issue tracker to clarify the status of all issues for the technical committee's review.

• Darren and Oliver to review and refine the website management roles and responsibilities document.

• Wendy to create documentation on how to make content entry changes to maintain consistency with templates.

• Darren to email Sanda about the preferred placement of concept URIs in the CV tables and coordinate with Oliver on implementation.

Summary

CDI Status Update and Issue Resolution

Wendy provided an update on the CDI status, mentioning that they are still expecting a submission by tomorrow. She also discussed two issues that Jeremy had raised, one related to the automated pipeline setup and the other concerning XML schemas. A comment was added to CDI-101.

Discussing XML Schema Management and Automation

Wendy, Jeremy, and Dan discussed the challenges of publishing multiple incompatible XML schemas. They agreed on publishing a single canonical schema in the package, with additional schemas separated and using different namespaces. Wendy suggested adding a comment reflecting this decision. The team plans to automate the process with a continuous integration pipeline triggered by publication from their modeling tool. Oliver was identified as the contact for the pipeline work. The package delivery is intended for tomorrow, though some details still need to be confirmed with others. The team aims to fully automate the schema management and publication process. Given we always said the spec was tool-agnostic, Jeremy and Flavio see no reason for publishing a second XML schema developed specifically for tools, even if they are very common. We recommend the second schema be made available at a later date as an add-on to the spec.

Discussing Package Issues and Jira Tracker

Wendy and Jeremy discussed the status of issues related to the package. Jeremy expressed concern about the number of open issues and the lack of a plan to address them before publishing. Wendy clarified that Flavio hadn't had a chance to update the status of these issues. Dan provided information that Flavio had fixed about 20 of the issues. Wendy suggested updating the Jira tracker to clarify what's been finished and what's future work. The team agreed on the importance of having an up-to-date issue tracker for the review process.

CDI Webinar Announcement

Wendy noted the text for the announcement of the CDI webinar focused on the upcoming voting process and asked for comments on the draft webinar details.

Technical Committee Website Management and Roles

Wendy discussed the management of the Alliance's website, focusing on content distribution, administrative roles, and website performance. She proposed a division of roles between Darren and Oliver, with Darren handling HTML, branding, and styling, and Oliver managing database-driven content and file management. Wendy also suggested the creation of an archivist role to ensure related pages receive necessary changes. Darren agreed to review the content management document and make necessary clarifications.

Synchronization Protocol and Content Updates Discussion

Wendy discussed the synchronization protocol and the process of updating content between Drupal and HubSpot. She mentioned that she is recording changes and keeping a folder of content changes. Wendy also noted that to provide Jared with access, Jon would have to lose his access to Drupal, but this might not be a good idea during the development period. She mentioned that new documents will be added using the HubSpot system, which should put them in the same place as those that were transferred from Drupal. Wendy also raised a question about additions and changes to the databases, which Oliver was asked to provide answers for.

Project Updates, Testing, and Webinar Planning

Wendy discussed the ongoing project, emphasizing the need for updates on assignments and inquired about the status of testing email templates and mail shot functionality. Darren confirmed that they had tested mail shots but needed a more real-world situation. Wendy also questioned the timeline for the project. Darren agreed that the flip over should happen after Eddie due to unresolved issues with the new website. She also emphasized the importance of reviewing the deadline dates on the shared spreadsheet. She is prioritizing the entry process document to ensure consistency in entry procedures before involving Jared.

Column Placement Discussion in Controlled Vocabulary HTML Table

Wendy and Darren discussed the possibility of adding concept URIs to the HTML pages, with Darren suggesting that it would be beneficial for their developer to insert another column in the tables to hold the concept URI. The team discussed the placement of a new column in a table. Wendy suggested that the column, which contains the concept URI, could be placed below the concept label in a different font or color to save space. However, she also noted that the column should be placed in the before the label column to avoid implying a change between languages. Oliver suggested placing the column in the second column to avoid blurring with the hierarchical indent. The team agreed to ask Sanda for her preference on the placement of the column.

Finalizing Feature Changes and Upcoming Tasks

Wendy and Darren discussed the final changes needed to make a feature live. Darren mentioned that a redirection was required, which was a simple task. Wendy emphasized the importance of making the concept URI feature prominent to encourage its use. For the following week, Darren didn't have specific topics, but Wendy mentioned the need to address the CDI submission. She also mentioned that she was working on high-level documentation for the Codebook. Dan was reminded to let Wendy know when the Beta 2 changes were completed, and to review the documents for any necessary comments or changes.

ATTENDEES: Wendy, Oliver, Darren, Dan, Flavio, Jeremy

AGENDA:

• DDI Lifecycle Codelist CV versions and distribution options via AW

• Agreement between TC and ICPSR regarding website management

• Defining admin and content positions for website

Quick recap

The team discussed improvements to DDI-Lifecycle Codelist version of CV information distribution, including versioning, file layout, and code list management. They explored website management roles, responsibilities, and the transition process for their project. The team also addressed document relocation and the need for a timeline and checklist to guide the project's progress.

Next steps

• Dan to create an example of the proposed versioning changes for DDI code lists of CVs and share it with the group.

• Wendy to write up clarification on the roles and responsibilities for website administration and content management.

• Wendy to create a Google spreadsheet with a timeline and checklist for the website migration, including dependencies and key dates.

• Dan to continue working on the ordered properties for DDI-Lifecycle 4.0 Beta 2.

• Wendy to contact John to clarify the location and organization of documents moved from the website to http://docs.ddialliance.org .

Summary

DDI Lifecycle Codelist CV versions and distribution options via AW:

Dan proposes creating a new file and disk layout for distributing DDILifecycle information and code lists, allowing for automatic listing of available versions. He suggests changing the versioning to reflect content changes in the DDI metadata version and storing the user-provided version in the DDI UserID. Darren agrees to consider these proposals for improving versioning and identification. Oliver clarifies that this would not require changes to the publication system.

• Versioning System for Translations Discussion

  • The team discussed the versioning system for translations and its implications. They agreed that a version change could occur if there are structural changes to the vocabulary, such as new categories or deleted categories. However, they also noted that a version change could occur without any content change, which could be confusing. The team also discussed the need for a programmatical way to determine if a version change is necessary for an individual concept, which would require integrating a comparison process into the creation process. They concluded that this would be a complex task, potentially requiring a new management system.
    Current CV Versioning policy
    https://ddialliance.org/controlled-vocabularies#policy

• Managing DDI Content Updates and Code Lists

  • Dan proposed a solution to manage and compare updates to the DDI Content, including a manifest file to list all available content and different versions. He suggested this to simplify the process of importing and translating DDI code lists into their tools. Darren and DDI agreed that any changes to the code lists should occur at the point of generation to ensure consistency. Oliver clarified that the current code lists are published in the DDI Alliance CV system as XML files. The team agreed to consider Dan's proposal for a more streamlined process.

• Versioning Mechanism and Transition Concerns

  • Oliver and Darren discussed the versioning mechanism between the SKOS and the code list representation. They considered the implications of having multiple versions of the same concept, such as a telephone entry, and how this would affect the transition between the two systems. Oliver expressed concerns about the uncertainty of where to land when transitioning from one version to another. The team also discussed the possibility of using an XSLT to produce the code list and category scheme, but Oliver was still unsure about the specifics of the transition process.

• Code Values and DDI Code List Discussion

  • Dan and Oliver discussed the concept of code values in the DDI CodeList, with Dan explaining that users would input these values in the CV properties. They also discussed the link between the CodeList and the SKOS ID for the code list, with Dan clarifying that the user ids in the codes are not versionable and always have the same version as their parent versionable. Oliver suggested the possibility of switching the user id into the code using an attribute pair. Dan confirmed that the code entries have the agency id, version, and user id, but they are not versionable objects and are treated differently in the DDI-Lifecycle.

• Discussing Controlled Vocabulary Terms in Data

  • Wendy and Dan discussed the use of controlled vocabulary terms in their data. Wendy expressed confusion about the purpose of these terms, particularly in relation to code lists. Dan clarified that the controlled vocabulary terms were meant to provide values for code value properties in the metadata, and should not be used as code lists for variable domains. Wendy agreed, noting that the terms could be useful for large datasets like country codes, but should not be reused. The conversation ended with Wendy expressing clarity on the purpose of the controlled vocabulary terms.

• Discussing Version Categorization Mechanism Changes

  • The team discussed the need to change the mechanism for categorizing versions, with Dan proposing a simpler system where the version increases to signify a new release. However, Darren expressed concerns about potential confusion if the changes aren't clearly explained. Oliver and DDI agreed that a detailed example of the proposed changes would be helpful for understanding the issue. They decided to create such an example and revisit the topic once it's ready.

Website Management Agreement Discussion

Wendy discussed the agreement between TC and ICPSR regarding website management. Darren suggested that the agreement could be between the TC and him, rather than ICPSR. The group agreed to proceed on this basis, with the understanding that the TC would manage the website. Darren also suggested that the website could be deployed onto the same AWS platform as http://docs.ddialliance.org next year, to keep everything on one ecosystem. The main point of clarification was ensuring that the TC's role in managing the website was clear.

• Website Management Roles and Responsibilities

  • The team discussed the roles and responsibilities of the four people managing their website. They agreed that two roles would be for content management and two for administration. The content roles would involve maintaining the website's content, currently dividing this in Product content and all else (primarily covering Admin and the Learn section). Distribution of content responsibility can shift was workloads and converage changes occur. Adminitrative roles would include branding, styling, and templates, as well as managing the database and file content. The administration roles would also include overseeing the website's performance, organization, and policy decisions. The team also discussed the possibility of an archivist role to monitor changes and ensure consistency across the site. They agreed that any major changes to the website would need to be reviewed and discussed by a broader group or the Executive Board, if there were costs involved.

• Admin roles

  • Maintain branding, organization, styling, templates

  • HubDB database management

  • File management

• Content roles

  • Product content

  • Admin/non-product content

• Archivist

  • Write up archivist

  • Analytics - could reside elsewhere

• Policy decisions - Initiated by TC with external review

Transition Timeline and Checklist Discussion

Wendy discussed the need for a timeline and checklist for the transition of their project. She agreed to create a Google spreadsheet to outline the tasks and dependencies. The goal is to have the transition completed by December 31st. Wendy also mentioned the need to handle content changes and ensure consistency between the ICPSR site and HubSpot site until final switch over is made. Darren suggested that the TC report on the Face-to-Face could be expanded into a chart with dates. Wendy also mentioned the agreement with ICPSR to shift over the entire ddi alliance.org DNS area and the status of the DDI-Lifecycle 4.0 Beta 2. Dan is still working on the ordered properties and once ready, they can send out the beta.

• Document Location Inquiry and Discussion

  • Wendy inquired about the location of documents moved by Jon into http://docs.ddialliance.org . Dan confirmed that he couldn't see any documents added to the http://docs.ddialliance.org , suggesting they might have been moved elsewhere. Darren found the top-level folder where Jon had copied all the files, which amounted to nearly a gigabyte's worth. Wendy decided to consult Jon to determine if this was their permanent location or a temporary one. Links to documents from web pages will need to be updated.
    Moved documents from website
    https://app-eu1.hubspot.com/files/144531336/?folderId=112327536857

FUTURE MEETINGS:
Wendy encouraged anyone with issues to send her a note for the next meeting.

ATTENDING: Wendy, Oliver, Jon, Darren, Dan, Jeremy, Flavio

Sep 05, 2024 09:56 AM Eastern Time (US and Canada) ID: 596 245 942
Quick recap
The team discussed the potential transfer of the ddialliance.org domain to their management, with a focus on setting up the same records in the current zone and then asking the current registrar to switch the name servers. They also discussed the coordination and management of legal and administrative tasks related to the UKDA project, with a focus on identifying roles and responsibilities for content management. Lastly, they discussed the need for a detailed report on the size of the content moved from the old site to HubSpot, the creation of a high-level document for codebook, and the progress and future plans for the DDI-Lifecycle version 4.

Next steps
Wendy to send a note to Jared about coordinating a call with Darren, Dan, and the appropriate ICPSR team members regarding the DNS transfer of http://ddialliance.org .

Wendy to follow up with Jared about the legal/management paperwork for the website and base configuration responsibilities.

Dan to submit a pull request with a first pass of marking ordered properties in DDI Lifecycle version 4 by the end of the month, in preparation for Beta 2 release.

Summary
DDI Alliance Domain Transfer Discussion

Wendy, Darren, and Dan discussed the potential of transferring the http://ddialliance.org domain to their management. They considered the pros and cons of this move, with the ICPSR team being in favor of the whole domain transfer. Darren suggested that the process should be straightforward, but they would need to set up the same records in the current zone and then ask the current registrar to switch the name servers. Dan proposed the possibility of transferring the resources to their account or setting it up to replicate it. Wendy agreed to set up a call with the right person in the DevOps or network team, including Darren and Dan, to further discuss the process.

UKDA Project Legal and Administrative Coordination

Wendy, Jon, and Darren discussed the coordination and management of legal and administrative tasks related to the UKDA project. They clarified the roles of named individuals in managing the AWS/HubSpot platform, with two individuals each for content and administration. They also discussed the need for clear roles and responsibilities, particularly for content management, and the potential for an oversight archivist role. The team agreed to identify these roles and individuals, with Darren offering to be an admin contact until the project goes live. Wendy suggested that they need to flesh out the roles and responsibilities further and identify the necessary people for the setup and ongoing management of the project.

Project Updates, Reports, and Expense Reimbursements

Wendy discussed the need to follow up with Jared regarding the legal aspects of a project and mentioned that she would be sending a note to Jared on this later in the day. She also mentioned the meeting report from the Technical Committee F2F and asked for feedback on the current draft. Jon suggested that the report from the face-to-face meeting was excellent and had no comments. Wendy also mentioned the need to fit the report into Jared's framework and asked for input on her list for the overall report. She also reminded the team about the 45-day deadline for expense reimbursement requests. The team discussed a form from Hilda that seemed to be mixing up two different things - the funding report for Jared and the annual or 6 monthly report from the working groups. It was decided that Wendy should send the report to Hilda and the financial report to Jared.

HubSpot Content Transition and Policy Discussion

Wendy expressed a need for a detailed report on the size of the content moved from the old site to HubSpot, including the number of pages and gigabytes of data. She also emphasized the importance of having a clear policy on what content should be archived and what should be retained as pages or moved to PDFs. Wendy suggested that a timeline for the transition should be established, with a goal of completing the move by EDDI with a hard deadline of December 31st. She also highlighted the need for communication with the DDI community about the changes and the expected timeline. Oliver agreed to work on a proposal for the transition of news elements from the old site to HubSpot.

Creating a High-Level DDI-Codebook Best Practices Document

Wendy discussed the need for a high-level document for codebook, which would contain best practices and information on the use of identifiers and controlled vocabularies, etc. She questioned whether this document should be part of the codebook package or a standalone best practices document. Dan suggested it should be a standalone document, as the usage document should not be tied to a specific version of codebook, given its backwards compatibility. Wendy agreed, noting that this would provide more flexibility in the document's tone and content.

DDI-Lifecycle Version 4 Progress and Future Plans

Wendy, Dan, and Flavio discussed the progress and future plans for the DDI-Lifecycle version 4.0 BETA2. Dan suggested a first pass at marking properties as ordered before setting out a BETA2 for review. Wendy agreed to this and mentioned a preliminary list from a previous issue. Dan proposed to submit a pull request with his first pass of ordered elements and asked for feedback from those interested in the RDF version. The team aimed to have the BETA2 version out by the end of the month (September). Flavio was tasked with reviewing the draft report for the scientific board meeting, which Wendy will send in by Monday. The funding report will be completed and sent to Jared. The team also discussed the roadmap for the next 12 to 18 months. This will be on next weeks agenda.

ATTENDEES: Wendy, Jon, Darren, Jeremy, Dan
REGRETS: Oliver
Relevant links:

XKOS:
https://rdf-vocabulary.ddialliance.org/xkos
CONTROLLED VOCABULARIES
https://ddialliance.org/controlled-vocabularies

redirect to:
https://rdf-vocabulary.ddialliance.org/cv

CDI example of HTML redirect
https://rdf-vocabulary.ddialliance.org/ddi-cdi/Agent

Quick recap

The team resumed collaboration with a content professional, discussed the progress of the RDF project, and considered using GitHub as a production tool. They also prioritized tasks for the upcoming quarters, identified key areas of focus, and discussed access and responsibilities for the new website. Lastly, they highlighted the need for finalizing content due to upcoming changes in best practices and the completion of the DNS change.

Next steps

• Darren to create a task list of remaining items for the website project, including data-driven pages, templates, styling, and mail shots.

• Darren to work with Michigan to redirect the RDF vocabulary URL and test the preservation of headers in the 301 redirect.

• TC members to review the roadmap document and provide feedback on priorities for the last half of 2024 and first half of 2025, including suggestions for new items or changes to existing ones.

Summary

Resumed Collaboration and Content Updates

Darren reported that they have resumed collaboration with the content professional, enabling re-editing of pages. He proposed creating an item list of outstanding tasks for the next team meeting. Darren also sought clarification on the tasks related to mail shots. Wendy emphasized the urgency of finalizing the content, highlighting the upcoming changes in best practices and the need to inform the working groups about the process. Lastly, Darren noted that the DNS change had been completed by DM , (Michigan).

RDF Project Progress and Redirect Testing

Darren discussed the progress of the RDF project, focusing on the upcoming redirect of the DDI-CDI page to the new RDF host. He also mentioned the need to test the redirect process to ensure the preservation of headers and the functionality of the redirect. Darren further shared his plan to improve the user experience by making concept URIs more accessible on the HTML pages and intended to discuss this with Oliver. The team also considered the idea of a poster session or webinar to present the project's progress and new infrastructure changes, with Jon suggesting a 15-20 minute session on the website. This could be combined with Roadmap session, and CV presentation.

GitHub Production Tool and RDF Ontology

Dan and Wendy discussed the use of GitHub as a production tool for XKOS and codebook. They recognized the need to expand the roadmap to include website-related work they recently took on. Wendy explained the Equivalency group's work on developing an RDF Ontology model to represent common content and concepts across DDI products. Wendy and Dan explored the idea of creating an underlying database mapping objects and content across different DDI products to enable consistent transformation and facilitate high-level understanding of DDI's overall coverage.

Prioritizing Tasks and Website Responsibilities

Wendy discussed the need for prioritizing tasks and identifying key areas of focus for the upcoming quarters. Wendy requested input from the team on their priorities and asked for a list of tasks that need to be broken out or given new subheadings. The team also discussed the access and responsibilities for the new website, with a limit of four people identified for administration and content. The team expects the CDI work to be completed by late September or early October.

ATTENDEES: Wendy, Jon, Oliver, Darren, Jeremy, Dan
CDI update

Roadmap/timeline requirement
There are documents on what is required and funding for implementation was part of this years request. They will prioritize this work product. Achim is back on Sept 11 and we should get roadmap and timeline shortly afterwards. Achim and Deirdre will work with Oliver on setting up pipelines in established DDI production work areas.

CDI timeline
They plan on submitting the package for voting process by Friday 20 September. Out goal will be to get it turned around by 27 September or shortly thereafter. Tentative date for the webinar is Thursday 3 October (of following week if necessary). The voting period should start within a 3-7 window on either side of this date. Wendy will coordinate with Jared. I noted that we will still be running a dual website at that point so the presentation of products may shift some pre- and post-voting period.

CV update
Request for DNS change should be made in next few days and be functional approximately 48 later. This will then be tested, followed by deployment, and then set up of RDF vocabulary resolution platform. Arofan will check on status 3 September.

Sanda wants to do more changes on the home page. Should be live next week. Entry shifts from Jared to Darren

HubSpot update
HubSpot needs to switch to US HubSpot outlet
Jon is talking to Jared on Monday to get the quote sorted and processed
We have a contact on the finance end at DDI
Darren has export .5 GIG of files and tables

Oliver - website transition

• Tried importing the News item and then stumbled into nightmare of html from work imports. Think of transferring last 2-3 years of news by hand into the system.

• Do yearly dump to PDF of news items.

• Make sure this is ok with Jared and note it as a policy.

• docs.ddialliance.org waiting for DNS change

Product section

• What is going where and how is this organized

• the doc site will contain information for people technically implementing

• Web will have informative text content for understanding individual products. Add links to key product content:

  • Technical/Implementation button

  • How to make this visual being able to use useful glif/icons

  • Just waiting for the new DNS update

Presentation proposal for EDDI on Roadmap

• Updating Roadmap (priorities, timelines, modifications)

• Next week review Roadmap and finalize submission - look at the roadmap in the meantime https://docs.google.com/spreadsheets/d/1b390D4CpPXqtqi7-e5GhrvWdHbNEG697/edit?gid=728820206#gid=728820206

• Other submissions to EDDI:

  • Darren - CV platform

  • Dan/Jeremy - Colectica

  • Oliver - Colectica

ZOOM SUMMARY (edited)

Meeting summary for Technical Committee (08/22/2024)

Quick recap

The team discussed issues related to payment processing and the need for a new management system, with a focus on using Hubspot's American side. They also reviewed the roadmap for upcoming projects, including the Achim's project, CDI timeline, and the need to update internal links. Lastly, they deliberated on the organization and layout of content on their website and Hubspot pages, and the importance of managing ongoing tasks.

Next steps

• All team members to review the roadmap document and provide comments on priorities and changes needed for the coming year before the next TC meeting.
• Dan to work on implementing icons or glyphs for common elements (e.g. documentation, downloads) on the product pages once the DNS entry is updated.
• DDI team to finalize the product section organization and content flow between the website and technical documentation.

Summary

Payment Processing and Roadmap Discussion
Wendy and Jon discussed issues related to payment processing and the need to find a new way to manage it. They decided to start using the American side of Hubspot to resolve the payment issue, with Jared's assistance. Darren had downloaded all necessary files to rebuild if needed. They also discussed the need to update internal links and identify moved documents. Lastly, Wendy presented a draft roadmap for the upcoming year, which would be updated online for priorities, timelines, and modifications.

Project Timelines and Database Import
The team discussed the roadmap timeline requirements for the CDI project, with Arofan's funding request for $9,000 (setting up the pipeline was part of the request and will be prioritized by the CDI group). The Achim is back on September 11th, and is a key component for this work. The CDI timeline included submitting a package for voting by September 20th, with a webinar planned for October 3rd.

Oliver and Darren discussed the process of importing a database and considered transferring the last two to three years of news into Hubspot. The CV update involved a request for DNS change expected to be functional within 48 hours. Darren planned to make changes to the home page after the redirection from the control recoveries Page was complete.

TC's Roadmap and Prioritization Discussion
Wendy presented a roadmap for TC work and asked for input on what needs to be expanded, changed, or prioritized. Wendy noted that the last cycle of certain product developments would be completed this year. Wendy also mentioned the need to shift focus to tools for assisting with usage and production management. Wendy asked for feedback on prioritizing tasks for this year and identifying any projects that can't be dropped.

updates regarding web site, Hubspot, and docs site.

Wendy and Darren discussed the ongoing work on the Hubspot and the need to reset it with the US side due to address-related issues. Darren confirmed that he had captured all the content to prevent loss. An issue with the docs site was raised by Dan, as the DNS still hadn't been added to the site. (Note this has now been accomplished). Lastly, Wendy brought up the product section, which she and Darren had been working on, and proposed to discuss its progress and vision in the next meeting.

Website Content Organization and Layout
Wendy and Dan discussed the organization and layout of content on their website and Hubspot pages. Wendy expressed a need for a clearer understanding of the flow from website pages to the options set up on the other side (docs and github content). Dan explained that the site would be for people wanting to learn more or technically implement one of the products, and suggested using icons to indicate different types of content. They also discussed the need to wait for a new DNS entry before making links, and the possibility of adding a page with all the versions of a product. The group also needed to decide what content needed to be included and where.

CDI Project Progress and Roadmap
Wendy emphasized the importance of managing ongoing tasks and the potential acceleration of the CDI project. A polished proposal for Eddie was requested, with feedback on the presentation also sought. The team discussed the CDI's roadmap plans, including automation funding and pipeline setup. Issues with CDI 1.0 and 1.1 versions were addressed, with Jeremy advising against releasing multiple incompatible schemas (CDI-101). The conversation ended without any significant decisions or action items, with Darren, Dan, and Oliver offering their availability for future communication.

THURSDAY
ATTENDEES: Wendy, Oliver, Jon, Darren, Jeremy, Dan

Thanks to Jon for replenishing various caffeine supplies.

Home page
https://144531336.hs-sites-eu1.com/en-us/

Editing page
https://app-eu1.hubspot.com/invite-user/2hS9GVzT8oBGMpNa?lang=en&via=link

Inventory of pages and status
https://docs.google.com/document/d/1DIPnvDMWo6SNxxf7bH4O5cwWCUWkrHL-aDlRNCqVM9M/edit?usp=sharing

Notes for editing guide:
From Website Pages
pulldown More tools [Advanced Menus]
Brings up Navigation menu

Select parent page and hover over white space at right of box to get Action icon
Click on Action

Settings
Change name and title
Update slug on URL to short meaningful NCNAME

Actions:

• Remaining subpages for Learn DONE

• Events - DONE

• CDI review ticket - Jeremy DONE

• Products have been moved DONE (down through product page - version pages being shifted to GitHub location)

• Data cleaning on data driven pages DONE

• Working group presentations to the web site from confluence DONE

• Documents have all been moved (copied) DONE

• Databases have been moved and now drive pages: Tools, Members, Contact (need review and pulling things together) DONE

• Currently importing all announcements (can take 6+ hours) DONE

• 4 automated build pipelines are now in doc.ddialliance.org (ddi-l 4.0, ddi-l 4.0 guidelines document, SDTL and CV) DONE

TO DO:

• wlt - Correcting internal page links from new site to http://ddialliance.org site - Wendy will correct these this weekend DONE

• Testing of new domain

• Review organization/navigation and make known changes at least in Products (TC area)

• Add link to Dan's site on doc.ddialliance.org for product content from the basic product page

• wlt - Go through and change DDI-L to DDI-Lifecycle etc to be consistent (DDI-CDI, DDI-C, Disco, SDTL, XKOS) DONE

• New site announcement to members/users (will complete closer to changeover date) - write Jared a newsletter piece on progress

• Brand/style guidelines - including consistent navigation approach

Wednesday
ATTENDEES: Wendy, Oliver, Jon, Darren, Jeremy, Dan

Thanks to Jon for replenishing various caffeine supplies.

• Received quote from HubSpot and discussed with Jared.

• Worked on issue #36 on DDI-L version 4.0 model, resolved all items in spreadsheet and entered in COGS

• Approved and merged pull request in ddimodel

• Met with Michael and Jared regarding HubSpot, DNS, and database content/usage. Resolved questions

• Resolved some issues concerning HTML page of CVs to remove duplication

• Exploring use of HubSpot features and how to do things like filters - identifying best approaches

• Fixed misspelling of Abstract in DDI-L version 4.0 BETA2

Discussion

• Where different doc types go? GitHub or S3

• GitHub is easier to load, drag, and drop

• Roadmap for moving from hand/semi-hand crafted to automated

• Check with Deirdra on GitHub location

ACTIONS:

• Jon is sending specific email Jared with link to product, quote, and reasoning. DONE

• Dan will follow up with Jared regarding DNS zone change

• Oliver will email Sanda with proposed solution including caveats for version info, and usage DONE

• Write guidelines for updating web site using HubSpot

• Thursday Darren will get some data cleaned and into tables DONE

• Oliver will create a table of tools, members, publications DONE

• Look at mail type functionality so that Jared can use it for targeted mailings DONE

FUTURE TO DO:

• Review examples to make sure they are parsable - moving to GitHub will let us provide a validation process to make sure these are accurate

• Profiles should be community generated but will need in various languages - recommended ways to express profiles

• ICPSR metadata with a JSON schema (look at this to see if there are objects not supported by DDI)

TUESDAY
ATTENDEES: Wendy, Oliver, Jon, Darren, Jeremy, Dan

Many thanks to Jon for supplying not only the rooms but the snacks and most of the the Coke Zero and coffee/tea to keep us all adequately caffeinated.

AGENDA:
https://docs.google.com/document/d/1DIPnvDMWo6SNxxf7bH4O5cwWCUWkrHL-aDlRNCqVM9M/edit?usp=sharing
Content spreadsheet:
https://docs.google.com/spreadsheets/d/1bcvxmwoxUWyhZfTkuR9ZrVtmCxz2mkE8cFYYyjpCLtM/edit?usp=sharing
Dump:
https://drive.google.com/drive/folders/1_imyqyrS59Of25Yz-NNcMYhwvhUkTCK_
Published site (HubSpot):
https://144531336.hs-sites-eu1.com/en-us/

DECISIONS:

• ICPSR has extended move date from Sept 1 to Dec 31 so do not need Plan C

• Complete work as soon as possible - preferably by EDDI

• Use HubSpot- explore upgrade options (limit 5 users, 30 pages in free version)

Discussions:

Choice of software - main point is to get out of self maintained
Distribution of content:

• Static and generated content (admin) on HubSpot, documents on AWS S3buckets - field level documentation generated from GitHub, documents

• Looked at how much needs to be on web and what can be off on doc.ddialliance and maintained on GitHub in terms of products. Need to flatten and provide consistent access.

ACTIVITIES:
Exploring the site dump for content specifics

QUESTIONS FOR MICHAEL:

• Request to change DNS for doc site

• Create a single table with all the values for membership and other data bases

• What is the difference between Publications and Bibliography?

ACTIONS:

• Darren will go in to HubSpot to check on functionality at different levels for databases, contact submissions, use for internal as well as external uses of database content. DONE

• Dan will set up Route 53 zone and send email to Michigan regarding actions needed DONE

• To set up the DNS for the technical documentation website, we need to request from ICPSR to delegate NS for http://docs.ddialliance.org to: (DONE)
  http://docs.ddialliance.org . IN NS ns-1840.awsdns-38.co.uk.
  http://docs.ddialliance.org . IN NS ns-1071.awsdns-05.org.
  http://docs.ddialliance.org . IN NS ns-901.awsdns-48.net.
  http://docs.ddialliance.org . IN NS ns-421.awsdns-52.com.

• Darren is setting up page templates so content of static pages can be loaded DONE

• Send Jared table of members as excel to see if it contains all that he needs. Is there additional information that needs to be exported (example is member email addrss maintained separately?) DONE

• Entered all first level pages under Learn (DONE)

• Need to check for links to forms etc that would need to be changed

• News and Members from database (can redo if we need to clarify) DONE

• About to do same for publication (DONE)

• About sections DONE

• LEARN first level pages and sub-levels entered DOE

• Moving published product documentation onto doc.ddialliance (will create landiing page) DONE

• Mirroring existing site (Plan C) DONE

• Membership navigation structure DONE

• Created base pages DONE

• Publications (need to chat about) DONE

• Work on styles started

• At limit for pages at no cost, HotSpot will give us a couple professional tools for a couple of weeks DONE

• Membership pages DONE

• Added Learn, Membership, Events, Publications, About to inventory spreadsheet completing sub-page listings (left off straight links to existing pages on hierarchies of other sections, documents, or other sites DONE

CHANGES TO PAGES

• Membership entry page content changed to a drop-down

• Lost brown/blue/green color boxes (for consistent item types like "download", "XML entry point", "Field Level documentation",etc. will look at adding icons)

ATTENDEES: Wendy, Jon, Oliver, Dan, Jeremy, Darren, Jared (guest)
Regrets: Flavio

F2F and Web site planning

See https://docs.google.com/document/d/1DIPnvDMWo6SNxxf7bH4O5cwWCUWkrHL-aDlRNCqVM9M/edit?usp=sharing

ACTIONS:
Jared:

1. Updated dump of database - completed 2024-08-08

2. Functionality of administrative section

3. Meeting time with Michael - set for 2024-08-14 8:30 EDT, 13:30 London

Wendy: (Notification to Content Managers)

1. Hold on updates during next week (Training, Admin, Marketing, Product) - completed 2024-08-08

Text of email:
Chantal, Kathryn, Jon, Jared, Hilde, and Arofan,

I am writing to you as content managers on the http://ddialliance.org site. I have included Hilde and Arofan as they are preparing DDI-CDI for a vote in September and may need content changes in the website regarding this.

The Technical Committee will be beginning the process of moving the http://ddialliance.org site from ICPSR to the UKDA. ICPSR will be creating downloads of the site content between now and Tuesday 13 August. We will then be creating a prototype of the new site. We will be running parallel sites while we check out the accuracy and stability of the new site. We would like to keep any changes to an absolute minimum for a period of time. We should be able to provide an estimate of this time period by the end of next week.

To that end please:

1. Do not make any changes to the current http://ddialliance.org site before August 20.

2. Follow guidelines that the TC will provide before August 20 regarding any necessary changes made for a period of time following August 20.

3. If you need to make content changes before August 20 or during the limited access period defined in the guidelines we will provide, please contact me to ensure that changes are reflected on both sites

4. We understand that upcoming events such as the CDI vote will require some changes as well as needed announcements. We will keep this period of limited change as brief as possible. We simply want to ensure that content changes are made to both sites in a consistent manner.

Thank you for your help with this move,

Use of Zoom Summary
Jared forwarded the summary from last week
Wendy has sent to members for review and comment
Looks useful, will need to configure meeting set up to have summary sent directly

Post meeting:
Jared raised 2 additional issues in an email which will be added to agenda outline

1. Web hosting expenses. What will they be and through which provider. If above a certain amount, I may need to vet the provider through UM.

2. Administration. I'd us to keep the number of people who can make direct edits on the site to a low number. Prior to the clean up a few years ago, we had 70+ users with edit permission. Most hadn't logged in for years (and I removed their access due to security risks), but the disparate editors sometimes made for dislocated content. My recommendation is to have the TC edit the Products section. For the other content (mainly involving Training and Marketing), I suggest requests work through me. This helps make sure we have consistent messaging. Thoughts?

ZOOM SUMMARY (edited)

Meeting summary for Technical Committee (08/08/2024)

Quick recap

The team discussed upcoming website changes, a security incident involving the University's website, and the issue of language repetition in HTML output. They also planned for the organization and storage of documents in the new 'docs.ddialliance.org' website, and the need for improved organization and descriptions for their working groups. Lastly, they arranged for transportation to UCL for upcoming meetings and discussed the transfer of the current site's content to a new database.

Next steps

• Jared to provide an updated database dump of the current website content to Darren.
• Jared to compile a list of current administrative functionalities on the website for the team.
• Jared to schedule a meeting with Michael (web team) for the following week to discuss potential needs.
• Wendy to send notification to content managers about holding off on website updates during the transition period.
• John to send his phone number to the team for emergencies during the London meeting.

Summary

Upcoming Travel, AI Tool, and New Team Members
Jon, Wendy, Dan, and Jeremy discussed their upcoming travel plans and the use of a new AI summary tool. Wendy expressed difficulty in accessing the DDI Alliance Mail to share the summary, and Jared offered to forward it.

Website Changes and Emus Discussion
The main agenda was then addressed - plans to announce upcoming website changes to members and users, gather feedback, and determine rollout strategy. Wendy shared a document outlining these plans and asked Jared for his input on prioritizing next steps.

University Website Security Incident Discussion
Jared discussed a security incident involving the University's website, where malicious ads were injected. The University's Information Assurance team, responsible for a major incident a year ago, wanted to take the site down due to the security incident. However, Jared disagreed and managed to get a commitment from the team to keep the existing site up until December, with a possibility of minor internal shifting. Wendy questioned the need for a backup plan, given the issues with the current setup and functionalities, and the ongoing work on the prototype.

Addressing HTML Repetition in RDF Release
The team discussed the issue of language repetition in the HTML output of the CV group, which was identified as a potential problem for the upcoming RDF release. Oliver and Wendy had been exploring the possibility of transforming the HTML output to eliminate duplication, a solution preferred over holding up the entire chain. Despite concerns about the aesthetic of the output, the team agreed that the priority was to have the RDF resolution system in place for the CDI release. The team planned to further discuss and resolve the issue in their next meeting.

News Item URL Structure Discussion
Jared and Darren discussed the potential changes to the URL structure of the news items on their website. Jared mentioned that there were over 50 news items, and Darren was unable to access the site to confirm the details. Oliver suggested setting up redirect rules if the URLs did change. There was also a discussion about the possibility of only importing news from the last year to support archives, but this would mean losing older content. Darren agreed to investigate the redirection issue, and Jon checked if the current URLs were supported by Hubspot.

Improving Member List and Publication Functions
Jared discussed the functionality of the Member List section and expressed interest in improving its display capabilities. Darren suggested that 80-90% of the new prototype should be completed within a week for feedback, with the remaining 10% to be addressed in the following weeks. Jared also brought up the topic of the publication section's functionality in the new site. Wendy requested Jared to list any additional functionalities to better understand the current system and identify any gaps or issues.

Organizing Documents on New Website
The team discussed the organization and storage of documents in the new 'docs.ddialliance.org' website. Jared, Darren, and Wendy discussed the need for a balance between HTML pages and PDF documents, with the intention of providing users with both easy-to-access HTML content and the security of PDF backups. They also discussed the potential for storing these documents in an S3 bucket, rather than internally on the website. The team agreed on the importance of having a clear and intentional structure for the new site.

Improving Working Group Organization and Communication
Jared, Jon, and Wendy discussed the need for improved organization and better descriptions for their working groups. They agreed on the importance of having text, not just links, for easier navigation and decided to move forward with the plan to transfer content while keeping functionality. Wendy proposed that a placeholder for an upcoming change be included in the next newsletter, with further details to be discussed later in the year with working groups. Wendy also requested that future communications be copied to Oliver, who is now the vice chair, and mentioned that she and Oliver would be reviewing the distribution of tasks within the committee. Lastly, Wendy requested a list of functionalities for the administrative sections from Jared for the following week.

Database Dump and Site Updates Strategy
Jared will ask Michael to provide another database dump for the current site's content, which will be sent to Darren. The team will discuss the best time for this transfer, likely on Wednesday. The team also agreed to hold off on making updates to the site until after the switch, with Jared suggesting he might create new content items instead. The team will reassess their approach after the prototype is up and before finalizing the site.

Transportation, Arrivals, and Notifications
Wendy and Jon discussed transportation arrangements for themselves and the others attending the meetings at UCL next week, with Jon recommending the Elizabeth line. Darren confirmed his arrival time on Tuesday morning. Wendy assigned an action for Jon to send emergency contact details. Wendy also assigned an action for herself to send a notification to the content managers and remind Jared about updating the database functionality and scheduling a meeting with Michael.

ATTENDEES: Wendy, Jon, Dan, Jeremy, Oliver, Flavio

Codebook - Pull Request 41

F2F
Jared's email
This is to check on the status of the web site migration. When do you plan to switch it over to the new host? When can we turn off the ICPSR-hosted site? Will we retain the same URLs at the new hosted site? Do you need our web team to be available for a few hours during the planned switchover? For content transfer, is this inventory still accurate?

I had one to-do relating to formalizing an agreement about who is doing what for web and resolution support. I recently spoke with the UM General Counsel office about creating legal agreements for web site hosting, maintenance, and support, as well as supporting the resolution system that has been set up to support CV and RDF vocabulary resolution. While we do not need details to create a draft for the web site actions, can you please specify the specific tasks and deliverables for supporting the resolution system?

I'm out next week without access to email or phone but will be back on the 5th. Can we talk then? I'm especially interested in how the Secretariat-owned sections, like Get Involved, Publications, About will function on the new site. I'm also interested in communicating these planned changes to the wider community.

Notes from CDI for TC:

Lifecycle using CDI as a bridge standard to SDMX
RDF expressions of DDI versions are moving forward
See if we can get Eurostat to look at the RDF from Lifecycle
UNECE is having a meeting 3rd week of October with side meeting of CDI/SDMX alignments
Eurostat has a conference in March NTTS (submission date 30th of September)
https://cros.ec.europa.eu/dashboard/ntts2025

CDIF
http://Schema.org in the middle pointing out to different things
Handling handoff between CDIF and DDI is CDI
There hasn't been a discussion of Products that should be involved

TC is accountable for some outreach activities and we need to coordinate this better with Scientific Board and other outreach activities. There should be broad support for the Product Suite with targeting taking place where appropriate.

FROM ZOOM SUMMARY (edited)

Meeting summary for Technical Committee (08/01/2024)

Quick recap

The team addressed various issues related to the code book, the website's hosting location, and the potential risks of transferring content to a functional site. They also discussed the need for improvements in the content management system, the display of language labels in CV groups, and the organization of documents on the docs.alliance.org website. Lastly, they touched upon upcoming developments from UNECE and Eurostat, and the importance of transparency and consultation with the technical committee.

Next steps

• Oliver to implement corrections for the HTML and SKOS files for the controlled vocabulary site by the end of the London meeting week.
• Wendy to write to Jared expressing that it's unreasonable for ICPSR to move the deadline for the site migration to September 1st with such short notice.
• Dan to add a task for the London meeting to create a static version of the DDI site as a backup plan.
• Jon and Darren to evaluate and provide feedback next Thursday on the feasibility of moving the majority of the site content by the end of August.

Summary

Addressing Code Book Issues and Feedback
Wendy addressed several outstanding issues related to Codebook. Wendy clarified the meanings of certain attributes, corrected a misspelling of 'orchid', and improved the formatting and clarity of the documentation. Wendy also planned to review codebook tree issues from Knut. Dan and Oliver provided feedback and suggestions for the commit message when merging the changes. Wendy thanked them for their input and agreed to implement their suggestions.

Website Hosting Change and Challenges
Jared from ICPSR notified the team that due to recent security issues, they need to move their website from the current hosting location by September 1st, much earlier than the previously agreed December 2025 deadline. The team discussed the challenges posed by this shortened timeline and the need for a temporary hosting solution during the transition period. Dan and Jon suggested pushing back on the new deadline as unreasonable with such short notice, but also proposed discussing interim options like a backup site with Jared and Dan in the next meeting.

Site Transfer and Backup Plan Discussion
Jon, Wendy, Dan, and Oliver discussed the potential risks and solutions related to moving the majority of their site over to a functional site by the end of August. Jon expressed his belief that the site's content could be successfully transferred, but highlighted the challenges with DNS and other related issues. As a backup plan, Dan suggested creating a static version of the site. Oliver agreed, noting that the only interactive part of the site was the search functionality, which could be removed. Jon proposed directing their concerns to Jared, suggesting an extension as a more reasonable solution. Wendy agreed to draft a letter to Jared, with Jon's review for accuracy.

Addressing Duplicate Language Labels
Wendy and Oliver discussed the issue of duplicate language labels in the HTML and SKOS files for the controlled vocabulary site. They agreed that this duplication was unintended and needed to be addressed, with potential solutions including transforming the SKOS file. They also discussed the availability of multiple language versions in the HTML for their publication plan, with Oliver explaining their current setup using secondary HTML pool methods through the REST API. Wendy acknowledged the convenience of having multiple languages, and they agreed to consider whether they wanted to adopt the same approach.

Addressing Content Management System Issues
Oliver and Wendy discussed issues related to the use of a content management system. Oliver explained that the system generates files each time a publish action is performed, rather than on the fly. He also pointed out that the current versioning system for languages was causing problems. Wendy expressed concerns about the time it would take to make changes, to which Oliver suggested two potential solutions: making corrections during his trip to London and starting a new service without the current system. Wendy agreed to these proposals, emphasizing the importance of getting the project up and running.

Addressing CV Groups Language Label Issue
The team discussed the issue of language labels not displaying correctly in the CV groups, as raised by the CV group. Dan expressed his opinion that this issue doesn't affect the usage and suggested improving the rendering. Oliver agreed but noted that users might prefer to wait until the error is fixed before setting it to hold. Wendy proposed a temporary fix and prioritized it, as it was only for a limited time. The team also discussed the need for a flag to notify users of errant duplication in multiple languages, which Wendy suggested would not significantly impact overall usability.

docs.alliance.org Website Structure
Wendy and Jon discussed the organization and presentation of documents and specifications on the docs.alliance.org website. Jon suggested that the site should provide options for users to make decisions and that specifications should have a presence on the main site. They also discussed the potential for confusion between the docs and main sites, and the need for clear guidelines. Dan added that the new system should accommodate automatically generated artifacts from specifications. The team agreed to further clarify the site's structure and content.

Upcoming Developments and Transparency Concerns
Wendy discussed upcoming developments from the UNECE and Eurostat, particularly regarding Lifecycle v4.0 and the use of RDF. Wendy also mentioned an upcoming conference with a submission deadline of 30th September. Jon expressed concerns about the lack of transparency and consultation with the technical committee regarding these initiatives. Wendy agreed to discuss these issues further with Oliver, the new vice chair, and to consider the committee's involvement in future activities.

ATTENDEES: Wendy, Dan, Jeremy, Adrian Dusa (guest)
Regrets: Jon, Darren, Oliver, Flavio

Codebook pull request

approved, merged, related issues closed

Codebook issue #18
Adrian Dusa will join the meeting for this discussion. Please look at the issue prior to the meeting.
https://github.com/ddialliance/ddi-c_2/issues/18

ADDED to Issue #18 by Dan: At this week's TC meeting, we discussed the current attribute documentation and if it makes sense to add explicit documentation at the attribute level such as

<xs:attribute name="..." ...>
<xs:annotation>
<xs:documentation>...</xs:documentation>
</xs:annotation>
</xs:attribute>

Adrian will provide examples of end products for suggested changes. These will be vetted with Jon regarding docbook HTML documenation creation. Timing this for the last set of structural changes to be made to schema prior to presentation for vote. Goal is to provide consistency in presentation for ease of reading and to better support capture and transference of documentation to other tools for codebook automated maintenance or new and existing codebook tools. May want to run this by Mehmood for WorldBank tool and Tools group (at least Olof for his tools).

DDI-L 4.0 BETA2 issues - continue to locate previous discussions and link or copy to issues. Whatever we can get done prior to F2F is great but at a minimum issues for discussion should be clearly defined to have a focused discussion.

Web/F2F prep
Jon has added some notes on doc.ddialiance.org organization as well as on the products section
Please look at these and comment or add to. One issue may be looking at when we get into reorganization of section content. For products some is being done before the F2F as we discussed in in March. Other reorganization may need to take place after the F2F within the context of overall revision of the web site. The primary goal of the F2F is to lift and move the current ICPSR site content and make sure there is no loss. Organization of new areas like doc.ddialliance.org are definately on the F2F agenda as we want those well organized and usable in September for new publications.

https://docs.google.com/document/d/1DIPnvDMWo6SNxxf7bH4O5cwWCUWkrHL-aDlRNCqVM9M/edit?usp=sharing

CHECK out the AI Companion and Start Summary for assistance with minutes - play with this next week

ATTENDEES: Wendy, Jon, Dan, Flavio, Jeremy

F2F preparation
Face-to-Face
https://docs.google.com/document/d/1DIPnvDMWo6SNxxf7bH4O5cwWCUWkrHL-aDlRNCqVM9M/edit?usp=sharing
Website Management
https://docs.google.com/document/d/1ihi4UlL4calygWp1wgFuO8Npqh_xPvtl4evAtXB-rEQ/edit?usp=sharing

Web pages

• Added page on filing issues and product level pages as per discussed earlier in the year

• Overview of what's in the suite and comparison of - rename this

• Contents listed on product page for each version (should list occur here or just link

• Can this be flatted out more

Codebook

• Decide layout plan (indention, line spacing) and prepare validation process in GitHub

• Delay cleanup of structure until last step

ATTENDEES: Wendy, Jon, Flavio, Dan, Jeremy

Codebook pull request - merged

DDI-CDI next steps -
Review processing issues - none filed
Verify URN for version 1.0 [Darren and Arofan] generic ddialliance.org/specification/DDI-CDI will be used for the general project page with links to each version
Check with about the listed issues

Face-to-Face meeting prep
Use next 4 meetings to ensure we have a clear listing of steps to complete during F-to-F
Includes getting a DDI-L v4.0 BETA2 out the door

4.0 critical issues in preparing BETA2

• Ordering

• References extension

Documentation websites on GitHub get it on CI automation -> docs.ddialliance.org
Process for putting all other docs on the documentation website - let DDI-CDI
There may be a standard Header/Footer for common look and feel

FUTURE AGENDA ITEMS:
Face-to-Face meeting prep
Website prep - get dates

ATTENDEES: Wendy, Jon, Oliver, Jeremy, Flavio, Dan

REGRETS: Darren

As of 1 July 2024 Jon Johnson will step down as Vice Chair of the Technical Committee. Oliver Hopt will serve as TC Vice Chair from 7/2024 through 6/2027.

Codebook pull request

• Approved and merged - covering references to DDI CV’s, typos, missing documentation

Codebook - Simple Text Review

• Approved 8 identified changes out of 86 simpleTextType that seemed logical (respUnit as parallel to anlyUnit) or where known non-DDI CVs exist within a discipline or organization (imputation types, availability status)

• Make those changes and retain spreadsheet for future Codebook Working Group

DDI-L v.4.0 BETA

• Priorities are ordering and internal enumeration issues

• Many issues administrative; Jon will add content to these

• Face-to-Face will be 3 days and we should be prepared to spend on BETA work to the extent possible

ACTION

• Wendy will provide a google doc to list technical and content management issues for the Alliance website https://docs.google.com/document/d/1ihi4UlL4calygWp1wgFuO8Npqh_xPvtl4evAtXB-rEQ/edit?usp=sharing

• Please add to this document with additional issues or expansion of listed issues

ATTENDEES: Wendy, Oliver, Dan, Jeremy, Flavio
REGRETS: Jon, Darren

Codebook pull request
Merged license update issues 97 and 93

Codebook review of simpleTextTypes
Issue arose in adding in documentation on existing DDI CVs of possible simpleTextTypes that should be conceptType or conceptualTextType (ex. “software”

ACTION:

Create annotated list of these on spreadsheet and send out
send out to SRG list to open for other to comment in open google doc if comments needed
Wendy do this and send it out

DDI-L V4.0 BETA issues and workplan
Pull request adding license (issue 32) merged
People should review issues and work on one of their choice
It would be good to have content changes (as opposed to administrative issues) addressed by August meeting so that if there is time we can move on getting BETA2 out for review.

ATTENDEES: Wendy, Darren, Dan, Jeremy, Oliver, Flavio
REGRETS: Jon

TC Chair and Vicechair for July 2024 - June 2027
Main concern is that the role is too much. We need to pull out Lifecycle and Codebook and look at the workload overall to spread out among the group.

Face-to-Face meeting
Thanks for your responses. I'll let you know the preferred times although we still need to adjust to the availability of the hosting agency(s).
13-15 August

Work on DDI-L V.4.0 BETA2
scheduling this work given people's availability
Preferred approach is to just starting to put it on the schedule
Marking the properties that need explicit ordering for RDF serialization

Codebook - please respond to the request for comments on the most recent issue. I will submit the pull request around 6 pm CDT Wednesday. High level documentation - outline
Pull request merged. One typo found DDICODE-540. Enter remaining schema updates DDICODE-540, 97, and 93. Change merged IN REVIEW items to closed. Start documentation issues that are related to high-level documentation.

ATTENDEES: Wendy, Dan, Oliver, Flavio
REGRETS: Jon

Codebook pull request
check vocabID or vocabName?
lines: 2558, 2797, 4336, 4574
Follow-up findings: vocabName is not an attribute (vocab); vocabID was added to replicate content from DDI-L. Documentation on vocabID was updated to clarify the use.
DDICODE-539 - Getting issue details... STATUS

Face-to-Face Meeting
Oliver not available in July 31st of July is first day available
Flavio remote availble in July/August
Dan
Begining of the week of 22nd OK
Begining of August 1-2nd goood
August OK except middle of month weekend
Poll - for availability

CDI review
Extend to end of June - email everyone with a prod to finish and file their work

Glossary review
Nudge to Training group people

Feedback from SB and EB meetings
no additional comments

ATTENDEES: Wendy, Jon, Darren, Oliver, Dan, Flavio

Update on CV and RDF resolution system
Overview of RDF system (on wrong system waiting for switch of DNS by Michael)
Showed timeline quiet launch 17-June
Formal TC approval to go ahead and go live on the 13th
Not all target languages come through in the same manner - Oliver working with algorithm
UKDA will take responsibility for maintaining this, but we need to formalize this - check with Jared on what is needed
XKOS is ready to go - only have a turtle would JSON be useful? Check with Franck

ACTION: Darren will talk to Franck next week regarding JSON for XKOS

DISCO is not yet published
It needs to be flagged that its not published
Can we put something out on the list
Are people using it? Would it be helpful
Who would be maintain - working group
PRIORITY: We probably shouldn't say its a w3c spec but it hasn't gone through
PRIORITY: We should update template to make it DDI - It is in the tracker as completed in 2019

Update on DDI-CDI targeted review
Sent request to see who is covering what...no additional responses yet
Flavio - UML/XMI and is checking on syntax rep review

Update on Glossary review
Several issues filed (12 have been filed, Training Group people have also volunteered)
ACTION: Follow up with working group on publication (web plus?)

Codebook
new pull request - Dan will clean up Wendy's mess and we'll re-commit content for pull-request
DDICODE outstanding issues - updates on 93 and 97 Darren will look at 95

July 4th CANCELLED due to British Elections

ATTENDEES: Wendy, Darren, Oliver, Dan, Flavio

Web pages/ site plan
https://docs.google.com/spreadsheets/d/1bcvxmwoxUWyhZfTkuR9ZrVtmCxz2mkE8cFYYyjpCLtM/edit?usp=sharing
From Jared:
What would be useful is to make a final decision whether the Technical Committee is committing to supporting the website infrastructure or if we should go with the vendor. If the former, will you support all functionality currently on the site or just portions? At a minimum, I need a place to post announcements, manage a membership list, and manage/edit all pages other than the /specification sections.
There's other functionality on the site, like the Bibliography or the Glossary or the Tools list that are nice but maybe could be hosted separately. Those are also decisions we'll need to make.

Discussion:

• Could support static or file content in something like S3 buckets

• COGS stuff could be bundled up as static content and hosted on docs.ddialliance.org

• Don't forsee any major problems in doing a serverless website that looks nice and bounces over to an object storage

• Could do a test migration in July during a 2-day meeting

• Can set up S3 buckets to test

• Tools etc. could be a non-database filter system

• Hotspot has the management system build into it that should support Jared's editing needs

• TC taking this on would save the transfer fees of the outside vendor. We would need to look at ongoing maintenance. Zero risk as we can build this as a parallel system this summer, test it out, and institute further changes following that.

• Do need to make sure we have document and project page organization determined so that new products and versions can be assured of URL consistency for linking purposes

ACTION: Darren will write to Jared

Scientific Board Plan
Clarified a few points. Will get final draft for tomorrow

CV publication

• Basically there

• DevOps team at Michigan have made the changes they needed to make

• Working with CV group to update static content on pages

• Problem of duplicate language (again) in the HTML

• CESSDA is not seeing the problem - if they won't fix we could filter out duplicates as an interim response

• Darren plans to go live the week after IASSIST if this works with the CV group

• Meeting on Monday to finalize the ontoloty access

• Oliver has discovered source of duplication: CESSDA has put an all of the versions together because they use a select bar. The Rest API ends up with 2 versions of both

• We could switch to this display but it may be complex for the user

• The CESSDA change dropped out some of our requirements - do we drop our requirements or ask them to make the change. We don't want what has come out with multiple versions (different versions) in different languages.

• We are concerned with the RDF version and we need to go live with this. The HTML is still a problem

Codebook outstanding issues
DDICODE-93
DDICODE-95
DDICODE-97

Darren will look at these

Presentation on DDI-L .v4.0BETA - who presents
Will sort this out off-line

Future agenda:
Chair, Vice-chair (3 year term)

ATTENDEES: Wendy, Jon, Darren, Dan, Jeremy

Web pages -

General:

• Want to control the website and not have it jobbed out

• There is static content that could be usable but not necessarily optimal

• Get it over and done and implemented and then think of the redesign

• Need to get basics done first (lift and shift)

• Opportunity to move to keep it simple then talk to groups involved (Training, TC, Publications)

Products:

• Putting up stuff around infrastructure and pipelines

• Top level URLs for the products will be hard coded or redirected

• If CDI has specific requirements they need to write them down

• This seems the obvious stuff to write this down

Document site:

• Timing of when we are going to split the document move to doc site

• Identification of documents should be part of the new site map

• Do we retain the static pages as just the publications and say the docs are over there (old docs)

• A page for product on the site but all in-depth documentation on the doc site

• Useful to have a profile of what goes on the doc site for future additions and help to groups in identifying what goes where

ACTION ITEMS:

• Darren and Jon will draft up a basic skeleton of the site map to kick against

• Add to next weeks agenda

• Known static content changes should be made before TC f-to-f meeting (ACTION: Wendy)

TC report to SB
Comments noted on draft

Update on current review work
I will continue to send reminders to reviews of both products during the review period

• Glossary List

  • Review underway

  • Several volunteers from Training Committee

• DDI-CDI Production:

  • Plans for sustainability of the standard; once its done in EA what is the pipeline (Jon will relook at the documents)

  • Olof - JSON

  • Pascal will begin in early June, may need extension

  • Dan - RDF output, looking primarily to see how they were transforming UML into RDF; special semantic relationships like traces in UML to identify patterns - how is this translated to canonical UML and then to syntaxes (are these just there to support their development of the model? this should be clarified)

Codebook (please look at before next week)

DDICODE-93 License - more detailed values for type attribute
DDICODE-95 DDICODE-95 - Getting issue details... STATUS deprecation - URI attributes needed
DDICODE-97 move license element to useStmt

ATTENDEES: Wendy, Jon, Oliver, Dan, Jeremy, Flavio, Christophe, Olof
APOLOGIES: Darren

Scientific Board Strategic Plan

[note Flavio and Dan are elected members on the SB. Wendy is ex-officio]
Flavio noted a couple of issues he'd already added
"horizon scanning capability"
"more robust" more of a clear source of truth

1.4 W3C and explore - RDF representation preparation in 2024 - why would we want to give away the variable cascade to another organization
Increase visibility, not "give away" needs to be more explicit

1.6 no date so when is this supposed to be worked on? non-open source like Colectica - what does it mean? The notion that we are working with commercial vendors and make no mention of those we currently work with. What exactly are we trying to develop? Work with related software (analysis tools) regarding input/output support?

2.2 we don't have an activity defined as such, what is in 2024? Clarify what this is mean here? architecture of content or of repository. Starting when?

2.3 addition of SDMX and HLG to this group specifically or as part of whole DDI interoperability. What do we mean by this in terms of organizational level?

2.4 The example is about domains not geographies but domain areas. Awkward, examples do not relate to geographies.

2.5 Progress toward ISO certification (we started, why is this 2025? What is meant by progress. More details - contact continuation, paperwork,

GENERAL: of strategic plan to prioritize and focus work on specific areas. Concern there is too much for too many. Who is going to do this.

3.1 TC is working on in terms of organizing documents and publication

3.2 difference between 3.2 and 3.4?
Maybe some collapse into a broader infrastructure for development, management, presentation

GENERAL: Verbiage is abstract making it difficult to differentiate and make sure that different aspects are identified as sub-bullets

Omitted support of DDI Codebook 2.6

3.8 don't understand what this means (modularity within a product), but between products? Why is the example is a provenance model.

Discussion on user mailing list (conceptual, and reference variable) was the actual issue we would face with 3.8. things differ between products and there are different implementations in various products. Need for metamodel to understand what is found in multiple products, but varying implementation. Example of metamodels (Groups).

Need both metamodel to understand differences, and in some areas a single mullti-use content.

Suggestions in the scientific model that haven't been discussed. They seem to be arising from a minimum number of sources

"Let's do a provenance model" opens up a can of worms. No background discussion. There is a process raising topical rears and pursuing their development across products in one way or another.

HOLES?: Lack of content regarding how to use DDI suite together. (maybe related to 2.6) maybe expand to include 2024?

ACTION:

• Wendy will add notes to document

• Start list of content for report, next year priorities for TC; send out to get comments before next week

ATTENDEES: Wendy, Jon, Darren, Dan, Christophe, Jeremy
APOLOGIES: Flavio, Oliver

Newsletter submission
Thank you for XKOS work on publication process
Alert people to recording on Lifecycle 4.0 BETA and continuing work

TC report to Scientific Board
Web page support through AWL
Infrastructure
Codebook
CDI review
4.0 BETA continue to finalization

Glossary Group document publication
TC to look at in terms of technical review
Check in Glossary that its DDI
Hayley get a few Training people to look at
Christophe will talk with INSEE folks regarding clarity
Month deadline

Glossary work

DDI Lifecycle github issue
https://github.com/ddialliance/ddimodel/issues/38

Rather than think of the specific item raised here
What is the purpose of the represented variable
If its available it should be in represented variable
there is a difference between practical use cases which might not be the dominant ones
Issue is primarily around missing values and problems of how coding or options change from capture through processing to instance and physical store differences
Currently clear change from represented to instance is going from universe to population (spatial temporal specifications)
Different sorts of data throw up different problems
One of the strengths of the schema it is quite directional (this can also)

ACTION: Wendy enter this discussion on #38 (DONE); others should add comments on role of RepresentedVariable or other points as needed

NEWS:
Guillaume is returning come September

ATTENDEES: Wendy, Jon, Darren, Flavio, Dan, Oliver, Christophe, Jared, Jeremy

1. AWS and Drupal site interaction and support
   Recap: some sections will move to AWS (specifications) remaining will be moved to wordpress
   Is it wise to split this down the middle and move to 2 sites. This might be difficult to split correctly and to maintain in the future.

--Is it possible to move everything to AWS? How complex might this be to maintain and supervise
--Need to move away from Drupel as it is not supported by ICPSR and want to move it to a vendor
--Fee for moving it over and an annual maintenance amount ($3-4,000 a year)
--Urgency because support for their version of Drupal had gone away, did get year's reprieve
--In the mean time the AWS option arose and discussion of how
--Lots of static content that would just be moved
--Some functinality - bibliography that is maintained in an ongoing basis; posting news etc.
--Happy if it did not got to vendor
--Can we support on-going posting and maintenance
--If not, we can still go to WordPress keeping basic content there and other on AWS

Current view of what goes to AWS:
(1) Content under the following virtual URLs will be moved to the new UMich AWS instance
 https://ddialliance.org/Specification/* - static content
 https://ddialliance.org/products/* - static content
 https://ddialliance.org/learn/resources - single static web page
 https://ddialliance.org/learn/resources/ddi-profiles - single static web page
 https://ddialliance.org/resources/tools - single static web page with filter
o Also move relevant referenced items under https://ddialliance.org/tool/*
 https://ddialliance.org/learn/markup-examples-by-version?field_ddi_product_tid=All
o Also move relevant referenced items under https://ddialliance.org/sites/default/files/*
 At the root level under http://ddialliance.org .
o https://ddialliance.org/Tables.dtd - downloadable document

--Clean up even before vendor
--Hidden stuff would not move and Darren's list may include that

--Splitting may have unintended consiquenses such as loss of database functions (publications, examples, etc.)
--Do we need all the functionality that is there (examples etc.) - if we add in with the vendor

--Would the options we are looking support these

--Issue about having to shadow the brand and sync it between 2 sites and with a 3rd party vendor

--What potential costs there would be with backup etc. - there is still need for web support (not completely volunteer)

--It seems to me that we are missing a written target sitemap (whether we have one or two platforms) that would guide the specifics of the migration.

--Could we take a few weeks out to design what we want the site to look like
Include Training Group
We have a current website that needs a spring clean but we have no idea what it should look like
Can we take 4 weeks to say this is where we want to end up (google doc) - we don't have much written down on this at the moment
Drupal to WordPress is somewhat automated there is also a lot of sorting and hand work to make the move.

https://docs.google.com/spreadsheets/d/1y54OKTmJeSV3YwxR6RBa7yJyHvD8P1drjPmG4C2vZoM/edit#gid=517461823
Lists out the sections and owner groups
Detail gets complex in the products sections
Resources has some sections
What exactly do we have - can we flatten it out?
ACTION:
We should look at what we want rather than just what we have
Create small group to look at this: Jared(co), Jon, Darren (co), Wendy

2. CDI technical review kickoff
   Flavio looked at it
   No draft of production process review issues - get from Arofan (send him a draft of request and list of people we've identified)
   Automation is still lacking so how is that evaluated
   We did write to them some time ago regarding - dig out letter we sent to them and verify all the parts are there
   Asking more information on production process itself; make sure its available
   Jon will look next Friday

3. Funding requests
   see google sheet: https://docs.google.com/document/d/111V1om6vbaJQCgxYI_znA03xDWW9HwW0iMGHxNVVnHU/edit?usp=sharing

4. Workplan for coming year
   Review workplan for tweaks

5. Updates on feedback for DDI LIFECYCLE V.4.0 BETA
   No only had one piece of feedback from Romain
   BETA 2 would be good to do when we have the ordered flags

ATTENDEES: Wendy, Jon, Darren, Flavio, Christophe, Dan

Pages from Alliance site

• We need to clarify what is moving from the current alliance site before it shifts to WordPress

• The Alliance wants to keep this site a simple as possible and there are currently some pages under LEARN/Resources that have underlying databases

• The majority of content we are looking at is static and is under Products

• Everything that is under Specification is easy but need to determine what needs to be moved

• The real question is about being able to identify specific nodes that can be handled by the remote site (RDF resolution system and CV services on UKDA site plus new DDI Alliance AWS space)

• One possible way is just a lift and shift move all the /Specification

• Landing page on WordPress site and then it sends you off to other system

• What Darren needs is a site map (node list) so he knows what gets moved

Specific questions:

docs.ddialliance.org - what is this intended to provide

• Currently we have a number of official documents associated with specifications that provide access to documentation such as field level documentation and high level documentation

• These are static sets and can be moved

• If it is difficult to tease out the nodes it may be possible to support static content under Products - all of this content is managed by TC

Resources:

• Tools, examples, profiles currently have underlying databases and we would like to retain this functionality

• These items are more in line with the products and it may be optional to move them under the Products umbrella. Learn could still retain a page that would describe and link to these resources

• To accomplish this we need to locate a workable opensource tool to manage search and display of these databases. TC will begin this search

Agency registry:

• DNS integration is being ended in 2025

• Could move to Amazon Route 53 DNS service

ACTION ITEMS:

• Obtain site map (node list)

• Transfer page listing for move to Jared’s spreadsheet

• Determine content that must move to retain or add functionality or to just support easy division of contents for routing

• Clarify who will physically be doing the set up on the AWS configuration

• Identify an open source tool to support access to database content for tools, examples, and profiles

• Determine consequences of moving content to alternate site based on node divisions proposed

ATTENDEES: Wendy, Jon, Oliver, Darren, Dan
APOLOGIES: Flavio

1-Codebook pull request
ok and merged

2-Funding requests (Deadline is 26 of April)

• Face-to-Face again

• Infrastructure costs - flag and could suggest it should be a regular budget line; software contingency budget as well as on-going costs

• 10K last year was infrastructure contingency from which a couple of months of AWS

• Alliance pays Colectica and this will move to AWS at some point
  As part of the move to AWS a bit of work needs to be done to convert from zone file to AWS (route 53) - Dan will write up

• Admin data meeting - talk to Flavio about meeting next year; funding request

3-Strategic Plan
Looks good - no questions or specific concerns from TC members at meeting

4-Administrative documents
Not ready for discussion

5-CDI submission to TC and production process review
Darren talked with Arofan about RDF URLs and everything is in line

Other:

• CV work is done ready to make the change after confirming a few minor points with ICPSR and the CV group
  https://ddicv.ukdataarchive.co.uk/ new format for the CV page (domain to be changed obviously)

• No ISI proposal but will look at the IAOS conferences coming up

FUTURE:
No Oliver until April 18
No meeting on 11th due to COSMOS conflict

ATTENDEES: Wendy, Dan, Oliver, Darren, Christophe

AGENDA

Resolution system

CV system

• Access to https://github.com/ddialliance/ddi-cv-platform (am hoping that Oliver will be at TC tomorrow).
  All is in hand

• Misalignment of version numbers in https://github.com/ddialliance/ddi-cv - logged at https://github.com/ddialliance/ddi-cv-platform/issues/1

• This has been resolved, however there is a problem with the resolving the old version numbers x.x rather than x.x.x

• They will check with CESSDA after talking to Sanda identify where the problem is in terms of inaccurate extensions

RDF hosting/resolution

• RDF hosting/resolution for CDI (upcoming) is not a dependent on the CESSDA CV version issue being fixed.

• DB will contact Arofan to clarify CDI RDF serialization’s readiness and double-check required URI patterns for CDI

• All CVs have unique URIs based on version numbers and are stored in Fuseki default graph.  For non-CV products e.g. CDI or DDI4 (that don’t necessarily have versioned URIs) we will create individual graphs for each product.

Codebook pull request review

OK and merged

4.0 BETA issues - Comments also added to issue discussion in GitHub

Item 8

• Changing from internal CV to external could be a Lifecycle 4.1 task

• Identify those that could be CV

• Add enumeration documentation to the item type

• Group ones should probably be taken over by CV group

• Dan will look through to make sure enumerations are coming through correctly - OWL specs may require URIs for enumerations (this would be an issue for next round of review - how referenced

• Wendy will look at CV options now. Darren can look at later. This is not urgent.

• Adding documentation on lists that are currently there. Create notes of what should be added to each README file.

Item 23
Not critical in short term. Good to get multiple eyes on this.

Item 36

• Should be addressed in 4.0 BETA 2

• Add additional notes on previous discussions

• Group instances to see what could be managed in the same way

FUTURE
Administrative documents (Jon) as available (Jon not on call)

ATTENDEES: Wendy, Dan, Flavio
POST-MEETING WRAPUP (due to forgetting US change to DST): Jon, Darren, Oliver, Christophe

Review of approach to issues 8, 23, and 32:

• Issue 8: (spreadsheet has been added to issue)

  • Identify good canditates for conrolled vocabulary

  • Group itesm need either documentation of definitions of types or use a controlled vocabulary. This is true of internal enumerations currently containing documentation.

• Issue 23: (spreadsheet has been added to issue)

  • When versionables reference other versionables looking for order needs - assigning of contents is required or used

  • DISCUSSION: How explicit order gets injected - clearly needed in some serializations but XML has two camps (order implied, order not implied)
    First step is identifying where order is needed

Additional Topics:
ISI presentation:

ISI do we want to propose something on DDI like last year in Ottawa?

Does TC want to send something - IPS

We have 3 weeks - kick around an idea for a topic

Something beyond use cases

Parts of DDI - multiple serializations

How content fits together and can move between products

3-4 people with or without discussant

XKOS:
Best Practice ready to go so look for email on that

ATTENDEES: Wendy, Jon, Oliver, Dan, Jeremy, Darren, Flavio, Christophe

Codebook

• Pull request has been amended and content is now correct

• Looking forward, the concept/conceptualText will be posted immediately following the resolution of the current pull request

• This will be followed by the remaining issues which are more targeted

ACTION ITEM:
Dan will note items that are NOT documentation change so they can be reviewed next week

DDI-L 4.0 Beta 1 Issues

• Another beta the middle or end of April after gathering COSMOS feedback

• An issue on representing sentinel variable values in Represented Variable

• All new content issues should go into GitHub. They will be tagged for releases.

• This will also allow broader discussion of content issues in clearer way than we have been able to do before.

ACTION ITEMS:
Wendy:
#36 - go back and pull together the earlier discussion of dealing with extended reference and review current need for continued use of this content
#8 - review enumerations as converted to CSV and conversion to outputs
#23 - locations of ordered attributes
Jon:
Administrative documents - add content on #16, #17, #30, #31, #32, #33, #34

ATTENDEES: Wendy, Jon, Oliver, Darren, Christophe
APPOLOGIES: Dan, Flavio

Webinar review

• A lot of people who we seldom or never seen

• Good suggestion:

  • Guidance on pull requests

  • We can set up templates

  • Contributors guidance document

• Missing documents

  • What is the model

  • Master?

  • Development version

• In terms of code the past published specifications can be compared

• Between 3.2 and 3.3 we could do a dif but we can't track between 3.3 and 4.0 because each generation is structurally different (dependent on mapping and change documentation)

• The ability to track individual changes

• How do we want to do this? If we publish 4.0 as a package

  • Track each time only model changes not differences in different serializations

  • We could do this locally [there will be so many structural changes a dif won’t work]

• At the moment published versions are in Bitbucket so we need a separate repository to hold published versions in GitHub to replace this

• Jon has started adding files (citation, license, contribution guidance, form for contributions

• One issue about having a missing license file we use cc-by [Note that the Alliance has determined that this is the license to be used by all products. We will need to check license on production programs (cc-by was not on the list of options]

• Document how things move through approval and that people can build on their own branch

• There is a GitHub project called DDI-CDI and DDI-CDI-resources

• Needs to be on DDI Alliance set of repositories

• There is no governance on the Alliance site

ACTION: Jon will start drafting governance documents

Status on CV and pipeline

• We have Simon George booked on 17-18th

• Can we have a repository for DDI-CV-platform (have DDI-CV-publishing)

  • Public
    https://github.com/orgs/ddialliance/repositories?type=all

  • Promo from Postman

Codebook

• Pull request for ddi-c_2
  https://github.com/ddialliance/ddi-c_2
  Note that the first ff3481b is bogus. It is the second commit 0cce353 that contains the documentation changes to codebook.xsd

• How to process through these?

  • People could put in comments in before next meeting for the documentation

  • The documentation content and the next (Concept/ConceptualText) are large. Some review should take place outside of meeting and then deal with questionable areas with group. Future edits should be more localized.

  • Oliver will check to see if there is a clear way to correct and entry error (alt is to reject pull request, kill this Fork and start over)

NEXT WEEK:
Codebook - Documentation review

FUTURE AGENDA ITEMS (as ready):

GitHub administration and guidance

Publication repository on Bitbucket - options/plan

ATTENDEES: Wendy, Jon, Darren, Flavio, Christophe, Jared (guest)
APOLOGIES: Dan

GitHub:

• Darren contacted Jared about AWS account and he needs clarification

• Background:

  • Trying to detach those things that are difficult to do with ICPSR infrastructure (underlying databases for examples, etc.; field level documentation; XML schema; etc.)

  • Set up AWS instances where these can be held

  • example: DDI Lifecycle model generates a lot of output and AWS is a solution

  • Separate web site for documentation and links to sources (docs.ddiaaliance.org)

  • Automation of creation pipeline of CV content from CESSDA output

• Currently we have 5 or 6 administrators already on GitHub - so we can manage rights for individuals

ACTIONS:

• Need to set up an account on AWS so that we can begin organizing this

• Jared needs to set up the account with appropriate permissions

• Darren will ask Michael for the DNS zones

• Wendy and Jon will create a list of items that might move from ICPSR to this new site

Clarification of document repository approach

• CDI will have a place to run their infrastructure

• Outputs will go on AWS

ACTIONS:

• Build a spreadsheet of where things are

• Start a TC ticket and add Darren as a watcher on that

DDI-L 4.0 BETA review of webinar

• Jon has done slides

• In middle of the webinar Jon would like to demo a change on a fork and generate

• 35-40 people are attending

• This is a BETA and there are already errors which need correction

• Public review in summer at the earliest

Questions regarding spreadsheet

• Specific example:

• The blue items on the spreadsheet are only removals

• Some are serialization specific and therefore need to be documented

ACTIONS:

• Add to key meaning of a,e,b in 3.3 column

• Finish sumamry of purpose of moving to Version 4.0 / use of COGS

• Get these posted and link information to Jon so he can add to slides (by Friday)

• What is the current status of CDI - finalizing documentation review checking images, Arofan is wrapping up the documentation and pulling together

XKOS update:

• XKOS best practices document almost finished integration of issues - probably out before COSMOS, maybe next week

• They will create an announcement and we can get this out and posted on the web page

ATTENDEES: Wendy, Jon, Darren, Dan, Oliver, Jeremy
APOLOGIES: Flavio

Infrastructure:
Set up account - and we need someone to work with Jared
In terms of setting up environment - need someone - Darren is willing to do set up
Getting permissions to the right people
Contact Olof and look for a person to follow-up and manage over time
Apache server serving a static site
Separate containers for different things - keep it simple
cover docs.ddialliance.org instances

AWS issue:
Darren - we can get Simon 2 weeks in March 11-18th) to get the CV set up at UKDA
Moving pipeline off Bitbucket to GitHub: Oliver has already moved it to GitHub and Oliver will send URL of repository
https://github.com/ddialliance/ddi-cv
The creation of repo content is still manual but should be turned into a pipeline in the nexts
Sanda can check out the html specifications. Pascal can also do checks

Webinar
Spreadsheet 3.3 to 4.0 - post for TC and webinar support
Jon has updated the presentation outline based on TC comments
Add Demo of COGS showing how easy it is to make changes and then generate their content for testing
Members planning to attend need to register

FUTURE AGENDA:
Next week: review of webinar in prep
Two weeks: Check in on CV, RDF, and repo pipeline - sanity check

ATTENDEES: Wendy, Dan, Jeremy, Flavio, Oliver
APLOGIES: Jon

DDI-L 4.0 BETA webinar

As of last Friday, we had 35 sign-ups.

Draft of short presentation contents
https://docs.google.com/document/d/1jeqDxr_4P3HM-oeW4j-g-WYr3aAKQeUwNooyilgGWbY/edit?usp=sharing
See document for comments

AWS set up

TC budget for AWS has been approved, so we need to get moving on setting up an account, presumably we need to get Jared to put the Alliance down to pay that.
Uses funding requested this year for infrastructure support. Next fiscal year this should be proposed as a new permanent budget line for association.

Talk to Darren and/or Olof about taking this on to get set up.
Regarding cloud deployment. Whoever sets it up we should management deployment using build scripts. The CV stuff is set up this way and there is something to start with.
We will need to add in things on the documentation. See how long Jared would need to set up the account. Be clear what needs to get done prior to setting up the cloud space and moving things into it.

Product Page

Implications for how the content is made available on the download and doc site for each product
Currently use docflex for the Field level documentation of the XML. Should this be continued with the COGS product. XML specific documentation as well as language specific documentation need for field level documentation.
Can we add to the doc site (links to specific output documentation)

A revised example will be made based on comments. Follow with examples of transfer of all product pages.

ATTENDEES: Wendy, Jon, Dan, Oliver, Christophe
APOLOGIES: Flavio

DDI Lifecycle 4.0 BETA webinar

• Went out on DDI Users Group and some socials

• Put a reminder out 2 weeks before event

• Check with Jared on numbers

• Jon will draft up something for next week

DDI infrastructure proposal

• Has contacted relevant working groups

• CV XKOS and SDTL are fine

• CDI group ok; some longer term issues that can be addressed as they come up in terms of production line

• Add some statement about governance around GitHub

• Can start setting up landing page for development work so we can inform people

Atlassian

• Some worries

• Many people are moving off it and this could cause the loss of open source support (due to lack of paying customers)

• We have moved off in terms of Bitbucket

• But we still have JIRA and Confluence

• Clean up JIRA in the next year to drop unused and archive old issue trackers

• Consider free support services we use to develop contingency plan across the board

NEXT WEEK:

• Product page mockup

• Draft of 4.0 BETA webinar content

• Administrative Data possibility of moving to next fiscal year

ATTENDEES: Wendy, Jon, Dan, Oliver, Flavio

Discuss page change drafts for products (TC-242, TC-243, TC-244)

• Product Page list of versions OK Get this done first as this resolves the Google reference issue

• The page of links for filing issues and how-to page are a good idea

• Product version page revision is a good first step by pushing critical information to the top and making it visible on the initial screen. Continue to revise based on information from developers/implementers and prepare updated draft along with an example from Lifecycle. These need to get updated prior to publication of CDI, new Codebook 2.6, and DDI Lifecycle 4.0.

• Issue to address in future: Current download packages are now served by an on-site (ICPSR) copy of the product and XML is provided for validation purposes through the ICPSR site. How should this change with multiple serializations (XML, RDF, JSON, UML-XMI, etc.)? The Bitbucket repository of published products currently serves as an archive and is not linked to the product pages consistently.

Organization and ongoing management of GitHub content
This was just a heads-up on the type of information TC needs to organize regarding GitHub space. It is primarily management information (Guidelines for development groups and users, tooling pipelines, space management, constraints regarding triggers for moving from free to fee based services, etc.). This is still in a period where we are obtaining comment. These notes from the CDI meeting will be more formally organized and provided in writing by CDI. The Scientific Board is aware of this work and is primarily monitoring the process to ensure input from development groups affected by the shift to GitHub.

TC pages under Learn dropdown on Alliance site
Resources:
We need to look at how examples, tools, and profiles are managed and organized in the new system. We will no longer have the on-site option of underlying databases. Primary decision is which, if any, of these should be managed off-site.
Getting Started:
See TC-245 for collecting information on determining the purpose of this page, what information should be provided, and how to present it. Members were asked to think about these issues and add comments to the issue so that we can move on revising this page in cooperation with the Training Group and probably future Marketing Group. This should be a priority. Jon will talk with Training leaders to get their initial thoughts.

NEXT WEEK:
Webinar for 4.0 Beta

No meeting was held. The following is the information sent out to the group for their attention and action:

TC issue tracker
I have reviewed and updated all entries
In particular see TC-242, TC-243, and TC-244 and add comments regarding layouts provided in attachments

Use filter TC-outstanding to filter out completed issues

Notes from DDI-CDI meeting on 2024-01-17

Comments about Git repository raised during CDI meeting on 2024-01-17 - note that Achim was not present so there will be more
Git repository management - how does TC support product groups in this?
Guidelines
Informational materials
Help identify persons if group is unable to provide this
Deirdre is willing to be involved as setting this up
Back-up people
Git repository
Tools/workflows

Organizing BETA access, support repository, policy, pull requests
Coordinate how to organize
Working with input from outside of the group (developers work)
Making clear what things are
Granularity of pull requests (issue based)
Branching - readme on how to do this, what is supported
Pricing models for AWS (space is not the issue, but deploying own servers/code base)

Each product needs to provide a canonical expression of the product
CDI is the UML model expressed as canonical XMI
Each product needs to be clear on this - how to visualize the model in the case of COGS base
Canonical model has to be publicly distributed

ATTENDEES: Wendy, Jon, Dan, Jeremy, Flavio, Oliver

Update on CDI work

• Variable Cascade issues - providing guidance through webinars, documentation (it appeared there weren't direct predicates linking along the chain, appeared there were just a single predicate used for different relationships)

• Will this require additional review?

Topics for future webinars

• Why are there various ways in dealing with the same content across products (classification/codelists etc)

DDI-L 4.0 BETA review

• Supplemental content - changes in the content model; why and what was done; update DDI Lifecycle Best Practices (how to use some of the schema structures to address 4.0 in addition to 3.x best practices)

• Updating model - just raising issues and modifying at the end

EDDI session on priorities

• Nothing specific struck a cord in terms of TC

Follow-up on - web pages and webinar with Jared

• Wendy has recieved comments on pages

• Jared has responded to Jon who will make arrangements for webinar on 4.0 BETA

ATTENDEES: Wendy, Jon, Dan, Jeremy, Flavio

DDI-Lifecycle version 4.0 beta follow-up

Webinar of some description
Q&A - Highlight of why we moved to this and how they can interract with it
or Anticipation - presentation

• Q&A seems best approach

• A bit of introduction to version 4.0 (about 5 minutes)

• Back end of February would be good timing

• Discussion pieces out from mid-Jan to mid-Feb: get a list together and see what is important to front-end

• 3:00 UK (10 eastern, 9 central) - 21st or 28th of Feb (Jon will follow-up on this)

• It's about a small group zoom with google form for registration

• We've only really asked about OWL serializations

• Do we want something more on the model conceptualization?

• On the serialization things?

• Model is the same as 3.3 with only changes made to so only specific changes for choice and physical description

ACTIVITY: (Wendy review and create list) pull discussion pieces together, clean up, post, and send links
Roll out discussion pieces.

Product pages on Alliance Site

Overview of products page can be updated but is in good shape overall
https://ddialliance.org/products/overview-of-current-products
Product pages:

• Less wordy

• More prominence for the access to the html documentation

• Update links

• Setting up pages that were held off the site (determine what should be off site support - RDF resolution

• All of the products have autogenerated documentation - move to a docs.documentation site and link from the main pages

• Examples, tools, etc are candidates for moving off-site

• Get comments from Jared (contact him for what feedback he's gotten)