TC Meeting Minutes

ATTENDEES: Wendy, Oliver, Darren, Dan, Flavio

REGRETS: Jon

AGENDA:

• DDI Newsletter

• DDI-CDI Publication

• Website work

• Website management group

Quick recap

The team discussed the DDI CDI publication, website coordination, and the formation of a future web management group. They also addressed the need for generating XML schema documentation, the placement of certain resources on the website, and the process of editing guidelines for the website. Lastly, they outlined the next steps for the project, including writing up more detailed terminology and providing information to CDI by the next day.

Next steps

• Darren to update the document with URL clarifications and send it to Wendy for review before sending to Achim.

• Wendy to create a Google Doc of Darren's document and link it to the meeting minutes.

• Darren to complete the database-driven pages for the website.

• Darren to email Jared about obtaining newsletter data if not found in the Drupal backup.

• Wendy to write up a more detailed proposal for the future web management group and circulate it to Jon, Jared, and Oliver.

• Darren to add information on directional information and editing guidelines for database pages to the web editing guidelines document.

• Wendy to continue adding screenshots to the web editing guidelines document.

• Technical Committee to consider content for the newsletter announcement regarding the website or other committee activities.

• Darren and Wendy to plan a webinar to introduce the new website to users once it's up and running.

DDI CDI Publication and Website Coordination

Wendy led the meeting, discussing the DDI CDI publication, website coordination, and the future web management group. She announced that the newsletter deadline had been extended to the 31st to accommodate the announcement of both the website and CDI publication. Flavio mentioned minor model corrections that needed to be made, which he planned to discuss with Arafan later. Wendy also discussed the need to modify web pages for CDI and move information into the overview. Darren's email was shared, and he clarified that most of the issues were straightforward, with many options for setting up redirects. The team agreed to provide any necessary information to the CDI team by the end of the day or the following day at the latest.

Canonical URL for Model Documentation

Darren discussed the need for a canonical URL for the model field level documentation, suggesting it could be either docs.ddilliance.org or a consistent pattern with specification/ddi-cdi/1.0/model/field level documentation that redirects to docs.ddillance.org. Wendy pointed out the need for a version number in the URL. Dan suggested including the XML schema documentation generated by DocFlex on the docs.ddiliance.org site, following a similar pattern to other products. Darren agreed, noting that the XML schema documentation hasn't been generated yet as it's been focused on the model.

DECISION: Keep the specification path as the canonical Specification/DDI-CDI/1.0/model/field-level-documentation but house in docs.ddidocumentation.org

XML Schema Documentation and Tool Status

Darren, Wendy, and Dan discussed the need for generating XML schema documentation, which is currently not being done. They agreed to bring this up with Jon in the next meeting. They also discussed the status of the tool used for generating this documentation. Darren mentioned that he had set up redirects on HubSpot for field level documentation and had a few more to do. The team also discussed the possibility of moving XSD artifacts to docs.ddilliance.org, but decided to discuss this further in a future meeting.

System Interaction and Development Management

Wendy discussed the need for higher level instructions on how people can interact with the system and manage development things. This was raised in the CDI meeting as is on their future agenda She suggested a meeting with the CDI and Technical Committee to ensure a consistent approach across the product. Darren confirmed that the XSD files are in Hubspot and the redirects are set up. He also asked Dan to deploy the package onto http://docs.ddialliance.org once it's ready. There was a discussion about the path for the DDI-CDI and the need to update the path on the docs site. Lastly, Darren and Wendy discussed the placement of the Turtle and JSON-LD packages, confirming that they should be hosted under the RDF URL.

Project Naming Convention and Website Cleanup

Wendy and Darren discussed the need provide Arofan and Achim with a clear list of URL, date, and version changes needed in the CDI publication package. Darren will provide a firm list of changes and URLs for the new version. Wendy will review before sending to Achim. Wendy will manage the standard updates to the web pages required for new publication.

Website Editing Guidelines and Resources

Wendy discussed the process of editing guidelines for a website, focusing on the setup of nesting patterns and the use of dropdowns. She raised the question of the inconsistent use of 2nd tier nesting on the drop down menus. Darren agreed that this was a content review issue. Wendy also mentioned the creation of a Confluence page of the Technical Committee, where she plans to include information about the website's technical infrastructure. She is currently working on the web editing guidelines and is open to feedback and questions.

Website Updates and Newsletter Management

Wendy discusses updating internal document links, related content, and database content on the new website. Darren provides updates on correcting file links and working on data-driven pages. They discuss handling newsletters, with Dan sharing an archival link to past newsletter PDFs and pages. Wendy suggests having submission forms for examples and tools that get reviewed before publishing to improve the process over the previous approach.

Location of old web pages to view past layout for verification:
https://web.archive.org/web/20241217184156/https://ddialliance.org/publications/newsletters

Forming Web Management Group

Wendy discussed the formation of a future web management group, which would consist of 4 to 5 people, including two editors for administrative and product content. The group would also include a technical support person and individuals with web layout and marketing skills. Wendy emphasized the importance of maintaining close ties with marketing and training. She also mentioned the need for an archivist to ensure consistency and alignment across different sections of the website. Additional people, volunteered or paid, could be added for specific activities such as a usage assessment. Darren agreed with Wendy's proposal, and the group decided to move forward with the plan. The web management group would be a working group of the Executive Board. Wendy will write this up in more detail and discuss with Jared, Jon, and Oliver. TC can review again when something is decided by this group.

Future items

Darren was tasked with providing information to CDI by the next day and adding directional information to the databases. Wendy has created a centralized place for web and technical infrastructure guidelines and the importance of maintaining consistency. The team discussed the timeline for CDI automation, with the first deadline in March. Wendy also reminded the team to provide suggestions for the website and newsletter. Lastly, Darren suggested a webinar to introduce the website to users once it's ready.

ATTENDEES: Wendy, Oliver, Dan, Jon, Darren, Flavio

Quick recap

The team discussed updates and tasks related to the website and DDI, including the progress of updating external URL links and the need for clear instructions for editing and updating the databases. They also discussed the need to clean up and reorganize the file structure for their project, and the progress of the DDI-Lifecycle v4.0 Beta 3 revie. Lastly, they initiated a discussion about future agenda items and encouraged team members to communicate their needs for upcoming meetings.

Next steps

• 
  Darren to continue updating website pages and links, aiming to finish by Monday morning.

• Oliver to be available tomorrow for Darren to check data-driven pages.

• Wendy to work on finishing the draft instructions for editing the website this week.

• Wendy to create a new subsection under the TC page for website management documentation.

• Darren to email Wendy tomorrow with his understanding of where CDI files are currently located and where they should go.

• Wendy to move the CDI product page back to "developing products" section until publication.

• Dan to send a reminder on January 15th for additional feedback on Beta 3.

• Dan to implement JSON, XML, and RDF versions of Beta 3 in the next couple of months.

• Dan to conduct round-trip testing between 3.3 and 4.0 versions.

• TC to follow up on CDI commitments for automation processes, particularly first quarter deadlines.

• Dan to continue work on representing profiles across different DDI Lifecycle representations.

• Dan to write up different ways to make profiles in various serializations for community feedback.

Summary

Website Updates and DDI Tasks

Wendy and Darren discussed updates and tasks related to the website and DDI. Darren mentioned that he was working on serving the website on SSL and expected to complete this by Monday. He also planned to check the rest of the data-driven pages the following day, and needed Wendy's assistance in review. Oliver was also expected to contribute to the project.

Website Updates and Documentation Plans

Darren and Wendy discussed the progress of updating external URL links to internal page links on the website. Darren agreed to continue working on this task and mentioned the need for a more detailed conversation about docs.ddilliance.org. Wendy agreed to include this on a future agenda. They also discussed the need for clear instructions for editing and updating the databases, which Wendy agreed to work on. Darren suggested documenting how the data modules work, and Wendy proposed creating a new subsection under the website management page for technical documentation. No other issues or questions were raised.

Project File Structure and Documentation

Wendy, Jon, and Darren discussed the need to clean up and reorganize the file structure for "http://docs.ddialliance.org ". Darren agreed to go through the files one by one, as most were configured as URLs rather than file links. Jon offered to help with the process. They also discussed the need to update the XML schema and RDF resolution pieces before publishing. Darren agreed to draft an email to Wendy to clarify his understanding of the project's current state and future plans. They also discussed the need to move the field level documentation for CDI to the correct location on Docs.ddi.

Beta 3 Review Progress and Implementation

Wendy and Dan discussed the progress of Beta 3 review. Dan mentioned that Vincent had reviewed the updated Json schema and found it to fix the issues from the DDI Hackathon. They also discussed the need for more feedback and the importance of implementation in Python. Dan mentioned that they would implement the Json, Xml, and Rdf versions in the next couple of months and conduct round trip testing between versions 3.3 and 4.0.

Future Agenda Items and Meeting Adjustments

Wendy suggested following up on the commitments made by CDI regarding automation processes. Dan also discussed his work on representing profiles across different representations and the need for more examples and proposals. He suggested writing a document about different ways to make profiles in different serializations and sending it out for comment. Wendy agreed that this could be a good first step towards creating supplementary publications to support the use of the DDI.

Wendy initiated a discussion about future agenda items, encouraging team members to communicate their needs for upcoming meetings. Flavio and Jon participated in the conversation, with Flavio joking about not invading Canada. Oliver requested a shift in a certain aspect, which Wendy agreed to. The conversation ended with Dan expressing gratitude and Wendy turning off the AI companion.

ATTENDEES: Wendy, Jon, Dan, Darren, Flavio
REGRETS: Oliver

Agenda: Update on Website work

Issue for schema validation
Example:
https://ddialliance.org/Specification/DDI-Lifecycle/3.3/XMLSchema/instance.xsd

https://app-eu1.hubspot.com/user-guide/144531336
CNAME record is now correct:
DNS Propagation Checker - Global DNS Checker Tool

Quick recap
The team discussed the ongoing issues with the website transfer to Hubspot, including DNS transfer problems and the need for setting up redirections for the specification URLs. They also discussed the potential consequences of rolling back the changes and the need to sort out publication plans for the CDI in January.

Next steps

• Darren to resolve DNS transfer issues with Candice within the next couple of hours.

• Darren to work on database-driven summary listing and hyperlink generation for individual record pages over the Christmas holidays.

• Darren to set up redirections for specification URLs to the correct locations on docs.ddalliance.org.

• Wendy to finalize and send the instructions for editing to Darren for review.

• Darren to contact Hubspot regarding the round-robin DNS issue if it persists.

• Wendy to send out a note on the user list if the site is expected to be down over the holidays due to transfer issues.

• Technical team to determine the appropriate locations for CDI publication and documentation by the January meeting.

• Wendy to coordinate with the team for the CDI publication in January, ensuring correct links and placements.

SUMMARY

DNS Transfer and SSL Certificate Issues

Wendy initiated a discussion about the status of the website, specifically the DNS transfer to HubSpot. Darren reported that the transfer was initiated the previous day, but some records were not yet picked up. He mentioned that Hubspot would automatically provision an SSL certificate once it was satisfied with all the DNS records. Darren also mentioned an issue with a CNAME record not being picked up by the Google Admin tools or any DNS tools, and he planned to discuss this with Candice and Jason, who are responsible for maintaining the zone file.

Hubspot Documentation and Redirection Issues

Wendy, Darren, and Flavio discussed ongoing issues with the documentation in HubSpot. Darren explained that he had found a solution to the problem and was available to work on it over the Christmas holidays. They also discussed the need for setting up redirections for the specification URLs to the correct location, with Darren offering to upload the necessary files into the correct folders on Hubspot. Dan confirmed that the generated documentation was currently the only thing added, and that the actual schemas would need to be put somewhere.

Website Transfer and DNS Discussion

Wendy and Darren discussed the process of moving their website to a new location. Wendy asked about the need to switch login addresses, to which Darren responded that the admin console URL would still be used for editing and content publishing. He also mentioned that he would have a better understanding of what Wendy could do once the domain transfer was completed. Wendy is finalizing the instructions for editing and plans to share them with Darren before granting Jared access. Darren suggested that if the DNS transfer issue couldn't be resolved by the end of the next day, they could either roll it back or continue to work on it when they return on the second. The team also discussed the potential for a broken link or a 404 error if the site was accessed via HTTPS. Jon suggested that not being able to access the website for a few days was not a significant issue.

Addressing Problem and Potential Consequences

Jon, Wendy, Darren, and Dan discussed a problem that needed to be resolved by the end of the next day. Jon suggested that the issue was not their responsibility and that the other party should either fix it or take the consequences. Wendy expressed concern about the potential consequences of rolling back the changes, suggesting that it might lead to the same problem. Darren agreed to contact Candice to discuss the issue further. The team decided to monitor the situation and take a decision later.

Addressing Site Downtime and Publication

Wendy suggested sending a note to the user list about potential site downtime due to transfer issues and DNS resolution problems. She also mentioned the need to sort out publication plans for the CDI in January, ensuring correct links and placement. Wendy asked the team to figure out technical fixes by their January meeting to facilitate this. Flavio confirmed that no major changes were expected, only ensuring things are in their final places and links are updated.

CDI Publication Delay to January

Wendy informed the team that due to EDDI and the holidays' influence, the CDI publication was delayed until January. Website issues may also be a factor. Member's clarified their availability and preferred means of contact between now and January 6.

The team wished each other well for the holidays.

ATTENDEES: Wendy, Oliver, Dan, Jeremy, Christophe
REGRETS: Jon

Website shift scheduled for 18 December

• Darren is continuing work on this and coordinating with ICPSR via Jared on 18 December shift

  • General clean-up of page formating

  • Database generated pages

  • Links between pages and to documents

  • DNS changes and mappings to retain previous links for users to major pages

• Training group will review their current content pages (as changes were not made in transfer) as basis for later input for improved section

• Wendy is completing Editing Instruction Manual for consistency and will complete first major draft by the 18th for Jared's use

DDI Codebook 2.6 preparation for review/vote

• Completed:

  • Field level documentation reformatted (PR38)

  • Update on dates and version information in schema and readme files (PR39)

• TO DO:

  • update member list

  • update change log

  • complete first full draft of high-level document

  • complete production process document and plan for moving to automated production process

  • update Field Level Documentation HTML pages

    • Having a bit of trouble getting the field level doc software running on windows 11, there will be a solution just not worked it out yet

DDI Lifecycle 4.0 planning for summer release

• Making some updates to the COGS tooling from Hackathon

  • JSON tooling

  • RDF content

• Update those tooling and get out a BETA3 review in January/February

• Also, now some samples of Lifecycle 4.0 from Hackathon (RDF, JSON, XML)

  • This will allow a new set of reviewers (content to get involved

• We have more on the NECTAR publisher which supports the JSON representation, full Lifecycle, CDI, questionnaire and study level information so people can create their own examples

• Work on a secondary examples section - move all the examples to this area and improve access from HubSpot

• Creating a standardized data set and then creating examples in each of the DDI products and have content expressed across the different products

• This also supports transformation and transport between different products

• DDI Developers docs

Activities from EDDI - interest in new working groups, organizing non-product specific Roadmap activities

• Types of Working Groups

  • Technical Group - manages publication

  • Product Groups

  • Content Groups

  • Practitioners Group - Best Practices as an output

• ADD in Christophe's email

• Variable Cascade - could be an example of this type of group as they already have content

• Start as a sub-group of TC

• Christophe will follow-up on this and keep the TC informed - he will contact others interested in this

EMAIL CHAIN

RE: Practicioner's Group
Jared to Christophe:

This is to follow up on our discussion about the Marketing Working Group and the potential new Practitioners Working Group you are interested in. I'm copying Jon so he's aware on the marketing side.

I've let Jon know you've kindly offered to provide use cases from Insee, as well as that you'd be open to participating in the working group. Jon: would you be able to add Christophe to future meeting invitations?

Christophe: Can you say more about the goals of the Practitioners Working Group, including expected outcomes and participants? Once we have a written proposal, we can figure out if this should go to the Executive Board or the Scientific Board. Either can review and approve a working group, depending on the group's scope.

Refining idea to application/use cases
Christophe to Jared:

My comment was in reaction to the ddi alliance's poster inviting proposals for new working groups and I think it's a very good message. Generally speaking, I think that between the DDI training group and the technical working groups, there is room for working groups on application cases. The working group on questionnaires is a very good example of a working group. We could imagine (by making a list to Santa Claus) a working group on setting up the variable cascade, a working group on managing variables over time, a working group on the methodology for associating a concept with a variable.

I don't know if we should set up a general working group for practitioners, as the content is likely to be heterogeneous and difficult to manage, and it might be more appropriate to set up working groups on practical and fairly well-defined use cases.

I'm thinking of making a list of the work that we're planning and if the marketing group can help us to get an overview of what would be interesting and possible to address collectively and relevant, that would be great.

Links with Marketing
Jared to Christophe:

I like your idea to establish WGs based on well-defined use cases. Sounds good to start with a list of work you're planning and working with the Marketing WG. Jon's on here so he can help coordinate with the Marketing WG once that list is ready.

Establishing sub-committee, broadening to TC interests
Jon to Jared, Christophe, Wendy, Oliver:

Thanks for this Christophe, I think the DDI representation group started by INSEE/CLOSER and Constances is a good example I think of what you are thinking of, I think Frank had the idea that it would find a more official home under the Alliance at some point? But perhaps a little less? I think as they would be cross product (or at least address the different issues, they would be suited to being Technical Committee sub-groups, which would be possibly short lived to develop application use cases, I think the questionnaire group is a bit wider as it does cover a wider remit so a formal working group probably might make more sense?

Would that work? Under the current way the Alliance sets up WG’s it’s a bit inflexible and has to have more ‘paperwork’. Would that address the use-cases you are thinking of?

Discussion regarding TC
Wendy to Jon, Jared, Christophe, Oliver

Can we kick around some thoughts on this during the TC meeting on Thursday? Right now we basically have product, content, and externally focused working groups (Marketing - EB, and Training - SB). I'd like to get a clearer picture and see where cross working group coordination would be needed in terms of on-going product development as well as furthering the TC/SB goals of use case development, best practices guides or other possible outputs from such groups.

ATTENDEES: Wendy, Oliver, Jon, Darren, Dan, Jeremy
REGRETS: Flavio

Quick recap

The team discussed the progress of their website project, with plans to proceed despite some issues and to launch by the end of the year. They also addressed issues related to a public URL and image functionality, and the interest of others in reviewing the DB generated content. Lastly, they discussed the use of existing tags for risk assessment purposes and the potential use of DDI metadata for these purposes.

Next steps

• Jon to write guidelines for reviewing the new website, focusing on navigation and overall structure.

• Dan to document ideas and recommendations for moving away from XML-specific profile validation tools towards more flexible, serialization-agnostic approaches.

• Wendy to attend a meeting in early 2025 to discuss DDI profile validation tools and potential expansion of the technical committee's work in this area. She will keep TC members informed.

Summary

Website Progress and Launch Plans

Wendy, Jon, Oliver, and Dan discussed the status of the website work. Wendy expressed uncertainty about the progress of database-driven pages, while Jon suggested they should move forward despite some issues. Jon proposed getting feedback from a limited number of people, possibly including someone from CLOSER, to identify any major navigation issues. Dan mentioned that they couldn't review the website at the moment due to lack of logins. Jon agreed to update the document outlining what they're asking people to review. The team decided to proceed with the website launch by the end of the year, with plans for further improvements in the following year.

Darren emailed during meeting that he would be working on this next week.

Wendy, Dan, Jon, and Oliver discussed issues related to a public URL and image functionality. Wendy confirmed the existence of a public URL and provided the link in the chat for Dan to access. Dan noticed that the images were not working, specifically mentioning the logo. Wendy agreed to add the URL to a document for future reference. Dan also suggested not sharing the URL in the meeting minutes for privacy reasons.

Discussing DDI Profile Validation Tool

Wendy discussed an upcoming meeting in early 2025 to discuss a DDI profile validation tool provided by CESSDA. As agreed in the past year, the DDI profile should not be part of the lifecycle standard and that it might be more beneficial to use existing tools for validation. They also discussed the need for better guidance on creating more flexible profiles in different serializations. Dan agreed to put his ideas into a more structured format to focus the discussion on what it would look like when the DDI profile is taken out of the lifecycle and put into this new format as well as role of TC in providing guidence for profiles in general and the use of standard tools for validation.

Using Tags for Risk Assessment

Wendy discussed her interaction with Jordan, a data librarian from USAID, who was seeking ways to use existing tags to identify combined identifiers for risk assessment purposes. Wendy suggested using the Data Relationship and Logical Record Descriptors to identify unique and complex identifiers. She also asked if anyone was using DDI metadata for similar purposes, to which Oliver responded that someone was doing something similar. Wendy will share the email chain so others can see if they can add to discussion or provide a contact.