ATTENDEES: Wendy, Jon, Darren, Flavio, Dan, Oliver, Christophe, Jared, Jeremy

1. AWS and Drupal site interaction and support
   Recap: some sections will move to AWS (specifications) remaining will be moved to wordpress
   Is it wise to split this down the middle and move to 2 sites. This might be difficult to split correctly and to maintain in the future.

--Is it possible to move everything to AWS? How complex might this be to maintain and supervise
--Need to move away from Drupel as it is not supported by ICPSR and want to move it to a vendor
--Fee for moving it over and an annual maintenance amount ($3-4,000 a year)
--Urgency because support for their version of Drupal had gone away, did get year's reprieve
--In the mean time the AWS option arose and discussion of how
--Lots of static content that would just be moved
--Some functinality - bibliography that is maintained in an ongoing basis; posting news etc.
--Happy if it did not got to vendor
--Can we support on-going posting and maintenance
--If not, we can still go to WordPress keeping basic content there and other on AWS

Current view of what goes to AWS:
(1) Content under the following virtual URLs will be moved to the new UMich AWS instance
 https://ddialliance.org/Specification/* - static content
 https://ddialliance.org/products/* - static content
 https://ddialliance.org/learn/resources - single static web page
 https://ddialliance.org/learn/resources/ddi-profiles - single static web page
 https://ddialliance.org/resources/tools - single static web page with filter
o Also move relevant referenced items under https://ddialliance.org/tool/*
 https://ddialliance.org/learn/markup-examples-by-version?field_ddi_product_tid=All
o Also move relevant referenced items under https://ddialliance.org/sites/default/files/*
 At the root level under http://ddialliance.org .
o https://ddialliance.org/Tables.dtd - downloadable document

--Clean up even before vendor
--Hidden stuff would not move and Darren's list may include that

--Splitting may have unintended consiquenses such as loss of database functions (publications, examples, etc.)
--Do we need all the functionality that is there (examples etc.) - if we add in with the vendor

--Would the options we are looking support these

--Issue about having to shadow the brand and sync it between 2 sites and with a 3rd party vendor

--What potential costs there would be with backup etc. - there is still need for web support (not completely volunteer)

--It seems to me that we are missing a written target sitemap (whether we have one or two platforms) that would guide the specifics of the migration.

--Could we take a few weeks out to design what we want the site to look like
Include Training Group
We have a current website that needs a spring clean but we have no idea what it should look like
Can we take 4 weeks to say this is where we want to end up (google doc) - we don't have much written down on this at the moment
Drupal to WordPress is somewhat automated there is also a lot of sorting and hand work to make the move.

https://docs.google.com/spreadsheets/d/1y54OKTmJeSV3YwxR6RBa7yJyHvD8P1drjPmG4C2vZoM/edit#gid=517461823
Lists out the sections and owner groups
Detail gets complex in the products sections
Resources has some sections
What exactly do we have - can we flatten it out?
ACTION:
We should look at what we want rather than just what we have
Create small group to look at this: Jared(co), Jon, Darren (co), Wendy

2. CDI technical review kickoff
   Flavio looked at it
   No draft of production process review issues - get from Arofan (send him a draft of request and list of people we've identified)
   Automation is still lacking so how is that evaluated
   We did write to them some time ago regarding - dig out letter we sent to them and verify all the parts are there
   Asking more information on production process itself; make sure its available
   Jon will look next Friday

3. Funding requests
   see google sheet: https://docs.google.com/document/d/111V1om6vbaJQCgxYI_znA03xDWW9HwW0iMGHxNVVnHU/edit?usp=sharing

4. Workplan for coming year
   Review workplan for tweaks

5. Updates on feedback for DDI LIFECYCLE V.4.0 BETA
   No only had one piece of feedback from Romain
   BETA 2 would be good to do when we have the ordered flags

ATTENDEES: Wendy, Jon, Darren, Flavio, Christophe, Dan

Pages from Alliance site

• We need to clarify what is moving from the current alliance site before it shifts to WordPress

• The Alliance wants to keep this site a simple as possible and there are currently some pages under LEARN/Resources that have underlying databases

• The majority of content we are looking at is static and is under Products

• Everything that is under Specification is easy but need to determine what needs to be moved

• The real question is about being able to identify specific nodes that can be handled by the remote site (RDF resolution system and CV services on UKDA site plus new DDI Alliance AWS space)

• One possible way is just a lift and shift move all the /Specification

• Landing page on WordPress site and then it sends you off to other system

• What Darren needs is a site map (node list) so he knows what gets moved

Specific questions:

docs.ddialliance.org - what is this intended to provide

• Currently we have a number of official documents associated with specifications that provide access to documentation such as field level documentation and high level documentation

• These are static sets and can be moved

• If it is difficult to tease out the nodes it may be possible to support static content under Products - all of this content is managed by TC

Resources:

• Tools, examples, profiles currently have underlying databases and we would like to retain this functionality

• These items are more in line with the products and it may be optional to move them under the Products umbrella. Learn could still retain a page that would describe and link to these resources

• To accomplish this we need to locate a workable opensource tool to manage search and display of these databases. TC will begin this search

Agency registry:

• DNS integration is being ended in 2025

• Could move to Amazon Route 53 DNS service

ACTION ITEMS:

• Obtain site map (node list)

• Transfer page listing for move to Jared’s spreadsheet

• Determine content that must move to retain or add functionality or to just support easy division of contents for routing

• Clarify who will physically be doing the set up on the AWS configuration

• Identify an open source tool to support access to database content for tools, examples, and profiles

• Determine consequences of moving content to alternate site based on node divisions proposed

ATTENDEES: Wendy, Jon, Oliver, Darren, Dan
APOLOGIES: Flavio

1-Codebook pull request
ok and merged

2-Funding requests (Deadline is 26 of April)

• Face-to-Face again

• Infrastructure costs - flag and could suggest it should be a regular budget line; software contingency budget as well as on-going costs

• 10K last year was infrastructure contingency from which a couple of months of AWS

• Alliance pays Colectica and this will move to AWS at some point
  As part of the move to AWS a bit of work needs to be done to convert from zone file to AWS (route 53) - Dan will write up

• Admin data meeting - talk to Flavio about meeting next year; funding request

3-Strategic Plan
Looks good - no questions or specific concerns from TC members at meeting

4-Administrative documents
Not ready for discussion

5-CDI submission to TC and production process review
Darren talked with Arofan about RDF URLs and everything is in line

Other:

• CV work is done ready to make the change after confirming a few minor points with ICPSR and the CV group
  https://ddicv.ukdataarchive.co.uk/ new format for the CV page (domain to be changed obviously)

• No ISI proposal but will look at the IAOS conferences coming up

FUTURE:
No Oliver until April 18
No meeting on 11th due to COSMOS conflict

ATTENDEES: Wendy, Dan, Oliver, Darren, Christophe

AGENDA

Resolution system

CV system

• Access to https://github.com/ddialliance/ddi-cv-platform (am hoping that Oliver will be at TC tomorrow).
  All is in hand

• Misalignment of version numbers in https://github.com/ddialliance/ddi-cv - logged at https://github.com/ddialliance/ddi-cv-platform/issues/1

• This has been resolved, however there is a problem with the resolving the old version numbers x.x rather than x.x.x

• They will check with CESSDA after talking to Sanda identify where the problem is in terms of inaccurate extensions

RDF hosting/resolution

• RDF hosting/resolution for CDI (upcoming) is not a dependent on the CESSDA CV version issue being fixed.

• DB will contact Arofan to clarify CDI RDF serialization’s readiness and double-check required URI patterns for CDI

• All CVs have unique URIs based on version numbers and are stored in Fuseki default graph.  For non-CV products e.g. CDI or DDI4 (that don’t necessarily have versioned URIs) we will create individual graphs for each product.

Codebook pull request review

OK and merged

4.0 BETA issues - Comments also added to issue discussion in GitHub

Item 8

• Changing from internal CV to external could be a Lifecycle 4.1 task

• Identify those that could be CV

• Add enumeration documentation to the item type

• Group ones should probably be taken over by CV group

• Dan will look through to make sure enumerations are coming through correctly - OWL specs may require URIs for enumerations (this would be an issue for next round of review - how referenced

• Wendy will look at CV options now. Darren can look at later. This is not urgent.

• Adding documentation on lists that are currently there. Create notes of what should be added to each README file.

Item 23
Not critical in short term. Good to get multiple eyes on this.

Item 36

• Should be addressed in 4.0 BETA 2

• Add additional notes on previous discussions

• Group instances to see what could be managed in the same way

FUTURE
Administrative documents (Jon) as available (Jon not on call)

ATTENDEES: Wendy, Dan, Flavio
POST-MEETING WRAPUP (due to forgetting US change to DST): Jon, Darren, Oliver, Christophe

Review of approach to issues 8, 23, and 32:

• Issue 8: (spreadsheet has been added to issue)

  • Identify good canditates for conrolled vocabulary

  • Group itesm need either documentation of definitions of types or use a controlled vocabulary. This is true of internal enumerations currently containing documentation.

• Issue 23: (spreadsheet has been added to issue)

  • When versionables reference other versionables looking for order needs - assigning of contents is required or used

  • DISCUSSION: How explicit order gets injected - clearly needed in some serializations but XML has two camps (order implied, order not implied)
    First step is identifying where order is needed

Additional Topics:
ISI presentation:

ISI do we want to propose something on DDI like last year in Ottawa?

Does TC want to send something - IPS

We have 3 weeks - kick around an idea for a topic

Something beyond use cases

Parts of DDI - multiple serializations

How content fits together and can move between products

3-4 people with or without discussant

XKOS:
Best Practice ready to go so look for email on that

ATTENDEES: Wendy, Jon, Oliver, Dan, Jeremy, Darren, Flavio, Christophe

Codebook

• Pull request has been amended and content is now correct

• Looking forward, the concept/conceptualText will be posted immediately following the resolution of the current pull request

• This will be followed by the remaining issues which are more targeted

ACTION ITEM:
Dan will note items that are NOT documentation change so they can be reviewed next week

DDI-L 4.0 Beta 1 Issues

• Another beta the middle or end of April after gathering COSMOS feedback

• An issue on representing sentinel variable values in Represented Variable

• All new content issues should go into GitHub. They will be tagged for releases.

• This will also allow broader discussion of content issues in clearer way than we have been able to do before.

ACTION ITEMS:
Wendy:
#36 - go back and pull together the earlier discussion of dealing with extended reference and review current need for continued use of this content
#8 - review enumerations as converted to CSV and conversion to outputs
#23 - locations of ordered attributes
Jon:
Administrative documents - add content on #16, #17, #30, #31, #32, #33, #34

ATTENDEES: Wendy, Jon, Oliver, Darren, Christophe
APPOLOGIES: Dan, Flavio

Webinar review

• A lot of people who we seldom or never seen

• Good suggestion:

  • Guidance on pull requests

  • We can set up templates

  • Contributors guidance document

• Missing documents

  • What is the model

  • Master?

  • Development version

• In terms of code the past published specifications can be compared

• Between 3.2 and 3.3 we could do a dif but we can't track between 3.3 and 4.0 because each generation is structurally different (dependent on mapping and change documentation)

• The ability to track individual changes

• How do we want to do this? If we publish 4.0 as a package

  • Track each time only model changes not differences in different serializations

  • We could do this locally [there will be so many structural changes a dif won’t work]

• At the moment published versions are in Bitbucket so we need a separate repository to hold published versions in GitHub to replace this

• Jon has started adding files (citation, license, contribution guidance, form for contributions

• One issue about having a missing license file we use cc-by [Note that the Alliance has determined that this is the license to be used by all products. We will need to check license on production programs (cc-by was not on the list of options]

• Document how things move through approval and that people can build on their own branch

• There is a GitHub project called DDI-CDI and DDI-CDI-resources

• Needs to be on DDI Alliance set of repositories

• There is no governance on the Alliance site

ACTION: Jon will start drafting governance documents

Status on CV and pipeline

• We have Simon George booked on 17-18th

• Can we have a repository for DDI-CV-platform (have DDI-CV-publishing)

  • Public
    https://github.com/orgs/ddialliance/repositories?type=all

  • Promo from Postman

Codebook

• Pull request for ddi-c_2
  https://github.com/ddialliance/ddi-c_2
  Note that the first ff3481b is bogus. It is the second commit 0cce353 that contains the documentation changes to codebook.xsd

• How to process through these?

  • People could put in comments in before next meeting for the documentation

  • The documentation content and the next (Concept/ConceptualText) are large. Some review should take place outside of meeting and then deal with questionable areas with group. Future edits should be more localized.

  • Oliver will check to see if there is a clear way to correct and entry error (alt is to reject pull request, kill this Fork and start over)

NEXT WEEK:
Codebook - Documentation review

FUTURE AGENDA ITEMS (as ready):

GitHub administration and guidance

Publication repository on Bitbucket - options/plan

ATTENDEES: Wendy, Jon, Darren, Flavio, Christophe, Jared (guest)
APOLOGIES: Dan

GitHub:

• Darren contacted Jared about AWS account and he needs clarification

• Background:

  • Trying to detach those things that are difficult to do with ICPSR infrastructure (underlying databases for examples, etc.; field level documentation; XML schema; etc.)

  • Set up AWS instances where these can be held

  • example: DDI Lifecycle model generates a lot of output and AWS is a solution

  • Separate web site for documentation and links to sources (docs.ddiaaliance.org)

  • Automation of creation pipeline of CV content from CESSDA output

• Currently we have 5 or 6 administrators already on GitHub - so we can manage rights for individuals

ACTIONS:

• Need to set up an account on AWS so that we can begin organizing this

• Jared needs to set up the account with appropriate permissions

• Darren will ask Michael for the DNS zones

• Wendy and Jon will create a list of items that might move from ICPSR to this new site

Clarification of document repository approach

• CDI will have a place to run their infrastructure

• Outputs will go on AWS

ACTIONS:

• Build a spreadsheet of where things are

• Start a TC ticket and add Darren as a watcher on that

DDI-L 4.0 BETA review of webinar

• Jon has done slides

• In middle of the webinar Jon would like to demo a change on a fork and generate

• 35-40 people are attending

• This is a BETA and there are already errors which need correction

• Public review in summer at the earliest

Questions regarding spreadsheet

• Specific example:

• The blue items on the spreadsheet are only removals

• Some are serialization specific and therefore need to be documented

ACTIONS:

• Add to key meaning of a,e,b in 3.3 column

• Finish sumamry of purpose of moving to Version 4.0 / use of COGS

• Get these posted and link information to Jon so he can add to slides (by Friday)

• What is the current status of CDI - finalizing documentation review checking images, Arofan is wrapping up the documentation and pulling together

XKOS update:

• XKOS best practices document almost finished integration of issues - probably out before COSMOS, maybe next week

• They will create an announcement and we can get this out and posted on the web page

ATTENDEES: Wendy, Jon, Darren, Dan, Oliver, Jeremy
APOLOGIES: Flavio

Infrastructure:
Set up account - and we need someone to work with Jared
In terms of setting up environment - need someone - Darren is willing to do set up
Getting permissions to the right people
Contact Olof and look for a person to follow-up and manage over time
Apache server serving a static site
Separate containers for different things - keep it simple
cover docs.ddialliance.org instances

AWS issue:
Darren - we can get Simon 2 weeks in March 11-18th) to get the CV set up at UKDA
Moving pipeline off Bitbucket to GitHub: Oliver has already moved it to GitHub and Oliver will send URL of repository
https://github.com/ddialliance/ddi-cv
The creation of repo content is still manual but should be turned into a pipeline in the nexts
Sanda can check out the html specifications. Pascal can also do checks

Webinar
Spreadsheet 3.3 to 4.0 - post for TC and webinar support
Jon has updated the presentation outline based on TC comments
Add Demo of COGS showing how easy it is to make changes and then generate their content for testing
Members planning to attend need to register

FUTURE AGENDA:
Next week: review of webinar in prep
Two weeks: Check in on CV, RDF, and repo pipeline - sanity check

ATTENDEES: Wendy, Dan, Jeremy, Flavio, Oliver
APLOGIES: Jon

DDI-L 4.0 BETA webinar

As of last Friday, we had 35 sign-ups.

Draft of short presentation contents
https://docs.google.com/document/d/1jeqDxr_4P3HM-oeW4j-g-WYr3aAKQeUwNooyilgGWbY/edit?usp=sharing
See document for comments

AWS set up

TC budget for AWS has been approved, so we need to get moving on setting up an account, presumably we need to get Jared to put the Alliance down to pay that.
Uses funding requested this year for infrastructure support. Next fiscal year this should be proposed as a new permanent budget line for association.

Talk to Darren and/or Olof about taking this on to get set up.
Regarding cloud deployment. Whoever sets it up we should management deployment using build scripts. The CV stuff is set up this way and there is something to start with.
We will need to add in things on the documentation. See how long Jared would need to set up the account. Be clear what needs to get done prior to setting up the cloud space and moving things into it.

Product Page

Implications for how the content is made available on the download and doc site for each product
Currently use docflex for the Field level documentation of the XML. Should this be continued with the COGS product. XML specific documentation as well as language specific documentation need for field level documentation.
Can we add to the doc site (links to specific output documentation)

A revised example will be made based on comments. Follow with examples of transfer of all product pages.

ATTENDEES: Wendy, Jon, Dan, Oliver, Christophe
APOLOGIES: Flavio

DDI Lifecycle 4.0 BETA webinar

• Went out on DDI Users Group and some socials

• Put a reminder out 2 weeks before event

• Check with Jared on numbers

• Jon will draft up something for next week

DDI infrastructure proposal

• Has contacted relevant working groups

• CV XKOS and SDTL are fine

• CDI group ok; some longer term issues that can be addressed as they come up in terms of production line

• Add some statement about governance around GitHub

• Can start setting up landing page for development work so we can inform people

Atlassian

• Some worries

• Many people are moving off it and this could cause the loss of open source support (due to lack of paying customers)

• We have moved off in terms of Bitbucket

• But we still have JIRA and Confluence

• Clean up JIRA in the next year to drop unused and archive old issue trackers

• Consider free support services we use to develop contingency plan across the board

NEXT WEEK:

• Product page mockup

• Draft of 4.0 BETA webinar content

• Administrative Data possibility of moving to next fiscal year

ATTENDEES: Wendy, Jon, Dan, Oliver, Flavio

Discuss page change drafts for products (TC-242, TC-243, TC-244)

• Product Page list of versions OK Get this done first as this resolves the Google reference issue

• The page of links for filing issues and how-to page are a good idea

• Product version page revision is a good first step by pushing critical information to the top and making it visible on the initial screen. Continue to revise based on information from developers/implementers and prepare updated draft along with an example from Lifecycle. These need to get updated prior to publication of CDI, new Codebook 2.6, and DDI Lifecycle 4.0.

• Issue to address in future: Current download packages are now served by an on-site (ICPSR) copy of the product and XML is provided for validation purposes through the ICPSR site. How should this change with multiple serializations (XML, RDF, JSON, UML-XMI, etc.)? The Bitbucket repository of published products currently serves as an archive and is not linked to the product pages consistently.

Organization and ongoing management of GitHub content
This was just a heads-up on the type of information TC needs to organize regarding GitHub space. It is primarily management information (Guidelines for development groups and users, tooling pipelines, space management, constraints regarding triggers for moving from free to fee based services, etc.). This is still in a period where we are obtaining comment. These notes from the CDI meeting will be more formally organized and provided in writing by CDI. The Scientific Board is aware of this work and is primarily monitoring the process to ensure input from development groups affected by the shift to GitHub.

TC pages under Learn dropdown on Alliance site
Resources:
We need to look at how examples, tools, and profiles are managed and organized in the new system. We will no longer have the on-site option of underlying databases. Primary decision is which, if any, of these should be managed off-site.
Getting Started:
See TC-245 for collecting information on determining the purpose of this page, what information should be provided, and how to present it. Members were asked to think about these issues and add comments to the issue so that we can move on revising this page in cooperation with the Training Group and probably future Marketing Group. This should be a priority. Jon will talk with Training leaders to get their initial thoughts.

NEXT WEEK:
Webinar for 4.0 Beta

No meeting was held. The following is the information sent out to the group for their attention and action:

TC issue tracker
I have reviewed and updated all entries
In particular see TC-242, TC-243, and TC-244 and add comments regarding layouts provided in attachments

Use filter TC-outstanding to filter out completed issues

Notes from DDI-CDI meeting on 2024-01-17

Comments about Git repository raised during CDI meeting on 2024-01-17 - note that Achim was not present so there will be more
Git repository management - how does TC support product groups in this?
Guidelines
Informational materials
Help identify persons if group is unable to provide this
Deirdre is willing to be involved as setting this up
Back-up people
Git repository
Tools/workflows

Organizing BETA access, support repository, policy, pull requests
Coordinate how to organize
Working with input from outside of the group (developers work)
Making clear what things are
Granularity of pull requests (issue based)
Branching - readme on how to do this, what is supported
Pricing models for AWS (space is not the issue, but deploying own servers/code base)

Each product needs to provide a canonical expression of the product
CDI is the UML model expressed as canonical XMI
Each product needs to be clear on this - how to visualize the model in the case of COGS base
Canonical model has to be publicly distributed

ATTENDEES: Wendy, Jon, Dan, Jeremy, Flavio, Oliver

Update on CDI work

• Variable Cascade issues - providing guidance through webinars, documentation (it appeared there weren't direct predicates linking along the chain, appeared there were just a single predicate used for different relationships)

• Will this require additional review?

Topics for future webinars

• Why are there various ways in dealing with the same content across products (classification/codelists etc)

DDI-L 4.0 BETA review

• Supplemental content - changes in the content model; why and what was done; update DDI Lifecycle Best Practices (how to use some of the schema structures to address 4.0 in addition to 3.x best practices)

• Updating model - just raising issues and modifying at the end

EDDI session on priorities

• Nothing specific struck a cord in terms of TC

Follow-up on - web pages and webinar with Jared

• Wendy has recieved comments on pages

• Jared has responded to Jon who will make arrangements for webinar on 4.0 BETA

ATTENDEES: Wendy, Jon, Dan, Jeremy, Flavio

DDI-Lifecycle version 4.0 beta follow-up

Webinar of some description
Q&A - Highlight of why we moved to this and how they can interract with it
or Anticipation - presentation

• Q&A seems best approach

• A bit of introduction to version 4.0 (about 5 minutes)

• Back end of February would be good timing

• Discussion pieces out from mid-Jan to mid-Feb: get a list together and see what is important to front-end

• 3:00 UK (10 eastern, 9 central) - 21st or 28th of Feb (Jon will follow-up on this)

• It's about a small group zoom with google form for registration

• We've only really asked about OWL serializations

• Do we want something more on the model conceptualization?

• On the serialization things?

• Model is the same as 3.3 with only changes made to so only specific changes for choice and physical description

ACTIVITY: (Wendy review and create list) pull discussion pieces together, clean up, post, and send links
Roll out discussion pieces.

Product pages on Alliance Site

Overview of products page can be updated but is in good shape overall
https://ddialliance.org/products/overview-of-current-products
Product pages:

• Less wordy

• More prominence for the access to the html documentation

• Update links

• Setting up pages that were held off the site (determine what should be off site support - RDF resolution

• All of the products have autogenerated documentation - move to a docs.documentation site and link from the main pages

• Examples, tools, etc are candidates for moving off-site

• Get comments from Jared (contact him for what feedback he's gotten)