- Created by Wendy Thomas, last modified on Aug 25, 2024
You are viewing an old version of this page. View the current version.
Compare with Current View Page History
« Previous Version 126 Next »
Earlier Meeting Minutes:
2022-2023 Minutes Page, 2020-2021 Minutes Page, 2018-2019 Minutes Page, 2016-2017 Minutes Page, Pre-2016 Minutes Page
ATTENDEES: Wendy, Jon, Oliver, Darren, Jeremy, Dan
CDI update
Roadmap/timeline requirement
There are documents on what is required and funding for implementation was part of this years request. They will prioritize this work product. Achim is back on Sept 11 and we should get roadmap and timeline shortly afterwards. Achim and Deirdre will work with Oliver on setting up pipelines in established DDI production work areas.
CDI timeline
They plan on submitting the package for voting process by Friday 20 September. Out goal will be to get it turned around by 27 September or shortly thereafter. Tentative date for the webinar is Thursday 3 October (of following week if necessary). The voting period should start within a 3-7 window on either side of this date. Wendy will coordinate with Jared. I noted that we will still be running a dual website at that point so the presentation of products may shift some pre- and post-voting period.
CV update
Request for DNS change should be made in next few days and be functional approximately 48 later. This will then be tested, followed by deployment, and then set up of RDF vocabulary resolution platform. Arofan will check on status 3 September.
Sanda wants to do more changes on the home page. Should be live next week. Entry shifts from Jared to Darren
HubSpot update
HubSpot needs to switch to US HubSpot outlet
Jon is talking to Jared on Monday to get the quote sorted and processed
We have a contact on the finance end at DDI
Darren has export .5 GIG of files and tables
Oliver - website transition
Tried importing the News item and then stumbled into nightmare of html from work imports. Think of transferring last 2-3 years of news by hand into the system.
Do yearly dump to PDF of news items.
Make sure this is ok with Jared and note it as a policy.
docs.ddialliance.org waiting for DNS change
Product section
What is going where and how is this organized
the doc site will contain information for people technically implementing
Web will have informative text content for understanding individual products. Add links to key product content:
Technical/Implementation button
How to make this visual being able to use useful glif/icons
Just waiting for the new DNS update
Presentation proposal for EDDI on Roadmap
Updating Roadmap (priorities, timelines, modifications)
Next week review Roadmap and finalize submission - look at the roadmap in the meantime https://docs.google.com/spreadsheets/d/1b390D4CpPXqtqi7-e5GhrvWdHbNEG697/edit?gid=728820206#gid=728820206
Other submissions to EDDI:
Darren - CV platform
Dan/Jeremy - Colectica
Oliver - Colectica
ATTENDEES: Wendy, Jon, Oliver, Dan, Jeremy, Darren, Jared (guest)
Regrets: Flavio
F2F and Web site planning
See https://docs.google.com/document/d/1DIPnvDMWo6SNxxf7bH4O5cwWCUWkrHL-aDlRNCqVM9M/edit?usp=sharing
ACTIONS:
Jared:
Updated dump of database - completed 2024-08-08
Functionality of administrative section
Meeting time with Michael - set for 2024-08-14 8:30 EDT, 13:30 London
Wendy: (Notification to Content Managers)
Hold on updates during next week (Training, Admin, Marketing, Product) - completed 2024-08-08
Text of email:
Chantal, Kathryn, Jon, Jared, Hilde, and Arofan,
I am writing to you as content managers on the http://ddialliance.org site. I have included Hilde and Arofan as they are preparing DDI-CDI for a vote in September and may need content changes in the website regarding this.
The Technical Committee will be beginning the process of moving the http://ddialliance.org site from ICPSR to the UKDA. ICPSR will be creating downloads of the site content between now and Tuesday 13 August. We will then be creating a prototype of the new site. We will be running parallel sites while we check out the accuracy and stability of the new site. We would like to keep any changes to an absolute minimum for a period of time. We should be able to provide an estimate of this time period by the end of next week.
To that end please:
Do not make any changes to the current http://ddialliance.org site before August 20.
Follow guidelines that the TC will provide before August 20 regarding any necessary changes made for a period of time following August 20.
If you need to make content changes before August 20 or during the limited access period defined in the guidelines we will provide, please contact me to ensure that changes are reflected on both sites
We understand that upcoming events such as the CDI vote will require some changes as well as needed announcements. We will keep this period of limited change as brief as possible. We simply want to ensure that content changes are made to both sites in a consistent manner.
Thank you for your help with this move,
Use of Zoom Summary
Jared forwarded the summary from last week
Wendy has sent to members for review and comment
Looks useful, will need to configure meeting set up to have summary sent directly
Post meeting:
Jared raised 2 additional issues in an email which will be added to aggenda outline
Web hosting expenses. What will they be and through which provider. If above a certain amount, I may need to vet the provider through UM.
Administration. I'd us to keep the number of people who can make direct edits on the site to a low number. Prior to the clean up a few years ago, we had 70+ users with edit permission. Most hadn't logged in for years (and I removed their access due to security risks), but the disparate editors sometimes made for dislocated content. My recommendation is to have the TC edit the Products section. For the other content (mainly involving Training and Marketing), I suggest requests work through me. This helps make sure we have consistent messaging. Thoughts?
ATTENDEES: Wendy, Jon, Dan, Jeremy, Oliver, Flavio
Codebook - Pull Request 41
F2F
Jared's email
This is to check on the status of the web site migration. When do you plan to switch it over to the new host? When can we turn off the ICPSR-hosted site? Will we retain the same URLs at the new hosted site? Do you need our web team to be available for a few hours during the planned switchover? For content transfer, is this inventory still accurate?
I had one to-do relating to formalizing an agreement about who is doing what for web and resolution support. I recently spoke with the UM General Counsel office about creating legal agreements for web site hosting, maintenance, and support, as well as supporting the resolution system that has been set up to support CV and RDF vocabulary resolution. While we do not need details to create a draft for the web site actions, can you please specify the specific tasks and deliverables for supporting the resolution system?
I'm out next week without access to email or phone but will be back on the 5th. Can we talk then? I'm especially interested in how the Secretariat-owned sections, like Get Involved, Publications, About will function on the new site. I'm also interested in communicating these planned changes to the wider community.
Notes from CDI for TC:
Lifecycle using CDI as a bridge standard to SDMX
RDF expressions of DDI versions are moving forward
See if we can get Eurostat to look at the RDF from Lifecycle
UNECE is having a meeting 3rd week of October with side meeting of CDI/SDMX alignments
Eurostat has a conference in March NTTS (submission date 30th of September)
https://cros.ec.europa.eu/dashboard/ntts2025
CDIF
http://Schema.org in the middle pointing out to different things
Handling handoff between CDIF and DDI is CDI
There hasn't been a discussion of Products that should be involved
TC is accountable for some outreach activities and we need to coordinate this better with Scientific Board and other outreach activities. There should be broad support for the Product Suite with targeting taking place where appropriate.
ATTENDEES: Wendy, Dan, Jeremy, Adrian Dusa (guest)
Regrets: Jon, Darren, Oliver, Flavio
Codebook pull request
approved, merged, related issues closed
Codebook issue #18
Adrian Dusa will join the meeting for this discussion. Please look at the issue prior to the meeting.
https://github.com/ddialliance/ddi-c_2/issues/18
ADDED to Issue #18 by Dan: At this week's TC meeting, we discussed the current attribute documentation and if it makes sense to add explicit documentation at the attribute level such as
<xs:attribute name="..." ...>
<xs:annotation>
<xs:documentation>...</xs:documentation>
</xs:annotation>
</xs:attribute>
Adrian will provide examples of end products for suggested changes. These will be vetted with Jon regarding docbook HTML documenation creation. Timing this for the last set of structural changes to be made to schema prior to presentation for vote. Goal is to provide consistency in presentation for ease of reading and to better support capture and transference of documentation to other tools for codebook automated maintenance or new and existing codebook tools. May want to run this by Mehmood for WorldBank tool and Tools group (at least Olof for his tools).
DDI-L 4.0 BETA2 issues - continue to locate previous discussions and link or copy to issues. Whatever we can get done prior to F2F is great but at a minimum issues for discussion should be clearly defined to have a focused discussion.
Web/F2F prep
Jon has added some notes on doc.ddialiance.org organization as well as on the products section
Please look at these and comment or add to. One issue may be looking at when we get into reorganization of section content. For products some is being done before the F2F as we discussed in in March. Other reorganization may need to take place after the F2F within the context of overall revision of the web site. The primary goal of the F2F is to lift and move the current ICPSR site content and make sure there is no loss. Organization of new areas like doc.ddialliance.org are definately on the F2F agenda as we want those well organized and usable in September for new publications.
https://docs.google.com/document/d/1DIPnvDMWo6SNxxf7bH4O5cwWCUWkrHL-aDlRNCqVM9M/edit?usp=sharing
CHECK out the AI Companion and Start Summary for assistance with minutes - play with this next week
ATTENDEES: Wendy, Jon, Dan, Flavio, Jeremy
F2F preparation
Face-to-Face
https://docs.google.com/document/d/1DIPnvDMWo6SNxxf7bH4O5cwWCUWkrHL-aDlRNCqVM9M/edit?usp=sharing
Website Management
https://docs.google.com/document/d/1ihi4UlL4calygWp1wgFuO8Npqh_xPvtl4evAtXB-rEQ/edit?usp=sharing
Web pages
Added page on filing issues and product level pages as per discussed earlier in the year
Overview of what's in the suite and comparison of - rename this
Contents listed on product page for each version (should list occur here or just link
Can this be flatted out more
Codebook
Decide layout plan (indention, line spacing) and prepare validation process in GitHub
Delay cleanup of structure until last step
ATTENDEES: Wendy, Jon, Flavio, Dan, Jeremy
Codebook pull request - merged
DDI-CDI next steps -
Review processing issues - none filed
Verify URN for version 1.0 [Darren and Arofan] generic ddialliance.org/specification/DDI-CDI will be used for the general project page with links to each version
Check with about the listed issues
Face-to-Face meeting prep
Use next 4 meetings to ensure we have a clear listing of steps to complete during F-to-F
Includes getting a DDI-L v4.0 BETA2 out the door
4.0 critical issues in preparing BETA2
Ordering
References extension
Documentation websites on GitHub get it on CI automation -> docs.ddialliance.org
Process for putting all other docs on the documentation website - let DDI-CDI
There may be a standard Header/Footer for common look and feel
FUTURE AGENDA ITEMS:
Face-to-Face meeting prep
Website prep - get dates
ATTENDEES: Wendy, Jon, Oliver, Jeremy, Flavio, Dan
REGRETS: Darren
As of 1 July 2024 Jon Johnson will step down as Vice Chair of the Technical Committee. Oliver Hopt will serve as TC Vice Chair from 7/2024 through 6/2027.
Codebook pull request
Approved and merged - covering references to DDI CV’s, typos, missing documentation
Codebook - Simple Text Review
Approved 8 identified changes out of 86 simpleTextType that seemed logical (respUnit as parallel to anlyUnit) or where known non-DDI CVs exist within a discipline or organization (imputation types, availability status)
Make those changes and retain spreadsheet for future Codebook Working Group
DDI-L v.4.0 BETA
Priorities are ordering and internal enumeration issues
Many issues administrative; Jon will add content to these
Face-to-Face will be 3 days and we should be prepared to spend on BETA work to the extent possible
ACTION
Wendy will provide a google doc to list technical and content management issues for the Alliance website https://docs.google.com/document/d/1ihi4UlL4calygWp1wgFuO8Npqh_xPvtl4evAtXB-rEQ/edit?usp=sharing
Please add to this document with additional issues or expansion of listed issues
ATTENDEES: Wendy, Oliver, Dan, Jeremy, Flavio
REGRETS: Jon, Darren
Codebook pull request
Merged license update issues 97 and 93
Codebook review of simpleTextTypes
Issue arose in adding in documentation on existing DDI CVs of possible simpleTextTypes that should be conceptType or conceptualTextType (ex. “software”
ACTION:
Create annotated list of these on spreadsheet and send out
send out to SRG list to open for other to comment in open google doc if comments needed
Wendy do this and send it out
DDI-L V4.0 BETA issues and workplan
Pull request adding license (issue 32) merged
People should review issues and work on one of their choice
It would be good to have content changes (as opposed to administrative issues) addressed by August meeting so that if there is time we can move on getting BETA2 out for review.
ATTENDEES: Wendy, Darren, Dan, Jeremy, Oliver, Flavio
REGRETS: Jon
TC Chair and Vicechair for July 2024 - June 2027
Main concern is that the role is too much. We need to pull out Lifecycle and Codebook and look at the workload overall to spread out among the group.
Face-to-Face meeting
Thanks for your responses. I'll let you know the preferred times although we still need to adjust to the availability of the hosting agency(s).
13-15 August
Work on DDI-L V.4.0 BETA2
scheduling this work given people's availability
Preferred approach is to just starting to put it on the schedule
Marking the properties that need explicit ordering for RDF serialization
Codebook - please respond to the request for comments on the most recent issue. I will submit the pull request around 6 pm CDT Wednesday. High level documentation - outline
Pull request merged. One typo found DDICODE-540. Enter remaining schema updates DDICODE-540, 97, and 93. Change merged IN REVIEW items to closed. Start documentation issues that are related to high-level documentation.
ATTENDEES: Wendy, Dan, Oliver, Flavio
REGRETS: Jon
Codebook pull request
check vocabID or vocabName?
lines: 2558, 2797, 4336, 4574
Follow-up findings: vocabName is not an attribute (vocab); vocabID was added to replicate content from DDI-L. Documentation on vocabID was updated to clarify the use.
DDICODE-539
-
Getting issue details...
STATUS
Face-to-Face Meeting
Oliver not available in July 31st of July is first day available
Flavio remote availble in July/August
Dan
Begining of the week of 22nd OK
Begining of August 1-2nd goood
August OK except middle of month weekend
Poll - for availability
CDI review
Extend to end of June - email everyone with a prod to finish and file their work
Glossary review
Nudge to Training group people
Feedback from SB and EB meetings
no additional comments
2024-05-30 NO MEETING - IASSIST
ATTENDEES: Wendy, Jon, Darren, Oliver, Dan, Flavio
Update on CV and RDF resolution system
Overview of RDF system (on wrong system waiting for switch of DNS by Michael)
Showed timeline quiet launch 17-June
Formal TC approval to go ahead and go live on the 13th
Not all target languages come through in the same manner - Oliver working with algorithm
UKDA will take responsibility for maintaining this, but we need to formalize this - check with Jared on what is needed
XKOS is ready to go - only have a turtle would JSON be useful? Check with Franck
ACTION: Darren will talk to Franck next week regarding JSON for XKOS
DISCO is not yet published
It needs to be flagged that its not published
Can we put something out on the list
Are people using it? Would it be helpful
Who would be maintain - working group
PRIORITY: We probably shouldn't say its a w3c spec but it hasn't gone through
PRIORITY: We should update template to make it DDI - It is in the tracker as completed in 2019
Update on DDI-CDI targeted review
Sent request to see who is covering what...no additional responses yet
Flavio - UML/XMI and is checking on syntax rep review
Update on Glossary review
Several issues filed (12 have been filed, Training Group people have also volunteered)
ACTION: Follow up with working group on publication (web plus?)
Codebook
new pull request - Dan will clean up Wendy's mess and we'll re-commit content for pull-request
DDICODE outstanding issues - updates on 93 and 97 Darren will look at 95
July 4th CANCELLED due to British Elections
ATTENDEES: Wendy, Darren, Oliver, Dan, Flavio
Web pages/ site plan
https://docs.google.com/spreadsheets/d/1bcvxmwoxUWyhZfTkuR9ZrVtmCxz2mkE8cFYYyjpCLtM/edit?usp=sharing
From Jared:
What would be useful is to make a final decision whether the Technical Committee is committing to supporting the website infrastructure or if we should go with the vendor. If the former, will you support all functionality currently on the site or just portions? At a minimum, I need a place to post announcements, manage a membership list, and manage/edit all pages other than the /specification sections.
There's other functionality on the site, like the Bibliography or the Glossary or the Tools list that are nice but maybe could be hosted separately. Those are also decisions we'll need to make.
Discussion:
Could support static or file content in something like S3 buckets
COGS stuff could be bundled up as static content and hosted on docs.ddialliance.org
Don't forsee any major problems in doing a serverless website that looks nice and bounces over to an object storage
Could do a test migration in July during a 2-day meeting
Can set up S3 buckets to test
Tools etc. could be a non-database filter system
Hotspot has the management system build into it that should support Jared's editing needs
TC taking this on would save the transfer fees of the outside vendor. We would need to look at ongoing maintenance. Zero risk as we can build this as a parallel system this summer, test it out, and institute further changes following that.
Do need to make sure we have document and project page organization determined so that new products and versions can be assured of URL consistency for linking purposes
ACTION: Darren will write to Jared
Scientific Board Plan
Clarified a few points. Will get final draft for tomorrow
CV publication
Basically there
DevOps team at Michigan have made the changes they needed to make
Working with CV group to update static content on pages
Problem of duplicate language (again) in the HTML
CESSDA is not seeing the problem - if they won't fix we could filter out duplicates as an interim response
Darren plans to go live the week after IASSIST if this works with the CV group
Meeting on Monday to finalize the ontoloty access
Oliver has discovered source of duplication: CESSDA has put an all of the versions together because they use a select bar. The Rest API ends up with 2 versions of both
We could switch to this display but it may be complex for the user
The CESSDA change dropped out some of our requirements - do we drop our requirements or ask them to make the change. We don't want what has come out with multiple versions (different versions) in different languages.
We are concerned with the RDF version and we need to go live with this. The HTML is still a problem
Codebook outstanding issues
DDICODE-93
DDICODE-95
DDICODE-97
Darren will look at these
Presentation on DDI-L .v4.0BETA - who presents
Will sort this out off-line
Future agenda:
Chair, Vice-chair (3 year term)
ATTENDEES: Wendy, Jon, Darren, Dan, Jeremy
Web pages -
General:
Want to control the website and not have it jobbed out
There is static content that could be usable but not necessarily optimal
Get it over and done and implemented and then think of the redesign
Need to get basics done first (lift and shift)
Opportunity to move to keep it simple then talk to groups involved (Training, TC, Publications)
Products:
Putting up stuff around infrastructure and pipelines
Top level URLs for the products will be hard coded or redirected
If CDI has specific requirements they need to write them down
This seems the obvious stuff to write this down
Document site:
Timing of when we are going to split the document move to doc site
Identification of documents should be part of the new site map
Do we retain the static pages as just the publications and say the docs are over there (old docs)
A page for product on the site but all in-depth documentation on the doc site
Useful to have a profile of what goes on the doc site for future additions and help to groups in identifying what goes where
ACTION ITEMS:
Darren and Jon will draft up a basic skeleton of the site map to kick against
Add to next weeks agenda
Known static content changes should be made before TC f-to-f meeting (ACTION: Wendy)
TC report to SB
Comments noted on draft
Update on current review work
I will continue to send reminders to reviews of both products during the review period
Glossary List
Review underway
Several volunteers from Training Committee
DDI-CDI Production:
Plans for sustainability of the standard; once its done in EA what is the pipeline (Jon will relook at the documents)
Olof - JSON
Pascal will begin in early June, may need extension
Dan - RDF output, looking primarily to see how they were transforming UML into RDF; special semantic relationships like traces in UML to identify patterns - how is this translated to canonical UML and then to syntaxes (are these just there to support their development of the model? this should be clarified)
Codebook (please look at before next week)
DDICODE-93 License - more detailed values for type attribute
DDICODE-95
DDICODE-95
-
Getting issue details...
STATUS
deprecation - URI attributes needed
DDICODE-97 move license element to useStmt
ATTENDEES: Wendy, Jon, Oliver, Dan, Jeremy, Flavio, Christophe, Olof
APOLOGIES: Darren
Scientific Board Strategic Plan
[note Flavio and Dan are elected members on the SB. Wendy is ex-officio]
Flavio noted a couple of issues he'd already added
"horizon scanning capability"
"more robust" more of a clear source of truth
1.4 W3C and explore - RDF representation preparation in 2024 - why would we want to give away the variable cascade to another organization
Increase visibility, not "give away" needs to be more explicit
1.6 no date so when is this supposed to be worked on? non-open source like Colectica - what does it mean? The notion that we are working with commercial vendors and make no mention of those we currently work with. What exactly are we trying to develop? Work with related software (analysis tools) regarding input/output support?
2.2 we don't have an activity defined as such, what is in 2024? Clarify what this is mean here? architecture of content or of repository. Starting when?
2.3 addition of SDMX and HLG to this group specifically or as part of whole DDI interoperability. What do we mean by this in terms of organizational level?
2.4 The example is about domains not geographies but domain areas. Awkward, examples do not relate to geographies.
2.5 Progress toward ISO certification (we started, why is this 2025? What is meant by progress. More details - contact continuation, paperwork,
GENERAL: of strategic plan to prioritize and focus work on specific areas. Concern there is too much for too many. Who is going to do this.
3.1 TC is working on in terms of organizing documents and publication
3.2 difference between 3.2 and 3.4?
Maybe some collapse into a broader infrastructure for development, management, presentation
GENERAL: Verbiage is abstract making it difficult to differentiate and make sure that different aspects are identified as sub-bullets
Omitted support of DDI Codebook 2.6
3.8 don't understand what this means (modularity within a product), but between products? Why is the example is a provenance model.
Discussion on user mailing list (conceptual, and reference variable) was the actual issue we would face with 3.8. things differ between products and there are different implementations in various products. Need for metamodel to understand what is found in multiple products, but varying implementation. Example of metamodels (Groups).
Need both metamodel to understand differences, and in some areas a single mullti-use content.
Suggestions in the scientific model that haven't been discussed. They seem to be arising from a minimum number of sources
"Let's do a provenance model" opens up a can of worms. No background discussion. There is a process raising topical rears and pursuing their development across products in one way or another.
HOLES?: Lack of content regarding how to use DDI suite together. (maybe related to 2.6) maybe expand to include 2024?
ACTION:
Wendy will add notes to document
Start list of content for report, next year priorities for TC; send out to get comments before next week
ATTENDEES: Wendy, Jon, Darren, Dan, Christophe, Jeremy
APOLOGIES: Flavio, Oliver
Newsletter submission
Thank you for XKOS work on publication process
Alert people to recording on Lifecycle 4.0 BETA and continuing work
TC report to Scientific Board
Web page support through AWL
Infrastructure
Codebook
CDI review
4.0 BETA continue to finalization
Glossary Group document publication
TC to look at in terms of technical review
Check in Glossary that its DDI
Hayley get a few Training people to look at
Christophe will talk with INSEE folks regarding clarity
Month deadline
DDI Lifecycle github issue
https://github.com/ddialliance/ddimodel/issues/38
Rather than think of the specific item raised here
What is the purpose of the represented variable
If its available it should be in represented variable
there is a difference between practical use cases which might not be the dominant ones
Issue is primarily around missing values and problems of how coding or options change from capture through processing to instance and physical store differences
Currently clear change from represented to instance is going from universe to population (spatial temporal specifications)
Different sorts of data throw up different problems
One of the strengths of the schema it is quite directional (this can also)
ACTION: Wendy enter this discussion on #38 (DONE); others should add comments on role of RepresentedVariable or other points as needed
NEWS:
Guillaume is returning come September
ATTENDEES: Wendy, Jon, Darren, Flavio, Dan, Oliver, Christophe, Jared, Jeremy
AWS and Drupal site interaction and support
Recap: some sections will move to AWS (specifications) remaining will be moved to wordpress
Is it wise to split this down the middle and move to 2 sites. This might be difficult to split correctly and to maintain in the future.
--Is it possible to move everything to AWS? How complex might this be to maintain and supervise
--Need to move away from Drupel as it is not supported by ICPSR and want to move it to a vendor
--Fee for moving it over and an annual maintenance amount ($3-4,000 a year)
--Urgency because support for their version of Drupal had gone away, did get year's reprieve
--In the mean time the AWS option arose and discussion of how
--Lots of static content that would just be moved
--Some functinality - bibliography that is maintained in an ongoing basis; posting news etc.
--Happy if it did not got to vendor
--Can we support on-going posting and maintenance
--If not, we can still go to WordPress keeping basic content there and other on AWS
Current view of what goes to AWS:
(1) Content under the following virtual URLs will be moved to the new UMich AWS instance
https://ddialliance.org/Specification/* - static content
https://ddialliance.org/products/* - static content
https://ddialliance.org/learn/resources - single static web page
https://ddialliance.org/learn/resources/ddi-profiles - single static web page
https://ddialliance.org/resources/tools - single static web page with filter
o Also move relevant referenced items under https://ddialliance.org/tool/*
https://ddialliance.org/learn/markup-examples-by-version?field_ddi_product_tid=All
o Also move relevant referenced items under https://ddialliance.org/sites/default/files/*
At the root level under http://ddialliance.org .
o https://ddialliance.org/Tables.dtd - downloadable document
--Clean up even before vendor
--Hidden stuff would not move and Darren's list may include that
--Splitting may have unintended consiquenses such as loss of database functions (publications, examples, etc.)
--Do we need all the functionality that is there (examples etc.) - if we add in with the vendor
--Would the options we are looking support these
--Issue about having to shadow the brand and sync it between 2 sites and with a 3rd party vendor
--What potential costs there would be with backup etc. - there is still need for web support (not completely volunteer)
--It seems to me that we are missing a written target sitemap (whether we have one or two platforms) that would guide the specifics of the migration.
--Could we take a few weeks out to design what we want the site to look like
Include Training Group
We have a current website that needs a spring clean but we have no idea what it should look like
Can we take 4 weeks to say this is where we want to end up (google doc) - we don't have much written down on this at the moment
Drupal to WordPress is somewhat automated there is also a lot of sorting and hand work to make the move.
https://docs.google.com/spreadsheets/d/1y54OKTmJeSV3YwxR6RBa7yJyHvD8P1drjPmG4C2vZoM/edit#gid=517461823
Lists out the sections and owner groups
Detail gets complex in the products sections
Resources has some sections
What exactly do we have - can we flatten it out?
ACTION:
We should look at what we want rather than just what we have
Create small group to look at this: Jared(co), Jon, Darren (co), Wendy
CDI technical review kickoff
Flavio looked at it
No draft of production process review issues - get from Arofan (send him a draft of request and list of people we've identified)
Automation is still lacking so how is that evaluated
We did write to them some time ago regarding - dig out letter we sent to them and verify all the parts are there
Asking more information on production process itself; make sure its available
Jon will look next FridayFunding requests
see google sheet: https://docs.google.com/document/d/111V1om6vbaJQCgxYI_znA03xDWW9HwW0iMGHxNVVnHU/edit?usp=sharingWorkplan for coming year
Review workplan for tweaksUpdates on feedback for DDI LIFECYCLE V.4.0 BETA
No only had one piece of feedback from Romain
BETA 2 would be good to do when we have the ordered flags
ATTENDEES: Wendy, Jon, Darren, Flavio, Christophe, Dan
Pages from Alliance site
We need to clarify what is moving from the current alliance site before it shifts to WordPress
The Alliance wants to keep this site a simple as possible and there are currently some pages under LEARN/Resources that have underlying databases
The majority of content we are looking at is static and is under Products
Everything that is under Specification is easy but need to determine what needs to be moved
The real question is about being able to identify specific nodes that can be handled by the remote site (RDF resolution system and CV services on UKDA site plus new DDI Alliance AWS space)
One possible way is just a lift and shift move all the /Specification
Landing page on WordPress site and then it sends you off to other system
What Darren needs is a site map (node list) so he knows what gets moved
Specific questions:
docs.ddialliance.org - what is this intended to provide
Currently we have a number of official documents associated with specifications that provide access to documentation such as field level documentation and high level documentation
These are static sets and can be moved
If it is difficult to tease out the nodes it may be possible to support static content under Products - all of this content is managed by TC
Resources:
Tools, examples, profiles currently have underlying databases and we would like to retain this functionality
These items are more in line with the products and it may be optional to move them under the Products umbrella. Learn could still retain a page that would describe and link to these resources
To accomplish this we need to locate a workable opensource tool to manage search and display of these databases. TC will begin this search
Agency registry:
DNS integration is being ended in 2025
Could move to Amazon Route 53 DNS service
ACTION ITEMS:
Obtain site map (node list)
Transfer page listing for move to Jared’s spreadsheet
Determine content that must move to retain or add functionality or to just support easy division of contents for routing
Clarify who will physically be doing the set up on the AWS configuration
Identify an open source tool to support access to database content for tools, examples, and profiles
Determine consequences of moving content to alternate site based on node divisions proposed
ATTENDEES: Wendy, Jon, Oliver, Darren, Dan
APOLOGIES: Flavio
1-Codebook pull request
ok and merged
2-Funding requests (Deadline is 26 of April)
Face-to-Face again
Infrastructure costs - flag and could suggest it should be a regular budget line; software contingency budget as well as on-going costs
10K last year was infrastructure contingency from which a couple of months of AWS
Alliance pays Colectica and this will move to AWS at some point
As part of the move to AWS a bit of work needs to be done to convert from zone file to AWS (route 53) - Dan will write upAdmin data meeting - talk to Flavio about meeting next year; funding request
3-Strategic Plan
Looks good - no questions or specific concerns from TC members at meeting
4-Administrative documents
Not ready for discussion
5-CDI submission to TC and production process review
Darren talked with Arofan about RDF URLs and everything is in line
Other:
CV work is done ready to make the change after confirming a few minor points with ICPSR and the CV group
https://ddicv.ukdataarchive.co.uk/ new format for the CV page (domain to be changed obviously)No ISI proposal but will look at the IAOS conferences coming up
FUTURE:
No Oliver until April 18
No meeting on 11th due to COSMOS conflict
ATTENDEES: Wendy, Dan, Oliver, Darren, Christophe
AGENDA
Resolution system
CV system
Access to https://github.com/ddialliance/ddi-cv-platform (am hoping that Oliver will be at TC tomorrow).
All is in handMisalignment of version numbers in https://github.com/ddialliance/ddi-cv - logged at https://github.com/ddialliance/ddi-cv-platform/issues/1
This has been resolved, however there is a problem with the resolving the old version numbers x.x rather than x.x.x
They will check with CESSDA after talking to Sanda identify where the problem is in terms of inaccurate extensions
RDF hosting/resolution
RDF hosting/resolution for CDI (upcoming) is not a dependent on the CESSDA CV version issue being fixed.
DB will contact Arofan to clarify CDI RDF serialization’s readiness and double-check required URI patterns for CDI
All CVs have unique URIs based on version numbers and are stored in Fuseki default graph. For non-CV products e.g. CDI or DDI4 (that don’t necessarily have versioned URIs) we will create individual graphs for each product.
Codebook pull request review
OK and merged
4.0 BETA issues - Comments also added to issue discussion in GitHub
Item 8
Changing from internal CV to external could be a Lifecycle 4.1 task
Identify those that could be CV
Add enumeration documentation to the item type
Group ones should probably be taken over by CV group
Dan will look through to make sure enumerations are coming through correctly - OWL specs may require URIs for enumerations (this would be an issue for next round of review - how referenced
Wendy will look at CV options now. Darren can look at later. This is not urgent.
Adding documentation on lists that are currently there. Create notes of what should be added to each README file.
Item 23
Not critical in short term. Good to get multiple eyes on this.
Item 36
Should be addressed in 4.0 BETA 2
Add additional notes on previous discussions
Group instances to see what could be managed in the same way
FUTURE
Administrative documents (Jon) as available (Jon not on call)
ATTENDEES: Wendy, Dan, Flavio
POST-MEETING WRAPUP (due to forgetting US change to DST): Jon, Darren, Oliver, Christophe
Review of approach to issues 8, 23, and 32:
Issue 8: (spreadsheet has been added to issue)
Identify good canditates for conrolled vocabulary
Group itesm need either documentation of definitions of types or use a controlled vocabulary. This is true of internal enumerations currently containing documentation.
Issue 23: (spreadsheet has been added to issue)
When versionables reference other versionables looking for order needs - assigning of contents is required or used
DISCUSSION: How explicit order gets injected - clearly needed in some serializations but XML has two camps (order implied, order not implied)
First step is identifying where order is needed
Additional Topics:
ISI presentation:
ISI do we want to propose something on DDI like last year in Ottawa?
Does TC want to send something - IPS
We have 3 weeks - kick around an idea for a topic
Something beyond use cases
Parts of DDI - multiple serializations
How content fits together and can move between products
3-4 people with or without discussant
XKOS:
Best Practice ready to go so look for email on that
ATTENDEES: Wendy, Jon, Oliver, Dan, Jeremy, Darren, Flavio, Christophe
Codebook
Pull request has been amended and content is now correct
Looking forward, the concept/conceptualText will be posted immediately following the resolution of the current pull request
This will be followed by the remaining issues which are more targeted
ACTION ITEM:
Dan will note items that are NOT documentation change so they can be reviewed next week
DDI-L 4.0 Beta 1 Issues
Another beta the middle or end of April after gathering COSMOS feedback
An issue on representing sentinel variable values in Represented Variable
All new content issues should go into GitHub. They will be tagged for releases.
This will also allow broader discussion of content issues in clearer way than we have been able to do before.
ACTION ITEMS:
Wendy:
#36 - go back and pull together the earlier discussion of dealing with extended reference and review current need for continued use of this content
#8 - review enumerations as converted to CSV and conversion to outputs
#23 - locations of ordered attributes
Jon:
Administrative documents - add content on #16, #17, #30, #31, #32, #33, #34
ATTENDEES: Wendy, Jon, Oliver, Darren, Christophe
APPOLOGIES: Dan, Flavio
Webinar review
A lot of people who we seldom or never seen
Good suggestion:
Guidance on pull requests
We can set up templates
Contributors guidance document
Missing documents
What is the model
Master?
Development version
In terms of code the past published specifications can be compared
Between 3.2 and 3.3 we could do a dif but we can't track between 3.3 and 4.0 because each generation is structurally different (dependent on mapping and change documentation)
The ability to track individual changes
How do we want to do this? If we publish 4.0 as a package
Track each time only model changes not differences in different serializations
We could do this locally [there will be so many structural changes a dif won’t work]
At the moment published versions are in Bitbucket so we need a separate repository to hold published versions in GitHub to replace this
Jon has started adding files (citation, license, contribution guidance, form for contributions
One issue about having a missing license file we use cc-by [Note that the Alliance has determined that this is the license to be used by all products. We will need to check license on production programs (cc-by was not on the list of options]
Document how things move through approval and that people can build on their own branch
There is a GitHub project called DDI-CDI and DDI-CDI-resources
Needs to be on DDI Alliance set of repositories
There is no governance on the Alliance site
ACTION: Jon will start drafting governance documents
Status on CV and pipeline
We have Simon George booked on 17-18th
Can we have a repository for DDI-CV-platform (have DDI-CV-publishing)
Public
https://github.com/orgs/ddialliance/repositories?type=allPromo from Postman
Codebook
Pull request for ddi-c_2
https://github.com/ddialliance/ddi-c_2
Note that the first ff3481b is bogus. It is the second commit 0cce353 that contains the documentation changes to codebook.xsdHow to process through these?
People could put in comments in before next meeting for the documentation
The documentation content and the next (Concept/ConceptualText) are large. Some review should take place outside of meeting and then deal with questionable areas with group. Future edits should be more localized.
Oliver will check to see if there is a clear way to correct and entry error (alt is to reject pull request, kill this Fork and start over)
NEXT WEEK:
Codebook - Documentation review
FUTURE AGENDA ITEMS (as ready):
GitHub administration and guidance
Publication repository on Bitbucket - options/plan
ATTENDEES: Wendy, Jon, Darren, Flavio, Christophe, Jared (guest)
APOLOGIES: Dan
GitHub:
Darren contacted Jared about AWS account and he needs clarification
Background:
Trying to detach those things that are difficult to do with ICPSR infrastructure (underlying databases for examples, etc.; field level documentation; XML schema; etc.)
Set up AWS instances where these can be held
example: DDI Lifecycle model generates a lot of output and AWS is a solution
Separate web site for documentation and links to sources (docs.ddiaaliance.org)
Automation of creation pipeline of CV content from CESSDA output
Currently we have 5 or 6 administrators already on GitHub - so we can manage rights for individuals
ACTIONS:
Need to set up an account on AWS so that we can begin organizing this
Jared needs to set up the account with appropriate permissions
Darren will ask Michael for the DNS zones
Wendy and Jon will create a list of items that might move from ICPSR to this new site
Clarification of document repository approach
CDI will have a place to run their infrastructure
Outputs will go on AWS
ACTIONS:
Build a spreadsheet of where things are
Start a TC ticket and add Darren as a watcher on that
DDI-L 4.0 BETA review of webinar
Jon has done slides
In middle of the webinar Jon would like to demo a change on a fork and generate
35-40 people are attending
This is a BETA and there are already errors which need correction
Public review in summer at the earliest
Questions regarding spreadsheet
Specific example:
The blue items on the spreadsheet are only removals
Some are serialization specific and therefore need to be documented
ACTIONS:
Add to key meaning of a,e,b in 3.3 column
Finish sumamry of purpose of moving to Version 4.0 / use of COGS
Get these posted and link information to Jon so he can add to slides (by Friday)
What is the current status of CDI - finalizing documentation review checking images, Arofan is wrapping up the documentation and pulling together
XKOS update:
XKOS best practices document almost finished integration of issues - probably out before COSMOS, maybe next week
They will create an announcement and we can get this out and posted on the web page
ATTENDEES: Wendy, Jon, Darren, Dan, Oliver, Jeremy
APOLOGIES: Flavio
Infrastructure:
Set up account - and we need someone to work with Jared
In terms of setting up environment - need someone - Darren is willing to do set up
Getting permissions to the right people
Contact Olof and look for a person to follow-up and manage over time
Apache server serving a static site
Separate containers for different things - keep it simple
cover docs.ddialliance.org instances
AWS issue:
Darren - we can get Simon 2 weeks in March 11-18th) to get the CV set up at UKDA
Moving pipeline off Bitbucket to GitHub: Oliver has already moved it to GitHub and Oliver will send URL of repository
https://github.com/ddialliance/ddi-cv
The creation of repo content is still manual but should be turned into a pipeline in the nexts
Sanda can check out the html specifications. Pascal can also do checks
Webinar
Spreadsheet 3.3 to 4.0 - post for TC and webinar support
Jon has updated the presentation outline based on TC comments
Add Demo of COGS showing how easy it is to make changes and then generate their content for testing
Members planning to attend need to register
FUTURE AGENDA:
Next week: review of webinar in prep
Two weeks: Check in on CV, RDF, and repo pipeline - sanity check
ATTENDEES: Wendy, Dan, Jeremy, Flavio, Oliver
APLOGIES: Jon
DDI-L 4.0 BETA webinar
As of last Friday, we had 35 sign-ups.
Draft of short presentation contents
https://docs.google.com/document/d/1jeqDxr_4P3HM-oeW4j-g-WYr3aAKQeUwNooyilgGWbY/edit?usp=sharing
See document for comments
AWS set up
TC budget for AWS has been approved, so we need to get moving on setting up an account, presumably we need to get Jared to put the Alliance down to pay that.
Uses funding requested this year for infrastructure support. Next fiscal year this should be proposed as a new permanent budget line for association.
Talk to Darren and/or Olof about taking this on to get set up.
Regarding cloud deployment. Whoever sets it up we should management deployment using build scripts. The CV stuff is set up this way and there is something to start with.
We will need to add in things on the documentation. See how long Jared would need to set up the account. Be clear what needs to get done prior to setting up the cloud space and moving things into it.
Product Page
Implications for how the content is made available on the download and doc site for each product
Currently use docflex for the Field level documentation of the XML. Should this be continued with the COGS product. XML specific documentation as well as language specific documentation need for field level documentation.
Can we add to the doc site (links to specific output documentation)
A revised example will be made based on comments. Follow with examples of transfer of all product pages.
ATTENDEES: Wendy, Jon, Dan, Oliver, Christophe
APOLOGIES: Flavio
DDI Lifecycle 4.0 BETA webinar
Went out on DDI Users Group and some socials
Put a reminder out 2 weeks before event
Check with Jared on numbers
Jon will draft up something for next week
DDI infrastructure proposal
Has contacted relevant working groups
CV XKOS and SDTL are fine
CDI group ok; some longer term issues that can be addressed as they come up in terms of production line
Add some statement about governance around GitHub
Can start setting up landing page for development work so we can inform people
Atlassian
Some worries
Many people are moving off it and this could cause the loss of open source support (due to lack of paying customers)
We have moved off in terms of Bitbucket
But we still have JIRA and Confluence
Clean up JIRA in the next year to drop unused and archive old issue trackers
Consider free support services we use to develop contingency plan across the board
NEXT WEEK:
Product page mockup
Draft of 4.0 BETA webinar content
Administrative Data possibility of moving to next fiscal year
ATTENDEES: Wendy, Jon, Dan, Oliver, Flavio
Discuss page change drafts for products (TC-242, TC-243, TC-244)
Product Page list of versions OK Get this done first as this resolves the Google reference issue
The page of links for filing issues and how-to page are a good idea
Product version page revision is a good first step by pushing critical information to the top and making it visible on the initial screen. Continue to revise based on information from developers/implementers and prepare updated draft along with an example from Lifecycle. These need to get updated prior to publication of CDI, new Codebook 2.6, and DDI Lifecycle 4.0.
Issue to address in future: Current download packages are now served by an on-site (ICPSR) copy of the product and XML is provided for validation purposes through the ICPSR site. How should this change with multiple serializations (XML, RDF, JSON, UML-XMI, etc.)? The Bitbucket repository of published products currently serves as an archive and is not linked to the product pages consistently.
Organization and ongoing management of GitHub content
This was just a heads-up on the type of information TC needs to organize regarding GitHub space. It is primarily management information (Guidelines for development groups and users, tooling pipelines, space management, constraints regarding triggers for moving from free to fee based services, etc.). This is still in a period where we are obtaining comment. These notes from the CDI meeting will be more formally organized and provided in writing by CDI. The Scientific Board is aware of this work and is primarily monitoring the process to ensure input from development groups affected by the shift to GitHub.
TC pages under Learn dropdown on Alliance site
Resources:
We need to look at how examples, tools, and profiles are managed and organized in the new system. We will no longer have the on-site option of underlying databases. Primary decision is which, if any, of these should be managed off-site.
Getting Started:
See TC-245 for collecting information on determining the purpose of this page, what information should be provided, and how to present it. Members were asked to think about these issues and add comments to the issue so that we can move on revising this page in cooperation with the Training Group and probably future Marketing Group. This should be a priority. Jon will talk with Training leaders to get their initial thoughts.
NEXT WEEK:
Webinar for 4.0 Beta
No meeting was held. The following is the information sent out to the group for their attention and action:
TC issue tracker
I have reviewed and updated all entries
In particular see TC-242, TC-243, and TC-244 and add comments regarding layouts provided in attachments
Use filter TC-outstanding to filter out completed issues
Notes from DDI-CDI meeting on 2024-01-17
Comments about Git repository raised during CDI meeting on 2024-01-17 - note that Achim was not present so there will be more
Git repository management - how does TC support product groups in this?
Guidelines
Informational materials
Help identify persons if group is unable to provide this
Deirdre is willing to be involved as setting this up
Back-up people
Git repository
Tools/workflows
Organizing BETA access, support repository, policy, pull requests
Coordinate how to organize
Working with input from outside of the group (developers work)
Making clear what things are
Granularity of pull requests (issue based)
Branching - readme on how to do this, what is supported
Pricing models for AWS (space is not the issue, but deploying own servers/code base)
Each product needs to provide a canonical expression of the product
CDI is the UML model expressed as canonical XMI
Each product needs to be clear on this - how to visualize the model in the case of COGS base
Canonical model has to be publicly distributed
ATTENDEES: Wendy, Jon, Dan, Jeremy, Flavio, Oliver
Update on CDI work
Variable Cascade issues - providing guidance through webinars, documentation (it appeared there weren't direct predicates linking along the chain, appeared there were just a single predicate used for different relationships)
Will this require additional review?
Topics for future webinars
Why are there various ways in dealing with the same content across products (classification/codelists etc)
DDI-L 4.0 BETA review
Supplemental content - changes in the content model; why and what was done; update DDI Lifecycle Best Practices (how to use some of the schema structures to address 4.0 in addition to 3.x best practices)
Updating model - just raising issues and modifying at the end
EDDI session on priorities
Nothing specific struck a cord in terms of TC
Follow-up on - web pages and webinar with Jared
Wendy has recieved comments on pages
Jared has responded to Jon who will make arrangements for webinar on 4.0 BETA
ATTENDEES: Wendy, Jon, Dan, Jeremy, Flavio
DDI-Lifecycle version 4.0 beta follow-up
Webinar of some description
Q&A - Highlight of why we moved to this and how they can interract with it
or Anticipation - presentation
Q&A seems best approach
A bit of introduction to version 4.0 (about 5 minutes)
Back end of February would be good timing
Discussion pieces out from mid-Jan to mid-Feb: get a list together and see what is important to front-end
3:00 UK (10 eastern, 9 central) - 21st or 28th of Feb (Jon will follow-up on this)
It's about a small group zoom with google form for registration
We've only really asked about OWL serializations
Do we want something more on the model conceptualization?
On the serialization things?
Model is the same as 3.3 with only changes made to so only specific changes for choice and physical description
ACTIVITY: (Wendy review and create list) pull discussion pieces together, clean up, post, and send links
Roll out discussion pieces.
Product pages on Alliance Site
Overview of products page can be updated but is in good shape overall
https://ddialliance.org/products/overview-of-current-products
Product pages:
Less wordy
More prominence for the access to the html documentation
Update links
Setting up pages that were held off the site (determine what should be off site support - RDF resolution
All of the products have autogenerated documentation - move to a docs.documentation site and link from the main pages
Examples, tools, etc are candidates for moving off-site
Get comments from Jared (contact him for what feedback he's gotten)
- No labels