Page Comparison

ATTENDEES: Wendy, Dan, Jon, Darren, Flavio

Implementation Language meeting around EDDI

Funding for Fall meeting on Implementation Languages has been approved. We need to develop a more fully developed outline for this meeting to discuss with CDI later in July.

• Week after would be better due to room availability

• EDDI Right after Thanksgiving (Wednesday-Thursday)

• Hackathon (check with Ingo)

• More generic meeting on the Friday for broader contribution and gory details the next week (3 days?)

• Identify specific implementations

• What needs to be done in a consistent manner

• What can be flexible

• Explanation of usage needed

Codebook 2.6 review

• wlt provide Jon with link - upload

COGS

No Jon next two weeks - no actions on COGS until then

August meeting of TC in Minneapolis

Funding has been approved

Specific goals and outputs from August meeting:

--Document changes in structure
--XML, RDF, JSON, UML/XMI (canonical), Sphinx and restructured text documentation
--Rules and output where possible - reconciling between the schema and outputs
--when we cut the cord that we are able to get back to the 3.3 schema
--What the different serializations look like - just doing vanilla transformations are a bit pointless
--Technical review - can roll out serializations separately
--XML and JSON are currently the most used and most stable and are probably ready first for review
--RDF and UML/XMI has been tested by EA
--If we wanted canonical XMI we have to change output or use the translation tool used by CDI
--Build in canonical XMI as an output (currently has 2 - Normative 2.4.2, 2.5 with diagram and diagram exchange) new flavor would be canonical
--Discussion of TC future leadership and roles
--Change log of XML structure between 3.3 and 3.4 to see what changed and why (supporting multiple serializations)
--Talking about changes - tree hierarchy is still the same but there are other things that are being updated to XML centric (CHOICE models) - substitution groups, removing a few things that are remnants of 3.0 and 3.1 (Identification and Reference properties)

CANCEL Next two meetings - June 30, July 7

ATTENDEES: Wendy, Oliver, Dan S

Updated on CDI status, Codebook review status, COGS input corrections, and LOD work at UKDA
Discussed August in-person meeting including increase in air fares and goals for the meeting

• Focus is on output of COGS

• Produce implementations that can be used for a broad technical review of new XML structures and new implementations - this is a technical review of the viability of these new structures and identifying issues that should be addressed.

• Discussion of TC direction over the next few years - new members, leadership changes, strategic directions

ATTENDEES: Wendy, Flavio, Darren

In person meeting

If funded by Exec Committee the TC In person meeting in Minneapolis at ISDRI has been approved for August 1-5, 2022. A note has been sent to TC members confirming the dates.

Presentation at Members Meeting and Scientific Board

The Members meeting presentation is only 3-5 minutes in length. Focus will be on funded activities and Codebook work. Will get a short statement from Darren so that this work presented accurately. Next year focus of coordinating multiple implementations of products and progressing on the production platform.

COGS decisions regarding Identification and Representation properties, use of citation

• Reviewed issues and had a brief discussion of properties

• Wendy will create Jira issues (relate to gitHub issues) so that a full discussion is recorded and retrained within the design issues discussed in TC. (GitHub is specific to the COGS implementation and this should reflect both the issue and decision). Google documents will be used to focus discussion and linked from the Jira issue (these will be downloaded as documents and attached when issue is decided).

• Members will be asked to review and comment

• COGS input issues (xml to csv) will be addressed as follows:

  • Modify transformation program where easy to do (new columns on csv with default values where information is not available in xml, etc.)

  • Export resulting csv files, edit, and reload - this is a one time import and the issues are primarily due to inconsistencies in the xml

  • Focus on import transformations should be on scripts that will be reused (example: import of canonical xmi from CDI to create a COGS copy)

  • XML will be modified for addition of managed representations for those few not currently available and inclusion of other physical data product content using name modifications for elements with same name in different namespaces. The intent is to support the shift to serialization and use of inclusion by reference and to provide all the record layout options for review without massive remodeling.

  • Handful of complex choice nesting locations will be mildly remodeled and documented in csv files.

ATTENDEES: Wendy, Jon, Darren, Oliver, Dan S.

Annual Report

No additional comments, will leave available through Sunday and then send in

Codebook

• Field level documentation HTML version has been completed

• Wendy will review Change Log and update, set up review page and create the google page for the high level documentation draft

• Review should start by end of May. This should last to the middle or end of August

• Anticipate easy review because there has been a lot of consultation - this should mean that developers can begin work using the Review version we some certainty

COGS work

Reference:

• Reference properties outside of the identification is the issue

• Reference is an entity and will act differently with serialization - RDF is a URI, XML / JSON model

• Review specifics of properties - Wendy will document

  • Example: Where you need to know a context of what you are referencing add another reference for context.

  • Including extra properties

• Option is to create Complex types

  • Reference becomes an element in the complex

  • Where is such extended content needed

Identifiers:

• don't think there is any difficulty by serialization. Some have their own identification (URI in RDF)

• Drop the difference between Versionable and Maintainable. There is only one way to identify an item (URN, AGENCY, ID, VERSION)

• Additional properties

  • Describable class extension of versionable

  • Properties found in versionable - review

• Identification is injected during creation of serializations

  • Inject identfication into any item type will have identification

  • Complex data types DON'T get Identification

• Another area of confusion for additional properties is the use of Name/Type/Description and Citation (currently used singularly or together in some elements)

  • Look at where citation is used and how. OtherMaterialTypes (clean-up later) Group? StudyUnit? ResourcePackage, Instance, etc.

CHOICE issues are being sorted

PhysicalDataProduct

• Add NCubeInstance to the basic RecordLayout now in the main schema

• Make the others derivatives of the BaseRecordLayoutType with unique element names and move the content to PhysicalDataProduct

• This will get everything in AND let us use one PhysicalDataProduct to import

Representations

• Recreate the few representation types that are not in the ManagedRepresentations.

• This gives them the same structure and we can determine later if we want to continue this division between representations that have references to other objects or what.

CV and RDF resolver:
The sandbox will be set up end of next week

https://rdf-vocabulary.ddialliance.org/cv
e.g. https://rdftest-vocabulary.ddialliance.org/cv/ModeOfCollection/1.0.1

ATTENDEES: Wendy, Jon, Darren, Oliver

CV work

• UKDS are finalising server setup next week.

• Oliver & Darren working together to get content in once it is setup. Target by the end of May.

• DNS needs to be setup at ICPSR.

DDItoCogs issues from Virtual Meeting

• Physical Structures (when to do remodel - couple of options for testing COGS input)

• Representations (when to do the remodel from mix of in-line and reference)

• Reference structures

• Identification

Standing on the Agenda

• TC will still have a shorter separate presentation (WG updates will be summarized by SB chairs)

Report for the Scientific Board Meeting

• Written report to the meeting

• Draft has been started - Wendy will complete and circulate

Date for F2F Technical Meeting

• Canvass for dates – August 2022

Codebook Review

• Field Level Documentation – Jon to do

• Documentation will be description of  changes

• Full High Level documentation to follow

ATTENDEES: Wendy, Jon, Darren, Dan S.

Update on CDI
--CDI is planning to get the package to the TC on June 1st. They realize we won't be able to deal with it until after IASSIST, but given the history of delays over the years, this lets them put a "win" in their column by getting it passed on.

Virtual Meeting:

• Updated the page with correct dates and links to the relevant issues in TC

• 9-11 CDT M-F plan on meetings. These may not be used but keep the slot free

Topics to discuss:

• Review where we are at with the input transformation

• Complex choice - status and how to redesign where need - chat first thing and then what we need to discuss later

Substitution Groups:

• Chose specific substitutions to use and how to implement

• XML substitution groups - is there stuff we want to merge or can we remodel that

• There is no model for the DCterms terms, these have been replaced by primitive types

Reference

• Design rules (managed references rather than modeled)

• Things that derived from Reference need to be added in other ways

• Handling additional reference information in terms of the overall model as it has to deal with multiple serialization targets

• References - is there some content that we need to retain that is not there

Rational to document:

• This will simplify the structure for everyone - in the future

• Draft of what was changed

• References were converted from a type to an auto-generated

Organization:

Monday

• Discussion on above

• Sort out who is doing what and when we get back together to discuss each point

• COGS validation process - what can be done intentionally and what needs to be done

Tuesday

• Discussion about what we are trying to do with the UML XMI input and output

• Realistically can the CDI model be collapsed into a COGS model

Thursday

• Can we have some consistency in the UML-XMI expressed by the canonical XMI

• What are the realistic possibilities of having a COGS maintenance pipeline for CDI

• Prioritization of outputs from COGS

Actions:

• Wendy will draft content on changes in the DDI Agency Registry for the DDI Alliance page, have Dan S. review it, post, and add information to the Scientific Board URN document

• Jon will add information from today’s meeting on the Virtual Meeting page including a draft meeting schedule

ATTENDEES: Wendy, Flavio, Darren

Funding requests

• Draft of the joint sponsored meeting on implementation languages sent to Arofan for CDI input

• TC meeting is just an update of last years with an adjustment of goals and outcomes

Codebook 2.6

• Finalize specification (header update) and make available to Darren (for CESSDA work)

• Complete draft of high level documentation by EOM

• Get out for public review as soon as possible in May

• Publicizing DDI Agency Registry work

• Draft content for web site in easy to understand words regarding capabilities and options

TC priorities for next year:

• Implementation languages - do not mean to hold up work of specific groups but want them to be aware of the work so we don't design ourselves into corners or have decisions driven by single products

• 2.6 completion and publication

• Requirements management

• Production framework

• Mapping within DDI Suite - translation work

• Explore options for Codebook in terms of design rules (what needs to be backward compatible)

• Syntax representations issues - maybe put new ones out is a beta-like mode for testing and identification of problem points - both Lifecycle and CDI plan RDF representations in the first half of fiscal year

No meeting

ATTENDEES: Wendy, Oliver, Darren, Flavio

DDI Agency Registry publication/announcement

• Sent email to Dan and Barry regarding this

• Would like to complete in the next 3 weeks

Identifying technical contacts in DDI

Notes from Jared's email
Next steps:

• At its next meeting, the Scientific Board will review and finalize the draft definitions of Scientific Representative and Technical Contact

  • note: from SB transition working group-content for Technical Contact was originally written by TC

• Jared will contact the member representatives requesting names for Scientific Representative and Technical Contacts. Both are optional.

• We still need to figure out the optimal way of sharing those names.

Preparing for May Virtual meeting - what we need to complete in April
SEE: TC Virtual Work Meeting - 2-6 May 2022
(linked from TC page upper left box of current activities)

Items below were noted today and are on the page

• UML/XMI review of what needs to be capture

• Updates to the ingest program of known easy to fix problems

• Outline the discussion issues around complex choices

• Identification/Reference details

DDI Alliance resolution system for CVs and RDF vocabularies
https://ddialliance.org/Specification/DDI-CV/AggregationMethod_1.0.html

• Darren talked to Miles and he feels its very straight forward beginning work on infrastructure the week of the 11th of April.

• The software PUBBY they were going to use is quite old - wouldn't really need if we use the CESSDA generated HTML
  https://github.com/cygri/pubby/tree/master/src/main

• Do we need a user interface for this - for general publication of CVs
  This can be done with html, skos, and codeList
  What would the user see if they followed a link. They would get the RDF or JSON structure. But the SKOS could contain the link to the html
  HTML as a mime type could redirect to the visual page

• The transforms occur in a bitbucket pipeline using XSL transform. This would run with free of charge run time (say once a month)
  Technical Design Specification
  Added link from TC page CURRENT ACTIVITIES box and from the original Resolution for DDI-CVs and RDF vocabularies

ATTENDEES: Wendy, Jon, Dan S., Flavio, Oliver

Reviewed current work plan through December 2022

• CV work - Oliver has been working on translation of output. Needs additional permissions at CESSDA.

• Identified focus for July-Dec 2022

• Identified activities requiring funding

May virtual meeting on COGS work:

Outstanding issues:

• Substitution Groups - Physical

• Complex choice - dealing with inheritance hierarchy, comprehensive modeling of current information

• Consistent modeling across model -

What can we do pre-meeting

• Listing of things that were skipped or changed

Documents recently discussed in TC draft google folder:

• WorkPlan for July Dec 2022

• Review Process for Official Technical Documents

• Integrated production framework

• Requirements and Process Management

• TC Work Plan 2021-22

No meeting

ATTENDEES: Wendy, Jon, Larry, Oliver, Dan S., Darren, Johan

CV documentation in codebook
CV group has requested that there be a link from within the schema to the specific CV
This means how we change for mid-release CV updates
DECISION: There will be a URI to the latest version that's what would go into the documentation
We can update only doing a minor update.
If documentation of new versions of Lifecycle are being generated we can easily integrated into the field level documentation
Is the SKOS a near thing in terms of exporting the SKOS we need to process. Oliver needs to look at it.

DOI's for Best Practices and High Level Documentation
Would prefer DOIs to on the HTML version of the Best Practices, or High Level Documents

Citing a standard you reference the specific home page URL
https://libraryguides.vu.edu.au/apa-referencing/7StandardsPatents cite standard number and publisher
Issue is do we want to maintain the versioning or take snapshots and deposit it ZENODO
Either we manage the versioning locally or we have to do the snapshot and copy to ZENODO

Step this back to the question of what is the need for a separate DOI for these official publications related to a product. We need a clearer sense of the use case for this as well as the broader need. Doing DOIs requires that we managing versioning more than an internal log of update changes within the document.

If we can't cite a document without a DOI then we need a DOI. File and management persistent identifier versioning situation

ACTION:
Raise a TC ticket and write down what we understand to be the problem
One point about persistence if we make changes in reorganization of our website we need to recognize the problem of breaking links.

Nail down what needs to be done for the documentation of the changes to the DDI Agency Registry both on the site and on the DDI Alliance page, then the announcement.
Mixture of a user guide and best practices document. Item for the Virtual TC - spend about an hour on the agenda. Coordinate with the SB URN working group

CANCEL Meeting on 10th March

ATTENDEES: Wendy, Jon, Larry, Oliver, Dan S., Darren, Johan, Carsten (2nd half for URN discussion)

Codebook:

Reviewed changes to agent types, conceptualTextTypes and concepts - spreadsheet should be implemented as listed

Enter these as a set for review, then enter documentation changes to 68 items. Note that 5 items need examples.

Mapping:

Johan - Project Shoc (social sciences and humanities open cloud) to create mapping to other standards and policies for conversion. There is a month and a half to the end of the project. Present to work to TC next week. Contact Flavio who will not be able to attend.

Recommendations part:

• The first choice - is that registering every URN that is created?

• Resolving DNS service are just for service resolution an end-point in a port (has been there for a long time)

• HTTP service end-points (3 built in web resolution, DDI which is the individual, and set with all related URNs)

• No differentiation between global and internal

Carsten:
What services should be operated to support URN resolution
What should be provided by the DDI Alliance
What should be provided by the member agencies

Diagram of URN resolution works (diagram in the SB WG paper)
Need to clarify what is already available
Once you know a URN belongs to DDI you can query DDI Agency Registry

If there is an established resolver used by agency
A service record is only providing an end-point (service and port)
There is nothing saying what type of service is being provided

You'd have to know the common name for the web/URN/verbatim service record and could use the DNS records to look up where those resolves are
Looking of service look-ups is not well supported across the board like by web services

How to come from DDI to the actual object
"N to T" arch id, handles, etc. at California Digital Libraries

This can do "N to T"
Have to know what the
No common resolution but there is a way of figuring out who owns the URN space and linking to the http://registry.ddialliance.org
There is a common system https://registry.ddialiance.org
You need to know the syntax of the DDI URN to proceed to go from ARPA to DDI (the registration in ARPA does that)
Don't want implementers to use that API to resolve everything, but to use that as a means of structuring local tool so that
A load limitation precludes having this available for handling all queries
So what's missing?
Identifier resolution and identifier end point. Individual identifiers can be resolved in terms of a consistent pattern within an agency or sub-agency. Individual variations from the pattern are not supported.
Published content owned by the DDI Alliance (primarily CV's and RDF vocabularies) - Darren and Oliver are completing that

General Discussion

• Access to the objects is in the court of the Agency

• In the recommendation (1) suggesting that the Alliance would register individual URNs (long paper)?
  If a DDI URN is found in the wild the DDI Alliance can direct you to where it is found
  That you can get a formal description of service

• Individual URNs were not considered an option

• There is a template approach (end-point at an agency level) they can provide additional breakdown at the initial endpoint (for example based on knowledge of internal structures)

• What the current service does not provide is validation of agency assigned IDs. It is their responsibility to state it is not a valid URN. Even with handle service you have to keep up-to-date with directions

• Use case not covered - is the Alliance taking on what is actually the agencies responsibility. The http://registry.ddialliance.org is simply a passthrough.

• Is it clear to the agency that this is what they'd have to do.

• What about a scenario where someone registers an agency and then deposits it in an archive.

• They'd keep their same agency ID and can define their own resolution. You could put the archive as the service for their agency. The resolution service would have to direct it to a differential agencies.

• Best Practice around this? If it is a distributed resolution environment and an agency has deposited in several different archives. They need a resolution system to address that.

• TC is open to further discussions with Carsten or with the URN Working Group as needed.

Background Information:

Carsten's email:
My understanding when taking on the lead of the URN TWG of the SB was that only the first step, Consumer<->DNS was working.
I understood our mission to clarify which of the other exchanges in the diagram needed to be implemented by either the Alliance or an Agency.
This is what our discussions on centralised vs decentralised solutions were mostly about.

From Dan's slides from EDDI (https://doi.org/10.5281/zenodo.5747652) I understood as follows:
Slide 8 says that the second step consumer<->registry for SRV records has been available for some time, but I don't know how it works and can't seem to find the documentation.
Slide 11-1&2 allowed me to do the third step consumer<->registry/Agency for the URL of a concrete URN, but Slide 11-3 did only allow me to retrieve the pattern via API, not the specific answer for a given URN.
The final step consumer<->repository is beyond the scope of the URN service.

Did I understand this correctly and where can I find a full documentation of steps two and three to follow end to end?
If I understand correctly, there is no mechanism foreseen to decide whether any URN is valid by the registry, but it would always return a URL that the repository then can't answer. (see screenshot)
I am happy to discuss this also tomorrow afternoon and please do forward this mail to those whom it concerns.

ATTENDEES: Wendy, Larry, Dan S., Oliver, Flavio, Darren

XKOS best practices:

• XKOS will have an invited technical review starting Feb 25, followed by a public review for comprehension lasting 6-8 weeks

• Publication will have a DOI (working with Jared on process)

• Wendy will draft a Guidelines for officially published documents (Best Practices, high level documentation) for submission to the Scientific Board for comment. This will be based on the XKOS process as the first document published under this process.

Codebook:

DDICODE-83 and general policy for use of conceptTextType

• Result of this is that ProcStat will not change as the only real use is internal processing information and those with processing systems have not requested this. There are no broader standards.

• Other items in document were approved.

The cutoff for deciding if an element should change from a simpleTextType to a conceptualtextType:

• If there is a standardized list it should be conceptualTextType

• If it is ONLY for processing (internal information) is probably overload

DDI Agency Registry:

• Link to presentation now available. Contact Barry about marketing announcement. Dan S. will work on working with Barry.

• Wendy will get these new capabilities into the URN Working Group paper to make sure they are complete

• Wendy will review directions in Administration section of DDI Agency Registry and write up an information page for the the Agency Registry page on the DDI site. Content will be sent to Dan for review when ready.

ATTENDEES: Wendy, Jon, Oliver, Flavio, Larry, Darren, Dan S.

Update on Codebook activities:

• Review of field level documentation has identified one or two additional needed changes (for consistency). These will be grouped and entered for review as a set.

• Updated work on high level documentation - addition of a "tree" representation of Codebook

Follow-up on TC review of URN Working Group paper:

• Timing of review by TC will be discussed in next URN WG meeting

• Wendy will update the URN paper with examples from Colectica Presentation
  DDI Registry presentation by Dan 10.5281/zenodo.5747653
  https://doi.org/10.5281/zenodo.5747652

CV LOD system

• SKOS recommendation

• There are also documents on the Atlassian site. These will be merged and TC members informed that it is ready for review and comments

• Versioning approach is being worked out and entered

• Transform of existing SKOS into a cleaner SKOS (Oliver)

• Publish transforms into bitbucket and then those will be grabbed and posted to the cloud

• 2nd week in march is slated for setup of the cloud platform

• New CV tools also have been proposed to be deployed in March

• Oliver has been in contact with CESSDA so that he is aware of where to do tweeks in URIs
  to deal with identifiers for codes

COGS

After the virtual meeting regarding COGS input we will be focusing on COGS output with a goal of identifying where similarity in formats for XML, RDF, JSON-LD, other formats is useful between products and where differences are needed to support specific applications. Planning needs to start for funding and scientific plan in conjunction with the groups developing products. For example, Pierre Antoin has a relationship with Scientific Board in terms of development of RDF JSON-LD for DDI-CDI and RDF in general.

This discussion is broader than TC and work has been done in each of the products with one or more formats that needs to be considered:

• DDI-L - XML (moving to full serialization), plans for RDF, JSON-LD, UML, etc.

• DDI-C - XML (hierarchical structure)

• XKOS - RDF (SKOS extension)

• SDTL - JSON

• Controlled Vocabularies - RDF (SKOS), XML (Codelist)

• DDI-CDI - XML (serialized), plans for RDF

Google groups and role of SRG list

• Update on Google Groups and continuation of the role of the SRG list to serve as a means of keeping interested people in touch with development discussions.

• Dan had question on specific feature of set up regarding prefixes. He will contact Jared directly

ATTENDEES: Wendy, Oliver, Larry, Flavio, Dan S., Barry

Codebook -

• Language by element repetition

• pull-requests merged

• Content and review of high level documentation - use review to edit draft

LOD Infrastructure

• Met with staff and what do with the URI's released by the CESSDA service

• CESSDA is doing a relaunch in March with minor changes which would cover all the system changes we needed for URI scheme

• Bitbucket repository will be set up soon so Oliver can pass over the scripts for production

• We may not need an extra transformation for target SKOS

• 2 major fixes are URI's to be really resolvable and not deliver empty descriptions for catagories if they don't occur in different languages

• URIs are generated with CESSDA output and we would be able to define the URIs produced

• Categories would have a consistent technical identifier rather than a clear text identifier

• SKOS to Codelist - Oliver will see if Darren has done any work on this, if not he will do this within the next week or so

Virtual Meeting

• April meeting - Dan and Jeremy busy the week of the 4th (6-11 unavailable)

• XMI/CDI:

  • Cardinality of both source and target

  • Relationship types - aggregations, compositions may not be relevant

  • Uses and dependencies (patterns)

  • Types are in the names now

  • Achim has done drafts of description of UML that can be used for this style of modeling

  • We would want options for getting this into COGS

  • Review of CDI XMI and of Achim's draft - Flavio and Larry

• Nail down dates as soon as possible so we can get some time frames for pre-meeting work

• The mix of data and metadata is important - JSON is a good vehicle

XKOS Best Practice Review

• Process (review of process itself)

• DOI application - multiple DOI's? ex. INSEE

Scientific Board WG on URN -

• Beneficial to make sure it addresses what is currently available and what is still needed

• It can resolve you to an individual URN - they can define a range of services

• Content hosting is not supported - URN resolution is solved

• TC should provide feedback on the draft of the document as this is the group that has been working on it for the past decade

• There are a lot of resolution options already available and we'd like to make sure they are focusing on new

ATTENDEES: Wendy, Dan S., Larry, Johan, Darren, Flavio

CV work
Discussion of low level specifics took place this week. Oliver expects transformations complete in a few weeks and Darren should complete some of his work. Something testable by end of February. CESSDA SKOS compliance probably not until end of year. We are not dependent upon CESSDA compliance to complete our work.

CODEBOOK

• Get a 2.6 out for review

• Work on weighting is not ready yet and should not hold up 2.6 given the need for DataVerse related. Better to get it correct than to put it in not quite finished. Backward compatibility rule. Don't hold but encourage movement on the sections that have been dropped (SDI and NCube).

• DDICODE-39 is another one to hold for further work. Note these issues in the release for review information for 2.6 and add a label post2.6 to these issues.

• Larry is looking into DDICODE-39 to see if there is a clear and unambiguous solution that could go into 2.6. If not it will be worked on for 2.7

• Set up high level document site for Codebook with Jon.

CDI is still finalizing documentation and will get to use dependent upon peoples work availability.

ADDITIONAL ISSUES we need to organize for discussion:

• XML is currently expressed in different styles between Codebook, Lifecycle, and CDI

  • what are the implications?

  • Are there overall rules that can be carried across the broader set of products and not just designed along a single one

• RDF usage is currently in SDTL and XKOS

  • These have their own RDF schemas and SDTL is integrated in PROV-1 and XKOS is integrated with SKOS as it is an extension

• Implementation languages for representations

Discuss when work plans need to be in for SB and how this coordinates with the financial requests.

NEXT WEEK NO TC due to Scientific Community Meeting

ATTENDEES: Wendy, Larry, Oliver, Dan S., Darren, Flavio, Johan

Requirements and Process Management - Integrated production framework and platform
https://docs.google.com/document/d/1TEy2zdxfARDgkOABb6kzuAmHK2oBJ0f5CmxL9F9pEbQ/edit#heading=h.avtxoxjt5vfy

• Good idea to have a sense of an integrated platform

• Infrastructure documentation - designing a standard SDLT

• We need to have a person who will oversee the design and implementation of the infrastructure

• Inventory of the existing buckets

• There shouldn't be any piece of production process or work we do that isn't in a public space

• Shouldn't rely on code or procedures that are in someone's head rather than in some published location

• We don't want systems reliant on an individual

• Availability of platform, both software and hardware, (needing to run on individual computers and requiring set up - example documentation set up for field level documentation)

• Initial discovery exercise - spreadsheet Wendy set up

• Look additionally at options

• If we need additional cloud space we will need to request beginning in the new fiscal year

• FFair(?) is set up to rebuild documentation each time a change is made

• Identify what is used by each product

• When you use free tools is everything exportable? yes, it just runs a rebuild script in an conformable environment with multiple platforms
  https://ci.appveyor.com/project/ddibot/ddimodel

• Darren will take a first stab

• CDI
  Jan 19 would be earliest but they are still working on the documentation so will probably be a bit later

Codebook
Will be scheduled for next week to see finalization for review

CV work

• Progress has been made and Oliver and Darren will meet next week

• Versioning at CESSDA is getting clearer (meeting on Monday)

• Profile of SKOS structure

https://ddi-alliance.atlassian.net/wiki/spaces/DDI4/pages/2766569473/Technical+Design+Specification#Metadata-Profile-(CVS%3C%3ERDF%3C%3EDDI-L)

Requirements and Implementation Management
Needs additional work - need to determine how to go about this

DDI Agency Registry update work
Acknowledged that Colectica has completed the work on the DDI Agency Registry update and presented on the work at EDDI. Announcement will be made when video of the presentation is published so that it can be referenced in the announcement.

ATTENDEES: Wendy, Jon, Dan S., Oliver, Larry, Barry

Review of Work Plan:

XKOS:

• Talk to Franck and settle on a process

• Light way would be to do it to the list - set up a comments process

• Comments period for Best Practices

• Meet XKOS needs and outline a basic approach to formal Best Practice papers

COGS Virtual meeting

• Jon will drive meeting

• Correct the input issues we know about

• Focus on input accuracy, support for output needs in XMI, input of XMI as well as current XML content, sorting through entry issues and complex nesting

• Easter is a problem week before or after 17th April (Oliver is available)

• Oliver may have some problems with the week of April 25

• Google docs for document working

• BitBucket features - due dates comments

• Slack possible but may not work for all

• Task and process assignments

• Who we want and what we want to get done, set up agenda early so we can nail it down early

Mapping Work:

• Mapping and comparison is progressing and is also being discussed in the Scientific Board - we need to coordinate

• Caution: we need to be clear on what is the mapping for?

  • Conceptual down to transformation

  • Managing ownership

Additional Notes:

, Dan, Oliver, Christophe, Jared, Jeremy

1. AWS and Drupal site interaction and support
   Recap: some sections will move to AWS (specifications) remaining will be moved to wordpress
   Is it wise to split this down the middle and move to 2 sites. This might be difficult to split correctly and to maintain in the future.

--Is it possible to move everything to AWS? How complex might this be to maintain and supervise
--Need to move away from Drupel as it is not supported by ICPSR and want to move it to a vendor
--Fee for moving it over and an annual maintenance amount ($3-4,000 a year)
--Urgency because support for their version of Drupal had gone away, did get year's reprieve
--In the mean time the AWS option arose and discussion of how
--Lots of static content that would just be moved
--Some functinality - bibliography that is maintained in an ongoing basis; posting news etc.
--Happy if it did not got to vendor
--Can we support on-going posting and maintenance
--If not, we can still go to WordPress keeping basic content there and other on AWS

Current view of what goes to AWS:
(1) Content under the following virtual URLs will be moved to the new UMich AWS instance
 https://ddialliance.org/Specification/* - static content
 https://ddialliance.org/products/* - static content
 https://ddialliance.org/learn/resources - single static web page
 https://ddialliance.org/learn/resources/ddi-profiles - single static web page
 https://ddialliance.org/resources/tools - single static web page with filter
o Also move relevant referenced items under https://ddialliance.org/tool/*
 https://ddialliance.org/learn/markup-examples-by-version?field_ddi_product_tid=All
o Also move relevant referenced items under https://ddialliance.org/sites/default/files/*
 At the root level under http://ddialliance.org .
o https://ddialliance.org/Tables.dtd - downloadable document

--Clean up even before vendor
--Hidden stuff would not move and Darren's list may include that

--Splitting may have unintended consiquenses such as loss of database functions (publications, examples, etc.)
--Do we need all the functionality that is there (examples etc.) - if we add in with the vendor

--Would the options we are looking support these

--Issue about having to shadow the brand and sync it between 2 sites and with a 3rd party vendor

--What potential costs there would be with backup etc. - there is still need for web support (not completely volunteer)

--It seems to me that we are missing a written target sitemap (whether we have one or two platforms) that would guide the specifics of the migration.

--Could we take a few weeks out to design what we want the site to look like
Include Training Group
We have a current website that needs a spring clean but we have no idea what it should look like
Can we take 4 weeks to say this is where we want to end up (google doc) - we don't have much written down on this at the moment
Drupal to WordPress is somewhat automated there is also a lot of sorting and hand work to make the move.

https://docs.google.com/spreadsheets/d/1y54OKTmJeSV3YwxR6RBa7yJyHvD8P1drjPmG4C2vZoM/edit#gid=517461823
Lists out the sections and owner groups
Detail gets complex in the products sections
Resources has some sections
What exactly do we have - can we flatten it out?
ACTION:
We should look at what we want rather than just what we have
Create small group to look at this: Jared(co), Jon, Darren (co), Wendy

2. CDI technical review kickoff
   Flavio looked at it
   No draft of production process review issues - get from Arofan (send him a draft of request and list of people we've identified)
   Automation is still lacking so how is that evaluated
   We did write to them some time ago regarding - dig out letter we sent to them and verify all the parts are there
   Asking more information on production process itself; make sure its available
   Jon will look next Friday

3. Funding requests
   see google sheet: https://docs.google.com/document/d/111V1om6vbaJQCgxYI_znA03xDWW9HwW0iMGHxNVVnHU/edit?usp=sharing

4. Workplan for coming year
   Review workplan for tweaks

5. Updates on feedback for DDI LIFECYCLE V.4.0 BETA
   No only had one piece of feedback from Romain
   BETA 2 would be good to do when we have the ordered flags