Wednesday progress report
| View file | ||
|---|---|---|
|
From Darren Bell
The TLDR context of this is that we have built a prototype data-subsetting and linkage tool that uses DDI-CDI Data Description Package as its logical data model and is implemented on a cloud NoSQL database (Amazon DynamoDB) with React.js and a simple serverless architecture. We had intended in this prototype to use Smart Energy data as exemplar “long” data but in the end, did not have sufficient time to negotiate the necessary licences with the various governance boards. Instead, for our prototype, we have used a combination of social science and environmental “Wide” data and synthesised time diary “Long” data. The core goal of demonstrating wide and long real-time linkage and subsetting in a single platform are still well exemplified by this arrangement and moreover, we still intend to integrate Smart Meter data in the next couple of iterations of the tool
The attached Word document is written by one of our data enginners Tom Gilders and gives an overview of the “data product builder” (as we call it) with some detail about datasets, query patterns and use of CDI and Data Privacy Vocabulary (DPV). There are links and references as well within the document.
...