DataPipeline - Examples

In a study where the study is a wave in a StudySeries, we do a single traversal of the Generic Longitudinal Business Process Model. From one wave to the next the traversal may be different. Each traversal is described in a DataPipeline. Extract Transform and Load (ETL) platforms support data pipelines to move data between systems. Using an ETL platform, data engineers create data pipelines to orchestrate the movement, transformation, validation, and loading of data, from source to final destination. The DataPipeline describes this “orchestration”. A prospective DataPipeline gives guidance to data engineers. It is a design pattern. A retrospective DataPipeline documents an ETL data pipeline after the fact.