Methodology
Note: This User story is currently being re-scoped in advance of preparations for the Dagstuhl 2014 sprint.
Suggested purpose: To describe of the study design specifications underlying the conduct of a research/business project. This is distinct from the execution of the design - which is detailed elsewhere (in Data Collection, Fieldwork, Data Processing and Data Analysis). However there should be a direct relationship able to be specified between the element of the methodological design and it's execution in the conduct of the study.
The description of the study methodology is largely reflected in the GLBPM, specifically Section 2 (Design). Preliminary work on describing the metadata requirements of this model are detailed in a Google Spreadsheet (thanks to Barry Radler and Jon Johnson). The steps in this Section to be documented (and/or adapted) are as follows:
- Identify Sources
- Design Sampling Methods
- Design Collection Instruments
- Specify Data Elements
- Specify Processing and Cleaning Methods
- Specify Analysis Plan
- Organise Research Team
- Design Infrastructure
There are several related current or proposed use cases that might be incorporated under this User Story - either as use cases of design or execution. These include
| Design | Execution |
|---|---|
| Study Inception | Data collection/Fieldwork |
| Data Management Planning | Data Processing Pipeline |
| Survey Development | Analysis and dissemination |
| Protocol specification | |
| Hypothesis generation | |
| Sampling, Weighting, ... |
PREVIOUS VERSION:
Full purpose: To provide an understanding of the study design addressing a business case (e.g., research question); to provide information which would allow, for example, (a) a researcher to field or repeat a study, or (b) a statistician to recreate a statistic or finding, or (c) a data user to understand the methods used to collect the data..
Full description: A series of design specifications (see topic areas below), not necessarily in a predefined order, that as a whole, constitute the methodology. Also known as the study design. Such design specifications probably include the information in the proposed view Protocol Specification by Jay (Re: Protocol Specification)
Discussion (Steve, Dan, Simon, Barry, Nicole - Toronto May 2014)
Proposals
Topics in Scope | Definition | Usage/example |
| Study Design | The "how"; specifications for how the research will be carried out and how the data will be obtained, collected, or captured. | |
| Business Case | The "why"; statement of why the research will be conducted and the research question(s) that will be addressed. | |
| Sample Frame | Identification of population upon which the survey will be conducted. | |
| Sampling | Specifying how the sample will be drawn from the frame. | See source link on sampling.
A sampling plan for any statistical activity for Dan Gillman |
| Estimation | Consideration of costs, size, bias, and confidence in the design of the variance estimates. | |
| Weighting | Representative power of each case in a data set and how they will be determined. | See source link on weighting. |
Data editing/transformations | Editing/cleaning or coding during or in context of data collection or administration of instrument (contingent upon mode). | Examples include looking for out-of-bound values, data inconsistencies, reporting errors, recoded variables (variables created from existing values from other variables), or coding open-ended text into categories. |
Imputation/non-response adjustment | Application or not to apply coding or a statistical technique to a dataset to populate missing values. |
|
Questionnaire development (logical instrument), mode effects | Creation of question types, response categories and options, skip logic, testing, mode of administration | Design of the data collection instruments and design of methodological tests (focus groups, cognitive interviews) to assist with instrument development. See source link on questionnaire design. |
Incorporating classifications | Specification for how the units will be subdivided in samples, tables or analyses. | Selection of strata, construction of data tables |
Analytic plan | Identification of analytic approach and specific statistical tests and manipulations. |
|
Questions
- We need to decide those views that are critical to describing methodology but that are out of scope, such as Logical Instrument, Classifications, etc.
Sources
GLPBM user story: https://docs.google.com/spreadsheet/ccc?key=0AmjuyyBzwvcodE9Qam5HbFhJRTV6U19rbnY5WUVZZEE&usp=drive_web#gid=1
Methodology paper on weighting: http://www1.unece.org/stat/platform/download/attachments/99486553/Weighting.zip?version=2&modificationDate=1399423569612&api=v2
Methodology paper on sampling: http://www1.unece.org/stat/platform/download/attachments/99486553/Sampling.zip?version=1&modificationDate=1399422349725&api=v2
Methodology paper on questionnaire design: (link TBD)
Pictures of Methodology brain storm (source materials yet to be sussed out):
Figure 1: Relationships? among methodology elements.
.JPG?version=1&modificationDate=1414049117399&cacheVersion=1&api=v2&width=333&height=250)
.JPG?api=v2){kind=link}