Job request: 20421
- Organisation:
- Bennett Institute
- Workspace:
- opioids-covid-research
- ID:
- vixzepinnwxnehvf
This page shows the technical details of what happened when the authorised researcher Andrea Schaffer requested one or more actions to be run against real patient data in the project, within a secure environment.
By cross-referencing the list of jobs with the
pipeline section below, you can infer what
security level
various outputs were written to. Researchers can never directly
view outputs marked as
highly_sensitive
;
they can only request that code runs against them. Outputs
marked as
moderately_sensitive
can be viewed by an approved researcher by logging into a highly
secure environment. Only outputs marked as
moderately_sensitive
can be requested for release to the public, via a controlled
output review service.
Jobs
-
- Job identifier:
-
mhy6nhjgzpabmcvx
-
- Job identifier:
-
hgp6afv567fdvplb
Pipeline
Show project.yaml
######################################
# This script defines the project pipeline -
# it specifies the execution orders for all the code in this
# repo using a series of actions.
######################################
version: '3.0'
expectations:
population_size: 10000
actions:
generate_dataset_table:
run: ehrql:v0 generate-dataset analysis/define_dataset_table.py
--output output/data/dataset_table.csv.gz
outputs:
highly_sensitive:
cohort: output/data/dataset_table.csv.gz
# Measures - prevalent and new prescribing - overall
measures_overall:
run: ehrql:v0 generate-measures analysis/measures_overall.py
--output output/measures/measures_overall.csv
--
--start-date "2018-01-01"
--intervals 54
outputs:
moderately_sensitive:
measure_csv: output/measures/measures_overall.csv
# Measures - prevalent prescribing - by demographic categories
measures_demo_prev:
run: ehrql:v0 generate-measures analysis/measures_demo_prev.py
--output output/measures/measures_demo_prev.csv
--
--start-date "2018-01-01"
--intervals 3
outputs:
moderately_sensitive:
measure_csv: output/measures/measures_demo_prev.csv
# Measures - new prescribing - by demographic categories
measures_demo_new:
run: ehrql:v0 generate-measures analysis/measures_demo_new.py
--output output/measures/measures_demo_new.csv
--
--start-date "2018-01-01"
--intervals 54
outputs:
moderately_sensitive:
measure_csv: output/measures/measures_demo_new.csv
# Measures - prevalent prescribing - by opioid type
measures_type:
run: ehrql:v0 generate-measures analysis/measures_type.py
--output output/measures/measures_type.csv
--
--start-date "2018-01-01"
--intervals 54
outputs:
moderately_sensitive:
measure_csv: output/measures/measures_type.csv
# Measures - prevalent and new prescribing - in people in care home
measures_carehome:
run: ehrql:v0 generate-measures analysis/measures_carehome.py
--output output/measures/measures_carehome.csv
--
--start-date "2018-01-01"
--intervals 54
outputs:
moderately_sensitive:
measure_csv: output/measures/measures_carehome.csv
# Measures - test
measures_test:
run: ehrql:v0 generate-measures analysis/measures_type_test.py
--output output/measures/measures_type_test.csv
--
--start-date "2019-01-01"
--intervals 24
outputs:
moderately_sensitive:
measure_csv: output/measures/measures_type_test.csv
## Process time series data - overall prescribing
process_ts_overall:
run: r:latest analysis/process/process_ts_overall.R
needs: [measures_overall]
outputs:
moderately_sensitive:
timeseries_csv: output/timeseries/ts_overall*.csv
## Process time series data - prescribing by demographics
process_ts_demo:
run: r:latest analysis/process/process_ts_demo.R
needs: [measures_demo_prev, measures_demo_new]
outputs:
moderately_sensitive:
timeseries_csv: output/timeseries/ts_demo*.csv
## Process time series data - prescribing by admin route
process_ts_type:
run: r:latest analysis/process/process_ts_type.R
needs: [measures_type]
outputs:
moderately_sensitive:
timeseries_csv: output/timeseries/ts_type*.csv
## Process time series data - prescribing to people in carehome
process_ts_carehome:
run: r:latest analysis/process/process_ts_carehome.R
needs: [measures_carehome]
outputs:
moderately_sensitive:
timeseries_csv: output/timeseries/ts_carehome*.csv
## Results table
table:
run: r:latest analysis/descriptive/table_stand.R
needs: [generate_dataset_table]
outputs:
moderately_sensitive:
table: output/tables/table_*.csv
# OLD COHORTEXTRACTOR CODE
# ## Cohort data
# generate_study_population_1:
# run: cohortextractor:latest generate_cohort
# --study-definition study_definition
# --index-date-range "2018-01-01 to 2018-12-01 by month"
# --output-dir=output
# --output-format=csv
# outputs:
# highly_sensitive:
# cohort: output/input_*.csv
# generate_study_population_2:
# run: cohortextractor:latest generate_cohort
# --study-definition study_definition
# --index-date-range "2019-01-01 to 2019-12-01 by month"
# --output-dir=output
# --output-format=csv
# outputs:
# highly_sensitive:
# cohort: output/input*.csv
# generate_study_population_3:
# run: cohortextractor:latest generate_cohort
# --study-definition study_definition
# --index-date-range "2020-01-01 to 2020-12-01 by month"
# --output-dir=output
# --output-format=csv
# outputs:
# highly_sensitive:
# cohort: output/inpu*.csv
# generate_study_population_4:
# run: cohortextractor:latest generate_cohort
# --study-definition study_definition
# --index-date-range "2021-01-01 to 2021-12-01 by month"
# --output-dir=output
# --output-format=csv
# outputs:
# highly_sensitive:
# cohort: output/inp*.csv
# generate_study_population_5:
# run: cohortextractor:latest generate_cohort
# --study-definition study_definition
# --index-date-range "2022-01-01 to 2022-03-01 by month"
# --output-dir=output
# --output-format=csv
# outputs:
# highly_sensitive:
# cohort: output/in*.csv
# ## Ethnicity
# generate_ethnicity_cohort:
# run: >
# cohortextractor:latest generate_cohort
# --study-definition study_definition_ethnicity
# outputs:
# highly_sensitive:
# cohort: output/input_ethnicity.csv
# # Data processing ----
# ## Add ethnicity
# join_cohorts:
# run: >
# cohort-joiner:v0.0.48
# --lhs output/input_*.csv
# --rhs output/input_ethnicity.csv
# --output-dir output/data
# needs: [generate_study_population_1, generate_study_population_2,
# generate_study_population_5, generate_study_population_3,
# generate_study_population_4, generate_ethnicity_cohort]
# outputs:
# highly_sensitive:
# cohort: output/data/input_*.csv
# ## Generate measures - full population
# generate_measures:
# run: >
# cohortextractor:latest generate_measures
# --study-definition study_definition
# --output-dir output/data
# needs: [join_cohorts]
# outputs:
# moderately_sensitive:
# measure_csv: output/data/measure_*.csv
Timeline
-
Created:
-
Started:
-
Finished:
-
Runtime: 01:44:37
These timestamps are generated and stored using the UTC timezone on the TPP backend.
Job information
- Status
-
Succeeded
- Backend
- TPP
- Workspace
- opioids-covid-research
- Requested by
- Andrea Schaffer
- Branch
- main
- Force run dependencies
- No
- Git commit hash
- c08d400
- Requested actions
-
-
measures_demo_prev
-
process_ts_demo
-
Code comparison
Compare the code used in this Job Request