Job request: 1087
- Organisation:
- Bennett Institute
- Workspace:
- covid-vaccineeffectiveness-research_doses
- ID:
- pmtrih52fdd4pzjg
This page shows the technical details of what happened when the authorised researcher Will Hulme requested one or more actions to be run against real patient data within a secure environment.
By cross-referencing the list of jobs with the pipeline section below, you can infer what security level the outputs were written to.
The output security levels are:
-
highly_sensitive
- Researchers can never directly view these outputs
- Researchers can only request code is run against them
-
moderately_sensitive
- Can be viewed by an approved researcher by logging into a highly secure environment
- These are the only outputs that can be requested for public release via a controlled output review service.
Jobs
-
- Job identifier:
-
cay3qlerhp2cfum2
-
- Job identifier:
-
brfeyw3ryqk7nr2x
-
- Job identifier:
-
5nakh65oymsjv6sz
-
- Job identifier:
-
4pdsud5gbtecnbay
-
- Job identifier:
-
usf7nlvmpudq6wof - Error:
- nonzero_exit: Job exited with an error code
-
- Job identifier:
-
wy275lye4rvhl4sa - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
wbello7rkzbqd42x - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
6mzmjufgwbvvn6ak - Error:
- dependency_failed: Not starting as dependency failed
Pipeline
Show project.yaml
version: '3.0'
expectations:
population_size: 100000
actions:
extract_all:
run: cohortextractor:latest generate_cohort --study-definition study_definition_all
outputs:
highly_sensitive:
cohort: output/input_all.csv
data_process_all:
run: r:latest analysis/R/data_process_all.R
needs: [extract_all]
outputs:
highly_sensitive:
data1: output/data/data_all.rds
data2: output/data/data_long_vax_dates.rds
data3: output/data/data_long_admission_dates.rds
data_properties_all:
run: r:latest analysis/R/data_properties.R output/data/data_all.rds output/data_properties
needs: [data_process_all]
outputs:
moderately_sensitive:
datasummary: output/data_properties/data_all*.txt
data_define_cohorts:
run: r:latest analysis/R/data_define_cohorts.R
needs: [data_process_all]
outputs:
highly_sensitive:
data: output/data/data_cohorts.rds
moderately_sensitive:
metadata: output/data/metadata_cohorts.rds
## VE models in over 80s
data_stset_over80s:
run: r:latest analysis/R/data_stset.R over80s
needs: [data_define_cohorts, data_process_all]
outputs:
highly_sensitive:
data: output/over80s/data/data_*.rds
data_properties_over80s:
run: r:latest analysis/R/data_properties.R output/over80s/postest/data/data_wide.rds output/over80s/postest/data_properties
needs: [data_stset_over80s]
outputs:
moderately_sensitive:
datasummary: output/cohort/outcome/data_properties/data_wide*.txt
descr_eventsperday_over80s:
run: r:latest analysis/R/plot_outcomes_per_day.R over80s postest
needs: [data_define_cohorts, data_stset_over80s]
outputs:
moderately_sensitive:
datasummary: output/over80s/postest/descr/events*.svg
# descr_table1_over80s:
# run: r:latest analysis/R/table1.R over80s postest
# needs: [data_define_cohorts, data_stset_over80s]
# outputs:
# moderately_sensitive:
# datasummary: output/over80s/postest/descr/table*.svg
models_msm_over80s:
run: r:latest analysis/R/models_msms.R over80s postest
needs: [data_define_cohorts, data_stset_over80s]
outputs:
highly_sensitive:
models: output/over80s/postest/modes/model*.rds
data: output/over80s/postest/models/data_weights.rds
moderately_sensitive:
weights: output/over80s/postest/weights.txt
weightsplot: output/over80s/postest/histogram_weights.svg
reportmodels_msm_over80s:
run: r:latest analysis/R/reportmodels_msms.R over80s
needs: [data_define_cohorts, models_msm_over80s]
outputs:
moderately_sensitive:
forest: output/models/msm/over80s/forest*.svg
trend: output/models/msm/over80s/secular_trends*.svg
eventrate: output/models/msm/over80s/event_rate.svg
tables: output/models/msm/over80s/*.csv
# data_stset_under65s:
# run: r:latest analysis/R/data_stset.R under65s
# needs: [data_cohorts, data_process_all]
# outputs:
# highly_sensitive:
# data: output/modeldata/data_*under65s.rds
#
# data_properties_under65s:
# run: r:latest analysis/R/data_properties.R output/modeldata/data_wide_under65s.rds output/data_properties
# needs: [data_stset_under65s]
# outputs:
# moderately_sensitive:
# datasummary: output/data_properties/data_wide_under65s*.txt
#
# models_msm_under65s:
# run: r:latest analysis/R/models_msms.R under65s
# needs: [data_cohorts, data_stset_under65s]
# outputs:
# moderately_sensitive:
# weights: output/models/msm/under65s/weights.txt
# forest: output/models/msm/under65s/forest*.svg
# trend: output/models/msm/under65s/secular_trends*.svg
# tables: output/models/msm/under65s/*.csv
# models_cox_over80s:
# run: r:latest analysis/R/models_cox_over80s.R
# needs: [data_stset_over80s]
# outputs:
# moderately_sensitive:
# zph: output/models/cox/over80s/zph*.png
# forest: output/models/cox/over80s/forest*.svg
# tables: output/models/cox/over80s/*.csv
# run_ve:
# run: cohortextractor:latest --version
# needs: [extract_all, data_process_all, data_properties_all, data_tte_over80s, models_cox_over80s]
# outputs:
# moderately_sensitive:
# whatever: output/.gitignore
Timeline
-
Created:
-
Started:
-
Finished:
-
Runtime: 02:50:42
These timestamps are generated and stored using the UTC timezone on the TPP backend.
Job request
- Status
-
Failed
- Backend
- TPP
- Requested by
- Will Hulme
- Branch
- doses
- Force run dependencies
- No
- Git commit hash
- 8d8cbd2
- Requested actions
-
-
extract_all -
data_process_all -
data_properties_all -
data_define_cohorts -
data_stset_over80s -
data_properties_over80s -
descr_eventsperday_over80s -
models_msm_over80s
-
Code comparison
Compare the code used in this job request