Job request: 4787
- Organisation:
- University of Bristol
- Workspace:
- covid-ve-change-over-time--process-data-for-models
- ID:
- ydjerqpdymy7lhvk
This page shows the technical details of what happened when the authorised researcher Elsie Horne requested one or more actions to be run against real patient data within a secure environment.
By cross-referencing the list of jobs with the pipeline section below, you can infer what security level the outputs were written to.
The output security levels are:
-
highly_sensitive
- Researchers can never directly view these outputs
- Researchers can only request code is run against them
-
moderately_sensitive
- Can be viewed by an approved researcher by logging into a highly secure environment
- These are the only outputs that can be requested for public release via a controlled output review service.
Jobs
-
- Job identifier:
-
rbplfwbmgvvp7bcq
-
- Job identifier:
-
3g2l24g4znuaggm7
-
- Job identifier:
-
wg2bbd7bm7f4fn5c - Error:
- nonzero_exit: Job exited with an error code
-
- Job identifier:
-
dgwewos62s2p2hfc - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
7udra7noqogzh5wm - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
ettqlr37cowpruvd - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
nuogbxsz2zvfw6gk - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
hemebdq2pl3gcawg - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
ssqiusio3kj3id6t - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
mk35em7kddn24kky - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
f3dif44bb6vgljau - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
d3f3fwa5dgsh7uqo - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
z3bsfy62djzv7aeu - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
5gblsfbwccjgcfxh - Error:
- dependency_failed: Not starting as dependency failed
Pipeline
Show project.yaml
version: '3.0'
expectations:
population_size: 100000
actions:
## ####################################
## preliminaries
## ####################################
design:
run: r:latest analysis/design.R
outputs:
moderately_sensitive:
study_dates_json: output/lib/study_parameters.json
study_dates_rds: output/lib/study_parameters.rds
jcvi_groups: output/lib/jcvi_groups.csv
elig_dates: output/lib/elig_dates.csv
regions: output/lib/regions.csv
model_varlist: output/lib/model_varlist.rds
## ####################################
## study definition
## ####################################
## generate dummy data for study_definition
dummy_data:
run: r:latest analysis/dummy_data.R
needs:
- design
outputs:
moderately_sensitive:
dummy_data: analysis/dummy_data.feather
## study definition
generate_study_population:
run: cohortextractor:latest generate_cohort --study-definition study_definition
--output-format feather
dummy_data_file: analysis/dummy_data.feather
needs:
- design
- dummy_data
outputs:
highly_sensitive:
cohort: output/input.feather
## ####################################
## preprocessing
## ####################################
## process data from study_definition
data_input_process:
run: r:latest analysis/preprocess/data_input_process.R
needs:
- design
- dummy_data
- generate_study_population
outputs:
highly_sensitive:
data_covs: output/data/data_covs.rds
data_vax_dates: output/data/data_*_vax_dates.rds
moderately_sensitive:
data_properties: output/tables/data_processed_tabulate.txt
## process recurring variables as long data
data_long_process:
run: r:latest analysis/preprocess/data_long_process.R
needs:
- design
- data_input_process
outputs:
highly_sensitive:
data_long_dates: output/data/data_long_*_dates.rds
## apply eligiblity criteria from boxes a and b
data_eligible_ab:
run: r:latest analysis/preprocess/data_eligible_ab.R
needs:
- design
- data_input_process
outputs:
highly_sensitive:
data_eligible_a: output/data/data_eligible_a.rds
data_eligible_b: output/data/data_eligible_b.rds
moderately_sensitive:
eligibility_count: output/lib/eligibility_count_ab.csv
group_age_ranges: output/lib/group_age_ranges.csv
## ####################################
## second_vax_period
## ####################################
## identify second vaccination time periods
## create dataset for identifying second vaccination time periods
data_2nd_vax_dates:
run: r:latest analysis/second_vax_period/data_2nd_vax_dates.R
needs:
- design
- data_input_process
- data_eligible_ab
outputs:
highly_sensitive:
data_vax_plot: output/second_vax_period/data/data_vax_plot.rds
second_vax_period_dates_rds: output/lib/second_vax_period_dates.rds
moderately_sensitive:
second_vax_period_dates_csv: output/lib/second_vax_period_dates.csv
## plot second vaccination time periods
plot_2nd_vax_dates:
run: r:latest analysis/second_vax_period/plot_2nd_vax_dates.R
needs:
- design
- data_eligible_ab
- data_2nd_vax_dates
outputs:
moderately_sensitive:
plots_by_region: output/second_vax_period/images/plot_by_region_*.png
## apply eligiblity criteria from boxes c and d
data_eligible_cd:
run: r:latest analysis/second_vax_period/data_eligible_cd.R
needs:
- design
- data_input_process
- data_eligible_ab
- data_2nd_vax_dates
outputs:
highly_sensitive:
data_eligible_c: output/data/data_eligible_c.rds
data_eligible_d: output/data/data_eligible_d.rds
## ####################################
## comparisons
## ####################################
## process data, apply model and generate report for JCVI group 02
## process covariates data
data_comparisons_process_02:
run: r:latest analysis/comparisons/data_comparisons_process.R 02
needs:
- design
- data_input_process
- data_long_process
- data_2nd_vax_dates
- data_eligible_cd
outputs:
highly_sensitive:
data_comparisons: output/jcvi_group_02/data/data_comparisons.rds
## process outcomes data
data_outcomes_process_02:
run: r:latest analysis/comparisons/data_outcomes_process.R 02
needs:
- design
- data_input_process
- data_long_process
- data_comparisons_process_02
outputs:
highly_sensitive:
data_outcomes: output/jcvi_group_02/data/data_outcomes.rds
## check gap between outcomes for combining
check_combine_outcomes_02:
run: r:latest analysis/comparisons/check_combine_outcomes.R 02
needs:
- data_outcomes_process_02
outputs:
highly_sensitive:
data_check_combine_outcomes: output/jcvi_group_02/data/check_combine_outcomes.rds
moderately_sensitive:
plot_check_combine_outcomes: output/jcvi_group_02/images/check_combine_outcomes.png
table_check_combine_outcomes: output/jcvi_group_02/tables/check_combine_outcomes.csv
## outcome = postest
## process tte data for postest
data_tte_process_02_postest:
run: r:latest analysis/comparisons/data_tte_process.R 02 postest
needs:
- data_comparisons_process_02
- data_outcomes_process_02
outputs:
highly_sensitive:
data_tte_brand_outcome: output/jcvi_group_02/data/data_tte_*_postest.rds
## apply cox model for postest
apply_model_cox_02_postest:
run: r:latest analysis/comparisons/apply_model_cox.R 02 postest
needs:
- design
- data_comparisons_process_02
- data_tte_process_02_postest
outputs:
highly_sensitive:
modelnumber: output/jcvi_group_02/models/*_postest_model*.rds
model_tidy_rds: output/jcvi_group_02/models/*_postest_modelcox_tidy.rds
moderately_sensitive:
model_glance: output/jcvi_group_02/models/*_postest_modelcox_glance.csv
model_tidy_csv: output/jcvi_group_02/models/*_postest_modelcox_tidy.csv
Timeline
-
Created:
-
Started:
-
Finished:
-
Runtime: 05:16:11
These timestamps are generated and stored using the UTC timezone on the TPP backend.
Job request
- Status
-
Failed
- Backend
- TPP
- Requested by
- Elsie Horne
- Branch
- process-data-for-models
- Force run dependencies
- Yes
- Git commit hash
- c145931
- Requested actions
-
-
run_all
-
Code comparison
Compare the code used in this job request