Job request: 9246
- Organisation:
- University of Bristol
- Workspace:
- waning-ve-2dose-1year-main
- ID:
- 7krm6lqgcqlr3qv2
This page shows the technical details of what happened when the authorised researcher Elsie Horne requested one or more actions to be run against real patient data within a secure environment.
By cross-referencing the list of jobs with the pipeline section below, you can infer what security level the outputs were written to.
The output security levels are:
-
highly_sensitive
- Researchers can never directly view these outputs
- Researchers can only request code is run against them
-
moderately_sensitive
- Can be viewed by an approved researcher by logging into a highly secure environment
- These are the only outputs that can be requested for public release via a controlled output review service.
Jobs
-
- Job identifier:
-
gcvips7f64veoae7
-
- Job identifier:
-
ir2q6rgmn5dx2efp
-
- Job identifier:
-
g4guys65qrfoozcu
-
- Job identifier:
-
zwedki3vnmn5p7tm
-
- Job identifier:
-
hm77pfpjooye276i - Error:
- nonzero_exit: Job exited with error code 1
-
- Job identifier:
-
w7xdfo3bfqdr3wmc - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
m2cs5nlwbiekyrp5 - Error:
- dependency_failed: Not starting as dependency failed
Pipeline
Show project.yaml
version: '3.0'
expectations:
population_size: 100000
actions:
## # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
## DO NOT EDIT project.yaml
## This file is created by create_project.R
## Edit and run create_project.R to update the project.yaml
## # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
## ####################################
## study definition
## ####################################
## generate dummy data for study_definition
dummy_data_vax:
run: r:latest analysis/dummy_data_vax.R
outputs:
moderately_sensitive:
dummy_data: analysis/dummy_data_vax.feather
## study definition
generate_study_population:
run: cohortextractor:latest generate_cohort --study-definition study_definition_vax
--output-format feather
dummy_data_file: analysis/dummy_data_vax.feather
needs:
- dummy_data_vax
outputs:
highly_sensitive:
cohort: output/input_vax.feather
## ####################################
## preprocessing
## ####################################
## process data from study_definition
data_input_process:
run: r:latest analysis/preprocess/data_input_process.R
needs:
- dummy_data_vax
- generate_study_population
outputs:
highly_sensitive:
data_wide_vax_dates: output/data/data_wide_vax_dates.rds
data_processed: output/data/data_processed.rds
moderately_sensitive:
data_properties: output/tables/data_*_tabulate.txt
## apply eligiblity criteria from boxes a and b
data_eligible_ab:
run: r:latest analysis/preprocess/data_eligible_ab.R
needs:
- data_input_process
outputs:
highly_sensitive:
data_eligible_a: output/data/data_eligible_a.rds
data_eligible_b: output/data/data_eligible_b.rds
moderately_sensitive:
eligibility_count_ab: output/tables/eligibility_count_ab.csv
group_age_ranges: output/lib/group_age_ranges.csv
## ####################################
## second_vax_period
## ####################################
## identify second vaccination time periods
## create dataset for identifying second vaccination time periods
data_2nd_vax_dates:
run: r:latest analysis/second_vax_period/data_2nd_vax_dates.R
needs:
- data_input_process
- data_eligible_ab
outputs:
highly_sensitive:
data_vax_plot: output/second_vax_period/data/data_vax_plot.rds
second_vax_period_dates_rds: output/second_vax_period/data/second_vax_period_dates.rds
moderately_sensitive:
second_vax_period_dates_txt: output/second_vax_period/tables/second_vax_period_dates.txt
## plot second vaccination time periods
plot_2nd_vax_dates:
run: r:latest analysis/second_vax_period/plot_2nd_vax_dates.R
needs:
- data_eligible_ab
- data_2nd_vax_dates
outputs:
moderately_sensitive:
plots_by_region: output/second_vax_period/images/plot_by_region_*.png
plots_by_region_data: output/second_vax_period/images/plot_by_region_*.txt
## apply eligiblity criteria from boxes c, d and e
data_eligible_cde:
run: r:latest analysis/preprocess/data_eligible_cde.R
needs:
- data_input_process
- data_eligible_ab
- data_2nd_vax_dates
outputs:
highly_sensitive:
data_eligible_e_vax: output/data/data_eligible_e_vax.rds
data_eligible_e_unvax: output/data/data_eligible_e_unvax.rds
data_eligible_e: output/data/data_eligible_e.csv
moderately_sensitive:
eligibility_count_cde: output/tables/eligibility_count_cde.csv
## ####################################
## study definition covs
## ####################################
generate_covs_data:
run: cohortextractor:latest generate_cohort --study-definition study_definition_covs
--output-format feather
needs:
- data_eligible_cde
outputs:
highly_sensitive:
cohort: output/input_covs.feather
## ####################################
## process covariates data
## ####################################
## (includes anytest_date)
data_covariates_process:
run: r:latest analysis/preprocess/data_covariates_process.R
needs:
- data_input_process
- data_eligible_cde
- generate_covs_data
outputs:
highly_sensitive:
data_covariates: output/data/data_all.rds
moderately_sensitive:
data_min_max_fu_csv: output/lib/data_min_max_fu.csv
## ####################################
## subsequent vaccination
## ####################################
## plot cumulative incidence of subsequent vaccination
plot_cumulative_incidence:
run: r:latest analysis/subsequent_vax/plot_cumulative_incidence.R
needs:
- data_covariates_process
outputs:
moderately_sensitive:
ci_vax: output/subsequent_vax/images/ci_vax.png
survtable: output/subsequent_vax/tables/survtable_redacted.csv
## ####################################
## table 1 for report
## ####################################
## create table 1 for all and for each subgroup
table1:
run: r:latest analysis/report/table1.R
needs:
- data_eligible_ab
- data_eligible_cde
- data_covariates_process
outputs:
moderately_sensitive:
eligibility_count_p1: output/tables/eligibility_count_p1.csv
eligibility_count_all: output/tables/eligibility_count_all.csv
table_csv: output/report/tables/table1_*_REDACTED.csv
table_html: output/report/tables/table1_*_REDACTED.html
## ####################################
## process time to event data
## ####################################
## process tte data
data_tte_process_BNT162b2:
run: r:latest analysis/comparisons/data_tte_process.R BNT162b2
needs:
- data_covariates_process
outputs:
highly_sensitive:
data_tte_brand_outcome: output/tte/data/data_tte_BNT162b2*.rds
moderately_sensitive:
event_counts_csv: output/tte/data/event_counts_BNT162b2.csv
event_counts_txt: output/tte/tables/event_counts_BNT162b2.txt
data_tte_process_ChAdOx1:
run: r:latest analysis/comparisons/data_tte_process.R ChAdOx1
needs:
- data_covariates_process
outputs:
highly_sensitive:
data_tte_brand_outcome: output/tte/data/data_tte_ChAdOx1*.rds
moderately_sensitive:
event_counts_csv: output/tte/data/event_counts_ChAdOx1.csv
event_counts_txt: output/tte/tables/event_counts_ChAdOx1.txt
data_tte_process_both:
run: r:latest analysis/comparisons/data_tte_process.R both
needs:
- data_covariates_process
outputs:
highly_sensitive:
data_tte_brand_outcome: output/tte/data/data_tte_both*.rds
moderately_sensitive:
event_counts_csv: output/tte/data/event_counts_both.csv
event_counts_txt: output/tte/tables/event_counts_both.txt
## check distribution of follow-up time in relation to variant dates
check_fu:
run: r:latest analysis/comparisons/check_fu.R
needs:
- data_covariates_process
outputs:
moderately_sensitive:
check_fu_plot: output/tte/images/check_fu_*.png
check_fu_plot_data: output/tte/images/check_fu_*.txt
Timeline
-
Created:
-
Started:
-
Finished:
-
Runtime: 05:51:47
These timestamps are generated and stored using the UTC timezone on the TPP backend.
Job request
- Status
-
Failed
- Backend
- TPP
- Workspace
- waning-ve-2dose-1year-main
- Requested by
- Elsie Horne
- Branch
- main
- Force run dependencies
- No
- Git commit hash
- 4f874f7
- Requested actions
-
-
dummy_data_vax -
generate_study_population -
data_input_process -
data_eligible_ab -
data_2nd_vax_dates -
plot_2nd_vax_dates -
data_eligible_cde
-
Code comparison
Compare the code used in this job request