Job request: 4793
- Organisation:
- University of Bristol
- Workspace:
- covid-ve-change-over-time--process-data-for-models
- ID:
- sgewv3pixqgm4qjk
This page shows the technical details of what happened when authorised researcher Elsie Horne requested one or more actions to be run against real patient data in the project, within a secure environment.
By cross-referencing the indicated Requested Actions with the
Pipeline section below, you can infer what
security level
various outputs were written to. Outputs marked as
highly_sensitive
can never be viewed directly by a researcher; they can only
request that code runs against them. Outputs marked as
moderately_sensitive
can be viewed by an approved researcher by logging into a highly
secure environment. Only outputs marked as
moderately_sensitive
can be requested for release to the public, via a controlled
output review service.
Jobs
-
- Job identifier:
-
z234xornsg5my6we
-
- Job identifier:
-
a5zd2duwdtdkaja5
-
- Job identifier:
-
s4bmfnyqaerv32kp
-
- Job identifier:
-
od5gpvkht6hx23u5
-
- Job identifier:
-
i7246aebgse5cdgx
-
- Job identifier:
-
a4nv4t7lj2zshp5t
-
- Job identifier:
-
wu2luze7hncsfajk
-
- Job identifier:
-
6e2mdw7zee2jvymf
-
- Job identifier:
-
gwtc4lkufr75oimx
-
- Job identifier:
-
fx3xrflr4mjrzpat
-
- Job identifier:
-
jctxa67cgu2ly3lc
-
- Job identifier:
-
2cfhmjyqeici45w4
-
- Job identifier:
-
3msbeh5lxm7s2vjc
-
- Job identifier:
-
bx647bfilcys2xza
-
- Job identifier:
-
fewnvk7snkmxskv7
-
- Job identifier:
-
v5dcu4437fyjnguq
-
- Job identifier:
-
xvyn3h3axcjtmr2c
-
- Job identifier:
-
6ne2qyeselwvkzoq
-
- Job identifier:
-
kndr3yl4x75xsx3w
-
- Job identifier:
-
x6orth53j36eqfij
-
- Job identifier:
-
ut4f7sftznvdkkkc
-
- Job identifier:
-
esol7f5lhjoxzqdh
-
- Job identifier:
-
zzrhsductrzjcqqx
Pipeline
Show project.yaml
version: '3.0'
expectations:
population_size: 100000
actions:
## ####################################
## preliminaries
## ####################################
design:
run: r:latest analysis/design.R
outputs:
moderately_sensitive:
study_dates_json: output/lib/study_parameters.json
study_dates_rds: output/lib/study_parameters.rds
jcvi_groups: output/lib/jcvi_groups.csv
elig_dates: output/lib/elig_dates.csv
regions: output/lib/regions.csv
model_varlist: output/lib/model_varlist.rds
outcomes: output/lib/outcomes.rds
## ####################################
## study definition
## ####################################
## generate dummy data for study_definition
dummy_data:
run: r:latest analysis/dummy_data.R
needs:
- design
outputs:
moderately_sensitive:
dummy_data: analysis/dummy_data.feather
## study definition
generate_study_population:
run: cohortextractor:latest generate_cohort --study-definition study_definition
--output-format feather
dummy_data_file: analysis/dummy_data.feather
needs:
- design
- dummy_data
outputs:
highly_sensitive:
cohort: output/input.feather
## ####################################
## preprocessing
## ####################################
## process data from study_definition
data_input_process:
run: r:latest analysis/preprocess/data_input_process.R
needs:
- design
- dummy_data
- generate_study_population
outputs:
highly_sensitive:
data_covs: output/data/data_covs.rds
data_vax_dates: output/data/data_*_vax_dates.rds
moderately_sensitive:
data_properties: output/tables/data_processed_tabulate.txt
## process recurring variables as long data
data_long_process:
run: r:latest analysis/preprocess/data_long_process.R
needs:
- design
- data_input_process
outputs:
highly_sensitive:
data_long_dates: output/data/data_long_*_dates.rds
## apply eligiblity criteria from boxes a and b
data_eligible_ab:
run: r:latest analysis/preprocess/data_eligible_ab.R
needs:
- design
- data_input_process
outputs:
highly_sensitive:
data_eligible_a: output/data/data_eligible_a.rds
data_eligible_b: output/data/data_eligible_b.rds
moderately_sensitive:
eligibility_count: output/lib/eligibility_count_ab.csv
group_age_ranges: output/lib/group_age_ranges.csv
## ####################################
## second_vax_period
## ####################################
## identify second vaccination time periods
## create dataset for identifying second vaccination time periods
data_2nd_vax_dates:
run: r:latest analysis/second_vax_period/data_2nd_vax_dates.R
needs:
- design
- data_input_process
- data_eligible_ab
outputs:
highly_sensitive:
data_vax_plot: output/second_vax_period/data/data_vax_plot.rds
second_vax_period_dates_rds: output/lib/second_vax_period_dates.rds
moderately_sensitive:
second_vax_period_dates_csv: output/lib/second_vax_period_dates.csv
## plot second vaccination time periods
plot_2nd_vax_dates:
run: r:latest analysis/second_vax_period/plot_2nd_vax_dates.R
needs:
- design
- data_eligible_ab
- data_2nd_vax_dates
outputs:
moderately_sensitive:
plots_by_region: output/second_vax_period/images/plot_by_region_*.png
## apply eligiblity criteria from boxes c and d
data_eligible_cd:
run: r:latest analysis/second_vax_period/data_eligible_cd.R
needs:
- design
- data_input_process
- data_eligible_ab
- data_2nd_vax_dates
outputs:
highly_sensitive:
data_eligible_c: output/data/data_eligible_c.rds
data_eligible_d: output/data/data_eligible_d.rds
## ####################################
## comparisons
## ####################################
## process data, apply model and generate report for JCVI group 02
## process covariates data
data_comparisons_process_02:
run: r:latest analysis/comparisons/data_comparisons_process.R 02
needs:
- design
- data_input_process
- data_long_process
- data_2nd_vax_dates
- data_eligible_cd
outputs:
highly_sensitive:
data_comparisons: output/jcvi_group_02/data/data_comparisons.rds
## process outcomes data
data_outcomes_process_02:
run: r:latest analysis/comparisons/data_outcomes_process.R 02
needs:
- design
- data_input_process
- data_long_process
- data_comparisons_process_02
outputs:
highly_sensitive:
data_outcomes: output/jcvi_group_02/data/data_outcomes.rds
## check gap between outcomes for combining
check_combine_outcomes_02:
run: r:latest analysis/comparisons/check_combine_outcomes.R 02
needs:
- data_outcomes_process_02
outputs:
highly_sensitive:
data_check_combine_outcomes: output/jcvi_group_02/data/check_combine_outcomes.rds
moderately_sensitive:
plot_check_combine_outcomes: output/jcvi_group_02/images/check_combine_outcomes.png
table_check_combine_outcomes: output/jcvi_group_02/tables/check_combine_outcomes.csv
## outcome = postest
## process tte data for postest
data_tte_process_02_postest:
run: r:latest analysis/comparisons/data_tte_process.R 02 postest
needs:
- data_comparisons_process_02
- data_outcomes_process_02
outputs:
highly_sensitive:
data_tte_brand_outcome: output/jcvi_group_02/data/data_tte_*_postest.rds
moderately_sensitive:
table_incidence: output/jcvi_group_02/tables/*_postest_incidence.txt
## apply cox model for postest
apply_model_cox_02_postest:
run: r:latest analysis/comparisons/apply_model_cox.R 02 postest
needs:
- design
- data_comparisons_process_02
- data_tte_process_02_postest
outputs:
highly_sensitive:
modelnumber: output/jcvi_group_02/models/*_postest_model*.rds
model_tidy_rds: output/jcvi_group_02/models/*_postest_modelcox_tidy.rds
model_summary_rds: output/jcvi_group_02/models/*_postest_modelcox_summary.rds
moderately_sensitive:
model_glance: output/jcvi_group_02/models/*_postest_modelcox_glance.csv
model_tidy_csv: output/jcvi_group_02/models/*_postest_modelcox_tidy.csv
## outcome = covidadmitted
## process tte data for covidadmitted
data_tte_process_02_covidadmitted:
run: r:latest analysis/comparisons/data_tte_process.R 02 covidadmitted
needs:
- data_comparisons_process_02
- data_outcomes_process_02
outputs:
highly_sensitive:
data_tte_brand_outcome: output/jcvi_group_02/data/data_tte_*_covidadmitted.rds
moderately_sensitive:
table_incidence: output/jcvi_group_02/tables/*_covidadmitted_incidence.txt
## apply cox model for covidadmitted
apply_model_cox_02_covidadmitted:
run: r:latest analysis/comparisons/apply_model_cox.R 02 covidadmitted
needs:
- design
- data_comparisons_process_02
- data_tte_process_02_covidadmitted
outputs:
highly_sensitive:
modelnumber: output/jcvi_group_02/models/*_covidadmitted_model*.rds
model_tidy_rds: output/jcvi_group_02/models/*_covidadmitted_modelcox_tidy.rds
model_summary_rds: output/jcvi_group_02/models/*_covidadmitted_modelcox_summary.rds
moderately_sensitive:
model_glance: output/jcvi_group_02/models/*_covidadmitted_modelcox_glance.csv
model_tidy_csv: output/jcvi_group_02/models/*_covidadmitted_modelcox_tidy.csv
## outcome = coviddeath
## process tte data for coviddeath
data_tte_process_02_coviddeath:
run: r:latest analysis/comparisons/data_tte_process.R 02 coviddeath
needs:
- data_comparisons_process_02
- data_outcomes_process_02
outputs:
highly_sensitive:
data_tte_brand_outcome: output/jcvi_group_02/data/data_tte_*_coviddeath.rds
moderately_sensitive:
table_incidence: output/jcvi_group_02/tables/*_coviddeath_incidence.txt
## apply cox model for coviddeath
apply_model_cox_02_coviddeath:
run: r:latest analysis/comparisons/apply_model_cox.R 02 coviddeath
needs:
- design
- data_comparisons_process_02
- data_tte_process_02_coviddeath
outputs:
highly_sensitive:
modelnumber: output/jcvi_group_02/models/*_coviddeath_model*.rds
model_tidy_rds: output/jcvi_group_02/models/*_coviddeath_modelcox_tidy.rds
model_summary_rds: output/jcvi_group_02/models/*_coviddeath_modelcox_summary.rds
moderately_sensitive:
model_glance: output/jcvi_group_02/models/*_coviddeath_modelcox_glance.csv
model_tidy_csv: output/jcvi_group_02/models/*_coviddeath_modelcox_tidy.csv
## outcome = death
## process tte data for death
data_tte_process_02_death:
run: r:latest analysis/comparisons/data_tte_process.R 02 death
needs:
- data_comparisons_process_02
- data_outcomes_process_02
outputs:
highly_sensitive:
data_tte_brand_outcome: output/jcvi_group_02/data/data_tte_*_death.rds
moderately_sensitive:
table_incidence: output/jcvi_group_02/tables/*_death_incidence.txt
## apply cox model for death
apply_model_cox_02_death:
run: r:latest analysis/comparisons/apply_model_cox.R 02 death
needs:
- design
- data_comparisons_process_02
- data_tte_process_02_death
outputs:
highly_sensitive:
modelnumber: output/jcvi_group_02/models/*_death_model*.rds
model_tidy_rds: output/jcvi_group_02/models/*_death_modelcox_tidy.rds
model_summary_rds: output/jcvi_group_02/models/*_death_modelcox_summary.rds
moderately_sensitive:
model_glance: output/jcvi_group_02/models/*_death_modelcox_glance.csv
model_tidy_csv: output/jcvi_group_02/models/*_death_modelcox_tidy.csv
## plot cox model for all outcomes
plot_model_cox_02:
run: r:latest analysis/comparisons/plot_cox.R 02
needs:
- design
- data_2nd_vax_dates
- apply_model_cox_02_postest
- apply_model_cox_02_covidadmitted
- apply_model_cox_02_coviddeath
- apply_model_cox_02_death
outputs:
moderately_sensitive:
plot: output/jcvi_group_02/images/plot_res_*.png
## tabulate cox model for all outcomes
tables_model_cox_02:
run: r:latest analysis/comparisons/tables_cox.R 02
needs:
- design
- data_2nd_vax_dates
- data_tte_process_02_postest
- data_tte_process_02_covidadmitted
- data_tte_process_02_coviddeath
- data_tte_process_02_death
- apply_model_cox_02_postest
- apply_model_cox_02_covidadmitted
- apply_model_cox_02_coviddeath
- apply_model_cox_02_death
outputs:
moderately_sensitive:
table_glance: output/jcvi_group_02/tables/*_modelcox_glance.txt
table_coefficients: output/jcvi_group_02/tables/*_modelcox_coefficients.txt
## process data, apply model and generate report for JCVI group 05
## process covariates data
data_comparisons_process_05:
run: r:latest analysis/comparisons/data_comparisons_process.R 05
needs:
- design
- data_input_process
- data_long_process
- data_2nd_vax_dates
- data_eligible_cd
outputs:
highly_sensitive:
data_comparisons: output/jcvi_group_05/data/data_comparisons.rds
## process outcomes data
data_outcomes_process_05:
run: r:latest analysis/comparisons/data_outcomes_process.R 05
needs:
- design
- data_input_process
- data_long_process
- data_comparisons_process_05
outputs:
highly_sensitive:
data_outcomes: output/jcvi_group_05/data/data_outcomes.rds
## check gap between outcomes for combining
check_combine_outcomes_05:
run: r:latest analysis/comparisons/check_combine_outcomes.R 05
needs:
- data_outcomes_process_05
outputs:
highly_sensitive:
data_check_combine_outcomes: output/jcvi_group_05/data/check_combine_outcomes.rds
moderately_sensitive:
plot_check_combine_outcomes: output/jcvi_group_05/images/check_combine_outcomes.png
table_check_combine_outcomes: output/jcvi_group_05/tables/check_combine_outcomes.csv
## outcome = postest
## process tte data for postest
data_tte_process_05_postest:
run: r:latest analysis/comparisons/data_tte_process.R 05 postest
needs:
- data_comparisons_process_05
- data_outcomes_process_05
outputs:
highly_sensitive:
data_tte_brand_outcome: output/jcvi_group_05/data/data_tte_*_postest.rds
moderately_sensitive:
table_incidence: output/jcvi_group_05/tables/*_postest_incidence.txt
## apply cox model for postest
apply_model_cox_05_postest:
run: r:latest analysis/comparisons/apply_model_cox.R 05 postest
needs:
- design
- data_comparisons_process_05
- data_tte_process_05_postest
outputs:
highly_sensitive:
modelnumber: output/jcvi_group_05/models/*_postest_model*.rds
model_tidy_rds: output/jcvi_group_05/models/*_postest_modelcox_tidy.rds
model_summary_rds: output/jcvi_group_05/models/*_postest_modelcox_summary.rds
moderately_sensitive:
model_glance: output/jcvi_group_05/models/*_postest_modelcox_glance.csv
model_tidy_csv: output/jcvi_group_05/models/*_postest_modelcox_tidy.csv
## outcome = covidadmitted
## process tte data for covidadmitted
data_tte_process_05_covidadmitted:
run: r:latest analysis/comparisons/data_tte_process.R 05 covidadmitted
needs:
- data_comparisons_process_05
- data_outcomes_process_05
outputs:
highly_sensitive:
data_tte_brand_outcome: output/jcvi_group_05/data/data_tte_*_covidadmitted.rds
moderately_sensitive:
table_incidence: output/jcvi_group_05/tables/*_covidadmitted_incidence.txt
## apply cox model for covidadmitted
apply_model_cox_05_covidadmitted:
run: r:latest analysis/comparisons/apply_model_cox.R 05 covidadmitted
needs:
- design
- data_comparisons_process_05
- data_tte_process_05_covidadmitted
outputs:
highly_sensitive:
modelnumber: output/jcvi_group_05/models/*_covidadmitted_model*.rds
model_tidy_rds: output/jcvi_group_05/models/*_covidadmitted_modelcox_tidy.rds
model_summary_rds: output/jcvi_group_05/models/*_covidadmitted_modelcox_summary.rds
moderately_sensitive:
model_glance: output/jcvi_group_05/models/*_covidadmitted_modelcox_glance.csv
model_tidy_csv: output/jcvi_group_05/models/*_covidadmitted_modelcox_tidy.csv
## outcome = coviddeath
## process tte data for coviddeath
data_tte_process_05_coviddeath:
run: r:latest analysis/comparisons/data_tte_process.R 05 coviddeath
needs:
- data_comparisons_process_05
- data_outcomes_process_05
outputs:
highly_sensitive:
data_tte_brand_outcome: output/jcvi_group_05/data/data_tte_*_coviddeath.rds
moderately_sensitive:
table_incidence: output/jcvi_group_05/tables/*_coviddeath_incidence.txt
## apply cox model for coviddeath
apply_model_cox_05_coviddeath:
run: r:latest analysis/comparisons/apply_model_cox.R 05 coviddeath
needs:
- design
- data_comparisons_process_05
- data_tte_process_05_coviddeath
outputs:
highly_sensitive:
modelnumber: output/jcvi_group_05/models/*_coviddeath_model*.rds
model_tidy_rds: output/jcvi_group_05/models/*_coviddeath_modelcox_tidy.rds
model_summary_rds: output/jcvi_group_05/models/*_coviddeath_modelcox_summary.rds
moderately_sensitive:
model_glance: output/jcvi_group_05/models/*_coviddeath_modelcox_glance.csv
model_tidy_csv: output/jcvi_group_05/models/*_coviddeath_modelcox_tidy.csv
## outcome = death
## process tte data for death
data_tte_process_05_death:
run: r:latest analysis/comparisons/data_tte_process.R 05 death
needs:
- data_comparisons_process_05
- data_outcomes_process_05
outputs:
highly_sensitive:
data_tte_brand_outcome: output/jcvi_group_05/data/data_tte_*_death.rds
moderately_sensitive:
table_incidence: output/jcvi_group_05/tables/*_death_incidence.txt
## apply cox model for death
apply_model_cox_05_death:
run: r:latest analysis/comparisons/apply_model_cox.R 05 death
needs:
- design
- data_comparisons_process_05
- data_tte_process_05_death
outputs:
highly_sensitive:
modelnumber: output/jcvi_group_05/models/*_death_model*.rds
model_tidy_rds: output/jcvi_group_05/models/*_death_modelcox_tidy.rds
model_summary_rds: output/jcvi_group_05/models/*_death_modelcox_summary.rds
moderately_sensitive:
model_glance: output/jcvi_group_05/models/*_death_modelcox_glance.csv
model_tidy_csv: output/jcvi_group_05/models/*_death_modelcox_tidy.csv
## plot cox model for all outcomes
plot_model_cox_05:
run: r:latest analysis/comparisons/plot_cox.R 05
needs:
- design
- data_2nd_vax_dates
- apply_model_cox_05_postest
- apply_model_cox_05_covidadmitted
- apply_model_cox_05_coviddeath
- apply_model_cox_05_death
outputs:
moderately_sensitive:
plot: output/jcvi_group_05/images/plot_res_*.png
## tabulate cox model for all outcomes
tables_model_cox_05:
run: r:latest analysis/comparisons/tables_cox.R 05
needs:
- design
- data_2nd_vax_dates
- data_tte_process_05_postest
- data_tte_process_05_covidadmitted
- data_tte_process_05_coviddeath
- data_tte_process_05_death
- apply_model_cox_05_postest
- apply_model_cox_05_covidadmitted
- apply_model_cox_05_coviddeath
- apply_model_cox_05_death
outputs:
moderately_sensitive:
table_glance: output/jcvi_group_05/tables/*_modelcox_glance.txt
table_coefficients: output/jcvi_group_05/tables/*_modelcox_coefficients.txt
Timeline
-
Created:
-
Started:
-
Finished:
-
Runtime: 02:12:10
These timestamps are generated and stored using the UTC timezone on the TPP backend.
Job information
- Status
-
Failed
- Backend
- TPP
- Requested by
- Elsie Horne
- Branch
- process-data-for-models
- Force run dependencies
- No
- Git commit hash
- 1f47a45
- Requested actions
-
-
data_tte_process_02_postest
-
apply_model_cox_02_postest
-
data_tte_process_02_covidadmitted
-
apply_model_cox_02_covidadmitted
-
data_tte_process_02_coviddeath
-
apply_model_cox_02_coviddeath
-
data_tte_process_02_death
-
apply_model_cox_02_death
-
plot_model_cox_02
-
tables_model_cox_02
-
data_comparisons_process_05
-
data_outcomes_process_05
-
check_combine_outcomes_05
-
data_tte_process_05_postest
-
apply_model_cox_05_postest
-
data_tte_process_05_covidadmitted
-
apply_model_cox_05_covidadmitted
-
data_tte_process_05_coviddeath
-
apply_model_cox_05_coviddeath
-
data_tte_process_05_death
-
apply_model_cox_05_death
-
plot_model_cox_05
-
tables_model_cox_05
-