Job request: 4772
- Organisation:
- University of Bristol
- Workspace:
- covid-ve-change-over-time--process-data-for-models
- ID:
- ojvehb2sxkuo7auv
This page shows the technical details of what happened when authorised researcher Elsie Horne requested one or more actions to be run against real patient data in the project, within a secure environment.
By cross-referencing the indicated Requested Actions with the
Pipeline section below, you can infer what
security level
various outputs were written to. Outputs marked as
highly_sensitive
can never be viewed directly by a researcher; they can only
request that code runs against them. Outputs marked as
moderately_sensitive
can be viewed by an approved researcher by logging into a highly
secure environment. Only outputs marked as
moderately_sensitive
can be requested for release to the public, via a controlled
output review service.
Jobs
-
- Job identifier:
-
fv6yu7ntpaca5gkb
-
- Job identifier:
-
qztvfrvu6naa4tm5
-
- Job identifier:
-
7lmxw25efj45wpxa
-
- Job identifier:
-
tw2fpdhbztm56vqe
-
- Job identifier:
-
ga7havjsanen3kxq
-
- Job identifier:
-
qmsecidxmqcqsfi5
-
- Job identifier:
-
6mhktyoa6nz5vkza
Pipeline
Show project.yaml
version: '3.0'
expectations:
population_size: 100000
actions:
## ####################################
## preliminaries
## ####################################
design:
run: r:latest analysis/design.R
outputs:
moderately_sensitive:
study_dates_json: output/lib/study_parameters.json
study_dates_rds: output/lib/study_parameters.rds
jcvi_groups: output/lib/jcvi_groups.csv
elig_dates: output/lib/elig_dates.csv
regions: output/lib/regions.csv
model_varlist: output/lib/model_varlist.rds
## ####################################
## vax
## ####################################
## data for eligibility criteria and vaccines
## generate dummy data for study_definition_vax
dummy_data_vax:
run: r:latest analysis/vax/dummy_data_vax.R
needs:
- design
outputs:
moderately_sensitive:
dummy_data_vax: analysis/vax/dummy_data_vax.feather
## study definition for eligiblity criteria and vaccine data
generate_study_population_vax:
run: cohortextractor:latest generate_cohort --study-definition study_definition_vax
--output-format feather
dummy_data_file: analysis/vax/dummy_data_vax.feather
needs:
- design
- dummy_data_vax
outputs:
highly_sensitive:
cohort_vax: output/input_vax.feather
## process data from study_definition_vax
data_vax_process:
run: r:latest analysis/vax/data_vax_process.R
needs:
- design
- dummy_data_vax
- generate_study_population_vax
outputs:
highly_sensitive:
data_vax_covs: output/data/data_vax_covs.rds
data_vax_dates: output/data/data_*_vax_dates.rds
moderately_sensitive:
data_properties: output/vax/tables/data_vax_processed_tabulate.txt
## apply eligiblity criteria from boxes a and b
data_eligible_ab:
run: r:latest analysis/vax/data_eligible_ab.R
needs:
- design
- data_vax_process
outputs:
highly_sensitive:
data_eligible_a: output/data/data_eligible_a.rds
data_eligible_b: output/data/data_eligible_b.rds
moderately_sensitive:
eligibility_count: output/lib/eligibility_count_ab.csv
group_age_ranges: output/lib/group_age_ranges.csv
## ####################################
## second_vax_period
## ####################################
## identify second vaccination time periods
## create dataset for identifying second vaccination time periods
data_2nd_vax_dates:
run: r:latest analysis/second_vax_period/data_2nd_vax_dates.R
needs:
- design
- data_vax_process
- data_eligible_ab
outputs:
highly_sensitive:
data_vax_plot: output/second_vax_period/data/data_vax_plot.rds
second_vax_period_dates_rds: output/lib/second_vax_period_dates.rds
moderately_sensitive:
second_vax_period_dates_csv: output/lib/second_vax_period_dates.csv
start_dates: output/lib/start_dates.csv
end_dates: output/lib/end_dates.csv
## identify and plot second vaccination time periods
plot_2nd_vax_dates:
run: r:latest analysis/second_vax_period/plot_2nd_vax_dates.R
needs:
- design
- data_eligible_ab
- data_2nd_vax_dates
outputs:
moderately_sensitive:
plots_by_region: output/second_vax_period/images/plot_by_region_*.png
## apply eligiblity criteria from boxes c and d
data_eligible_cd:
run: r:latest analysis/second_vax_period/data_eligible_cd.R
needs:
- design
- data_vax_process
- data_eligible_ab
- data_2nd_vax_dates
outputs:
highly_sensitive:
data_eligible_c: output/data/data_eligible_c.rds
data_eligible_d: output/data/data_eligible_d.rds
## ####################################
## covs
## ####################################
## data for covariates
## generate dummy data for study_definition_covs
dummy_data_covs:
run: r:latest analysis/covs/dummy_data_covs.R
needs:
- design
- dummy_data_vax
- data_2nd_vax_dates
outputs:
moderately_sensitive:
dummy_data_vax: analysis/covs/dummy_data_covs.feather
## study definition for covariates
generate_study_population_covs:
run: cohortextractor:latest generate_cohort --study-definition study_definition_covs
--output-format feather
dummy_data_file: analysis/covs/dummy_data_covs.feather
needs:
- design
- data_2nd_vax_dates
- dummy_data_covs
outputs:
highly_sensitive:
cohort_vax: output/input_covs.feather
## process recurring variables as long data
data_long_process:
run: r:latest analysis/covs/data_long_process.R
needs:
- design
- data_eligible_cd
- generate_study_population_covs
outputs:
highly_sensitive:
data_long_dates: output/data/data_long_*_dates.rds
## ####################################
## comparisons
## ####################################
## process data, apply model and generate report for JCVI group 01
## process covariates data
data_comparisons_process_01:
run: r:latest analysis/comparisons/data_comparisons_process.R 01
needs:
- design
- data_vax_process
- data_2nd_vax_dates
- data_eligible_cd
- generate_study_population_covs
- data_long_process
outputs:
highly_sensitive:
data_comparisons: output/jcvi_group_01/data/data_comparisons.rds
## process outcomes data
data_outcomes_process_01:
run: r:latest analysis/comparisons/data_outcomes_process.R 01
needs:
- design
- generate_study_population_covs
- data_long_process
- data_comparisons_process_01
outputs:
highly_sensitive:
data_outcomes: output/jcvi_group_01/data/data_outcomes.rds
## check gap between outcomes for combining
check_combine_outcomes_01:
run: r:latest analysis/comparisons/check_combine_outcomes.R 01
needs:
- data_outcomes_process_01
outputs:
highly_sensitive:
data_check_combine_outcomes: output/jcvi_group_01/data/check_combine_outcomes.rds
moderately_sensitive:
plot_check_combine_outcomes: output/jcvi_group_01/images/check_combine_outcomes.png
table_check_combine_outcomes: output/jcvi_group_01/tables/check_combine_outcomes.csv
## outcome = postest
## process tte data for postest
data_tte_process_01_postest:
run: r:latest analysis/comparisons/data_tte_process.R 01 postest
needs:
- data_comparisons_process_01
- data_outcomes_process_01
outputs:
highly_sensitive:
data_tte_brand_outcome: output/jcvi_group_01/data/data_tte_*_postest.rds
## apply cox model for postest
apply_model_cox_01_postest:
run: r:latest analysis/comparisons/apply_model_cox.R 01 postest
needs:
- design
- data_comparisons_process_01
- data_tte_process_01_postest
outputs:
highly_sensitive:
modelnumber: output/jcvi_group_01/models/*_postest_model*.rds
model_tidy_rds: output/jcvi_group_01/models/*_postest_modelcox_tidy.rds
moderately_sensitive:
model_glance: output/jcvi_group_01/models/*_postest_modelcox_glance.csv
model_tidy_csv: output/jcvi_group_01/models/*_postest_modelcox_tidy.csv
Timeline
-
Created:
-
Started:
-
Finished:
-
Runtime: 05:43:05
These timestamps are generated and stored using the UTC timezone on the TPP backend.
Job information
- Status
-
Succeeded
- Backend
- TPP
- Requested by
- Elsie Horne
- Branch
- process-data-for-models
- Force run dependencies
- Yes
- Git commit hash
- 80377c3
- Requested actions
-
-
design
-
dummy_data_vax
-
generate_study_population_vax
-
data_vax_process
-
data_eligible_ab
-
data_2nd_vax_dates
-
plot_2nd_vax_dates
-
Code comparison
Compare the code used in this Job Request
- No previous Job Request available for comparison