Job request: 15354
- Organisation:
- University of Bristol
- Workspace:
- post-covid-respiratory
- ID:
- mmyry23w2g4emnbo
This page shows the technical details of what happened when the authorised researcher Alex Whitmarsh requested one or more actions to be run against real patient data within a secure environment.
By cross-referencing the list of jobs with the pipeline section below, you can infer what security level the outputs were written to.
The output security levels are:
-
highly_sensitive
- Researchers can never directly view these outputs
- Researchers can only request code is run against them
-
moderately_sensitive
- Can be viewed by an approved researcher by logging into a highly secure environment
- These are the only outputs that can be requested for public release via a controlled output review service.
Jobs
-
- Job identifier:
-
hbkqdqc43c3e6i5d
-
- Job identifier:
-
eqgurskh4newnwqr
-
- Job identifier:
-
vq2q5zqmhavwn5j5
-
- Job identifier:
-
5757tbcpmokul6qa - Error:
- nonzero_exit: Job exited with an error: A transient database error occurred, your job may run if you try it again, if it keeps failing then contact tech support
-
- Job identifier:
-
brz6jcc65vaabw46 - Error:
- nonzero_exit: Job exited with an error: A transient database error occurred, your job may run if you try it again, if it keeps failing then contact tech support
-
- Job identifier:
-
soyijujwy3bxt2i5
-
- Job identifier:
-
riqrygj2uwrtbfau
-
- Job identifier:
-
pno27thucsctu44q - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
7a765hc7xdkyqkgp - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
wi7qczpxwgltj554 - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
augp2uxirkxlkleh - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
akwl2qf6c7iafu3j - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
kfroz53tbz3r54pd - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
2se3ml6u54dmilof - Error:
- dependency_failed: Not starting as dependency failed
Pipeline
Show project.yaml
version: '3.0'
expectations:
population_size: 10000
actions:
## # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
## DO NOT EDIT project.yaml DIRECTLY
## This file is created by create_project_actions.R
## Edit and run create_project_actions.R to update the project.yaml
## # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
vax_eligibility_inputs:
run: r:latest analysis/metadates.R
outputs:
highly_sensitive:
study_dates_json: output/study_dates.json
vax_jcvi_groups: output/vax_jcvi_groups.csv
vax_eligible_dates: output/vax_eligible_dates.csv
generate_study_population_prelim:
run: cohortextractor:latest generate_cohort --study-definition study_definition_prelim
--output-format feather
needs:
- vax_eligibility_inputs
outputs:
highly_sensitive:
cohort: output/input_prelim.feather
generate_index_dates:
run: r:latest analysis/prelim.R
needs:
- vax_eligibility_inputs
- generate_study_population_prelim
outputs:
highly_sensitive:
index_dates: output/index_dates.csv
generate_study_population_prevax:
run: cohortextractor:latest generate_cohort --study-definition study_definition_prevax
--output-format csv.gz
needs:
- vax_eligibility_inputs
- generate_index_dates
outputs:
highly_sensitive:
cohort: output/input_prevax.csv.gz
generate_study_population_vax:
run: cohortextractor:latest generate_cohort --study-definition study_definition_vax
--output-format csv.gz
needs:
- generate_index_dates
- vax_eligibility_inputs
outputs:
highly_sensitive:
cohort: output/input_vax.csv.gz
generate_study_population_unvax:
run: cohortextractor:latest generate_cohort --study-definition study_definition_unvax
--output-format csv.gz
needs:
- vax_eligibility_inputs
- generate_index_dates
outputs:
highly_sensitive:
cohort: output/input_unvax.csv.gz
preprocess_data_prevax:
run: r:latest analysis/preprocess/preprocess_data.R prevax
needs:
- generate_index_dates
- generate_study_population_prevax
outputs:
moderately_sensitive:
describe: output/not-for-review/describe_input_prevax_stage0.txt
describe_venn: output/not-for-review/describe_venn_prevax.txt
highly_sensitive:
cohort: output/input_prevax.rds
venn: output/venn_prevax.rds
preprocess_data_vax:
run: r:latest analysis/preprocess/preprocess_data.R vax
needs:
- generate_index_dates
- generate_study_population_vax
outputs:
moderately_sensitive:
describe: output/not-for-review/describe_input_vax_stage0.txt
descrive_venn: output/not-for-review/describe_venn_vax.txt
highly_sensitive:
cohort: output/input_vax.rds
venn: output/venn_vax.rds
preprocess_data_unvax:
run: r:latest analysis/preprocess/preprocess_data.R unvax
needs:
- generate_index_dates
- generate_study_population_unvax
outputs:
moderately_sensitive:
describe: output/not-for-review/describe_input_unvax_stage0.txt
describe_venn: output/not-for-review/describe_venn_unvax.txt
highly_sensitive:
cohort: output/input_unvax.rds
venn: output/venn_unvax.rds
stage1_data_cleaning_all:
run: r:latest analysis/preprocess/Stage1_data_cleaning.R all
needs:
- preprocess_data_prevax
- preprocess_data_vax
- preprocess_data_unvax
- vax_eligibility_inputs
outputs:
moderately_sensitive:
refactoring: output/not-for-review/meta_data_factors_*.csv
QA_rules: output/review/descriptives/QA_summary_*.csv
IE_criteria: output/review/descriptives/Cohort_flow_*.csv
histograms: output/not-for-review/numeric_histograms_*.svg
highly_sensitive:
cohort: output/input_*.rds
describe_file-input_prevax_stage1:
run: r:latest analysis/describe_file.R input_prevax_stage1 rds
needs:
- stage1_data_cleaning_all
outputs:
moderately_sensitive:
describe_model_input: output/describe-input_prevax_stage1.txt
describe_file-input_vax_stage1:
run: r:latest analysis/describe_file.R input_vax_stage1 rds
needs:
- stage1_data_cleaning_all
outputs:
moderately_sensitive:
describe_model_input: output/describe-input_vax_stage1.txt
describe_file-input_unvax_stage1:
run: r:latest analysis/describe_file.R input_unvax_stage1 rds
needs:
- stage1_data_cleaning_all
outputs:
moderately_sensitive:
describe_model_input: output/describe-input_unvax_stage1.txt
stage2_missing_table1_all:
run: r:latest analysis/descriptives/Stage2_missing_table1.R all
needs:
- stage1_data_cleaning_all
outputs:
moderately_sensitive:
Missing_RangeChecks: output/not-for-review/Check_missing_range_*.csv
DateChecks: output/not-for-review/Check_dates_range_*.csv
Descriptive_Table: output/review/descriptives/Table1_*.csv
Timeline
-
Created:
-
Started:
-
Finished:
-
Runtime: 60:15:26
These timestamps are generated and stored using the UTC timezone on the TPP backend.
Job request
- Status
-
Failed
- Backend
- TPP
- Workspace
- post-covid-respiratory
- Requested by
- Alex Whitmarsh
- Branch
- main
- Force run dependencies
- Yes
- Git commit hash
- b580ab0
- Requested actions
-
-
generate_study_population_prevax -
generate_study_population_vax -
generate_study_population_unvax -
preprocess_data_prevax -
preprocess_data_vax -
preprocess_data_unvax -
stage1_data_cleaning_all -
describe_file-input_prevax_stage1 -
describe_file-input_vax_stage1 -
describe_file-input_unvax_stage1 -
stage2_missing_table1_all
-
Code comparison
Compare the code used in this job request