Job request: 16078
- Organisation:
- ONS
- Workspace:
- covid_mental_health
- ID:
- nqhxd3vc7z5vdsvi
This page shows the technical details of what happened when the authorised researcher Klaudia Rzepnicka requested one or more actions to be run against real patient data within a secure environment.
By cross-referencing the list of jobs with the pipeline section below, you can infer what security level the outputs were written to.
The output security levels are:
-
highly_sensitive
- Researchers can never directly view these outputs
- Researchers can only request code is run against them
-
moderately_sensitive
- Can be viewed by an approved researcher by logging into a highly secure environment
- These are the only outputs that can be requested for public release via a controlled output review service.
Jobs
-
- Job identifier:
-
7uyeayvwk3rvd73j
-
- Job identifier:
-
pshr3xjgqtcfgs6k - Error:
- nonzero_exit: Job exited with an error
-
- Job identifier:
-
456jf6plhqx27ack - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
bitk6g3llhhbdu5v - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
nwznd6ksm37u75lj - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
4vi4julouwc6lvt5 - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
x4mau4jprc2zarpc - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
p452behkow5a7z2k - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
25grwtiy5xrc7svd - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
73wxk6tf6yagjuad - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
3d6v2v4j2rjv2lrw - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
gher2lifavwtt64v - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
czfzddli77hys33z - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
ktiqdfrjfj73n67s - Error:
- dependency_failed: Not starting as dependency failed
Pipeline
Show project.yaml
# describes how each step in your analysis should be run
version: '3.0'
expectations:
population_size: 1000
actions:
# combine_codelists:
# run: r:latest analysis/combine_codelists.R
# outputs:
# highly_sensitive:
# cmd: 'codelists/ons-cmd-codes.csv'
#smi: 'codelists/ons-smi-codes.csv'
#self_harm : 'codelists/ons-self-harm-codes.csv'
generate_wide_non_health:
run: cohortextractor:latest generate_cohort --study-definition study_definition_non_health --with-end-date-fix
#needs: [combine_codelists]
outputs:
highly_sensitive:
cohort: output/input_non_health.csv
generate_wide_health_mh:
run: cohortextractor:latest generate_cohort --study-definition study_definition_health_mh --with-end-date-fix
#needs: [combine_codelists]
outputs:
highly_sensitive:
cohort: output/input_health_mh.csv
generate_wide_health_non_mh:
run: cohortextractor:latest generate_cohort --study-definition study_definition_health_non_mh --with-end-date-fix
#needs: [combine_codelists]
outputs:
highly_sensitive:
cohort: output/input_health_non_mh.csv
generate_wide_ons_cis_new:
run: >
databuilder:v0
generate-dataset
--output output/dataset_ons_cis_new.csv
analysis/dataset_definition_ons_cis_new.py
outputs:
highly_sensitive:
dataset: output/dataset_ons_cis_new.csv
combine_wide_data:
run: r:latest analysis/1_combine_wide_data.R
needs: [generate_wide_non_health, generate_wide_health_mh, generate_wide_health_non_mh, generate_wide_ons_cis_new]
outputs:
highly_sensitive:
cohort: output/input_cis_wide.csv
transform_cis_wide_to_long:
run: r:latest analysis/2_cis_wide_to_long.R
needs: [combine_wide_data]
outputs:
highly_sensitive:
cohort: output/input_cis_long.csv
reconcile_snomed_ctv3:
run: r:latest analysis/3_reconcile_snomed_ctv3.R
needs: [transform_cis_wide_to_long]
outputs:
highly_sensitive:
cohort: output/input_reconciled.csv
#bmi: output/bmi_summary_table_info_before_cap.csv
derive_exposed:
run: r:latest analysis/4_exposed_population.R
needs: [reconcile_snomed_ctv3]
outputs:
highly_sensitive:
cohort: output/cis_exposed.csv
derive_controls:
run: r:latest analysis/5_control_population.R
needs: [reconcile_snomed_ctv3]
outputs:
highly_sensitive:
cohort: output/cis_control.csv
perform_matching:
run: r:latest analysis/6_create_controls.R
needs: [derive_exposed, derive_controls]
outputs:
highly_sensitive:
incidence: output/incidence_group.csv
prevalence: output/prevalence_group.csv
exacerbated: output/exacerbated_group.csv
adjust_matched_groups:
run: r:latest analysis/7_adjust_groups.R
needs: [perform_matching]
outputs:
highly_sensitive:
incidence: output/adjusted_incidence_group.csv
prevalence: output/adjusted_prevalence_group.csv
descriptive_stats:
run: r:latest analysis/8_descriptive_statistics.R
needs: [adjust_matched_groups]
outputs:
moderately_sensitive:
incidence_cat_stats: output/1_descriptives_incidence_cat.csv
incidence_con_stats: output/2_descriptives_incidence_con.csv
prevalence_cat_stats: output/3_descriptives_prevalence_cat.csv
prevalence_con_stats: output/4_descriptives_prevalence_con.csv
cumulative_incidence:
run: r:latest analysis/9_cumulative_incidence_curves.R
needs: [adjust_matched_groups]
outputs:
highly_sensitive:
inc_t: output/incidence_t.csv
prev_t: output/prevalence_t.csv
moderately_sensitive:
inc_surv: output/incidence_surv.jpg
prev_surv: output/prevalence_surv.jpg
hazard_ratios:
run: r:latest analysis/10_new_hazard_ratio_code.R
needs: [cumulative_incidence]
outputs:
moderately_sensitive:
temp: output/5_cox_hazard_ratio_incidence_table.csv
temp2: output/6_cox_hazard_ratio_prevalence_table.csv
surv1: output/1_survfit_plot_incidence_noadj.jpg
surv2: output/2_survfit_plot_prevalence_noadj.jpg
surv3: output/3_survfit_plot_incidence_min.jpg
surv4: output/4_survfit_plot_prevalence_min.jpg
surv5: output/5_survfit_plot_incidence_full.jpg
surv6: output/6_survfit_plot_prevalence_full.jpg
shoen_1: output/inc_no_adj_schoenfeld_res.csv
shoen_2: output/inc_min_adj_schoenfeld_res.csv
shoen_3: output/inc_full_adj_schoenfeld_res.csv
shoen_4: output/prev_no_adj_schoenfeld_res.csv
shoen_5: output/prev_min_adj_schoenfeld_res.csv
shoen_6: output/prev_full_adj_schoenfeld_res.csv
shoen_jp1: output/inc_no_adj_schoenfeld_res.jpg
shoen_jp2: output/inc_min_adj_schoenfeld_res.jpg
shoen_jp3: output/inc_full_adj_schoenfeld_res.jpg
shoen_jp4: output/prev_no_adj_schoenfeld_res.jpg
shoen_jp5: output/prev_min_adj_schoenfeld_res.jpg
shoen_jp6: output/prev_full_adj_schoenfeld_res.jpg
distribution_of_follow_up_time:
run: r:latest analysis/11_distribution_of_follow_up_time.R
needs: [cumulative_incidence]
outputs:
moderately_sensitive:
distribution_inc: output/distribution_of_follow_up_time_incidence.jpg
distribution_prev: output/distribution_of_follow_up_time_prevalence.jpg
density1: output/distribution_of_follow_up_time_incidence_density.jpg
density2: output/distribution_of_follow_up_time_prevalence_density.jpg
event_rates_incidence:
run: r:latest analysis/12_event_rates_incidence.R
needs: [cumulative_incidence]
outputs:
moderately_sensitive:
incidence_event_rates: output/event_counts_and_rates_incidence.csv
event_rates_prevalence:
run: r:latest analysis/13_event_rates_prevalence.R
needs: [cumulative_incidence]
outputs:
moderately_sensitive:
incidence_event_rates: output/event_counts_and_rates_prevalence.csv
Timeline
-
Created:
-
Started:
-
Finished:
-
Runtime: 00:01:17
These timestamps are generated and stored using the UTC timezone on the TPP backend.
Job request
- Status
-
Failed
- Backend
- TPP
- Workspace
- covid_mental_health
- Requested by
- Klaudia Rzepnicka
- Branch
- main
- Force run dependencies
- No
- Git commit hash
- 45b7958
- Requested actions
-
-
generate_wide_ons_cis_new -
combine_wide_data -
transform_cis_wide_to_long -
reconcile_snomed_ctv3 -
derive_exposed -
derive_controls -
perform_matching -
adjust_matched_groups -
descriptive_stats -
cumulative_incidence -
hazard_ratios -
distribution_of_follow_up_time -
event_rates_incidence -
event_rates_prevalence
-
Code comparison
Compare the code used in this job request