Job request: 18067
- Organisation:
- ONS
- Workspace:
- covid_mental_health
- ID:
- rni7x7fn57nii6mn
This page shows the technical details of what happened when the authorised researcher Klaudia Rzepnicka requested one or more actions to be run against real patient data within a secure environment.
By cross-referencing the list of jobs with the pipeline section below, you can infer what security level the outputs were written to.
The output security levels are:
-
highly_sensitive
- Researchers can never directly view these outputs
- Researchers can only request code is run against them
-
moderately_sensitive
- Can be viewed by an approved researcher by logging into a highly secure environment
- These are the only outputs that can be requested for public release via a controlled output review service.
Jobs
-
- Job identifier:
-
6wgxlb7cxusmsjsd - Error:
- cancelled_by_user: Cancelled by user
-
- Job identifier:
-
bhx3f43lce4vzymw - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
54pg7gcuvxqw3rq6 - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
j4rayb5vzkcby3qz - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
4evkvhbs27yo6q23 - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
h2zbriea7er6ahat - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
smzsam7vtwb34kxr - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
d2wxwpey2l4icdop - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
wdegq3hyht56jrco - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
z5gfmqjfjg5odkfw - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
ddmfe75sbc343tks - Error:
- cancelled_by_user: Cancelled by user
-
- Job identifier:
-
oqhk3pxnwk3aix7x - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
xs753upawjxjomkw - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
muiwarsbpmittaea - Error:
- dependency_failed: Not starting as dependency failed
Pipeline
Show project.yaml
# describes how each step in your analysis should be run
version: '3.0'
expectations:
population_size: 1000
actions:
generate_wide_non_health:
run: cohortextractor:latest generate_cohort --study-definition study_definition_non_health --with-end-date-fix
outputs:
highly_sensitive:
cohort: output/input_non_health.csv
generate_wide_health_mh:
run: cohortextractor:latest generate_cohort --study-definition study_definition_health_mh --with-end-date-fix
outputs:
highly_sensitive:
cohort: output/input_health_mh.csv
generate_wide_health_non_mh:
run: cohortextractor:latest generate_cohort --study-definition study_definition_health_non_mh --with-end-date-fix
outputs:
highly_sensitive:
cohort: output/input_health_non_mh.csv
generate_wide_ons_cis_new:
run: >
databuilder:v0
generate-dataset
--output output/dataset_ons_cis_new.csv
analysis/dataset_definition_ons_cis_new.py
outputs:
highly_sensitive:
dataset: output/dataset_ons_cis_new.csv
combine_wide_data:
run: r:latest analysis/1_combine_wide_data.R
needs: [generate_wide_non_health, generate_wide_health_mh, generate_wide_health_non_mh, generate_wide_ons_cis_new]
outputs:
highly_sensitive:
cohort: output/input_cis_wide.csv
transform_cis_wide_to_long:
run: r:latest analysis/2_cis_wide_to_long.R
needs: [combine_wide_data]
outputs:
highly_sensitive:
cohort: output/input_cis_long.csv
reconcile_snomed_ctv3:
run: r:latest analysis/3_reconcile_snomed_ctv3.R
needs: [transform_cis_wide_to_long]
outputs:
highly_sensitive:
cohort: output/input_reconciled.csv
#bmi: output/bmi_summary_table_info_before_cap.csv
derive_exposed:
run: r:latest analysis/4_exposed_population.R
needs: [reconcile_snomed_ctv3]
outputs:
highly_sensitive:
cohort: output/cis_exposed.csv
derive_controls:
run: r:latest analysis/5_control_population.R
needs: [reconcile_snomed_ctv3]
outputs:
highly_sensitive:
cohort: output/cis_control.csv
perform_matching:
run: r:latest analysis/6_create_controls.R
needs: [derive_exposed, derive_controls]
outputs:
highly_sensitive:
incidence: output/incidence_group.csv
prevalence: output/prevalence_group.csv
testing_exposed_post_pre_matching:
run: r:latest analysis/6_6_pre_post_match_testing.R
needs: [derive_exposed, derive_controls,perform_matching]
outputs:
moderately_sensitive:
t1: output/matching_rates_incidence.csv
t2: output/matching_rates_prevalence.csv
adjust_matched_groups:
run: r:latest analysis/7_adjust_groups.R
needs: [perform_matching]
outputs:
highly_sensitive:
incidence: output/adjusted_incidence_group.csv
prevalence: output/adjusted_prevalence_group.csv
descriptive_stats:
run: r:latest analysis/8_descriptive_statistics.R
needs: [adjust_matched_groups]
outputs:
moderately_sensitive:
incidence_cat_stats: output/1_descriptives_incidence_cat.csv
incidence_con_stats: output/2_descriptives_incidence_con.csv
prevalence_cat_stats: output/3_descriptives_prevalence_cat.csv
prevalence_con_stats: output/4_descriptives_prevalence_con.csv
cumulative_incidence:
run: r:latest analysis/9_cumulative_incidence_curves.R
needs: [adjust_matched_groups]
outputs:
highly_sensitive:
inc_t: output/incidence_t.csv
prev_t: output/prevalence_t.csv
moderately_sensitive:
inc_surv: output/incidence_surv.jpg
prev_surv: output/prevalence_surv.jpg
hazard_ratios:
run: r:latest analysis/10_new_hazard_ratio_code.R
needs: [cumulative_incidence]
outputs:
moderately_sensitive:
temp: output/5_cox_hazard_ratio_incidence_table.csv
temp2: output/6_cox_hazard_ratio_prevalence_table.csv
surv1: output/1_survfit_plot_incidence_noadj.jpg
surv2: output/2_survfit_plot_prevalence_noadj.jpg
surv3: output/3_survfit_plot_incidence_min.jpg
surv4: output/4_survfit_plot_prevalence_min.jpg
surv5: output/5_survfit_plot_incidence_full.jpg
surv6: output/6_survfit_plot_prevalence_full.jpg
shoen_1: output/inc_no_adj_schoenfeld_res.csv
shoen_2: output/inc_min_adj_schoenfeld_res.csv
shoen_3: output/inc_full_adj_schoenfeld_res.csv
shoen_4: output/prev_no_adj_schoenfeld_res.csv
shoen_5: output/prev_min_adj_schoenfeld_res.csv
shoen_6: output/prev_full_adj_schoenfeld_res.csv
shoen_jp1: output/inc_no_adj_schoenfeld_res.jpg
shoen_jp2: output/inc_min_adj_schoenfeld_res.jpg
shoen_jp3: output/inc_full_adj_schoenfeld_res.jpg
shoen_jp4: output/prev_no_adj_schoenfeld_res.jpg
shoen_jp5: output/prev_min_adj_schoenfeld_res.jpg
shoen_jp6: output/prev_full_adj_schoenfeld_res.jpg
distribution_of_follow_up_time:
run: r:latest analysis/11_distribution_of_follow_up_time.R
needs: [cumulative_incidence]
outputs:
moderately_sensitive:
distribution_inc: output/distribution_of_follow_up_time_incidence.jpg
distribution_prev: output/distribution_of_follow_up_time_prevalence.jpg
density1: output/distribution_of_follow_up_time_incidence_density.jpg
density2: output/distribution_of_follow_up_time_prevalence_density.jpg
event_rates_incidence:
run: r:latest analysis/12_event_rates_incidence.R
needs: [cumulative_incidence]
outputs:
moderately_sensitive:
incidence_event_rates: output/event_counts_and_rates_incidence.csv
event_rates_prevalence:
run: r:latest analysis/13_event_rates_prevalence.R
needs: [cumulative_incidence]
outputs:
moderately_sensitive:
incidence_event_rates: output/event_counts_and_rates_prevalence.csv
testing_interactions_and_anova:
run: r:latest analysis/14_testing_interactions_and_anova.R
needs: [cumulative_incidence]
outputs:
moderately_sensitive:
d1: output/99_SCH_RESIDUALS_indextime_spline_interaction_INCIDENCE.jpg
d2: output/99_COEFF_indextime_spline_interaction_INCIDENCE_full.csv
d3: output/99_ANOVA_indextime_spline_interaction_INCIDENCE.csv
d4: output/99_SCH_RESIDUALS_indextime_spline_interaction_INCIDENCE.csv
d5: output/99_SCH_RESIDUALS_indextime_spline_interaction_PREVALENCE.jpg
d6: output/99_SCH_RESIDUALS_indextime_spline_interaction_PREVALENCE.csv
sc1: output/99_COEFF_indextime_spline_interaction_PREVALENCE.csv
sc2: output/99_ANOVA_indextime_spline_interaction_PREVALENCE.csv
sc3: output/99_SCH_RESIDUALS_waves_interaction_INCIDENCE.jpg
sc4: output/99_ANOVA_waves_interaction_INCIDENCE.csv
sc5: output/99_COEFF_waves_interaction_INCIDENCE.csv
sc6: output/99_SCH_RESIDUALS_waves_interaction_INCIDENCE.csv
#emmeans:
# run: r:latest analysis/15_emmeans.R
# needs: [cumulative_incidence]
# outputs:
# moderately_sensitive:
# d1: output/99_emmeans_incidence.csv
# d2: output/99_emmeans_incidence.jpg
# d3: output/99_emmeans_3df_incidence.csv
# d4: output/99_emmeans_3df_incidence.jpg
# d5: output/99_emmeans_4df_incidence.csv
# d6: output/99_emmeans_4df_incidence.jpg
# d7: output/BIC_all_3models.cvs
# d8: output/different_degrees_of_freedom_avova_all_adjustments.cvs
#hr_for_interaction_with_spline:
# run: r:latest analysis/16_hr_for_interaction_with_spline.R
# needs: [cumulative_incidence]
# outputs:
# moderately_sensitive:
# d1: output/100_hazard_ratios_by_modifier_incidence.csv
# d2: output/100_hazard_ratios_by_modifier_variable.jpg
Timeline
-
Created:
-
Started:
-
Finished:
-
Runtime: 00:00:29
These timestamps are generated and stored using the UTC timezone on the TPP backend.
Job request
- Status
-
Failed
- Backend
- TPP
- Workspace
- covid_mental_health
- Requested by
- Klaudia Rzepnicka
- Branch
- main
- Force run dependencies
- No
- Git commit hash
- 0eaec45
- Requested actions
-
-
transform_cis_wide_to_long -
reconcile_snomed_ctv3 -
derive_exposed -
derive_controls -
perform_matching -
testing_exposed_post_pre_matching -
adjust_matched_groups -
descriptive_stats -
cumulative_incidence -
hazard_ratios -
distribution_of_follow_up_time -
event_rates_incidence -
event_rates_prevalence -
testing_interactions_and_anova
-
Code comparison
Compare the code used in this job request