Job request: 6332

View Repo View project.yaml

This page shows the technical details of what happened when authorised researcher Venexia Walker requested one or more actions to be run against real patient data in the post-covid-vaccinated project, within a secure environment.

By cross-referencing the indicated Requested Actions with the Pipeline section below, you can infer what security level various outputs were written to. Outputs marked as highly_sensitive can never be viewed directly by a researcher; they can only request that code runs against them. Outputs marked as moderately_sensitive can be viewed by an approved researcher by logging into a highly secure environment. Only outputs marked as moderately_sensitive can be requested for release to the public, via a controlled output review service.

Jobs

ID Status Action
ackght3kk7ri7jcm succeeded Analysis_cox_eating_disorders
mfrz7bdu3w7eg3wp failed Analysis_cox_addiction
g4absyq5sjaursn5 failed Analysis_cox_angina
rywq3zfujlmuqyid failed Analysis_cox_anxiety_general
rehib42gp5ir6gno failed Analysis_cox_anxiety_ocd
g4abcguew2yw3pdd failed Analysis_cox_anxiety_ptsd
sshvpkcg5iwfowws failed Analysis_cox_ate
apcc36hwzcqebniu failed Analysis_cox_depression
ir5t2dkuwl6l2tu5 failed Analysis_cox_hf
5jzislqkd52lvue4 succeeded Analysis_cox_self_harm_10plus
jqyy4jpjibtw74zp succeeded Analysis_cox_self_harm_15plus
a2pnhw5nigqbsbxt failed Analysis_cox_serious_mental_illness
boee2hydf2jvyax6 succeeded Analysis_cox_stroke_sah_hs
isntckuce2opgsx3 failed Analysis_cox_suicide
7bw6jjtr2sriewz7 failed Analysis_cox_vte

Pipeline

Show Hide project.yaml
version: '3.0'

expectations:

  population_size: 400000

actions:

  ## # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # 
  ## DO NOT EDIT project.yaml DIRECTLY 
  ## This file is created by create_project_actions.R 
  ## Edit and run create_project_actions.R to update the project.yaml 
  ## # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # 

  vax_eligibility_inputs:
    run: r:latest analysis/vax_eligibility_inputs.R
    outputs:
      highly_sensitive:
        vax_study_dates_json: output/vax_study_dates.json
        vax_jcvi_groups: output/vax_jcvi_groups.csv
        vax_eligible_dates: output/vax_eligible_dates.csv

  generate_study_population_electively_unvaccinated:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_electively_unvaccinated
      --output-format feather
    needs:
    - vax_eligibility_inputs
    outputs:
      highly_sensitive:
        cohort: output/input_electively_unvaccinated.feather

  generate_study_population_vaccinated:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_vaccinated
      --output-format feather
    outputs:
      highly_sensitive:
        cohort: output/input_vaccinated.feather

  generate_study_population_index:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_index
      --output-format feather
    outputs:
      highly_sensitive:
        cohort: output/input_index.feather

  preprocess_data_vaccinated:
    run: r:latest analysis/preprocess_data.R vaccinated
    needs:
    - generate_study_population_index
    - generate_study_population_vaccinated
    - generate_study_population_electively_unvaccinated
    outputs:
      moderately_sensitive:
        describe: output/describe_input_vaccinated_*.txt
      highly_sensitive:
        cohort: output/input_vaccinated.rds
        venn: output/venn_vaccinated.rds

  preprocess_data_electively_unvaccinated:
    run: r:latest analysis/preprocess_data.R electively_unvaccinated
    needs:
    - generate_study_population_index
    - generate_study_population_vaccinated
    - generate_study_population_electively_unvaccinated
    outputs:
      moderately_sensitive:
        describe: output/describe_input_electively_unvaccinated_*.txt
      highly_sensitive:
        cohort: output/input_electively_unvaccinated.rds
        venn: output/venn_electively_unvaccinated.rds

  stage1_data_cleaning_both:
    run: r:latest analysis/Stage1_data_cleaning.R both
    needs:
    - preprocess_data_vaccinated
    - preprocess_data_electively_unvaccinated
    outputs:
      moderately_sensitive:
        refactoring: output/meta_data_factors_*.csv
        QA_rules: output/QA_summary_*.csv
        IE_criteria: output/Cohort_flow_*.csv
      highly_sensitive:
        cohort: output/input_*_stage1.rds

  stage2_missing_table1_both:
    run: r:latest analysis/Stage2_missing_table1.R both
    needs:
    - stage1_data_cleaning_both
    outputs:
      moderately_sensitive:
        Missing_RangeChecks: output/Check_missing_range_*.csv
        DateChecks: output/Check_dates_range_*.csv
        Descriptive_Table: output/Table1_*.csv

  stage4_table2_both:
    run: r:latest analysis/table_2.R both
    needs:
    - stage1_data_cleaning_both
    outputs:
      moderately_sensitive:
        table2: output/table2_*.csv

  stage4_venn_diagram_both:
    run: r:latest analysis/venn_diagram.R both
    needs:
    - preprocess_data_vaccinated
    - preprocess_data_electively_unvaccinated
    - stage1_data_cleaning_both
    outputs:
      moderately_sensitive:
        venn_diagram: output/venn_diagram_*.svg
        venn_diagram_number_check: output/venn_diagram_number_check_*.csv

  ## Apply cox model for ami 

  Analysis_cox_ami:
    run: r:latest analysis/01_pipe.R ami
    needs:
    - stage1_data_cleaning_both
    outputs:
      moderately_sensitive:
        compiled_hrs_html: output/suppressed_compiled_HR_results_ami.html
        compiled_event_counts_html: output/suppressed_compiled_event_counts_ami.html
        analyses_not_run: output/analyses_not_run_ami.csv
        compiled_hrs_csv: output/suppressed_compiled_HR_results_ami.csv
        compiled_event_counts_csv: output/suppressed_compiled_event_counts_ami.csv
      highly_sensitive:
        compiled_hrs: output/compiled_HR_results_ami.csv
        compiled_event_counts: output/compiled_event_counts_ami.csv

  ## Apply cox model for stroke_isch 

  Analysis_cox_stroke_isch:
    run: r:latest analysis/01_pipe.R stroke_isch
    needs:
    - stage1_data_cleaning_both
    outputs:
      moderately_sensitive:
        compiled_hrs_html: output/suppressed_compiled_HR_results_stroke_isch.html
        compiled_event_counts_html: output/suppressed_compiled_event_counts_stroke_isch.html
        analyses_not_run: output/analyses_not_run_stroke_isch.csv
        compiled_hrs_csv: output/suppressed_compiled_HR_results_stroke_isch.csv
        compiled_event_counts_csv: output/suppressed_compiled_event_counts_stroke_isch.csv
      highly_sensitive:
        compiled_hrs: output/compiled_HR_results_stroke_isch.csv
        compiled_event_counts: output/compiled_event_counts_stroke_isch.csv

  ## Apply cox model for dvt 

  Analysis_cox_dvt:
    run: r:latest analysis/01_pipe.R dvt
    needs:
    - stage1_data_cleaning_both
    outputs:
      moderately_sensitive:
        compiled_hrs_html: output/suppressed_compiled_HR_results_dvt.html
        compiled_event_counts_html: output/suppressed_compiled_event_counts_dvt.html
        analyses_not_run: output/analyses_not_run_dvt.csv
        compiled_hrs_csv: output/suppressed_compiled_HR_results_dvt.csv
        compiled_event_counts_csv: output/suppressed_compiled_event_counts_dvt.csv
      highly_sensitive:
        compiled_hrs: output/compiled_HR_results_dvt.csv
        compiled_event_counts: output/compiled_event_counts_dvt.csv

  ## Apply cox model for pe 

  Analysis_cox_pe:
    run: r:latest analysis/01_pipe.R pe
    needs:
    - stage1_data_cleaning_both
    outputs:
      moderately_sensitive:
        compiled_hrs_html: output/suppressed_compiled_HR_results_pe.html
        compiled_event_counts_html: output/suppressed_compiled_event_counts_pe.html
        analyses_not_run: output/analyses_not_run_pe.csv
        compiled_hrs_csv: output/suppressed_compiled_HR_results_pe.csv
        compiled_event_counts_csv: output/suppressed_compiled_event_counts_pe.csv
      highly_sensitive:
        compiled_hrs: output/compiled_HR_results_pe.csv
        compiled_event_counts: output/compiled_event_counts_pe.csv

  ## Apply cox model for tia 

  Analysis_cox_tia:
    run: r:latest analysis/01_pipe.R tia
    needs:
    - stage1_data_cleaning_both
    outputs:
      moderately_sensitive:
        compiled_hrs_html: output/suppressed_compiled_HR_results_tia.html
        compiled_event_counts_html: output/suppressed_compiled_event_counts_tia.html
        analyses_not_run: output/analyses_not_run_tia.csv
        compiled_hrs_csv: output/suppressed_compiled_HR_results_tia.csv
        compiled_event_counts_csv: output/suppressed_compiled_event_counts_tia.csv
      highly_sensitive:
        compiled_hrs: output/compiled_HR_results_tia.csv
        compiled_event_counts: output/compiled_event_counts_tia.csv

  ## Apply cox model for stroke_sah_hs 

  Analysis_cox_stroke_sah_hs:
    run: r:latest analysis/01_pipe.R stroke_sah_hs
    needs:
    - stage1_data_cleaning_both
    outputs:
      moderately_sensitive:
        compiled_hrs_html: output/suppressed_compiled_HR_results_stroke_sah_hs.html
        compiled_event_counts_html: output/suppressed_compiled_event_counts_stroke_sah_hs.html
        analyses_not_run: output/analyses_not_run_stroke_sah_hs.csv
        compiled_hrs_csv: output/suppressed_compiled_HR_results_stroke_sah_hs.csv
        compiled_event_counts_csv: output/suppressed_compiled_event_counts_stroke_sah_hs.csv
      highly_sensitive:
        compiled_hrs: output/compiled_HR_results_stroke_sah_hs.csv
        compiled_event_counts: output/compiled_event_counts_stroke_sah_hs.csv

  ## Apply cox model for hf 

  Analysis_cox_hf:
    run: r:latest analysis/01_pipe.R hf
    needs:
    - stage1_data_cleaning_both
    outputs:
      moderately_sensitive:
        compiled_hrs_html: output/suppressed_compiled_HR_results_hf.html
        compiled_event_counts_html: output/suppressed_compiled_event_counts_hf.html
        analyses_not_run: output/analyses_not_run_hf.csv
        compiled_hrs_csv: output/suppressed_compiled_HR_results_hf.csv
        compiled_event_counts_csv: output/suppressed_compiled_event_counts_hf.csv
      highly_sensitive:
        compiled_hrs: output/compiled_HR_results_hf.csv
        compiled_event_counts: output/compiled_event_counts_hf.csv

  ## Apply cox model for angina 

  Analysis_cox_angina:
    run: r:latest analysis/01_pipe.R angina
    needs:
    - stage1_data_cleaning_both
    outputs:
      moderately_sensitive:
        compiled_hrs_html: output/suppressed_compiled_HR_results_angina.html
        compiled_event_counts_html: output/suppressed_compiled_event_counts_angina.html
        analyses_not_run: output/analyses_not_run_angina.csv
        compiled_hrs_csv: output/suppressed_compiled_HR_results_angina.csv
        compiled_event_counts_csv: output/suppressed_compiled_event_counts_angina.csv
      highly_sensitive:
        compiled_hrs: output/compiled_HR_results_angina.csv
        compiled_event_counts: output/compiled_event_counts_angina.csv

  ## Apply cox model for ate 

  Analysis_cox_ate:
    run: r:latest analysis/01_pipe.R ate
    needs:
    - stage1_data_cleaning_both
    outputs:
      moderately_sensitive:
        compiled_hrs_html: output/suppressed_compiled_HR_results_ate.html
        compiled_event_counts_html: output/suppressed_compiled_event_counts_ate.html
        analyses_not_run: output/analyses_not_run_ate.csv
        compiled_hrs_csv: output/suppressed_compiled_HR_results_ate.csv
        compiled_event_counts_csv: output/suppressed_compiled_event_counts_ate.csv
      highly_sensitive:
        compiled_hrs: output/compiled_HR_results_ate.csv
        compiled_event_counts: output/compiled_event_counts_ate.csv

  ## Apply cox model for vte 

  Analysis_cox_vte:
    run: r:latest analysis/01_pipe.R vte
    needs:
    - stage1_data_cleaning_both
    outputs:
      moderately_sensitive:
        compiled_hrs_html: output/suppressed_compiled_HR_results_vte.html
        compiled_event_counts_html: output/suppressed_compiled_event_counts_vte.html
        analyses_not_run: output/analyses_not_run_vte.csv
        compiled_hrs_csv: output/suppressed_compiled_HR_results_vte.csv
        compiled_event_counts_csv: output/suppressed_compiled_event_counts_vte.csv
      highly_sensitive:
        compiled_hrs: output/compiled_HR_results_vte.csv
        compiled_event_counts: output/compiled_event_counts_vte.csv

  ## Apply cox model for depression 

  Analysis_cox_depression:
    run: r:latest analysis/01_pipe.R depression
    needs:
    - stage1_data_cleaning_both
    outputs:
      moderately_sensitive:
        compiled_hrs_html: output/suppressed_compiled_HR_results_depression.html
        compiled_event_counts_html: output/suppressed_compiled_event_counts_depression.html
        analyses_not_run: output/analyses_not_run_depression.csv
        compiled_hrs_csv: output/suppressed_compiled_HR_results_depression.csv
        compiled_event_counts_csv: output/suppressed_compiled_event_counts_depression.csv
      highly_sensitive:
        compiled_hrs: output/compiled_HR_results_depression.csv
        compiled_event_counts: output/compiled_event_counts_depression.csv

  ## Apply cox model for anxiety_general 

  Analysis_cox_anxiety_general:
    run: r:latest analysis/01_pipe.R anxiety_general
    needs:
    - stage1_data_cleaning_both
    outputs:
      moderately_sensitive:
        compiled_hrs_html: output/suppressed_compiled_HR_results_anxiety_general.html
        compiled_event_counts_html: output/suppressed_compiled_event_counts_anxiety_general.html
        analyses_not_run: output/analyses_not_run_anxiety_general.csv
        compiled_hrs_csv: output/suppressed_compiled_HR_results_anxiety_general.csv
        compiled_event_counts_csv: output/suppressed_compiled_event_counts_anxiety_general.csv
      highly_sensitive:
        compiled_hrs: output/compiled_HR_results_anxiety_general.csv
        compiled_event_counts: output/compiled_event_counts_anxiety_general.csv

  ## Apply cox model for anxiety_ocd 

  Analysis_cox_anxiety_ocd:
    run: r:latest analysis/01_pipe.R anxiety_ocd
    needs:
    - stage1_data_cleaning_both
    outputs:
      moderately_sensitive:
        compiled_hrs_html: output/suppressed_compiled_HR_results_anxiety_ocd.html
        compiled_event_counts_html: output/suppressed_compiled_event_counts_anxiety_ocd.html
        analyses_not_run: output/analyses_not_run_anxiety_ocd.csv
        compiled_hrs_csv: output/suppressed_compiled_HR_results_anxiety_ocd.csv
        compiled_event_counts_csv: output/suppressed_compiled_event_counts_anxiety_ocd.csv
      highly_sensitive:
        compiled_hrs: output/compiled_HR_results_anxiety_ocd.csv
        compiled_event_counts: output/compiled_event_counts_anxiety_ocd.csv

  ## Apply cox model for anxiety_ptsd 

  Analysis_cox_anxiety_ptsd:
    run: r:latest analysis/01_pipe.R anxiety_ptsd
    needs:
    - stage1_data_cleaning_both
    outputs:
      moderately_sensitive:
        compiled_hrs_html: output/suppressed_compiled_HR_results_anxiety_ptsd.html
        compiled_event_counts_html: output/suppressed_compiled_event_counts_anxiety_ptsd.html
        analyses_not_run: output/analyses_not_run_anxiety_ptsd.csv
        compiled_hrs_csv: output/suppressed_compiled_HR_results_anxiety_ptsd.csv
        compiled_event_counts_csv: output/suppressed_compiled_event_counts_anxiety_ptsd.csv
      highly_sensitive:
        compiled_hrs: output/compiled_HR_results_anxiety_ptsd.csv
        compiled_event_counts: output/compiled_event_counts_anxiety_ptsd.csv

  ## Apply cox model for eating_disorders 

  Analysis_cox_eating_disorders:
    run: r:latest analysis/01_pipe.R eating_disorders
    needs:
    - stage1_data_cleaning_both
    outputs:
      moderately_sensitive:
        compiled_hrs_html: output/suppressed_compiled_HR_results_eating_disorders.html
        compiled_event_counts_html: output/suppressed_compiled_event_counts_eating_disorders.html
        analyses_not_run: output/analyses_not_run_eating_disorders.csv
        compiled_hrs_csv: output/suppressed_compiled_HR_results_eating_disorders.csv
        compiled_event_counts_csv: output/suppressed_compiled_event_counts_eating_disorders.csv
      highly_sensitive:
        compiled_hrs: output/compiled_HR_results_eating_disorders.csv
        compiled_event_counts: output/compiled_event_counts_eating_disorders.csv

  ## Apply cox model for serious_mental_illness 

  Analysis_cox_serious_mental_illness:
    run: r:latest analysis/01_pipe.R serious_mental_illness
    needs:
    - stage1_data_cleaning_both
    outputs:
      moderately_sensitive:
        compiled_hrs_html: output/suppressed_compiled_HR_results_serious_mental_illness.html
        compiled_event_counts_html: output/suppressed_compiled_event_counts_serious_mental_illness.html
        analyses_not_run: output/analyses_not_run_serious_mental_illness.csv
        compiled_hrs_csv: output/suppressed_compiled_HR_results_serious_mental_illness.csv
        compiled_event_counts_csv: output/suppressed_compiled_event_counts_serious_mental_illness.csv
      highly_sensitive:
        compiled_hrs: output/compiled_HR_results_serious_mental_illness.csv
        compiled_event_counts: output/compiled_event_counts_serious_mental_illness.csv

  ## Apply cox model for self_harm_10plus 

  Analysis_cox_self_harm_10plus:
    run: r:latest analysis/01_pipe.R self_harm_10plus
    needs:
    - stage1_data_cleaning_both
    outputs:
      moderately_sensitive:
        compiled_hrs_html: output/suppressed_compiled_HR_results_self_harm_10plus.html
        compiled_event_counts_html: output/suppressed_compiled_event_counts_self_harm_10plus.html
        analyses_not_run: output/analyses_not_run_self_harm_10plus.csv
        compiled_hrs_csv: output/suppressed_compiled_HR_results_self_harm_10plus.csv
        compiled_event_counts_csv: output/suppressed_compiled_event_counts_self_harm_10plus.csv
      highly_sensitive:
        compiled_hrs: output/compiled_HR_results_self_harm_10plus.csv
        compiled_event_counts: output/compiled_event_counts_self_harm_10plus.csv

  ## Apply cox model for self_harm_15plus 

  Analysis_cox_self_harm_15plus:
    run: r:latest analysis/01_pipe.R self_harm_15plus
    needs:
    - stage1_data_cleaning_both
    outputs:
      moderately_sensitive:
        compiled_hrs_html: output/suppressed_compiled_HR_results_self_harm_15plus.html
        compiled_event_counts_html: output/suppressed_compiled_event_counts_self_harm_15plus.html
        analyses_not_run: output/analyses_not_run_self_harm_15plus.csv
        compiled_hrs_csv: output/suppressed_compiled_HR_results_self_harm_15plus.csv
        compiled_event_counts_csv: output/suppressed_compiled_event_counts_self_harm_15plus.csv
      highly_sensitive:
        compiled_hrs: output/compiled_HR_results_self_harm_15plus.csv
        compiled_event_counts: output/compiled_event_counts_self_harm_15plus.csv

  ## Apply cox model for suicide 

  Analysis_cox_suicide:
    run: r:latest analysis/01_pipe.R suicide
    needs:
    - stage1_data_cleaning_both
    outputs:
      moderately_sensitive:
        compiled_hrs_html: output/suppressed_compiled_HR_results_suicide.html
        compiled_event_counts_html: output/suppressed_compiled_event_counts_suicide.html
        analyses_not_run: output/analyses_not_run_suicide.csv
        compiled_hrs_csv: output/suppressed_compiled_HR_results_suicide.csv
        compiled_event_counts_csv: output/suppressed_compiled_event_counts_suicide.csv
      highly_sensitive:
        compiled_hrs: output/compiled_HR_results_suicide.csv
        compiled_event_counts: output/compiled_event_counts_suicide.csv

  ## Apply cox model for addiction 

  Analysis_cox_addiction:
    run: r:latest analysis/01_pipe.R addiction
    needs:
    - stage1_data_cleaning_both
    outputs:
      moderately_sensitive:
        compiled_hrs_html: output/suppressed_compiled_HR_results_addiction.html
        compiled_event_counts_html: output/suppressed_compiled_event_counts_addiction.html
        analyses_not_run: output/analyses_not_run_addiction.csv
        compiled_hrs_csv: output/suppressed_compiled_HR_results_addiction.csv
        compiled_event_counts_csv: output/suppressed_compiled_event_counts_addiction.csv
      highly_sensitive:
        compiled_hrs: output/compiled_HR_results_addiction.csv
        compiled_event_counts: output/compiled_event_counts_addiction.csv

State

State is inferred from the related Jobs.

Status: Failed

Timings

Timings set to UTC timezone.

  • Created:
  • Started:
  • Finished:
  • Runtime: 182:52:25

Config

  • Backend:
    TPP
  • Workspace:
    post-covid-vaccinated
  • Branch:
    main
  • Creator:
    venexia
  • Force run dependencies:
    False
  • Git Commit Hash:
    430b38e
  • Requested actions:
    • Analysis_cox_stroke_sah_hs
    • Analysis_cox_hf
    • Analysis_cox_angina
    • Analysis_cox_ate
    • Analysis_cox_vte
    • Analysis_cox_depression
    • Analysis_cox_anxiety_general
    • Analysis_cox_anxiety_ocd
    • Analysis_cox_anxiety_ptsd
    • Analysis_cox_eating_disorders
    • Analysis_cox_serious_mental_illness
    • Analysis_cox_self_harm_10plus
    • Analysis_cox_self_harm_15plus
    • Analysis_cox_suicide
    • Analysis_cox_addiction