Skip to content

Job request: 8067

Organisation:
University of Manchester
Workspace:
cc_3
ID:
brw7l3n5o7uu67ey
View repo View project.yaml

This page shows the technical details of what happened when authorised researcher Ya-Ting Yang requested one or more actions to be run against real patient data in the project, within a secure environment.

By cross-referencing the indicated Requested Actions with the Pipeline section below, you can infer what security level various outputs were written to. Outputs marked as highly_sensitive can never be viewed directly by a researcher; they can only request that code runs against them. Outputs marked as moderately_sensitive can be viewed by an approved researcher by logging into a highly secure environment. Only outputs marked as moderately_sensitive can be requested for release to the public, via a controlled output review service.

Jobs

  • Action:
    process_Rmatching
    Status:
    Status: Succeeded
    Job identifier:
    iquo3cflzphqvzut
  • Action:
    table1
    Status:
    Status: Succeeded
    Job identifier:
    bf65gkhdxuxfimzu
  • Action:
    table2_ab
    Status:
    Status: Succeeded
    Job identifier:
    6naawzsxvibwgr4k

Pipeline

Show project.yaml 
version: '3.0'

expectations:
  population_size: 1000

actions:
  

  generate_study_population_covid_admission:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_covid_admission
    outputs:
      highly_sensitive:
        cohort: output/input_covid_admission.csv

  process_1:  
    run: r:latest analysis/process_1.R
    needs: [generate_study_population_covid_admission]
    outputs:
      highly_sensitive:
        case: output/case_covid_icu_death.csv
       # case2: output/case_covid_icu_death_2.csv
        control: output/control_covid_hosp.csv
      #  control2: output/control_covid_hosp_2.csv

  matching: # matching with replacement # died covid
    run: r:latest -e 'rmarkdown::render("analysis/matching.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_1]
    outputs:
      moderately_sensitive:
        html: output/matching.html
      highly_sensitive: 
        rds1: output/matched_patients.rds
        rds2: output/unmatched_patients.rds
        csv: output/matched_patients_id.csv
  
  process_Rmatching: # add variables 
    run: r:latest analysis/process_Rmatching.R
    needs: [extract_variables,matching]
    outputs:
      highly_sensitive:
        cohort1: output/matched_outcome.rds
        rds1: output/abtype79.rds
        rds2: output/comor17.rds

  table1: # matching var
    run: r:latest -e 'rmarkdown::render("analysis/table1.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_1,process_Rmatching]
    outputs:
      moderately_sensitive:
        html: output/table1.html

  table2: # confounders
    run: r:latest -e 'rmarkdown::render("analysis/table2.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_Rmatching]
    outputs:
      moderately_sensitive:
        html: output/table2.html

  table2_ab: # ab
    run: r:latest -e 'rmarkdown::render("analysis/table2_ab.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_Rmatching]
    outputs:
      moderately_sensitive:
        html: output/table2_ab.html


  # matching2: # matching with replacement # died any
  #   run: r:latest -e 'rmarkdown::render("analysis/matching2.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
  #   needs: [process_1]
  #   outputs:
  #     moderately_sensitive:
  #       html: output/matching2.html
  #     highly_sensitive: 
  #       rds1: output/matched_patients_2.rds
  #       rds2: output/unmatched_patients_2.rds
  #       csv: output/matched_patients_id_2.csv

  # check_matching:
  #   run: r:latest -e 'rmarkdown::render("analysis/check_matching.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
  #   needs: [matching]
  #   outputs:
  #     moderately_sensitive:
  #       html: output/check_matching.html
  
  # check_matching2:
  #   run: r:latest -e 'rmarkdown::render("analysis/check_matching2.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
  #   needs: [matching2]
  #   outputs:
  #     moderately_sensitive:
  #       html: output/check_matching2.html

  extract_variables: 
    run: cohortextractor:latest generate_cohort --study-definition study_definition_outcome
    needs: [matching]
    outputs:
      highly_sensitive:
        cohort: output/input_outcome.csv

  # extract_variables2: 
  #   run: cohortextractor:latest generate_cohort --study-definition study_definition_outcome2
  #   needs: [matching2]
  #   outputs:
  #     highly_sensitive:
  #       cohort: output/input_outcome2.csv
  
  # process_Rmatching: # add variables 
  #   run: r:latest analysis/process_Rmatching.R
  #   needs: [extract_variables,matching]
  #   outputs:
  #     highly_sensitive:
  #       cohort1: output/matched_outcome.rds

  # process_Rmatching2: # add variables 
  #   run: r:latest analysis/process_Rmatching2.R
  #   needs: [extract_variables2,matching2]
  #   outputs:
  #     highly_sensitive:
  #       cohort1: output/matched_outcome2.rds


  # check_input: # 
  #   run: r:latest -e 'rmarkdown::render("analysis/check_input.Rmd", knit_root_dir = "/workspace", output_dir = "output")'
  #   needs: [process_Rmatching2,extract_variables2,matching2,process_Rmatching,extract_variables,matching,process_1, generate_study_population_covid_admission]
  #   outputs:
  #     moderately_sensitive:
  #       html: output/check_input.html

  ### for covid icu/ covid death
  # process_Rmatching: # add variables 
  #   run: r:latest analysis/process_Rmatching.R
  #   needs: [extract_variables,matching]
  #   outputs:
  #     highly_sensitive:
  #       cohort1: output/matched_outcome.rds
  #       rds1: output/abtype79.rds
  #       rds2: output/comor17.rds

  # check_exposures_cont: # continuous variables
  #   run: r:latest -e 'rmarkdown::render("analysis/check_exposures_cont.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
  #   needs: [process_Rmatching]
  #   outputs:
  #     moderately_sensitive:
  #       html: output/check_exposures_cont.html
  
  # check_exposures_cat: # category variables
  #   run: r:latest -e 'rmarkdown::render("analysis/check_exposures_cat.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
  #   needs: [process_Rmatching]
  #   outputs:
  #     moderately_sensitive:
  #       html: output/check_exposures_cat.html

  # modeling: # ab level
  #   run: r:latest -e 'rmarkdown::render("analysis/modeling.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
  #   needs: [process_Rmatching]
  #   outputs:
  #     moderately_sensitive:
  #       html: output/modeling.html

  # modeling2: # ab level+ CCI + covid vaccine
  #   run: r:latest -e 'rmarkdown::render("analysis/modeling2.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
  #   needs: [process_Rmatching]
  #   outputs:
  #     moderately_sensitive:
  #       html: output/modeling2.html

  # modeling3: # all
  #   run: r:latest -e 'rmarkdown::render("analysis/modeling3.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
  #   needs: [process_Rmatching]
  #   outputs:
  #     moderately_sensitive:
  #       html: output/modeling3.html

  # describe_ab: 
  #   run: r:latest -e 'rmarkdown::render("analysis/describe_ab.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
  #   needs: [process_Rmatching]
  #   outputs:
  #     moderately_sensitive:
  #       html: output/describe_ab.html
  
  # abtype modeling
  # modeling4: # all
  #   run: r:latest -e 'rmarkdown::render("analysis/modeling4.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
  #   needs: [process_Rmatching]
  #   outputs:
  #     moderately_sensitive:
  #       html: output/modeling4.html

  
  #   broad ab modeling
  # modeling5: # all
  #   run: r:latest -e 'rmarkdown::render("analysis/modeling5.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
  #   needs: [process_Rmatching]
  #   outputs:
  #     moderately_sensitive:
  #       html: output/modeling5.html




  ### for covid icu/ any death
 ### for covid icu/ covid death
  # process_Rmatching: # add variables 
  #   run: r:latest analysis/process_Rmatching.R
  #   needs: [extract_variables,matching]
  #   outputs:
  #     highly_sensitive:
  #       cohort1: output/matched_outcome.rds
  #       rds1: output/abtype79.rds
  #       rds2: output/comor17.rds

  check_exposures_cont: # continuous variables
    run: r:latest -e 'rmarkdown::render("analysis/check_exposures_cont.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_Rmatching]
    outputs:
      moderately_sensitive:
        html: output/check_exposures_cont.html
  
  check_exposures_cat: # category variables
    run: r:latest -e 'rmarkdown::render("analysis/check_exposures_cat.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_Rmatching]
    outputs:
      moderately_sensitive:
        html: output/check_exposures_cat.html

  modeling: # ab level
    run: r:latest -e 'rmarkdown::render("analysis/modeling.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_Rmatching]
    outputs:
      moderately_sensitive:
        html: output/modeling.html

  modeling2: # ab level+ CCI + covid vaccine
    run: r:latest -e 'rmarkdown::render("analysis/modeling2.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_Rmatching]
    outputs:
      moderately_sensitive:
        html: output/modeling2.html

  modeling3: # all
    run: r:latest -e 'rmarkdown::render("analysis/modeling3.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_Rmatching]
    outputs:
      moderately_sensitive:
        html: output/modeling3.html

  describe_ab: 
    run: r:latest -e 'rmarkdown::render("analysis/describe_ab.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_Rmatching]
    outputs:
      moderately_sensitive:
        html: output/describe_ab.html
  
  # abtype modeling
  modeling4: # all
    run: r:latest -e 'rmarkdown::render("analysis/modeling4.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_Rmatching]
    outputs:
      moderately_sensitive:
        html: output/modeling4.html

  
    # broad ab modeling
  modeling5: # all
    run: r:latest -e 'rmarkdown::render("analysis/modeling5.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_Rmatching]
    outputs:
      moderately_sensitive:
        html: output/modeling5.html

Timeline

  • Created:

  • Started:

  • Finished:

  • Runtime: 00:02:12

These timestamps are generated and stored using the UTC timezone on the TPP backend.

Job information

Status
Succeeded
Backend
TPP
Workspace
cc_3
Requested by
Ya-Ting Yang
Branch
CC_outcome_3
Force run dependencies
No
Git commit hash
16b0f09
Requested actions
  • process_Rmatching
  • table1
  • table2_ab

Code comparison

Compare the code used in this Job Request