Skip to content

Job request: 13899

Organisation:
University of Manchester
Workspace:
cc_3_sens
ID:
ld7x3oq34lfvh52q

This page shows the technical details of what happened when the authorised researcher Ya-Ting Yang requested one or more actions to be run against real patient data in the project, within a secure environment.

By cross-referencing the list of jobs with the pipeline section below, you can infer what security level various outputs were written to. Researchers can never directly view outputs marked as highly_sensitive ; they can only request that code runs against them. Outputs marked as moderately_sensitive can be viewed by an approved researcher by logging into a highly secure environment. Only outputs marked as moderately_sensitive can be requested for release to the public, via a controlled output review service.

Jobs

  • Action:
    process_1
    Status:
    Status: Succeeded
    Job identifier:
    f3azb5cknmmb7prp
  • Action:
    matching_icu
    Status:
    Status: Succeeded
    Job identifier:
    aodww7por6y2rcrt
  • Action:
    extract_variables_icu
    Status:
    Status: Succeeded
    Job identifier:
    sah4sxqy3jz4for6
  • Action:
    process_Rmatching_icu
    Status:
    Status: Succeeded
    Job identifier:
    3wizan26j45p3qn2
  • Action:
    model_icu
    Status:
    Status: Succeeded
    Job identifier:
    vq5nbmeeyjhi4svn
  • Action:
    model_2_icu
    Status:
    Status: Succeeded
    Job identifier:
    uv2e65qhlti465ld

Pipeline

Show project.yaml
version: '3.0'

expectations:
  population_size: 1000

actions:
  
  generate_study_population_covid_admission:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_covid_admission
    outputs:
      highly_sensitive:
        cohort: output/input_covid_admission.csv

  process_1:  
    run: r:latest analysis/process_1.R
    needs: [generate_study_population_covid_admission]
    outputs:
      highly_sensitive:
        case: output/case_covid_icu_death.csv
        control: output/control_covid_hosp.csv
        case2: output/case_covid_icu.csv

# combine ICU+death outcome
  matching: # matching with replacement # died covid
    run: r:latest -e 'rmarkdown::render("analysis/matching.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_1]
    outputs:
      moderately_sensitive:
        html: output/matching.html
      highly_sensitive: 
        rds1: output/matched_patients.rds
        rds2: output/unmatched_patients.rds
        csv: output/matched_patients_id.csv

  check_unmatched:
    run: r:latest -e 'rmarkdown::render("analysis/check_unmatched.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [matching]
    outputs:
      moderately_sensitive:
        html: output/check_unmatched.html
  
  check_process_1: 
    run: r:latest -e 'rmarkdown::render("analysis/check_process_1.Rmd", knit_root_dir = "/workspace", output_dir = "output")'
    needs: [generate_study_population_covid_admission]
    outputs:
      moderately_sensitive:
        html: output/check_process_1.html

  extract_variables: 
    run: cohortextractor:latest generate_cohort --with-end-date-fix --study-definition study_definition_outcome
    needs: [matching]
    outputs:
      highly_sensitive:
        cohort: output/input_outcome.csv

  process_Rmatching: # add variables 
    run: r:latest analysis/process_Rmatching.R
    needs: [extract_variables,matching]
    outputs:
      highly_sensitive:
        cohort1: output/matched_outcome.rds
        rds1: output/abtype79.rds
        rds2: output/comor17.rds

  table1: # matching var
    run: r:latest -e 'rmarkdown::render("analysis/table1.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_1,process_Rmatching]
    outputs:
      moderately_sensitive:
        html: output/table1.html

  table1_round: 
    run: r:latest analysis/table1.R
    needs: [process_1,process_Rmatching]
    outputs:
      moderately_sensitive:
        csv1: output/table1_unmatched.csv
        csv2: output/table1_matched.csv
        csv3: output/table1_random.csv
        
  table2: # confounders
    run: r:latest -e 'rmarkdown::render("analysis/table2.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_Rmatching]
    outputs:
      moderately_sensitive:
        html: output/table2.html

  table2_round: 
    run: r:latest analysis/table2.R
    needs: [process_Rmatching]
    outputs:
      moderately_sensitive:
        csv1: output/table2_matched.csv
        csv3: output/table2_random.csv

  table2_ab: # ab
    run: r:latest -e 'rmarkdown::render("analysis/table2_ab.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_Rmatching]
    outputs:
      moderately_sensitive:
        html: output/table2_ab.html


  table3_round: 
    run: r:latest analysis/table3.R
    needs: [process_Rmatching]
    outputs:
      moderately_sensitive:
        csv1: output/table3.csv
        csv3: output/table3_ab.csv

  model: 
    run: r:latest analysis/model/model.R
    needs: [process_Rmatching]
    outputs:
      moderately_sensitive:
        csv1: output/model_1_crude.csv
        csv2: output/model_1_adjusted.csv
  
  model_2: 
    run: r:latest -e 'rmarkdown::render("analysis/model_2.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_Rmatching]
    outputs:
      moderately_sensitive:
        html: output/model_2.html

  
# only ICU outcome
  matching_icu: # matching with replacement # died covid
    run: r:latest -e 'rmarkdown::render("analysis/matching_icu.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_1]
    outputs:
      moderately_sensitive:
        html: output/matching_icu.html
      highly_sensitive: 
        rds1: output/matched_patients_icu.rds
        rds2: output/unmatched_patients_icu.rds
        csv: output/matched_patients_id_icu.csv

  extract_variables_icu: 
    run: cohortextractor:latest generate_cohort --with-end-date-fix --study-definition study_definition_outcome_icu
    needs: [matching_icu]
    outputs:
      highly_sensitive:
        cohort: output/input_outcome_icu.csv

  process_Rmatching_icu:  
    run: r:latest analysis/process_Rmatching_icu.R
    needs: [extract_variables_icu,matching_icu]
    outputs:
      highly_sensitive:
        cohort1: output/matched_outcome_icu.rds
        rds1: output/abtype79_icu.rds
        rds2: output/comor17_icu.rds
  
  model_icu: 
    run: r:latest analysis/model/model_icu.R
    needs: [process_Rmatching_icu]
    outputs:
      moderately_sensitive:
        csv1: output/model_1_crude_icu.csv
        csv2: output/model_1_adjusted_icu.csv
  
  model_2_icu: 
    run: r:latest -e 'rmarkdown::render("analysis/model_2_icu.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_Rmatching_icu]
    outputs:
      moderately_sensitive:
        html: output/model_2_icu.html

Timeline

  • Created:

  • Started:

  • Finished:

  • Runtime: 00:55:15

These timestamps are generated and stored using the UTC timezone on the TPP backend.

Job information

Status
Succeeded
Backend
TPP
Workspace
cc_3_sens
Requested by
Ya-Ting Yang
Branch
CC_outcome_3_sens
Force run dependencies
No
Git commit hash
b576012
Requested actions
  • process_1
  • matching_icu
  • extract_variables_icu
  • process_Rmatching_icu
  • model_icu
  • model_2_icu

Code comparison

Compare the code used in this Job Request