Job request: 8071

  • ID: fuw4ado6nzqhekju
  • Workspace: cc_3
View Repo View project.yaml

This page shows the technical details of what happened when authorised researcher rriefu requested one or more actions to be run against real patient data in the BRIT Antibiotic Research project, within a secure environment.

By cross-referencing the indicated Requested Actions with the Pipeline section below, you can infer what security level various outputs were written to. Outputs marked as highly_sensitive can never be viewed directly by a researcher; they can only request that code runs against them. Outputs marked as moderately_sensitive can be viewed by an approved researcher by logging into a highly secure environment. Only outputs marked as moderately_sensitive can be requested for release to the public, via a controlled output review service.

Jobs

ID Status Action
lk2hkmb7toos2ojd succeeded model

Pipeline

Show Hide project.yaml
version: '3.0'

expectations:
  population_size: 1000

actions:
  

  generate_study_population_covid_admission:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_covid_admission
    outputs:
      highly_sensitive:
        cohort: output/input_covid_admission.csv

  process_1:  
    run: r:latest analysis/process_1.R
    needs: [generate_study_population_covid_admission]
    outputs:
      highly_sensitive:
        case: output/case_covid_icu_death.csv
       # case2: output/case_covid_icu_death_2.csv
        control: output/control_covid_hosp.csv
      #  control2: output/control_covid_hosp_2.csv

  matching: # matching with replacement # died covid
    run: r:latest -e 'rmarkdown::render("analysis/matching.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_1]
    outputs:
      moderately_sensitive:
        html: output/matching.html
      highly_sensitive: 
        rds1: output/matched_patients.rds
        rds2: output/unmatched_patients.rds
        csv: output/matched_patients_id.csv
  
  process_Rmatching: # add variables 
    run: r:latest analysis/process_Rmatching.R
    needs: [extract_variables,matching]
    outputs:
      highly_sensitive:
        cohort1: output/matched_outcome.rds
        rds1: output/abtype79.rds
        rds2: output/comor17.rds

  table1: # matching var
    run: r:latest -e 'rmarkdown::render("analysis/table1.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_1,process_Rmatching]
    outputs:
      moderately_sensitive:
        html: output/table1.html

  table2: # confounders
    run: r:latest -e 'rmarkdown::render("analysis/table2.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_Rmatching]
    outputs:
      moderately_sensitive:
        html: output/table2.html

  table2_ab: # ab
    run: r:latest -e 'rmarkdown::render("analysis/table2_ab.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_Rmatching]
    outputs:
      moderately_sensitive:
        html: output/table2_ab.html

  model: 
    run: r:latest -e 'rmarkdown::render("analysis/model.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_Rmatching]
    outputs:
      moderately_sensitive:
        html: output/model.html
        
  # matching2: # matching with replacement # died any
  #   run: r:latest -e 'rmarkdown::render("analysis/matching2.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
  #   needs: [process_1]
  #   outputs:
  #     moderately_sensitive:
  #       html: output/matching2.html
  #     highly_sensitive: 
  #       rds1: output/matched_patients_2.rds
  #       rds2: output/unmatched_patients_2.rds
  #       csv: output/matched_patients_id_2.csv

  # check_matching:
  #   run: r:latest -e 'rmarkdown::render("analysis/check_matching.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
  #   needs: [matching]
  #   outputs:
  #     moderately_sensitive:
  #       html: output/check_matching.html
  
  # check_matching2:
  #   run: r:latest -e 'rmarkdown::render("analysis/check_matching2.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
  #   needs: [matching2]
  #   outputs:
  #     moderately_sensitive:
  #       html: output/check_matching2.html

  extract_variables: 
    run: cohortextractor:latest generate_cohort --study-definition study_definition_outcome
    needs: [matching]
    outputs:
      highly_sensitive:
        cohort: output/input_outcome.csv

  # extract_variables2: 
  #   run: cohortextractor:latest generate_cohort --study-definition study_definition_outcome2
  #   needs: [matching2]
  #   outputs:
  #     highly_sensitive:
  #       cohort: output/input_outcome2.csv
  
  # process_Rmatching: # add variables 
  #   run: r:latest analysis/process_Rmatching.R
  #   needs: [extract_variables,matching]
  #   outputs:
  #     highly_sensitive:
  #       cohort1: output/matched_outcome.rds

  # process_Rmatching2: # add variables 
  #   run: r:latest analysis/process_Rmatching2.R
  #   needs: [extract_variables2,matching2]
  #   outputs:
  #     highly_sensitive:
  #       cohort1: output/matched_outcome2.rds


  # check_input: # 
  #   run: r:latest -e 'rmarkdown::render("analysis/check_input.Rmd", knit_root_dir = "/workspace", output_dir = "output")'
  #   needs: [process_Rmatching2,extract_variables2,matching2,process_Rmatching,extract_variables,matching,process_1, generate_study_population_covid_admission]
  #   outputs:
  #     moderately_sensitive:
  #       html: output/check_input.html

  ### for covid icu/ covid death
  # process_Rmatching: # add variables 
  #   run: r:latest analysis/process_Rmatching.R
  #   needs: [extract_variables,matching]
  #   outputs:
  #     highly_sensitive:
  #       cohort1: output/matched_outcome.rds
  #       rds1: output/abtype79.rds
  #       rds2: output/comor17.rds

  # check_exposures_cont: # continuous variables
  #   run: r:latest -e 'rmarkdown::render("analysis/check_exposures_cont.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
  #   needs: [process_Rmatching]
  #   outputs:
  #     moderately_sensitive:
  #       html: output/check_exposures_cont.html
  
  # check_exposures_cat: # category variables
  #   run: r:latest -e 'rmarkdown::render("analysis/check_exposures_cat.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
  #   needs: [process_Rmatching]
  #   outputs:
  #     moderately_sensitive:
  #       html: output/check_exposures_cat.html

  # modeling: # ab level
  #   run: r:latest -e 'rmarkdown::render("analysis/modeling.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
  #   needs: [process_Rmatching]
  #   outputs:
  #     moderately_sensitive:
  #       html: output/modeling.html

  # modeling2: # ab level+ CCI + covid vaccine
  #   run: r:latest -e 'rmarkdown::render("analysis/modeling2.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
  #   needs: [process_Rmatching]
  #   outputs:
  #     moderately_sensitive:
  #       html: output/modeling2.html

  # modeling3: # all
  #   run: r:latest -e 'rmarkdown::render("analysis/modeling3.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
  #   needs: [process_Rmatching]
  #   outputs:
  #     moderately_sensitive:
  #       html: output/modeling3.html

  # describe_ab: 
  #   run: r:latest -e 'rmarkdown::render("analysis/describe_ab.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
  #   needs: [process_Rmatching]
  #   outputs:
  #     moderately_sensitive:
  #       html: output/describe_ab.html
  
  # abtype modeling
  # modeling4: # all
  #   run: r:latest -e 'rmarkdown::render("analysis/modeling4.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
  #   needs: [process_Rmatching]
  #   outputs:
  #     moderately_sensitive:
  #       html: output/modeling4.html

  
  #   broad ab modeling
  # modeling5: # all
  #   run: r:latest -e 'rmarkdown::render("analysis/modeling5.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
  #   needs: [process_Rmatching]
  #   outputs:
  #     moderately_sensitive:
  #       html: output/modeling5.html




  ### for covid icu/ any death
 ### for covid icu/ covid death
  # process_Rmatching: # add variables 
  #   run: r:latest analysis/process_Rmatching.R
  #   needs: [extract_variables,matching]
  #   outputs:
  #     highly_sensitive:
  #       cohort1: output/matched_outcome.rds
  #       rds1: output/abtype79.rds
  #       rds2: output/comor17.rds

  check_exposures_cont: # continuous variables
    run: r:latest -e 'rmarkdown::render("analysis/check_exposures_cont.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_Rmatching]
    outputs:
      moderately_sensitive:
        html: output/check_exposures_cont.html
  
  check_exposures_cat: # category variables
    run: r:latest -e 'rmarkdown::render("analysis/check_exposures_cat.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_Rmatching]
    outputs:
      moderately_sensitive:
        html: output/check_exposures_cat.html

  modeling: # ab level
    run: r:latest -e 'rmarkdown::render("analysis/modeling.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_Rmatching]
    outputs:
      moderately_sensitive:
        html: output/modeling.html

  modeling2: # ab level+ CCI + covid vaccine
    run: r:latest -e 'rmarkdown::render("analysis/modeling2.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_Rmatching]
    outputs:
      moderately_sensitive:
        html: output/modeling2.html

  modeling3: # all
    run: r:latest -e 'rmarkdown::render("analysis/modeling3.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_Rmatching]
    outputs:
      moderately_sensitive:
        html: output/modeling3.html

  describe_ab: 
    run: r:latest -e 'rmarkdown::render("analysis/describe_ab.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_Rmatching]
    outputs:
      moderately_sensitive:
        html: output/describe_ab.html
  
  # abtype modeling
  modeling4: # all
    run: r:latest -e 'rmarkdown::render("analysis/modeling4.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_Rmatching]
    outputs:
      moderately_sensitive:
        html: output/modeling4.html

  
    # broad ab modeling
  modeling5: # all
    run: r:latest -e 'rmarkdown::render("analysis/modeling5.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
    needs: [process_Rmatching]
    outputs:
      moderately_sensitive:
        html: output/modeling5.html

State

State is inferred from the related Jobs.

Status: Succeeded

Timings

Timings set to UTC timezone.

  • Created:
  • Started:
  • Finished:
  • Runtime: 00:00:55

Config

  • Backend:
    TPP
  • Workspace:
    cc_3
  • Branch:
    CC_outcome_3
  • Creator:
    rriefu
  • Force run dependencies:
    False
  • Git Commit Hash:
    37911f0
  • Requested actions:
    • model