Skip to content

Job request: 11732

Organisation:
University of Bristol
Workspace:
vaccine-effectiveness-in-children
ID:
6g5w6s2yxg2zp252
View repo View project.yaml

This page shows the technical details of what happened when the authorised researcher Will Hulme requested one or more actions to be run against real patient data within a secure environment.

By cross-referencing the list of jobs with the pipeline section below, you can infer what security level the outputs were written to.

The output security levels are:

  • highly_sensitive
    • Researchers can never directly view these outputs
    • Researchers can only request code is run against them
  • moderately_sensitive
    • Can be viewed by an approved researcher by logging into a highly secure environment
    • These are the only outputs that can be requested for public release via a controlled output review service.

Jobs

  • Action:
    extract_controlpotential_over12_1
    Status:
    Succeeded
    Job identifier:
    r4bd3yrdfyzpl4r2
  • Action:
    extract_treated_over12
    Status:
    Succeeded
    Job identifier:
    mb3stdrlxxokxaaf
  • Action:
    process_treated_over12
    Status:
    Succeeded
    Job identifier:
    5kdzyf5wkhcojxc7
  • Action:
    process_controlpotential_over12_1
    Status:
    Failed
    Job identifier:
    dfrv3ryw3ar42mxw
    Error:
    nonzero_exit: Job exited with error code 1
  • Action:
    process_controlactual_over12_1
    Status:
    Failed
    Job identifier:
    k6xfjz5ke6f34ji4
    Error:
    dependency_failed: Not starting as dependency failed
  • Action:
    process_controlpotential_over12_2
    Status:
    Failed
    Job identifier:
    ea7ryegbrh2mdehp
    Error:
    dependency_failed: Not starting as dependency failed
  • Action:
    km_over12_all_postest
    Status:
    Failed
    Job identifier:
    sobu3wrs5uozpoho
    Error:
    dependency_failed: Not starting as dependency failed
  • Action:
    km_over12_all_emergency
    Status:
    Failed
    Job identifier:
    rihahdy7mx7dwzz4
    Error:
    dependency_failed: Not starting as dependency failed
  • Action:
    km_over12_all_covidemergency
    Status:
    Failed
    Job identifier:
    2sr5w5xtludxto4o
    Error:
    dependency_failed: Not starting as dependency failed
  • Action:
    km_over12_all_covidadmitted
    Status:
    Failed
    Job identifier:
    6wb4ym4yi6wzinzx
    Error:
    dependency_failed: Not starting as dependency failed
  • Action:
    km_over12_all_covidcritcare
    Status:
    Failed
    Job identifier:
    p5w7x76zil6j774j
    Error:
    dependency_failed: Not starting as dependency failed
  • Action:
    km_over12_all_coviddeath
    Status:
    Failed
    Job identifier:
    6g7ruwsl4m42ns7i
    Error:
    dependency_failed: Not starting as dependency failed
  • Action:
    km_over12_all_noncoviddeath
    Status:
    Failed
    Job identifier:
    gf3tgo6jw74tgqxu
    Error:
    dependency_failed: Not starting as dependency failed
  • Action:
    process_controlactual_over12_2
    Status:
    Failed
    Job identifier:
    6ff3qujedn2z6rfm
    Error:
    dependency_failed: Not starting as dependency failed
  • Action:
    extract_controlfinal_over12
    Status:
    Failed
    Job identifier:
    667c24u5on7zkby3
    Error:
    dependency_failed: Not starting as dependency failed
  • Action:
    process_controlfinal_over12
    Status:
    Failed
    Job identifier:
    w5q2djkqvwolusc5
    Error:
    dependency_failed: Not starting as dependency failed
  • Action:
    extract_controlpotential_over12_2
    Status:
    Failed
    Job identifier:
    ls4aa3l4i2famnkh
    Error:
    dependency_failed: Not starting as dependency failed
  • Action:
    extract_controlactual_over12_2
    Status:
    Failed
    Job identifier:
    ranb5vc2bqjmsy7l
    Error:
    dependency_failed: Not starting as dependency failed
  • Action:
    match_potential_over12_2
    Status:
    Failed
    Job identifier:
    5ao4lnf5xrl7zdec
    Error:
    dependency_failed: Not starting as dependency failed
  • Action:
    match_potential_over12_1
    Status:
    Failed
    Job identifier:
    qs3az6awbt73fjgk
    Error:
    dependency_failed: Not starting as dependency failed
  • Action:
    extract_controlactual_over12_1
    Status:
    Failed
    Job identifier:
    qzeevm3rnssg4cwi
    Error:
    dependency_failed: Not starting as dependency failed

Pipeline

Show project.yaml 
version: '3.0'

expectations:

  population_size: 100000

actions:

  ## # # # # # # # # # # # # # # # # # # # 
  ## DO NOT EDIT project.yaml DIRECTLY 
  ## This file is created by create-project.R 
  ## Edit and run create-project.R to update the project.yaml 
  ## # # # # # # # # # # # # # # # # # # # 
  ##   
  ## # # # # # # # # # # # # # # # # # # # 
  ## Over 12s cohort 
  ## # # # # # # # # # # # # # # # # # # # 
  ## # # # # # # # # # # # # # # # # # # # 
  ## Extract and match 
  ## # # # # # # # # # # # # # # # # # # # 

  extract_treated_over12:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_treated
      --output-file output/over12/extract/input_treated.feather --param cohort=over12
    outputs:
      highly_sensitive:
        extract: output/over12/extract/input_treated.feather

  process_treated_over12:
    run: r:latest analysis/treated/process_treated.R over12
    needs:
    - extract_treated_over12
    outputs:
      highly_sensitive:
        rds: output/over12/treated/*.rds

  extract_controlpotential_over12_1:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_controlpotential
      --output-file output/over12/matchround1/extract/input_controlpotential.feather
      --param cohort=over12 --param matching_round=1 --param index_date=2021-09-20
    needs: []
    outputs:
      highly_sensitive:
        cohort: output/over12/matchround1/extract/input_controlpotential.feather

  process_controlpotential_over12_1:
    run: r:latest analysis/matching/process_controlpotential.R over12 1
    needs:
    - extract_controlpotential_over12_1
    outputs:
      highly_sensitive:
        rds: output/over12/matchround1/process/*.rds

  match_potential_over12_1:
    run: r:latest analysis/matching/match_potential.R over12 1
    needs:
    - process_treated_over12
    - process_controlpotential_over12_1
    outputs:
      highly_sensitive:
        rds: output/over12/matchround1/potential/*.rds
        csv: output/over12/matchround1/potential/*.csv.gz

  extract_controlactual_over12_1:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_controlactual
      --output-file output/over12/matchround1/extract/input_controlactual.feather
      --param cohort=over12 --param matching_round=1
    needs:
    - match_potential_over12_1
    outputs:
      highly_sensitive:
        cohort: output/over12/matchround1/extract/input_controlactual.feather

  process_controlactual_over12_1:
    run: r:latest analysis/matching/process_controlactual.R over12 1
    needs:
    - process_treated_over12
    - match_potential_over12_1
    - extract_controlpotential_over12_1
    - extract_controlactual_over12_1
    outputs:
      highly_sensitive:
        rds: output/over12/matchround1/actual/*.rds
        csv: output/over12/matchround1/actual/*.csv.gz

  extract_controlpotential_over12_2:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_controlpotential
      --output-file output/over12/matchround2/extract/input_controlpotential.feather
      --param cohort=over12 --param matching_round=2 --param index_date=2021-10-04
    needs:
    - process_controlactual_over12_1
    outputs:
      highly_sensitive:
        cohort: output/over12/matchround2/extract/input_controlpotential.feather

  process_controlpotential_over12_2:
    run: r:latest analysis/matching/process_controlpotential.R over12 2
    needs:
    - extract_controlpotential_over12_2
    outputs:
      highly_sensitive:
        rds: output/over12/matchround2/process/*.rds

  match_potential_over12_2:
    run: r:latest analysis/matching/match_potential.R over12 2
    needs:
    - process_treated_over12
    - process_controlpotential_over12_2
    - process_controlactual_over12_1
    outputs:
      highly_sensitive:
        rds: output/over12/matchround2/potential/*.rds
        csv: output/over12/matchround2/potential/*.csv.gz

  extract_controlactual_over12_2:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_controlactual
      --output-file output/over12/matchround2/extract/input_controlactual.feather
      --param cohort=over12 --param matching_round=2
    needs:
    - match_potential_over12_2
    outputs:
      highly_sensitive:
        cohort: output/over12/matchround2/extract/input_controlactual.feather

  process_controlactual_over12_2:
    run: r:latest analysis/matching/process_controlactual.R over12 2
    needs:
    - process_treated_over12
    - match_potential_over12_2
    - extract_controlpotential_over12_2
    - extract_controlactual_over12_2
    - process_controlactual_over12_1
    outputs:
      highly_sensitive:
        rds: output/over12/matchround2/actual/*.rds
        csv: output/over12/matchround2/actual/*.csv.gz

  extract_controlfinal_over12:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_controlfinal
      --output-file output/over12/extract/input_controlfinal.feather --param cohort=over12
      --param n_matching_rounds=2
    needs:
    - process_controlactual_over12_2
    outputs:
      highly_sensitive:
        extract: output/over12/extract/input_controlfinal.feather

  process_controlfinal_over12:
    run: r:latest analysis/matching/process_controlfinal.R over12
    needs:
    - process_controlactual_over12_1
    - process_controlactual_over12_2
    - extract_controlfinal_over12
    - process_treated_over12
    outputs:
      highly_sensitive:
        extract: output/over12/match/*.rds

  ## # # # # # # # # # # # # # # # # # # # 
  ## Model 
  ## # # # # # # # # # # # # # # # # # # # 

  km_over12_all_postest:
    run: r:latest analysis/km.R over12 all postest
    needs:
    - process_controlfinal_over12
    outputs:
      moderately_sensitive:
        csv: output/over12/models/km/all/postest/*.csv
        rds: output/over12/models/km/all/postest/*.rds
        png: output/over12/models/km/all/postest/*.png

  km_over12_all_emergency:
    run: r:latest analysis/km.R over12 all emergency
    needs:
    - process_controlfinal_over12
    outputs:
      moderately_sensitive:
        csv: output/over12/models/km/all/emergency/*.csv
        rds: output/over12/models/km/all/emergency/*.rds
        png: output/over12/models/km/all/emergency/*.png

  km_over12_all_covidemergency:
    run: r:latest analysis/km.R over12 all covidemergency
    needs:
    - process_controlfinal_over12
    outputs:
      moderately_sensitive:
        csv: output/over12/models/km/all/covidemergency/*.csv
        rds: output/over12/models/km/all/covidemergency/*.rds
        png: output/over12/models/km/all/covidemergency/*.png

  km_over12_all_covidadmitted:
    run: r:latest analysis/km.R over12 all covidadmitted
    needs:
    - process_controlfinal_over12
    outputs:
      moderately_sensitive:
        csv: output/over12/models/km/all/covidadmitted/*.csv
        rds: output/over12/models/km/all/covidadmitted/*.rds
        png: output/over12/models/km/all/covidadmitted/*.png

  km_over12_all_covidcritcare:
    run: r:latest analysis/km.R over12 all covidcritcare
    needs:
    - process_controlfinal_over12
    outputs:
      moderately_sensitive:
        csv: output/over12/models/km/all/covidcritcare/*.csv
        rds: output/over12/models/km/all/covidcritcare/*.rds
        png: output/over12/models/km/all/covidcritcare/*.png

  km_over12_all_coviddeath:
    run: r:latest analysis/km.R over12 all coviddeath
    needs:
    - process_controlfinal_over12
    outputs:
      moderately_sensitive:
        csv: output/over12/models/km/all/coviddeath/*.csv
        rds: output/over12/models/km/all/coviddeath/*.rds
        png: output/over12/models/km/all/coviddeath/*.png

  km_over12_all_noncoviddeath:
    run: r:latest analysis/km.R over12 all noncoviddeath
    needs:
    - process_controlfinal_over12
    outputs:
      moderately_sensitive:
        csv: output/over12/models/km/all/noncoviddeath/*.csv
        rds: output/over12/models/km/all/noncoviddeath/*.rds
        png: output/over12/models/km/all/noncoviddeath/*.png

  ## #### End ####

Timeline

  • Created:

  • Started:

  • Finished:

  • Runtime: 03:21:03

These timestamps are generated and stored using the UTC timezone on the TPP backend.

Job request

Status
Failed
Backend
TPP
Workspace
vaccine-effectiveness-in-children
Requested by
Will Hulme
Branch
main
Force run dependencies
No
Git commit hash
afb5e2c
Requested actions
  • run_all

Code comparison

Compare the code used in this job request