Skip to content

Job request: 7276

Organisation:
University of Bristol
Workspace:
covid-ve-change-over-time--define-covs-at-svp
ID:
dxtltj5upjdnbreu

This page shows the technical details of what happened when the authorised researcher Elsie Horne requested one or more actions to be run against real patient data within a secure environment.

By cross-referencing the list of jobs with the pipeline section below, you can infer what security level the outputs were written to.

The output security levels are:

  • highly_sensitive
    • Researchers can never directly view these outputs
    • Researchers can only request code is run against them
  • moderately_sensitive
    • Can be viewed by an approved researcher by logging into a highly secure environment
    • These are the only outputs that can be requested for public release via a controlled output review service.

Jobs

  • Action:
    dummy_data_vax
    Status:
    Status: Succeeded
    Job identifier:
    yzmkku723jsik74e
  • Action:
    generate_study_population
    Status:
    Status: Succeeded
    Job identifier:
    hc6mnradfozxybik
  • Action:
    data_input_process
    Status:
    Status: Succeeded
    Job identifier:
    sgfw5zhpoanyl6dm
  • Action:
    data_tte_process_BNT162b2
    Status:
    Status: Failed
    Job identifier:
    aetspyrvj3gmkovc
    Error:
    dependency_failed: Not starting as dependency failed
  • Action:
    data_2nd_vax_dates
    Status:
    Status: Failed
    Job identifier:
    xlkpwzazynltfq5z
    Error:
    dependency_failed: Not starting as dependency failed
  • Action:
    data_min_max_fu
    Status:
    Status: Failed
    Job identifier:
    scxyczfgd5haxdd3
    Error:
    dependency_failed: Not starting as dependency failed
  • Action:
    plot_cumulative_incidence
    Status:
    Status: Failed
    Job identifier:
    nf2e2z3z5zmrzmcv
    Error:
    dependency_failed: Not starting as dependency failed
  • Action:
    table1
    Status:
    Status: Failed
    Job identifier:
    h242ayqnzqz66o5k
    Error:
    dependency_failed: Not starting as dependency failed
  • Action:
    data_tte_process_both
    Status:
    Status: Failed
    Job identifier:
    4kkuwclruiznj2j3
    Error:
    dependency_failed: Not starting as dependency failed
  • Action:
    generate_covs_data
    Status:
    Status: Failed
    Job identifier:
    mj7dm2xwk2drv3ix
    Error:
    dependency_failed: Not starting as dependency failed
  • Action:
    data_covariates_process
    Status:
    Status: Failed
    Job identifier:
    7mgse77rgtqbsqoz
    Error:
    dependency_failed: Not starting as dependency failed
  • Action:
    data_eligible_cde
    Status:
    Status: Failed
    Job identifier:
    7tb4nmq5dorohhk4
    Error:
    dependency_failed: Not starting as dependency failed
  • Action:
    data_tte_process_ChAdOx1
    Status:
    Status: Failed
    Job identifier:
    i2bdoeillp5fcpww
    Error:
    dependency_failed: Not starting as dependency failed
  • Action:
    data_eligible_ab
    Status:
    Status: Failed
    Job identifier:
    ef3qcyirjmxrg4ab
    Error:
    cancelled_by_user: Cancelled by user
  • Action:
    plot_2nd_vax_dates
    Status:
    Status: Failed
    Job identifier:
    6kiiomvkevmw3whj
    Error:
    dependency_failed: Not starting as dependency failed

Pipeline

Show project.yaml
version: '3.0'

expectations:

  population_size: 100000

actions:

  ## # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # 
  ## DO NOT EDIT project.yaml 
  ## This file is created by create-project.R 
  ## Edit and run create-project.R to update the project.yaml 
  ## # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # 
  ## #################################### 
  ## study definition 
  ## #################################### 
  ## generate dummy data for study_definition 

  dummy_data_vax:
    run: r:latest analysis/dummy_data_vax.R
    outputs:
      moderately_sensitive:
        dummy_data: analysis/dummy_data_vax.feather

  ## study definition 

  generate_study_population:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_vax
      --output-format feather
    dummy_data_file: analysis/dummy_data_vax.feather
    needs:
    - dummy_data_vax
    outputs:
      highly_sensitive:
        cohort: output/input_vax.feather

  ## #################################### 
  ## preprocessing 
  ## #################################### 
  ## process data from study_definition 

  data_input_process:
    run: r:latest analysis/preprocess/data_input_process.R
    needs:
    - dummy_data_vax
    - generate_study_population
    outputs:
      highly_sensitive:
        data_wide_vax_dates: output/data/data_wide_vax_dates.rds
        data_processed: output/data/data_processed.rds
      moderately_sensitive:
        data_properties: output/tables/data_*_tabulate.txt

  ## apply eligiblity criteria from boxes a and b 

  data_eligible_ab:
    run: r:latest analysis/preprocess/data_eligible_ab.R
    needs:
    - data_input_process
    outputs:
      highly_sensitive:
        data_eligible_a: output/data/data_eligible_a.rds
        data_eligible_b: output/data/data_eligible_b.rds
      moderately_sensitive:
        eligibility_count_ab: output/tables/eligibility_count_ab.csv
        group_age_ranges: output/lib/group_age_ranges.csv

  ## #################################### 
  ## second_vax_period 
  ## #################################### 
  ## identify second vaccination time periods 
  ## create dataset for identifying second vaccination time periods 

  data_2nd_vax_dates:
    run: r:latest analysis/second_vax_period/data_2nd_vax_dates.R
    needs:
    - data_input_process
    - data_eligible_ab
    outputs:
      highly_sensitive:
        data_vax_plot: output/second_vax_period/data/data_vax_plot.rds
        second_vax_period_dates_rds: output/second_vax_period/data/second_vax_period_dates.rds
      moderately_sensitive:
        second_vax_period_dates_txt: output/second_vax_period/tables/second_vax_period_dates.txt

  ## plot second vaccination time periods 

  plot_2nd_vax_dates:
    run: r:latest analysis/second_vax_period/plot_2nd_vax_dates.R
    needs:
    - data_eligible_ab
    - data_2nd_vax_dates
    outputs:
      moderately_sensitive:
        plots_by_region: output/second_vax_period/images/plot_by_region_*.png

  ## apply eligiblity criteria from boxes c, d and e 

  data_eligible_cde:
    run: r:latest analysis/preprocess/data_eligible_cde.R
    needs:
    - data_input_process
    - data_eligible_ab
    - data_2nd_vax_dates
    outputs:
      highly_sensitive:
        data_eligible_e_vax: output/data/data_eligible_e_vax.rds
        data_eligible_e_unvax: output/data/data_eligible_e_unvax.rds
        data_eligible_e: output/data/data_eligible_e.csv
      moderately_sensitive:
        eligibility_count_cde: output/tables/eligibility_count_cde.csv

  ## #################################### 
  ## study definition covs 
  ## #################################### 

  generate_covs_data:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_covs
      --output-format feather
    needs:
    - data_eligible_cde
    outputs:
      highly_sensitive:
        cohort: output/input_covs.feather

  ## #################################### 
  ## process covariates data 
  ## #################################### 
  ## (includes anytest_date) 

  data_covariates_process:
    run: r:latest analysis/preprocess/data_covariates_process.R
    needs:
    - data_input_process
    - data_eligible_cde
    - generate_covs_data
    outputs:
      highly_sensitive:
        data_covariates: output/data/data_all.rds

  ## min and max follow-up dates for plots 

  data_min_max_fu:
    run: r:latest analysis/comparisons/data_min_max_fu.R
    needs:
    - data_covariates_process
    outputs:
      moderately_sensitive:
        data_min_max_fu_csv: output/lib/data_min_max_fu.csv

  ## #################################### 
  ## subsequent vaccination 
  ## #################################### 
  ## plot cumulative incidence of subsequent vaccination 

  plot_cumulative_incidence:
    run: r:latest analysis/subsequent_vax/plot_cumulative_incidence.R
    needs:
    - data_covariates_process
    outputs:
      moderately_sensitive:
        ci_vax: output/subsequent_vax/images/ci_vax.png
        survtable: output/subsequent_vax/tables/survtable_redacted.csv

  ## #################################### 
  ## table 1 for report 
  ## #################################### 
  ## create table 1 for all and for each subgroup 

  table1:
    run: r:latest analysis/report/table1.R
    needs:
    - data_covariates_process
    outputs:
      moderately_sensitive:
        eligibility_count_p1: output/tables/eligibility_count_p1.csv
        table_csv: output/report/tables/table1_*_REDACTED.csv
        table_html: output/report/tables/table1_*_REDACTED.html

  ## #################################### 
  ## process time to event data 
  ## #################################### 
  ## process tte data 

  data_tte_process_BNT162b2:
    run: r:latest analysis/comparisons/data_tte_process.R BNT162b2
    needs:
    - data_covariates_process
    outputs:
      highly_sensitive:
        data_tte_brand_outcome: output/tte/data/data_tte_BNT162b2*.rds
      moderately_sensitive:
        event_counts: output/tte/tables/event_counts_BNT162b2.csv

  data_tte_process_ChAdOx1:
    run: r:latest analysis/comparisons/data_tte_process.R ChAdOx1
    needs:
    - data_covariates_process
    outputs:
      highly_sensitive:
        data_tte_brand_outcome: output/tte/data/data_tte_ChAdOx1*.rds
      moderately_sensitive:
        event_counts: output/tte/tables/event_counts_ChAdOx1.csv

  data_tte_process_both:
    run: r:latest analysis/comparisons/data_tte_process.R both
    needs:
    - data_covariates_process
    outputs:
      highly_sensitive:
        data_tte_brand_outcome: output/tte/data/data_tte_both*.rds
      moderately_sensitive:
        event_counts: output/tte/tables/event_counts_both.csv

Timeline

  • Created:

  • Started:

  • Finished:

  • Runtime: 06:46:01

These timestamps are generated and stored using the UTC timezone on the TPP backend.

Job request

Status
Failed
Backend
TPP
Requested by
Elsie Horne
Branch
define-covs-at-svp
Force run dependencies
Yes
Git commit hash
02b9ca2
Requested actions
  • run_all

Code comparison

Compare the code used in this job request

  • No previous job request available for comparison