Job request: 2888

Organisation:: Bennett Institute
Workspace:: comparative-ve-research_main
ID:: qammykcrul324y2o

This page shows the technical details of what happened when the authorised researcher Will Hulme requested one or more actions to be run against real patient data within a secure environment.

By cross-referencing the list of jobs with the pipeline section below, you can infer what security level the outputs were written to.

The output security levels are:

highly_sensitive
- Researchers can never directly view these outputs
- Researchers can only request code is run against them
moderately_sensitive
- Can be viewed by an approved researcher by logging into a highly secure environment
- These are the only outputs that can be requested for public release via a controlled output review service.

Jobs

Action:

hcw_extract

Status:

Status: Succeeded

Job identifier:

6ysa4bclp3fpkdvx
Action:

design

Status:

Status: Succeeded

Job identifier:

xzfezvc6ir3nrfpe
Action:

extract

Status:

Status: Succeeded

Job identifier:

5bd3uyiydwzi7fpj
Action:

hcw_process

Status:

Status: Succeeded

Job identifier:

q5i2cbgiukw3mic7
Action:

hcw_properties

Status:

Status: Succeeded

Job identifier:

5hdkxcg4qsoqhyjj
Action:

hcw_descr

Status:

Status: Succeeded

Job identifier:

5jvxvh4hscchpda6
Action:

data_process

Status:

Status: Succeeded

Job identifier:

zwtrafuf5m32eow5
Action:

data_selection

Status:

Status: Succeeded

Job identifier:

qrqymohd2jxwimdf
Action:

data_properties

Status:

Status: Succeeded

Job identifier:

by645qgsechwgdop
Action:

model_postest_timesincevax

Status:

Status: Succeeded

Job identifier:

ymz3gphqc2tz47xt
Action:

model_emergency_timesincevax

Status:

Status: Succeeded

Job identifier:

7jjkf5yxntzemydw
Action:

model_test_timesincevax

Status:

Status: Succeeded

Job identifier:

j6q4krvbj4uheem4
Action:

model_postest_calendar

Status:

Status: Succeeded

Job identifier:

5t6tirn5zb4k2zna
Action:

model_test_calendar

Status:

Status: Succeeded

Job identifier:

yrvoiysvg7dz6goy
Action:

descr_km

Status:

Status: Failed

Job identifier:

stpyszvxjrhpkt5a

Error:

nonzero_exit: Job exited with an error code
Action:

model_covidadmitted_timesincevax

Status:

Status: Succeeded

Job identifier:

2ejncwc2x5qgg3oa
Action:

descr_table1_allvax

Status:

Status: Succeeded

Job identifier:

jeixaupb5icyqvlt
Action:

model_emergency_calendar

Status:

Status: Succeeded

Job identifier:

z5icpxo5epmvxh3o
Action:

model_covidadmitted_calendar

Status:

Status: Succeeded

Job identifier:

est6ku44meqxxfur
Action:

descr_irr

Status:

Status: Succeeded

Job identifier:

zcdt533kczgzd72c
Action:

descr_table1

Status:

Status: Succeeded

Job identifier:

65b6axehwmb5ixv2
Action:

report_postest_calendar

Status:

Status: Succeeded

Job identifier:

47yneautux6xhuvh
Action:

report_emergency_calendar

Status:

Status: Succeeded

Job identifier:

qxf7qpxbae3iuv54
Action:

report_emergency_timesincevax

Status:

Status: Succeeded

Job identifier:

enh5zz3egajga7ri
Action:

report_test_calendar

Status:

Status: Succeeded

Job identifier:

cjrouc7235a3r77g
Action:

report_postest_timesincevax

Status:

Status: Succeeded

Job identifier:

mcadbo6ecl7cwvnj
Action:

report_covidadmitted_calendar

Status:

Status: Succeeded

Job identifier:

nrq6lnt2cmcc3uwd
Action:

report_covidadmitted_timesincevax

Status:

Status: Succeeded

Job identifier:

vii6sbpiqo7dxn2z
Action:

report_test_timesincevax

Status:

Status: Succeeded

Job identifier:

fdiiiw3hz3i55ai7

Pipeline

Show project.yaml

version: '3.0'

expectations:

  population_size: 100000

actions:

  hcw_extract:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_hcw
      --output-format feather
    outputs:
      highly_sensitive:
        cohort: output/input_hcw.feather

  hcw_process:
    run: r:latest analysis/hcw/process.R
    needs:
    - hcw_extract
    outputs:
      highly_sensitive:
        data: output/data/hcw_data_processed.rds
        datavax: output/data/hcw_data_vax.rds

  hcw_properties:
    run: r:latest analysis/process/data_properties.R output/data/hcw_data_processed.rds
      output/data_properties
    needs:
    - hcw_process
    outputs:
      moderately_sensitive:
        cohort: output/data_properties/hcw_data_processed*.txt

  hcw_descr:
    run: r:latest analysis/hcw/descr.R
    needs:
    - hcw_process
    outputs:
      moderately_sensitive:
        png: output/hcw/*.png
        svg: output/hcw/*.svg
        html: output/hcw/*.html
        csv: output/hcw/*.csv

  design:
    run: r:latest analysis/process/design.R
    outputs:
      moderately_sensitive:
        metadata: output/data/metadata*

  extract:
    run: cohortextractor:latest generate_cohort --study-definition study_definition
      --output-format feather
    outputs:
      highly_sensitive:
        cohort: output/input.feather

  data_process:
    run: r:latest analysis/process/data_process.R
    needs:
    - extract
    outputs:
      highly_sensitive:
        cohort: output/data/data_processed.rds

  data_properties:
    run: r:latest analysis/process/data_properties.R output/data/data_processed.rds
      output/data_properties
    needs:
    - data_process
    outputs:
      highly_sensitive:
        cohort: output/data_properties/data_processed*.txt

  data_selection:
    run: r:latest analysis/process/data_selection.R
    needs:
    - data_process
    outputs:
      highly_sensitive:
        data_allvax: output/data/data_cohort_allvax.rds
        data: output/data/data_cohort.rds
      moderately_sensitive:
        flow: output/data/flowchart.csv

  descr_table1:
    run: r:latest analysis/descriptive/table1.R
    needs:
    - design
    - data_selection
    outputs:
      moderately_sensitive:
        html: output/descriptive/tables/table1*.html
        csv: output/descriptive/tables/table1*.csv

  descr_table1_allvax:
    run: r:latest analysis/descriptive/table1_allvax.R
    needs:
    - design
    - data_selection
    outputs:
      moderately_sensitive:
        html: output/descriptive/tables/table1_allvax.html
        csv: output/descriptive/tables/table1_allvax.csv

  descr_irr:
    run: r:latest analysis/descriptive/table_irr.R output/data/data_processed.rds
      output/data_properties
    needs:
    - design
    - data_selection
    outputs:
      moderately_sensitive:
        html: output/descriptive/tables/table_irr.html
        csv: output/descriptive/tables/table_irr.csv

  descr_km:
    run: r:latest analysis/descriptive/km.R output/data/data_processed.rds output/data_properties
    needs:
    - design
    - data_selection
    outputs:
      moderately_sensitive:
        png: output/descriptive/km/plot_survival*.png
        svg: output/descriptive/km/plot_survival*.svg

  model_test_timesincevax:
    run: r:latest analysis/models/model_cox.R test timesincevax
    needs:
    - design
    - data_selection
    outputs:
      highly_sensitive:
        data: output/test/timesincevax/data_cox_split.rds
        models: output/test/timesincevax/model*.rds
      moderately_sensitive:
        logs: output/test/timesincevax/log*.txt
        glance: output/test/timesincevax/glance*.csv

  report_test_timesincevax:
    run: r:latest analysis/models/report_cox.R test timesincevax
    needs:
    - design
    - model_test_timesincevax
    outputs:
      moderately_sensitive:
        svg: output/test/timesincevax/forest*.svg
        png: output/test/timesincevax/forest*.png
        csv: output/test/timesincevax/estimates*.csv

  model_test_calendar:
    run: r:latest analysis/models/model_cox.R test calendar
    needs:
    - design
    - data_selection
    outputs:
      highly_sensitive:
        data: output/test/calendar/data_cox_split.rds
        models: output/test/calendar/model*.rds
      moderately_sensitive:
        logs: output/test/calendar/log*.txt
        glance: output/test/calendar/glance*.csv

  report_test_calendar:
    run: r:latest analysis/models/report_cox.R test calendar
    needs:
    - design
    - model_test_calendar
    outputs:
      moderately_sensitive:
        svg: output/test/calendar/forest*.svg
        png: output/test/calendar/forest*.png
        csv: output/test/calendar/estimates*.csv

  model_postest_timesincevax:
    run: r:latest analysis/models/model_cox.R postest timesincevax
    needs:
    - design
    - data_selection
    outputs:
      highly_sensitive:
        data: output/postest/timesincevax/data_cox_split.rds
        models: output/postest/timesincevax/model*.rds
      moderately_sensitive:
        logs: output/postest/timesincevax/log*.txt
        glance: output/postest/timesincevax/glance*.csv

  report_postest_timesincevax:
    run: r:latest analysis/models/report_cox.R postest timesincevax
    needs:
    - design
    - model_postest_timesincevax
    outputs:
      moderately_sensitive:
        svg: output/postest/timesincevax/forest*.svg
        png: output/postest/timesincevax/forest*.png
        csv: output/postest/timesincevax/estimates*.csv

  model_postest_calendar:
    run: r:latest analysis/models/model_cox.R postest calendar
    needs:
    - design
    - data_selection
    outputs:
      highly_sensitive:
        data: output/postest/calendar/data_cox_split.rds
        models: output/postest/calendar/model*.rds
      moderately_sensitive:
        logs: output/postest/calendar/log*.txt
        glance: output/postest/calendar/glance*.csv

  report_postest_calendar:
    run: r:latest analysis/models/report_cox.R postest calendar
    needs:
    - design
    - model_postest_calendar
    outputs:
      moderately_sensitive:
        svg: output/postest/calendar/forest*.svg
        png: output/postest/calendar/forest*.png
        csv: output/postest/calendar/estimates*.csv

  model_emergency_timesincevax:
    run: r:latest analysis/models/model_cox.R emergency timesincevax
    needs:
    - design
    - data_selection
    outputs:
      highly_sensitive:
        data: output/emergency/timesincevax/data_cox_split.rds
        models: output/emergency/timesincevax/model*.rds
      moderately_sensitive:
        logs: output/emergency/timesincevax/log*.txt
        glance: output/emergency/timesincevax/glance*.csv

  report_emergency_timesincevax:
    run: r:latest analysis/models/report_cox.R emergency timesincevax
    needs:
    - design
    - model_emergency_timesincevax
    outputs:
      moderately_sensitive:
        svg: output/emergency/timesincevax/forest*.svg
        png: output/emergency/timesincevax/forest*.png
        csv: output/emergency/timesincevax/estimates*.csv

  model_emergency_calendar:
    run: r:latest analysis/models/model_cox.R emergency calendar
    needs:
    - design
    - data_selection
    outputs:
      highly_sensitive:
        data: output/emergency/calendar/data_cox_split.rds
        models: output/emergency/calendar/model*.rds
      moderately_sensitive:
        logs: output/emergency/calendar/log*.txt
        glance: output/emergency/calendar/glance*.csv

  report_emergency_calendar:
    run: r:latest analysis/models/report_cox.R emergency calendar
    needs:
    - design
    - model_emergency_calendar
    outputs:
      moderately_sensitive:
        svg: output/emergency/calendar/forest*.svg
        png: output/emergency/calendar/forest*.png
        csv: output/emergency/calendar/estimates*.csv

  model_covidadmitted_timesincevax:
    run: r:latest analysis/models/model_cox.R covidadmitted timesincevax
    needs:
    - design
    - data_selection
    outputs:
      highly_sensitive:
        data: output/covidadmitted/timesincevax/data_cox_split.rds
        models: output/covidadmitted/timesincevax/model*.rds
      moderately_sensitive:
        logs: output/covidadmitted/timesincevax/log*.txt
        glance: output/covidadmitted/timesincevax/glance*.csv

  report_covidadmitted_timesincevax:
    run: r:latest analysis/models/report_cox.R covidadmitted timesincevax
    needs:
    - design
    - model_covidadmitted_timesincevax
    outputs:
      moderately_sensitive:
        svg: output/covidadmitted/timesincevax/forest*.svg
        png: output/covidadmitted/timesincevax/forest*.png
        csv: output/covidadmitted/timesincevax/estimates*.csv

  model_covidadmitted_calendar:
    run: r:latest analysis/models/model_cox.R covidadmitted calendar
    needs:
    - design
    - data_selection
    outputs:
      highly_sensitive:
        data: output/covidadmitted/calendar/data_cox_split.rds
        models: output/covidadmitted/calendar/model*.rds
      moderately_sensitive:
        logs: output/covidadmitted/calendar/log*.txt
        glance: output/covidadmitted/calendar/glance*.csv

  report_covidadmitted_calendar:
    run: r:latest analysis/models/report_cox.R covidadmitted calendar
    needs:
    - design
    - model_covidadmitted_calendar
    outputs:
      moderately_sensitive:
        svg: output/covidadmitted/calendar/forest*.svg
        png: output/covidadmitted/calendar/forest*.png
        csv: output/covidadmitted/calendar/estimates*.csv

Timeline

Created: 4 years, 3 months ago 14 Jul 2021 15:24:30 UTC
Started: 4 years, 3 months ago 14 Jul 2021 15:25:07 UTC
Finished: 4 years, 3 months ago 14 Jul 2021 20:53:33 UTC
Runtime: 10:15:47

These timestamps are generated and stored using the UTC timezone on the TPP backend.

Job request

Status: Failed
Backend: TPP
Workspace: comparative-ve-research_main
Requested by: Will Hulme
Branch: main
Force run dependencies: Yes
Git commit hash: fdacaf7
Requested actions: run_all

Code comparison

Compare the code used in this job request