Job request: 1582

Organisation:: Bennett Institute
Workspace:: ncs-hospital-disruption-2
ID:: p6uft6fx6uxve5bq

This page shows the technical details of what happened when the authorised researcher Louis Fisher requested one or more actions to be run against real patient data within a secure environment.

By cross-referencing the list of jobs with the pipeline section below, you can infer what security level the outputs were written to.

The output security levels are:

highly_sensitive
- Researchers can never directly view these outputs
- Researchers can only request code is run against them
moderately_sensitive
- Can be viewed by an approved researcher by logging into a highly secure environment
- These are the only outputs that can be requested for public release via a controlled output review service.

Jobs

Action:

generate_cohort_5

Status:

Status: Succeeded

Job identifier:

qgxrsbcywbe5unat
Action:

generate_cohort_2

Status:

Status: Succeeded

Job identifier:

qnloie3tr3y2wuog
Action:

generate_cohort_1

Status:

Status: Succeeded

Job identifier:

bi7xkoscz3kznfri
Action:

generate_cohort_4

Status:

Status: Succeeded

Job identifier:

xigr33ctgneskfnb
Action:

generate_cohort_3

Status:

Status: Succeeded

Job identifier:

aumyzqmricipo7dt
Action:

generate_cohort_imd_ethnicity

Status:

Status: Succeeded

Job identifier:

of2utvzlsbkb4ucr
Action:

join_ethnicity_imd

Status:

Status: Succeeded

Job identifier:

4sqelgaj5zbb6zal
Action:

calculate_measures

Status:

Status: Succeeded

Job identifier:

lyro2owh36deflxh
Action:

calculate_measures_demographics

Status:

Status: Succeeded

Job identifier:

opc7gvzep7xpqlg5
Action:

standardise_and_plot

Status:

Status: Succeeded

Job identifier:

zd6me4beeirlvz3z
Action:

standardise_and_plot_demographics

Status:

Status: Succeeded

Job identifier:

ehflgwkfpedgkycu

Pipeline

Show project.yaml

version: '3.0'

expectations:
  population_size: 10000

actions:


  generate_cohort_imd_ethnicity:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_imd_ethnicity
    outputs:
      highly_sensitive:
        cohort: output/input_imd_ethnicity.csv
  

  generate_cohort_1:
    # week 1 2019 to 21 2019 inclusive
    run: cohortextractor:latest generate_cohort --study-definition study_definition --index-date-range "2018-12-31 to 2019-05-26 by week"
    outputs:
      highly_sensitive:
        cohort: output/input_*.csv

  generate_cohort_2:
    # week 22 2019 to 42 2019 inclusive
    run: cohortextractor:latest generate_cohort --study-definition study_definition --index-date-range "2019-05-27 to 2019-10-20 by week"
    outputs:
      highly_sensitive:
        cohort: output/input*.csv

  generate_cohort_3:
    # week  43 2019 to 11 2020 inclusive
    run: cohortextractor:latest generate_cohort --study-definition study_definition --index-date-range "2019-10-21 to 2020-03-15 by week"
    outputs:
      highly_sensitive:
        cohort: output/inpu*.csv

  generate_cohort_4:
    # week 12 2020 to 32 2020 inclusive
    run: cohortextractor:latest generate_cohort --study-definition study_definition --index-date-range "2020-03-16 to 2020-08-09 by week"
    outputs:
      highly_sensitive:
        cohort: output/inp*.csv

  generate_cohort_5:
    # week 33 2020 to week 53 2020 inclusive
    run: cohortextractor:latest generate_cohort --study-definition study_definition --index-date-range "2020-08-10 to 2021-01-03 by week"
    outputs:
      highly_sensitive:
        cohort: output/in*.csv

  join_ethnicity_imd:
    run: python:latest python analysis/join_ethnicity_imd.py
    needs: [generate_cohort_1, generate_cohort_2, generate_cohort_3, generate_cohort_4, generate_cohort_5, generate_cohort_imd_ethnicity]
    outputs:
      highly_sensitive:
        cohort: output/i*.csv
   
  calculate_measures:
    run: cohortextractor:latest generate_measures --study-definition study_definition
    needs: [join_ethnicity_imd]
    outputs:
      moderately_sensitive:
        measure: output/measure_*_rate.csv

  calculate_measures_demographics:
    run: python:latest python analysis/generate_measures_demographics.py
    needs: [join_ethnicity_imd]
    outputs:
      moderately_sensitive:
        csvs: output/measure_*_rate_*.csv
        

  standardise_and_plot:
    run: python:latest python analysis/time_series_plots.py
    needs: [calculate_measures]
    outputs:
      moderately_sensitive:
        tables: output/*_rate.csv
        graph: output/time_series_plot.svg

  standardise_and_plot_demographics:
    run: python:latest python analysis/time_series_plots_demographics.py
    needs: [calculate_measures_demographics]
    outputs:
      moderately_sensitive:
        tables: output/*_rate_*_breakdown.csv
        tables_combined: output/combined_disease_breakdown_*.csv
        graph: output/time_series_plot_*.svg
        combined_graph: output/combined_diseases_time_series.svg

Timeline

Created: 4 years, 6 months ago 09 Apr 2021 15:32:43 UTC
Started: 4 years, 6 months ago 09 Apr 2021 15:33:19 UTC
Finished: 4 years, 6 months ago 10 Apr 2021 07:11:51 UTC
Runtime: 29:11:16

These timestamps are generated and stored using the UTC timezone on the TPP backend.

Job request

Status: Succeeded
Backend: TPP
Workspace: ncs-hospital-disruption-2
Requested by: Louis Fisher
Branch: icd-10-fix
Force run dependencies: Yes
Git commit hash: 5135658
Requested actions: run_all

Code comparison

Compare the code used in this job request