Skip to content

Job request: 4379

Organisation:
Bennett Institute
Workspace:
pincer-measures
ID:
nizj2zka6uqdvvu7

This page shows the technical details of what happened when the authorised researcher Louis Fisher requested one or more actions to be run against real patient data within a secure environment.

By cross-referencing the list of jobs with the pipeline section below, you can infer what security level the outputs were written to.

The output security levels are:

  • highly_sensitive
    • Researchers can never directly view these outputs
    • Researchers can only request code is run against them
  • moderately_sensitive
    • Can be viewed by an approved researcher by logging into a highly secure environment
    • These are the only outputs that can be requested for public release via a controlled output review service.

Jobs

Pipeline

Show project.yaml
version: '3.0'

expectations:
  population_size: 1000

actions:

  generate_study_population_test:
    run: cohortextractor:latest generate_cohort --study-definition study_definition --index-date-range "2019-09-01 to 2019-09-01 by month" --output-format feather
    outputs:
      highly_sensitive:
        cohort: output/input_*.feather
  
  check_test:
    run: python:latest python analysis/test_study_def.py
    needs: [generate_study_population_test]
    outputs:
      moderately_sensitive:
        egfr: output/EGFR_*.csv


  # generate_study_population_1:
  #   run: cohortextractor:latest generate_cohort --study-definition study_definition --index-date-range "2019-09-01 to 2020-05-01 by month" --output-format feather
  #   outputs:
  #     highly_sensitive:
  #       cohort: output/input_*.feather

  # generate_study_population_2:
  #   run: cohortextractor:latest generate_cohort --study-definition study_definition --index-date-range "2020-06-01 to 2021-02-01 by month" --output-format feather
  #   outputs:
  #     highly_sensitive:
  #       cohort: output/input*.feather

  # generate_study_population_3:
  #   run: cohortextractor:latest generate_cohort --study-definition study_definition --index-date-range "2021-03-01 to 2021-09-01 by month" --output-format feather
  #   outputs:
  #     highly_sensitive:
  #       cohort: output/inpu*.feather

  # generate_study_population_ethnicity:
  #   run: cohortextractor:latest generate_cohort --study-definition study_definition_ethnicity --output-format feather
  #   outputs:
  #     highly_sensitive:
  #       cohort: output/input_ethnicity.feather
 
  # join_ethnicity_region:
  #   run: python:latest python analysis/join_ethnicity_region.py
  #   needs: [generate_study_population_1, generate_study_population_2, generate_study_population_3, generate_study_population_ethnicity]
  #   outputs:
  #     highly_sensitive:
  #       cohort: output/inp*.feather
          

  # calculate_numerators:
  #   run: python:latest python analysis/calculate_numerators.py
  #   needs: [join_ethnicity_region]
  #   outputs:
  #     highly_sensitive:
  #       cohort: output/indicator_e_f_*.feather

  # calculate_composite_indicators:
  #   run: python:latest python analysis/composite_indicators.py
  #   needs: [calculate_numerators, join_ethnicity_region]
  #   outputs:
  #     moderately_sensitive:
  #       counts: output/*_composite_measure.csv

  # generate_measures:
  #   run: cohortextractor:latest generate_measures --study-definition study_definition --output-dir=output
  #   needs: [join_ethnicity_region]
  #   outputs:
  #       moderately_sensitive:
  #         measure_csv: output/measure_*_rate.csv

  
  # generate_measures_demographics:
  #   run: python:latest python analysis/calculate_measures.py
  #   needs: [calculate_numerators, join_ethnicity_region]
  #   outputs:
  #     moderately_sensitive:
  #       counts: output/indicator_measure_*.csv
  #       measure_csv: output/measure*_rate.csv
  #       demographics: output/demographics_summary_*.csv
        

  # generate_summary_counts:
  #   run: python:latest python analysis/summary_statistics.py
  #   needs: [join_ethnicity_region, generate_measures, generate_measures_demographics, calculate_numerators]
  #   outputs:
  #     moderately_sensitive:
  #       patient_count: output/patient_count_*.json
  #       practice_count: output/practice_count_*.json
  #       summary: output/indicator_summary_statistics_*.json
  
  # generate_plots:
  #   run: python:latest python analysis/plot_measures.py
  #   needs: [generate_measures, generate_measures_demographics, calculate_composite_indicators]
  #   outputs:
  #     moderately_sensitive:
  #       counts: output/figures/plot_*.jpeg
  #       combined: output/figures/combined_plot_*.png
  #       demographics: output/demographic_aggregates.csv
  
  # generate_notebook:
  #   run: jupyter:latest jupyter nbconvert /workspace/analysis/report.ipynb --execute --to html --template basic --output-dir=/workspace/output --ExecutePreprocessor.timeout=86400 --no-input
  #   needs: [generate_plots, generate_summary_counts]
  #   outputs:
  #     moderately_sensitive:
  #       notebook: output/report.html

  # generate_dem_notebook:
  #   run: jupyter:latest jupyter nbconvert /workspace/analysis/demographic_report.ipynb --execute --to html --template basic --output-dir=/workspace/output --ExecutePreprocessor.timeout=86400 --no-input
  #   needs: [generate_plots]
  #   outputs:
  #     moderately_sensitive:
  #       notebook: output/demographic_report.html

  # run_tests:
  #   run: python:latest python -m pytest --junit-xml=output/pytest.xml --verbose
  #   outputs:
  #     moderately_sensitive:
  #       log: output/pytest.xml

Timeline

  • Created:

  • Started:

  • Finished:

  • Runtime: 01:17:29

These timestamps are generated and stored using the UTC timezone on the TPP backend.

Job request

Status
Succeeded
Backend
TPP
Workspace
pincer-measures
Requested by
Louis Fisher
Branch
main
Force run dependencies
No
Git commit hash
791e07b
Requested actions
  • generate_study_population_test
  • check_test

Code comparison

Compare the code used in this job request