Job Request: 2574 pvy3abb5lzq4dzz4

This page shows the technical details of what happened when authorised researcher HelenCEBM requested one or more actions to be run against real patient data in the Surgery Outcomes project, within a secure environment.

By cross-referencing the indicated Requested Actions with the Pipeline section below, you can infer what security level various outputs were written to. Outputs marked as highly_sensitive can never be viewed directly by a researcher; they can only request that code runs against them. Outputs marked as moderately_sensitive can be viewed by an approved researcher by logging into a highly secure environment. Only outputs marked as moderately_sensitive can be requested for release to the public, via a controlled output review service.

State

State is inferred from the related Jobs
Status: succeeded

Config

Backend: tpp
Workspace: surgery-outcomes-full (master)
Creator: HelenCEBM
Force run dependencies?: False
Git Commit Hash: e22cb13
Requested Actions:
  • logistic_regression

Timings

Created: 1 month, 2 weeks ago
Started: 1 month, 2 weeks ago
Finished: 1 month, 2 weeks ago
Runtime: 00:00:12

Pipeline

version: '3.0'

expectations:
population_size: 5000

actions:

generate_study_population:
run: cohortextractor:latest generate_cohort --study-definition study_definition --index-date-range "2020-04-01 to 2020-04-01 by month"
outputs:
highly_sensitive:
cohort: output/input_2020*.csv


generate_study_population_2019:
run: cohortextractor:latest generate_cohort --study-definition study_definition --index-date-range "2019-04-01 to 2019-04-01 by month"
outputs:
highly_sensitive:
cohort: output/input_2019*.csv

filter_population_for_matching:
run: python:latest python analysis/filter_population.py
needs: [generate_study_population]
outputs:
highly_sensitive:
filtered_cohort: output/filtered_2020_for_matching.csv
covid_cohort: output/filtered_2020_covid_positive.csv

matching:
run: python:latest python analysis/match_running.py
needs: [generate_study_population, filter_population_for_matching, generate_study_population_2019]
outputs:
moderately_sensitive:
matching_report: output/matching_report_2019.txt
highly_sensitive:
matched_cohort: output/matched_matches_2019.csv

generate_notebook_data_checks:
run: jupyter:latest jupyter nbconvert /workspace/notebooks/data_checks.ipynb --execute --to html --output-dir=/workspace/output --ExecutePreprocessor.timeout=86400
needs: [filter_population_for_matching]
outputs:
moderately_sensitive:
notebook: output/data_checks.html

generate_notebook:
run: jupyter:latest jupyter nbconvert /workspace/notebooks/analysis.ipynb --execute --to html --output-dir=/workspace/output --ExecutePreprocessor.timeout=86400
needs: [generate_study_population_2019, filter_population_for_matching, matching]
outputs:
moderately_sensitive:
notebook: output/analysis.html
baseline_char: output/baseline*
counts: output/patient_count*
outcomes_summary: output/summary*
outcomes_detailed: output/detailed*
outcomes_mortality: output/mortality*

logistic_regression:
run: python:latest python analysis/logistic_regression.py
needs: [generate_study_population]
outputs:
moderately_sensitive:
#odds_ratios: output/odds_ratios.csv
confidence_intervals: output/confidence_intervals.csv

Jobs

ID Status Action
b7ipjh27hgdelpbz succeeded logistic_regression View