Job request: 17379

Organisation:: Bennett Institute
Workspace:: the-impact-of-covid-19-on-the-care-of-people-with-sickle-cell-disease-interactive
ID:: dr6pcafuxkqa3ywo

This page shows the technical details of what happened when the authorised researcher Brian MacKenna (PHC) requested one or more actions to be run against real patient data within a secure environment.

By cross-referencing the list of jobs with the pipeline section below, you can infer what security level the outputs were written to.

The output security levels are:

highly_sensitive
- Researchers can never directly view these outputs
- Researchers can only request code is run against them
moderately_sensitive
- Can be viewed by an approved researcher by logging into a highly secure environment
- These are the only outputs that can be requested for public release via a controlled output review service.

Jobs

Action:

generate_study_population_01GZ0RWS3KFFC34P9RSW6YBFGP

Status:

Status: Failed

Job identifier:

xdrpzz5bks6qs3qz

Error:

nonzero_exit: Job exited with an error
Action:

generate_study_population_weekly_01GZ0RWS3KFFC34P9RSW6YBFGP

Status:

Status: Failed

Job identifier:

eppy33xo2dlkucqx

Error:

nonzero_exit: Job exited with an error
Action:

generate_study_population_ethnicity_01GZ0RWS3KFFC34P9RSW6YBFGP

Status:

Status: Succeeded

Job identifier:

wrfzrfl2sts2qxpl
Action:

generate_measures_01GZ0RWS3KFFC34P9RSW6YBFGP

Status:

Status: Failed

Job identifier:

sefngqyf37yaus6m

Error:

dependency_failed: Not starting as dependency failed
Action:

join_cohorts_01GZ0RWS3KFFC34P9RSW6YBFGP

Status:

Status: Failed

Job identifier:

bfy2mhpcpko236a3

Error:

dependency_failed: Not starting as dependency failed
Action:

plot_measure_01GZ0RWS3KFFC34P9RSW6YBFGP

Status:

Status: Failed

Job identifier:

gapbqbgyadxmr6qz

Error:

dependency_failed: Not starting as dependency failed
Action:

generate_report_01GZ0RWS3KFFC34P9RSW6YBFGP

Status:

Status: Failed

Job identifier:

aqdzdfnnybbjahd5

Error:

dependency_failed: Not starting as dependency failed
Action:

top_5_table_01GZ0RWS3KFFC34P9RSW6YBFGP

Status:

Status: Failed

Job identifier:

yaj5ziyx35hfjcfw

Error:

dependency_failed: Not starting as dependency failed
Action:

event_counts_01GZ0RWS3KFFC34P9RSW6YBFGP

Status:

Status: Failed

Job identifier:

qhj34b4rqdq2ptji

Error:

dependency_failed: Not starting as dependency failed

Pipeline

Show project.yaml

version: '3.0'

expectations:
  population_size: 1000

actions:

  generate_study_population_ethnicity_01GZ0RWS3KFFC34P9RSW6YBFGP:
    run: cohortextractor:latest generate_cohort
      --study-definition study_definition_ethnicity
      --param end_date="2023-03-31"
      --output-dir output/01GZ0RWS3KFFC34P9RSW6YBFGP --output-format=feather
    outputs:
      highly_sensitive:
        cohort: output/01GZ0RWS3KFFC34P9RSW6YBFGP/input_ethnicity.feather

  generate_study_population_weekly_01GZ0RWS3KFFC34P9RSW6YBFGP:
    run: cohortextractor:latest generate_cohort
      --study-definition study_definition
      --param codelist_1_path="interactive_codelists/codelist_1.csv"
      --param codelist_1_type="medication"
      --param codelist_2_path="interactive_codelists/codelist_2.csv"
      --param codelist_2_type="event"
      --param codelist_1_frequency="weekly"
      --param time_value="None"
      --param time_ever="True"
      --param time_scale=""
      --param time_event="before"
      --param codelist_2_comparison_date="end_date"
      --param operator="AND"
      --param population="all"
      --param breakdowns=""
      --index-date_range="2023-04-10 to 2023-04-10 by week"
      --output-dir=output/01GZ0RWS3KFFC34P9RSW6YBFGP
      --output-format=feather
      --output-file=output/01GZ0RWS3KFFC34P9RSW6YBFGP/input_weekly_2023-04-10.feather
    outputs:
      highly_sensitive:
        cohort: output/01GZ0RWS3KFFC34P9RSW6YBFGP/input_weekly_2023-04-10.feather

  generate_study_population_01GZ0RWS3KFFC34P9RSW6YBFGP:
    run: cohortextractor:latest generate_cohort
      --study-definition study_definition
      --param codelist_1_path="interactive_codelists/codelist_1.csv"
      --param codelist_1_type="medication"
      --param codelist_2_path="interactive_codelists/codelist_2.csv"
      --param codelist_2_type="event"
      --param codelist_1_frequency="monthly"
      --param time_value="None"
      --param time_ever="True"
      --param time_scale=""
      --param time_event="before"
      --param codelist_2_comparison_date="end_date"
      --param operator="AND"
      --param population="all"
      --param breakdowns="sex,age,ethnicity,imd,region"
      --index-date-range="2019-09-01 to 2023-03-31 by month"
      --output-dir=output/01GZ0RWS3KFFC34P9RSW6YBFGP
      --output-format=feather
    outputs:
      highly_sensitive:
        cohort: output/01GZ0RWS3KFFC34P9RSW6YBFGP/input_*.feather

  join_cohorts_01GZ0RWS3KFFC34P9RSW6YBFGP:
    run: >
      cohort-joiner:v0.0.38
        --lhs output/01GZ0RWS3KFFC34P9RSW6YBFGP/input_20*.feather
        --rhs output/01GZ0RWS3KFFC34P9RSW6YBFGP/input_ethnicity.feather
        --output-dir output/01GZ0RWS3KFFC34P9RSW6YBFGP/joined
    needs: [generate_study_population_01GZ0RWS3KFFC34P9RSW6YBFGP, generate_study_population_ethnicity_01GZ0RWS3KFFC34P9RSW6YBFGP]
    outputs:
      highly_sensitive:
        cohort: output/01GZ0RWS3KFFC34P9RSW6YBFGP/joined/input_20*.feather

  generate_measures_01GZ0RWS3KFFC34P9RSW6YBFGP:
    run: >
      python:latest -m analysis.measures
        --breakdowns=sex
        --breakdowns=age
        --breakdowns=ethnicity
        --breakdowns=imd
        --breakdowns=region
        --input-dir="output/01GZ0RWS3KFFC34P9RSW6YBFGP/joined"

    needs: [join_cohorts_01GZ0RWS3KFFC34P9RSW6YBFGP]
    outputs:
      moderately_sensitive:
        measure: output/01GZ0RWS3KFFC34P9RSW6YBFGP/joined/measure_all.csv
        decile_measure: output/01GZ0RWS3KFFC34P9RSW6YBFGP/joined/measure_practice_rate_deciles.csv

  top_5_table_01GZ0RWS3KFFC34P9RSW6YBFGP:
    run: >
      python:latest python analysis/top_5.py
      --codelist-1-path="interactive_codelists/codelist_1.csv"
      --codelist-2-path="interactive_codelists/codelist_2.csv"
      --output-dir="output/01GZ0RWS3KFFC34P9RSW6YBFGP"
    needs: [generate_measures_01GZ0RWS3KFFC34P9RSW6YBFGP]
    outputs:
      moderately_sensitive:
        tables: output/01GZ0RWS3KFFC34P9RSW6YBFGP/joined/top_5*.csv

  plot_measure_01GZ0RWS3KFFC34P9RSW6YBFGP:
    run: >
      python:latest python analysis/plot_measures.py
        --breakdowns=sex
        --breakdowns=age
        --breakdowns=ethnicity
        --breakdowns=imd
        --breakdowns=region
        --output-dir output/01GZ0RWS3KFFC34P9RSW6YBFGP
    needs: [generate_measures_01GZ0RWS3KFFC34P9RSW6YBFGP]
    outputs:
      moderately_sensitive:
        measure: output/01GZ0RWS3KFFC34P9RSW6YBFGP/plot_measure*.png
        deciles: output/01GZ0RWS3KFFC34P9RSW6YBFGP/deciles_chart.png

  event_counts_01GZ0RWS3KFFC34P9RSW6YBFGP:
    run: >
      python:latest -m analysis.event_counts --input-dir="output/01GZ0RWS3KFFC34P9RSW6YBFGP" --output-dir="output/01GZ0RWS3KFFC34P9RSW6YBFGP"
    needs: [join_cohorts_01GZ0RWS3KFFC34P9RSW6YBFGP, generate_study_population_weekly_01GZ0RWS3KFFC34P9RSW6YBFGP]
    outputs:
      moderately_sensitive:
        measure: output/01GZ0RWS3KFFC34P9RSW6YBFGP/event_counts.json

  generate_report_01GZ0RWS3KFFC34P9RSW6YBFGP:
    run: >
      python:latest python analysis/render_report.py
      --output-dir="output/01GZ0RWS3KFFC34P9RSW6YBFGP"
      --population="all"
      --breakdowns=sex
      --breakdowns=age
      --breakdowns=ethnicity
      --breakdowns=imd
      --breakdowns=region
      --codelist-1-name="Opioid-containing medicines (oral - excluding drugs for substance misuse) - dm+d"
      --codelist-2-name="Sickle (SPL-AtRiskv4) (SNOMED CT)"
      --codelist-1-link="opensafely/opioid-containing-medicines-oral-excluding-drugs-for-substance-misuse-dmd/6eb4a72e"
      --codelist-2-link="nhsd/sickle-spl-atriskv4-snomed-ct/7083ed37"
      --time-value="None"
      --time-scale=""
      --time-event="before"
      --start-date="2019-09-01"
      --end-date="2023-03-31"
      
      --time-ever
      
    needs: [event_counts_01GZ0RWS3KFFC34P9RSW6YBFGP, top_5_table_01GZ0RWS3KFFC34P9RSW6YBFGP, plot_measure_01GZ0RWS3KFFC34P9RSW6YBFGP]
    outputs:
      moderately_sensitive:
        notebook: output/01GZ0RWS3KFFC34P9RSW6YBFGP/report.html

Timeline

Created: 2 years, 7 months ago 27 Apr 2023 07:12:30 UTC
Started: 2 years, 7 months ago 27 Apr 2023 07:11:54 UTC
Finished: 2 years, 7 months ago 27 Apr 2023 09:18:35 UTC
Runtime: 02:07:01

These timestamps are generated and stored using the UTC timezone on the TPP backend.

Job request

Status: Failed
Backend: TPP
Workspace: the-impact-of-covid-19-on-the-care-of-people-with-sickle-cell-disease-interactive
Requested by: Brian MacKenna (PHC)
Branch: main
Force run dependencies: Yes
Git commit hash: b17d37c
Requested actions: run_all

Code comparison

Compare the code used in this job request