Skip to content

Job request: 16837

Organisation:
The London School of Hygiene & Tropical Medicine
Workspace:
healthcare_utilisation_openprompt
ID:
nngdmqy6hnhzh73c
View repo View project.yaml

This page shows the technical details of what happened when the authorised researcher Liang-Yu Lin requested one or more actions to be run against real patient data in the project, within a secure environment.

By cross-referencing the list of jobs with the pipeline section below, you can infer what security level various outputs were written to. Researchers can never directly view outputs marked as highly_sensitive ; they can only request that code runs against them. Outputs marked as moderately_sensitive can be viewed by an approved researcher by logging into a highly secure environment. Only outputs marked as moderately_sensitive can be requested for release to the public, via a controlled output review service.

Jobs

Pipeline

Show project.yaml 
version: '3.0'

expectations:
  population_size: 500

actions:

  generate_long_covid_exposure_dataset:
    run: 
      databuilder:v0 generate-dataset
        analysis/dataset_definition_unmatched_exp_lc.py
        --output output/dataset_exp_lc_unmatched.csv
    outputs:
      highly_sensitive:
        cohort: output/dataset_exp_lc_unmatched.csv

  check_stp_regions:
    needs: [generate_long_covid_exposure_dataset]
    run: r:latest analysis/dm00_check_stp_regions.R
    outputs:
      moderately_sensitive: 
        stp_region: output/stp_regions_counts.csv

  generate_list_gp_use_long_covid_dx:
    run: 
      databuilder:v0 generate-dataset
        analysis/dataset_definition_lc_gp_list.py
        --output output/dataset_lc_gp_list.csv
    outputs:
      highly_sensitive:
        cohort: output/dataset_lc_gp_list.csv

  generate_dataset_comparator_exclude_gp_no_long_covid:
    needs: [generate_list_gp_use_long_covid_dx]
    run: 
      databuilder:v0 generate-dataset
        analysis/dataset_definition_unmatched_comparator.py
        --output output/dataset_comparator_unmatched.csv
    outputs:
      highly_sensitive:
        cohort: output/dataset_comparator_unmatched.csv

  split_unmatched_data_by_stp_regions:
    needs: [generate_long_covid_exposure_dataset, generate_dataset_comparator_exclude_gp_no_long_covid]
    run: r:latest analysis/dm00_split_stp_for_matching.R
    outputs: 
      moderately_sensitive: 
        stp_exp_table: output/exp_stp_names_numbers.csv
        stp_com_table: output/com_stp_names_numbers.csv
      highly_sensitive: 
        exp_stp_: output/exp_stp_.csv
        exp_stp_5: output/exp_stp_05.csv
        exp_stp_24: output/exp_stp_24.csv
        exp_stp_26: output/exp_stp_26.csv
        exp_stp_23: output/exp_stp_23.csv
        exp_stp_6: output/exp_stp_06.csv
        exp_stp_13: output/exp_stp_13.csv
        exp_stp_49: output/exp_stp_49.csv
        exp_stp_15: output/exp_stp_15.csv
        exp_stp_41: output/exp_stp_41.csv
        exp_stp_9: output/exp_stp_09.csv
        exp_stp_17: output/exp_stp_17.csv
        exp_stp_25: output/exp_stp_25.csv
        exp_stp_16: output/exp_stp_16.csv
        exp_stp_7: output/exp_stp_07.csv
        exp_stp_42: output/exp_stp_42.csv
        exp_stp_21: output/exp_stp_21.csv
        exp_stp_40: output/exp_stp_40.csv
        exp_stp_27: output/exp_stp_27.csv
        exp_stp_14: output/exp_stp_14.csv
        exp_stp_20: output/exp_stp_20.csv
        exp_stp_29: output/exp_stp_29.csv
        exp_stp_43: output/exp_stp_43.csv
        exp_stp_8: output/exp_stp_08.csv
        exp_stp_37: output/exp_stp_37.csv
        exp_stp_22: output/exp_stp_22.csv
        exp_stp_12: output/exp_stp_12.csv
        exp_stp_33: output/exp_stp_33.csv
        exp_stp_35: output/exp_stp_35.csv
        exp_stp_36: output/exp_stp_36.csv
        exp_stp_10: output/exp_stp_10.csv
        exp_stp_44: output/exp_stp_44.csv
        com_stp_: output/com_stp_.csv
        com_stp_5: output/com_stp_05.csv
        com_stp_24: output/com_stp_24.csv
        com_stp_26: output/com_stp_26.csv
        com_stp_23: output/com_stp_23.csv
        com_stp_6: output/com_stp_06.csv
        com_stp_13: output/com_stp_13.csv
        com_stp_49: output/com_stp_49.csv
        com_stp_15: output/com_stp_15.csv
        com_stp_41: output/com_stp_41.csv
        com_stp_9: output/com_stp_09.csv
        com_stp_17: output/com_stp_17.csv
        com_stp_25: output/com_stp_25.csv
        com_stp_16: output/com_stp_16.csv
        com_stp_7: output/com_stp_07.csv
        com_stp_42: output/com_stp_42.csv
        com_stp_21: output/com_stp_21.csv
        com_stp_40: output/com_stp_40.csv
        com_stp_27: output/com_stp_27.csv
        com_stp_14: output/com_stp_14.csv
        com_stp_20: output/com_stp_20.csv
        com_stp_29: output/com_stp_29.csv
        com_stp_43: output/com_stp_43.csv
        com_stp_8: output/com_stp_08.csv
        com_stp_37: output/com_stp_37.csv
        com_stp_22: output/com_stp_22.csv
        com_stp_12: output/com_stp_12.csv
        com_stp_33: output/com_stp_33.csv
        com_stp_35: output/com_stp_35.csv
        com_stp_36: output/com_stp_36.csv
        com_stp_10: output/com_stp_10.csv
        com_stp_44: output/com_stp_44.csv

  # match_comparators:
  #   run:
  #     python:latest python analysis/match.py
  #   needs: [generate_dataset_comparator_exclude_gp_no_long_covid, generate_long_covid_exposure_dataset]
  #   outputs: 
  #     highly_sensitive:
  #       matched_cases: output/matched_cases.csv
  #       matched_matches: output/matched_matches.csv
  #       matched_all: output/matched_combined.csv
  #     moderately_sensitive: 
  #       matching_report: output/matching_report.txt

Timeline

  • Created:

  • Started:

  • Finished:

  • Runtime: 00:02:20

These timestamps are generated and stored using the UTC timezone on the TPP backend.

Job information

Status
Failed
Backend
TPP
Workspace
healthcare_utilisation_openprompt
Requested by
Liang-Yu Lin
Branch
main
Force run dependencies
No
Git commit hash
3872deb
Requested actions
  • split_unmatched_data_by_stp_regions

Code comparison

Compare the code used in this Job Request