Skip to content

Job request: 12269

Organisation:
The London School of Hygiene & Tropical Medicine
Workspace:
long_term_kidney_outcomes_after_covid
ID:
tohalmo5t6dch3lm
View repo View project.yaml

This page shows the technical details of what happened when the authorised researcher Viyaasan Mahalingasivam requested one or more actions to be run against real patient data in the project, within a secure environment.

By cross-referencing the list of jobs with the pipeline section below, you can infer what security level various outputs were written to. Researchers can never directly view outputs marked as highly_sensitive ; they can only request that code runs against them. Outputs marked as moderately_sensitive can be viewed by an approved researcher by logging into a highly secure environment. Only outputs marked as moderately_sensitive can be requested for release to the public, via a controlled output review service.

Pipeline

Show project.yaml 
version: "3.0"

expectations:
  population_size: 50000

actions:

# Extract data required for matching for covid (all STPs)
  extract_covid_matching:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_covid_matching
    outputs:
      highly_sensitive:
        cohort: output/input_covid_matching.csv

# Extract data required for matching for contemporary (all STPs)
  extract_contemporary_matching:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_contemporary_matching
    outputs:
      highly_sensitive:
        cohort: output/input_contemporary_matching.csv

  split_stps_without_exclusions:
    run: stata-mp:latest analysis/split_stps_without_exclusions.do
    needs: [extract_covid_matching, extract_contemporary_matching]
    outputs:
      highly_sensitive:
        covid_stp5: output/input_covid_matching_stp5.csv
        covid_stp6: output/input_covid_matching_stp6.csv
        covid_stp7: output/input_covid_matching_stp7.csv
        covid_stp8: output/input_covid_matching_stp8.csv
        covid_stp9: output/input_covid_matching_stp9.csv
        covid_stp10: output/input_covid_matching_stp10.csv
        covid_stp12: output/input_covid_matching_stp12.csv
        covid_stp13: output/input_covid_matching_stp13.csv
        covid_stp14: output/input_covid_matching_stp14.csv
        covid_stp15: output/input_covid_matching_stp15.csv
        covid_stp16: output/input_covid_matching_stp16.csv
        covid_stp17: output/input_covid_matching_stp17.csv
        covid_stp20: output/input_covid_matching_stp20.csv
        covid_stp21: output/input_covid_matching_stp21.csv
        covid_stp22: output/input_covid_matching_stp22.csv
        covid_stp23: output/input_covid_matching_stp23.csv
        covid_stp24: output/input_covid_matching_stp24.csv
        covid_stp25: output/input_covid_matching_stp25.csv
        covid_stp26: output/input_covid_matching_stp26.csv
        covid_stp27: output/input_covid_matching_stp27.csv
        covid_stp29: output/input_covid_matching_stp29.csv
        covid_stp33: output/input_covid_matching_stp33.csv
        covid_stp35: output/input_covid_matching_stp35.csv
        covid_stp36: output/input_covid_matching_stp36.csv
        covid_stp37: output/input_covid_matching_stp37.csv
        covid_stp40: output/input_covid_matching_stp40.csv
        covid_stp41: output/input_covid_matching_stp41.csv
        covid_stp42: output/input_covid_matching_stp42.csv
        covid_stp43: output/input_covid_matching_stp43.csv
        covid_stp44: output/input_covid_matching_stp44.csv
        covid_stp49: output/input_covid_matching_stp49.csv
        contemporary_stp5: output/input_contemporary_matching_stp5.csv
        contemporary_stp6: output/input_contemporary_matching_stp6.csv
        contemporary_stp7: output/input_contemporary_matching_stp7.csv
        contemporary_stp8: output/input_contemporary_matching_stp8.csv
        contemporary_stp9: output/input_contemporary_matching_stp9.csv
        contemporary_stp10: output/input_contemporary_matching_stp10.csv
        contemporary_stp12: output/input_contemporary_matching_stp12.csv
        contemporary_stp13: output/input_contemporary_matching_stp13.csv
        contemporary_stp14: output/input_contemporary_matching_stp14.csv
        contemporary_stp15: output/input_contemporary_matching_stp15.csv
        contemporary_stp16: output/input_contemporary_matching_stp16.csv
        contemporary_stp17: output/input_contemporary_matching_stp17.csv
        contemporary_stp20: output/input_contemporary_matching_stp20.csv
        contemporary_stp21: output/input_contemporary_matching_stp21.csv
        contemporary_stp22: output/input_contemporary_matching_stp22.csv
        contemporary_stp23: output/input_contemporary_matching_stp23.csv
        contemporary_stp24: output/input_contemporary_matching_stp24.csv
        contemporary_stp25: output/input_contemporary_matching_stp25.csv
        contemporary_stp26: output/input_contemporary_matching_stp26.csv
        contemporary_stp27: output/input_contemporary_matching_stp27.csv
        contemporary_stp29: output/input_contemporary_matching_stp29.csv
        contemporary_stp33: output/input_contemporary_matching_stp33.csv
        contemporary_stp35: output/input_contemporary_matching_stp35.csv
        contemporary_stp36: output/input_contemporary_matching_stp36.csv
        contemporary_stp37: output/input_contemporary_matching_stp37.csv
        contemporary_stp40: output/input_contemporary_matching_stp40.csv
        contemporary_stp41: output/input_contemporary_matching_stp41.csv
        contemporary_stp42: output/input_contemporary_matching_stp42.csv
        contemporary_stp43: output/input_contemporary_matching_stp43.csv
        contemporary_stp44: output/input_contemporary_matching_stp44.csv
        contemporary_stp49: output/input_contemporary_matching_stp49.csv
      moderately_sensitive:
        log: logs/split_stps_without_exclusions.log

  match_contemporary_stp35_without_exclusions:
    run: python:latest python analysis/match_contemporary_stp35_without_exclusions.py
    needs: [extract_covid_matching, extract_contemporary_matching, split_stps_without_exclusions]
    outputs:
      moderately_sensitive:
        matching_report: output/matching_report_contemporary_stp35_without_exclusions.txt
      highly_sensitive:
        matched_matches: output/matched_matches_contemporary_stp35_without_exclusions.csv

Timeline

  • Created:

  • Started:

  • Finished:

  • Runtime:

These timestamps are generated and stored using the UTC timezone on the TPP backend.

Job information

Status
Failed 
JobRequestError: split_stps_without_exclusions failed on a previous run and must be re-run
Backend
TPP
Workspace
long_term_kidney_outcomes_after_covid
Requested by
Viyaasan Mahalingasivam
Branch
main
Force run dependencies
No
Git commit hash
b8924b7
Requested actions
  • run_all

Code comparison

Compare the code used in this Job Request