Skip to content

Job request: 11468

Organisation:
The London School of Hygiene & Tropical Medicine
Workspace:
long_term_kidney_outcomes_after_covid
ID:
a66fufnjic2xrrrh
View repo View project.yaml

This page shows the technical details of what happened when the authorised researcher Viyaasan Mahalingasivam requested one or more actions to be run against real patient data within a secure environment.

By cross-referencing the list of jobs with the pipeline section below, you can infer what security level the outputs were written to.

The output security levels are:

  • highly_sensitive
    • Researchers can never directly view these outputs
    • Researchers can only request code is run against them
  • moderately_sensitive
    • Can be viewed by an approved researcher by logging into a highly secure environment
    • These are the only outputs that can be requested for public release via a controlled output review service.

Jobs

Pipeline

Show project.yaml 
version: "3.0"

expectations:
  population_size: 10000

actions:

# Match covid_all_for_matching to comparators in potential_historical_general_population and potential_contemporary_general_population
# https://docs.opensafely.org/case-control-studies/

# Extract data for covid_england
  extract_covid_england:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_covid_england
    outputs:
      highly_sensitive:
        cohort: output/input_covid_england.csv

#Data management for covid_england
  covid_england:
    run: stata-mp:latest analysis/covid_england.do
    needs: [extract_covid_england]
    outputs:
      highly_sensitive:
        analysis_dataset: output/covid_england.dta
      moderately_sensitive:
        log: logs/covid_england.log

#Descriptive statitics for covid_england
  covid_descriptive:
    run: stata-mp:latest analysis/covid_descriptive.do
    needs: [covid_england]
    outputs:
      highly_sensitive:
        analysis_dataset: output/covid_descriptive.dta
      moderately_sensitive:
        log: logs/covid_descriptive.log

# Extract data required for matching for covid_stp01
  extract_covid_stp01_matching:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_covid_stp01_matching
    outputs:
      highly_sensitive:
        cohort: output/input_covid_stp01_matching.csv

#Data management for covid_stp01_matching
  clean_covid_stp01_matching:
    run: stata-mp:latest analysis/covid_stp01_matching.do
    needs: [extract_covid_stp01_matching]
    outputs:
      highly_sensitive:
        analysis_dataset: output/covid_stp01_matching.csv
      moderately_sensitive:
        log: logs/covid_stp01_matching.log

# Extract data required for matching for contemporary_stp01
  extract_contemporary_stp01_matching:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_contemporary_stp01_matching
    outputs:
      highly_sensitive:
        cohort: output/input_contemporary_stp01_matching.csv

#Data management for contemporary_stp01_matching
  clean_contemporary_stp01_matching:
    run: stata-mp:latest analysis/contemporary_stp01_matching.do
    needs: [extract_contemporary_stp01_matching]
    outputs:
      highly_sensitive:
        analysis_dataset: output/contemporary_stp01_matching.csv
      moderately_sensitive:
        log: logs/contemporary_stp01_matching.log

# Match covid_stp01_matching to contemporary_stp01_matching
  match_contemporary_stp01:
    run: python:latest python analysis/match_contemporary_stp01.py
    needs: [extract_covid_stp01_matching, clean_covid_stp01_matching, extract_contemporary_stp01_matching, clean_contemporary_stp01_matching]
    outputs:
      moderately_sensitive:
        matching_report: output/matching_report_contemporary_stp01.txt
      highly_sensitive:
        matched_matches: output/matched_matches_contemporary_stp01.csv

# Extract data required for matching for covid_northeast
  extract_covid_northeast_matching:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_covid_northeast_matching
    outputs:
      highly_sensitive:
        cohort: output/input_covid_northeast_matching.csv

#Data management for covid_northeast_matching
  clean_covid_northeast_matching:
    run: stata-mp:latest analysis/covid_northeast_matching.do
    needs: [extract_covid_northeast_matching]
    outputs:
      highly_sensitive:
        analysis_dataset: output/covid_northeast_matching.csv
      moderately_sensitive:
        log: logs/covid_northeast_matching.log

# Extract data required for matching for contemporary_northeast
  extract_contemporary_northeast_matching:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_contemporary_northeast_matching
    outputs:
      highly_sensitive:
        cohort: output/input_contemporary_northeast_matching.csv

#Data management for potential_contemporary_population_northeast
  clean_contemporary_northeast_matching:
    run: stata-mp:latest analysis/contemporary_northeast_matching.do
    needs: [extract_contemporary_northeast_matching]
    outputs:
      highly_sensitive:
        analysis_dataset: output/contemporary_northeast_matching.csv
      moderately_sensitive:
        log: logs/contemporary_northeast_matching.log

# Match covid_northeast to potential_contemporary_population_northeast
  match_contemporary_northeast:
    run: python:latest python analysis/match_contemporary_northeast.py
    needs: [extract_covid_northeast_matching, clean_covid_northeast_matching, extract_contemporary_northeast_matching, clean_contemporary_northeast_matching]
    outputs:
      moderately_sensitive:
        matching_report: output/matching_report_contemporary_northeast.txt
      highly_sensitive:
        matched_matches: output/matched_matches_contemporary_northeast.csv


# Extract data for covid_critical_care
  extract_covid_critical_care:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_covid_critical_care
    outputs:
      highly_sensitive:
        cohort: output/input_covid_critical_care.csv

  covid_critical_care_count:
    run: stata-mp:latest analysis/covid_critical_care.do
    needs: [extract_covid_critical_care]
    outputs:
      highly_sensitive:
        analysis_dataset: output/covid_critical_care.dta
      moderately_sensitive:
        log: logs/covid_critical_care.log

# Extract data for potential_contemporary_general_population
  extract_potential_contemporary_general_population:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_potential_contemporary_general_population
    outputs:
      highly_sensitive:
        cohort: output/input_potential_contemporary_general_population.csv

# Extract data for potential_contemporary_general_population (North East region)
  extract_potential_contemporary_general_population_northeast:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_potential_contemporary_general_population_northeast
    outputs:
      highly_sensitive:
        cohort: output/input_potential_contemporary_general_population_northeast.csv

Timeline

  • Created:

  • Started:

  • Finished:

  • Runtime: 06:21:27

These timestamps are generated and stored using the UTC timezone on the TPP backend.

Job request

Status
Failed
Backend
TPP
Workspace
long_term_kidney_outcomes_after_covid
Requested by
Viyaasan Mahalingasivam
Branch
main
Force run dependencies
No
Git commit hash
de22899
Requested actions
  • extract_covid_stp01_matching
  • clean_covid_stp01_matching
  • extract_contemporary_stp01_matching
  • clean_contemporary_stp01_matching
  • match_contemporary_stp01

Code comparison

Compare the code used in this job request