Database maintenance mode enabled

Databases undergo regular routine maintenance. Find out more about what database maintenance means.

Job request: 2569

Organisation:: The London School of Hygiene & Tropical Medicine
Workspace:: hh-classification-research
ID:: zvwer323lmf5bka6

This page shows the technical details of what happened when the authorised researcher Kevin Wing requested one or more actions to be run against real patient data within a secure environment.

By cross-referencing the list of jobs with the pipeline section below, you can infer what security level the outputs were written to.

The output security levels are:

highly_sensitive
- Researchers can never directly view these outputs
- Researchers can only request code is run against them
moderately_sensitive
- Can be viewed by an approved researcher by logging into a highly secure environment
- These are the only outputs that can be requested for public release via a controlled output review service.

Jobs

Action:

generate_cohort

Status:

Status: Failed

Job identifier:

nzz47cx6rj7cc4gi

Error:

cancelled_by_user: Cancelled by user
Action:

03a_hhClassif_an_hist_hhSizebyEthnicity

Status:

Status: Failed

Job identifier:

lh7vzwjzioeg5crd

Error:

dependency_failed: Not starting as dependency failed
Action:

01_hhClassif_cr_analysis_dataset

Status:

Status: Failed

Job identifier:

lp7mzljadea2a32q

Error:

dependency_failed: Not starting as dependency failed

Pipeline

Show project.yaml

version: "3.0"

expectations:
  population_size: 100000

actions:
  generate_cohort:
    run: cohortextractor:latest generate_cohort --study-definition study_definition
    outputs:
      highly_sensitive:
        cohort: output/input.csv

  W2_generate_cohort:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_W2
    outputs:
      highly_sensitive:
        cohort: output/input_W2.csv

  01_hhClassif_cr_analysis_dataset:
    run: stata-mp:latest analysis/01_hhClassif_cr_analysis_dataset.do MAIN
    needs: [generate_cohort]
    outputs:
      highly_sensitive:
        allHHSizes: output/allHH_beforeDropping_largerThan10_MAIN.dta
        allHHSizes1_20: output/allHH_sizedBetween1And20_MAIN.dta
        allvars: output/hhClassif_analysis_datasetMAIN.dta
        allvarsEthMiss: output/hhClassif_analysis_dataset_with_missing_ethnicityMAIN.dta
        allVarsAgeSpline: output/hhClassif_analysis_dataset_ageband_*MAIN.dta 
        allVarsAgeStratStset: output/hhClassif_analysis_dataset_STSET_*_ageband_*MAIN.dta 
      moderately_sensitive:
        log: logs/01_hhClassif_cr_analysis_dataset.log

  01_hhClassif_cr_analysis_dataset_W2:
    run: stata-mp:latest analysis/01_hhClassif_cr_analysis_dataset.do W2
    needs: [W2_generate_cohort]
    outputs:
      highly_sensitive:
        allHHSizes: output/allHH_beforeDropping_largerThan10_W2.dta
        allHHSizes1_20: output/allHH_sizedBetween1And20_W2.dta
        allvars: output/hhClassif_analysis_datasetW2.dta
        allvarsEthMiss: output/hhClassif_analysis_dataset_with_missing_ethnicityW2.dta
        allVarsAgeSpline: output/hhClassif_analysis_dataset_ageband_*W2.dta 
        allVarsAgeStratStset: output/hhClassif_analysis_dataset_STSET_*_ageband_*W2.dta 
      moderately_sensitive:
        log: logs/01_hhClassif_cr_analysis_dataset_W2.log

  02_hhClassif_an_data_checks:
    run: stata-mp:latest analysis/02_hhClassif_an_data_checks.do MAIN
    needs: [01_hhClassif_cr_analysis_dataset]
    outputs:
      moderately_sensitive:
        log: logs/02_hhClassif_an_data_checks.log

  02_hhClassif_an_data_checks_W2:
    run: stata-mp:latest analysis/02_hhClassif_an_data_checks.do W2
    needs: [01_hhClassif_cr_analysis_dataset_W2]
    outputs:
      moderately_sensitive:
        log: logs/02_hhClassif_an_data_checks_W2.log

  03a_hhClassif_an_hist_hhSizebyEthnicity:
    run: stata-mp:latest analysis/03a_hhClassif_an_hist_hhSizebyEthnicity.do MAIN
    needs: [01_hhClassif_cr_analysis_dataset]
    outputs:
      moderately_sensitive:
        log: logs/03a_hhClassif_an_hist_hhSizebyEthnicity_MAIN.log
        gph1: output/overallHHSizeDist_MAIN.gph 
        gph2: output/whiteHHSizeDist_MAIN.gph 
        gph3: output/southAsianHHSizeDist_MAIN.gph
        gph4: output/blackHHSizeDist_MAIN.gph
        gph5: output/HHdistHists_MAIN.pdf

  03a_hhClassif_an_hist_hhSizebyEthnicity_W2:
    run: stata-mp:latest analysis/03a_hhClassif_an_hist_hhSizebyEthnicity.do W2
    needs: [01_hhClassif_cr_analysis_dataset_W2]
    outputs:
      moderately_sensitive:
        log: logs/03a_hhClassif_an_hist_hhSizebyEthnicity_W2.log
        gph1: output/overallHHSizeDist_W2.gph 
        gph2: output/whiteHHSizeDist_W2.gph 
        gph3: output/southAsianHHSizeDist_W2.gph
        gph4: output/blackHHSizeDist_W2.gph
        gph5: output/HHdistHists_W2.pdf

  03b_hhClassif_an_hist_ov65hhSizebyEthnicity:
    run: stata-mp:latest analysis/03b_hhClassif_an_hist_ov65hhSizebyEthnicity.do MAIN
    needs: [01_hhClassif_cr_analysis_dataset]
    outputs:
      moderately_sensitive:
        log: logs/03b_hhClassif_an_hist_ov65hhSizebyEthnicity_MAIN.log
        gph1: output/ov65OverallHHSizeDist_MAIN.gph 
        gph2: output/ov65WhiteHHSizeDist_MAIN.gph 
        gph3: output/ov65SouthAsianHHSizeDist_MAIN.gph
        gph4: output/ov65BlackHHSizeDist_MAIN.gph
        gph5: output/ov65HHdistHists_MAIN.pdf

  03c_hhClassif_an_descriptive_table_1:
    run: stata-mp:latest analysis/03b_hhClassif_an_descriptive_table_1.do MAIN
    needs: [01_hhClassif_cr_analysis_dataset]
    outputs:
      moderately_sensitive:
        log: logs/03b_hhClassif_an_descriptive_table_1_MAIN.log
        tableTxt: output/table1_hhClassifMAIN.txt
        tableExcel: output/table1_hhClassifMAIN.xlsx

  03c_hhClassif_an_descriptive_table_1_W2:
    run: stata-mp:latest analysis/03b_hhClassif_an_descriptive_table_1.do W2
    needs: [01_hhClassif_cr_analysis_dataset_W2]
    outputs:
      moderately_sensitive:
        log: logs/03b_hhClassif_an_descriptive_table_1_W2.log
        tableTxt: output/table1_hhClassifW2.txt
        tableExcel: output/table1_hhClassifW2.xlsx

  04_hhClassif_an_univariable_analysis:
    needs: [01_hhClassif_cr_analysis_dataset]
    run: stata-mp:latest analysis/04_hhClassif_an_univariable_analysis.do MAIN
    outputs:
      moderately_sensitive:
        log: logs/04_hhClassif_an_univariable_analysis_MAIN.log
        data: output/hhClassif_univariableAnalysis_*_ageband_*MAIN.ster

  04_hhClassif_an_univariable_analysis_W2:
    needs: [01_hhClassif_cr_analysis_dataset_W2]
    run: stata-mp:latest analysis/04_hhClassif_an_univariable_analysis.do W2
    outputs:
      moderately_sensitive:
        log: logs/04_hhClassif_an_univariable_analysis_W2.log
        data: output/hhClassif_univariableAnalysis_*_ageband_*W2.ster

  05_hhClassif_an_multivariable_analysis:
    needs: [01_hhClassif_cr_analysis_dataset]
    run: stata-mp:latest analysis/05_hhClassif_an_multivariable_analysis.do MAIN
    outputs:
      moderately_sensitive:
        log: logs/05_hhClassif_an_multivariable_analysis_MAIN.log
        data: output/hhClassif_multivariableAllEthnicities_*_ageband_*MAIN.ster

  05_hhClassif_an_multivariable_analysis_W2:
    needs: [01_hhClassif_cr_analysis_dataset_W2]
    run: stata-mp:latest analysis/05_hhClassif_an_multivariable_analysis.do W2
    outputs:
      moderately_sensitive:
        log: logs/05_hhClassif_an_multivariable_analysis_W2.log
        data: output/hhClassif_multivariableAllEthnicities_*_ageband_*W2.ster

  #06_hhClassif_an_multivariable_analysis_eth5Interaction:
  #  needs: [01_hhClassif_cr_analysis_dataset]
  #  run: stata-mp:latest analysis/06_hhClassif_an_multivariable_analysis_eth5Interaction.do MAIN
  #  outputs:
  #    moderately_sensitive:
  #      log: logs/06_hhClassif_an_eth5Interaction_analysis_MAIN.log
  #      data: output/hhClassif_multvariableAnalysisEth5Interaction_*_ageband_*MAIN.ster

  #06_hhClassif_an_multivariable_analysis_eth5Interaction_W2:
  #  needs: [01_hhClassif_cr_analysis_dataset_W2]
  #  run: stata-mp:latest analysis/06_hhClassif_an_multivariable_analysis_eth5Interaction.do W2
  #  outputs:
  #    moderately_sensitive:
  #      log: logs/06_hhClassif_an_eth5Interaction_analysis_W2.log
  #      data: output/hhClassif_multvariableAnalysisEth5Interaction_*_ageband_*W2.ster

  07_hhClassif_an_multivariable_analysis_perEth5Group:
    needs: [01_hhClassif_cr_analysis_dataset]
    run: stata-mp:latest analysis/07_hhClassif_an_multivariable_analysis_perEth5Group.do MAIN
    outputs:
      moderately_sensitive:
        log: logs/07_hhClassif_an_multivariable_analysis_perEth5Group_MAIN.log
        data: output/hhClassif_multvariableAnalysisPerEth5_*_ageband_*_ethnicity_*MAIN.ster

  07_hhClassif_an_multivariable_analysis_perEth5Group_W2:
    needs: [01_hhClassif_cr_analysis_dataset_W2]
    run: stata-mp:latest analysis/07_hhClassif_an_multivariable_analysis_perEth5Group.do W2
    outputs:
      moderately_sensitive:
        log: logs/07_hhClassif_an_multivariable_analysis_perEth5Group_W2.log
        data: output/hhClassif_multvariableAnalysisPerEth5_*_ageband_*_ethnicity_*W2.ster

Timeline

Created: 4 years, 4 months ago 18 Jun 2021 05:53:04 UTC
Started: 4 years, 4 months ago 18 Jun 2021 05:53:13 UTC
Finished: 4 years, 4 months ago 18 Jun 2021 06:10:11 UTC
Runtime: 00:16:55

These timestamps are generated and stored using the UTC timezone on the TPP backend.

Job request

Status: Failed
Backend: TPP
Workspace: hh-classification-research
Requested by: Kevin Wing
Branch: master
Force run dependencies: Yes
Git commit hash: ed1b109
Requested actions: 03a_hhClassif_an_hist_hhSizebyEthnicity

Code comparison

Compare the code used in this job request