Job request: 7766
- Organisation:
- Bennett Institute
- Workspace:
- long-covid-sick-notes
- ID:
- ksll4eagt775wyj5
This page shows the technical details of what happened when the authorised researcher Alex Walker requested one or more actions to be run against real patient data within a secure environment.
By cross-referencing the list of jobs with the pipeline section below, you can infer what security level the outputs were written to.
The output security levels are:
-
highly_sensitive
- Researchers can never directly view these outputs
- Researchers can only request code is run against them
-
moderately_sensitive
- Can be viewed by an approved researcher by logging into a highly secure environment
- These are the only outputs that can be requested for public release via a controlled output review service.
Jobs
-
- Job identifier:
-
n2trfflhwsepx55l
-
- Job identifier:
-
kl3r3zesqn6kxwc5
-
- Job identifier:
-
l44euss4ama2ah75
-
- Job identifier:
-
ygwczhp5bmmt3tzh - Error:
- nonzero_exit: Job exited with error code 143
-
- Job identifier:
-
6d3sm2whgd5ridq3 - Error:
- nonzero_exit: Job exited with error code 143
-
- Job identifier:
-
d34pfkwl2tknpjo4 - Error:
- nonzero_exit: Job exited with error code 143
-
- Job identifier:
-
5epi6dprikbfwwnv - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
s5oxpvrsfmxrywm6 - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
36tjgnb6giskkh6l - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
cfo4xdr7ctbyspts - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
qgo5ujhztujxoepy - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
h42vrtzauu6gnw6h - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
ptuhq4td7e4xfjge - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
m2jrkjx3laggmlll - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
zruobrpiouciesbg - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
tnypdwmi3bigjbri - Error:
- Internal error
-
- Job identifier:
-
64wb6tbcvubkmqbp - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
wvzyjdp4lejh4jcf - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
wowpztya7g7pkmvc - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
pcvddr5qgspsozhb - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
cgzcpblm5zh3qaq5 - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
3fmxrpqgqm3mphjr - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
igc3pauutgri7wpp - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
zyl4p5ubis3wngsi - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
6odzrtkmfkmtcail - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
76pdvcdwbefjiplo - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
vsuzotmwzbnnfikm - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
det76ydedqrjixql - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
o7vnyagrkqbqmhfq - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
2ifzzx4dperxwscc - Error:
- dependency_failed: Not starting as dependency failed
Pipeline
Show project.yaml
version: '3.0'
expectations:
population_size: 10000
actions:
generate_study_population_covid_2020:
run: cohortextractor:latest generate_cohort --study-definition study_definition_covid_2020 --output-dir=output/cohorts
outputs:
highly_sensitive:
cohort: output/cohorts/input_covid_2020.csv
generate_study_population_covid_2021:
run: cohortextractor:latest generate_cohort --study-definition study_definition_covid_2021 --output-dir=output/cohorts
outputs:
highly_sensitive:
cohort: output/cohorts/input_covid_2021.csv
generate_study_population_general_2019:
run: cohortextractor:latest generate_cohort --study-definition study_definition_general_2019 --output-dir=output/cohorts
outputs:
highly_sensitive:
cohort: output/cohorts/input_general_2019.csv
generate_study_population_general_2020:
run: cohortextractor:latest generate_cohort --study-definition study_definition_general_2020 --output-dir=output/cohorts
outputs:
highly_sensitive:
cohort: output/cohorts/input_general_2020.csv
generate_study_population_general_2021:
run: cohortextractor:latest generate_cohort --study-definition study_definition_general_2021 --output-dir=output/cohorts
outputs:
highly_sensitive:
cohort: output/cohorts/input_general_2021.csv
generate_study_population_pneumonia_2019:
run: cohortextractor:latest generate_cohort --study-definition study_definition_pneumonia_2019 --output-dir=output/cohorts
outputs:
highly_sensitive:
cohort: output/cohorts/input_pneumonia_2019.csv
matching_2019:
run: python:latest python analysis/match_running.py "2019" --output-dir=output/cohorts
needs: [generate_study_population_covid_2020, generate_study_population_general_2019]
outputs:
moderately_sensitive:
matching_report: output/cohorts/matching_report_general_2019.txt
highly_sensitive:
matched_cohort: output/cohorts/matched_matches_general_2019.csv
matching_2020:
run: python:latest python analysis/match_running.py "2020" --output-dir=output/cohorts
needs: [generate_study_population_covid_2020, generate_study_population_general_2020]
outputs:
moderately_sensitive:
matching_report: output/cohorts/matching_report_general_2020.txt
highly_sensitive:
matched_cohort: output/cohorts/matched_matches_general_2020.csv
matching_2021:
run: python:latest python analysis/match_running.py "2021" --output-dir=output/cohorts
needs: [generate_study_population_covid_2021, generate_study_population_general_2021]
outputs:
moderately_sensitive:
matching_report: output/cohorts/matching_report_general_2021.txt
highly_sensitive:
matched_cohort: output/cohorts/matched_matches_general_2021.csv
reconcile_sick_note_spells_covid_2020:
run: python:latest python analysis/reconcile_sick_note_spells.py "_covid_2020" "input"
needs: [generate_study_population_covid_2020]
outputs:
highly_sensitive:
cohort_with_duration: output/cohorts/input_covid_2020_with_duration.csv
reconcile_sick_note_spells_covid_2021:
run: python:latest python analysis/reconcile_sick_note_spells.py "_covid_2021" "input"
needs: [generate_study_population_covid_2021]
outputs:
highly_sensitive:
cohort_with_duration: output/cohorts/input_covid_2021_with_duration.csv
reconcile_sick_note_spells_general_2019:
run: python:latest python analysis/reconcile_sick_note_spells.py "_general_2019" "matched_matches"
needs: [matching_2019]
outputs:
highly_sensitive:
cohort_with_duration: output/cohorts/input_general_2019_with_duration.csv
reconcile_sick_note_spells_general_2020:
run: python:latest python analysis/reconcile_sick_note_spells.py "_general_2020" "matched_matches"
needs: [matching_2020]
outputs:
highly_sensitive:
cohort_with_duration: output/cohorts/input_general_2020_with_duration.csv
reconcile_sick_note_spells_general_2021:
run: python:latest python analysis/reconcile_sick_note_spells.py "_general_2021" "matched_matches"
needs: [matching_2021]
outputs:
highly_sensitive:
cohort_with_duration: output/cohorts/input_general_2021_with_duration.csv
reconcile_sick_note_spells_pneumonia_2019:
run: python:latest python analysis/reconcile_sick_note_spells.py "_pneumonia_2019" "input"
needs: [generate_study_population_pneumonia_2019]
outputs:
highly_sensitive:
cohort_with_duration: output/cohorts/input_pneumonia_2019_with_duration.csv
covid_2020_rates_cohort:
run: stata-mp:latest analysis/000_cr_define_covariates_simple_rates.do "covid_2020" --output-dir=output/cohorts
needs: [reconcile_sick_note_spells_covid_2020]
outputs:
highly_sensitive:
analysis_dataset: output/cohorts/cohort_rates_covid_2020.dta
covid_2021_rates_cohort:
run: stata-mp:latest analysis/000_cr_define_covariates_simple_rates.do "covid_2021" --output-dir=output/cohorts
needs: [reconcile_sick_note_spells_covid_2021]
outputs:
highly_sensitive:
analysis_dataset: output/cohorts/cohort_rates_covid_2021.dta
general_2019_rates_cohort:
run: stata-mp:latest analysis/000_cr_define_covariates_simple_rates.do "general_2019" --output-dir=output/cohorts
needs: [reconcile_sick_note_spells_general_2019]
outputs:
highly_sensitive:
analysis_dataset: output/cohorts/cohort_rates_general_2019.dta
general_2020_rates_cohort:
run: stata-mp:latest analysis/000_cr_define_covariates_simple_rates.do "general_2020" --output-dir=output/cohorts
needs: [reconcile_sick_note_spells_general_2020]
outputs:
highly_sensitive:
analysis_dataset: output/cohorts/cohort_rates_general_2020.dta
general_2021_rates_cohort:
run: stata-mp:latest analysis/000_cr_define_covariates_simple_rates.do "general_2021" --output-dir=output/cohorts
needs: [reconcile_sick_note_spells_general_2021]
outputs:
highly_sensitive:
analysis_dataset: output/cohorts/cohort_rates_general_2021.dta
pneumonia_2019_rates_cohort:
run: stata-mp:latest analysis/000_cr_define_covariates_simple_rates.do "pneumonia_2019" --output-dir=output/cohorts
needs: [reconcile_sick_note_spells_pneumonia_2019]
outputs:
highly_sensitive:
analysis_dataset: output/cohorts/cohort_rates_pneumonia_2019.dta
covid_2020_rates:
run: stata-mp:latest analysis/100_cr_simple_rates.do "covid_2020" --output-dir=output/tabfig
needs: [covid_2020_rates_cohort]
outputs:
moderately_sensitive:
rates: output/tabfig/rates_summary_covid_2020.csv
covid_2021_rates:
run: stata-mp:latest analysis/100_cr_simple_rates.do "covid_2021" --output-dir=output/tabfig
needs: [covid_2021_rates_cohort]
outputs:
moderately_sensitive:
rates: output/tabfig/rates_summary_covid_2021.csv
general_2019_rates:
run: stata-mp:latest analysis/100_cr_simple_rates.do "general_2019" --output-dir=output/tabfig
needs: [general_2019_rates_cohort]
outputs:
moderately_sensitive:
rates: output/tabfig/rates_summary_general_2019.csv
general_2020_rates:
run: stata-mp:latest analysis/100_cr_simple_rates.do "general_2020" --output-dir=output/tabfig
needs: [general_2020_rates_cohort]
outputs:
moderately_sensitive:
rates: output/tabfig/rates_summary_general_2020.csv
general_2021_rates:
run: stata-mp:latest analysis/100_cr_simple_rates.do "general_2021" --output-dir=output/tabfig
needs: [general_2021_rates_cohort]
outputs:
moderately_sensitive:
rates: output/tabfig/rates_summary_general_2021.csv
pneumonia_2019_rates:
run: stata-mp:latest analysis/100_cr_simple_rates.do "pneumonia_2019" --output-dir=output/tabfig
needs: [pneumonia_2019_rates_cohort]
outputs:
moderately_sensitive:
rates: output/tabfig/rates_summary_pneumonia_2019.csv
append_cohorts:
run: stata-mp:latest analysis/200_cr_data_management_matching.do --output-dir=output/cohorts
needs: [covid_2021_rates_cohort, covid_2020_rates_cohort, pneumonia_2019_rates_cohort, general_2021_rates_cohort, general_2020_rates_cohort, general_2019_rates_cohort]
outputs:
moderately_sensitive:
log: output/cohorts/append_cohorts.txt
highly_sensitive:
dataset: output/cohorts/combined_covid_2020_pneumonia.dta
dataset2: output/cohorts/combined_covid_2021_pneumonia.dta
dataset3: output/cohorts/combined_covid_2020_general_2019.dta
dataset4: output/cohorts/combined_covid_2021_general_2019.dta
dataset5: output/cohorts/combined_covid_general_2020.dta
dataset6: output/cohorts/combined_covid_general_2021.dta
cox_models:
run: stata-mp:latest analysis/201_cox_models.do
needs: [append_cohorts]
outputs:
moderately_sensitive:
log: output/cohorts/cox_models.txt
dataset: output/tabfig/cox_model_summary.csv
describe_duration_diagnoses:
run: jupyter:latest jupyter nbconvert /workspace/notebooks/describe_duration_diagnoses.ipynb --execute --to html --template basic --output-dir=/workspace/output --ExecutePreprocessor.timeout=86400 --no-input
needs: [covid_2021_rates_cohort, covid_2020_rates_cohort, general_2019_rates_cohort, general_2020_rates_cohort, general_2021_rates_cohort, pneumonia_2019_rates_cohort]
outputs:
moderately_sensitive:
notebook: output/describe_duration_diagnoses.html
table: output/tabfig/med_mean_overall.csv
table2: output/tabfig/med_mean_age_group.csv
table3: output/tabfig/med_mean_sex.csv
table4: output/tabfig/med_mean_ethnicity.csv
table5: output/tabfig/med_mean_imd.csv
table6: output/tabfig/med_mean_region.csv
subplots: output/tabfig/diag_*.png
Timeline
-
Created:
-
Started:
-
Finished:
-
Runtime: 77:01:50
These timestamps are generated and stored using the UTC timezone on the TPP backend.
Job request
- Status
-
Failed
- Backend
- TPP
- Workspace
- long-covid-sick-notes
- Requested by
- Alex Walker
- Branch
- master
- Force run dependencies
- Yes
- Git commit hash
- 324a24e
- Requested actions
-
-
run_all
-
Code comparison
Compare the code used in this job request