Job request: 11455
- Organisation:
- Bennett Institute
- Workspace:
- asthma_sro
- ID:
- 4qvjt4gwooltovje
This page shows the technical details of what happened when the authorised researcher Rose Higgins requested one or more actions to be run against real patient data within a secure environment.
By cross-referencing the list of jobs with the pipeline section below, you can infer what security level the outputs were written to.
The output security levels are:
-
highly_sensitive
- Researchers can never directly view these outputs
- Researchers can only request code is run against them
-
moderately_sensitive
- Can be viewed by an approved researcher by logging into a highly secure environment
- These are the only outputs that can be requested for public release via a controlled output review service.
Jobs
-
- Job identifier:
-
eufagjpjnnu7srxq - Error:
- nonzero_exit: Job exited with error code 5: Something went wrong with the database, please contact tech support
-
- Job identifier:
-
lhcbunoq5c5y4fbt - Error:
- cancelled_by_user: Cancelled by user
-
- Job identifier:
-
t37kywr3hpcyz66n - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
3imrovxqn3uuot65 - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
i4zpjejlht4m5hvb - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
tssmw5xnoakkzw67 - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
hghekakgfucxn25n - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
6i2z26mwf3qmvvhz - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
ry2jnc2gpbswktfs - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
xbcmhf5rophisqme - Error:
- dependency_failed: Not starting as dependency failed
Pipeline
Show project.yaml
version: '3.0'
expectations:
population_size: 1000
actions:
# Asthma register (ast_reg) Dictionary method actions
generate_study_population_ast_reg:
run: >
cohortextractor:latest generate_cohort
--study-definition study_definition_ast_reg
--index-date-range "2019-03-01 to 2022-03-31 by month"
--output-dir=output
--output-format=csv
outputs:
highly_sensitive:
cohort: output/input_ast_reg*.csv
generate_study_population_ethnicity:
run: cohortextractor:latest generate_cohort --study-definition study_definition_ethnicity --output-dir=output
outputs:
highly_sensitive:
cohort: output/input_ethnicity.csv
join_ethnicity_ast_reg:
run: >
cohort-joiner:v0.0.46
--lhs output/input_ast*.csv
--rhs output/input_ethnicity.csv
--output-dir output/joined
needs: [generate_study_population_ethnicity, generate_study_population_ast_reg]
outputs:
highly_sensitive:
cohort: output/joined/input_ast*.csv
generate_measures_ast_reg:
run: >
cohortextractor:latest generate_measures
--study-definition study_definition_ast_reg
--output-dir=output/joined
needs: [join_ethnicity_ast_reg]
outputs:
moderately_sensitive:
measure_csv: output/joined/measure_ast_reg_*_rate.csv
join_measures_register:
run: python:latest python analysis/join_and_round.py
--input-list output/joined/measure_ast_reg_total_rate.csv
--input-list output/joined/measure_ast_reg_age_band_rate.csv
--input-list output/joined/measure_ast_reg_care_home_rate.csv
--input-list output/joined/measure_ast_reg_ethnicity_rate.csv
--input-list output/joined/measure_ast_reg_imd_rate.csv
--input-list output/joined/measure_ast_reg_learning_disability_rate.csv
--input-list output/joined/measure_ast_reg_region_rate.csv
--input-list output/joined/measure_ast_reg_sex_rate.csv
--output-dir output/joined/summary
--output-name "measure_register.csv"
needs: [generate_measures_ast_reg]
outputs:
moderately_sensitive:
# Only output the single summary file
measure_csv: output/joined/summary/measure_register.csv
calculate_rates_ast_reg:
run: python:latest python analysis/rate_calculations.py
needs: [generate_measures_ast_reg]
outputs:
moderately_sensitive:
tables: output/rate_table_*.csv
plots: output/plot_*.png
decile_chart: output/decile_chart.png
#############################
# Plotting
#############################
generate_qof_deciles_charts:
run: >
deciles-charts:v0.0.15
--input-files output/joined/measure_*_practice_rate.csv
--output-dir output/joined
config:
show_outer_percentiles: false
tables:
output: true
charts:
output: true
needs: [generate_measures_ast_reg]
outputs:
moderately_sensitive:
cohort: output/joined/deciles_*_*.*
generate_qof_groups:
run: >
python:latest python analysis/group_charts.py
--input-files output/joined/measure_ast_reg_*.csv
--output-dir output/joined
--date-lines "2019-03-31" "2020-03-31" "2021-03-31"
--scale "percentage"
needs: [generate_measures_ast_reg]
outputs:
moderately_sensitive:
cohort: output/joined/group_chart_*.png
generate_all_breakdowns:
run: >
python:latest python analysis/panel_plots.py
--input-file output/joined/summary/measure_register.csv
--measures-pattern "*_rate"
--output-dir output/joined/summary
--output-name "asthma_register_by_demographic_group"
--date-lines "2019-03-31" "2020-03-31" "2021-03-31"
--scale "percentage"
needs: [join_measures_register]
outputs:
moderately_sensitive:
cohort: output/joined/summary/asthma_register_by_demographic_group.png
generate_notebook_ast_reg:
run: jupyter:latest jupyter nbconvert /workspace/analysis/Table1_notebook.ipynb --execute --to html --output-dir=/workspace/output --ExecutePreprocessor.timeout=86400 --no-input
needs: [join_measures_register]
outputs:
moderately_sensitive:
notebook: output/Table1_notebook.html
Timeline
-
Created:
-
Started:
-
Finished:
-
Runtime: 07:25:13
These timestamps are generated and stored using the UTC timezone on the TPP backend.
Job request
- Status
-
Failed
- Backend
- TPP
- Workspace
- asthma_sro
- Requested by
- Rose Higgins
- Branch
- master
- Force run dependencies
- Yes
- Git commit hash
- 570dbbf
- Requested actions
-
-
run_all
-
Code comparison
Compare the code used in this job request