Job request: 11463
- Organisation:
- The UK Renal Registry
- Workspace:
- renal-short-data-report
- ID:
- o2xepw5ukneptnbu
This page shows the technical details of what happened when the authorised researcher Shalini Santhakumaran requested one or more actions to be run against real patient data in the project, within a secure environment.
By cross-referencing the list of jobs with the
pipeline section below, you can infer what
security level
various outputs were written to. Researchers can never directly
view outputs marked as
highly_sensitive
;
they can only request that code runs against them. Outputs
marked as
moderately_sensitive
can be viewed by an approved researcher by logging into a highly
secure environment. Only outputs marked as
moderately_sensitive
can be requested for release to the public, via a controlled
output review service.
Jobs
-
- Job identifier:
-
mkilgxebhnbzth57
Pipeline
Show project.yaml
version: "3.0"
expectations:
population_size: 1000
actions:
generate_study_population_calculators:
run: cohortextractor:latest generate_cohort
--study-definition study_definition_calculators
--output-format csv.gz
outputs:
highly_sensitive:
cohort: output/input_calculators.csv.gz
calculate_calculators:
run: python:latest python analysis/scripts/calculate_calculators.py
needs: [generate_study_population_calculators]
outputs:
highly_sensitive:
counts: output/input_calculators_calculated.csv.gz
generate_notebook_crcl:
run: jupyter:latest jupyter nbconvert /workspace/analysis/scripts/crcl_gault.ipynb --execute --to html --template basic --output-dir=/workspace/output --ExecutePreprocessor.timeout=86400 --no-input
needs: [calculate_calculators]
outputs:
moderately_sensitive:
notebook: output/crcl_gault.html
generate_study_population:
run: cohortextractor:latest generate_cohort
--study-definition study_definition
--index-date-range "2019-01-01 to 2021-01-01 by month"
--output-format csv.gz
outputs:
highly_sensitive:
cohort: output/input*.csv.gz
generate_study_population_2:
run: cohortextractor:latest generate_cohort
--study-definition study_definition
--index-date-range "2021-02-01 to 2022-07-01 by month"
--output-format csv.gz
outputs:
highly_sensitive:
cohort: output/inpu*.csv.gz
generate_study_population_ukrr_ethnicity:
run: cohortextractor:latest generate_cohort
--study-definition study_definition_ukrr_ethnicity
--output-dir=output
--output-format csv.gz
outputs:
highly_sensitive:
cohort: output/input_ukrr_ethnicity.csv.gz
join_cohorts:
run: >
cohort-joiner:v0.0.46
--lhs output/input_20*.csv.gz
--rhs output/input_ukrr_ethnicity.csv.gz
--output-dir output/joined
needs: [generate_study_population, generate_study_population_2, generate_study_population_ukrr_ethnicity]
outputs:
highly_sensitive:
cohort: output/joined/input_20*.csv.gz
generate_study_population_paeds:
run: cohortextractor:latest generate_cohort
--study-definition study_definition_paeds
--output-format csv.gz
outputs:
highly_sensitive:
cohort: output/input_paeds.csv.gz
generate_table_1:
run: python:latest python analysis/scripts/table_1.py --study_def_paths="output/joined/input_*.csv.gz" --demographics="age_band,sex,region,imd,ethnicity"
needs: [join_cohorts]
outputs:
moderately_sensitive:
counts: output/table_1.csv
at_risk: output/table_1_at_risk.csv
get_counts:
run: python:latest python analysis/scripts/combine_operators.py
needs: [join_cohorts]
outputs:
moderately_sensitive:
counts: output/*_count.csv
generate_top_5_tables:
run: python:latest python analysis/scripts/top_codes.py
needs: [get_counts]
outputs:
moderately_sensitive:
counts: output/top_5_code_table_*.csv
generate_study_population_check_SC:
run: cohortextractor:latest generate_cohort
--study-definition study_definition_check_SC
--output-format csv.gz
outputs:
highly_sensitive:
cohort: output/input_check_SC.csv.gz
generate_measures:
run: cohortextractor:latest generate_measures
--study-definition study_definition
--output-dir=output/joined
needs: [join_cohorts]
outputs:
moderately_sensitive:
measure_csv: output/joined/measure_*_rate.csv
generate_plots:
run: python:latest python analysis/scripts/plot_measures.py
needs: [generate_measures]
outputs:
moderately_sensitive:
counts: output/figures/plot_*.jpeg
generate_plots_numeric_values:
run: python:latest python analysis/scripts/plot_numeric_values.py
needs: [join_cohorts]
outputs:
moderately_sensitive:
figures: output/*dis*.jpeg
generate_upset_plots:
run: python:latest python analysis/scripts/ckd_discrepancies.py
needs: [join_cohorts]
outputs:
moderately_sensitive:
figures: output/ckd_staging_upset_*.png
time: output/time_diff_ckd.jpeg
# generate_notebook:
# run: jupyter:latest jupyter nbconvert /workspace/analysis/report.ipynb --execute --to html --template basic --output-dir=/workspace/output --ExecutePreprocessor.timeout=86400 --no-input
# needs: [generate_plots, get_counts]
# outputs:
# moderately_sensitive:
# notebook: output/report.html
generate_ukrr_checks:
run: r:latest -e 'rmarkdown::render("analysis/ukrr-analysis/ukrr-checks.Rmd", knit_root_dir = "/workspace", output_dir = "output/notebooks")'
needs: [generate_study_population, generate_study_population_ukrr_ethnicity]
outputs:
moderately_sensitive:
report: output/notebooks/ukrr-checks.html
generate_ukrr_report:
run: r:latest -e 'rmarkdown::render("analysis/ukrr-analysis/ukrr-report.Rmd", knit_root_dir = "/workspace", output_dir = "output/notebooks")'
needs: [join_cohorts]
outputs:
moderately_sensitive:
report: output/notebooks/ukrr-report.html
generate_ukrr_discrepancies:
run: r:latest -e 'rmarkdown::render("analysis/ukrr-analysis/ukrr-discrepancies.Rmd", knit_root_dir = "/workspace", output_dir = "output/notebooks")'
needs: [join_cohorts]
outputs:
moderately_sensitive:
report: output/notebooks/ukrr-discrepancies.html
# generate_kidney_week:
# run: r:latest -e 'rmarkdown::render("analysis/ukrr-analysis/kidney_week.Rmd", knit_root_dir = "/workspace", output_dir = "output/notebooks")'
# needs: [join_cohorts]
# outputs:
# moderately_sensitive:
# report: output/notebooks/kidney_week.html
generate_ukrr_tables_figures:
run: r:latest -e 'rmarkdown::render("analysis/ukrr-analysis/ukrr-tables-figures.Rmd", knit_root_dir = "/workspace", output_dir = "output/notebooks")'
needs: [join_cohorts]
outputs:
moderately_sensitive:
report: output/notebooks/ukrr-tables-figures.html
generate_ukrr_tables_figures_paeds:
run: r:latest -e 'rmarkdown::render("analysis/ukrr-analysis/ukrr-tables-figures-paeds.Rmd", knit_root_dir = "/workspace", output_dir = "output/notebooks")'
needs: [generate_study_population_paeds]
outputs:
moderately_sensitive:
report: output/notebooks/ukrr-tables-figures-paeds.html
generate_check_SC:
run: r:latest -e 'rmarkdown::render("analysis/ukrr-analysis/check-SC.Rmd", knit_root_dir = "/workspace", output_dir = "output/notebooks")'
needs: [generate_study_population_check_SC, generate_study_population_ukrr_ethnicity]
outputs:
moderately_sensitive:
report: output/notebooks/check-SC.html
generate_modality_difference_table:
run: r:latest -e 'rmarkdown::render("analysis/ukrr-analysis/modality-difference-table.Rmd", knit_root_dir = "/workspace", output_dir = "output/notebooks")'
needs: [join_cohorts]
outputs:
moderately_sensitive:
report: output/notebooks/modality-difference-table.html
# generate_frequency_plots:
# run: r:latest analysis/test_frequency.R
# needs: [join_cohorts_ethnicity]
# outputs:
# moderately_sensitive:
# figures: output/figures/months_with_*.png
Timeline
-
Created:
-
Started:
-
Finished:
-
Runtime: 00:23:08
These timestamps are generated and stored using the UTC timezone on the TPP backend.
Job information
- Status
-
Succeeded
- Backend
- TPP
- Workspace
- renal-short-data-report
- Requested by
- Shalini Santhakumaran
- Branch
- main
- Force run dependencies
- No
- Git commit hash
- ddb1ec2
- Requested actions
-
-
generate_modality_difference_table
-
Code comparison
Compare the code used in this Job Request