Job request: 24195

Organisation:: King's College London
Workspace:: incidence_longtermconditions
ID:: 5p6cv64lyq5thvkt

This page shows the technical details of what happened when the authorised researcher Mark Russell requested one or more actions to be run against real patient data within a secure environment.

By cross-referencing the list of jobs with the pipeline section below, you can infer what security level the outputs were written to.

The output security levels are:

highly_sensitive
- Researchers can never directly view these outputs
- Researchers can only request code is run against them
moderately_sensitive
- Can be viewed by an approved researcher by logging into a highly secure environment
- These are the only outputs that can be requested for public release via a controlled output review service.

Jobs

Action:

generate_dataset

Status:

Status: Succeeded

Job identifier:

ppfylu2tnguin2w3
Action:

measures_dataset_pmr_2016

Status:

Status: Succeeded

Job identifier:

cexf25dr26wukr4t
Action:

measures_dataset_pmr_2017

Status:

Status: Succeeded

Job identifier:

opdxiyaadob5l2om
Action:

measures_dataset_pmr_2018

Status:

Status: Succeeded

Job identifier:

axzqejrmvd6hkgqf
Action:

measures_dataset_pmr_2019

Status:

Status: Succeeded

Job identifier:

okc7oba6ornp7pet
Action:

measures_dataset_pmr_2020

Status:

Status: Succeeded

Job identifier:

bgmuavie7xwk6wbe
Action:

measures_dataset_pmr_2021

Status:

Status: Succeeded

Job identifier:

azg7c6ugnjhi3pe3
Action:

measures_dataset_pmr_2022

Status:

Status: Succeeded

Job identifier:

dvp363uuzuzczj6s
Action:

measures_dataset_pmr_2023

Status:

Status: Succeeded

Job identifier:

qqc7o4jzmsgdhwk3
Action:

measures_dataset_pmr_2024

Status:

Status: Succeeded

Job identifier:

lyw2d2gkzgne5bzo
Action:

measures_dataset_rheumatoid_2016

Status:

Status: Succeeded

Job identifier:

qep2ik5nucqpxozf
Action:

measures_dataset_rheumatoid_2017

Status:

Status: Succeeded

Job identifier:

urtrqipuhs4yotvi
Action:

measures_dataset_rheumatoid_2018

Status:

Status: Succeeded

Job identifier:

jk7ijsxfyp3bwcuu
Action:

measures_dataset_rheumatoid_2019

Status:

Status: Succeeded

Job identifier:

24x36dlcf5wgvgpo
Action:

measures_dataset_rheumatoid_2020

Status:

Status: Succeeded

Job identifier:

fi74sxx37dgywwbp
Action:

measures_dataset_rheumatoid_2021

Status:

Status: Succeeded

Job identifier:

7cv5tlqdjqbqa6a4
Action:

measures_dataset_rheumatoid_2022

Status:

Status: Succeeded

Job identifier:

bcvi5hrury26aazu
Action:

measures_dataset_rheumatoid_2023

Status:

Status: Succeeded

Job identifier:

hftlj5pvxo7ojnjp
Action:

measures_dataset_rheumatoid_2024

Status:

Status: Succeeded

Job identifier:

vnyohxnynaffha2l
Action:

run_incidence_graphs

Status:

Status: Succeeded

Job identifier:

dnnqjakmoeg6sgdm
Action:

run_sarima

Status:

Status: Failed

Job identifier:

l3eo7zcpnkhafkxg

Error:

cancelled_by_user: Cancelled by user

Pipeline

Show project.yaml

version: '3.0'

expectations:
  population_size: 1000

actions:
             
  generate_dataset:
    run: ehrql:v1 generate-dataset analysis/dataset_definition.py
      --output output/dataset_definition.csv
      #--
      #--diseases "{diseases}"
    outputs:
      highly_sensitive:
        cohort: output/dataset_definition.csv

  measures_dataset_rheumatoid_2016:
    run: ehrql:v1 generate-measures analysis/dataset_definition_measures.py
      --output output/measures/measures_dataset_rheumatoid_2016.csv
      --
      --start-date "2016-04-01"
      --intervals 12
      --disease "rheumatoid"
    needs: [generate_dataset]
    outputs:
      highly_sensitive:
        measure_csv: output/measures/measures_dataset_rheumatoid_2016.csv

  measures_dataset_pmr_2016:
    run: ehrql:v1 generate-measures analysis/dataset_definition_measures.py
      --output output/measures/measures_dataset_pmr_2016.csv
      --
      --start-date "2016-04-01"
      --intervals 12
      --disease "pmr"
    needs: [generate_dataset]
    outputs:
      highly_sensitive:
        measure_csv: output/measures/measures_dataset_pmr_2016.csv

  measures_dataset_rheumatoid_2017:
    run: ehrql:v1 generate-measures analysis/dataset_definition_measures.py
      --output output/measures/measures_dataset_rheumatoid_2017.csv
      --
      --start-date "2017-04-01"
      --intervals 12
      --disease "rheumatoid"
    needs: [generate_dataset]
    outputs:
      highly_sensitive:
        measure_csv: output/measures/measures_dataset_rheumatoid_2017.csv

  measures_dataset_pmr_2017:
    run: ehrql:v1 generate-measures analysis/dataset_definition_measures.py
      --output output/measures/measures_dataset_pmr_2017.csv
      --
      --start-date "2017-04-01"
      --intervals 12
      --disease "pmr"
    needs: [generate_dataset]
    outputs:
      highly_sensitive:
        measure_csv: output/measures/measures_dataset_pmr_2017.csv

  measures_dataset_rheumatoid_2018:
    run: ehrql:v1 generate-measures analysis/dataset_definition_measures.py
      --output output/measures/measures_dataset_rheumatoid_2018.csv
      --
      --start-date "2018-04-01"
      --intervals 12
      --disease "rheumatoid"
    needs: [generate_dataset]
    outputs:
      highly_sensitive:
        measure_csv: output/measures/measures_dataset_rheumatoid_2018.csv

  measures_dataset_pmr_2018:
    run: ehrql:v1 generate-measures analysis/dataset_definition_measures.py
      --output output/measures/measures_dataset_pmr_2018.csv
      --
      --start-date "2018-04-01"
      --intervals 12
      --disease "pmr"
    needs: [generate_dataset]
    outputs:
      highly_sensitive:
        measure_csv: output/measures/measures_dataset_pmr_2018.csv

  measures_dataset_rheumatoid_2019:
    run: ehrql:v1 generate-measures analysis/dataset_definition_measures.py
      --output output/measures/measures_dataset_rheumatoid_2019.csv
      --
      --start-date "2019-04-01"
      --intervals 12
      --disease "rheumatoid"
    needs: [generate_dataset]
    outputs:
      highly_sensitive:
        measure_csv: output/measures/measures_dataset_rheumatoid_2019.csv

  measures_dataset_pmr_2019:
    run: ehrql:v1 generate-measures analysis/dataset_definition_measures.py
      --output output/measures/measures_dataset_pmr_2019.csv
      --
      --start-date "2019-04-01"
      --intervals 12
      --disease "pmr"
    needs: [generate_dataset]
    outputs:
      highly_sensitive:
        measure_csv: output/measures/measures_dataset_pmr_2019.csv

  measures_dataset_rheumatoid_2020:
    run: ehrql:v1 generate-measures analysis/dataset_definition_measures.py
      --output output/measures/measures_dataset_rheumatoid_2020.csv
      --
      --start-date "2020-04-01"
      --intervals 12
      --disease "rheumatoid"
    needs: [generate_dataset]
    outputs:
      highly_sensitive:
        measure_csv: output/measures/measures_dataset_rheumatoid_2020.csv

  measures_dataset_pmr_2020:
    run: ehrql:v1 generate-measures analysis/dataset_definition_measures.py
      --output output/measures/measures_dataset_pmr_2020.csv
      --
      --start-date "2020-04-01"
      --intervals 12
      --disease "pmr"
    needs: [generate_dataset]
    outputs:
      highly_sensitive:
        measure_csv: output/measures/measures_dataset_pmr_2020.csv

  measures_dataset_rheumatoid_2021:
    run: ehrql:v1 generate-measures analysis/dataset_definition_measures.py
      --output output/measures/measures_dataset_rheumatoid_2021.csv
      --
      --start-date "2021-04-01"
      --intervals 12
      --disease "rheumatoid"
    needs: [generate_dataset]
    outputs:
      highly_sensitive:
        measure_csv: output/measures/measures_dataset_rheumatoid_2021.csv

  measures_dataset_pmr_2021:
    run: ehrql:v1 generate-measures analysis/dataset_definition_measures.py
      --output output/measures/measures_dataset_pmr_2021.csv
      --
      --start-date "2021-04-01"
      --intervals 12
      --disease "pmr"
    needs: [generate_dataset]
    outputs:
      highly_sensitive:
        measure_csv: output/measures/measures_dataset_pmr_2021.csv

  measures_dataset_rheumatoid_2022:
    run: ehrql:v1 generate-measures analysis/dataset_definition_measures.py
      --output output/measures/measures_dataset_rheumatoid_2022.csv
      --
      --start-date "2022-04-01"
      --intervals 12
      --disease "rheumatoid"
    needs: [generate_dataset]
    outputs:
      highly_sensitive:
        measure_csv: output/measures/measures_dataset_rheumatoid_2022.csv

  measures_dataset_pmr_2022:
    run: ehrql:v1 generate-measures analysis/dataset_definition_measures.py
      --output output/measures/measures_dataset_pmr_2022.csv
      --
      --start-date "2022-04-01"
      --intervals 12
      --disease "pmr"
    needs: [generate_dataset]
    outputs:
      highly_sensitive:
        measure_csv: output/measures/measures_dataset_pmr_2022.csv

  measures_dataset_rheumatoid_2023:
    run: ehrql:v1 generate-measures analysis/dataset_definition_measures.py
      --output output/measures/measures_dataset_rheumatoid_2023.csv
      --
      --start-date "2023-04-01"
      --intervals 12
      --disease "rheumatoid"
    needs: [generate_dataset]
    outputs:
      highly_sensitive:
        measure_csv: output/measures/measures_dataset_rheumatoid_2023.csv

  measures_dataset_pmr_2023:
    run: ehrql:v1 generate-measures analysis/dataset_definition_measures.py
      --output output/measures/measures_dataset_pmr_2023.csv
      --
      --start-date "2023-04-01"
      --intervals 12
      --disease "pmr"
    needs: [generate_dataset]
    outputs:
      highly_sensitive:
        measure_csv: output/measures/measures_dataset_pmr_2023.csv

  measures_dataset_rheumatoid_2024:
    run: ehrql:v1 generate-measures analysis/dataset_definition_measures.py
      --output output/measures/measures_dataset_rheumatoid_2024.csv
      --
      --start-date "2024-04-01"
      --intervals 6
      --disease "rheumatoid"
    needs: [generate_dataset]
    outputs:
      highly_sensitive:
        measure_csv: output/measures/measures_dataset_rheumatoid_2024.csv

  measures_dataset_pmr_2024:
    run: ehrql:v1 generate-measures analysis/dataset_definition_measures.py
      --output output/measures/measures_dataset_pmr_2024.csv
      --
      --start-date "2024-04-01"
      --intervals 6
      --disease "pmr"
    needs: [generate_dataset]
    outputs:
      highly_sensitive:
        measure_csv: output/measures/measures_dataset_pmr_2024.csv

  # run_data_avail:
  #   run: stata-mp:latest analysis/101_data_availability.do
  #   needs: [generate_dataset]
  #   outputs:
  #     moderately_sensitive:
  #       log1: logs/data_avail_tables.log   
  #       data1: output/data/data_check_*.csv
  
  run_incidence_graphs:
    run: stata-mp:latest analysis/100_incidence_graphs.do
    needs: [generate_dataset, measures_dataset_rheumatoid_2016, measures_dataset_pmr_2016, measures_dataset_rheumatoid_2017, measures_dataset_pmr_2017, measures_dataset_rheumatoid_2018, measures_dataset_pmr_2018, measures_dataset_rheumatoid_2019, measures_dataset_pmr_2019, measures_dataset_rheumatoid_2020, measures_dataset_pmr_2020, measures_dataset_rheumatoid_2021, measures_dataset_pmr_2021, measures_dataset_rheumatoid_2022, measures_dataset_pmr_2022, measures_dataset_rheumatoid_2023, measures_dataset_pmr_2023, measures_dataset_rheumatoid_2024, measures_dataset_pmr_2024]
    outputs:
      moderately_sensitive:
        log1: logs/descriptive_tables.log   
        figure1: output/figures/incidence_*.svg
        figure2: output/figures/prevalence_*.svg
        figure3: output/figures/inc_comp_*.svg
        figure4: output/figures/prev_comp_*.svg
        figure5: output/figures/inc_adj_*.svg
        figure6: output/figures/inc_ma_sex_*.svg
        figure7: output/figures/prev_adj_*.svg
        figure8: output/figures/inc_adj_ma_*.svg
        figure9: output/figures/inc_adj_ma2_*.svg
        figure10: output/figures/adj_ma_sex_*.svg
        figure11: output/figures/adj_ma_sex2_*.svg
        figure12: output/figures/adj_ma_age_*.svg
        figure13: output/figures/adj_ma_age2_*.svg
        table1: output/tables/arima_nonstandardised.csv
        table2: output/tables/arima_standardised.csv

  run_sarima:
    run: r:latest analysis/200_sarima.R
    needs: [run_incidence_graphs]
    outputs:
      moderately_sensitive:
        log1: logs/sarima_log.txt   
        figure1: output/figures/observed_*.svg
        figure2: output/figures/raw_pre_covid_*.svg
        figure3: output/figures/differenced_pre_covid_*.svg
        figure4: output/figures/seasonal_pre_covid_*.svg
        figure5: output/figures/raw_acf_*.svg
        figure6: output/figures/differenced_acf_*.svg
        figure7: output/figures/seasonal_acf_*.svg
        figure8: output/figures/auto_residuals_*.svg
        figure9: output/figures/obs_pred_*.svg
        table1: output/tables/change_incidence_byyear.csv
        table2: output/tables/values_*.csv

Timeline

Created: 9 months, 3 weeks ago 28 Jan 2025 13:44:29 UTC
Started: 9 months, 3 weeks ago 29 Jan 2025 03:07:34 UTC
Finished: 9 months, 3 weeks ago 29 Jan 2025 23:08:15 UTC
Runtime: 60:51:54

These timestamps are generated and stored using the UTC timezone on the TPP backend.

Job request

Status: Failed
Backend: TPP
Workspace: incidence_longtermconditions
Requested by: Mark Russell
Branch: main
Force run dependencies: Yes
Git commit hash: 8c10e7c
Requested actions: run_all

Code comparison

Compare the code used in this job request