Job request: 4935

View Repo View project.yaml

This page shows the technical details of what happened when authorised researcher ali-fahmi requested one or more actions to be run against real patient data in the amr_repeat_abs project, within a secure environment.

By cross-referencing the indicated Requested Actions with the Pipeline section below, you can infer what security level various outputs were written to. Outputs marked as highly_sensitive can never be viewed directly by a researcher; they can only request that code runs against them. Outputs marked as moderately_sensitive can be viewed by an approved researcher by logging into a highly secure environment. Only outputs marked as moderately_sensitive can be requested for release to the public, via a controlled output review service.

Jobs

ID Status Action
5aicnlo2stonieek failed generate_study_population_repeat_ABs
aox5b5jl2dqu26em failed generate_notebook_repeated_ABs

Pipeline

Show Hide project.yaml
version: '3.0'

expectations:
  population_size: 1000

actions:

  # generate_study_population:
  #   run: cohortextractor:latest generate_cohort --study-definition study_definition --index-date-range "2019-01-01 to today by month" --skip-existing --output-dir=output/measures --output-format=csv.gz
  #   outputs:
  #     highly_sensitive:
  #       cohort: output/measures/input_*.csv.gz

  generate_study_population_repeat_ABs:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_repeat_ABs --index-date-range "2019-01-01 to today by month" --skip-existing --output-dir=output/repeat_ABs_data --output-format=csv.gz
    outputs:
      highly_sensitive:
        cohort: output/repeat_ABs_data/input_repeat_ABs_*.csv.gz

  generate_study_population_elderly:
    run: cohortextractor:latest generate_cohort --study-definition study_definition_elderly 
      --output-format=csv.gz
    outputs:
      highly_sensitive:
        cohort: output/input_elderly.csv.gz

  # generate_measures:
  #   run: cohortextractor:latest generate_measures --study-definition study_definition --skip-existing --output-dir=output/measures
  #   needs: [generate_study_population]
  #   outputs:
  #     moderately_sensitive:
  #       measure_csv: output/measures/measure_*.csv
        
  describe_elderly_agedis:
    run: r:latest analysis/tables/gen_csv_age_check.R
    needs: [generate_study_population_elderly]
    outputs:
      moderately_sensitive:
        agetable: output/age_quant.csv

  # describe:
  #   run: r:latest analysis/plot/overall_ab_prescribing.R
  #   needs: [generate_measures]
  #   outputs:
  #     moderately_sensitive:
  #       cohort: output/overall.png
  #       boxplot: output/overallbox.png

  # describe_percentile:
  #   run: r:latest analysis/plot/overall_ab_prescribing_2575percentile.R
  #   needs: [generate_measures]
  #   outputs:
  #     moderately_sensitive:
  #       percentile: output/overall_25th_75th_percentile.png

  # describe_starpu:
  #   run: r:latest analysis/plot/starpu_ab_prescribing.R
  #   needs: [generate_measures]
  #   outputs:
  #     moderately_sensitive:
  #       cohort: output/starpuline.png
  #       boxplot: output/starpubox.png
  
  # describe_infection_ab_UTI:
  #   run: r:latest analysis/plot/infection_ab_UTI.R
  #   needs: [generate_study_population, generate_measures]
  #   outputs:
  #      moderately_sensitive:
  #       plot: output/UTI.png
  
  generate_notebook_repeated_ABs:
    run: jupyter:latest jupyter nbconvert /workspace/analysis/repeated_ABs.ipynb --execute --to html --output-dir=/workspace/output/repeat_ABs_outputs --ExecutePreprocessor.timeout=86400
    needs: [generate_study_population_repeat_ABs]
    outputs:
      moderately_sensitive:
        notebook: output/repeat_ABs_outputs/repeated_ABs.html 
        figures: output/repeat_ABs_outputs/*

State

State is inferred from the related Jobs.

Status: Failed

Timings

Timings set to UTC timezone.

  • Created:
  • Started:
  • Finished:
  • Runtime: 87:59:09

Config

  • Backend:
    TPP
  • Workspace:
    amr_repeat_abs
  • Branch:
    study_eval_repeated_abs
  • Creator:
    ali-fahmi
  • Force run dependencies:
    True
  • Git Commit Hash:
    8129962
  • Requested actions:
    • generate_notebook_repeated_ABs