Skip to content

Job request: 14856

Organisation:
Bennett Institute
Workspace:
winter-pressures
ID:
o3prnqwpnin75bqj

This page shows the technical details of what happened when the authorised researcher Iain Dillingham requested one or more actions to be run against real patient data within a secure environment.

By cross-referencing the list of jobs with the pipeline section below, you can infer what security level the outputs were written to.

The output security levels are:

  • highly_sensitive
    • Researchers can never directly view these outputs
    • Researchers can only request code is run against them
  • moderately_sensitive
    • Can be viewed by an approved researcher by logging into a highly secure environment
    • These are the only outputs that can be requested for public release via a controlled output review service.

Jobs

Pipeline

Show project.yaml
version: "3.0"

expectations:
  population_size: 5000

actions:
  # Other data
  # ----------
  # Add actions for other data to this section. Prefix them with a suitable name; place
  # scripts in a similarly named sub-directory of the analysis directory; write outputs
  # to a similarly named sub-directory of the output directory.
  #
  # For example, let's call our other data "metrics". We would prefix our actions
  # "metrics_"; we would place our scripts in analysis/metrics; we would write outputs
  # to output/metrics.

  # Appointments data
  # -----------------
  appointments_generate_dataset_sql:
    run: >
      sqlrunner:latest
        analysis/appointments/dataset_query.sql
        --output output/appointments/dataset_long.csv
        --dummy-data-file analysis/appointments/dummy_dataset_long.csv
    outputs:
      highly_sensitive:
        dataset: output/appointments/dataset_long.csv

  appointments_read_csv:
    run: python:latest
      python
      -m analysis.appointments.read_csv
    needs: [appointments_generate_dataset_sql]
    outputs:
      moderately_sensitive:
        log: output/appointments/read_csv.log

  # appointments_generate_dataset:
  #   run: >
  #     databuilder:v0
  #       generate-dataset
  #       analysis/appointments/dataset_definition.py
  #       --output output/appointments/dataset_wide.arrow
  #   outputs:
  #     highly_sensitive:
  #       dataset: output/appointments/dataset_wide.arrow

  # appointments_get_freq_na_values:
  #   run: >
  #     python:latest
  #       python
  #       -m analysis.appointments.get_freq_na_values
  #   needs: [appointments_generate_dataset]
  #   outputs:
  #     moderately_sensitive:
  #       dataset: output/appointments/freq_na_values.csv

  # appointments_reshape_dataset:
  #   run: >
  #     python:latest
  #       python
  #       -m analysis.appointments.reshape_dataset
  #   needs: [appointments_generate_dataset]
  #   outputs:
  #     highly_sensitive:
  #       dataset: output/appointments/dataset_long.arrow

  # appointments_generate_measure:
  #   run: >
  #     python:latest
  #       python
  #       -m analysis.appointments.generate_measure
  #   needs: [appointments_reshape_dataset]
  #   outputs:
  #     moderately_sensitive:
  #       measure: output/appointments/measure_median_lead_time_in_days_by_nunique_patient_id.csv

  # appointments_generate_deciles_charts:
  #   run: >
  #     deciles-charts:v0.0.33
  #       --input-files output/appointments/measure_*.csv
  #       --output-dir output/appointments
  #   config:
  #     show_outer_percentiles: true
  #   needs: [appointments_generate_measure]
  #   outputs:
  #     moderately_sensitive:
  #       deciles_charts: output/appointments/deciles_chart_*.png
  #       deciles_tables: output/appointments/deciles_table_*.csv

Timeline

  • Created:

  • Started:

  • Finished:

  • Runtime: 00:25:40

These timestamps are generated and stored using the UTC timezone on the TPP backend.

Job request

Status
Succeeded
Backend
TPP
Workspace
winter-pressures
Requested by
Iain Dillingham
Branch
main
Force run dependencies
No
Git commit hash
64d2bec
Requested actions
  • appointments_read_csv

Code comparison

Compare the code used in this job request