Job request: 4258
- Organisation:
- NHSE/I
- Workspace:
- prostate_cancer_psa_testing_2021
- ID:
- pppon6zuuxiublud
This page shows the technical details of what happened when the authorised researcher Pedro Oliveira requested one or more actions to be run against real patient data within a secure environment.
By cross-referencing the list of jobs with the pipeline section below, you can infer what security level the outputs were written to.
The output security levels are:
-
highly_sensitive
- Researchers can never directly view these outputs
- Researchers can only request code is run against them
-
moderately_sensitive
- Can be viewed by an approved researcher by logging into a highly secure environment
- These are the only outputs that can be requested for public release via a controlled output review service.
Jobs
-
- Job identifier:
-
qhjdtoce5grwi33g
-
- Job identifier:
-
ae4ae2pkijrvub4b
Pipeline
Show project.yaml
version: '3.0'
expectations:
population_size: 1000
actions:
generate_study_population:
run: cohortextractor:latest generate_cohort --study-definition study_definition --index-date-range "2019-04-01 to 2021-09-30 by month" --output-dir=output --output-format csv.gz
outputs:
highly_sensitive:
cohort: output/input_*.csv.gz
#This has been taken from https://github.com/opensafely/sro-template/blob/master/project.yaml
#The output files are not there, but assume it will give the 30 input files witht the suffices being the months
#The .gz extension code above is to turn it into a compressed file
generate_measures:
run: cohortextractor:latest generate_measures --study-definition study_definition --output-dir=output
#Give errors if used: --index-date-range "2019-04-01 to 2021-09-30 by month" --output-format csv
needs: [generate_study_population]
outputs:
moderately_sensitive:
measure_csv: output/measure_*.csv
# run_model:
# run: python:analysis/PSA_graphs.py
# needs: [generate_measures]
# outputs:
# moderately_sensitive:
# measure_csv: output/PSA_test_total_table.csv
#Code below functioning as of 2021 10 27
#New 2021/10/11 - PSA_graphs.py working in ypnb book locally, but gives error whem running model
run_model:
#python:latest is in principle needed. All slashes in paths should be forward slashes
run: python:latest python analysis/PSA_graphs.py
needs: [generate_measures]
outputs:
moderately_sensitive:
plots: output/*.jpg #Tests_long_list.jpg, output/Tests_short_list.jpg - comma to separate lists may not work
#Putting a second line with plots and a different output gives duplicate key error
Timeline
-
Created:
-
Finished:
-
Runtime:
These timestamps are generated and stored using the UTC timezone on the TPP backend.
Job request
- Status
-
Succeeded
- Backend
- TPP
- Workspace
- prostate_cancer_psa_testing_2021
- Requested by
- Pedro Oliveira
- Branch
- main
- Force run dependencies
- No
- Git commit hash
- f9a24a4
- Requested actions
-
-
generate_study_population -
generate_measures
-
Code comparison
Compare the code used in this job request