This page shows the technical details of what happened when authorised researcher Louis Fisher requested one or more actions to be run against real patient data in the Service Restoration Observatory project, within a secure environment.
By cross-referencing the indicated Requested Actions with the Pipeline section below, you can infer what
security level various outputs were written to. Outputs
marked as highly_sensitive can never be viewed directly by a
researcher; they can only request that code runs against them. Outputs marked as moderately_sensitive can be
viewed by an approved researcher by logging into a highly secure environment. Only outputs marked as
moderately_sensitive can be requested for release to the public, via a controlled output review service.
Jobs
| ID | Status | Action |
|---|---|---|
| 4sbadfcqyciyunec | failed | generate_study_population |
| rzebg5dnzvuda65u | failed | generate_study_population_ethnicity |
| ruzgqqh5xerdo7sd | failed | join_ethnicity |
| sqf6jbvlno2brype | succeeded | generate_study_population_practice_count |
| wso3hgggmizs6acu | failed | generate_measures |
| 2zodsryp7trisley | failed | generate_measure_ethnicity |
| bg5lhpojjm7yq6al | failed | get_patient_count |
| p3gpkvvgxcb7n3ya | failed | create_notebook |
| s2sqt2ybd4h5w3is | failed | generate_notebook |
Pipeline
Show Hide project.yaml
version: '3.0'
expectations:
population_size: 10000
actions:
# convert_codelist:
# run: python:latest python analysis/convert_codelist.py
# outputs:
# highly_sensitive:
# cohort: codelists/opensafely-pulse-oximetry-converted.csv
generate_study_population:
run: cohortextractor:latest generate_cohort --study-definition study_definition --index-date-range "2020-07-01 to 2021-02-01 by week" --output-dir=output
outputs:
highly_sensitive:
cohort: output/input_*.csv
generate_study_population_ethnicity:
run: cohortextractor:latest generate_cohort --study-definition study_definition_ethnicity --output-dir=output
outputs:
highly_sensitive:
cohort: output/input_ethnicity.csv
join_ethnicity:
run: python:latest python analysis/join_ethnicity.py
needs: [generate_study_population, generate_study_population_ethnicity]
outputs:
highly_sensitive:
cohort: output/input*.csv
generate_study_population_practice_count:
run: cohortextractor:latest generate_cohort --study-definition study_definition_practice_count --output-dir=output
outputs:
highly_sensitive:
cohort: output/input_practice_count.csv
generate_measures:
run: cohortextractor:latest generate_measures --study-definition study_definition --output-dir=output
needs: [join_ethnicity]
outputs:
moderately_sensitive:
measure_csv: output/measure_*.csv
generate_measure_ethnicity:
run: python:latest python analysis/generate_measure_ethnicity.py
needs: [join_ethnicity]
outputs:
moderately_sensitive:
measure: output/measure_ethnicity.csv
get_patient_count:
run: python:latest python analysis/get_patients_counts.py
needs: [generate_study_population]
outputs:
moderately_sensitive:
text: output/patient_count.json
create_notebook:
run: python:latest python analysis/create_notebook.py
needs: [get_patient_count, join_ethnicity]
outputs:
moderately_sensitive:
notebook: analysis/SRO_Notebook.ipynb
generate_notebook:
run: jupyter:latest jupyter nbconvert /workspace/analysis/SRO_Notebook.ipynb --execute --to html --output-dir=/workspace/output --ExecutePreprocessor.timeout=86400 --no-input
needs: [create_notebook, generate_measures, generate_measure_ethnicity, generate_study_population_practice_count, generate_study_population, join_ethnicity]
outputs:
moderately_sensitive:
notebook: output/SRO_Notebook.html
State
State is inferred from the related Jobs.
Status: Failed
Timings
Timings set to UTC timezone.
- Created:
- Started:
- Finished:
- Runtime: 00:12:44
Config
-
- Backend:
-
TPP
-
- Workspace:
- sro-pulse-oximetry-2
-
- Branch:
template
-
- Creator:
- LFISHER7
-
- Force run dependencies:
- False
-
- Git Commit Hash:
- 4cf71b3
-
Requested actions:
run_all