This page shows the technical details of what happened when authorised researcher Ben Butler-Cole requested one or more actions to be run against real patient data in the Service Restoration Observatory project, within a secure environment.
By cross-referencing the indicated Requested Actions with the Pipeline section below, you can infer what
security level various outputs were written to. Outputs
highly_sensitive can never be viewed directly by a
researcher; they can only request that code runs against them. Outputs marked as
moderately_sensitive can be
viewed by an approved researcher by logging into a highly secure environment. Only outputs marked as
moderately_sensitive can be requested for release to the public, via a controlled output review service.
Show Hide project.yaml
version: '3.0' expectations: population_size: 1000 actions: generate_study_population: run: cohortextractor:latest generate_cohort --study-definition study_definition --index-date-range "2019-01-01 to 2021-04-01 by month" --output-dir=output --output-format=feather outputs: highly_sensitive: cohort: output/input_*.feather get_patient_count: run: python:latest python analysis/get_patients_counts.py needs: [generate_study_population] outputs: moderately_sensitive: text: output/patient_count.json generate_study_population_ethnicity: run: cohortextractor:latest generate_cohort --study-definition study_definition_ethnicity --output-dir=output --output-format=feather outputs: highly_sensitive: cohort: output/input_ethnicity.feather join_ethnicity: run: python:latest python analysis/join_ethnicity.py needs: [generate_study_population, generate_study_population_ethnicity] outputs: highly_sensitive: cohort: output/input*.feather get_practice_count: run: python:latest python analysis/get_practice_count.py needs: [join_ethnicity] outputs: moderately_sensitive: text: output/practice_count.json generate_measures: run: cohortextractor:latest generate_measures --study-definition study_definition --output-dir=output needs: [join_ethnicity] outputs: moderately_sensitive: measure_csv: output/measure_*.csv generate_measures_cleaned: run: python:latest python analysis/clean_measures.py needs: [generate_measures] outputs: moderately_sensitive: measure_csv: output/measure_cleaned_*.csv generate_measures_demographics: run: python:latest python analysis/calculate_measures.py needs: [join_ethnicity] outputs: moderately_sensitive: measure: output/combined_measure_*.csv generate_notebook: run: jupyter:latest jupyter nbconvert /workspace/notebooks/sentinel_measures.ipynb --execute --to html --template basic --output-dir=/workspace/output --ExecutePreprocessor.timeout=86400 --no-input needs: [generate_measures, get_practice_count, get_patient_count] outputs: moderately_sensitive: notebook: output/sentinel_measures.html subplots: output/sentinel_measures_subplots.png generate_notebook_demographics: run: jupyter:latest jupyter nbconvert /workspace/notebooks/sentinel_measures_demographics.ipynb --execute --to html --template basic --output-dir=/workspace/output --ExecutePreprocessor.timeout=86400 --no-input needs: [generate_measures, generate_measures_demographics] outputs: moderately_sensitive: notebook: output/sentinel_measures_demographics.html get_population_count: run: python:latest python analysis/population_counts.py needs: [join_ethnicity] outputs: moderately_sensitive: text: output/*_count.csv demographic_changes: run: python:latest python notebooks/demographic_change.py needs: [generate_measures_demographics] outputs: moderately_sensitive: csv: output/demographics_differences.csv csv_sorted: output/demographics_differences_sorted.csv # run_tests: # run: python:latest python -m pytest --junit-xml=output/pytest.xml --verbose # outputs: # moderately_sensitive: # log: output/pytest.xml
State is inferred from the related Jobs.
JobRequestError: All requested actions were already scheduled to run: __error__ (State.FAILED)demographic_changes (State.SUCCEEDED)generate_measures (State.SUCCEEDED)generate_measures_cleaned (State.SUCCEEDED)generate_measures_demographics (State.SUCCEEDED)generate_notebook (State.SUCCEEDED)generate_notebook_demographics (State.SUCCEEDED)generate_study_population (State.SUCCEEDED)generate_study_population_ethnicity (State.SUCCEEDED)generate_study_population_practice_count (State.SUCCEEDED)get_patient_count (State.SUCCEEDED)get_population_count (State.SUCCEEDED)get_practice_count (State.SUCCEEDED)join_ethnicity (State.SUCCEEDED)run_all (State.SUCCEEDED)
Timings set to UTC timezone.
- Runtime: 00:00:00