Job request: 11644
- Organisation:
- University of Bristol
- Workspace:
- vaccine-effectiveness-in-children
- ID:
- 7kvmbb6omxjpmjkz
This page shows the technical details of what happened when the authorised researcher Will Hulme requested one or more actions to be run against real patient data in the project, within a secure environment.
By cross-referencing the list of jobs with the
pipeline section below, you can infer what
security level
various outputs were written to. Researchers can never directly
view outputs marked as
highly_sensitive
;
they can only request that code runs against them. Outputs
marked as
moderately_sensitive
can be viewed by an approved researcher by logging into a highly
secure environment. Only outputs marked as
moderately_sensitive
can be requested for release to the public, via a controlled
output review service.
Jobs
-
- Job identifier:
-
weg35mh2uwx55qu4
-
- Job identifier:
-
7oq6rmmrbwklb2mm
-
- Job identifier:
-
zv5cmwcuwponi5vy
-
- Job identifier:
-
3clxzsubilstb6zp
-
- Job identifier:
-
pt6jvhxus7pazc6y
-
- Job identifier:
-
sdaebzntq7zhaqmo
-
- Job identifier:
-
dconbyjsqwkjh7pg
-
- Job identifier:
-
5qztafpdqgoqfxv3
Pipeline
Show project.yaml
version: '3.0'
expectations:
population_size: 1000
actions:
extract_treated:
run: cohortextractor:latest generate_cohort
--study-definition study_definition_treated
--output-file output/input_treated.feather
outputs:
highly_sensitive:
cohort: output/input_treated.feather
data_process_treated:
run: r:latest analysis/data_process_treated.R over12
needs:
- extract_treated
outputs:
highly_sensitive:
rds: output/data/data_treated_eligible.rds
moderately_sensitive:
flowchart: output/data/flowchart_treated_eligible.csv
# Skim data
skim_data_treated:
run: r:latest analysis/data_skim.R output/data/data_treated_eligible.rds output/data_properties
needs: [data_process_treated]
outputs:
moderately_sensitive:
txt1: output/data_properties/data_treated_eligible_skim.txt
txt2: output/data_properties/data_treated_eligible_coltypes.txt
txt3: output/data_properties/data_treated_eligible_tabulate.txt
## matching round 1
extract_control_potential1:
run: cohortextractor:latest generate_cohort
--study-definition study_definition_control_potential
--output-file output/input_control_potential1.feather
--output-format feather
--param agegroup=over12
--param matching_round=1
--param index_date='2021-09-20'
outputs:
highly_sensitive:
cohort: output/input_control_potential1.feather
data_process_control_potential1:
run: r:latest analysis/data_process_control.R over12 1
needs:
- extract_control_potential1
outputs:
highly_sensitive:
rds: output/data/data_control_potential1.rds
matching1:
run: r:latest analysis/matching.R over12 1
needs:
- data_process_treated
- data_process_control_potential1
outputs:
highly_sensitive:
rds: output/match/data_potential_matchstatus1.rds
#rds: output/match/data_potential_matched1.rds
csv: output/match/potential_matchedcontrols1.csv.gz
# skim_control_potential1:
# run: r:latest analysis/data_skim.R output/match/data_potential_matched1.rds output/data_properties
# needs: [matching1]
# outputs:
# moderately_sensitive:
# txt1: output/data_properties/data_potential_matched1_skim.txt
# txt2: output/data_properties/data_potential_matched1_coltypes.txt
# txt3: output/data_properties/data_potential_matched1_tabulate.txt
extract_control_match1:
run: cohortextractor:latest generate_cohort
--study-definition study_definition_control_match
--output-file output/input_control_match1.feather
--param agegroup=over12
--param matching_round=1
--param index_date='2021-09-20'
needs:
- matching1
outputs:
highly_sensitive:
cohort: output/input_control_match1.feather
matching_filter1:
run: r:latest analysis/matching_filter.R over12 1
needs:
- matching1
- data_process_treated
- extract_control_match1
- extract_control_potential1 # this is only necessary for the dummy data
outputs:
highly_sensitive:
rds1: output/match/data_matchstatus_allrounds1.rds
rds2: output/match/data_successful_matchedcontrols1.rds
csv: output/match/cumulative_matchedcontrols1.csv.gz
extract_finalmatched:
run: cohortextractor:latest generate_cohort
--study-definition study_definition_finalmatched
--output-file output/input_finalmatched.feather
needs:
- matching_filter1
#- matching_filter2
outputs:
highly_sensitive:
cohort: output/input_finalmatched.feather
data_process_finalmatched:
run: r:latest analysis/data_process_finalmatched.R over12
needs:
- data_process_treated
- extract_finalmatched
- matching_filter1
#- matching_filter2
outputs:
highly_sensitive:
rds: output/data/data_finalmatched.rds
model_km_over12_all_postest:
run: r:latest analysis/km.R over12 all postest
needs:
- data_process_finalmatched
outputs:
moderately_sensitive:
csv: output/comparison/over12/all/postest/*csv
png: output/comparison/over12/all/postest/*png
Timeline
-
Created:
-
Started:
-
Finished:
-
Runtime: 01:33:37
These timestamps are generated and stored using the UTC timezone on the TPP backend.
Job information
- Status
-
Failed
- Backend
- TPP
- Workspace
- vaccine-effectiveness-in-children
- Requested by
- Will Hulme
- Branch
- main
- Force run dependencies
- No
- Git commit hash
- 05fefc4
- Requested actions
-
-
extract_control_potential1
-
data_process_control_potential1
-
matching1
-
extract_control_match1
-
matching_filter1
-
extract_finalmatched
-
data_process_finalmatched
-
model_km_over12_all_postest
-
Code comparison
Compare the code used in this Job Request