Job request: 5682
- Organisation:
- The London School of Hygiene & Tropical Medicine
- Workspace:
- vaccine-neuro-cohort
- ID:
- sey26kf5g7wegcxf
This page shows the technical details of what happened when the authorised researcher Anna Schultze requested one or more actions to be run against real patient data within a secure environment.
By cross-referencing the list of jobs with the pipeline section below, you can infer what security level the outputs were written to.
The output security levels are:
-
highly_sensitive
- Researchers can never directly view these outputs
- Researchers can only request code is run against them
-
moderately_sensitive
- Can be viewed by an approved researcher by logging into a highly secure environment
- These are the only outputs that can be requested for public release via a controlled output review service.
Jobs
-
- Job identifier:
-
3gmjytxzxtl6kq2v
-
- Job identifier:
-
n6foekunf3dga2h5
-
- Job identifier:
-
mpc4u4gu3unq6f36 - Error:
- nonzero_exit: Job exited with error code 137: likely means it ran out of memory
-
- Job identifier:
-
kgxtqlhejm66ye3a - Error:
- dependency_failed: Not starting as dependency failed
-
- Job identifier:
-
oy23oepcowla6fex - Error:
- dependency_failed: Not starting as dependency failed
Pipeline
Show project.yaml
version: '3.0'
expectations:
population_size: 1000
actions:
# EXTRACT DATA FOR EXPOSED AND POTENTIAL CONTROLS
## Extract data on vaccinated people
## NB; This study definition only contains the data needed for matching
generate_exposed_cohort:
run: cohortextractor:latest generate_cohort --study-definition study_definition_exposed
outputs:
highly_sensitive:
cohort: output/input_exposed.csv
## Extract data on potential concurrent controls
generate_concurrent_controls:
run: cohortextractor:latest generate_cohort --study-definition study_definition_concurrent_controls
outputs:
highly_sensitive:
cohort: output/input_concurrent_controls.csv
## Extract data on potential historical controls
# MATCH EXPOSED TO CONTROLS
# Note that exposed and resulting matches need to_c be extracted into separate csvs to allow appropriate covariate information to be extracted
## Match vaccinated people to concurrent controls
match_concurrent:
run: python:latest python analysis/matching.py
needs: [generate_exposed_cohort, generate_concurrent_controls]
outputs:
moderately_sensitive:
matching_report: output/matching_report_concurrent.txt
highly_sensitive:
matched_cases: output/matched_cases_concurrent.csv
matched_matches: output/matched_matches_concurrent.csv
matched_all: output/matched_combined_concurrent.csv
## Match vaccinated people to historical controls
## Extract time-dependent data for matched concurrent controls
add_covariates_concurrent_controls:
run: cohortextractor:latest generate_cohort --study-definition study_definition_complete_concurrent
needs: [match_concurrent]
outputs:
highly_sensitive:
cohort: output/input_complete_concurrent.csv
## Extract time-dependent data for matched historical controls
# CREATE THE REQUIRED POPULATIONS
## ie, merge case and match data, apply remaining inclusions and exclusions
apply_exclusion_criteria:
run: stata-mp:latest analysis/01_apply_exclusion_criteria.do
needs: [add_covariates_concurrent_controls, match_concurrent]
outputs:
moderately_sensitive:
log: output/logs/01_apply_exclusion_criteria.log
highly_sensitive:
interim_data: output/concurrent_cohort.dta
Timeline
-
Created:
-
Started:
-
Finished:
-
Runtime: 62:55:05
These timestamps are generated and stored using the UTC timezone on the TPP backend.
Job request
- Status
-
Failed
- Backend
- TPP
- Workspace
- vaccine-neuro-cohort
- Requested by
- Anna Schultze
- Branch
- main
- Force run dependencies
- Yes
- Git commit hash
- 86b5cf0
- Requested actions
-
-
run_all
-
Code comparison
Compare the code used in this job request
- No previous job request available for comparison