Job Request: 2969 ln5bs6y55kr3tiql

This page shows the technical details of what happened when authorised researcher Millie Green requested one or more actions to be run against real patient data in the Factors Associated with Covid-19 Vaccination project, within a secure environment.

By cross-referencing the indicated Requested Actions with the Pipeline section below, you can infer what security level various outputs were written to. Outputs marked as highly_sensitive can never be viewed directly by a researcher; they can only request that code runs against them. Outputs marked as moderately_sensitive can be viewed by an approved researcher by logging into a highly secure environment. Only outputs marked as moderately_sensitive can be requested for release to the public, via a controlled output review service.

State

State is inferred from the related Jobs
Status: failed

Config

Backend: emis
Workspace: factors-associated-with-covid-19-vaccination-emis-test (EMIS)
Creator: acagreen17
Force run dependencies?: False
Git Commit Hash: 74304b6
Requested Actions:
  • data_process
  • data_properties
  • model_final

Timings

Created: 1 week ago
Started: 1 week ago
Finished: 1 week ago
Runtime: 00:05:54

Pipeline

######################################

# This script defines the project pipeline - it specifys the execution orders for all the code in this
# repo using a series of actions.

######################################

version: '3.0'

expectations:
population_size: 1000000

actions:

# Extract data for study population flow chart
generate_study_population_flow_chart_data:
run: cohortextractor:latest generate_cohort --study-definition study_definition_flow_chart
outputs:
highly_sensitive:
cohort: output/input_flow_chart.csv

# Calculate numbers for study population flow chart
flow_chart:
run: r:latest -e 'rmarkdown::render("analysis/R/Markdown/Study_definition_flow_chart.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
needs: [generate_study_population_flow_chart_data]
outputs:
moderately_sensitive:
html: output/Study_definition_flow_chart.html

# Extract study data
generate_study_population:
run: cohortextractor:latest generate_cohort --study-definition study_definition
outputs:
highly_sensitive:
cohort: output/input.csv

# Process data
data_process:
run: r:latest analysis/R/Scripts/00_process_data.R
needs: [generate_study_population]
outputs:
highly_sensitive:
data1: output/data/data_all.rds
data2: output/data/data_modelling.rds

# Summarise data
data_properties:
run: r:latest analysis/R/Scripts/01_data_properties.R output/data/data_all.rds output/data_properties
needs: [generate_study_population, data_process]
outputs:
moderately_sensitive:
datasummary: output/data_properties/data_all*.txt

# # More data summaries
# data_summaries:
# run: r:latest -e 'rmarkdown::render("analysis/R/Markdown/Data_summaries.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
# needs: [generate_study_population, data_process]
# outputs:
# moderately_sensitive:
# html: output/Data_summaries.html

# # Coxph models
# cox_models:
# run: r:latest analysis/R/Scripts/02a_models_coxph.R
# needs: [generate_study_population, data_process]
# outputs:
# highly_sensitive :
# models: output/models/testing/mod*.rds

# # Coxme model
# coxme_model:
# run: r:latest analysis/R/Scripts/02b_models_coxme.R
# needs: [generate_study_population, data_process]
# outputs:
# highly_sensitive :
# models: output/models/testing/mod_test*.rds

# # Model summaries
# cox_models_summaries:
# run: r:latest -e 'rmarkdown::render("analysis/R/Markdown/Model_comparisons.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
# needs: [generate_study_population, data_process, cox_models, coxme_model]
# outputs:
# moderately_sensitive:
# html: output/Model_comparisons.html

# # AFT vs Cox models feasibility
# aft_vs_cox_poc:
# run: r:latest analysis/R/Scripts/02c_models_cox_vs_aft_poc.R
# needs: [generate_study_population, data_process]
# outputs:
# highly_sensitive :
# models: output/models/testing/cox_vs_aft/mod*.rds
# moderately_sensitive:
# tables: output/models/testing/cox_vs_aft/table*.csv
# plots: output/models/testing/cox_vs_aft/plot*.svg

# # AFT models
# aft_vs_cox_models:
# run: r:latest analysis/R/Scripts/02d_models_cox_vs_aft.R
# needs: [generate_study_population, data_process]
# outputs:
# highly_sensitive :
# models: output/models/cox_vs_aft/mod*.rds

# # Model comparison
# aft_vs_cox_comparison:
# run: r:latest -e 'rmarkdown::render("analysis/R/Markdown/Cox_vs_aft_model_comparison.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
# needs: [generate_study_population, data_process, aft_vs_cox_models]
# outputs:
# moderately_sensitive:
# html: output/Cox_vs_aft_model_comparison.html

# Final (stratified cox) model
model_final:
run: r:latest analysis/R/Scripts/03_model_final.R
needs: [generate_study_population, data_process]
outputs:
highly_sensitive :
models: output/models/final/mod*.rds

# Strata summaries
strata_summary:
run: r:latest analysis/R/Scripts/04_strata_summary.R
needs: [data_process, model_final]
outputs:
moderately_sensitive:
plots: output/models/final/plot_strata*.svg
csv: output/models/final/cmlhaz_quantiles.csv

# # Results summary
# results_summary:
# run: r:latest -e 'rmarkdown::render("analysis/R/Markdown/Results_summary.Rmd", knit_root_dir = "/workspace", output_dir="/workspace/output")'
# needs: [data_process, model_final]
# outputs:
# moderately_sensitive:
# html: output/Results_summary.html

# Table and figure
table_and_figure:
run: r:latest analysis/R/Scripts/05_table_and_figure.R
needs: [generate_study_population, data_process, model_final]
outputs:
moderately_sensitive:
tables: output/models/final/tab*.csv
plots: output/models/final/plot*.svg



Jobs

ID Status Action
udlvr4r5l5jczspq succeeded data_process View
wpuksucf3isgci7a succeeded data_properties View
yws4iegvrdrreecy failed model_final View