Job request: 27179

Organisation:: Bennett Institute
Workspace:: migration-short-data-report
ID:: h6bspzf2qhzaxo6u

This page shows the technical details of what happened when the authorised researcher Yamina Boukari requested one or more actions to be run against real patient data within a secure environment.

By cross-referencing the list of jobs with the pipeline section below, you can infer what security level the outputs were written to.

The output security levels are:

highly_sensitive
- Researchers can never directly view these outputs
- Researchers can only request code is run against them
moderately_sensitive
- Can be viewed by an approved researcher by logging into a highly secure environment
- These are the only outputs that can be requested for public release via a controlled output review service.

Jobs

Action:

generate_dataset_for_census_2011

Status:

Succeeded

Job identifier:

fgsmdpdcrfogla3w
Action:

generate_dataset_for_census_2021

Status:

Succeeded

Job identifier:

ok4bd3kgjkaov4mn
Action:

generate_full_study_cohort

Status:

Succeeded

Job identifier:

adupyws7beaazw3i
Action:

generate_annual_migrant_counts_3cat_withdoe

Status:

Succeeded

Job identifier:

qclppbywokgagxuc
Action:

generate_annual_migrant_counts_migration_status_types

Status:

Running

Job identifier:

2eo7a7ifhdqhl5fp
Action:

generate_annual_migration_coding_counts

Status:

Succeeded

Job identifier:

abky6of4ohegwf2a
Action:

generate_primary_care_planned_encounters

Status:

Running

Job identifier:

p5or7uyicfb6ncsf
Action:

generate_histogram_migration_codes_without_date_restrictions

Status:

Pending

Job identifier:

fnapm46qjram5ooj

Status message:

waiting_on_workers: Waiting on available workers
Action:

generate_demographics_full_study_table_migration_types

Status:

Pending

Job identifier:

2i5b4rusxr2oduwv

Status message:

waiting_on_workers: Waiting on available workers
Action:

generate_summary_migration_codes_without_date_restrictions

Status:

Pending

Job identifier:

ryotfyxa33s3arrx

Status message:

waiting_on_workers: Waiting on available workers
Action:

generate_date_of_uk_entry_description_combinations

Status:

Pending

Job identifier:

cv54gchwatbqjakv

Status message:

waiting_on_workers: Waiting on available workers
Action:

generate_age_summary_census_2011_mig_3cat_withdoe

Status:

Pending

Job identifier:

f3stetz6nlwz6wsk

Status message:

waiting_on_workers: Waiting on available workers
Action:

generate_age_summary_census_2021_mig_3cat_withdoe

Status:

Pending

Job identifier:

klivybqx4yhycztd

Status message:

waiting_on_workers: Waiting on available workers
Action:

generate_demographics_full_study_table_3cat_withdoe

Status:

Pending

Job identifier:

fjk5wozo2esdm3rd

Status message:

waiting_on_workers: Waiting on available workers
Action:

generate_migration_coding_summary

Status:

Pending

Job identifier:

ojtrynib7mjott7y

Status message:

waiting_on_workers: Waiting on available workers
Action:

generate_date_variable_checks_summary

Status:

Pending

Job identifier:

4ot54kgclknl74vl

Status message:

waiting_on_workers: Waiting on available workers
Action:

generate_migration_code_combinations_summary

Status:

Pending

Job identifier:

rupqmze7f5d62y63

Status message:

waiting_on_workers: Waiting on available workers
Action:

generate_date_of_uk_entry_description

Status:

Pending

Job identifier:

5l4e5o4hnijv4y5r

Status message:

waiting_on_workers: Waiting on available workers
Action:

generate_demographics_census_2021_study_table_3cat_withdoe

Status:

Pending

Job identifier:

cy4hiazi7liyl4xk

Status message:

waiting_on_workers: Waiting on available workers
Action:

generate_demographics_census_2011_study_table_mig_3cat_withdoe

Status:

Pending

Job identifier:

3ryo6sdxpkkhpawc

Status message:

waiting_on_workers: Waiting on available workers

Pipeline

Show project.yaml

version: '5.0'

actions:
  generate_full_study_cohort:
    run: ehrql:v1 generate-dataset analysis/dataset_definition_full_study_cohort.py --output output/cohorts/full_study_cohort.arrow
    outputs:
      highly_sensitive:
         dataset: output/cohorts/full_study_cohort.arrow

  generate_dataset_for_census_2021:
    run: ehrql:v1 generate-dataset analysis/dataset_definition_census_cohorts.py 
      --output output/cohorts/census_2021_study_cohort.arrow
      --
      --census-date "2021-03-21"
    outputs:
      highly_sensitive:
        dataset: output/cohorts/census_2021_study_cohort.arrow

  generate_dataset_for_census_2011:
    run: ehrql:v1 generate-dataset analysis/dataset_definition_census_cohorts.py 
      --output output/cohorts/census_2011_study_cohort.arrow
      --
      --census-date "2011-03-27"
    outputs:
      highly_sensitive:
        dataset: output/cohorts/census_2011_study_cohort.arrow

  generate_demographics_full_study_table_3cat_withdoe:
    run: r:v2 analysis/process_full_cohort_data.R output/tables/demographics_full_study_cohort_3cat_withdoe.csv mig_status_3_cat_withdoe
    needs:
    - generate_full_study_cohort
    outputs:
      moderately_sensitive:
        csv: output/tables/demographics_full_study_cohort_3cat_withdoe.csv

  # generate_demographics_full_study_table_6cat_withdoe:
  #   run: r:v2 analysis/process_full_cohort_data.R output/tables/demographics_full_study_cohort_6cat_withdoe.csv mig_status_6_cat_withdoe
  #   needs:
  #   - generate_full_study_cohort
  #   outputs:
  #     moderately_sensitive:
  #       csv: output/tables/demographics_full_study_cohort_6cat_withdoe.csv

  generate_demographics_full_study_table_migration_types:
    run: r:v2 analysis/process_full_cohort_data_migration_types.R 
    needs:
    - generate_full_study_cohort
    outputs:
      moderately_sensitive:
        csv: output/tables/demographics_full_study_cohort_migration_types.csv

  # generate_demographics_uk_entry_cohort:
  #   run: r:v2 analysis/process_date_of_uk_entry_cohort.R 
  #   needs:
  #   - generate_full_study_cohort
  #   outputs:
  #     moderately_sensitive:
  #       csv: output/tables/demographics_date_of_uk_entry_and_no_other_migration_info_cohort.csv

  generate_demographics_census_2011_study_table_mig_3cat_withdoe:
    run: r:v2 analysis/process_census_cohort_data.R output/cohorts/census_2011_study_cohort.arrow output/tables/demographics_census_2011_cohort_3cat_withdoe.csv mig_status_3_cat_withdoe
    needs:
    - generate_dataset_for_census_2011
    outputs:
      moderately_sensitive:
        csv: output/tables/demographics_census_2011_cohort_3cat_withdoe.csv

  # generate_demographics_census_2011_study_table_mig_6cat_withdoe:
  #   run: r:v2 analysis/process_census_cohort_data.R output/cohorts/census_2011_study_cohort.arrow output/tables/demographics_census_2011_cohort_6cat_withdoe.csv mig_status_6_cat_withdoe
  #   needs:
  #   - generate_dataset_for_census_2011
  #   outputs:
  #     moderately_sensitive:
  #       csv: output/tables/demographics_census_2011_cohort_6cat_withdoe.csv

  generate_demographics_census_2021_study_table_3cat_withdoe:
    run: r:v2 analysis/process_census_cohort_data.R output/cohorts/census_2021_study_cohort.arrow output/tables/demographics_census_2021_cohort_3cat_withdoe.csv mig_status_3_cat_withdoe
    needs:
    - generate_dataset_for_census_2021
    outputs:
      moderately_sensitive:
        csv: output/tables/demographics_census_2021_cohort_3cat_withdoe.csv
  
  # generate_demographics_census_2021_study_table_6cat_withdoe:
  #   run: r:v2 analysis/process_census_cohort_data.R output/cohorts/census_2021_study_cohort.arrow output/tables/demographics_census_2021_cohort_6cat_withdoe.csv mig_status_6_cat_withdoe
  #   needs:
  #   - generate_dataset_for_census_2021
  #   outputs:
  #     moderately_sensitive:
  #       csv: output/tables/demographics_census_2021_cohort_6cat_withdoe.csv

  generate_age_summary_census_2011_mig_3cat_withdoe:
    run: r:v2 analysis/census_cohort_age_summary.R output/cohorts/census_2011_study_cohort.arrow output/tables/age_summary_census_2011_cohort_3cat_withdoe.csv mig_status_3_cat_withdoe
    needs:
    - generate_dataset_for_census_2011
    outputs:
      moderately_sensitive:
        csv: output/tables/age_summary_census_2011_cohort_3cat_withdoe.csv

  generate_age_summary_census_2021_mig_3cat_withdoe:
    run: r:v2 analysis/census_cohort_age_summary.R output/cohorts/census_2021_study_cohort.arrow output/tables/age_summary_census_2021_cohort_3cat_withdoe.csv mig_status_3_cat_withdoe
    needs:
    - generate_dataset_for_census_2021
    outputs:
      moderately_sensitive:
        csv: output/tables/age_summary_census_2021_cohort_3cat_withdoe.csv

  generate_date_of_uk_entry_description:
    run: r:v2 analysis/date_of_entry_to_uk.R
    needs:
    - generate_full_study_cohort
    outputs:
      moderately_sensitive:
        csv: output/tables/date_of_uk_entry_description.csv

  generate_date_of_uk_entry_description_combinations:
    run: r:v2 analysis/date_of_entry_to_uk_combinations.R
    needs:
    - generate_full_study_cohort
    outputs:
      moderately_sensitive:
        csv: output/tables/date_of_uk_entry_combinations.csv

  generate_annual_migrant_counts_3cat_withdoe:
    run: ehrql:v1 generate-measures analysis/generate_annual_migrant_counts_3cat_withdoe.py --output output/tables/annual_counts/3cat_withdoe/:csv
    outputs:
      moderately_sensitive:
        csv: output/tables/annual_counts/3cat_withdoe/*.csv 
  
  # generate_annual_migrant_counts_6cat_withdoe:
  #   run: ehrql:v1 generate-measures analysis/generate_annual_migrant_counts_6cat_withdoe.py --output output/tables/annual_counts/6cat_withdoe/:csv
  #   outputs:
  #     moderately_sensitive:
  #       csv: output/tables/annual_counts/6cat_withdoe/*.csv

  generate_annual_migrant_counts_migration_status_types:
    run: ehrql:v1 generate-measures analysis/generate_annual_migrant_counts_migration_status_types.py --output output/tables/annual_counts/migration_status_types/:csv
    outputs:
      moderately_sensitive:
        csv: output/tables/annual_counts/migration_status_types/*.csv

  generate_migration_coding_summary:
    run: r:v2 analysis/migration_coding.R
    needs:
    - generate_full_study_cohort
    outputs:
      moderately_sensitive:
        csv: output/tables/migration_coding_summary.csv

  generate_migration_code_combinations_summary:
    run: r:v2 analysis/migration_coding_combinations.R
    needs:
    - generate_full_study_cohort
    outputs:
      moderately_sensitive:
        csv: output/tables/migration_code_combinations_summary.csv

  generate_date_variable_checks_summary:
    run: r:v2 analysis/date_variable_checks.R
    needs:
    - generate_full_study_cohort
    outputs:
      moderately_sensitive:
        csv: output/tables/date_variable_checks.csv

  generate_annual_migration_coding_counts:
    run: ehrql:v1 generate-measures analysis/annual_code_counts.py --output output/tables/annual_counts/migration_coding_occ_comparison.csv
    outputs:
      moderately_sensitive:
        csv: output/tables/annual_counts/migration_coding_occ_comparison.csv

  generate_primary_care_planned_encounters:
    run: ehrql:v1 generate-measures analysis/primary_care_activity_comparisons.py --output output/tables/annual_counts/primary_care_comparison.csv
    outputs:
      moderately_sensitive:
        csv: output/tables/annual_counts/primary_care_comparison.csv

  generate_histogram_migration_codes_without_date_restrictions:
    run: r:v2 analysis/histogram_migration_codes_without_date_restrictions.r
    needs:
    - generate_full_study_cohort
    outputs:
      moderately_sensitive:
        png: output/figures/histogram_migration_coding_no_restrictions.png

  generate_summary_migration_codes_without_date_restrictions:
    run: r:v2 analysis/summary_migration_codes_without_date_restrictions.r
    needs:
    - generate_full_study_cohort
    outputs:
      moderately_sensitive:
        csv: output/tables/summary_migration_coding_without_restrictions_by_mig_status.csv

Job statistics

Status	Count	Percentage
Pending	13	65%
Running	2	10%
Succeeded	5	25%
Failed	0	0%

5 / 20 (25%) complete

Timeline

Created: 14 hours ago 22 Oct 2026 14:44:05 UTC
Started: 11 hours ago 22 Oct 2026 18:24:43 UTC
Runtime: 09:59:47

These timestamps are generated and stored using the UTC timezone on the TPP backend.

Job request

Status: Running
Backend: TPP
Workspace: migration-short-data-report
Requested by: Yamina Boukari
Branch: main
Force run dependencies: Yes
Git commit hash: 4b77836
Requested actions: generate_full_study_cohort

generate_dataset_for_census_2021

generate_dataset_for_census_2011

generate_demographics_full_study_table_3cat_withdoe

generate_demographics_full_study_table_migration_types

generate_demographics_census_2011_study_table_mig_3cat_withdoe

generate_demographics_census_2021_study_table_3cat_withdoe

generate_age_summary_census_2011_mig_3cat_withdoe

generate_age_summary_census_2021_mig_3cat_withdoe

generate_date_of_uk_entry_description

generate_annual_migrant_counts_3cat_withdoe

generate_annual_migrant_counts_migration_status_types

generate_migration_coding_summary

generate_migration_code_combinations_summary

generate_date_variable_checks_summary

generate_annual_migration_coding_counts

generate_primary_care_planned_encounters

generate_histogram_migration_codes_without_date_restrictions

run_all

Code comparison

Compare the code used in this job request