mkdir -p failed for path /home/moltis/.config/matplotlib: [Errno 13] Permission denied: '/home/moltis/.config/matplotlib'
Matplotlib created a temporary cache directory at /tmp/matplotlib-ep8gx_14 because there was an issue with the default path (/home/moltis/.config/matplotlib); it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
FWI mode: hourly Processed dir: /mnt/fast_data/workspaces/pea-met-network/data/processed
1. Data Loading¶
Load hourly processed data for all stations. Station list is derived from disk.
Stations found: ['cavendish', 'greenwich', 'north_rustico', 'stanhope', 'stanley_bridge', 'tracadie'] cavendish: 26092 hourly rows (2023-04-01 00:00:00+00:00 to 2026-03-23 03:00:00+00:00)
greenwich: 26092 hourly rows (2023-04-01 00:00:00+00:00 to 2026-03-23 03:00:00+00:00) north_rustico: 26113 hourly rows (2023-04-01 00:00:00+00:00 to 2026-03-24 00:00:00+00:00)
stanhope: 24144 hourly rows (2023-04-01 00:00:00+00:00 to 2025-12-31 23:00:00+00:00) stanley_bridge: 26092 hourly rows (2023-04-01 00:00:00+00:00 to 2026-03-23 03:00:00+00:00)
tracadie: 23956 hourly rows (2023-06-29 00:00:00+00:00 to 2026-03-23 03:00:00+00:00) Combined: 152489 hourly rows, 6 stations
station total_records
cavendish 26092
greenwich 26092
north_rustico 26113
stanhope 24144
stanley_bridge 26092
tracadie 23956
2.2 Temporal Coverage Summary¶
Visual overview of data availability by station over time.
Saved: notebooks/figures/temporal_coverage.png
2.3 Missingness Heatmaps¶
Each heatmap is split into two panels:
Left panel — Core Sensors / FWI: Variables directly used as FWI inputs (temp, RH, wind, rain) plus FWI output indices (FFMC, DMC, DC, ISI, BUI, FWI on daily). These are the columns that matter for the fire weather pipeline.
Right panel — Extended Sensors: Auxiliary measurements (wind direction, solar radiation, barometric pressure, dew point, gust speed) that are not used in FWI computation. Many stations lack some or all of these sensors, so 100% missing values here indicate a missing instrument, not a data problem.
Hourly heatmap — raw observations from
station_hourly.csv.Daily heatmap — daily aggregates with FWI components from
<station>_daily_compliant.csv(falls back tostation_daily.csv).
Saved: notebooks/figures/missingness_heatmap_hourly.png
Hourly missingness summary (core):
air_temperature_c relative_humidity_pct wind_speed_kmh rain_mm
station
cavendish 14.5 14.5 0.0 0.0
greenwich 4.1 16.2 4.0 0.1
north_rustico 4.1 4.1 0.0 0.0
stanhope 0.3 0.3 0.3 0.2
stanley_bridge 0.0 17.2 0.0 0.0
tracadie 0.0 12.8 0.0 0.0
Hourly missingness summary (extended):
wind_direction_deg solar_radiation_w_m2 barometric_pressure_kpa dew_point_c wind_gust_speed_kmh
station
cavendish 14.5 14.5 100.0 14.5 14.9
greenwich 12.1 12.1 100.0 16.2 12.5
north_rustico 2.1 1.3 1.3 1.3 4.3
stanhope 2.1 100.0 100.0 0.3 100.0
stanley_bridge 18.4 18.4 34.7 100.0 32.1
tracadie 13.0 13.0 18.8 100.0 15.5
Saved: notebooks/figures/missingness_heatmap_daily.png
Daily missingness summary (core + FWI):
air_temperature_c relative_humidity_pct wind_speed_kmh rain_mm ffmc dmc dc isi bui fwi
station
cavendish 14.6 14.6 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
greenwich 4.1 16.2 4.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
north_rustico 4.0 4.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
stanhope 0.6 0.7 0.6 0.0 0.0 0.0 0.0 0.0 0.0 0.0
stanley_bridge 0.0 17.2 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
tracadie 0.0 12.8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Daily missingness summary (extended):
barometric_pressure_kpa dew_point_c solar_radiation_w_m2 wind_direction_deg wind_gust_speed_kmh
station
cavendish 100.0 14.6 14.6 14.6 15.0
greenwich 100.0 16.2 12.1 12.1 12.5
north_rustico 1.2 1.2 1.3 1.9 4.1
stanhope 100.0 0.6 NaN 1.2 NaN
stanley_bridge 34.6 100.0 18.4 18.4 32.0
tracadie 18.7 100.0 12.9 12.9 15.5
2.4 Imputation Summary¶
Gaps filled per station per variable from the pipeline imputation report.
Imputation report: 717 entries Gaps filled per station per variable: variable air_temperature_c barometric_pressure_kpa dew_point_c relative_humidity_pct solar_radiation_w_m2 wind_direction_deg wind_speed_kmh station cavendish 3789 26092 3787 3787 3786 3786 3890 greenwich 10158 26092 4216 4229 3147 3145 3249 north_rustico 1076 330 331 1075 331 547 1188 stanhope 77 24144 68 79 0 516 68 stanley_bridge 4486 9043 26092 26092 4800 4800 8371 tracadie 3076 4508 23956 23956 3107 3107 3809 Imputation method breakdown: method linear_interpolation 575 preserve 142
Saved: notebooks/figures/imputation_summary.png
3. Fire Weather Index Analysis¶
Time series of FWI system components for all stations with computed indices.
3.1 FWI Component Time Series¶
Daily-mean FWI components for all stations. FFMC uses hourly values; DMC, DC, BUI, FWI use daily means.
Stations with FWI data: ['cavendish', 'greenwich', 'north_rustico', 'stanhope', 'stanley_bridge', 'tracadie']
Plotted FWI for: ['cavendish', 'greenwich', 'north_rustico', 'stanhope', 'stanley_bridge', 'tracadie'] Saved: notebooks/figures/fwi_timeseries.png
3.1b QF-Aware FWI Time Series¶
FWI values plotted with background shading indicating data quality.
- Yellow band: at least one FWI input was synthetically imputed (
qf == 1) - Red band: at least one FWI input was chain-break gap-filled (
qf == 9) - No shading: all FWI inputs from observed data (
qf == 0)
Stations without per-variable _qf columns (stanhope) are skipped.
Saved: notebooks/figures/fwi_qf_aware.png
3.2 FWI Mode Comparison¶
Overlay FWI values from hourly and compliant modes to show divergence (e.g., during chain breaks).
Mode comparison for: ['cavendish', 'greenwich', 'north_rustico', 'stanhope', 'stanley_bridge', 'tracadie'] Saved: notebooks/figures/fwi_mode_comparison.png
3.3 FWI Value Statistics¶
Descriptive statistics (min, max, mean, std) for FWI codes per station, sourced from the QA/QC report.
QA/QC report exists but has no fwi_stat_ columns. Columns available: ['station', 'hourly_rows', 'daily_rows', 'date_range_start', 'date_range_end', 'completeness', 'fwi_mode', 'pre_imp_missing_pct_air_temperature_c', 'pre_imp_missing_pct_relative_humidity_pct', 'pre_imp_missing_pct_wind_speed_kmh', 'pre_imp_missing_pct_rain_mm', 'post_imp_missing_pct_air_temperature_c', 'post_imp_missing_pct_air_temperature_c_qf', 'post_imp_missing_pct_barometric_pressure_kpa', 'post_imp_missing_pct_bui', 'post_imp_missing_pct_bui_qf', 'post_imp_missing_pct_dc', 'post_imp_missing_pct_dc_qf', 'post_imp_missing_pct_dew_point_c', 'post_imp_missing_pct_dmc', 'post_imp_missing_pct_dmc_qf', 'post_imp_missing_pct_ffmc', 'post_imp_missing_pct_ffmc_qf', 'post_imp_missing_pct_fwi', 'post_imp_missing_pct_fwi_qf', 'post_imp_missing_pct_isi', 'post_imp_missing_pct_isi_qf', 'post_imp_missing_pct_rain_mm', 'post_imp_missing_pct_rain_mm_qf', 'post_imp_missing_pct_relative_humidity_pct', 'post_imp_missing_pct_relative_humidity_pct_qf', 'post_imp_missing_pct_solar_radiation_w_m2', 'post_imp_missing_pct_wind_direction_deg', 'post_imp_missing_pct_wind_gust_speed_kmh', 'post_imp_missing_pct_wind_speed_kmh', 'post_imp_missing_pct_wind_speed_kmh_qf', 'duplicate_count', 'out_of_range_temp_count', 'out_of_range_rh_count', 'out_of_range_wind_count', 'quality_enforced_count', 'quality_flagged_count', 'out_of_range_pre_enforcement', 'out_of_range_post_enforcement', 'fwi_chain_breaks', 'carry_forward_days', 'carry_forward_pct', 'ffmc_min', 'ffmc_max', 'ffmc_mean', 'ffmc_std', 'dmc_min', 'dmc_max', 'dmc_mean', 'dmc_std', 'dc_min', 'dc_max', 'dc_mean', 'dc_std', 'isi_min', 'isi_max', 'isi_mean', 'isi_std', 'bui_min', 'bui_max', 'bui_mean', 'bui_std', 'fwi_min', 'fwi_max', 'fwi_mean', 'fwi_std']
3.4 Chain Break Analysis¶
Analysis of FWI chain breaks by station and cause, sourced from the chain break diagnostic report.
Chain break report: 143 entries Breaks per station by cause: cause input_missing startup station cavendish 19 2 greenwich 42 0 north_rustico 10 0 stanhope 44 2 stanley_bridge 11 0 tracadie 11 2 Breakdown by missing input: missing_input relative_humidity_pct 62 air_temperature_c, relative_humidity_pct, wind_speed_kmh 29 air_temperature_c 28 air_temperature_c, relative_humidity_pct 18
Saved: notebooks/figures/chain_breaks.png
4. Data Quality Report¶
Quality metrics from the QA/QC report, quality enforcement actions, and cross-station imputation audit.
4.1 Pre/Post Imputation Missingness¶
Side-by-side heatmap comparing missingness before and after imputation for the four core FWI input variables (temperature, relative humidity, wind speed, precipitation). Both panels share the same color scale so improvements are visually obvious.
Note: Pre-imputation data is only available for the four core variables — those are the ones the pipeline imputes. Extended sensor variables (solar radiation, barometric pressure, etc.) are not imputed by the pipeline.
Saved: notebooks/figures/missingness_pre_post_imputation.png
Pre/Post Imputation Missingness Comparison:
station variable pre_% post_% improved_%
greenwich air_temperature_c 38.93 4.06 34.87
greenwich rain_mm 0.13 0.13 0.00
greenwich relative_humidity_pct 16.21 16.19 0.02
greenwich wind_speed_kmh 12.45 3.96 8.49
cavendish air_temperature_c 14.52 14.52 0.00
cavendish rain_mm 0.03 0.03 0.00
cavendish relative_humidity_pct 14.51 14.51 0.00
cavendish wind_speed_kmh 14.91 0.00 14.91
north_rustico air_temperature_c 4.12 4.12 0.00
north_rustico rain_mm 0.04 0.04 0.00
north_rustico relative_humidity_pct 4.12 4.12 0.00
north_rustico wind_speed_kmh 4.55 0.00 4.55
stanley_bridge air_temperature_c 17.19 0.00 17.19
stanley_bridge rain_mm 0.00 0.00 0.00
stanley_bridge relative_humidity_pct 100.00 17.19 82.81
stanley_bridge wind_speed_kmh 32.08 0.00 32.08
tracadie air_temperature_c 12.84 0.00 12.84
tracadie rain_mm 0.00 0.00 0.00
tracadie relative_humidity_pct 100.00 12.84 87.16
tracadie wind_speed_kmh 15.90 0.00 15.90
stanhope air_temperature_c 0.32 0.32 0.00
stanhope rain_mm 0.19 0.19 0.00
stanhope relative_humidity_pct 0.33 0.33 0.00
stanhope wind_speed_kmh 0.28 0.28 0.00
4.2 Quality Enforcement Actions¶
Count of quality enforcement actions by type and station.
Quality enforcement report: 5215 actions Actions by type and station: action flag_only set_nan station cavendish 126 12 greenwich 1161 607 north_rustico 2559 87 stanhope 315 66 stanley_bridge 56 1 tracadie 136 89
4.3 Out-of-Range Summary¶
Out-of-range counts for temperature, RH, and wind per station from the QA/QC report.
Out-of-Range Summary:
station out_of_range_temp_count out_of_range_rh_count out_of_range_wind_count out_of_range_pre_enforcement out_of_range_post_enforcement
greenwich 0 0 0 0 0
cavendish 0 0 0 0 0
north_rustico 0 0 0 0 0
stanley_bridge 0 0 0 0 0
tracadie 0 0 0 0 0
stanhope 0 0 0 0 0
4.2b Quality Enforcement Summary (Visual)¶
Stacked bar chart of quality enforcement actions by station. Each bar segment represents an action type (set_nan, clip, etc.).
Saved: notebooks/figures/quality_enforcement_stacked.png
4.2c Synthetic Data Proportion Heatmap¶
Percentage of rows with qf > 0 (synthetic or chain-break) per station per variable.
Higher values indicate greater reliance on imputed/synthetic data.
Stations without _qf columns for a variable are shown as N/A.
Saved: notebooks/figures/synthetic_proportion_heatmap.png
4.2d Quality Flag Timeline¶
Colored bands showing when quality flags were raised per station.
Each row is a station; colors indicate the type of quality issue detected.
Stations without _quality_flags data are shown as empty.
Saved: notebooks/figures/quality_flag_timeline.png
4.4 Cross-Station Imputation Audit¶
Donor-to-recipient imputation pairs and counts.
Cross-station imputation audit: 78623 entries Columns: ['station', 'timestamp_utc', 'variable', 'imputed_value', 'quality_flag', 'source', 'method', 'donor_priority']
4.4b Cross-Station Donor Audit (Visual)¶
Dual-panel chart showing:
- Left: Donor contribution bars (how many values each donor provided)
- Right: Imputation method breakdown by station
Saved: notebooks/figures/cross_station_donor_audit.png
5. Principal Component Analysis¶
PCA on hourly temperature to identify station similarity patterns.
Station matrix shape: (26113, 6) Stations: ['cavendish', 'greenwich', 'north_rustico', 'stanhope', 'stanley_bridge', 'tracadie']
PCA Loadings:
station component loading explained_variance_ratio
cavendish PC1 0.413668 0.963466
greenwich PC1 0.396266 0.963466
north_rustico PC1 0.413277 0.963466
stanhope PC1 0.406986 0.963466
stanley_bridge PC1 0.405674 0.963466
tracadie PC1 0.413334 0.963466
cavendish PC2 -0.167839 0.018571
greenwich PC2 0.900793 0.018571
north_rustico PC2 -0.171177 0.018571
stanhope PC2 -0.054382 0.018571
stanley_bridge PC2 -0.323366 0.018571
tracadie PC2 -0.153547 0.018571
Saved: notebooks/figures/pca_scree.png Explained variance ratios: [0.9635, 0.0186, 0.0098, 0.0062, 0.0011, 0.0008]
Saved: notebooks/figures/pca_biplot.png
6. Hierarchical Clustering¶
Stations clustered by temperature correlation to identify redundancy groups.
Saved: notebooks/figures/clustering_dendrogram.png
Cluster assignments: stanhope tracadie cavendish north_rustico stanley_bridge greenwich Pairwise distance matrix: station cavendish greenwich north_rustico stanhope stanley_bridge tracadie station cavendish 0.000 0.093 0.006 0.033 0.028 0.006 greenwich 0.093 0.000 0.086 0.095 0.102 0.058 north_rustico 0.006 0.086 0.000 0.035 0.026 0.005 stanhope 0.033 0.095 0.035 0.000 0.057 0.031 stanley_bridge 0.028 0.102 0.026 0.057 0.000 0.026 tracadie 0.006 0.058 0.005 0.031 0.026 0.000
7. Redundancy Analysis¶
Combining PCA, clustering, and benchmarking against the Stanhope reference to identify redundant stations.
Benchmark results:
station reference_station overlap_count mean_abs_diff correlation observations
cavendish stanhope 21858 1.815121 0.966669 [0.7825000000000001, 0.4033333333333333, 0.20833333333333326, 0.6875, 0.7641666666666669, 0.31, 0.09166666666666656, 0.27416666666666667, 0.9775, 2.0741666666666667, 1.8708333333333333, 1.355, 0.34833333333333316, 2.1275, 1.9224999999999994, 1.0341666666666667, 0.4358333333333333, 0.3258333333333332, 0.061666666666666536, 0.385, 0.6675, 1.3608333333333331, 1.6858333333333333, 1.4425, 0.8008333333333333, 0.5425, 0.4474999999999998, 0.206666666666667, 0.8541666666666665, 0.5783333333333336, 0.047499999999999876, 0.11750000000000016, 0.5533333333333332, 0.39166666666666705, 1.8816666666666668, 1.03, 1.010833333333333, 0.6891666666666669, 0.4716666666666667, 0.8075000000000001, 1.1208333333333331, 1.3058333333333332, 2.419166666666667, 3.455833333333333, 3.476666666666666, 3.01, 2.100833333333333, 1.7983333333333333, 2.1883333333333335, 2.3058333333333327, 2.064166666666667, 1.4574999999999996, 0.8300000000000001, 0.7508333333333335, 0.41166666666666707, 0.746666666666667, 2.2258333333333336, 3.730833333333333, 4.753333333333334, 5.203333333333334, 5.3625, 5.308333333333334, 4.73, 4.906666666666666, 3.4991666666666665, 1.6374074074074074, 0.3166666666666669, 2.6056666666666666, 1.8436666666666666, 1.6303333333333332, 0.7153333333333327, 1.8246666666666664, 1.4136666666666666, 0.5606666666666669, 2.0686666666666667, 0.865, 0.18466666666666665, 0.7433333333333332, 0.028666666666666618, 1.4756666666666665, 3.2643333333333335, 4.341666666666667, 4.652666666666667, 4.5523333333333325, 0.7736666666666672, 0.6036666666666664, 1.8376666666666672, 3.6559999999999997, 2.684333333333333, 3.2223333333333333, 4.569999999999999, 5.542333333333334, 5.037000000000001, 3.2506666666666666, 3.6006666666666667, 2.5900000000000003, 0.006999999999999895, 0.21300000000000002, 1.5613333333333332, 2.111666666666667, ...]
greenwich stanhope 24067 1.540169 0.905163 [0.131, 1.272, 1.776, 1.6753333333333327, 1.3593333333333337, 1.0616666666666668, 0.4973333333333335, 0.8983333333333332, 1.579333333333333, 2.796333333333333, 2.403, 1.596, 0.30799999999999983, 2.551, 2.433666666666667, 2.436, 1.4793333333333332, 0.9983333333333333, 0.7026666666666668, 0.5543333333333336, 0.9340000000000002, 2.0716666666666668, 2.3259999999999996, 2.262666666666667, 1.614, 0.8383333333333329, 1.033666666666667, 0.3340000000000001, 0.5190000000000001, 0.6160000000000001, 0.07799999999999985, 0.04800000000000004, 0.9796666666666667, 0.3923333333333332, 0.9243333333333337, 0.301333333333333, 0.07833333333333314, 0.44599999999999973, 0.16566666666666663, 0.3240000000000003, 1.0553333333333335, 1.745333333333333, 2.252333333333333, 2.981333333333333, 2.879333333333333, 2.4543333333333335, 1.6183333333333334, 1.6399999999999997, 2.2116666666666664, 2.530333333333333, 2.5700000000000003, 2.155, 1.5509999999999997, 1.5199999999999996, 1.189, 0.06699999999999928, 1.2399999999999998, 2.4606666666666666, 3.372, 4.061, 4.348333333333334, 4.505333333333333, 4.098333333333334, 4.56, 3.0966666666666667, 1.7093333333333338, 0.17433333333333367, 1.8640000000000003, 1.8049999999999997, 1.5246666666666657, 0.48566666666666647, 1.4106666666666663, 0.5083333333333331, 0.8073333333333332, 1.334, 0.44133333333333336, 0.852, 0.40733333333333344, 0.41466666666666663, 2.330333333333333, 4.1450000000000005, 5.676333333333333, 6.049666666666667, 5.577999999999999, 0.5589999999999993, 1.0030000000000001, 2.7380000000000004, 3.351, 2.309, 3.887333333333334, 4.211666666666666, 4.692333333333334, 5.078666666666667, 4.711333333333333, 4.510333333333333, 3.167666666666667, 0.07866666666666666, 0.3703333333333334, 0.5583333333333333, 2.6606666666666667, ...]
north_rustico stanhope 22994 1.872865 0.964902 [3.2779999999999996, 4.0600000000000005, 3.0125, 3.3594999999999997, 5.177999999999999, 5.859999999999999, 5.2125, 5.2595, 4.9555, 0.5805000000000002, 0.3480000000000001, 0.6470000000000001, 1.1544999999999999, 0.42549999999999977, 0.607, 1.1465, 0.9923333333333335, 1.2554999999999996, 2.831, 5.4375, 5.569500000000001, 0.42600000000000016, 0.7905000000000006, 2.645999999999999, 3.653500000000001, 2.3629999999999995, 1.8560000000000008, 1.9699999999999998, 1.1725000000000003, 1.1084999999999994, 0.3155000000000001, 1.5074999999999998, 0.47550000000000026, 0.44649999999999945, 1.4965000000000002, 1.5819999999999999, 1.9894999999999996, 0.16349999999999998, 0.488, 0.23050000000000015, 0.956, 2.5140000000000002, 4.861000000000001, 3.7095000000000002, 5.258, 7.7545, 3.6179999999999986, 0.7385000000000002, 2.0774999999999997, 0.3949999999999996, 5.0165, 4.219999999999999, 4.367999999999999, 2.4349999999999996, 0.4775000000000009, 0.07399999999999984, 0.6455000000000002, 0.06300000000000061, 1.0415, 0.6745000000000001, 1.8719999999999999, 0.9215, 1.787, 2.569, 2.8975, 2.7225000000000006, 1.4784999999999995, 0.5105, 0.25849999999999973, 0.9654999999999996, 0.9155000000000002, 0.9110000000000005, 0.39700000000000024, 0.39449999999999985, 6.627, 5.5705, 2.4085, 0.44650000000000034, 1.5984999999999996, 0.8835000000000006, 0.7670000000000003, 1.9605000000000015, 1.8565000000000005, 1.9209999999999985, 1.0989999999999993, 0.7234999999999996, 0.7985000000000007, 1.2915, 1.1304999999999996, 0.573500000000001, 0.29800000000000093, 0.22299999999999986, 0.2054999999999998, 0.3130000000000006, 0.702, 0.5025000000000004, 0.49350000000000005, 2.8790000000000013, 4.922, 2.186, ...]
stanley_bridge stanhope 24066 2.099542 0.942855 [0.7825000000000001, 0.4033333333333333, 0.20833333333333326, 0.6875, 0.7641666666666669, 0.31, 0.09166666666666656, 0.27416666666666667, 0.9775, 2.0741666666666667, 1.8708333333333333, 1.355, 0.34833333333333316, 2.1275, 1.9224999999999994, 1.0341666666666667, 0.4358333333333333, 0.3258333333333332, 0.061666666666666536, 0.385, 0.6675, 1.3608333333333331, 1.6858333333333333, 1.4425, 0.8008333333333333, 0.5425, 0.4474999999999998, 0.206666666666667, 0.8541666666666665, 0.5783333333333336, 0.047499999999999876, 0.11750000000000016, 0.5533333333333332, 0.39166666666666705, 1.8816666666666668, 1.03, 1.010833333333333, 0.6891666666666669, 0.4716666666666667, 0.8075000000000001, 1.1208333333333331, 1.3058333333333332, 2.419166666666667, 3.455833333333333, 3.476666666666666, 3.01, 2.100833333333333, 1.7983333333333333, 2.1883333333333335, 2.3058333333333327, 2.064166666666667, 1.4574999999999996, 0.8300000000000001, 0.7508333333333335, 0.41166666666666707, 0.746666666666667, 2.2258333333333336, 3.730833333333333, 4.753333333333334, 5.203333333333334, 5.3625, 5.308333333333334, 4.73, 4.906666666666666, 3.4991666666666665, 1.6374074074074074, 0.3166666666666669, 2.6056666666666666, 1.8436666666666666, 1.6303333333333332, 0.7153333333333327, 1.8246666666666664, 1.4136666666666666, 0.5606666666666669, 2.0686666666666667, 0.865, 0.18466666666666665, 0.7433333333333332, 0.028666666666666618, 1.4756666666666665, 3.2643333333333335, 4.341666666666667, 4.652666666666667, 4.5523333333333325, 0.7736666666666672, 0.6036666666666664, 1.8376666666666672, 3.6559999999999997, 2.684333333333333, 3.2223333333333333, 4.569999999999999, 5.542333333333334, 5.037000000000001, 3.2506666666666666, 3.6006666666666667, 2.5900000000000003, 0.006999999999999895, 0.21300000000000002, 1.5613333333333332, 2.111666666666667, ...]
tracadie stanhope 21940 1.822442 0.968903 [1.1550000000000011, 0.9824999999999982, 0.8283333333333331, 0.6266666666666687, 0.33333333333333215, 0.39416666666667055, 0.7650000000000006, 1.5183333333333309, 0.5924999999999976, 1.1966666666666725, 0.043333333333333, 1.0366666666666688, 1.1408333333333331, 1.1508333333333347, 1.8183333333333387, 1.5458333333333307, 0.769166666666667, 0.086666666666666, 2.557500000000001, 3.0933333333333337, 2.8599999999999994, 2.2641666666666644, 1.975833333333334, 1.7858333333333292, 1.6616666666666688, 1.5225000000000009, 1.6858333333333313, 1.803333333333331, 1.4700000000000024, 0.8125, 0.33500000000000085, 0.16999999999999815, 1.4050000000000011, 2.2225, 2.2025000000000006, 1.3141666666666687, 1.0933333333333337, 0.19583333333333286, 0.7155555555555608, 0.17499999999999716, 0.398333333333337, 0.21083333333333343, 0.8533333333333317, 1.8975000000000009, 1.3825000000000003, 2.1125000000000007, 2.233333333333338, 2.0383333333333304, 1.8616666666666646, 1.1433333333333344, 1.134999999999998, 0.8524999999999991, 0.8716666666666661, 0.9283333333333275, 0.09416666666666984, 0.6916666666666664, 2.231666666666669, 2.1633333333333304, 1.4450000000000003, 2.8283333333333367, 1.7749999999999986, 2.0249999999999986, 1.4258333333333333, 1.2699999999999996, 0.3758333333333326, 0.0591666666666697, 1.2349999999999994, 2.7025000000000006, 3.9775000000000027, 4.57833333333333, 4.006666666666664, 3.7441666666666684, 2.7775, 2.576666666666661, 2.0799999999999983, 0.6533333333333324, 0.4833333333333343, 0.03999999999999915, 0.1700000000000017, 1.5308333333333373, 2.6466666666666683, 2.288333333333334, 1.7424999999999997, 1.1666666666666679, 2.575833333333332, 2.3833333333333364, 1.8758333333333361, 0.9899999999999984, 0.036666666666668846, 1.1208333333333336, 3.6866666666666674, 3.418333333333333, 3.5199999999999996, 2.973333333333329, 1.4858333333333285, 1.4608333333333299, 1.1708333333333272, 1.047500000000003, 1.4683333333333373, 1.6616666666666653, ...]
Station Recommendations:
station reference_station recommendation risk_probability ci_lower ci_upper risk_band evidence assumptions limitations
cavendish stanhope remove 0.121140 0.012887 0.288408 low benchmark correlation=0.967; uncertainty=low (0.01-0.29); pca=[PC1=0.414, PC2=-0.168]; cluster=position 3/6 Distributional uncertainty is estimated with scipy.stats.gaussian_kde over observation-derived station-reference divergence samples; when benchmark fixtures omit raw observations, a documented synthetic distribution is generated from mean difference, correlation, and overlap support. Sample support is adequate for a coarse uncertainty bound.
greenwich stanhope defer 0.028022 0.000000 0.060345 low benchmark correlation=0.905; uncertainty=low (0.00-0.06); pca=[PC1=0.396, PC2=0.901]; cluster=position 6/6 Distributional uncertainty is estimated with scipy.stats.gaussian_kde over observation-derived station-reference divergence samples; when benchmark fixtures omit raw observations, a documented synthetic distribution is generated from mean difference, correlation, and overlap support. Sample support is adequate for a coarse uncertainty bound.
north_rustico stanhope remove 0.110791 0.013123 0.265487 low benchmark correlation=0.965; uncertainty=low (0.01-0.27); pca=[PC1=0.413, PC2=-0.171]; cluster=position 4/6 Distributional uncertainty is estimated with scipy.stats.gaussian_kde over observation-derived station-reference divergence samples; when benchmark fixtures omit raw observations, a documented synthetic distribution is generated from mean difference, correlation, and overlap support. Sample support is adequate for a coarse uncertainty bound.
stanley_bridge stanhope defer 0.063608 0.010394 0.144060 low benchmark correlation=0.943; uncertainty=low (0.01-0.14); pca=[PC1=0.406, PC2=-0.323]; cluster=position 5/6 Distributional uncertainty is estimated with scipy.stats.gaussian_kde over observation-derived station-reference divergence samples; when benchmark fixtures omit raw observations, a documented synthetic distribution is generated from mean difference, correlation, and overlap support. Sample support is adequate for a coarse uncertainty bound.
tracadie stanhope remove 0.123004 0.014229 0.279774 low benchmark correlation=0.969; uncertainty=low (0.01-0.28); pca=[PC1=0.413, PC2=-0.154]; cluster=position 2/6 Distributional uncertainty is estimated with scipy.stats.gaussian_kde over observation-derived station-reference divergence samples; when benchmark fixtures omit raw observations, a documented synthetic distribution is generated from mean difference, correlation, and overlap support. Sample support is adequate for a coarse uncertainty bound.
8. Uncertainty Quantification¶
Risk probabilities and confidence intervals for station removal.
Uncertainty Quantification (KDE-based risk):
station reference_station risk_probability ci_lower ci_upper risk_band assumptions limitations
cavendish stanhope 0.121140 0.012887 0.288408 low Distributional uncertainty is estimated with scipy.stats.gaussian_kde over observation-derived station-reference divergence samples; when benchmark fixtures omit raw observations, a documented synthetic distribution is generated from mean difference, correlation, and overlap support. Sample support is adequate for a coarse uncertainty bound.
greenwich stanhope 0.028022 0.000000 0.060345 low Distributional uncertainty is estimated with scipy.stats.gaussian_kde over observation-derived station-reference divergence samples; when benchmark fixtures omit raw observations, a documented synthetic distribution is generated from mean difference, correlation, and overlap support. Sample support is adequate for a coarse uncertainty bound.
north_rustico stanhope 0.110791 0.013123 0.265487 low Distributional uncertainty is estimated with scipy.stats.gaussian_kde over observation-derived station-reference divergence samples; when benchmark fixtures omit raw observations, a documented synthetic distribution is generated from mean difference, correlation, and overlap support. Sample support is adequate for a coarse uncertainty bound.
stanley_bridge stanhope 0.063608 0.010394 0.144060 low Distributional uncertainty is estimated with scipy.stats.gaussian_kde over observation-derived station-reference divergence samples; when benchmark fixtures omit raw observations, a documented synthetic distribution is generated from mean difference, correlation, and overlap support. Sample support is adequate for a coarse uncertainty bound.
tracadie stanhope 0.123004 0.014229 0.279774 low Distributional uncertainty is estimated with scipy.stats.gaussian_kde over observation-derived station-reference divergence samples; when benchmark fixtures omit raw observations, a documented synthetic distribution is generated from mean difference, correlation, and overlap support. Sample support is adequate for a coarse uncertainty bound.
Saved: notebooks/figures/uncertainty_risk.png
Conclusion¶
TBD — fill in after review.