In Day 18, we introduced permutation testing to avoid standard Gaussian assumptions. However, running point-by-point tests across 64 channels and 500 time points yields 32,000 statistical tests! Controlling for Family-Wise Error Rate (FWER) using simple Bonferroni or FDR corrections is far too conservative because neighboring EEG channels and adjacent time samples are inherently correlated.
Cluster-Based Permutation Tests (Maris & Oostenveld, 2007) solve this by taking advantage of electrophysiological continuity: true biological effects do not appear at isolated, solitary pixels — they form spatially and temporally connected clusters.
The Step-by-Step Cluster Pipeline
Based on Benedikt Ehinger's Methodology
Extract Activation Differences Across Subjects
For each subject, calculate the trial-averaged response difference between conditions (e.g., Condition A − Condition B) across all sensors and time points.
Calculate Pointwise Test Statistics ($t$-values)
Compute an independent or dependent samples $t$-statistic at every single data point $(channel, time)$. The $t$-value measures the condition difference scaled by subject-to-subject variability.
Apply Cluster-Forming Threshold & Group Adjacencies
Threshold the $t$-map using an initial uncorrected cutoff (e.g., $p_{\text{threshold}} < 0.05$ or $|t| > 2.0$). Group contiguous suprathreshold points into candidate clusters using spatial neighbor graphs (adjacency matrices) and temporal continuity.
Calculate Cluster Mass ($\Sigma t$)
For each candidate cluster, sum all the $t$-values within the cluster: $\text{Cluster Mass} = \sum t_i$. A large cluster mass indicates a strong, spatially/temporally extended effect.
Build Null Distribution via Permutations & Compare
Randomly swap condition labels for subjects 1,000+ times. On each iteration, re-run steps 2–4 and record the maximum cluster mass. Compare your observed cluster masses against this permutation null distribution to assign non-parametric $p$-values.
Benedikt Ehinger's Core Takeaway: The cluster-based permutation test does not evaluate whether an isolated time point is significant. Instead, it tests whether the total mass of a connected spatio-temporal cluster exceeds what could occur by chance under label shuffling.
MNE-Python Implementation
Spatio-Temporal Clustering
In MNE-Python, mne.channels.find_ch_adjacency computes the spatial sensor graph, and spatio_temporal_cluster_1samp_test performs the clustering and permutation loop across channels and time points.
import numpy as np import mne from mne.stats import spatio_temporal_cluster_1samp_test # Data shape: (n_subjects, n_times, n_channels) X = condition_A_epochs.get_data() - condition_B_epochs.get_data() X = np.transpose(X, (0, 2, 1)) # Reshape to (n_subjects, n_times, n_channels) # 1. Compute spatial channel adjacency graph adjacency, _ = mne.channels.find_ch_adjacency(condition_A_epochs.info, ch_type='eeg') # 2. Set cluster-forming threshold (uncorrected p < 0.05 equivalent t-value) p_threshold = 0.05 t_threshold = scipy.stats.t.ppf(1 - p_threshold / 2, df=n_subjects - 1) # 3. Run Spatio-Temporal Cluster Permutation Test t_obs, clusters, cluster_p_values, H0 = spatio_temporal_cluster_1samp_test( X, threshold=t_threshold, adjacency=adjacency, n_permutations=1000, tail=0, n_jobs=-1 ) # 4. Find significant clusters good_clusters = np.where(cluster_p_values < 0.05)[0] print(f"Found {len(good_clusters)} significant cluster(s)!")
Critical Interpretation & FAQs
Avoid Common Misconceptions
Does a cluster p-value tell you exact spatial/temporal boundaries?
No! As emphasized in FieldTrip and Ehinger's guides, the $p$-value applies to the cluster as a whole entity, not to individual sensors or millisecond latency bins inside the cluster. You can state that "there is a significant difference between conditions in this spatial-temporal region," but you cannot claim that exact millisecond $t=182 \text{ ms}$ or channel $\text{Cz}$ individually has $p < 0.05$.
Why is the Max-Statistic essential?
By extracting only the maximum cluster mass across the entire scalp and time window in each permutation iteration, the test controls the Family-Wise Error Rate (FWER) at the specified alpha level (e.g. $\alpha = 0.05$), ensuring robust protection against false positives.