Reliable evaluation

Subject-Independent EEG Classification

This study tests whether an EEG classifier can recognise rest and imagined movements in participants it has never seen during training. We evaluated a four-stream temporal CNN on 16,192 trials from 103 PhysioNet participants, keeping participants separate across training, validation and testing.

SubmittedResearch intern · model development and evaluation

Code ↗

The problem

How well does an EEG classifier generalize to participants it has never seen?

The four classes are rest, left-fist imagery, right-fist imagery and both-feet imagery. The classification cohort comprises 103 PhysioNet participants and 16,192 trials.

My role & contributions

I developed the four-stream CNN and implemented nested, participant-disjoint evaluation and reproducibility materials.

Supervision: Dr. Bikash Kumar Pradhan

Method & system

Four parallel temporal CNN streams process 64-channel EEG using kernel widths of 7, 9, 11 and 13 samples. Stream features are pooled and concatenated for four-class classification. The kernel widths represent temporal scales, not specific brain-frequency bands. — Four-stream architecture · October 24 manuscript, Figure 3. Temporal features are pooled, combined and passed to the classifier.

Participant-disjoint evaluation

Outer participant splitKeep outer-test participants separate from outer-training participants
Inner validationSplit inner-validation participants from the training pool for checkpoint selection
Model selectionTrain the four-stream Multi-Scale Temporal CNN and select checkpoints using inner validation
Final outer evaluationEvaluate on untouched outer-test participants; repeat across ten test folds

Participant-disjoint evaluation: each outer-test fold contains people who are separate from its corresponding training and inner-validation pools.

Filter 64-channel EEG at 4–40 Hz and normalise each channel within each epoch.
Process the signals through four parallel temporal convolution streams with kernel widths of 7, 9, 11 and 13 samples. These widths specify temporal scales, not individual brain-frequency bands.
Combine the four streams’ features for four-class classification.
Evaluate using nested, subject-disjoint 10-fold cross-validation. Select checkpoints using separate inner-validation participants, never outer-test participants.

PythonPyTorchPhysioNetSubject-disjoint validation

Evaluation & results

The reference model has 83.08 million parameters. The results below use seed 42 and report mean ± standard deviation across ten outer folds. Checkpoint selection uses separate inner-validation participants; outer-test participants contribute only to final evaluation.

83.08-million-parameter reference model · seed 42 · mean ± standard deviation across ten outer folds
Measure	Result	Context
Accuracy	70.23 ± 1.87%	Mean ± standard deviation across ten outer folds.
Balanced accuracy	58.38 ± 2.70%	Mean ± standard deviation across ten outer folds.
Macro-F1	59.70 ± 2.54%	Mean ± standard deviation across ten outer folds.

Pooled out-of-fold macro one-versus-rest ROC–AUC: 0.839. This pooled ranking measure is reported separately from the fold-mean metrics above.

Across three random seeds, mean accuracy was 69.83 ± 0.36%, with variability measured across the three seed-specific fold means.

Aggregate outer-test confusion matrix for the seed-42, 83.08-million-parameter reference model. Rows are actual classes; columns are predictions. Class codes: 0 rest, 1 left-fist imagery, 2 right-fist imagery, 3 both-feet imagery. Correct predictions by row: 7,968 rest (86.0%), 1,218 left fist (52.3%), 1,141 right fist (49.9%) and 1,047 both feet (45.4%). — Aggregate outer-test confusion matrix · Figure 5. Cells show trial counts and row-normalised percentages across ten folds. Codes: 0 rest; 1 left fist; 2 right fist; 3 both feet.

Model-capacity trade-off: accuracy, balanced accuracy and macro-F1 across variants from about 0.55 to 83.08 million parameters, with computational cost and batch-one GPU latency in a second panel. The 19.94-million-parameter variant achieved 70.23% mean accuracy and 59.68% macro-F1 using 76% fewer parameters than the reference. — Model-capacity trade-off · supplementary Figure S2. The parameter axis is logarithmic. The computational panel reports Conv/Linear GFLOPs and batch-one latency from the original NVIDIA Tesla T4 runs.

Interpretation

Class imbalance and variation between participants affect how overall accuracy should be interpreted; balanced accuracy and macro-F1 provide complementary class-sensitive summaries. A separate neurophysiological band-power analysis included 85 participants after artefact rejection. It provides physiological context, rather than an explanation of the CNN’s decisions, and is distinct from the 103-person classification cohort.

Future work

Extend evaluation across participants and class distributions, alongside online BCI experiments and clinical validation.

← Back to research Discuss this work ↗