Fig 1. Overview of Linearized Encoding Analysis (LEA)
Methodological issues
Reverse double-dipping
This article (Kim, 2026) elucidates a methodological pitfall of cross-validation for evaluating predictive models applied to naturalistic neuroimaging data—namely, ‘reverse double-dipping’ (RDD). In a broader context, this problem is also known as ‘leakage in training examples’, which is difficult to detect in practice. RDD can occur when predictive modeling is applied to data from a conventional neuroscientific design, characterized by a limited set of stimuli repeated across trials and/or participants. It results in spurious predictive performances due to overfitting to repeated signals, even in the presence of independent noise. Through comprehensive simulations and real-world examples following theoretical formulation, the article underscores how such information leakage can occur and how severely it could compromise the results and conclusions when it is combined with widely spread informal reverse inference. The article concludes with practical recommendations for researchers to avoid RDD in their experiment design and analysis.
Fig 2. Reserve double-dipping: data dips you, twice!
Time series prediction
Fig 3. Spurious correlation between smooth time series
Resources
Kim, 2024-09-07, Linearized Encoding Modeling: a Predictive Analysis Methodology for Music Perception, Korean Society for Music Perception and Cognition (KSMPC) Summer School 24, Session 3 lecture. [slides][code][repo]
This article elucidates a methodological pitfall of cross-validation for evaluating predictive models applied to naturalistic neuroimaging data—namely, “stimulus-driven leakage.” While this problem is well known as “leakage in training examples” in machine learning, it may be difficult to detect in practice due to conventions in neuroscience. Stimulus-driven leakage can occur when predictive modelling is applied to data from a conventional neuroscientific design, characterised by a limited set of stimuli repeated across trials and/or participants. It results in spurious predictive performance due to overfitting to repeated signals, even in the presence of independent noise. Through comprehensive simulations and real-world examples, following a theoretical formulation, the article underscores how such data leakage can occur and how severely it can compromise results and conclusions when combined with widespread informal reverse inference. The article concludes with practical recommendations for researchers to avoid stimulus-driven leakage in their experimental design and analysis.
@article{kim2026sdl,author={Kim, Seung-Goo},title={Stimulus-Driven Leakage in Naturalistic Neuroimaging},journal={Imaging Neuroscience},year={2026},month=nov,doi={10.1162/IMAG.a.1463},url={https://doi.org/10.1162/IMAG.a.1463},}
Speech comprehension entails the neural mapping of the acoustic speech signal onto learned linguistic units. This acousto-linguistic transformation is bi-directional, whereby higher-level linguistic processes (e.g. semantics) modulate the acoustic analysis of individual linguistic units. Here, we investigated the cortical topography and linguistic modulation of the most fundamental linguistic unit, the phoneme. We presented natural speech and "phoneme quilts" (pseudo-randomly shuffled phonemes) in either a familiar (English) or unfamiliar (Korean) language to native English speakers while recording functional magnetic resonance imaging. This allowed us to dissociate the contribution of acoustic vs. linguistic processes toward phoneme analysis. We show that (i) the acoustic analysis of phonemes is modulated by linguistic analysis and (ii) that for this modulation, both of acoustic and phonetic information need to be incorporated. These results suggest that the linguistic modulation of cortical sensitivity to phoneme classes minimizes prediction error during natural speech perception, thereby aiding speech comprehension in challenging listening situations.
@article{kim2024cc,volume={34},number={4},pages={bhae155},journal={Cerebral Cortex},author={Kim, Seung-Goo and Martino, Federico De and Overath, Tobias},date-modified={2024-04-25 14:43:05 +0200},doi={10.1093/cercor/bhae155},title={Linguistic modulation of the neural encoding of phonemes},year={2024},}
Proceedings – The Joint Conference of the 17th International Conference on Music Perception and Cognition (ICMPC) and the 7th Conference of the Asia-Pacific Society for the Cognitive Sciences of Music (APSCOM), Nov 2023
@article{kim2023icmpc,author={Kim, Seung-Goo and Overath, Tobias and Sammler, Daniela},journal={Proceedings -- The Joint Conference of the 17th International Conference on Music Perception and Cognition (ICMPC) and the 7th Conference of the Asia-Pacific Society for the Cognitive Sciences of Music (APSCOM)},date={2023-08-01},date-modified={2024-04-25 18:48:05 +0200},organization={International Conference on Music Perception and Cognition (ICMPC)},title={Emotion-relevant Representations of Music Extracted by Convolutional Neural Networks Are Encoded in Medial Prefrontal Cortex},year={2023},}
This article discusses recent developments and advances in the neuroscience of music to understand the nature of musical emotion. In particular, it highlights how system identification techniques and computational models of music have advanced our understanding of how the human brain processes the textures and structures of music and how the processed information evokes emotions. Musical models relate physical properties of stimuli to internal representations called features, and predictive models relate features to neural or behavioral responses and test their predictions against independent unseen data. The new frameworks do not require orthogonalized stimuli in controlled experiments to establish reproducible knowledge, which has opened up a new wave of naturalistic neuroscience. The current review focuses on how this trend has transformed the domain of the neuroscience of music.
@article{kim2022fn,pages={928841},author={Kim, Seung-Goo},doi={10.3389/fnins.2022.928841},issn={1662-453X},journal={Frontiers in Neuroscience},title={On the encoding of natural music in computational models and human brains},volume={16},year={2022},bdsk-url-1={https://www.frontiersin.org/journals/neuroscience/articles/10.3389/fnins.2022.928841},bdsk-url-2={https://doi.org/10.3389/fnins.2022.928841}}
Even without formal training, humans experience a wide range of emotions in response to changes in musical features, such as tonality and rhythm, during music listening. While many studies have investigated how isolated elements of tonal and rhythmic properties are processed in the human brain, it remains unclear whether these findings with such controlled stimuli are generalizable to complex stimuli in the real world. In the current study, we present an analytical framework of a linearized encoding analysis based on a set of music information retrieval features to investigate the rapid cortical encoding of tonal and rhythmic hierarchies in natural music. We applied this framework to a public domain EEG dataset (OpenMIIR) to deconvolve overlapping EEG responses to various musical features in continuous music. In particular, the proposed framework investigated the EEG encoding of the following features: tonal stability, key clarity, beat, and meter. This analysis revealed a differential spatiotemporal neural encoding of beat and meter, but not of tonal stability and key clarity. The results demonstrate that this framework can uncover associations of ongoing brain activity with relevant musical features, which could be further extended to other relevant measures such as time-resolved emotional responses in future studies.