Hypothesis: This proposed harmonic consonance-dissonance music perception task is expected to reveal a monotonic relationship between harmonic consonance and pleasantness ratings by NH listeners. Additionally, CI recipients will be able to distinguish between the most consonant and most dissonant music samples, although with less contrast between each condition than the NH cohort and with lower overall ratings of sound quality. Finally, we hypothesize that listeners with extensive music training would show more pronounced differences in pleasantness ratings across the four tiers of consonance to dissonance. 

Background: Harmonic consonance and dissonance play crucial roles in how music is perceived, affecting its perceived quality and pleasantness. However, tools to evaluate these musical aspects, especially for CI users, are scarce, leading to significant knowledge gaps. This study aimed to refine previous methods by emphasizing the variability and typically lower scores among CI users, aligning these findings more closely with their reported experiences and existing literature.

Methods: A total of 34 participants (21 NH and 13 CI) completed the 30-minute music task, which involved listening to music samples with various levels of harmonic consonance-dissonance and then rating the samples on a 5-point “pleasantness” scale. Participants also provided details about their musical training and listening habits. 

Results: Data analysis revealed significant differences in how NH and CI users perceive consonance and dissonance, particularly when dissonance is pronounced enough to be discernible over the spectral resolution limitations of CIs. A correlation between formal musical training and better performance on the task was observed in both NH and CI groups, suggesting a heightened sensitivity to harmonic dissonance among trained listeners.

Conclusions: The findings challenge prior assumptions about the limited music perception capabilities of CI users and propose the potential for a shorter, refined version of this test for clinical use or further research. This task could aid in optimizing CI configurations for enhanced music enjoyment.

Subjects

A group of 21 NH listeners, including 13 women and 8 men, with an average age of 33.5 ± 14.0 (mean ± one standard deviation, range: 24 to 71 years) participated in the study (Table 1). Prior to participation, NH subjects completed a hearing screening which consisted of responding to pure-tone stimuli presented at 25 dB HL at octave intervals from 250 – 8000 Hz in each ear. A separate group of 13 CI recipients, including 6 women and 7 men, were aged 57.4 ± 10.1 years (range: 42 to 77 years) and were previously implanted with MED-EL devices. Twelve were unilaterally implanted and one was bilaterally implanted, yielding 14 CI ears tested. The CI cohort had a distribution of electrode array insertion depths ranging from 450 to 673 degrees, with an average of 564 ± 59 deg. Ten ears had received a Flex28 (28 mm) array and four ears a FlexSoft (31.5 mm) array.

All participants completed a music history questionnaire outlining the duration, setting, and age of starting music training, as well as specific instruments played. Participants were also asked how frequently they currently listen to music during an average week and what musical genre(s) they prefer. 

Among the NH cohort, 15 out of 21 subjects reported formal music training; those who received music training began at age 8.3 years (± 2.0 years) and lasted 8.2 years (± 3.2 years). The primary musical instrument studied by each subject with musical training was as follows: piano (7), guitar (3), violin (2), flute (2), and trombone (1). Five subjects reported currently practicing a musical instrument. None reported being self-taught musicians. Within the NH cohort, all but one subject reported listening to music in their day-to-day life, and there was a wide variability in the number of hours per week spent listening to music (18.5 ± 20.2 hours; range: 0 to 72 hours/week). When subjects were asked to name their favorite genre of music, there was a roughly even mix of respondents choosing Hip-Hop, Pop/Rock, Soul/R&B, and Jazz. 

In the CI cohort, 9 of the 13 subjects reported formal music training; those with music training began at age 9.4 years on average (± 3.2 years) and lasted 11.3 years (± 13.2 years). One subject without formal training did report being a self-taught musician, with 10 years of practice. The primary instruments included: voice (3), piano (2), violin (2), clarinet (1), guitar (1), and bass fiddle (1). Four of the ten are still practicing an instrument currently. All but two of the CI recipients reported listening to music on a weekly basis, for an average of 14.6 ± 11.3 hours per week. The CI group’s genre preferences were divided between predominantly Folk, Pop/Rock, and Classical. 

The Institutional Review Board at the University of California, San Francisco (UCSF) approved this study and informed consent was obtained from all participants. Subjects were recruited from flyers on UCSF campus, from a database of potential research volunteers maintained by our lab, and by word of mouth.

Musical Stimuli

This music sound quality task evaluates two aspects of music perception: 1) overall pleasantness of synthetic piano music samples and 2) the change in a listener’s rating of pleasantness across different levels, or tiers, of harmonic consonance. The task was described at length previously in Caldwell et al. (2016), which consisted of 3 levels, termed Tier A (most consonant), Tier B, and Tier C (most dissonant). For this study, we expanded on the Caldwell et al. study to include a fourth level, Tier D, which was more dissonant than the previous Tier C. 

For this test, we utilized 12 distinct melodies (2 for practice and 10 for the main test) and paired each melody with 4 possible versions of accompanying chord structure, resulting in 48 music samples in all. With few exceptions, the notes of the melodies were in the treble clef and ranged from F4 - F5 (349 - 698 Hz) and the chord accompaniments were in the bass clef and ranged from A2 - E4 (110 - 330 Hz), with harmonics present through around 8000 Hz.  

It may be helpful for the reader to review the composition of a single melody with the four versions of chord accompaniments in different formats; therefore, these melodies are presented in musical score format (Figure 1), spectrogram format (Figure 2), as well as in an audio file that is available online (Supplementary Digital Content 1). For each melody, the accompanying chord structures employ varying levels of harmonic dissonance, as follows: 

Tier A: Consonant major triad with no dissonant notes

Tier B: One dissonant note (flattened minor 7th replacing the consonant 5th in the chord)

Tier C: Two dissonant notes (flattened minor 6th and 7th replacing the consonant 3rd and 5th)

Tier D: Two dissonant notes (flattened minor 2nd and major 2nd replacing the consonant 3rd and 5th)

Tiers A, B, and C are identical to that which was used in previously published work (Caldwell et al., 2016). Tier D was composed based on previous studies of varying levels of dissonance (Blood et al., 1999), and the degree of pleasantness would then be confirmed through our NH listener group.

Each music sample was 8 bars in length and approximately 14.8 seconds in duration, and were presented at the same 130 beats per minute tempo. The intensity of the musical samples were normalized to reduce perceived variance in volume across the sample. Musical stimuli were generated by GarageBand (Apple Inc., Cupertino, CA, USA), utilizing a synthetic piano.

Data Collection 

The present task was administered using an online survey tool (Qualtrics LLC, Seattle, WA, USA). At the beginning of the task, listeners were given a practice music clip and were asked to adjust the volume of their computer output until they achieved a comfortable listening level. For each trial, both practice and main test, a listener was presented with a music clip and asked, “Ignoring all other aspects of the music, how PLEASANT was the sound?” The listener responded by selecting one of the following five options: not at all pleasant, slightly pleasant, moderately pleasant, very pleasant, and extremely pleasant. Users completed a practice section consisting of 8 music samples. This subset represented 2 distinct melodies, with all 4 versions of each melody, so that users could hear the full range of consonance-dissonance prior to beginning the test. The remaining 40 music samples (10 melodies, 4 versions of each) were used for the final assessment. At the end of the task, the respondents were offered a text box for providing comments on their experience. 

The NH cohort listened in the binaural condition with supra-aural headphones. The CI cohort streamed the audio from a single test laptop via a MED-EL AudioLink to their CI sound processor. All scene classifiers, noise reduction algorithms, and microphone directionality features were disabled. CI volume was set to 90-100% and sensitivity fixed at 75%. If the subject used a hearing device on the contralateral ear then it was removed for testing. If there was residual acoustic hearing in the contralateral ear, then a foam earplug was inserted.

For other investigators who may use this test metric for their own purposes, the authors recommend taking the frequency range of any hearing devices (e.g., hearing aids, CIs, etc.) into consideration. The fundamental frequencies of the notes in melodies designed for this assessment range from 110 to 700 Hz, while the harmonic content generated by the synthetic piano music samples tapered off around 8000 Hz. These frequencies are all within the standard frequency allocation table of MED-EL CIs (70-8500 Hz), and are mostly within the frequency ranges of Cochlear Americas (188-7938 Hz) and Advanced Bionics (250-10,000 Hz) CI devices.

Data and Statistical Analyses 

To assess the impact of demographic variables and experiential factors on performance in the consonance-dissonance music task, we examined the mean Tier A (most consonant) differences among the specified groups. Additionally, we analyzed the difference in sound quality ratings from Tier A through Tier D (inclusive of all tiers), referred to here as the ‘slope’, which serves as a proxy for sensitivity to harmonic dissonance, albeit limited by the participants’ overall assessment of pleasantness of musical consonance and dissonance. These analyses incorporated variables such as age, musical training, and listening habits.

We then analyzed the differences in performance on the consonance-dissonance music task across groups and conditions of interest. For input to the statistical software (JASP, Version 0.18.3; Amsterdam, Netherlands) and in all following graphs, the five potential responses from listeners were assigned the following numerical values: 0 (not at all pleasant), 25 (slightly pleasant), 50 (moderately pleasant), 75 (very pleasant), and 100 (extremely pleasant). To create linear regressions from responses across consonance-dissonance tiers, the four tiers were assigned values as follows: 100 (Tier A), 66.7 (Tier B), 33.3 (Tier C), and 0 (Tier D). 

A mixed analysis of variance (ANOVA) was conducted on pleasantness ratings of music clips to determine whether the within-subject factor of consonance-dissonance tier (Tiers: A, B, C, D) affected perception. One between-subject factor (Cohort: NH, CI), was included in the model. Mauchly’s test indicated that the assumption of sphericity was violated (χ2(5) = 37.1, p < 0.001) and a Greenhouse-Geisser correction was applied to the degrees of freedom (ε = 0.560). Tukey’s test for Honestly Significance Difference (HSD) explored all pairwise comparisons, with adjustments made to the p-value for a family of 28. 

All demographic and CI factors were inspected for normal distribution with the Shapiro-Wilk test, and the group means tested for differences with either a student’s t-test (if parametric) or Mann-Whitney U test (if non-parametric). Correlations of these factors with test outcomes were explored with either Pearson’s (for parametric) or Spearman’s (for non-parametric) tests.