Understanding insincere language, for example sarcasm and teasing, is a fundamental element of communication and crucial for maintaining social relationships. This can be a challenging task for cochlear implant users (CIs) who receive degraded suprasegmental information important for perceiving a speaker’s attitude. We measured perception of speaker sincerity (literal positive, literal negative, sarcasm, and teasing) in 16 adults with CIs using an established video inventory. Participants were presented with audio-only and audio-visual social interactions between two people with and without supporting verbal context. They were instructed to describe the content of the conversation and answer whether the speakers meant what they said. Results showed that subjects could not always identify speaker sincerity, even when the content of the conversation was perfectly understood. This deficit was greater for perceiving insincere relative to sincere utterances. Performance improved when additional visual cues or verbal context cues were provided. Subjects who were better at perceiving the content of the interactions in the audio-only condition benefited more from having additional visual cues for judging the speaker’s sincerity, suggesting that the two modalities compete for cognitive recourses. Percentage scores for understanding the content also did not correlate with that for extracting speaker sincerity, suggesting that what was said vs. how it was said were perceived using unrelated segmental versus suprasegmental cues. Our results further showed that subjects who had access to lower-order resolved harmonic information provided by a hearing aids in the contralateral ear identified speaker sincerity better than those who used implants alone. The results suggest that measuring speech recognition alone in CI users do not fully describe the outcome and our findings stress the importance of measuring social communication functions in people with CIs.