The pioneer work on the irrelevant speech effect (ISE) can be traced back at Colle and Welsh’s report in 1976, and since then many behavioral and neuropsychological experiments have been carried out on this topic. In previous studies, the ISE refers to, compared to a silent-control condition, the disruption of serial recall due to the presentation of auditory distractors. To date, there are many models to explain ISE which have made great progress, such as phonological store hypothesis, changing state hypothesis, feature model and so on. However, most of the work were based on the working memory model, and none of them could satisfactorily explain the discrepancies among different models. And there are ongoing debates on how attention plays a role in the ISE. In the current study, we shifted the observed window to the earliest stage of visual perception (visual awareness or consciousness) and aimed to find out if the irrelevant speech would influence visual information processing earlier than in the memory stage. Three experiments were conducted in this study. All the participants were required to detect the target stimulus which was presented near subjective threshold under three different acoustic conditions. In Experiment 1, a 3 (the type of the irrelevant speech: monosyllabic word vs. pure tone vs. silence) × 3 (the type of picture: gray solid circle vs. cartoon face vs. cartoon clock) mix design was implemented. In Experiment 2, a 3 (the type of the irrelevant speech: monosyllabic word vs. pure tone vs. silence) × 2 (the type of picture: real face vs. real house) within-subject design was implemented. In Experiment 3, in order to examine the neural correlates of the irrelevant speech effect that was found in the previous two experiments, a single factor with 3 levels of the irrelevant sound background (monosyllabic word vs. pure tone vs. silence) were designed and the corresponding event-related potentials (VAN) were recorded. All the behavioral results were analyzed according to the Signal Detection Theory. The results in Experiments 1 and 2 revealed that the irrelevant speech did impair participants’ behavioral performances in the visual awareness task. The ERP results in Experiments 3 were in line with those of Experiments 1 and 2 in that the visual awareness negativity (VAN) around 200ms disappeared under the condition of irrelevant speech, whereas not under the condition of pure tone or silence. In summary, it is concluded that the irrelevant speech damages visual awareness, and this damage was not on the content of visual awareness, nor on the concurrent attentional mechanism.