This study aims to illustrate a psychometric and interpretive procedure for detecting gender-biased items in a standardized English listening comprehension test for secondary school students. The target test, which has been administered nationwide twice a year through Education Broadcast System since 1991, is composed of 20 multiple-choice items designed to assess students'' ability to understand monologic and dialogic spoken discourses about a range of familiar topics. Data for analysis were collected from 875 first-year students in a middle school (450 male, 425 female). In order to avoid theory-laden and method-contingent results, two classical (M-H and logistic regression) and two IRT-based methods (Rasch and IRTLRDIF) were employed in parallel. The final phase of analysis identified two uniform and two non-uniform items that functioned unequally by gender. The follow-up content analysis revealed that conversation topics were the most likely cause of the bias. Some implications for item development and fairness in the assessment of foreign language listening ability and directions of future research are discussed on the basis of the findings.