The Music Library includes materials on all aspects of music, but is primarily directed toward support for the curriculum offered by the Music Department. The library contains more than 15,000 sound recordings, 17,000 music scores, 12,000 books about music, and 2,000 bound periodicals. It provides playback facilities for the collection's CDs, LPs, cassettes, videos, CD-ROMs, and has three public Internet workstations and two photocopy machines.

Books and scores are shelved in open stacks in the Harold Newman Reading Room on the lower level. Scores are classed by the Columbia-Vassar adaptation of the G.S. Dickinson classification for music. Bound periodicals are shelved alphabetically by title. The Music Library is open to anyone with a validated ID from any of the colleges of CUNY. Books may be returned to any CUNY library; scores must be returned directly to the Music Library. Sound recordings circulate (2 weeks) only to City College Music Department faculty for classroom preparation.

The Music Library, created in 1948, was installed in its fifth campus home in 1993 when it moved to a newly-renovated bi-level space under the apse of the Great Hall in Shepard Hall. The Music Department is located nearby in the same building. Most of the furniture and shelving were designed by the William A. Hall Partnership, architects for the project. A mural by William Clarke Rice, featuring the college's first building on Twenty-third Street, is mounted over the entrance.