Since the formation of the Islamic society by Prophet Muhammad, mosque libraries fulfilled several important roles that deserve scholarly attention. Therefore, this article elicits the significance of these various roles within Muslim societies through Islamic history. Until present day, almost all mosque libraries provide a variety of collections, concerning both religious and non-religious issues, such as social science and education. In line with these roles, this paper presents different examples of mosque libraries that, through history, contributed significantly to the development of Islamic civilization, like the library of the al-Azhar mosque in Egypt, the Grand Mosque Library in Cordova, etc. In addition, several aspects determining the (in)significance of mosque libraries through history, such as library management and factors contributing to the stagnation of mosque libraries, are investigated.