The Caldwell-Lake George Library collection dates back to the 1880s when it was bequeathed after the death of local resident DeWitt Clinton Hay. The library's building was constructed in 1906, and the original librarian, Mary Hubbell, served the community until 1958. There have only been five librarians at Caldwell over the course of its storied history.

Among Hay’s collection were several old tomes, some of which now reside in the library’s archive. Many of them are still in circulation and sit on the library’s shelves. 

The library today is a member of the Southern Adirondack Library System and provides many services for the local community and visitors of Lake George to enjoy. There is a small parking lot out back, free Wi-Fi and computer use, constantly changing new and popular hot ticket books to browse from, a full calendar of events, and the library staff is very knowledgeable about the local history and happenings of Lake George.