As I was going through Appendix N over the weekend, some of the lacunae present in Gygax's list of fantasy (and science fiction) works are almost as interesting as the things he does actually include.
The author Gary just plain missed was Clark Ashton Smith, the one member of the trio of great early American fantasists to survive the '30s (the others being H.P. Lovecraft and Robert E. Howard). CAS's stories are probably those of most benefit to those stuck in the rut of "vanilla" fantasy, as they are dense with both flavor and a semi-Theosophical bent that will mash things up straightaway. His stories are available online at Eldritch Dark. For those wanting print editions, the University of Nebraska Press's Bison Frontiers of Imagination series reprinted the old Arkham House collections Out of Space and Time and Lost Worlds which contain the meat of this work.
A major pioneer of heroic fantasy was left out. E.R. Eddison's The Worm Ouroboros (full text at Wikisource) had been reprinted by Ballantine a few years before their famous Adult Fantasy line set the standard for fantasy literature. Eddison's work was missed when first written but quickly gained popularity after The Lord of the Rings and was quite relevant in the period when AD&D was published.
Gygax included several women in his list but missed Ursula K. LeGuin, whose work is consciously different from much of contemporary fantasy. Her Earthsea cycle (A Wizard of Earthsea, The Tombs of Atuan, The Farthest Shore, Tehanu) is exemplary, particularly because Ged often has to use the kind of lateral thinking that best marks old-school RPG play. It's also a good source for dragons (in A Wizard of Earthsea) and literary tombs similar to D&D dungeons (in The Tombs of Atuan). And if you like island adventure it's a great source of inspiration on that front.
The appendix is also focused, to a fault, on swords & sorcery and fantasy literature. Except for Three Hearts and Three Lions it never touches on chivalric romances, for all of D&D's medieval pretensions. There is no Arthurian literature at all, which seems strange to me because I grew up thinking of Arthur as a very central figure in fantastic precursors. The Once and Future King would have been relevant, as would Malory's Le Morte d'Arthur (Penguin: Volume 1, Volume 2; free Kindle ebook: Volume 1, Volume 2; Project Gutenberg: Volume 1, Volume 2). Arthur of course features in Deities & Demigods and especially in OD&D there is a strong argument for a bit of Arthurian flavor in knightly jousts.
Finally, as has come up repeatedly in the discussion about Arneson, you can't really say that D&D was not inspired by John Norman's Gor series. This was a sword & planet adventure series that gets into some controversy over its idea of sexual slavery, but was a big influence on how Arneson ran Blackmoor. So if you want some of that flavor (it apparently starts out a good deal better than the later volumes), start with Tarnsman of Gor and go as far as you're interested in going.
There are other things we could add to the list all day, but I'm interested in talking about other works that were generally available in the late 1970s, had a fantasy or sci-fi flavor, and especially the ones that had an influence on early D&D players or a clear potential for in-game influence.