Having recently had the pleasure of playing in Allan Grohe's Castle Greyhawk, one of the things that struck me was that the Greyhawk dungeons are made not out of grey bricks (maybe that's a bit on the nose?) but actually out of black basalt. And that sort of surprised me, because I typically think of dungeons as being, well, places with lots of big sturdy grey stone bricks.
The dungeons beneath the Louvre are, as you can see, pale - almost white. There are other things we can think of as well; dungeons carved from the local stone and taking on its colors, dungeons built up with fired-clay bricks, or even plaster-coated walls.
What is on the walls, floors and ceilings can really switch up the look and feel of a dungeon level. Black basalt makes Greyhawk feel forbidding and alien, whereas the stark bricks of the Louvre are harsh and unrelenting. Brown, red and grey are all colors that can vary up between warmth, human design, and intensely difficult areas. More exotic stones are also possible; other colors, or striated marble or sandstone, or rocks containing bisected quartz, for instance. And other forms of architecture are also possible, such as the cyclopean style where unworked rocks are loosely fit together. Or rooms could be accidental, effectively enormous geodes that were struck when digging.
Of course, stone's not the only thing that the walls might be made of. For instance, a dungeon in an old salt mine could have walls that are literally covered with salt. An abandoned coal mine would pose obvious problems for people who are using torches to light their way, especially if it's got a ready source of firedamp. Salt mines are a good place for some truly monumental caverns as well.
These decisions don't have to be made for an entire dungeon. Different building materials can be a tip-off of different construction; perhaps there are rough-hewn stone walls built by (naturally occurring) kobolds that are clearly differentiated from ancient brick walls, and in the deeper levels you get to strange cyclopean architecture and more alien materials. It's a good way to differentiate levels and sub-levels, if you want to make the differentiation.
There's also the question of humidity. Being underground, dungeons should be humid. Their temperature is relative to the air temperature outside, but it's easy for an underground hole to have a humidity so high that they could develop condensation on the walls, ceiling or floor. (This is of course a great case for a red herring.) This would be worse in an area with dirt floors, and better with solid floors. In colder areas dungeons will always be cold; in hot areas they will tend to stay hot.
We spend a lot of time in the dungeon; I think a bit of variety can go a long way to mark your dungeon as different from others.