I wanted to revisit this thread because I came across a product that seems to work exceptionally well for N-scale dirt...
Here is one view. The product was fully saturated with matte-medium and then left to dry. This is now fully dried. The good thing is that the dirt retains about the same coloring after gluing as it did before the matte medium was applied. In other words there was very little darkening as a result of the glue:

Here's another image of the dirt actually applied on the layout. The dirt isn't quite dry in all spots, but you can see the difference between it's wet and dry looks. I applied matte medium to the side of the hill... I then sprinkled on the dirt. I then used a brush to even it out. Finally it received a good soaking of matte medium from a spray bottle.

I think the final result is excellent.
And again, this is secured with matte medium so it can easily be changed if need be.
What is this product?
"Fine Natural Soil & Dirt", by Scenic Express.