DKS: That could be a problem in some areas.
The stuff we get here in eastern Kentucky is matte, because it is crushed stone. There's very little sand here, just limestone, so it comes from the same quarries as the local gravel and larger stuff. The common mortar sand is yellowish, unlike the off-white bulk limestone, but it's still locally crushed.
In Pasco, WA, the local sand is basically "ground up Rocky Mountains and local basalt", ground by the Ice Age Missoula floods, and deposited in the Pasco Basin. It hasn't had time to be rounded, so is also mostly matte. It's a medium brownish-gray, kind of an odd color, really, because of the iron content. Not iron in the rocks, but actual "iron filings", that can be pulled out with a magnet. Not recommended for ballast, but usable for other scenery, as long as it's solidly glued down. The "iron filings" tend to give the sand a slightly rusty tinge. I suspect they came from the local basalt, which also acquires a brownish crust after long weathering.
Those are the only two areas where I have much experience with the sand.