You've got a number of different trains of thought going on here. The way I understand Tom Mann, he's talking about the perceived size and appearance of ballast, while others are talking about the actual size of ballast. First off, ballast does come in different sizes. At the AN, we once got a car of ballast the size of softballs. As soon as the Roadmaster got the hopper doors closed, Vulcan Materials heard about it and it was on its way back. Don't know who bought ballast like that, but the AN sure didn't! If you want to go the other way, I don't think there was any stone ballast in Orleans Parish, Louisiana, when I lived there, just oyster shells. The GM&O actually used river sand on its main line.
Now, I am inclined to agree with Tom, I ballasted my portable layout using HO ballast (N was not available back then). I used Highball N ballast on my photo diorama and agree it looks a lot like tile grout. Having tile floors in much of my house I know tile grout.
This goes back to the "theater set" concept of a model railroad as opposed to railroad models. What looks right becomes more important than being exactly to scale in the overall impression of reality reduced to a tiny fraction of the actual scene.
