A couple of thoughts:
1. Kato #4s are notorious for having the points "picked" by wheel flanges and causing derailments. The fix is to file slots in the stock rails that allow the points to go into those recess slots so that the flanges do not pick them. So, if that is your problem, you could work on the #4s and keep your track plan. But, if the derailing is due to curvature pulling long equipment off the track, then you will need the #6s.
2. Kato #6s, as they come from the factory, result is wider track spacing in yards. They can be cut down to keep the same track spacing as the #4s, which is Kato's standard double track spacing. To see if that works for you, see how your equipment manages Kato's double crossover, which is 4 #6s with track spacing the same as #4s. Or, if space is not a problem, just use the #6s as they are.