Actually, the only thing that matters regarding this particular type of damage is the sidewall; there is little difference in off-road capability whether you are using 18's, 20's, or even 15's, accept the sidewall height (all other things like ply being equal -- and yes it will still be a rougher ride, but that goes to Your ability to tolerate it, not the equipments).

Average sidewall height for 18's is about 6 inches; for 20's it's about 4.5-5. That is why the damage occurs. Stepping up to a larger series tire (ie a 295/45r20) with a 5.5-6 inch sidewall mitigates the difference pretty well. Of course, if you really want to get out there hard in rocky, craggy terrain, your best bet would be to drop to 15's and have 8 inches of sidewall!

The other issue here is rim width; a 9.5 will be a bit less likely to be damaged than a 10, but there is a limit to how narrow one can go based on the rubber.