Wheat bread has more fibre in it and bacterias find it more difficult to find sustenance from it. What molds need to act is moisture and if in bread has more fiber, it´s texture is less moisture than those breads with less fiber.
Well, that´s just my theory.