The seals are better than most Australian cars (which have to be good). The effort to make the car pristine again after long trips over dirt roads is fairly minimal, and I like the efforts that Jaguar have gone to in order to keep stuff OUT of the car. I look at some UK vehicles that have taken a beating (thanks to endless rain, salt and snow), and reckon that sand/gravel/dirt is much more forgiving on the car overall. And we tend not to suffer soggy footwells or door aquariums here - there simply isn't enough rain!
The only downside to long unsealed road travel is that I've bent the odd wheel - again, this is not unusual on Australian roads, and there are plenty of companies that offer to true up out-of-round wheels.
Interestingly, when the facelift model was released in Australia, the XF 2.2 made a circumnavigation of Australia (14,000km) in 24 days - lots of unsealed roads, too - and broke MPG records (3.98 l/100km) in doing so. The couple who achieved this went on to circumnavigate Tasmania (1136km) in an XF260 2.0 and averaged 3.75 l/100km (on a single tank) - Tassie has a lot of hills, which makes it more of a challenge than it might seem.