I left the fender on, no issues removing the swing arm. (As mentioned before). Good opportunity to clean and lube all the suspension linkage needle bearings and of course swing arm bearings.
It is a pain to get the drive shaft back into the u - joint with the swing arm installed, I recommend inserting the shaft as you install the swing arm, moly paste on splines too. (Not meaning to offend if you already know all this.)