I think unplugging while in the middle of full-current charging (when the current is highest) might be worse than plugging at the beginning of a charge (when the current is smaller). Since many unplug only after charging has completed, probably a non-issue for them. I often unplug at partial charge and I think it is safer for the bike and cord to do it at the outlet in the long run.
The other aspect is a quick and decisive motion to connect and disconnect. Not a slow and careful approach that would let the arching linger longer than necessary ...
The potential damage due to the arching is slow and cumulative - it will happen over many charge cycles. Will it ever get bad enough to notice? If you are lucky enough - no, it won't get bad enough to cause overheating or a bad connection. But if the contacts are already weak for some other reason, it might happen faster.
Now if that rubber plug on the bike would somehow remind me to put it back over the charge port on the bike after charging, it would be ideal
I keep forgetting it off and dangling after a charge, leaving the port exposed while I ride...
Well there is another reason that I plug the charger cord into the side of the bike instead of the wall outlet. My bike is parked in front of the wall and I have to lean over it and extend my arm to reach the wall outlet. Leading to the possibility of me loosing my balance and falling on to the Zero, possibly pushing it forward and off of the side stand. (It did happen once a couple of years ago. ) Anyway, so far I have charged my Zero hundreds of times by plugging the cord into the bike's terminal last without any problems occurring.