I've carried heavy things (maybe 20lbs) in the top box on my FXS. Never really noticed a huge reduction in agility, but that was all on the streets of Chicago, not trails or anything an FX is better suited for. I've seen a handful of people that have installed chargers into top cases and it is a tempting idea. I just like my top case to be free for actual storage, so I clamped my charger onto the grab bars on the passenger seat.
See here for a couple shots of my clamped on charger. I thought I posted a thread about mounting my J-1772 adapter on the fender, but I guess I didn't actually click post. I'll have to rewrite that and post it.
The other reason I'd probably not want to permanently mount a lot of weight in place of the top box, is because it's such a long lever at that point. It's a couple feet from the main frame at that point. Depending on your rides and the forces applied to that lever it could cause the rack, or subframe to fail prematurely.