Been there, done that. If you leave the bike on and push with eco set on regen with off-throttle, you can build up some regen after a bit too. After pushing and probably more important...coasting a bit after the bike stopped... it got recharged a little and I was able to ride the Zero S a little more (1 block) after a reboot. Sort of like in The Princess Bride where Westley was only MOSTLY dead instead of all dead.
Hm. If you leave it turned on, the headlight stays on, which is a 50W draw IIRC. Also the 12V converter, the dashboard, all the controllers...let's be conservative and say 60W total. So in addition to moving the bike, you have to supply an extra 60W just to break even, let alone some more to provide any regen. Lance Armstrong, at his drug-fueled maximum, could provide maybe 700 watts of power for a fairly short amount of time. This out-of-shape 52-yo would have a hard time just breaking even. Obviously a downward slope helps, but I was headed uphill, which would tip the balance even farther against me.
But I abbreviated my experience. Here's the full details of the end of my ride: At maybe 1000 yards from home, I still had 3% showing, which I felt pretty good about...I'd been losing a percent in maybe a half mile or so for a while. But as I turned right and started ascending the penultimate (not very steep) hill, the display immediately dropped to 2%, and I had very little power under my right hand. That seemed fine; by this time I had just a few hundred yards to go, and I figured the controller was cutting back power to help me stretch the last little bit of my range. But within a hundred yards or so, it ticked to 1%, and I lost all power, coasting to a stop maybe 300 yards from my driveway. Honestly, I felt a little offended that the display still showed I had 1% SoC available, but it wouldn't give me anything. But there were only maybe 100 yards of mild uphill, followed by 30 yards or so of mild downhill, and then a couple hundred of mild uphill to my driveway. Not terribly daunting, but first let's see if I'm truly empty. I turned the key off, then on. Still showed 1%, no response to the throttle. Power cycle it again, still 1% and no response. Otay, Panky, time to get off and start pushing, for which I turned the key off. After reaching the top of the first hill, just 100 yards or so, I got back in the saddle, turned the key on....and it gave me power! I stayed on it, not wanting to upset anything, through the few yards of downhill followed by the couple hundred of uphill to my driveway. It stopped again just as I turned into my driveway, but at that point I was overjoyed to push it the last 10 feet.
So. It certainly didn't regen while I was pushing it uphill with the key off. I'm thinking that while I was pushing it, in the absence of any load, the cell voltage recovered a tiny bit (which batteries do), passed some threshold and the controller gave me just a little bit of a last gasp. I'm wondering if your experience was similar?