The caps were getting hot. That might be the failure. Overload by too high resistance and inductance of source might have led to an overload situation. The middle one of the 3 680µF popped, at cool situation with around 50V without any current flow, it was around an hour sitting in this situation while balancing. The charging power supply is nice and was off at that time.
For insiders: The problem about testing with D-current using VESC with motors slowly turning while having masses of ohmic copper losses is, that the Capacitors and the DC-link circuit are not tested. Also the switching voltage losses inside the Fets is not considered correctly. I estimate the real life losses of my VESC 100% more than in D-current test run. Additionally to the losses inside the cells, cables, fuses, BMS, relays.