I don’t know if turning of the inverter has anything to do with it but you can leave the inverter on. When you have ac on your rv, as when you start your generator or plug into shore power, the inverter has a circuit in it that bypass the batteries and goes straight through the inverter. Try that the next time.
__________________
|