The VAT rules related to your particular business need to be understood if you are to pay the right amount. Paying too little means that you have to pay interest when you discover the liability and declare it. If you fail to spot it before HMRC do, you may also be charged a penalty.
If you overpay because of your own mistake, you do not get any interest when you find and claim it. Moreover, you cannot make that claim once three years have passed since the end of the VAT period in which you overpaid.