I realised what was causing the error. When the Tax Category rule is present, it automatically populates the In State or Out State GST in the Purchase Taxes and Charges Template before the form probably gets to process some other parts. This causes all taxes to show up.
If we remove the tax from Purchase Taxes and Charges Template and then select the same tax manually after this, the tax rate is correctly updated to contain only IGST or SGST + CGST.
Pretty sure its a bug, that this doesn’t work when tax category automatically populates the Purchase Taxes and Charges Template.