If I have full physical custody and I die, will my husband have any visitation/custody rights to my 2 kids with my ex?

I take it there is no step-parent adoption or other legal rights having been granted and your husband is not the Dad.  He will need to acquire rights based on their having lived with him, etc., before your death (assuming that is the case).  He should see a lawyer if he wants to get that in line before you die.