Please check your data import template as if you are importing and you are setting up a Custom Field to be a associated with a certain tag group then please make sure you have spelt the name of the tag group correctly.
If there is a spelling mistake and the data is imported the custom field will be associated to 'All contacts' (as shown above) instead of your tag group as it would not be able to find a match.
For assistance please contact our Support Helpdesk.