To rename or delete a tag, go to the top menu in Manage > Tags.
The list of all tags available on your Grytics will be displayed. To perform your action, find your tag, then in the Actions column, click on the "..." > rename or delete.

You can remove tags you have assigned to items. Simply select the elements on the page by checking the box to the left of the element, then click in the top right on Bulk Actions > Untag.