Once you've clicked on Domains, your screen should look similar to the screenshot below.
Now that you're on Domains, it's time to select the domains you want to remove a label or labels from.
In the example below, I've searched for example (1) to find any domains that include the word example and selected them all (2). Oncey ou've made your select, click ↑ Labels on the bottom of your screen (3).
Clicking ↑ Labels brings up a search box that lets you search for existing labels or create new ones. In the example below, I only have one label so it's as easy as clicking the checkbox next to it to uncheck it (1) and clicking Apply (2).
As you can see from the example below, the domains are no longer labeled with Example.
You can also remove labels from an individual domain by clicking on + and then following the steps outlined above.