I hope that this final dungeon isn't so close to where Team Evil are that they can hear the fighting, figure out the direction, and create some kind of shortcut. I think it's quite unlikely, but...
I think one of the great unwritten rules of D&D is that fighting one group doesn't aggro the next.
The thing is the Azurites don't use a single color; they use a single hue. The use light blue, dark blue, black, white, glossy blue, off-white with a bluish tint. They sky's the limit, as long as it's blue.