By stating that a person is 'of colour' you are already making a racist statement by excluding or differentiating them from yourself, or people of the 'white' colour. A person "of colour" by logic is every person on earth. By tradition, perhaps, it is a recognition of non-white humans, or non European descendants, but all this does is alienate and segregate humans with words this time, instead of physical fences.
Colour isn't always race; race isn't always culture; culture isn't always character.
My Italianesque pale olive skin (at least the parts that get sunshine)
make me a person 'of colour' just as much as the darker browns of the
Maori. Using the term "of colour" does nothing but perpetuate the idea
that non-white races are "different" from Europeans. Our cultures may
very well be different, but then, house to house cultures and behaviours
are different as well. In the end, we are all subtly different with
different shades of skin colour, freckles, hair colour, etc... yet,
fundamentally similar, if not the same, in our human capacity.