Saying the words "people of color" isn't the same thing as saying "colored people," with all the racist history implied by that phrase.

Indeed, it was developed as an alternative to "African American."

Like others, I have come to use the phrase "black people and people of color" to designate the wide spectrum of people experiencing racism without eliding too many differences together.

Still, "black people" (even when describing specifically people of African descent) can also be problematic, as Jason notes, because the backgrounds of people who've recently (that is, post colonial) immigrated are very different in many ways than those whose African descent was filtered through slavery.

And yet, racists in America treat them exactly the same as they treat the native born black Americans whose presence is directly related to slavery.

I suggest that there are times when it's important to acknowledge the backgrounds may be different, but their experiences with racism subsequent to their immigration will in many ways be similar.