Nobody should have to put up with something as vile and pointless as racism. However, like any other industry, complain to your boss, in this case your manager.
Let them make the decision whether to withdraw to whole team.
Let them make the decision whether to withdraw to whole team.