is pointing out that prejudice and racism exist in society
When he says that he, as a darker skinned man, is sent into the kitchen when "company" (probably white people) comes, he is showing that people are still racist and treat black people like they are a part of society that should be hidden and not seen in public.