why did the poorer classes of white continue to support the slave system even though they did not own slaves or benefit from it