During the civil war, many Southern states codified into law this expectation that all white men should be on the lookout for Black people sneaking around, living life free and stuff. States like Georgia brought their "citizens arrest" law into existence in 1863 for this exact reason.
"Now we are all slave patrol!"
("We" means "white people.")