James Oliver, Sr was born in 1891 in Lenawee County, Michigan as the son of Charles Eugene and Helen L. Oliver.

He was married to Nora and they had at least one child, James Oliver, Jr.