The story of "Rich Joe" Vann is actually pretty incredible. He was in the room when his father was murdered, in a tavern near the site of the family-owned ferry. Spirited off by a trusted slave, "Rich Joe" returned to his palatial estate near present-day Chatsworth. James Vann, who had become a powerful chief in the Cherokee Nation wanted his son to inherit the wealth James had accumulated, instead of his wives as was the Cherokee tradition at the time. In council the Cherokee granted Joseph Vann the inheritance.
Over the years Joseph Vann became as well-known as his father to both to the Cherokee and other Southeastern tribes and Georgians. Word of his home (today's Vann House), along the old Federal Highway, spread throughout the state and was the envy of many settlers. In 1819, while on a trip from Augusta to Nashville, President James Monroe spent the night with three other men. Originally set to spend the night in the Spartan Moravian mission at Spring Place, Monroe traveled to meet Vann in his house about a mile away. When he saw the comfortable surroundings, President Monroe requested permission to spend the night in the Vann House rather than the mission.
After the Sixth Georgia Land Lottery
leading up to the Cherokee Trail of Tears
, Vann was evicted by Colonel William Bishop of the brutal Georgia Guard. Bishop claimed Vann had violated Georgia law by allowing a white man to work for him without a permit. The Colonel used the house as his local headquarters and permitted his brother, Absalom Bishop, to live there, which did not sit well with Spencer Riley, who won the house in the lottery. Riley tried to settle in the house, something Col. Bishop did not like, so the Colonel had his men throw a smoldering log on the steps, forcing Riley out and returning Absolam to the house.
All this didn't matter to Joseph Vann, who was living in Tennessee at the time.