Judge Isaac H. Bronson and his family were originally from New York, where he was a lawyer and a member of the Twenty-Fifth Congress. His impressive career there led President Martin van Buren to appoint Bronson circuit judge of East Florida. The Bronson family then relocated to St. Augustine, Florida, which was the headquarters of the court. When Florida became a state in 1845, Bronson was appointed one of the four circuit state judges.
In 1852, he was entrusted a large tract of land in Palatka, Florida by three prominent families of that town. He set aside a portion of this land along the river to construct his estate, which he called "Sunny Point". A staunch abolitionist, Bronson used no slave labor in the construction of the house. He planted citrus in the land surrounding the stately home, and for a time it was a working citrus plantation. The Bronson family became well known in Palatka, and they were deeply involved in helping their new hometown prosper. Sunny Point became a famous gathering place for Florida's upper crust and political elite. His daughters Gertrude and Emma were married at the estate.
Unfortunately, Isaac Bronson passed away only a year after he built Sunny Point. Bronson's wife Sophronia remained there until the onset of the Civil War, when she moved back up to New York - never to return.