Born in Newtown, Ohio, Judson Harmon graduated Denison University in 1866. During the Civil War, Harmon spent a summer in the Ohio home guard and helped repel a potential invasion by Confederate cavalry raiders. Following the war, he received a law degree from the University of Cincinnati Law School in 1869 and secured appointment as a judge.

Although originally a Republican, Harmon crossed party lines to support the Democrats after 1872. In 1895, Grover Cleveland selected Harmon to succeed Richard Olney as attorney general. In that capacity, Harmon prosecuted the Trans-Missouri Freight Association under the new Sherman Anti-Trust Act.

Leaving office in 1897, Harmon returned to his Cincinnati law practice. An attractive moderate Democratic figure in Ohio politics, Harmon received the Democratic nomination for governor in 1908 and became the first former cabinet official to be elected to a governorship. He served two terms as Ohio's governor, defeating Republican Warren Harding in 1910. While in Columbus, he endorsed the sixteenth amendment, supported workman's compensation, and oversaw the creation of a public utility commission.

Harmon continued to practice law despite serving as governor and became a professor at Cincinnati Law School later in life.