Born in Deruyter, New York on June 28, 1836, Gage was forced to abandon his education at age 14 when poverty afflicted his family. After working as a bookkeeper and cashier in Chicago banks, Gage climbed the ladder within the industry, eventually become the leader and treasurer of the "Honest Money League of the North West." By 1882, he had become the vice president and executive officer of the First National Bank of Chicago.

Gage switched party affiliations in 1884, throwing his support behind Democrat Grover Cleveland for President. He declined an offer to become treasury secretary during Cleveland's second term, but in 1897, newly elected President William McKinley offered Gage, his backup choice, the position again; this time, Gage accepted. The appointment received widespread support from major newspapers. In turn, Gage helped to finance the Spanish-American War while contributing to the passage of the Gold Standard Act of 1900. President Theodore Roosevelt retained Gage as treasury secretary after McKinley's assassination, but Gage soon resigned in protest of TR's interference in departmental matters. Following his tenure in the cabinet, Gage became president of the U.S. Trust Company of New York until 1906. He died on January 26, 1927.