Elihu Spencer Miller was born in Princeton, New Jersey, on September 3, 1817. He attended the College of New Jersey, graduating in 1836. Miller studied law in the offices of James S. Green in Princeton and later with Reverdy Johnson in Baltimore, where he first passed the Bar. He was admitted to the Pennsylvania Bar in 1843 and continued to practice law in Philadelphia for the remainder of his life.
When the law department at the University of Pennsylvania was re-established in 1852, Miller was chosen to fill the chair of real estate and equity jurisprudence, a position he held for twenty years; he also served as dean of the Law School for his last four years at the University. Miller strenuously opposed moving the Law School to West Philadelphia, maintaining that an ideal Law School should be in the midst of professional law offices. When the decision to move was made over his opposition in 1873, he resigned his professorship, feeling unable to meet the demands of his practice and teaching on the more distant campus.
Miller was a distinguished member of the Law Association, holding a post on its Board of Censors. In 1865, along with William M. Evarts, he was involved a case challenging the Constitutionality of the recently imposed Federal Income Tax. Miller died in his Philadelphia office on March 6, 1879.