John Jacob Astor IV and his mother Caroline Schermerhorn Astor residence designed by Richard Morris Hunt c. 1896 at 840 Fifth Avenue and East 65th Street in New York City. The Astors moved into this house after demolishing their brownstone at 34th Street and Fifth Avenue to build the Astoria portion of the Waldorf-Astoria Hotel. Click HERE for more on the Astor residence which was demolished in 1926 and replaced with Temple Emanu-El. The above photo was taken in 1897 and below in 1898 (which is one of the best images of Fifth Avenue before the turn of the century I've seen). Photos from the Museum of the City of New York.