During the recent years, the use of pozzolanic materials (e.g., volcanic ash) in concrete and cement manufacturing has increased significantly since it can reduce the environment hazard associated with using Portland cement. In this paper, the effect of elevated temperatures on the physical and mechanical characteristics of building mortar produced with volcanic ash is experimentally explored. In order to evaluate the performance of the mortar, four different proportions of volcanic ash (0, 5, 15, and 25%)—as weight replacement of the cement—were prepared. A series of tests were conducted after 28, 90, and 120 days under different temperatures (25, 200, 500, and 800 °C). This paper demonstrates that the replacement of cement by a proportion of volcanic ash can sustain an acceptable level of compressive strength and improve the overall characterization of the mortar while reducing the amount of CO2 released. The mortar with 15% ratio of the volcanic ash replacement showed better flexural and the tensile strength. This paper also highlights that the volcanic ash replacement affects the late-age properties of the mortar more than the early age ones at both ambient and elevated temperatures.