According to legend, the people around this area used to see a vision of an old lady appearing on the hill where the pagoda now sits. Upon seeing people, she always said that someday a great leader would build a pagoda at this site to bring peace to the country. One day, when Lord Nguyen Hoang passed by this area, upon hearing the story, he ordered the construction of the pagoda in 1601. He named it Thien Mu Pagoda - Heavenly Lady Pagoda.

Overlooking the southwest bank of the Huong River (Perfume River) in Hue , the Pagoda has two sections. The front of the Pagoda can be accessed from the Perfume river and has the Phuoc Duyen tower which can be seen from many points in Hue. This area also houses the great bell which was cast in 1710. It also has the turtoise bearing a great stele carved in 1715.

The rear of the pagoda is the main hall where Buddha is enshrined. It is also an area where the monks of the temple live and practice Buddhism. The Pagoda was the center of Buddhism in central Vietnam. In 1963, in a defying act against the Diem regime, Thich Quang Duc burned himself in downtown Saigon. Today, the car that took him to Saigon is stored in the rear of Thien Mu Pagoda.