The Armenian Patriarchate of Jerusalem is located in the Old City of Jerusalem (Palestine), in the Armenian Quarter which takes up the one-sixth of the old city and occupies the entire southwest corner of the town. The Armenian Patriarch and the Brotherhood of St. James, together with the Greek and Roman Catholic patriarchs, are the sole guardians of the Dominical Sites, the holiest shrines of Christendom.

The Armenian presence in Jerusalem dates back to early Christian times. From as early as the fourth century we have records about Armenian monks in the Holy Land. Over the ensuing centuries Armenian monks and pilgrims built several monasteries, with as many as seventy institutions mentioned by a seventh century Armenian writer. Armenian mosaics with Armenian inscriptions from the fifth and sixth centuries indicate a very early Armenian presence in the city.

Originally, the city of Jerusalem had one bishop and chronologically first in the line of bishops had been St. James, the Brother of the Lord. Armenian bishops from Greater Armenia visited the Holy Land and some may have lived there for extended periods of time. These pilgrim bishops, priests and laymen probably suffered persecution under Byzantine rule as a result of the schism in the church after the Council of Chalcedon in 451, since the bishops of Jerusalem adhered to the faith of the Byzantine Empire, whereas the Armenians remained true to the doctrines of the early church. When the Arabs seized Jerusalem in 637, the Armenians took the opportunity to set up their own bishop, a cleric named Abraham, to head he followers of the Armenian faith. It became traditional for the Armenian patriarchs to consider this Abraham as the first of the 91 succeeding bishops.