The Venerable Andrew Forster takes over from the Right Reverend Ken Good, who retired last May.
Lord-Lieutenant for the County Borough of Derry Dr Angela Garvey was among those at the St Columb’s Cathedral service on Sunday.
The new bishop said he was both “excited and daunted” by his new role.
Bishop Forster told the congregation at Sunday’s service he wanted people “to have bigger hearts for those who are on the margins of society”.
“For those who come to us as refugees, for those who are persecuted for their faith, for those who know day by day what it is to live with insecurity and in fear.
“We need to be people with bigger hearts, generous hearts and giving hearts.”
Previously he served as Rector of Drumcliffe and Archdeacon of Elphin and Ardagh and as chaplain at Queen’s University in Belfast.
His first church role was as Curate in Willowfield Parish Church, Belfast.
A second service will be held on 12 January at St Eunan’s Cathedral in Raphoe, County Donegal, that will see the bishop officially installed as Bishop of Raphoe.