At the Bryanston Methodist church in Johannesburg, President Jacob Zuma urged South Africans not to forget the values Mr Mandela stood for.
At the Regina Mundi Catholic Church in Soweto, priest Sebastian Roussouw said the late president had been "a light in the darkness".
A national memorial service is due to be held on Tuesday.Mr Mandela's body will lie in state in Pretoria on the following three days and he will be given a state funeral on 15 December.
About 60 heads of state and government have confirmed they will attend the memorial service or the funeral, the government says.
South Africans have been holding vigils since Mr Mandela died at home at the age of 95, after several months of ill health.
Addressing the congregation in Johannesburg - including members of the Mandela family - Mr Zuma praised Mr Mandela for his commitment to peace and reconciliation.
"He stood for freedom, he fought against those who oppressed others. He wanted everyone to be free."
In Sunday's service at the Regina Mundi Church - which acted as a vital meeting place during the apartheid era - priest Sebastian Roussouw praised Mr Mandela for his "humility and forgiveness".
"Madiba did not doubt the light. He paved the way for a better future, but he cannot do it alone," he said, referring to Mr Mandela by his clan name.
In Cape Town, Archbishop Thabo Makgoba said Mr Mandela was a powerful and continuing reminder that individuals do have the power to make change happen in the world.
Mr Mandela will be buried in his home of Qunu on Sunday - security there has been stepped up
Mr Mandela's successor as president, Thabo Mbeki, will attend a service at the Oxford Shul synagogue in Johannesburg in the afternoon.
Other senior politicians and ANC officials are attending services across the country, in what correspondents say it an attempt to involve more scattered communities.
'Guard of honour'
The focal points for public remembrance have so far been Mr Mandela's house in the Johannesburg suburb of Houghton and his old home in Soweto.
Mourners and wellwishers there have lit candles and laid thousands of wreaths of flowers.
Mr Mandela's body will lie in state on Wednesday, Thursday and Friday at the heart of the South African government in Pretoria, before his funeral on Sunday.
His body will be taken each day at 07:00 from the mortuary to the city hall through the streets of Pretoria. Members of the public have been encouraged to line the route and form a "guard of honour".
The family will view the body on Wednesday morning before the public are allowed to file past from 12:00.
