They have also agreed to the “principle of dialogue”, as two West African presidents come to mediate.
Mr Kafando and the prime minister were detained by the presidential guard during a cabinet meeting on Wednesday.
At least three people have died in protests after an ally of ex-President Blaise Compaore was named leader.
The US, France and the African Union (AU) have condemned the takeover in the former French colony.
The influential Balai Citoyen civil society group has put the number of people who have died in demonstrations against the the presidential guard (RSP) in the capital, Ouagadougou, at 10.
An unknown number of protesters have also been detained.
The coup leaders imposed a night-time curfew across the West African state, and have ordered the closure of land and air borders.
Elections were due to be held on October 11 nearly a year after a popular uprising forced Mr Compaore, the long-time ruler of the West African country, from power.
The headquarters of Mr Compaore’s Congress for Democracy and Progress (CDP) party were ransacked in Ouagadougou as news of the coup spread, the AFP news agency reports.
The AU and regional body the Economic Community of West African States (Ecowas), have called for the immediate release of “hostages”, referring to Mr Kafando and interim Prime Minister Isaac Zida.
Senegal’s President Macky Sall, who is going to mediate along with Thomas Boni Yayi of Benin, said he was heading to Ouagadougou in his role as the current chairman of Ecowas.