The practice, which involves attempting to change someone’s sexual orientation or gender identity, is being outlawed for lesbian, gay and bisexual individuals but not trans people.
LGBT charity Stonewall says conversion therapy is based on an assumption that being lesbian, gay, bi, or trans is a mental illness that can be “cured”.
Groups have criticised the Government’s last minute decision earlier this week to only impose a partial ban.
Mr Javid cited a report by paediatrician and former president of the Royal College of Paediatrics and Child Health Dr Hilary Cass, which highlighted the lack of evidence about the best approach to treating gender dysphoria in children.
“When it comes to conversion therapy it is absolutely right, as the Government has said, that we ban the so-called conversion therapy for LGB people,” he told Sky News.
“When it comes to trans I do think that we need to be more careful. As Health Secretary, in my mind, is the recent report by Dr Hilary Cass who is one of the most experienced paediatricians in this country.
“She just published an interim report a few weeks ago and she talked about how children and young people when they say they have gender dysphoria it is right for medical experts to be able to question that and to determine what the cause might be.
“[Whether] it is a case of genuine gender identity dysphoria or could it be that that individual is suffering from some child sex abuse or could it be linked to bullying.
“I think it is right to take the approach that we have, which is ban conversion therapy for LGB people but take a must more sensitive approach when it comes to trans.”
Mr Javid added that electric shock therapy, sometimes used in the treatment, should not take place.
He said: “I don’t think that anyone should be administered with electric shock…I think that would count as abuse and it would already be against the law.”
Prime Minister Boris Johnson also defended the decision not to include trans people in the Government’s proposed ban, saying there are “complexities and sensitivities” which need to be worked through.
Speaking during a visit to a hospital in Welwyn Garden City, he said: “I’m sorry we haven’t been able to reach agreement with the organisations concerned but that will in no way diminish our determination to tackle prejudice wherever we can.”
The Government’s LGBT+ business champion Iain Anderson has resigned and more than 100 organisations have pulled out of the UK’s first ever global LGBT+ conference over the decision not to ban the practice.
The landmark Safe To Be Me conference, due to take place in June, is now set to be axed by Downing Street after support for the event collapsed.
The Terrence Higgins Trust was among the organisations to boycott the event.
A Terrence Higgins Trust spokesman said: “Trans rights are human rights – progress without or at the expense of trans people is not progress. We stand together and will not be divided.”
Umbrella body, the LGBT+ Consortium, branded the Government’s U-turn on conversion therapy “abhorrent”.
The Government has said it is going to look into ways it can ban trans conversion therapy.