The Home Office has stressed it will not discriminate against ethnic minority passengers arriving in the UK.
A recent report by the Equality and Human Rights Commission (EHRC) concluded people of Pakistani origin were 52 times more likely to be stopped at British airports and ports by officials.
The Terrorism Act allows officers to detain people for up to nine hours for questioning when arriving in the UK but the Home Office maintains it can be vital in tackling possible terrorism.
A statement from the department said: "Schedule 7 [of the Terrorism Act 2000] is a vital border security power which helps the police, Security Service and other agencies detect and disrupt terrorist threats."
It is estimated 20 prosecutions a year for terrorist-related crimes occurred between 2005 and 2009 as a result of the Act and it can play a key role in locating important individuals.
According to the EHRC study, a total of 53,992 people were stopped in 2012/13. Passengers who were Asian were 11.3 times more likely to be stopped than those who were white, while black people were 6.3 times more likely to be stopped, and the rate for mixed-race travellers was 3.6 times.
However, a number of human right groups are calling for an urgent review of the policy, which could see a number of changes introduced, including a reduction in the time a person can be held from nine to six hours and a ban on intimate searches, as well as the right to consult with a solicitor after an hour of questioning.
Ratna Lachman, of JUST West Yorkshire, told BBC News more needs to be done to ensure people are detained for the correct reasons.
"Stop-and-searches are happening based on stereotyping rather than being intelligence-led. We need more checks and balances in the operation of this measure,” stressed Ms Lachman.
The UK Border Agency, which operates within the Home Office's remit, has been repeatedly criticised for its poor performance and could be abolished in the coming months.