More than 1,000 violent crimes, including five killings, have been committed by prisoners released early with electronic tags, it has emerged.
Home Office figures show tagged offenders have committed one murder and four manslaughters, among other crimes, since the scheme began in 1999.
Minister Gerry Sutcliffe said a balance had to be struck between rehabilitating prisoners, and public safety.
But the Conservatives said it showed a "shocking disregard for public safety".
Under the Home Detention Curfew, prisoners can be released up to four and a half months early, as long as they wear an electronic tag.
MPs on the Commons Public Accounts Commitee said it cost £70 a day less to enforce a curfew, than keep people in jail.
But chairman Edward Leigh said too many crimes were being committed by people released early with tags, and not enough information was getting back to the prison governors who made the decision to release them.
In one case, Danny Cann from south London battered a man to death with a baseball bat, after being freed early from a jail term for robbery. He was later convicted of murder.
Mr Sutcliffe, a Home Office minister, said less than 4% of people offended while wearing tags.
"Most people see it as a good scheme to help them back into society," he told the BBC.
"There has to be a punishment, as far as the sentence is concerned, but there has to be rehabilitation as well."
The prison population reached a record 79,843 at the weekend with, in theory, just 125 more spaces left.
In response, on Monday, Home Secretary John Reid outlined a series of measures to relieve pressure on prisons - including freeing up 500 spaces in police cells from Thursday.
It is not clear whether they will be needed yet, as Mr Sutcliffe said the number of prisoners had been "reduced" this week.
Juliet Lyon, director of the Prison Reform Trust, said: "Compared with the soaring reconviction rates for all those leaving our overcrowded jails and the cost to the public purse of more than £11bn a year from reoffending, the Home Detention Curfew makes sound economic sense."
For the Conservatives, shadow home secretary David Davis said the report raised serious questions about the way tagging was being used.
"With so many serious offences being committed it is clear the government is showing a shocking disregard for public safety."
While ministers said a reoffending rate while tagged of below 4% was good - his view was that those offences would not have happened had the prisoners not been freed early under the tagging scheme.
He said he was in favour of tagging - but only for offenders to wear for a period once they have completed their sentence.
Liberal Democrat home affairs spokesman Nick Clegg said flaws in the tagging system could be "laid squarely at the government's feet for failing to implement the system competently in practice".
Former senior probation officer David Fraser said the government should abandon tagging altogether.
"Supervising persistent offenders in the community, with or without a tag, is disastrous for the public," he said.
"The public need to be protected from crime. It is absolutely amazing that [the government] are able, somehow, to ignore this. What must happen? They are sleepwalking into civil unrest, in my view."
Comment from Forum: Wearing a tag doesn't prevent further crimes, it means you can do your crime but must be home by 7o'clock. Geff Grate, UK
Story from BBC NEWS
Sounds pretty good, unless you're the officer who has to visit the families of the murder victims.
We now seem to have a society that is at loggerheads over our penal system, which is full to capacity. Rehabilitation is working in some cases, but a lot of released prisoners re-offend, most moving up the felony ladder.
What can be done?