Communications firms are being asked to record all internet contacts between people as part of a modernisation in UK police surveillance tactics.
The home secretary scrapped plans for a database but wants details to be held and organised for security services.
The new system would track all e-mails, phone calls and internet use, including visits to social network sites.
The Tories said the Home Office had "buckled under Conservative pressure" in deciding against a giant database.
Announcing a consultation on a new strategy for communications data and its use in law enforcement, Jacqui Smith said there would be no single government-run database.
Communications data is an essential tool for law enforcement agencies to track murderers and paedophiles, save lives and tackle crime
Jacqui Smith
Home Secretary
But she also said that "doing nothing" in the face of a communications revolution was not an option.
The Home Office will instead ask communications companies - from internet service providers to mobile phone networks - to extend the range of information they currently hold on their subscribers and organise it so that it can be better used by the police, MI5 and other public bodies investigating crime and terrorism.
Ministers say they estimate the project will cost £2bn to set up, which includes some compensation to the communications industry for the work it may be asked to do.
"Communications data is an essential tool for law enforcement agencies to track murderers, paedophiles, save lives and tackle crime," Ms Smith said.
"Advances in communications mean that there are ever more sophisticated ways to communicate and we need to ensure that we keep up with the technology being used by those who seek to do us harm.
"It is essential that the police and other crime fighting agencies have the tools they need to do their job, However to be clear, there are absolutely no plans for a single central store."
'Contact not content'
Communication service providers (CSPs) will be asked to record internet contacts between people, but not the content, similar to the existing arrangements to log telephone contacts.
REASONS TO CHANGE WHAT CAN BE KEPT
More communication via computers rather than phones
Companies won't always keep all data all the time
Anonymity online masks criminal identities
More online services provided from abroad
Data held in many locations and difficult to find
Source: Home Office consultation
But, recognising that the internet has changed the way people talk, the CSPs will also be asked to record some third party data or information partly based overseas, such as visits to an online chatroom and social network sites like Facebook or Twitter.
Security services could then seek to examine this data along with information which links it to specific devices, such as a mobile phone, home computer or other device, as part of investigations into criminal suspects.
The plan expands a voluntary arrangement under which CSPs allow security services to access some data which they already hold.
The security services already deploy advanced techniques to monitor telephone conversations or intercept other communications, but this is not used in criminal trials.
Ms Smith said that while the new system could record a visit to a social network, it would not record personal and private information such as photos or messages posted to a page.
"What we are talking about is who is at one end [of a communication] and who is at the other - and how they are communicating," she said.
Existing legal safeguards under the Regulation of Investigatory Powers Act would continue to apply. Requests to see the data would require top level authorisation within a public body such as a police force. The Home Office is running a separate consultation on limiting the number of public authorities that can access sensitive information or carry out covert surveillance.
'Orwellian'
Liberal Democrat home affairs spokesman Chris Huhne said: "I am pleased that the Government has climbed down from the Big Brother plan for a centralised database of all our emails and phone calls.
"However, any legislation that requires individual communications providers to keep data on who called whom and when will need strong safeguards on access.
"It is simply not that easy to separate the bare details of a call from its content. What if a leading business person is ringing Alcoholics Anonymous, or a politician's partner is arranging to hire a porn video?
"There has to be a careful balance between investigative powers and the right to privacy."
DATA CONSULTATION
Download the document [676 KB]
Most computers will open this document automatically, but you may need Adobe Reader
Download the reader here
Shadow home secretary Chris Grayling said: "The big problem is that the government has built a culture of surveillance which goes far beyond counter terrorism and serious crime. Too many parts of Government have too many powers to snoop on innocent people and that's really got to change.
"It is good that the home secretary appears to have listened to Conservative warnings about big brother databases. Now that she has finally admitted that the public don't want their details held by the State in one place, perhaps she will look at other areas in which the Government is trying to do precisely that."
Guy Herbert of campaign group NO2ID said: "Just a week after the home secretary announced a public consultation on some trivial trimming of local authority surveillance, we have this: a proposal for powers more intrusive than any police state in history.
"Ministers are making a distinction between content and communications data into sound-bite of the year. But it is spurious.
"Officials from dozens of departments and quangos could know what you read online, and who all your friends are, who you emailed, when, and where you were when you did so - all without a warrant."
The consultation runs until 20 July 2009.
-----------
Opinions?
The home secretary scrapped plans for a database but wants details to be held and organised for security services.
The new system would track all e-mails, phone calls and internet use, including visits to social network sites.
The Tories said the Home Office had "buckled under Conservative pressure" in deciding against a giant database.
Announcing a consultation on a new strategy for communications data and its use in law enforcement, Jacqui Smith said there would be no single government-run database.
Communications data is an essential tool for law enforcement agencies to track murderers and paedophiles, save lives and tackle crime
Jacqui Smith
Home Secretary
But she also said that "doing nothing" in the face of a communications revolution was not an option.
The Home Office will instead ask communications companies - from internet service providers to mobile phone networks - to extend the range of information they currently hold on their subscribers and organise it so that it can be better used by the police, MI5 and other public bodies investigating crime and terrorism.
Ministers say they estimate the project will cost £2bn to set up, which includes some compensation to the communications industry for the work it may be asked to do.
"Communications data is an essential tool for law enforcement agencies to track murderers, paedophiles, save lives and tackle crime," Ms Smith said.
"Advances in communications mean that there are ever more sophisticated ways to communicate and we need to ensure that we keep up with the technology being used by those who seek to do us harm.
"It is essential that the police and other crime fighting agencies have the tools they need to do their job, However to be clear, there are absolutely no plans for a single central store."
'Contact not content'
Communication service providers (CSPs) will be asked to record internet contacts between people, but not the content, similar to the existing arrangements to log telephone contacts.
REASONS TO CHANGE WHAT CAN BE KEPT
More communication via computers rather than phones
Companies won't always keep all data all the time
Anonymity online masks criminal identities
More online services provided from abroad
Data held in many locations and difficult to find
Source: Home Office consultation
But, recognising that the internet has changed the way people talk, the CSPs will also be asked to record some third party data or information partly based overseas, such as visits to an online chatroom and social network sites like Facebook or Twitter.
Security services could then seek to examine this data along with information which links it to specific devices, such as a mobile phone, home computer or other device, as part of investigations into criminal suspects.
The plan expands a voluntary arrangement under which CSPs allow security services to access some data which they already hold.
The security services already deploy advanced techniques to monitor telephone conversations or intercept other communications, but this is not used in criminal trials.
Ms Smith said that while the new system could record a visit to a social network, it would not record personal and private information such as photos or messages posted to a page.
"What we are talking about is who is at one end [of a communication] and who is at the other - and how they are communicating," she said.
Existing legal safeguards under the Regulation of Investigatory Powers Act would continue to apply. Requests to see the data would require top level authorisation within a public body such as a police force. The Home Office is running a separate consultation on limiting the number of public authorities that can access sensitive information or carry out covert surveillance.
'Orwellian'
Liberal Democrat home affairs spokesman Chris Huhne said: "I am pleased that the Government has climbed down from the Big Brother plan for a centralised database of all our emails and phone calls.
"However, any legislation that requires individual communications providers to keep data on who called whom and when will need strong safeguards on access.
"It is simply not that easy to separate the bare details of a call from its content. What if a leading business person is ringing Alcoholics Anonymous, or a politician's partner is arranging to hire a porn video?
"There has to be a careful balance between investigative powers and the right to privacy."
DATA CONSULTATION
Download the document [676 KB]
Most computers will open this document automatically, but you may need Adobe Reader
Download the reader here
Shadow home secretary Chris Grayling said: "The big problem is that the government has built a culture of surveillance which goes far beyond counter terrorism and serious crime. Too many parts of Government have too many powers to snoop on innocent people and that's really got to change.
"It is good that the home secretary appears to have listened to Conservative warnings about big brother databases. Now that she has finally admitted that the public don't want their details held by the State in one place, perhaps she will look at other areas in which the Government is trying to do precisely that."
Guy Herbert of campaign group NO2ID said: "Just a week after the home secretary announced a public consultation on some trivial trimming of local authority surveillance, we have this: a proposal for powers more intrusive than any police state in history.
"Ministers are making a distinction between content and communications data into sound-bite of the year. But it is spurious.
"Officials from dozens of departments and quangos could know what you read online, and who all your friends are, who you emailed, when, and where you were when you did so - all without a warrant."
The consultation runs until 20 July 2009.
-----------
Opinions?
