By Ian Johnston
Published: 9:08PM BST 19 Sep 2009
A five-year research programme, called Project Indect, aims to develop computer programmes which act as "agents" to monitor and process information from web sites, discussion forums, file servers, peer-to-peer networks and even individual computers.
Its main objectives include the "automatic detection of threats and abnormal behaviour or violence".
Project Indect, which received nearly £10 million in funding from the European Union, involves the Police Service of Northern Ireland (PSNI) and computer scientists at York University, in addition to colleagues in nine other European countries.
Shami Chakrabarti, the director of human rights group Liberty, described the introduction of such mass surveillance techniques as a "sinister step" for any country, adding that it was "positively chilling" on a European scale.
The Indect research, which began this year, comes as the EU is pressing ahead with an expansion of its role in fighting crime, terrorism and managing migration, increasing its budget in these areas by 13.5% to nearly £900 million.
The European Commission is calling for a "common culture" of law enforcement to be developed across the EU and for a third of police officers – more than 50,000 in the UK alone – to be given training in European affairs within the next five years.
According to the Open Europe think tank, the increased emphasis on co-operation and sharing intelligence means that European police forces are likely to gain access to sensitive information held by UK police, including the British DNA database. It also expects the number of UK citizens extradited under the controversial European Arrest Warrant to triple.
Stephen Booth, an Open Europe analyst who has helped compile a dossier on the European justice agenda, said these developments and projects such as Indect sounded "Orwellian" and raised serious questions about individual liberty.
"This is all pretty scary stuff in my book. These projects would involve a huge invasion of privacy and citizens need to ask themselves whether the EU should be spending their taxes on them," he said.
"The EU lacks sufficient checks and balances and there is no evidence that anyone has ever asked 'is this actually in the best interests of our citizens?'"
Miss Chakrabarti said: "Profiling whole populations instead of monitoring individual suspects is a sinister step in any society.
"It's dangerous enough at national level, but on a Europe-wide scale the idea becomes positively chilling."
According to the official website for Project Indect, which began this year, its main objectives include "to develop a platform for the registration and exchange of operational data, acquisition of multimedia content, intelligent processing of all information and automatic detection of threats and recognition of abnormal behaviour or violence".
It talks of the "construction of agents assigned to continuous and automatic monitoring of public resources such as: web sites, discussion forums, usenet groups, file servers, p2p [peer-to-peer] networks as well as individual computer systems, building an internet-based intelligence gathering system, both active and passive".
York University's computer science department website details how its task is to develop "computational linguistic techniques for information gathering and learning from the web".
"Our focus is on novel techniques for word sense induction, entity resolution, relationship mining, social network analysis [and] sentiment analysis," it says.
A separate EU-funded research project, called Adabts – the Automatic Detection of Abnormal Behaviour and Threats in crowded Spaces – has received nearly £3 million. Its is based in Sweden but partners include the UK Home Office and BAE Systems.
It is seeking to develop models of "suspicious behaviour" so these can be automatically detected using CCTV and other surveillance methods. The system would analyse the pitch of people's voices, the way their bodies move and track individuals within crowds.
Project coordinator Dr Jorgen Ahlberg, of the Swedish Defence Research Agency, said this would simply help CCTV operators notice when trouble was starting.
"People usually don't start to fight from one second to another," he said. "They start by arguing and pushing each other. It's not that 'oh you are pushing each other, you should be arrested', it's to alert an operator that something is going on.
"If it's a shopping mall, you could send a security guard into the vicinity and things [a fight] maybe wouldn't happen."
Open Europe believes intelligence gathered by Indect and other such systems could be used by a little-known body, the EU Joint Situation Centre (SitCen), which it claims is "effectively the beginning of an EU secret service". Critics have said it could develop into "Europe's CIA".
The dossier says: "The EU's Joint Situation Centre (SitCen) was originally established in order to monitor and assess worldwide events and situations on a 24-hour basis with a focus on potential crisis regions, terrorism and WMD-proliferation.
"However, since 2005, SitCen has been used to share counter-terrorism information.
"An increased role for SitCen should be of concern since the body is shrouded in so much secrecy.
"The expansion of what is effectively the beginning of an EU 'secret service' raises fundamental questions of political oversight in the member states."
Superintendent Gerry Murray, of the PSNI, said the force's main role would be to test whether the system, which he said could be operated on a countrywide or European level, was a worthwhile tool for the police.
"A lot of it is very academic and very science-driven [at the moment]. Our budgets are shrinking, our human resources are shrinking and we are looking for IT technology that will help us five years down the line in reducing crime and combating criminal gangs," he said.
"Within this Project Indect there is an ethical board which will be looked at: is it permissible within the legislation of the country who may use it, who oversees it and is it human rights compliant."