NYT: NSA's info snarfing had to be reduced because the data was overwhelming

This is something I've remarked on in comments previously. Its possible to have too much information if the retention selection criteria are too loose.

... In a world of ballooning communications, the agency is sometimes simply overwhelmed. In 2008, the N.S.A.’s Middle East and North Africa group set about updating its Sigint collection capabilities. The “ambitious scrub” of selectors — essentially search terms — cut the number of terms automatically searched from 21,177 to 7,795 and the number of messages added to the agency’s Pinwale database from 850,000 a day to 450,000 a day.

The reduction in volume was treated as a major achievement, opening the way for new collection on Iranian leadership and Saudi and Syrian diplomats, the report said...

***Put stuff here about combinatorial explosion
***Put stuff here about curse of dimensionality

posted by Purp at 11:01 AM

| Access Comments