Search Technology Gets Smarter

By Maria Trombly | Securities Industry News | April 3, 2006

As the volume of information coursing through securities firms continues to expand, so does the need for search engines capable of anticipating and producing more relevant, targeted results than those of currently available software. With an assist from artificial intelligence that delves deeper than would a conventional keyword search, technology developers are getting closer to that ideal.

These innovations are driven to a large degree by compliance needs, according to Susan Feldman, an analyst at International Data Corp. (IDC) in Framingham, Mass. "IDC believes that the next two years will be a time of ferment, change, and growth in the information-access software and solutions markets," she says. This year she sees the arrival of advanced language analysis tools that will take searching to new levels.

The emerging search technologies could permit users to specify information they want on, say, companies that issue more than $1 billion in debt, or bonds yielding more than 5 percent, or a certain type of unsecured debt. IBM Corp. recently worked on a project that involved searching Securities and Exchange Commission filings for specific information within blocks of text.

"We are leveraging analytics to identify any references to debt," explains Marc Andrews, IBM's director of strategy and business development for content discovery. "A lot of it is in paragraph form and not in any structured format. Our system will go in and tag references to debt, extract from those references the type of debt, and identify the dollar amount indicated and the maturity. It's all about being able to extract knowledge from unstructured information and make it available to multiple business processes and applications."

IBM's technology includes clustering, which groups search results according to content or by categories. Andrews says the search functions at sites such as nasdaq.com, tdwaterhouse.com and eluxury.com are publicly available examples of the IBM technology at work.

Another search engine with artificial intelligence was rolled out in March by Jersey City, N.J.-based Accoona Corp. It is a version of the Google News search with technology from Fast Search & Transfer (Fast), an Oslo-based company with more than 3,500 installations worldwide. The Chinese government is one of the backers of privately held Accoona, which also has a Chinese-language search. The company offers a free business-information search function, with data from Dun & Bradstreet and Accoona's own database of business information, which covers more than 60 million businesses worldwide.

Accoona CEO Stuart Kauder says it's important to understand the meaning behind phrases in order for clustering to work. This can prevent synonyms or alternate spellings from ending up in separate clusters. "Our artificial intelligence can understand the meaning of words," he says. "We have 100 separate algorithms to determine what a specific term may mean."

According to Andrew McKay, Fast's vice president of marketing, the company's search technology can understand the meaning of words in 77 languages, and it has partnerships with major technology integrators such as Accenture. Wall Street firms already use Fast's technology for compliance, risk management, business intelligent, data cleansing and collaboration, he says.

As a compliance tool, the technology can generate alerts about suspicious activity based on contextual analysis and behavior patterns--not just a standard keyword alert. And research analysts can use automatic background searching to generate alerts based on company news events.

"One of our financial customers in Europe monitors a bunch of trends and gets notified instantly when something gets triggered," says McKay. "This includes a qualitative search of company sites, news, blogs--quantitative and qualitative analysis at the same time. We can even do sentiment analysis, of whether people like or hate things." Such searches can scan both structured and unstructured content. "The artificial barrier between intranet and extranet content goes away," he says.

There is another application of intelligent alerting: bringing relevant documents to people's attention before they specifically ask for them.

But who has time to sort through search results? Artificial intelligence can do it for you by reading through the materials, figuring out what they're about, and then organizing the results into clusters.

Another approach involves dynamic clustering. If you search for "Smith" are you looking for the college? The person? The company? The profession? An intelligent search engine would be able to identify the major clusters that the results fall into and allow you to quickly narrow your search.

The clusters can be predefined based on established taxonomies. For example, source materials can be classified by content, and a search can be narrowed down to a particular document type or category.

Factiva, the joint news-database venture of Dow Jones & Co. and Reuters Group, launched a version of intelligent clustering technology in January. Factiva uses a pre-set taxonomy that covers more than 640 different subjects, more than 860 industry categories, 1.5 million companies and 22 languages.

"We have more than 300 million documents going back 30 years," says Dennis Cahill, Factiva's VP of products. Over the last five years, the company has been investing heavily in intelligent searching, he says.

Dynamic clustering is also available in an electronic document-discovery product that EMC Corp. of Hopkinton, Mass. released in February. "One of the frustrating things that our users are complaining about is relevance," says Lubor Ptacek, EMC's director of product marketing. "You get 50,000 hits with every Google search, but most of it is not very relevant."

He adds: "There is a huge focus right now in the corporate world on the topic of e-discovery, often combined with legal challenges. Doing it manually costs just an amazing amount of money." Dynamic clustering, he says, puts powerful search tools in the hands of non-specialists.

"All of a sudden, you don't have techies, you have lawyers" making effective use of the systems, he explains. "They speak lawyerese; the technology needs to be simple so that they can use it, and also provide some of the guidance a lawyer might need."