|
Mining Text for Meaning |
|
|
|
Tuesday, 29 December 2009 18:13 |
|
|
Sources report that more than 80% of the world’s data is in text form. Many companies, in order to grow, need to find meaning in this textual information, and research into text analysis has been ongoing for several decades.
To mine meaning from text, it is first structured: Each sentence is grammatically analyzed for parts of speech and syntactic relationships. Different analyses have different goals. The following list is not inclusive:
- Clustering text
- Categorizing text
- Extracting concepts
- Analyzing the sentiment (i.e., determining the tone of the writing)
- Summarizing
- Deciphering relationships among elements within the text
Existing text mining systems allow for methods that are statistic and probabilistic in nature. While these approaches have proven useful, they have inherent limits that do not provide the sophisticated understanding of the text that a human analyst provides.
Current research into text analysis aims to move beyond statistical and probabilistic relationships to provide an understanding of how humans process, generate, and comprehend language, thereby allowing for more symbolic approaches, approaches that “understand” relationships between ideas described in paragraphs.
In the near future, more complex search queries—queries based on ideas and themes rather than key words alone—will yield more focused information retrieval. Other benefits of advanced text analysis approaches are applicable to:
- Business intelligence
- Automated translation
- Information sharing
- Criminal investigation
We at AGS Analytics are close to rolling out the next major revision of our text analysis system. We’d like to speak with you about your company’s needs now so that we can tailor our software to your goals. Please call or e-mail us. Initial consultation is free—our intrinsic energy is a bonus. |
|
Last Updated on Tuesday, 29 December 2009 20:34 |
|
Importance of Data Accuracy |
|
|
|
Saturday, 07 November 2009 16:33 |
|
|
Managing data and parsing useful information can be quite daunting when faced with the potential for inaccurate models and erroneous results. It is important to work with standardized rules that help develop the best possible models to better manage data in a meaningful way. Data forms are as numerous as the transactions they represent. Data can be a list of numbers, text characters, locations, product categories, a specific time, financial histories, travel patterns, or any other figure used to represent an event. For each of these data points, Data Mining can be successfully employed to uncover existing patterns of behaviors that provide insight and offer predictable indicators for future transactions. Asking the right questions, however, is the key to discovering meaningful results that provide a critical level of guidance from the data itself.
Valid results are the key to useful analysis techniques that help provide a clear picture of future behavior. Random errors can skew results in unpredictable ways, altering the final analysis and cause significant problems in predictability patterns. In any set of complex data, there are values that are found outside the normal criteria that, if not identified, can easily exaggerate results and provide useless and sometimes damaging results. For this reason, cross-validation and standardized data-management techniques are employed.
Accurate measures begin with a unified approach to data management. Specific standards and practices for data mining professionals have been promoted at professional organizations such as the Association for Computing Machinery's special interest group, Knowledge Discovery in Data, and the IEEE Computer Society's Technical Committee on Data Engineering that publishes "IEEE Transactions on Knowledge and Data Engineering." Today, professional data- mining specialists must understand the rigorous criteria and methods these groups have identified, in order to successfully bridge the gap between useful analysis and anecdotal summaries. Specific standards govern classic analysis techniques of general statistics clustering through to next-generation techniques such as decision trees (Six Sigma), regression analysis, and network frameworks. For this reason, it is vitally important to partner with knowledgeable data-mining professionals dedicated to accurate, meaningful analysis tailored to your specific needs. |
|
Last Updated on Sunday, 08 November 2009 16:28 |
|
|
What’s So Difficult about Data Mining? |
|
|
|
Sunday, 01 November 2009 20:21 |
|
|
Given a spreadsheet, anyone with a B.S. in statistics can pick out an apparent pattern in data. But the truth is that, by themselves, obvious patterns in data may be irrelevant to a company’s goals. It takes an experienced analyst to determine if that pattern is actually useful to predicting future trends that can help your company meet its bottom line.
With a combined experience of more than four decades, we at AGS Analytics have developed software systems for mining data and finding trends that organizations have used to further their research or to save or earn themselves money:
- We worked with federal government scientists to convert and combine large text databases and to extract semantic information from patterns in the data in order to test and prove psychological theories of human language processing.
- We helped the Air Force identify skills among its workforce that could be improved through training. Then we developed the methodology and software to find the appropriate training programs at the lowest cost.
- Working with investment managers, we identified investment strategies that worked best in specific market and economic circumstances. Our results showed which methods to use during specific market conditions—and which to avoid.
Data mining’s use is not limited to federal agencies and financial institutions as in the above examples. Its abilities can benefit organizations involved with sales, marketing, research, customer relationship management, law enforcement, and human resources. We can show you how specifically.
We’re “number” people, and we love what we do. Please call to discuss your needs with one of our analysts. Initial consultation is free—our intrinsic energy is a bonus. |
|
Last Updated on Sunday, 08 November 2009 16:28 |
|
What is Data Mining and How Can It Benefit My Company? |
|
|
|
Sunday, 01 November 2009 15:30 |
|
|
Data mining reveals hidden gems among seemingly incomprehensible amounts of data. What makes data mining special is that it can be used to predict future behavior rather than focus on past events. Last April stores in Dallas may have sold out of your umbrellas, but can you expect the same spate of sales this year? Did stores display the umbrellas near the galoshes? Did they drop the price, distribute coupons or promote a special on rain gear? Did the increase in sales of backpacks (or any other item) contribute to the umbrella blowout? Data mining professionals and systems analyze the relationships among relevant patterns in data—sometimes seemingly unrelated data—to predict future trends.
Data mining has benefited not only companies with products to sell, but medical researchers with vaccines to develop, law enforcement with lives to protect, engineers with highways to build, even professional sports teams with games to win. Data mining reveals patterns in numbers, yes, but data mining tools can dig through locations, product categories, addresses, transactions, even straight text like in newspapers. We at AGS Analytics have developed innovative software that makes sense of the data for your business. We are pros in the myriad analysis methods and can help you apply the results to improve your bottom line.
Please call or e-mail to discuss your needs or to ask our experts about how data mining can benefit your organization. Initial consultation is free—our intrinsic energy is a bonus. |
|
Last Updated on Sunday, 08 November 2009 16:28 |
|