Industry & Business

By definition, practitioners in multidisciplinary fields such as bioinformatics need to keep abreast of the latest research trends in several areas. However, the relevance of research work focused in areas outside their normal specialty may not be immediately obvious to all observers.

For instance, a multi-million dollar, 4-year research project on statistical methods for data mining, called Euredit and funded by the European Union, is creating interesting possibilities for many bioinformatics endeavors, including microarray analysis and forecasting trends.

Euredit’s goal was to analyze census surveys. Due to human nature, these surveys usually contain missing or incorrect data. European governments funded research to determine and improve statistical techniques that can be used to clean this data by filling in gaps or highlighting errors. This research was carried out by government national statistics offices, universities, and research-based companies. The cleaned data produced by this research was then used to better estimate the need for services and programs in communities. The result of this intensive research study was a number of new algorithms relevant to disciplines where data mining is important, with bioinformatics perhaps one of the top beneficiaries. Read the rest of this entry »

Increasingly, sophisticated methods are available for analyzing financial data and helping decision makers. Many of these methods have been described in articles that have appeared in Financial Engineering News. But in practice the data that will be used by these methods can be full of errors; it is dirty data. And it is often the more sophisticated methods that are most affected by dirty data, time series and variance models, such as GARCH, seem to be particularly sensitive to the presence of bad values in the data. While it is sometimes possible to use robust techniques that are less sensitive to bad observations, for example, using a median instead of a mean, it makes sense to deal with the bad data before the modeling takes place. Improve the quality of the data and you are very likely to improve the quality of the results. So before using the data it should be cleaned, that is, as many of the errors in the data as possible are corrected.

Here we will look specifically at cleaning numerical information, but the need for data cleaning is perhaps more important in text information where problems with misspelling and use of different abbreviations etc. complicate the process further.

What can be wrong with the data? There is a hierarchy of problems that are encountered:

1. No values have been input

2. Impossible values have been input

3. Inconsistent values have been input

4. Unlikely values have been input

While the first seems straightforward, there needs to be a distinction made between structural missing and observational missing. Structural missing will relate to values you would not expect to be there, for instance, share price changes will not be available when stock markets are closed at weekends or holidays. The models used need to be able to cope with such values, inventing values to fill such gaps is not a good way to proceed. On the other hand, observational missing are just values that have gone astray. Clearly where possible they should be looked for, but this may either be impossible or just too expensive. Read the rest of this entry »

Nearly everyone has heard or read a story lately regarding the climate of the real estate market. It is important to remember that real estate is locally driven, sometimes differentiated all the way down to a specific or unique neighborhood. Statistics used to describe the state of the national real estate market do not bear the same weight or the same meaning on many local markets.

The best source for statistics about current home prices and trends in any region is the local Multiple Listing Service, or MLS. A Multiple Listing Service is a cooperative of real estate brokerage firms. The MLS maintains a robust database containing a consolidation of property listings submitted by member brokerages over many years. It is available to all member brokers and their agents and provides a fast, effective way to list, view and market properties and conduct market analysis, benefiting both the buying and selling public as a whole.

Market statistics provided by the MLS are explained through the use of terminology, such as median, mean, and percentage change. Here is a look at some simple and easy to understand definitions of these terms. A median, the most frequently used and cited statistical measure in real estate, is simply the middle value of a set of numbers. In other words, it is the exact point at which half of the values are higher and half are lower. With home prices, the median is the exact sales price where 50 percent of the homes sold for less than the reference price and 50 percent sold for more. A median is especially relevant when discussing home prices, because it is not influenced by extremely high or extremely low prices. Read the rest of this entry »

Automation or Industrial Automation  is the use of computers to control industrial machinery and processes, replacing human operators. It is a step beyond mechanization, where human operators are provided with machinery to help them in their jobs. The most visible part of automation can be said to be industrial robotics. Some advantages are repeatability, tighter quality control, waste reduction, integration with business systems, increased productivity and reduction of labour. Some disadvantages are high initial costs and increased dependence on maintenance.

By the middle of the 20th century, automation had existed for many years on a small scale, using mechanical devices to automate the production of simply shaped items. However the concept only became truly practical with the addition of the computer, whose flexibility allowed it to drive almost any sort of task. Computers with the required combination of power, price, and size first started to appear in the 1960s, and since then have taken over the vast majority of assembly line tasks (some food production/inspection being a notable exception).

In most cases specialised hardened computers referred to as PLCs (Programmable Logic Controllers) are used to synchronize the flow of inputs from sensors and events with the flow of outputs to actuators and events. This leads to precisely controlled actions that permit a tight control of the process or machine. Read the rest of this entry »