The Analysis Challenge
Data Overload
Given the high payback from preventing upsets and improving processes, what is blocking greater success? Simply put: data overload.
Producing, collecting, transmitting and storing vast amounts of data gets easier every day. Technological advances in remote sensing, wireless transmission, and data storage have led to unprecedented capacity for data capture and archival.
But these technological advances have not necessarily improved our ability to extract useful information from this data. Today's large, complex industrial operations generate millions of potentially error-ridden data records every hour from scores of devices in many formats from many locations. And it all must be summarized, organized and analyzed to convert it into something of value. But when faced with terabytes of data, even the most straightforward analysis can seem overwhelming, even impossible.
In fact, the challenges are so great and so overwhelming, that most organizations simply don't do analysis! They rely solely on experience and the lessons learned from previous observations to react to current operating conditions. But this approach is only as good as the experience of the observer and restricts understanding to local domain knowledge. Conditions outside the area of domain expertise may have a profound impact.
