Big Data? What’s in it for Me?
Monday, January 30, 2017
Posted by: Darchevia Woods
Humayun Latif, VP Technology, Strategic Systems International
As Big Data becomes a pervasive term for the modern business, most decisions makers need a starting point to evaluate its suitability and application. Why does an organization need to consider a Big Data solution and does it help address specific challenges?
There are a handful of working definitions for Big Data, but it is most simply described as data sets so large and complex that become difficult or impossible to process using traditional database management applications. Big Data refers to a process that is used when traditional data mining and handling techniques cannot uncover the insights and meaning of the underlying data. Data that is unstructured or time sensitive or simply very large cannot be processed by relational database engines.
Big Data is not a single technology, technique or initiative. Rather, it is an approach towards organizing, storing, accessing and analyzing data across many areas of business and technology.
What factors should be evaluated when deciding whether a Big Data solution makes sense for your data and analytics needs? Big Data approaches are typically more suitable in the following situations: are preferred over traditional databases because they enable organizations to handle large data sets and generate useful information and insights from data with minimal delay time, unlike traditional databases.
Handling Large Volume Of Data: Organizations that need to handle large quantities of data as part of their daily operations may want to consider Big Data. Big Data technologies
Gaining Insight: Traditional database approaches are more suitable in deriving analytics from more structured types of data; data in which direct relationships between various entities are more easily visible and predictable. Organizations these days are collecting data from many different sources. This data is not only large in volume but differs in its nature from the types of data that has typically been part of enterprise databases. The foremost difference in today’s data is that it may not be very well structured, categorized or organized. Relationships or relevance between disparate sets of data may not be obvious. Establishing common patterns, links, correlations with this type of data and deriving intelligence out of it is easier with a Big Data approach.
Efficient Information Management: Vast repositories of data often contain useful and vital information that can be very valuable for business needs. Big Data implementations are typically more efficient in how they deal with data. This is achieved by one or more of the following approaches that are a part of most Big Data implementations:
· Distributed Storage
· Distributed Processing
· Simple algorithms (vs. complex queries)
· Improved and simpler scalability (as a result of the above) the help of distributed systems, where parts of the data is stored in different
The following considerations, also known as the 5V’s are also useful guidelines in determining the suitability of a Big Data initiative:
Volume refers to the vast amounts of data generated every second. With a Big Data solution, we can now store and use these data sets which include locations, brought together by software.
Velocity refers to the speed at which new data is generated and the speed at which data is distributed. Big Data allows us now to analyze the data while it is being generated, without ever putting it into databases.
Variety refers to the different types of data we can now use. Approximately, 80% of the world’s data is now unstructured, and therefore can’t easily be put into tables. With Big Data we can now harness different types of data (structured and unstructured) and bring them together with more traditional, structured data.
Veracity refers to the trustworthiness of the data. With many forms of data, quality and accuracy are less governable but Big Data now allows us to work with these types of data. The volumes often make up for the lack of quality or accuracy.
Value: It is great to have access to Big Data but unless we can turn it into tangible value its full potential will not be realized. So one can safely argue that ‘value’ is the most important V of Big Data. Big data lends itself nicely to data mining, data analytics and other more abstract data related initiatives where huge amounts of data that businesses collect can be put to good use and yield value and business opportunities.
Strategic Systems International (SSI) is a data and software engineering firm headquartered in Chicago with 25+ year experience in building applications for large enterprises and SAAS companies with an onshore/offshore delivery model. We are a team of quants and technologists that seek to solve complex problems through simple technology and data-enabled solutions.