The amount of data flowing through and around modern enterprises is growing exponentially, but it can present significant challenges to those focused on success.
Without having some way to organize the vast amount of data that often surrounds a business presence, it is difficult for decision makers to "know what the data knows," and further, how they can use that information advantageously.
Fortunately, great advancements in machine learning and data science have yielded a massive opportunity for enterprises of all sizes to more efficiently and effectively gain a better sense of their data (and all of the processes, protocols and workflows that surround it) and gain a more comprehensive, truly unified view of performance (on both a historical and predictive basis) that can serve as their guide to achieving business objectives.
Just how big is the opportunity? McKinsey Global Institute's 2017 report, "Big data: The next frontier for innovation, competition, and productivity," indicated that big data can create as much as $700 billion in value to consumer and business end-users. Reaching that level will require a sufficient investment in the right technologies and infrastructure as well as the right personnel.
In this month's
Website Magazine feature article, learn why big data is so challenging, a way to put it in context and the considerations that must be made to ensure success with such initiatives from a technical perspective.
The Big Data Challenge
What enterprises are really after with big data efforts is intelligence. The challenge, of course, is analyzing the many forms of data (from Web analytics to data mining to event processing) in a way that it can be used to truly improve a company's services or the investments they have made-both time and financial.
For example, data intelligence is often used to analyze the internal data of a company's workforce or operations in order to make better hiring - and firing - decisions in the future. It is also used to deliver personalized user experiences, optimize the lifetime value of consumers, measure partnership and integration effectiveness and ultimately build a richer understanding of performance on nearly every business front.
What is holding back most organizations is inadequate analytical and technical know-how (as well as data privacy and security issues) and these are proving to be significant challenges for businesses both large and small. A formal education and extensive experience in the practical big data arts and sciences is the only way enterprises can ensure their future success, but in the interim, it is essential to understand the value that such initiatives can offer enterprises willing to make the investment.
Data in Context
Big data is often discussed in relation to specific challenges and opportunities for a business and often in the context of what are known as the five V's: value, variability/variety, velocity, volume and veracity. Let's explore how each of these aspects may impact an enterprise now and in the future.
Value: From the perspective of a business, it is the value (often the financial value) of big data efforts that captures attention at the outset of an initiative of this nature. The value comes from insight discovery and pattern recognition that leads to more effective operations, stronger (and more profitable) customer relationships and other quantifiable business results. Having the right software and personnel in place helps businesses extract the value they seek.
Variety/Variability: Data is not static. Its dynamic nature (i.e., growing, shifting and shrinking) requires that companies continually seek to capture, manage and analyze. In sentiment analysis, for example, that might be shifts in the meaning of words or phrases. In cluster analysis, it may be grouping various behaviors to customers (e.g., everyone that has ever submitted a support ticket, but abandoned their mobile shopping cart). In order to gain the best possible view into a business, there needs to be a range of different data types. That might include unstructured data, semi-structured data and raw data. Variety in data can also mean where it comes from and who is providing it so internal and external data (discussed below) are frequent considerations.
Velocity: There is an immense amount of data in the digital world and it often flows freely and often quite quickly. The speed at which companies receive, store and manage their data is vital to decision making so having a reliable infrastructure and an appropriate technical framework is another key to extracting the most value from big data efforts.
Volume: The amount of data also presents opportunities and challenges in the context of a business. Too little volume leads to inaccuracies while too much can be cumbersome and overwhelming. The upside of course is that more data can mean greater efficiency and thanks to significant advancements in all sorts of big data technologies it is becoming reality.
Veracity: In order to have confidence in data, it needs to be accurate too. Data can be dependent upon the viewer so there cannot be any question as to its "truthiness" and it is more important than ever. That is a lot to take into account when considering a big data initiative but a deeper understanding of these factors and variables will yield more useful conversations about how to use the information available to guide and improve decision making within any enterprise. To realize the benefits of big data however, there needs to be action.
Data Monetization
Data is at the center of the modern digital enterprise. One of the keys to success is the ability to generate additional revenue streams by using data effectively.
Discover the best ways to monetize the value of your data today.
Data in Action
Having access to data and being able to analyze it does not really mean much if enterprises are not able to profit from their investment. Companies that use big data analytics to influence the decisions they make in regard to their business typically experience incredible improvements to both their financial performance and ability to make decisions faster than the competition. That is why it is so important to at least understand how a broader big data initiative works.
Determine Customer Demands, Needs, Wants & Aspirations
Only by profiling target customers, verticals and their own parameters for success are enterprises able to capitalize on their data stores in an efficient and effective manner. This stage of big data initiatives demands a meticulous surveying of consumers, influencers and stakeholders and will be the guiding force in their success.
Aggregate Data Assets
Success with big data (or any data for that matter) starts with identifying data that differentiates, exploring use cases to solve and providing insights that matter to those that can benefit directly from its access. The information can be raw and unrefined in one data silo, but integrated with another and it could become a powerful new insight. Say for example that a retailer is providing their transaction data report to a shipping optimization solution. If that optimized data does not find its way back in to the retailer's data store, it cannot be used to optimize other processes. Data on package delivery notifications, for example, could be used for retargeting initiatives with these customers, or in aggregate with some other data like package failure rates.
What Type of Data are We Talking About?
There is internal and external data and there is a thin but distinct difference between the two. Internal data, for example, is the data provided through consumer interaction with a digital presence (users that visited/opened its website or application or used the customer support system), or perhaps accounting information like billing/budgeting data or sales activity logs.
External data sources, however, are different. Based on a collected email from a lead generation partner, for example, marketers might then collect a personal email address. That address can be associated with other similar profiles and should there be a match, it is possible to start building a profile of a user, their location, industry, related emails and domains, their company Web presence, how many employees are within that enterprise and perhaps even social media activity. What aggregating and analyzing all this data provides is an opportunity to develop information-based business decisions.
In addition, internal data is collected from an enterprise's own customer relationship management (CRM), marketing automation, support and financial systems. External data, on the other hand, is gathered from extensive Web crawls using highly advanced dynamic algorithms, public data repositories and even manual collection.
Data Integration
Data science platforms serve as a centralized integration instance, where the work of data scientists, engineers and developers come together and can be streamlined and shared.
What makes for the ideal platform? Find out in
Website Magazine's "
Data Integration Roundup", profiling the top providers and the essential steps to take to ensure integration is flawless when it comes to the data, systems, technologies, security and personnel that should be in place.
Technical Data Development
For enterprises ready and willing to jump into these initiatives, not only will they need to understand their own data-related business objectives and plan how they will use that data in the future, but they will also need to design and build an underlying technical environment and infrastructure - one that is ideally low cost and low complexity - that is stable, scalable and provides access to the necessary user toolsets to discover insights, make decisions and solve real-world problems.
Make no mistake - the technical aspect of big data is complicated. It is for this reason that data specialists are some of the highest paid business professionals in the world. Their familiarity and experience with analytics and computer science are being rewarded because they have developed the skills now in demand. What are those skills? How can they be developed? And what resources are available?
In addition to required levels of intellectual curiosity, business acumen and communication skills, those succeeding in big data projects tend to have the right education, but also (and perhaps more importantly) knowledge of tools such as SAS and/or R - as well as actual programming.
Proficiency in common coding languages including Python, Java, Perl or C/C++ should be expected in big data projects as well. Computer science experience of this nature (including the ability to write and execute often complex queries) is essential, but so are the tools an analyst uses.
One of the most common names heard in big data-related conversations is that of Apache Hadoop, an open source software platform for distributed storage and distributed processing of data sets in the cloud. Hadoop services also provide data access, data governance, security, and operations support, which is why expertise in these areas is so critical to the success of businesses today.
Hadoop is one the most recognizable names in the world of big data, but it is far from the only software/solution provider that is influencing how today's businesses manage and use their data (see sidebar).
Big Data in Motion
There is big potential in big data when the right vision, the right strategy and the right personnel and technology are in place. Let this big data starter pack from
Website Magazine, and the resources included within it, serve as a starting point as you seek out to find what your data wants you to know.
MANAGE AND USE YOUR DATA
Review many of the tools used by the
Website Magazine community in their own big data and analytics initiatives on the ‘Net.