Is Big Data a ‘big deal’ for modern companies?

0

The use of Big Data is slated to become the basis of competition for the modern capitalist machine and growth for individual firms, enhancing productivity as well as creating significant value for the world economy by reducing waste and increasing the quality of products and services.

The issue now, apart from smaller companies in Malaysia not utilising Big Data is that, even for those that do, companies are far better at talking about data than putting it to use.

With terms such as Petabyte and Exabyte being thrown around in the new business lexicon, it is no wonder most businesses in Malaysia are unable to take it seriously.

While the Internet buzzes with new millennial preach, most serious business people are focused on its partners, customers and employees making all the techie mumbo jumbo irrelevant.

Yet Big Data is important because it will transform how companies manage their enterprises and force managerial practices to evolve.

The evolution of the business sphere

A study estimated that by 2024, global enterprise servers would annually process the digital equivalent of a stack of books extending more than 4.37 light-years to Alpha Centauri.

Gartner analysts noted that, “Big Data is a way to preserve context that is missing in the refined structured data stores ― this means a balance between intentionally ‘dirty’ data and data cleaned from unnecessary digital exhaust, sampling or no sampling.

“A capability to combine multiple data sources creates new expectations for consistent quality, for example, to accurately account for differences in granularity, velocity of changes, lifespan, perishability and dependencies of participating datasets.

“Convergence of social, mobile, cloud and Big Data technologies presents new requirements ― getting the right information to the consumer quickly, ensuring reliability of external data you don’t have control over, validating the relationships among data elements, looking for data synergies and gaps, creating provenance of the data you provide to others, spotting skewed and biased data.”

Among the biggest advantages of Big Data to companies now are the new ways of linking datasets that are playing a large role in generating new insights.

In marketing, familiar uses of Big Data includes ‘recommendation engines’ like those used by companies such as Netflix and Amazon to make purchase suggestions based on the prior interests of one customer as compared to millions of others.

American credit-card companies have found unusual associations in the course of mining data to evaluate the risk of default: people who buy anti-scuff pads for their furniture, for example, are highly likely to make their payments.

In the public realm, there are all kinds of applications: allocating police resources by predicting where and when crimes are most likely to occur; finding associations between air quality and health; or using genomic analysis to speed the breeding of crops like rice for drought resistance.

Forbes noted that 87 per cent of enterprises believe Big Data analytics will redefine the competitive landscape of their industries within the next three years. Eighty-nine per cent believe that companies that do not adopt a Big Data analytics strategy in the next year risk losing market share and momentum.

Government agencies such as the Multimedia Development Corporation (MDeC) are looking to promote greater adoption of this technology locally, fully aware the impact of the adoption of such technology on business efficiency.

“Although the awareness of Big Data analytics (BDA) amongst Malaysian companies is already high, their readiness or willingness to actually adopt it is still relatively low,” Datuk Yasmin Mahmood, chief executive officer at MDeC pointed out in a media report.

In this year’s Big Data week, MDeC unveiled a plan to increase the number of local data scientists from the current 80 to 1,500 by the year 2020.

Media reports noted that from July 2015 onwards, the universities that will offer undergraduate and post-graduate data science and computer science courses with data and business analytics specialisations are Asia Pacific University (APU), Malaysia Multimedia University (MMU), International Islamic University Malaysia (IIUM), Sunway University, Monash University, University Institute Technology Mara (UiTM) and University Teknologi Petronas (UTP).

“There are currently 80 data scientists in our local data analytics industry. To meet our target of 1,500 data scientists by 2020, we need more support from the institutes of higher learning in Malaysia to offer courses that will encourage young people in the country to explore this high-demand career.

“We need to ensure a consistent and sustainable supply of skilled workforce to drive the adoption of BDA (Big Data analytics) technologies, cultivate a culture of innovation and essentially, build a BDA ecosystem that will impact all sectors of the Malaysian economy,” she said.

“Big Data is one of those technologies whose contribution is more than the sum of its parts. Whilst companies often associate Big Data with the amount of data they acquire, the different sources of data and speed at which data is coming at you, the true business value is from the insight that you can derive from the analysis of the data, and the competitive advantage it brings through differentiation,” said Ivan Teh, managing director of Fusionex.

“More companies are now excited about Big Data but would still require support and motivation,” added Yasmin.

“This year, we have around 40 BDA and related solutions from universities, local businesses and international tech companies showcased at Big Data Week. This excitement leads us to draw upon our competencies in the national BDA thought-leadership to enable entrepreneurs and businesses to start exploring the power of their data without having to invest in expensive consultancy or training.”

Advantage for Malaysian companies

The use of Big Data is becoming a crucial way for leading companies to outperform their peers. In most industries, established competitors and new entrants alike will leverage data-driven strategies to innovate, compete, and capture value.

Big Data will help to create new growth opportunities and entirely new categories of companies, such as those that aggregate and analyse industry data.

Many enterprises are investing the majority of their time in analysis (36 per cent) and just 13 per cent are using Big Data analytics to predict outcomes, and only 16 per cent using their analytics applications to optimise processes and strategies.

Moving beyond analysis to predictive analytics and optimisation is the upside potential the majority of the C-level respondents see as essential to staying competitive in their industries in the future.

Many of these will be companies that sit in the middle of large information flows where data about products and services, buyers and suppliers, consumer preferences and intent can be captured and analysed.

In addition to the sheer scale of Big Data, the real-time and high frequency nature of the data are also important. For example, ‘nowcasting,’ the ability to estimate metrics such as consumer confidence, immediately, something that previously could only be done retrospectively, is becoming more extensively used, adding considerable power to prediction.

Similarly, the high frequency of data allows users to test theories in near real-time and to a level never before possible.

Just like large enterprises, SMEs can use Big Data to better understand its customers, tap into new markets and cut out unnecessary costs across the business, all in real-time.

More adventurous SMEs are even tapping into the capabilities of advanced data scientists, without the budgets or capability to bring such resources in-house.

However, when companies start collecting and storing all the data available, which can quickly reach petabytes in volume, having the capital to invest in the associated hardware and software can create a real problem for mid-market firms.

Cloud computing, the ability to rent compute power from an external provider as and when you need it, may provide SMEs with the solution for the moment.

Consumers too could reap benefits from Big Data. Mobile applications such as Waze or Google Maps take smart routing using real-time traffic information, which is one of the most heavily-used applications of personal-location data.

As the penetration of smart phones increases, and free navigation applications are included in these devices, the use of smart routing is set to rise exponentially.

A new world of transparency

As information becomes more readily accessible, companies that have relied on proprietary data as a competitive asset are slowly being outpaced due to the sheer evolution of technology.

For example, the real-estate industry that trades on information irregularities such as controlled access to transaction data and tightly held information of the bid and asks behaviour of buyers. Acquiring both requires a significant expense and effort.

But in recent years, social media and the Internet allows almost anyone to bypass agents, permitting buyers and sellers to exchange perspectives on the value of properties and creating parallel sources for real-estate data.

Another swipe at such privately-owned information is the assembly by some companies of readily available satellite imagery that, when processed and analysed, contain clues about competitors’ physical facilities.

One big challenge is the fact that the mountains of data many companies are amassing often lurk in departmental ‘silos’ impeding timely exploitation and information hoarding within business units can also  be a problem. Often, this prevents these companies from forming a coherent view of individual customers or understanding links among financial markets.

Some manufacturers are attempting to pry open these departmental enclaves: they are integrating data from multiple systems, inviting collaboration among formerly walled-off functional units, and even seeking information from external suppliers and customers to co-create products. In advanced-manufacturing sectors such as automotive, for example, suppliers from around the world make thousands of components.

More integrated data platforms now allow companies and their supply chain partners to collaborate during the design phase―a crucial determinant of final manufacturing costs.

Competitive advantage for everyone

Big Data ushers in the possibility of a fundamentally different type of decision-making. Using controlled forecasting, corporations can test and analyse results to guide decisions and changes.

In effect, experimentation can help distinguish causation from correlation, thus reducing the variability of outcomes.

Online companies in particular are continuous testers.

Bill Lee, a partner at Ernst and Young told BizHive Weekly that, “An organisation typically has three tiers or types of data/information.  Tier-1 data are the transaction data; tier-2 data is generated from combining tier-1 data through analytics to create insight; and tier-3 data is generated through combining tier-2 data with Big Data, or data which is unstructured in nature collected from non-traditional sources (powerpoint, word, email or spreadsheet files and from the world wide web).

“Tier-3 insights or data is generated therefore from Big Data combined with an organisation’s internal structured data, and it can provide more accurate current or forecasted information. You can hardly or ever get 100 per cent accuracy from analytics but any confidence level plus minus five per cent from 9 per cent can be considered excellent.”

He explained that Big Data could provide a competitive advantage today because it provides potential insight, which is not possible from traditional means, and not many organisations have such capabilities today.

Once all or most organisations have this capability, Big Data will no longer offer a competitive advantage but a basic need for survival.

However, Lee also noted that for the moment; not all organisations need to access Big Data.

“It is very expensive to gather, store and use Big Data.  The use cases are also limited to analytics ‘matured’ organisations, which have huge transactions (millions a day) or need specialised insights, which can only be gleamed from Big Data sources like social media.

“It would make more sense for many organisations to master the ‘small data’, which they already own using predictive and prescriptive techniques to gleam insights.”

At the moment, there are no difference between the analytic techniques used for big or small data therefore the values created by big or small data are the same.

The objectives of analytics, big or small data, are to enhance revenue, optimise resources and manage risk more effectively.

They are productivity tools, which allow the organisation to enhance value from quantitative, or data driven methods.

Forecasting

In a previous interview, Datuk Fadilah Baharin, the director general for the Department of Statistics Malaysia explained that, “Information is critical in forecasting whether it be for business or government sectors citing example of a person losing their phone, that the loss of the phone is of secondary importance as compared to the information stored in it.”

Lee added to this noting that, current or traditional non-quantitative/data driven methods are basically hypothesis-driven reactive methods.  This means that a decision is made from watching what has happened and to rely on one’s experience with past events and decisions to make new decisions.

Forecasting or predictive method are data driven and because it provides a view of what may happen with a confidence level, it allows management to make an informed proactive decision to prevent an unwanted event from happening or to promote a preferred event.

Through this data driven method, it enhances the organisation’s ability to compete in a more efficient and effective manner.

Where such controlled experiments aren’t feasible, companies can use more natural experiments to identify the sources of variability in performance.

One government organisation, for instance, collected data on multiple groups of employees doing similar work at different sites.

Simply making the data available spurred lagging workers to improve their performance.

Adding to that, Lee explained, “The use of big or small data, predictive and prescriptive methods, acquisition of software and hardware and the building models to make it happen are all the ‘science’ or analytics.

“For analytics to work you need the art of analytics.  The art of analytics requires an organisation to prepare its structure, work flow, policies, skills, mindset and culture to effectively implement the analytics data driven requirements.

“So the use of big or small data in analytics actually requires the organisation to go through a transformation of the way it operates.

“It changes its sole reliance on experience and gut feeling or the decisions of its managers and employees to include what its data is saying about what the best decision may be.

An analytics-driven organisation is data driven, has the skills and the ability to create insights through data and makes decisions from the experience of its workforce, complemented and augmented by what its data is saying.

“It will speed up decisions, become more effective and efficient. It will enhance revenue acquisition, optimise the way resources are used and be more effective in the way risks are managed.”

Big Data enables a major step beyond what until recently was considered state of the art, by making real-time personalisation possible. A retailer can now be able to track the behavior of individual customers from Internet click streams, update their likings, and model their likely behavior on the spot.

These retailers will then be able to recognise when customers are nearing a purchase decision and nudge the transaction to completion by bundling preferred products, offered with reward program benefits.

So is Big Data really a big deal?

The era of Big Data could yield new management principles. In the early days of professionalised corporate management, leaders discovered that minimum efficient scale was a key determinant of competitive success.

Likewise, future competitive benefits are likely to accrue to companies that can not only capture more and better data but also use that data effectively at scale. We hope that by reflecting on such issues and the five questions that follow, executives will be better able to recognise how Big Data could upend assumptions behind their strategies, as well as the speed and scope of the change that’s now under way.

With Big Data and analytics generally considered the domain of the giant corporation, many SME business owners end up overlooking the numerous opportunities these tools can provide for small and medium-sized enterprises.

Unfortunately, such an oversight can result in SMEs losing out to competitors who are effectively using data as a means of improving performance and gaining new insights.

Although smaller companies typically have limited resources and smaller budgets, they do have other advantages: typically, a more flexible IT infrastructure, with fewer legacy system issues or disparate databases, and an ability to change practices quickly.

Another challenge facing SMEs is that even if they could access Big Data the way larger corporations do, they may not have the same capacity as larger players to ‘analyse’ the new data sets, or even to build up in house teams that can do this job.

This challenge will be particularly acute for established SMEs, which have been operating for a few years already.

Many of today’s new start-ups are born into a world of data, so they are created thinking about how they can use data. Indeed some businesses have created business models that are completely reliant on data.

The bottom line is: improved performance, better risk management, and the ability to unearth insights that would otherwise remain hidden. As the price of sensors, communications devices, and analytic software continues to fall, more and more companies will be joining this managerial revolution.

But for the moment, Lee said, “Don’t be swayed by the attractiveness of Big Data and drawn to investing time and effort to bring Big Data capabilities into your organization. “

He advised that most organization will already have tons of small data, which it has spent millions to gather and store. Analytics can be applied to these small data to very positive effect.

“It will be costly to get your small data ready and used through analytics but far less than Big Data and the benefits from small data will bring enough positive outcomes which can keep your company competitive for years to come.  So your focus should be big analytics and not so much Big Data.”

Companies must develop a ‘data culture’ where executives, employees and strategic partners are active participants in managing a meaningful data lifecycle.

Tomorrow’s successful companies will be equipped to harness new sources of information and take responsibility over accurate data creation and maintenance. This will enable businesses to turn data from information into business insights.