This article is part of the MITB Thought Leadership Series published by the SMU School of Information Systems’ Master of IT in Business programme.
By Johnson Poh, Adjunct Faculty, MITB (Analytics), SMU School of Information Systems
Technology has turned the tables in favour of consumers, enabling them to find goods and services faster and access more choices. Companies now compete more intensely to capture consumers’ mindshare and scour for ways to keep their products relevant. But every coin has two sides. While technology has empowered consumers with choice, it has also offered companies a plethora of data to understand consumers better. This puts the odds in favour of companies that can leverage on data to gain consumer insights and meet their business objectives.
Companies that have honed their data craft have become industry gamechangers. Amazon, Ebay, Lazada have transformed the retail industry through personalised product recommendation and customised consumer content because of their data-focused foundation, which has supported their business strategies. Similarly, Netflix’s data-driven management has led the paradigm shift in the entertainment industry through on-demand video streaming and predicting content that viewers want to watch. Data-driven companies such as these are better prepared for surviving the next decade in the new economy.
Naturally, the technology sector has been quick at developing cutting-edge data capabilities and implementing them in an array of industries with much success, either by themselves or with strategic partners. Any company that understands how to leverage on data capabilities can develop a strong data-ready culture, regardless of industry. Big data processing frameworks, coupled with advances in cloud computing technologies, mean that implementing big data solutions have become more convenient and affordable.
While big data can be used to gain insights more efficiently for swifter business decisions, getting companies to adopt a data-driven approach is fraught with inertia. Starting out can be daunting. How do you structure an effective big data team to deliver on business objectives? How much of an investment do you make on infrastructure and manpower? Transiting into a data-ready culture does not have to be a major overhaul of organisation structure, but a well thought out roadmap with long-term objectives is necessary to ease the transition.
An understanding of three key areas is required to guide companies in developing their roadmap.
- Designing a data repository;
- Investing in the right data architecture;
- Building an effective big data team in tune with business objectives.
The repository
Companies should not be afraid to start small by using data they already have to experiment and create a feedback loop. As you experiment, the relevance of new data fields may emerge that you may wish to start collecting. Some questions that can guide your data design and construct are:
- What data do I need to meet my business objectives?
- What existing data do I have?
- What data do I not have, but would like to start collecting?
- Is there data that is difficult to collect but I need to find proxies to?
Remember, companies that refine their data repository through continuous experimentation will reap the benefits of drawing deeper insights.
The architecture
Having a good understanding of data software in the market ensures that companies only spend on what is necessary. Software offers a range of functions from managing big data processes, predictive and advanced analytics to visualisation capabilities. Some companies need the full suite of data products, while others simply need software in modular form. Getting the right data capabilities to meet your business needs keep the cost on your balance sheet at bay.
For example, companies just starting out may want to focus on developing a data science workbench and take on a modular approach in developing a data architecture. Using open-sourced frameworks such as Hadoop and Spark are affordable and low cost for fulfilling their data analysis needs.
As companies advance along their data-ready roadmap, they may find the need to scale up in capacity and complexity. When business strategies evolve from pure analysis to predictive modelling, growing data repositories require larger storage and advanced data analysis for more efficient sifting of insights. These companies may find the need to customise software and may go a step further to invest in their own R&D and design data applications that improve their company’s workflow and productivity.
The team
Apart from a robust data repository and architecture, choosing the right people to lead the effort is important. Having a combination of technical know-how and the business objectives is essential. An effective big data team should be led by a business leader, who is grounded in technical concepts with hands-on experience in implementation.
In a big data team, getting the right composition of data engineers, data scientists and software developers is important. For instance, companies at the onset of their data roadmap will require more data engineers, who are implementers and managers of big data platforms and repositories. As companies advance along their roadmap, more data scientists will be required to build data analytics and predictive algorithms. Finally, for a data-driven culture to permeate throughout the whole organisation, software engineers are necessary for developing front-end dashboards and data visualisation tools that aid business product managers in their day-to-day decision making and operations.
At the minimum, an effective big data team needs to be proficiently trained in fields such as mathematics, statistics and computing. While this is a given, the integration of big data science teams within organisations also demands that every team member has a strong appreciation of the business objectives and the ability to communicate with key stakeholders, instead of acting in silo. A commitment to securing stakeholder buy-in is part and parcel of a well-integrated data science team in the organisation. Consistent engagement with the company’s IT offices, business owners, and perhaps more importantly, the company’s consumers are key to achieving success with a big data strategy.
Across industries, a growing number of companies have started to build their own big data teams in-house. As companies prepare their data roadmap for the future and instil a data-ready culture within their organisation, it is integral for business leaders to pay attention to people, processes and technology in equal measure to realise the promise of big data.
