Cloud Technology
Big Data and Cloud Computing – The Way Forward
Creating value is a business imperative. In today’s digital age, leveraging the benefits of big data and cloud computing can help organisations derive value from better, smarter, real-time decisions.
Cloud computing may very well be the solution that corporations need when tackling issues of scalability and flexibility of data.
Rapid technological advances have emerged as a notable feature in driving important business decisions. In the digital age, it is about how human beings are applying technologies in new and innovative ways. Key to this is data, which has unprecedented volume, speed and global reach, where people and things are connected more than ever in today’s world. With improving analytical methods, machines will be able to learn from the data collected, allowing for meaningful insights to be supplied to organisations to make better decisions.
Big data is a large volume of data that cannot be sufficiently managed by traditional and relational technologies. It can be non-relational, unstructured or structured data streaming in from diverse sources at a massive scale. By processing large volumes of diverse data quickly and reliably, big data has evolved from merely collecting and reporting data to generating actionable insights. The transformation of big data starts from the collection of the right data sets, followed by the creation of a data lake, and finally, analysis to derive insights and solve business problems.
The global big data market has grown rapidly from US$8 billion in 2011 to US$35 billion in 2017 and it is expected to reach US$103 billion by 20271. This growth has been facilitated by declining technological costs and an improvement in the analytics talent pool. In Malaysia, the journey towards becoming a digital nation with the adoption of a digitally-integrated lifestyle has received positive support, with significant investments by venture capital funds in the life sciences, information and communications technology (ICT), and manufacturing sectors, increasing from US$0.78 billion in 2006 to US$1.5 billion in 20162.
Risk management is crucial in an organisation’s framework to ensure a stable, secure and trusted environment when it comes to data management and adoption of cloud technology.
Convergence of Big Data and Cloud Computing
Big data has a set of characteristics, often defined by the five “V”s:
- Variety – Harnessing different types of unstructured and structured data
- Veracity – Acquiring data from trustworthy and credible sources
- Value – Leveraging data and insights to generate value
- Volume – Processing high volumes of data
- Velocity – Collecting and processing data on a real-time basis
This set of characteristics offers substantial benefits to organisations who are willing to take advantage of it. Among the benefits of big data are improving design and development, optimising supply chain management, streamlining operations and enhancing the consumer experience.
While there is no doubt that big data has brought about considerable benefits to organisations, the challenges lie in data storage due to a lack of infrastructure, and data management due to multiple sources and formats. Moreover, the growth of data dramatically increases the challenges of scalability and flexibility. To address these challenges, cloud computing may be a fitting solution.
Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources that can be rapidly provisioned and released with minimal management effort or service provider interaction. Cloud has transformed the way organisations manage scalability, resiliency and cost efficiency, so organisations should consider leveraging the cloud when using big data.
Cloud computing will enable omnichannel access and services can be made available anywhere.
What Do the Combination of Big Data and Cloud Computing Offer?
If an organisation is to succeed in the digital age, big data alone is not sufficient; hence, the calls for the adoption of cloud computing in order to be more impactful. One benefit of adopting cloud is the availability of a simplified, flexible and scalable infrastructure that allows organisations to be nimble and ready for changing consumer expectations.
The enormous volume of data that grows by the day raises the need for a significant commitment of hardware and processing resources. Organisations could leverage cloud instead of investing in physical infrastructure to accommodate peaks in data traffic and storage. Cloud is a flexible infrastructure that can be scaled according to needs, which makes it easier and more convenient to manage workloads with minimal management. Hence, cloud computing is a perfect partner for big data.
Big data and cloud computing deliver value to organisations by reducing cost. There is no upfront capital expenditure since cloud computing is a pay-as-you-use solution, allowing end-users to pay only for what they use. This solution reduces capital expenditure (CapEx) and moves the cost to operating expenses (OpEx). Cloud enables organisations to drive costs down without the need to invest heavily in setting up and maintaining hardware infrastructure.
In the digital age, digital media are increasingly making an abundance of information available to anyone; and with the convergence of web and mobile, increasingly more consumers are using smart devices to access sites and services. Organisations can, therefore, use cloud computing to improve access (omnichannel access) and allow services to be available anywhere.
Cloud adoption has been on the rise, with a growth from 93% in 2015 to 96% in 2018, as found in a 2018 survey by RightScale3. With cloud computing technology on the rise, organisations can now leverage the cloud to perform complex analysis of data, allowing organisations to benefit from the integration of data computing services and a cloud platform.
“If an organisation is to succeed in the digital age, big data alone is not sufficient; hence, the calls for the adoption of cloud computing in order to be more impactful."
Organisations need to employ the right measures to prevent data leakage and loss.
Implementing a Framework is Key to Overcoming Challenges
Despite the advantages offered by the combination of big data and cloud computing, it also creates a new unfamiliar platform that raises some concerns and that needs to be tested.
The fundamental challenge is security and control in the cloud environment. In the cloud, organisations face numerous security and data privacy challenges related to identity and access management (IAM), multi-tenant architectures, service availability (malicious insider, external threats to service), data loss and leakage, data privacy issues and security governance. Big data deployments in the cloud require the deployment of new security tools and platforms (authentication, access control, encryption, event logging and monitoring).
Another key challenge is the loss of control and performance when adopting cloud. The control ownership varies depending on agreements between cloud providers and organisations. Since organisations lack the visibility of the underlying computing resources, organisations face risks in meeting Service Level Agreements (SLAs) or availability requirements. Organisations and cloud providers need to work together to define SLA specifications to ensure that quality, availability, reliability and performance are up to the organisations’ expectations. Moreover, the type of cloud an organisation chooses matters. Not all cloud service providers offer the same level of controls and trust levels. A more rigorous control environment is required for mission-critical applications, infrastructure and platforms.
To overcome these challenges and effectively move ahead with their business goals, organisations should have a framework to enable a secure and trusted ecosystem. The first part of the framework is to assess and monitor the profile; organisations should periodically examine their risk profile and develop a plan to address any key areas of exposure. The next part of the framework is to certify and comply, where organisations should focus on the legal, governance, compliance and audit aspects to confirm that using cloud computing for big data does not violate any laws or agreements. Finally, the last step in the framework is to improve and enhance any new capabilities, so that organisations can transform data in the cloud into enterprise information assets and one true source of data to yield added business value for the organisations.
“No matter what industry an organisation is in, a key success factor for organisations is the availability of relevant data at the right time.”
