Big data as a technology passed through various stages of evolution during the last few years, which still keeps it hot in the list of tech buzzwords! Starting with handling the 3 V’s of data – Volume (of data to be handled), Velocity (of data generated) and Variety (of data generated), it has spread wings to more V’s – Veracity (to ensure data integrity and reliability), Vulnerability (to address privacy and confidentiality concerns) and Value (of information)!
As Google showed the way, collection and collation of huge volumes of data and applying the right analytics to gain valuable insights into the business and optimization possibilities is the key to extracting the full potential of the data-driven industry. Today Chief Data Officers are building strategies to organize their data and to derive business intelligence from it to drive radical transformation of businesses in many sectors such as industrial, retail, logistics, healthcare etc.
BDaaS (Big Data-as-a-Service) is gaining momentum, enabling external experts to take the company’s customer data to the cloud and to provide analytical insights for decision making. Offered as a managed service, it frees up the customer from substantial initial investment and helps offer RoI-driven spending. This article focusses on BDaaS, describing the potential that enables our customers to conceptualize and launch new business models.
Large corporations with structured and centralized ERP systems wouldn’t benefit as much from BDaaS as compared to unorganized sectors comprising of diverse players each with their own fragmented IT infrastructures. For instance, unorganized retail is a heterogeneous sector with a geographically distributed supply chain that spans across medium and small players, having considerable differences in their levels of process maturity. Stand-alone islands of software application are encountered many times and so are ad-hoc (or legacy) structures of data storage and archival. B2B companies providing services to geographically spread out customers in many traditional supply chains like chemicals/reagents for laboratory use, petrochemical (non-fuel) derivatives and medical drugs could benefit from the transformational potential of BDaaS.
Suppose you are a B2B player in one of these or similar sectors, let us take a closer look at your business and customer data! Could your expertise in the industry be leveraged to identify a new data-driven model by “integrating your customer data” to offer new intelligence gleaned from it? This integration gives you the data in ‘sector level’ rather than ‘individual’ customer level. You will be able to identify sector level intelligence and provide it to all your customers which will be mutually beneficial for all.
In order to accomplish this outcome, you will most often need external expertise in big data to work collaboratively with you (or your domain consultants) in order to build a BDaaS platform to offer your customers. The value of business intelligence that the platform brings helps them win in their businesses and their patronage in turn helps your business model succeed.
So has been our experience working with a world leader in the pharmacy supply chain across North America. Besides supplying medicines and medical equipment to their customers, they also provide inventory and patient management software to their customers. The software installed in each of the numerous hospitals gathers transactional data over time. We worked closely with the customer’s consultants on the feasibility of data integration and created a centralized control center using big data technologies such as Spark and Kafka. Hosted on the cloud, the platform captures streaming data from different hospitals and pushes them to the centralized system that offers a metered BDaaS service to end-customers, the analytics insights helping them to optimize their businesses.
The path to big data implementation, however, was filled with several challenges, a few of which are:
With the regulatory requirements concerning medical information like the HIPPA standards, compliance is mandatory. Only non-sensitive data at a lower level of granularity is collected, that respects privacy concerns of the individual hospitals of exposure of their patients’ sensitive information. This is the key factor to the success of the project both from customer buy-in and regulatory compliance points of view. The collected data is pushed to cloud securely with transport layer security.
Verity of data
The data being heterogeneous and scattered is the foremost challenge while implementing big data solutions. Even though most hospitals use our customer’s software a few others use their own legacy software. Data could be isolated even across the departments in the same organization! We built data collector modules which can be customized easily to collect data from various sources and push it to the cloud. Rationalizing the relevant data fields from these diversified sources and integrating it provides a lot of insight into possibilities of analytics.
Time to market and initial investment
Being a metered service we had to make sure that customer’s cost is kept linear with usage. Databricks big data platform with reliable Open Source Kafka data injector gives us a balanced and scalable framework to meet this objective.
After data was made available from all sources centrally for analysis it was discovered that information on the availability of particular medicines in each hospital along with demand predictability has the potential to reduce the associated transportation costs by around 20%. Data-driven drill down revealed for instance that for a particular area with a prevalence of influenza but with shortage of the corresponding medicine, the system can identify the best possible area (nearest, where there is enough stock but no demand currently) from which this medicine can be arranged. Supply chain demand mitigation by coordinating drug supply between customers can significantly save inventory and transportation cost for customers. More importantly, it saves precious reaction time for their end-users, which would not have been possible without the magic of BDaaS.
In your own strategy to connect your fragmented customer data centrally to provide mutually beneficial information, the role of an experienced big data partner is indeed crucial. Combine the power of your domain expertise with big data specialists to create new data-driven business models which besides increasing your revenues could make you the hub to all customers thereby increasing the bonding of existing ones and attracting new ones.