Business Challenge
Our client’s radio stations and businesses generate large volumes of data that are geographically distributed. With so many data sources under the control of numerous stakeholders, the data integrity and quality can be unpredictable.
Our client consulted SoftServe to develop a data quality framework that will run on top of Google Cloud Services (GCS) to ensure data quality and integrity. The client’s engineering team built the first version of this framework and then SoftServe helped take it to the next level - working with their data team to pull and store data in a single Data Lake for further processing and analysis. This will enable our client to create a reliable data source for business, finance, and advertisement analytics to improve business operations.
Project Description
With the goal of creating a cloud-based data quality framework, SoftServe’s team conducted an onsite Discovery Phase to develop a proposed solution for implementation. Currently in the prototyping phase, our experts are leveraging numerous platforms and technologies, including Apache Beam, Java, GCP, DataFlow, BigQuery, and BigTable.
Using a set of data services GCP provides for data processing, we are building a framework that will help our client to build flexible data processing pipelines.
Business Value
Having provided a Discovery Phase report and Architecture Vision, the SoftServe team is currently working on a PoC.
The data quality framework SoftServe is building will speed-up development of complex data processing pipelines and expand our client’s data lake. With this new data framework our client will be able to aggregate data to be later used for business process optimization. Having a robust and flexible data quality framework, our client will be able to create new revenue pipelines faster.