Two insurance providers send a patient file on the same day. One calls him John. The other calls him Johnny. Same date of birth, same address, same prescriptions, but to the receiving system, that's two different people. Multiply that by many payers, hundreds of thousands of patients, and years of accumulated files, and you have a big data problem.
SoftServe built a centralized patient data platform that ingests files from every payer, resolves duplicate identities, and standardizes everything into one trusted record. Now there's a single governed patient view that health advisors, analysts, and machine-learning teams can all build on.
More data does not mean a clear view of the patient
A large U.S. healthcare and pharmacy organization was pulling patient data from dozens of payers. The data was in disarray, the John/Johnny problem was just one of many examples. What they wanted should have been a given. Take everything known about a patient, from every payer, and put it into one record, in one place.
Sounds simple, even easy. Here's the reality.
Each payer delivered its data differently. Demographics, medical and pharmacy claims, and care-gap records arrived through legacy processes. Modern formats and interfaces existed; almost nobody used them.
And no two payers labeled the data the same. Source systems were thinly documented, and requirements shifted from week to week as stakeholders came and went. Because every file was a new full snapshot, nobody upstream had any notion of what had changed since the last one.
The result was a fragmented data landscape that made it hard to give analytics teams timely, trusted information, so they could build a dependable patient view. The ultimate project goal was to build that one unified, trustworthy patient record no matter how the data arrived.
Creating consistency without controlling the sources
Healthcare organizations can't make the payers send data the same way. Many providers, each with its own systems and its own habits, were never going to agree on a format. If patient data was going to be consistent, that consistency had to be built on the client's side.
The design started from a different premise. Instead of building around one fixed set of source files, the team built reusable patterns that could absorb change, whatever a payer sent, however a payer changed it, and whichever new payer showed up next.
Two decisions that shaped the project
First, preserve the raw data. SoftServe kept every source file exactly as it arrived in Delta Lake and did all the standardizing downstream from that untouched copy. Because the originals were always there, the team could reprocess whenever logic changed, trace any record back to its source, and hand the machine-learning team the raw history they needed, without ever asking a payer to resend a file. It also meant the client could change how it interpreted the data without losing the original context.
Second, build in stages. The work began small, one payer, the core data layers, an early quality framework, and two downstream integrations. Just enough to prove the architecture before scaling. Once it held, SoftServe productized the platform, and everything built along the way became a lasting capability.
Staging let the team learn from real data instead of waiting for requirements that never stopped coming, and it kept the foundational decisions separate from source-specific details, so the platform could absorb the next payer without a redesign.
One patient view, in seven stages
The project team designed and implemented a centralized patient data platform on Microsoft Azure, with Databricks and Delta Lake at its core.
Here's how the platform works, stage by stage, from the moment a file lands to the moment a health advisor can trust what's in front of them. Each stage solves a specific problem, with a vendor product where one existed, and custom engineering when one did not.
Arrival with Microsoft Azure
A file lands on the SFTP server. Azure Data Factory runs the ingestion, and before a single row moves forward, every file gets checked for its name, for duplicates, for empties, and anything that fails is caught at the door.
Memory with Databricks
Cleared files flow into that Databricks lakehouse, the raw foundation with schema evolution switched on, so the platform bends rather than breaks when a payer quietly changes its format. From here on, everything downstream reads from the preserved originals rather than touching them.
Change detection, built by SoftServe
Payers only ever sent full snapshots, so a file might carry 50,000 patient records when only a few hundred had changed. Rather than reprocess everything, SoftServe built its own change-data-capture (CDC) mechanism that compares each delivery against the last and sends only the genuine differences forward. That one decision rewrote the economics of the entire pipeline.
Identity with Verato
Now back to John and Johnny. The same person could still show up differently across two payers, a formal name here, a nickname there, and left alone the platform would create two profiles for one human being. Verato, an external patient-identity service, recognizes when records are the same person and assigns one shared identifier across every payer. Duplicates collapse into a single accurate profile.
A common language with USCDI
A shared identity only helps if the data around it speaks the same language. SoftServe mapped every payer's format into one common patient model based on USCDI, the U.S. healthcare interoperability standard. Downstream teams no longer had to learn each payer's quirks.
The quality gate, built by SoftServe
Between raw and ready sits an automated quality framework that catches malformed data early, where it's cheap to fix, instead of deep in the pipeline where it isn't. Some rules were set by the destination itself, since the downstream business platform would only accept data that met its constraints. Quality was built into the product.
Delivery and the round trip with Salesforce and Snowflake
Finally, the record surfaces where people use it. Salesforce Health Cloud is the front end, where health advisors open a clean, unified profile and act on it. When an advisor updates a record, that change streams back into the lakehouse so the business app never becomes an isolated copy. Analysts get their own environment in Snowflake. One governed foundation serves everyone, and the journey is a loop.
What changed for the business
The organization went from patient data it couldn't trust to a single, reliable patient view. The same payer files still arrive in their usual disarray, but now every one of them lands in one governed foundation and consolidates into a unified profile in Salesforce Health Cloud.
What that means:
- Health advisors open one clean, unified patient profile and act on it, instead of stitching together fragments from different payers.
- Analytics and machine-learning teams draw from a standardized, fully historical dataset, with the original source records preserved for whatever they need next.
- The next payer is a mapping job, so growth no longer means re-engineering the platform every time.
- Compliance and interoperability come built in, because the platform standardizes the data to a recognized healthcare standard that is ready for controlled sharing.
These are the results from a solid plan and an expert team. We got here by making the data trustworthy first, then letting every team build on the same foundation. Because the architecture doesn't need to be rebuilt for every new payer, application, or analytics initiative, the organization can keep growing at a fraction of the cost and effort.
Ready to build your own foundation?
Start small, prove the foundation on one payer first, then productize what works rather than betting everything up front. That staged, data-first approach is how SoftServe runs data engagements, and it's laid out in The Playbook for Data That Wins.
