[Remote] DATABRICKS DATA ENGINEER
Note: The job is a remote job and is open to candidates in USA. National Data Solutions is supporting a data warehouse and RCM modernization program for a large national healthcare client. They are seeking a Databricks Data Engineer with strong healthcare claims data experience to build data pipelines and validate claims data integrity during the program's initial phase.
Responsibilities
- Build and configure nightly automated data extraction pipelines from multiple source systems including practice management, clearinghouse, and EHR platforms
- Normalize and canonicalize claims data across heterogeneous sources into a unified enterprise data model
- Validate claims data integrity, completeness, and accuracy — including charge, payment, adjustment, denial, and ERA/EOB data — against a 24-month historical baseline
- Identify and document data quality gaps, feed failures, and reconciliation discrepancies
- Implement data lineage, refresh cadence, and governance controls within the Databricks environment
- Support Power BI connectivity from the Databricks semantic layer
- Coordinate directly with NDS domain leads and client stakeholders on data dictionary alignment, KPI logic, and validation findings
- Participate in UAT and resolve data quality issues surfaced during dashboard validation
Skills
- 3+ years of hands-on Databricks engineering experience including Delta Lake, Unity Catalog, and Databricks Workflows
- Direct experience with healthcare claims data — charge entry, claim submission, remittance, denial management, or AR — in a data engineering or analytics context
- Demonstrated ability to validate and reconcile complex multi-source claims datasets
- Proficiency in PySpark and SQL
- Experience connecting Databricks to Power BI via DirectQuery or semantic layer
- Working knowledge of HIPAA data handling requirements and PHI masking in non-production environments
- Strong communication and client-facing skills — you will interact regularly with NDS leads and client stakeholders and are expected to present findings, ask the right questions, and manage expectations professionally
- Ability to work independently against a defined scope and surface blockers clearly and promptly
- Familiarity with clearinghouse data structures, ERA/EOB formats, or CARC/RARC code sets
- Prior work in RCM, home health, or managed care analytics environments
Company Overview