[Remote] Senior Data Engineer with Java, Apache Spark
Note: The job is a remote job and is open to candidates in USA. EPAM Systems is a technology-driven organization operating in the financial data domain, building a production-grade analytics and data ingestion platform. This role focuses on designing, building and operating large-scale data ingestion and analytics solutions for batch and real-time processing of financial data, supporting critical downstream analytics and reporting use cases.
Responsibilities
- Design, develop and maintain Apache Spark batch pipelines and Kafka streaming solutions
- Build serverless microservices using AWS Lambda and Azure Functions
- Develop data ingestion connectors for databases, file systems, message queues and financial data feeds
- Write production-grade code in Java, Scala and Python
- Manage infrastructure using Terraform across multiple environments
- Containerize and deploy applications using Docker and Kubernetes (EKS/AKS)
- Implement and maintain CI/CD pipelines in GitLab CI
- Write and maintain unit and integration tests with high code coverage
- Monitor platform health and troubleshoot production issues in distributed systems
Skills
- 5+ years of experience in data engineering or backend platform development
- Proficiency in Java and/or Scala as primary languages with Python as secondary
- Expertise in Apache Spark including Spark SQL and Spark Streaming
- Skills in Apache Kafka and event-driven architectures
- Competency in SQL and data modeling
- Familiarity with AWS cloud services and serverless computing
- Knowledge of Terraform for Infrastructure as Code
- Skills in Docker and Kubernetes (EKS/AKS)
- Familiarity with CI/CD pipelines such as GitLab CI, Jenkins or equivalent
- Background in cloud data warehouses such as Amazon Redshift
- Understanding of Microsoft Azure services and Azure-native tooling
- Knowledge of Iceberg, Hudi or Delta Lake
Benefits
- Delivering innovative solutions to industry leaders, making a global impact
- Enjoyable working environment, whether it is the vibrant office or the comfort of your own home
- Opportunity to work abroad for up to two months per year
- Relocation opportunities within our offices in 55+ countries
- Corporate and social events
- Leadership development, career advising, soft skills and well-being programs
- Certifications, including GCP, Azure and AWS
- Unlimited access to LinkedIn Learning and Udemy
- Free English classes with certified teachers
- Participation in the Employee Stock Purchase Plan
- Monetary bonuses for engaging in the referral program
- Comprehensive medical & family care package
- Five trust days per year (sick leave without a medical certificate)
- Benefits package (sports activities, a variety of stores and services)
- Four trust days per year for personal needs
- Discounts for fitness clubs
- Benefits package (hotels, restaurants, stores and services)
- Discounts in local language schools, including online courses for the Kazakh language
- Medical & family care package
- Six trust days per year (sick leave without a medical certificate)
- Coverage of psychology sessions of your choice
- Benefits package (sports activities, a variety of stores and services)
- Housing support program: preferential mortgage access via Otbasy Bank partnership
- Discounts for fitness clubs and sports programs
- Discounts in local language schools, including offline courses for the Uzbek language
- Discounts for fitness clubs, dance schools and sports programs
Company Overview
Company H1B Sponsorship