[Remote] Data Engineer II
Note: The job is a remote job and is open to candidates in USA. NPS Prism is a market-leading, cloud-based CX benchmarking and operational improvement platform owned by Bain & Company. We are seeking a highly skilled and experienced Data Engineer to design and implement robust data solutions for scalable business needs, utilizing cloud platforms and big data technologies.
Responsibilities
- Design, build, and optimize ETL/ELT workflows using tools like Databricks, SQL, Python/pyspark & Alteryx (Good to have)
- Develop and maintain robust, scalable, and efficient data pipelines for processing large datasets. from source to emerging data
- Work on cloud platforms (Azure, AWS) to build and manage data lakes, data warehouses, and scalable data architectures
- Utilize cloud services like Azure Data Factory, AWS Glue, or for data processing and orchestration
- Use Databricks for big data processing, analytics, and real-time data processing
- Leverage Apache Spark for distributed computing and handling complex data transformations
- Create and manage SQL-based data solutions, ensuring high availability, scalability, and performance
- Develop and enforce data quality checks and validation mechanisms
- Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders, to deliver impactful data solutions
- Understand business requirements and translate them into technical solutions
- Leverage CI/CD pipelines to streamline development, testing, and deployment of data engineering workflows
- Work with DevOps tools like Git, Jenkins, or Azure DevOps for version control and automation
- Maintain clear documentation for data workflows, pipelines, and processes
- Optimize data systems for performance, scalability, and cost-efficiency
Skills
- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field
- 3–6 years of experience in Data Engineering or related roles
- Hands-on experience with big data processing frameworks, data lakes, and cloud-native services
- Proficiency in Python, SQL, and PySpark for data processing and manipulation
- Proven experience in Databricks and Apache Spark
- Expertise in working with cloud platforms like Azure, AWS
- Sound knowledge of ETL processes and tools like Alteryx
- Leveraging data lakes, data warehouses, and data pipelines
- Build a Data Pipeline from scratch
- Strong understanding of distributed systems and big data technologies
- Basic understanding of DevOps principles and familiarity with CI/CD pipelines
- Hands-on experience with tools like Git, Jenkins, or Azure DevOps
- Strong problem-solving skills and a knack for optimizing data solutions
- Excellent communication (oral and written) skills
- Familiarity with data visualization tools like Power BI, Tableau, or similar
- Knowledge of streaming technologies such as Kafka or Event Hubs
Company Overview
Company H1B Sponsorship