Job Requirements
Job Title: Data Engineer (Java/PySpark, Python, Iceberg)
We are looking for a Data Engineer with strong hands-on experience in Java, Python, PySpark, and Data Engineering to design and build scalable data pipelines and data processing solutions.
Key Responsibilities
Design, develop, and maintain scalable data pipelines using Java and PySpark.
Build and optimize data ingestion, transformation, and processing frameworks for large-scale datasets.
Develop solutions for batch and near real-time data processing.
Work with Apache Iceberg tables and modern data lake architectures.
Collaborate with data architects, analysts, and business stakeholders to understand data requirements.
Ensure data quality, governance, security, and performance across data platforms.
Write efficient and reusable code using Python and Java.
Optimize Spark jobs for performance, scalability, and reliability.
Integrate data from multiple structured and unstructured sources.
Participate in code reviews, troubleshooting, and production support activities.
Required Skills
Experience Band - 4 to 8 years
Strong experience in Java and PySpark development.
Solid understanding of Data Engineering concepts and best practices.
Experience designing and implementing ETL/ELT pipelines.
Good knowledge of distributed computing frameworks such as Apache Spark.
Experience working with SQL and large-scale data processing.
Understanding of data modeling, partitioning, and performance optimization techniques.
Familiarity with version control systems such as Git.
Strong problem-solving and analytical skills.
Preferred Skills
Knowledge of Python development.
Experience with Apache Iceberg and Data Lakehouse architectures.
Experience with cloud platforms such as AWS, Azure, or GCP.
Exposure to workflow orchestration tools such as Airflow.
Hands-on experience with Kafka or other streaming technologies.
Experience with ETL tools is a plus, but candidates should have strong coding and data engineering expertise.
What We Are Looking For
Strong hands-on Data Engineer with a coding-first mindset.
Proven experience in Java/PySpark development.
Knowledge of Python and Apache Iceberg.
ETL experience with tools like IBM Datastage is desirable.
Work Experience
5-7Years
Job Title: Data Engineer (Java/PySpark, Python, Iceberg)
We are looking for a Data Engineer with strong hands-on experience in Java, Python, PySpark, and Data Engineering to design and build scalable data pipelines and data processing solutions.
Key Responsibilities
Design, develop, and maintain scalable data pipelines using Java and PySpark.
Build and optimize data ingestion, transformation, and processing frameworks for large-scale datasets.
Develop solutions for batch and near real-time data processing.
Work with Apache Iceberg tables and modern data lake architectures.
Collaborate with data architects, analysts, and business stakeholders to understand data requirements.
Ensure data quality, governance, security, and performance across data platforms.
Write efficient and reusable code using Python and Java.
Optimize Spark jobs for performance, scalability, and reliability.
Integrate data from multiple structured and unstructured sources.
Participate in code reviews, troubleshooting, and production support activities.
Required Skills
Experience Band - 4 to 8 years
Strong experience in Java and PySpark development.
Solid understanding of Data Engineering concepts and best practices.
Experience designing and implementing ETL/ELT pipelines.
Good knowledge of distributed computing frameworks such as Apache Spark.
Experience working with SQL and large-scale data processing.
Understanding of data modeling, partitioning, and performance optimization techniques.
Familiarity with version control systems such as Git.
Strong problem-solving and analytical skills.
Preferred Skills
Knowledge of Python development.
Experience with Apache Iceberg and Data Lakehouse architectures.
Experience with cloud platforms such as AWS, Azure, or GCP.
Exposure to workflow orchestration tools such as Airflow.
Hands-on experience with Kafka or other streaming technologies.
Experience with ETL tools is a plus, but candidates should have strong coding and data engineering expertise.
What We Are Looking For
Strong hands-on Data Engineer with a coding-first mindset.
Proven experience in Java/PySpark development.
Knowledge of Python and Apache Iceberg.
ETL experience with tools like IBM Datastage is desirable.
Work Experience
5-7Years