Senior Data Engineer

About the role

A Senior Data Engineer is sought to help implement data analytics products using on-premise and cloud data platforms. The position is based in Guadalajara and offers a hybrid arrangement.

Key responsibilities

• Work with big data platforms such as Databricks and Apache Spark at large scale, pushing the limits of data processing and model enablement.
• Build and maintain robust ETL/ELT pipelines for ingestion, transformation and aggregation of large-scale datasets on Hadoop and enterprise data platforms.
• Develop high-performance data processing jobs with PySpark/Spark and Python on platforms such as Cloudera and Databricks.
• Optimize pipeline performance and cost through partitioning, file formats, compute tuning and efficient query patterns.
• Contribute to CI/CD for data workflows, including testing, code reviews and deployment automation.
• Partner with Product Managers to understand users and use cases, and apply that knowledge to scope and build new modules and features.

Required profile

• Strong hands-on experience building production-grade pipelines on big data platforms (Hadoop ecosystem and cloud platforms such as Databricks).
• Experience with orchestration and integration tools such as Apache Airflow, Apache NiFi or Talend.
• Working knowledge of DevOps/CI-CD practices: version control (Git), automated testing, release pipelines and observability.
• Strong problem-solving skills, able to debug complex data issues and communicate clearly with technical and non-technical stakeholders.
• Experience developing Java-based applications is an added advantage.

Required skills

• Python, Spark, Hive, Impala and SQL
• Databricks (preferred) and Snowflake
• Apache Airflow, Apache NiFi or Talend