- Lead/Administer the enterprise data platform (DWH, Data Lake).
- Create and maintain optimal/reliable data pipeline architecture to meet business needs.
- Define and operate the infrastructure required for optimal extraction, transformation and loading (ETL) of data from a wide variety of data sources using SQL and 'big data' technologies as needed (Hadoop, MapReduce, Hive, Spark, Kafka, Pig, data streaming, NoSQL, SQL, programming)
- Design and implement life cycle management processes (DevOps) to enable continuous integration and continuous deployment (CICD) of data systems.
- Integrate data from various resources (including external data sources and IoT) and manage the big data as a key enterprise asset.
- Create and maintain backend data solutions for data analysts and data scientists. Assist them in unlocking insight from enterprise data.
- Identify, design, and implement internal process and framework (e.g. elimination of manual processes, optimizing data delivery, evolving data infrastructure capabilities, etc.)
- Work with stakeholders including product, data and architecture SME to assist with data-related technical issues and support their data infrastructure needs.
- Ensure compliance to data architecture and security requirements.
- Identify data quality issue and make recommendation for addressing root causes.
- Setup observability/monitoring to measure reliability of the data pipelines and act quickly (e.g. operational support) in case of incident.
Lead Data Engineer - Dorval, Canada - Bombarbier
Description
Bombardier is a global leader, creating innovative and game-changing planes. Our products and services provide world-class transportation experiences that set new standards in passenger comfort, energy, efficiency, reliability and safety. We are a global organization focused on working together with a team spirit.
In your role, you will
As our ideal candidate
• You hold a bachelor's degree in computer science, Statistics, Informatics, Information Systems or another quantitative field.
• You have 10 years of experience in a Data Engineer / Data Specialist role .
• You have experience coaching/leading a small team (technical leadership).
• You have knowledge of Agile / SCRUM project delivery, DevOps and CICD practices.
• You have advanced knowledge of SQL, query authoring and relational databases.
• You have experience optimizing 'big data' pipelines (storage, file format, partitioning, spark, python, streaming)
• You are efficient at performing root cause analysis to address issues and applying long-term fix.
• You have experience designing and building data transformation, data structures, metadata framework and automated workload management.
• You have experience in data protection measures, data privacy and collaborating with Cyber team
• You have good knowledge of Azure data services (Azure Data Factory, Synapse, Azure Data Lake Storage, Event Hub, Polybase, Databricks, Delta lake, Cognitive Services, ...)
• You have good knowledge of Object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.
• You have good people skills and are a team player, motivated by developing other people.
• You are a good communicator, who can simplify complex technical issue for non-technical people. You can work in an environment with a mix of French and English languages.
Your ideas move people.
Job Lead Data Engineer ( Data Integration)
Primary Location Administrative Centre (CA)
Organization Aerospace Canada
Shift
Employee Status Regular