Lead Data Engineer - ref. q82735803
Job Description
A Lead Data Engineer works closely with a multidisciplinary Agile team to build high quality data pipelines driving analytic solutions. These solutions will generate insights allowing to advance data-driven decision-making capabilities. This role requires deep understanding of data architecture, data engineering, data analysis, reporting, and a basic understanding of data science techniques and workflows.The ideal candidate is a skilled data / software engineer with experience creating data products supporting analytic solutions. They work as part of a technical, cross functional analytics team.
KEY ACCOUNTABILITIES:
Design, develop, optimize, and maintain data architecture and pipelines that adhere to Extract, Transform, and Load (ETL) principles and business goals
Solve complex data problems to deliver insights that helps our business to achieve their goals
Create data products for analytics and data scientist team members to improve their productivity Advise, consult, mentor and coach other data and analytic professionals on data standards and practices
Foster a culture of sharing, re-use, design for scale stability, and operational efficiency of data and analytical solutions
Lead the evaluation, implementation and deployment of emerging tools and process for analytic data engineering in order to improve our productivity as a team
Develop and deliver communication and education plans on analytic data engineering capabilities, standards, and processes
Partner with business analysts and solutions architects to develop technical architectures for strategic enterprise projects and initiatives
Develop and deliver some data science courses part of advance analytics and digital academy
Performs other duties as assigned.
QUALIFICATIONS & SKILLS:
Minimum Qualifications:
Bachelor's degree or MS required; Computer Science, MIS, or Engineering preferred
5 years of experience working in data engineering or architecture role, 7+ preferred
Expertise in SQL and data analysis and experience with programming language (Python, R or Scala preferred)
Minimum Experience:
Experience developing and maintaining data warehouses in big data solutions
Should have domain knowledge of metal and mining or energy or heavy industry or oil and gas experience or a combination of either industries.
Good Knowledge of programming languages and developing data analytics models (R, Python, Spark, Java, Scala, etc.)
Experience with dashboard development and BI tools such as Tableau, Power BI, Looker, Shiny, Qlik, etc.
Experience with developing solutions on cloud computing services and infrastructure in the data and analytics space (preferred)
Experience in data architecture including the ability to define data retention policies, monitor performance and advise any necessary infrastructure changes
Good knowledge of data and analytics, e.g. dimensional modeling, ETL, reporting tools, data governance and warehousing, structured and unstructured data
Practical knowledge in traditional ETL tools (Informatica, Alteryx, SAP data services etc.) as well as big database systems like the Hadoop ecosystem and cloud technologies (Azure, Spark, etc.)
Database development experience using Hadoop or BigQuery and experience with a variety of relational, NoSQL, and cloud database technologies Big Data Development experience using Hive, Impala, Spark and familiarity with Kafka (Preferred), familiarity with the different operating system (Preferred)
Exposure to Machine Learning, data science, computer vision, AI, statistics, and/or applied mathematics
Good working knowledge of industry 4.0 application, big data, autonomous robots, system integration, cloud computing, additive manufacturing, augmented reality etc.
Experience in automation & control system and training in similar fields is valued.
AUTHORITY/ DECISION MAKING:
Thinking within broadly defined policies, standards and objectives. The determination of what needs to be done in applying polices is largely left up to the incumbent who must establish the plan, determine the priorities and prescribe the processes needed to achieve the objectives
Subject to general direction and broadly defined functional policy objectives. Actions which will impact other functional or operating areas usually require approval before they may be implemented.
Agile/Digital skills:
Experience working on collaborative Agile product team
Individual skills:
Self-motivated with strong problem-solving and learning skills
Flexibility to changes in work direction as the project develops
Excellent communication, listening, and influencing skills
Mindset & Behaviors:
Flexibility in quickly adjusting positions based on stakeholders input
Ability to work at an abstract level and gain consensus