Data Engineer Lead
We are currently building a company-wide Data Lake in the Cloud (AWS). It is actively used by our Sales, Marketing & Pricing teams Data Scientists in our Pricing & Analytics Center of excellence team.
We are implementing cutting-edge technologies for optimizing HP pricing and marketing spending worldwide. They need to have access & work on this wide range & variety of data in the most efficient manner.
The team has a worldwide impact, having teams located on all continents. You will be leading a team of 3 to 6 data engineers, who are responsible for bringing any data requested by the teams of data scientists into this data lake and you will make sure that those assets are properly built (data architecture, efficient ETL processing), documented, and evangelized to a broad audience of users.
Organize your team in a “follow the sun” (Global workflow) organization, and leverage to have footprints in all continents
Assigns responsibilities, provides direction, leadership and coaching, removes barriers as needed to enable teams to execute their roles and achieve objectives and goals.
People recruitment and development
Cost management, and direction-setting within the area of responsibility.
Give guidance, advise and bring into our data lake any data required, constrained by our context and technology stack
Leverage our stack in the fastest & most efficient manner: S3, Kubernetes cluster, Dataiku DSS, Spark (pySpark, Scala, Spark R), python, R, Julia languages, data partitioning
Guide and coach customers to access and use this data appropriately
Architecture organization of data subjects, and segment by gold/silver/platinum qualities
Typically, 8+ years of relevant experience
Bachelor's degree in Computer Science, Engineering
Experience directly leading and developing a team
Familiar with Agile Project Management.
In-depth understanding of technology in direct responsibility.
Deep understanding of Spark (experience on Spark optimization would be ideal)
Proficiency with pySpark
Experience working in a big data environment is required (hdfs, partitioning concept)
SQL proficiency, experience of ETL activities
Experience with Cloud environment (AWS preferred)
Experience with Kubernetes, docker
Nice to have:
Go & Julia
Experience with Dataiku DSS
Python & R, Scala is a plus
What we offer:
Opportunity to work in an international organization with colleagues coming from all over the world.
Diverse, continued internal growth and career opportunities. Including HP’s own learning platform and LinkedIn Learning.
An attractive benefitspackage.
Our Women Network organizes activities suchas;Networking, the promotion of STEM vocations, talks on; improving business acumen, work life balance and skills of the future etc.