Data Engineer, DevOps - Remote OK

Company: Splunk
Location: Denver
Posted on: May 6, 2021

Job Description:

As a Data DevOps Engineer in the Enterprise DataWarehouse team you will work closely with a team of Data, Business Intelligence and Dev/Prod Ops engineers to help automate CI/CD deployment in a consistent and reliable way to non-production and production data infrastructure, at scale. As part this team, you will build CI/CD automation across GitLab/Github/Bitbucket, Airflow, Python ETL/ELT, Database scripts, containerization (Docker) and container-orchestration systems such as Kubernetes. You will work towards supporting deployment, data flow architecture and automation to deliver the highly available, performant and secure environments of Splunk's Enterprise Data Warehouse.Responsibilities* Iterate on release management methodologies to increase the quality & velocity of data warehouse deployments. Deploy and maintain CI/CD pipelines across multiple environments* You will collaborate with Production Engineering team's on-call responsibilities in rotation.* Experience working with Python based ETL/ELT, API and Cloud Data Warehouse databases. Data warehousing, Data Engineering experience, particularly with Snowflake is a plus.* Design, develop and operate terabyte-scale data pipelines and services that meet goals of low latency, high availability, resiliency, security and quality .* Define best practices, work with data engineering teams on best practices and ensure they are followed. Teach others how to do the right things. Automate everything.* Enable self-service as much as possible via automation so that application teams can do what they need without you, focus on the engineering aspect primarily* Automate schema updates/migration via common pipelines and pattern to be used by application teams in their deployment processesRequirements:* 8+ years of relevant work experience surrounding CI/CD, Test Automation and data infrastructure operations.* Demonstrated ability to develop advanced CI/CD pipelines in GitLab, Python, Airflow (eg using dynamic data pipelines, automated unit testing, performance testing, etc.)* Experience in writing and troubleshooting ETL jobs (Python, Airflow) and complex database SQL scripts.You have a strong python programming background in writing clean, testable code. Develop monitors and build alerts around error conditions and performance.* Experience in data technologies server administration, SLA/incident management, RBAC automation and controls, user communication and support.* Strong problem solving skills and ability to work independently or in team settings on complex production issues. Participate in On-Call support as needed.

Keywords: Splunk, Denver , Data Engineer, DevOps - Remote OK, Other , Denver, Colorado