Careers

Who we are looking for:

The data engineering team's mission is to provide high availability and high resiliency as a core service to our ACV applications. The team is responsible for ETL’s using different ingestion and transformation techniques. We are responsible for a range of critical tasks aimed at ensuring smooth and efficient functioning and high availability of ACVs data platforms. We are a crucial bridge between Infrastructure Operations, Data Infrastructure, Analytics, and Development teams providing valuable feedback and insights to continuously improve platform reliability, functionality, and overall performance.

We are seeking a talented data professional as a Senior Data Engineer to join our Data Engineering team. This role requires a strong focus and experience in software development, multi-cloud based technologies, in memory data stores, and a strong desire to learn complex systems and new technologies. It requires a sound foundation in database and infrastructure architecture, deep technical knowledge, software development, excellent communication skills, and an action-based philosophy to solve hard software engineering problems.

What you will do:

As a Data Engineer at ACV Auctions you HAVE FUN !! You will design, develop, write, and modify code. You will be responsible for development of ETLs, application architecture, optimizing databases & SQL queries. You will work alongside other data engineers and data scientists in the design and development of solutions to ACV’s most complex software problems. It is expected that you will be able to operate in a high performing team, that you can balance high quality delivery with customer focus, and that you will have a record of delivering and guiding team members in a fast-paced environment.

• Design, develop, and maintain scalable ETL pipelines using Python and SQL to ingest, process, and transform data from diverse sources.

• Write clean, efficient, and well-documented code in Python and SQL.

• Utilize Git for version control and collaborate effectively with other engineers.

• Implement and manage data orchestration workflows using industry-standard orchestration tools (e.g., Apache Airflow, Prefect)..

• Apply a strong understanding of major data structures (arrays, dictionaries, strings, trees, nodes, graphs, linked lists) to optimize data processing and storage.

• Support multi-cloud application development.

• Contribute, influence, and set standards for all technical aspects of a product or service including but not limited to, testing, debugging, performance, and languages.

• Support development stages for application development and data science teams, emphasizing in MySQL and Postgres database development.

• Influence company wide engineering standards for tooling, languages, and build systems.

• Leverage monitoring tools to ensure high performance and availability; work with operations and engineering to improve as required.

• Ensure that data development meets company standards for readability, reliability, and performance.

• Collaborate with internal teams on transactional and analytical schema design.

• Conduct code reviews, develop high-quality documentation, and build robust test suites

• Respond-to and troubleshoot highly complex problems quickly, efficiently, and effectively.

• Mentor junior data engineers.

• Assist/lead technical discussions/innovation including engineering tech talks

• Assist in engineering innovations including discovery of new technologies, implementation strategies, and architectural improvements.

• Participate in on-call rotation

What you will need:

• Bachelor’s degree in computer science, Information Technology, or a related field (or equivalent work experience)

• Ability to read, write, speak, and understand English.

• 4+ years of experience programming in Python

• 3+ years of experience with ETL workflow implementation (Airflow, Python)

• 3+ years work with continuous integration and build tools.

• 3+ years of experience with Cloud platforms preferably in AWS or GCP

• Knowledge of database architecture, infrastructure, performance tuning, and optimization techniques.

• Deep Knowledge in day-day tools and how they work including deployments, k8s, monitoring systems, and testing tools.

• Proficient in databases (RDB), SQL, and can contribute to schema definitions.

• Self-sufficient debugger who can identify and solve complex problems in code.

• Deep understanding of major data structures (arrays, dictionaries, strings).

• Experience with Domain Driven Design.

• Experience with containers and Kubernetes.

• Experience with database monitoring and diagnostic tools, preferably Data Dog.

• Hands-on skills and the ability to drill deep into the complex system design and implementation.

• Proficiency in SQL query writing and optimization.

• Experience with database security principles and best practices.

• Experience with in-memory data processing

• Experience working with data warehousing concepts and technologies, including dimensional modeling and ETL frameworks

• Strong communication and collaboration skills, with the ability to work effectively in a fast paced global team environment.

• Experience working with:

  • SQL data-layer development experience; OLTP schema design

  • Using and integrating with cloud services, specifically: AWS RDS, Aurora, S3, GCP

  • Github, Jenkins, Python, Docker, Kubernetes

Nice to Have Qualifications

• Experience with Airflow, Docker, Visual Studio, Pycharm, Redis, Kubernetes, Fivetran, Spark, Dataflow, Dataproc, EMR

• Experience with database monitoring and diagnostic tools, preferably DataDog

• Hands-on experience with Kafka or other event streaming technologies.

• Hands-on experience with micro-service architecture