View Our Website View All Jobs

Engineering Manager, Archive-It

The Internet Archive has over 30PB of unique digital information running across an integrated cluster of over 700 VMs on over 600 "bare-metal" hosts in multiple data centers. Archive-It is the Internet Archive’s subscription web archiving service, used by over 500 libraries, archives, universities, non-profits, governments, and others to build curated web archive collections. The service supports hundreds of user-scheduled web crawls per day via a full-featured collection management web application, maintains a public-access website,, and oversees an aggregate collection of around 1PB and 23 billion URLs, as well as associated repositories, processing clusters, databases, APIs, and affiliated services.

We are looking for a production-oriented, "hands-on" engineering team manager with experience effectively managing staff and participating in projects across a busy group of remote and on-site engineers. The ideal candidate will bring a user-oriented approach to technical leadership with a focus on advancing the service, managing development projects from origin to release, ensuring operational resiliency, and implementing best practices around testing and deployment. Candidates should also be skilled in management communications and able to work collaboratively with a distributed team of engineers, managers, and program staff. The Engineering Manager, Archive-It reports to the Director of Web Archiving and works closely with the Director of Engineering, Web Archiving Program Managers, and Archive-It and Web Group engineering teams.

Job Summary

The Engineering Manager, Archive-It is responsible for managing, contributing, and mentoring the Archive-It engineering team and is responsible for developing, maintaining, and supporting Archive-It services world-wide.

Responsibilities & Duties

  • Manage Archive-It engineering team responsible for developing, maintaining, and supporting Archive-It services. This includes all publicly-facing applications as well as internal systems and services.
  • Work directly with Director, Program Managers, and team to plan, schedule, and manage to completion new features, services, and development projects for Archive-It.
  • Identify, plan, and manage the implementation of improvements in processes and operational infrastructure for Archive-It and Internet Archive.
  • Coordinate and contribute to the transition of new software and services from a development into a production footing. This includes establishing procedures and policy that will ensure agile, sustainable development and deployment practices and overall service growth and expansion.
  • Coordinate, recruit, hire, schedule, and fire engineering staff as needed to achieve application feature and service objectives.

Qualifications & Skills

  • Experience as manager and mentor of an engineering team
  • Strong advocate for the end user experience of web-delivered services and an overall “customer service” mentality
  • Experience in highly available 24x7 production environment and managing large server cluster infrastructure.
  • Ability to document, communicate, and share critical knowledge with both engineering and product staff
  • Passion for automation, data-driven decision making, and information reporting
  • Work history that includes production-level programming in high-transaction environments
  • Fluency in Linux system administration, Unix shell scripting, and Python, with familiarity in Java, no-SQL databases, etc.
  • Experience deploying and administering search and web-host services
  • Excellent and creative problem solver, communicator, and mentor.
  • Experienced in open source practices and passion for staying current with trends
  • BS Computer Science, or equivalent work experience

Preferred Qualifications

  • Experience with Ansible, Git, Nagios, ELK stack, etc.
  • Experience deploying and maintaining big data analytics tools and systems, especially Hadoop
  • Excellent oral and written communication and documentation skills, including presenting to and communicating with external, non-technical partners
  • MS in Computer Science or equivalent work experience
  • Flexibility and a sense of humor

Benefits & Perks

The Internet Archive provides a comprehensive benefits package including; PTO, paid holidays, medical, dental, vision, FSA, commuter, STD, LTD, 403B/Roth accounts and Friday lunches at IA HQ.

Internet Archive is a proud Equal Opportunity Employer M/F/D/V/L/G/B/T and will consider for employment, qualified applicants with criminal histories in a manner consistent with the requirements of the Fair Chance Ordinance.



Read More

Apply for this position

Apply with Indeed
Attach resume as .pdf, .doc, or .docx (limit 2MB) or Paste resume

Paste your resume here or Attach resume file

To comply with government Equal Employment Opportunity / Affirmative Action reporting regulations, we are requesting (but NOT requiring) that you enter this personal data. This information will not be used in connection with any employment decisions, and will be used solely as permitted by state and federal law. Your voluntary cooperation would be appreciated. Learn more.
Veteran/Disability status