The position will work in the Web Archiving Group, supporting development of web archiving and data services for partners ranging from national libraries and governments to universities, researchers, and collaborating technology partners. The role will help build, maintain, and scale new and existing software and services, including at-scale data processing pipelines, access and harvesting technologies, APIs, and contribute to improving our systems and services used by libraries and open knowledge organizations around the world. The position will also help to synthesize user and business driven technical needs into new engineering and development projects and features across our portfolio of paid services. This role contributes to managing data at scale, building new tools and technologies, monitoring and deploying production systems, working closely with a distributed team of program/product staff and engineers, and liaising directly with an international set of collaborators and clients on web and data services.
Responsibilities & Duties
Preferred Skills & Requirements
3-4 years of experience in Python and Unix/Linux shell
Reporting Structure: The Web Data Engineer reports to the Director, Web Archiving & Data Services and works closely with program staff in Web Archiving & Data Services team, and with the broader IA operations/infrastructure and engineering teams.
Location: San Francisco, CA or remote
References must be made available upon request.
Internet Archive is an Equal Opportunity Employer. Internet Archive complies with the Fair Chance Ordinance. Internet Archive is a 501(c)(3) non-profit library founded in 1996.