You will be responsible for optimizing or even re-designing our own state-of-the-art Artificial Intelligence Assistant.
Additionally, you will work in our content acquisition pipeline and our new solution for data ingestion and web scraping, defining the architecture and implementing the solution in Python following the best practices.
The ideal candidate is an experienced Python developer that enjoys defining an architecture and building solutions from the ground up, and ideally with some background working in Web Scraping.
- You develop small, well-crafted components that plug into our data processing pipeline.
- You write scalable queries that impact millions of rows while keeping execution time low.
- You make improvements to our testing procedure that allow us to move faster and with more confidence.
- You collaborate with data scientists and machine learning experts in order to tackle challenging problems in a scalable way.
- You work with cutting edge technologies like delta lake to help us provide our users with innovative tools.
- You build distributed systems for web scraping, data processing