Building a production-ready RSS pipeline with Python, Docker, PostgreSQL, and Kestra The post I Built My Second ETL Pipeline. This Time, I Started Thinking Like a Data Engineer appeared first on Towards Data Science .
By Ibrahim Salami
A practical walkthrough using text-to-SQL as the example The post Why I Stopped Using One Agent and Built a Multi-Agent Pipeline Instead appeared first on Towards Data Science .
By Priyansh Bhardwaj
The article recounts the author's experience of moving a Dockerized data pipeline from a local laptop to AWS, highlighting the challenges that arose when the environment changed. It explores lessons learned about container behavior, networking intricacies, and hidden assumptions that were previously taken for granted in a local setup. The post serves as a practical guide for developers facing similar transitions to cloud infrastructure.
By Ibrahim Salami
I tried to make my ETL pipeline production-ready. Three things broke.
By Ibrahim Salami
What I thought was a scheduling problem turned out to be a portability problem first The post I Tried to Schedule My ETL Pipeline. Here’s What I Didn’t Expect.
By Ibrahim Salami
The article discusses how the reliability mechanisms added to large language model (LLM) pipelines can lead to confident but incorrect outputs, especially when the correct answer is absent. It examines the behavior of pipelines in such scenarios and highlights the paradox where safeguards intended to improve accuracy may actually reinforce errors. The piece underscores the importance of understanding pipeline responses when faced with missing or ambiguous information.
By Hubert García Gordon