Towards Data Science
Jun 12
I Thought Data Engineering Was Just Writing Scripts. I Was Wrong.
I tried to make my ETL pipeline production-ready. Three things broke.
I Built My Second ETL Pipeline. This Time, I Started Thinking Like a Data Engineer
Building a production-ready RSS pipeline with Python, Docker, PostgreSQL, and Kestra The post I Built My Second ETL Pipeline. This Time, I Started Thinking Like a Data Engineer appeared first on Towards Data Science .
Related stories
Towards Data Science
Jun 24
Your First Task as a Data Engineer in a New Company? Make the ETL Pipeline Testable
A practical data engineering onboarding workflow for environment setup, automated testing, and AI-assisted development. The post Your First Task as a Data Engineer in a New Company?
Towards Data Science
Jun 19
I Tried to Schedule My ETL Pipeline. Here’s What I Didn’t Expect.
What I thought was a scheduling problem turned out to be a portability problem first The post I Tried to Schedule My ETL Pipeline. Here’s What I Didn’t Expect.
Towards Data Science
Jun 24
Why I Stopped Using One Agent and Built a Multi-Agent Pipeline Instead
A practical walkthrough using text-to-SQL as the example The post Why I Stopped Using One Agent and Built a Multi-Agent Pipeline Instead appeared first on Towards Data Science .
Towards Data Science
Jun 25
One Month Into Learning Data Engineering in Public: Here’s What I Didn’t Write About
A reflection on the first month of learning data engineering in public, and what actually kept me going. The post One Month Into Learning Data Engineering in Public: Here’s What I Didn’t Write About appeared first on Towards Data Science .
Towards Data Science
Jul 17
A Production RAG Pipeline in Action: Every Answer Typed and Cited
Enterprise Document Intelligence [Vol. 1 #9B] - One call wires the four upgraded bricks together, run on a paper, a NIST standard, and a report with a broken TOC The post A Production RAG Pipeline in Action: Every Answer Typed and Cited appeared first on Towards Data Science .
Towards Data Science
Aug 25
A New Towards Data Science: A Faster Site and a Brand-New Contributor Portal
Towards Data Science has announced a major overhaul of its website and contributor portal. The new site promises faster performance and a brand‑new portal for writers, aiming to improve the experience for both readers and contributors. The update is positioned as a significant upgrade for anyone who reads, writes, or both on the platform.
Towards Data Science
Aug 3
The AI Was the Easy Part: What Is a Forward-Deployed Engineer in a Supply Chain?
What actually makes a Forward Deployed Engineer, told through one supply chain project. The post The AI Was the Easy Part: What Is a Forward-Deployed Engineer in a Supply Chain?
Towards Data Science
Aug 6
Loop Engineering for Cross-References: When RAG Answers ‘see Section 7.2’ Instead of the Actual Answer
Enterprise Document Intelligence [Vol. 1 #11] - When the first answer points elsewhere in the document, the pipeline loops back to fetch the linked context The post Loop Engineering for Cross-References: When RAG Answers ‘see Section 7.
Towards Data Science
4d ago
AI Made Data Scientists Faster. Now It’s Expanding the Job.
AI has accelerated data scientists’ productivity, but its influence extends beyond speed. The technology is reshaping who owns data, how judgment is exercised, and the overall career trajectory of data scientists. These changes signal a broader transformation in the field’s structure and responsibilities.
Towards Data Science
Jul 14
How I’m Making Sure My Analytics Career Doesn’t Get Eaten by AI
The analytics career I signed up for five years ago doesn't exist anymore, and honestly, I am fine with that. The post How I’m Making Sure My Analytics Career Doesn’t Get Eaten by AI appeared first on Towards Data Science .
Towards Data Science
Jul 17
One RAG Pipeline, Four Very Different PDFs: Same Four Bricks, Every Answer Typed and Cited
Enterprise Document Intelligence [Vol. 1 #9B] - One call wires the four upgraded bricks together, run on a paper, a NIST standard, and a report with a broken TOC The post One RAG Pipeline, Four Very Different PDFs: Same Four Bricks, Every Answer Typed and Cited appeared first on Towards Data Science .