Towards Data Science
Jun 12
I Thought Data Engineering Was Just Writing Scripts. I Was Wrong.
I tried to make my ETL pipeline production-ready. Three things broke.
I Built My Second ETL Pipeline. This Time, I Started Thinking Like a Data Engineer
Building a production-ready RSS pipeline with Python, Docker, PostgreSQL, and Kestra The post I Built My Second ETL Pipeline. This Time, I Started Thinking Like a Data Engineer appeared first on Towards Data Science .
Related stories
Towards Data Science
Jun 24
Your First Task as a Data Engineer in a New Company? Make the ETL Pipeline Testable
A practical data engineering onboarding workflow for environment setup, automated testing, and AI-assisted development. The post Your First Task as a Data Engineer in a New Company?
Towards Data Science
Jun 19
I Tried to Schedule My ETL Pipeline. Here’s What I Didn’t Expect.
What I thought was a scheduling problem turned out to be a portability problem first The post I Tried to Schedule My ETL Pipeline. Here’s What I Didn’t Expect.
Towards Data Science
Jun 24
Why I Stopped Using One Agent and Built a Multi-Agent Pipeline Instead
A practical walkthrough using text-to-SQL as the example The post Why I Stopped Using One Agent and Built a Multi-Agent Pipeline Instead appeared first on Towards Data Science .
Towards Data Science
Jun 25
One Month Into Learning Data Engineering in Public: Here’s What I Didn’t Write About
A reflection on the first month of learning data engineering in public, and what actually kept me going. The post One Month Into Learning Data Engineering in Public: Here’s What I Didn’t Write About appeared first on Towards Data Science .
Towards Data Science
Jul 17
A Production RAG Pipeline in Action: Every Answer Typed and Cited
Enterprise Document Intelligence [Vol. 1 #9B] - One call wires the four upgraded bricks together, run on a paper, a NIST standard, and a report with a broken TOC The post A Production RAG Pipeline in Action: Every Answer Typed and Cited appeared first on Towards Data Science .
Towards Data Science
Aug 3
The AI Was the Easy Part: What Is a Forward-Deployed Engineer in a Supply Chain?
What actually makes a Forward Deployed Engineer, told through one supply chain project. The post The AI Was the Easy Part: What Is a Forward-Deployed Engineer in a Supply Chain?
Towards Data Science
Aug 6
Loop Engineering for Cross-References: When RAG Answers ‘see Section 7.2’ Instead of the Actual Answer
Enterprise Document Intelligence [Vol. 1 #11] - When the first answer points elsewhere in the document, the pipeline loops back to fetch the linked context The post Loop Engineering for Cross-References: When RAG Answers ‘see Section 7.
Towards Data Science
Jul 14
How I’m Making Sure My Analytics Career Doesn’t Get Eaten by AI
The analytics career I signed up for five years ago doesn't exist anymore, and honestly, I am fine with that. The post How I’m Making Sure My Analytics Career Doesn’t Get Eaten by AI appeared first on Towards Data Science .
Towards Data Science
Jul 17
One RAG Pipeline, Four Very Different PDFs: Same Four Bricks, Every Answer Typed and Cited
Enterprise Document Intelligence [Vol. 1 #9B] - One call wires the four upgraded bricks together, run on a paper, a NIST standard, and a report with a broken TOC The post One RAG Pipeline, Four Very Different PDFs: Same Four Bricks, Every Answer Typed and Cited appeared first on Towards Data Science .
Towards Data Science
Jun 3
I Spent May Evaluating Different Engines for OCR
Testing fourteen engines on ninety-three human documents The post I Spent May Evaluating Different Engines for OCR appeared first on Towards Data Science .
Towards Data Science
Jun 11
BI Is Dead, Long Live BI
The true bottleneck was never the analysis. The post BI Is Dead, Long Live BI appeared first on Towards Data Science .