Towards Data Science
Jun 11
PySpark for Beginners: Beyond the Basics
Take the next step to building real workflows with Spark on your laptop The post PySpark for Beginners: Beyond the Basics appeared first on Towards Data Science .
PySpark for Beginners: Building Intermediate-Level Skills
A practical next step into partitions, shuffles, joins, caching, and execution plans. The post PySpark for Beginners: Building Intermediate-Level Skills appeared first on Towards Data Science .
Related stories
Towards Data Science
Sep 2
A Practical Introduction to PySpark Window Functions
The article titled "A Practical Introduction to PySpark Window Functions" explains why the standard groupBy function isn’t enough for certain data processing tasks. It introduces PySpark window functions as a more powerful alternative, providing readers with a practical guide to implementing these functions in their data workflows.
OpenAI Blog
Jul 14
Data science workflows with ChatGPT Work
Learn practical ChatGPT Work workflows for root-cause briefs, KPI memos, scoped analyses, and dashboard specifications.
arXiv Computer Vision
Aug 25
VersaDB: A High-Performance AI Storage Database for Unifying Mutimodal Datasets
arXiv:2608.22795v1 Announce Type: new Abstract: The AI field has been rapidly developing, leading to the emergence of a large number of AI training datasets of various types. These datasets contain d...
arXiv AI
Jun 17
Querying an astronomical database using large language models: the ALeRCE text-to-SQL system
arXiv:2606. 18108v1 Announce Type: cross Abstract: We develop a text-to-SQL (structured query language) system based on large language models (LLMs) using in-context learning and apply it to the Automatic Learning for the Rapid Classification of Events (ALeRCE) astronomical database.
arXiv Machine Learning
Aug 18
PandasCorpus: A Resource of Real-World Pandas Workflows and Usage Patterns
arXiv:2608. 14742v1 Announce Type: cross Abstract: Pandas has emerged as the de facto library for data processing and machine learning, widely used for tasks, such as data loading, transformation, and analysis.
arXiv AI
Jun 30
Beyond IID: How General Are Tabular Foundation Models, Really?
arXiv:2606. 30410v1 Announce Type: cross Abstract: Foundation models for predictive machine learning on tabular data have recently gained significant traction in academia and industry.
OpenAI Blog
Feb 12
Introducing GPT-5.3-Codex-Spark
Introducing GPT-5. 3-Codex-Spark—our first real-time coding model.
Hugging Face Trending Papers
Jun 13
CODA-BENCH: Can Code Agents Handle Data-Intensive Tasks?
Advanced agents are increasingly demonstrating the potential to operate as autonomous engineers, creating a growing demand for evaluation benchmarks that capture the complexity of real-world development. Such environments typically involve both complex code and large-scale data (i.
arXiv Machine Learning
Aug 6
stratum: A System Infrastructure for Massive Agent-Centric ML Workloads
arXiv:2603. 03589v3 Announce Type: replace-cross Abstract: Recent advances in large language models (LLMs) transform how machine learning (ML) pipelines are developed and evaluated.
arXiv AI
Aug 6
CheMLFlow: An Open-Source Platform for Cheminformatics and Materials Informatics Applications
arXiv:2608. 04942v1 Announce Type: cross Abstract: CheMLFlow is an open-source platform for building and executing end-to-end, high-throughput, and agentic workflows for scientific and technological applications.
arXiv AI
Jun 30
Experience Graphs: The Data Foundation for Self-Improving Agents
arXiv:2606. 29823v1 Announce Type: cross Abstract: The database community has repeatedly advanced the state of the art by recognizing that new workloads demand new system architectures.