OpenAI Blog
Prompt Caching in the API
Read the original on OpenAI Blog →Offering automatic discounts on inputs that the model has recently seen
Summary generated by The Flow from the publisher's feed. The full article lives at OpenAI Blog.
Related stories
arXiv AI
Jul 24
MiniCache: Reusable Program Caching with Small Model Interfaces for Efficient LLM Inference
arXiv:2607. 20507v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used for program-aided reasoning, agentic decision making, and structured task execution, but these applications often incur high inference cost.
Hugging Face Trending Papers
Aug 5
Fewer Tokens, Smaller Cache: Reward-Coordinated Efficient Reasoning
Large Reasoning Models (LRMs) excel on complex tasks through long chain-of-thought (CoT) reasoning, but their lengthy intermediate steps cause severe overthinking that inflates inference cost. KV-cache compression is a common solution, yet existing reasoning-oriented methods apply a uniform policy across the trajectory and judge compression only by what it removes from the cache.
Hugging Face Trending Papers
Jun 8
DynaCF: Mitigating Shortcut Learning in Reward Models via Dynamic Counterfactual Sensitivity
Reward models trained from pairwise preferences often exploit superficial shortcut cues rather than learning true response quality. We propose DynaCF, a dynamic reweighting framework for mitigating shortcut learning in reward model training.
arXiv AI
Aug 6
Fewer Tokens, Smaller Cache: Reward-Coordinated Efficient Reasoning
arXiv:2608. 04771v1 Announce Type: new Abstract: Large Reasoning Models (LRMs) excel on complex tasks through long chain-of-thought (CoT) reasoning, but their lengthy intermediate steps cause severe overthinking that inflates inference cost.
arXiv Machine Learning
Jun 9
DynaCF: Mitigating Shortcut Learning in Reward Models via Dynamic Counterfactual Sensitivity
arXiv:2606. 09043v1 Announce Type: new Abstract: Reward models trained from pairwise preferences often exploit superficial shortcut cues rather than learning true response quality.
arXiv AI
6d ago
Keep the Future, Drop the Rollout: RIFT for World Action Models
arXiv:2608. 11521v1 Announce Type: cross Abstract: World action models (WAMs) condition robot actions on predicted futures, but iterative video rollout increases deployment latency.