The paper introduces Workload Aware Column Imprint-Hash Join (WACI‑HJ), a new edge query processing technique that predicts upcoming workloads to pre‑compute hash bins, reducing unnecessary cacheline scans. Compared to the prior CI‑HJ method, WACI‑HJ cuts cacheline reads by 54% and improves query execution time by 10%, while also delivering measurable gains in CPU, RAM, and I/O energy efficiency. Experiments on benchmark and Smart Transportation datasets demonstrate its effectiveness for both scaled and skewed data, enabling faster real‑time traffic analysis and congestion management.
arXiv:2606. 11257v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) pipelines are compute-intensive, combining embedding, retrieval, reranking, and large language model (LLM) generation.
The paper evaluates seven graph database engines, including Corvic AI, on a synthetic biomedical property graph with 1.02 million nodes and 5.34 million rows. It benchmarks query latency, bulk‑ingest throughput, point‑update latency, and correctness across a twenty‑query workload that covers neighborhood lookups, bounded paths, set intersections, anti‑joins, aggregation, ranking, temporal filters, full scans, and relational joins. The study finds that no single engine is universally fastest; performance depends on query shape, and the largest cost difference arises from bulk‑ingest throughput, which varies by three orders of magnitude and dominates total cost for workloads with fewer than about 10⁵ queries per data refresh.
arXiv:2601. 07048v5 Announce Type: replace-cross Abstract: Approximate nearest neighbor search (ANNS) is a core problem in machine learning and information retrieval applications.
arXiv:2610.08268v1 Announce Type: cross Abstract: Pervasive intelligent applications are increasingly deployed on mobile and Internet of Things (IoT) edge devices. Consequently, Large Language Models...
SemanticXR is a device‑cloud system that enables real‑time, open‑vocabulary semantic mapping and querying for XR applications while respecting power, bandwidth, and memory limits. By treating semantically identifiable objects as first‑class units, the system coordinates communication, execution, and memory across device and server, achieving a 2.2× faster server‑side mapping latency and keeping upstream bandwidth below 2.5 Mbps. On the device, an object‑level sparse local map with incremental updates delivers sub‑100 ms query latency for up to 10,000 objects, supports tens of thousands of objects within a 500 MB footprint, and adds only about 2 % to idle power.
arXiv:2609.05760v1 Announce Type: cross Abstract: We present RAGMark, a modular benchmarking framework for advanced Retrieval-Augmented Generation (RAG) systems targeting small-scale multi-GPU enviro...
arXiv:2608. 11688v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) models are attractive for edge deployment because they provide high model capacity while activating only a small subset of parameters per token, improving compute efficiency.
arXiv:2609.08307v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as backends for intelligent web services, but serving them across the edge continuum requires bala...
The paper introduces TopoCompress, a token compression framework designed for distributed edge Mixture-of-Experts (MoE) inference. It jointly optimizes token compression, expert deployment, GPU-CPU residency, and routing to reduce cross-server communication and resource usage. The method uses a two-timescale alternating optimization, with an online loop compressing low-importance tokens and an offline loop updating expert placement based on accumulated traffic.
The paper introduces ReadKV, a query‑adaptive quantization method for KV caches that stores keys and values in progressive codes, allowing each decoding query to allocate different precision prefixes for keys and values. By optimizing a calibrated distortion objective under fixed budgets, ReadKV achieves near‑optimal allocation and demonstrates that query‑dependent access can outperform any query‑independent reader at the same read budget. Experiments on six base models show that reading only four bits on average from an eight‑bit cache increases perplexity by at most 0.66% while reducing logical reads and retained capacity compared to a 16‑bit cache, and it achieves lower latency than existing codecs on an NVIDIA A10G.
The paper introduces BEA, a fine‑grained estimator that predicts the time‑to‑first‑token (TTFT) for language model queries by replaying token‑batch formation based on current workload snapshots. BEA achieves about 5% error with 0.1 ms prediction time, enabling the latency‑aware router BEAR to balance accuracy, cost, and latency across heterogeneous LLM pools. BEAR outperforms seven baselines, improving utility of on‑target responses by 63% on average and by 30.5% at each arrival rate.
SparKV is an adaptive framework for on-device large language model inference that intelligently decides whether to stream or compute each key-value (KV) cache chunk, overlapping communication and computation to cut latency. It dynamically adjusts offline-generated schedules at runtime to cope with changing wireless connectivity and edge resource availability. Experiments show SparKV can reduce Time-to-First-Token by 1.3×–5.1× and lower per-request energy consumption by 1.5×–3.3× with minimal impact on response quality.