IEEE Transactions on Cognitive Communications and Networking · 2023 · 51 citations · 28 references
ComputingCommunicationComputer ScienceArtificial IntelligenceMachine LearningSemanticsContext DependenceContext-aware ComputingCommunication EfficiencyContext InferenceContext ReasoningContext RepresentationConversational AgentsHuman CommunicationSymbol MappingVirtual Agent Interaction
Machine learning has improved communication efficiency by reinstating the semantics of bits, and human communication shows that reasoning about others' contexts—contextual reasoning—enables more efficient exchanges. The authors propose a novel stochastic model of semantic communication, termed semantics‑native communication (SNC), and a framework that uses agent states as side information to enable iterative, local communication between a speaker and a virtual listener. The framework models the speaker and listener as agents that iteratively exchange symbols while incorporating each other's inferred state as side information, effectively performing contextual reasoning. The authors prove that contextual reasoning converges to a minimal bit length while meeting a target reliability, and simulations show that SNC with contextual reasoning substantially reduces bit lengths and remains robust to imperfect agent states through quantized mapping.
Recently, machine learning (ML) has shown its effectiveness in improving communication efficiency by reinstating the semantics of bits. To understand its underlying principles, we propose a novel stochastic model of semantic communication, dubbed semantics-native communication (SNC). Inspired from human communication, we consider a point-to-point SNC scenario where a speaker has an intention of referring to an entity, extracts its semantic concepts, and maps these concepts into symbols that are communicated to a target listener through the traditional Shannon communication channel. Next, we recall rational humans who can communicate more efficiently by reasoning about the others' contexts before communication, referred to as contextual reasoning. This motivates us to propose a novel SNC framework harnessing agent states as side information in a way that the speaker locally and iteratively communicates with a virtual agent having the listener's state, and vice versa. Theoretically, we prove the convergence of contextual reasoning, at which it minimizes the bit length while guaranteeing a target reliability. Simulation results corroborate that contextual reasoning based SNC can significantly reduce bit lengths, and be a robust solution to imperfect agent states by quantizing the entity-concept-symbol mapping before contextual reasoning.
28
A Mathematical Theory of Communication
Claude E. Shannon · Bell System Technical Journal · 1948 · 80.7K citations
Distilling the Knowledge in a Neural Network
Geoffrey E. Hinton, Oriol Vinyals · arXiv (Cornell University) · 2015 · 14K citations · Full text
Reinforcement Learning: An Introduction
Jeffrey D. Johnson, Jinghong Li, Zengshi Chen · Neurocomputing · 2000 · 8.7K citations
Sinkhorn Distances: Lightspeed Computation of Optimal Transport
Marco Cuturi · 2013 · 2.2K citations
Deep Learning Enabled Semantic Communication Systems
Huiqiang Xie, Zhijin Qin, Geoffrey Ye Li et al. · IEEE Transactions on Signal Processing · 2021 · 1.3K citations · Full text