arXiv (Cornell University) · 2021 · 38 citations · 31 references
ComputingCommunicationComputer ScienceKnowledge RepresentationNatural Language ProcessingSemanticsContext DependenceContext-aware ComputingDialogue SystemsMeaning RepresentationNatural Language UnderstandingCommunication EffectivenessCommunication ReliabilityCommunication SemanticsContext RepresentationConversational AgentsHuman CommunicationSpeech UnderstandingVirtual Agent Interaction
Spurred by a huge interest in the post-Shannon communication, it has recently been shown that leveraging semantics can significantly improve the communication effectiveness across many tasks. In this article, inspired by human communication, we propose a novel stochastic model of System 1 semantics-native communication (SNC) for generic tasks, where a speaker has an intention of referring to an entity, extracts the semantics, and communicates its symbolic representation to a target listener. To further reach its full potential, we additionally infuse contextual reasoning into SNC such that the speaker locally and iteratively self-communicates with a virtual agent built on the physical listener's unique way of coding its semantics, i.e., communication context. The resultant System 2 SNC allows the speaker to extract the most effective semantics for its listener. Leveraging the proposed stochastic model, we show that the reliability of System 2 SNC increases with the number of meaningful concepts, and derive the expected semantic representation (SR) bit length which quantifies the extracted effective semantics. It is also shown that System 2 SNC significantly reduces the SR length without compromising communication reliability.
31
A Mathematical Theory of Communication
Claude E. Shannon · Bell System Technical Journal · 1948 · 80.7K citations
Distilling the Knowledge in a Neural Network
Geoffrey E. Hinton, Oriol Vinyals · arXiv (Cornell University) · 2015 · 14K citations · Full text
<i>The Mathematical Theory of Communication</i>
Claude E. Shannon, Warren Weaver, Norbert Wiener · Physics Today · 1950 · 9.9K citations
Sinkhorn Distances: Lightspeed Computation of Optimal Transport
Marco Cuturi · 2013 · 2.2K citations
Deep Learning Enabled Semantic Communication Systems
Huiqiang Xie, Zhijin Qin, Geoffrey Ye Li et al. · IEEE Transactions on Signal Processing · 2021 · 1.3K citations · Full text