ACM Transactions on Multimedia Computing Communications and Applications · 2022 · 10 citations · 14 references
ComputingComputer ScienceArtificial IntelligenceMachine LearningNatural Language ProcessingDeep LearningLyrics AnalysisMelodic AnalysisSpeech SynthesisText GenerationAlgorithmic Melody GenerationInterpretabilityLyric InterpretationLyrics SemanticsMelody ExtractionModel InterpretabilitySemantic FeaturesTraining
Melody generation aims to learn the distribution of real melodies to generate new melodies conditioned on lyrics, which has been a very interesting topic in the area of artificial intelligence and music. However, a challenging issue still limits the quality and reliability of melody generation conditioned on lyrics: how to enhance the interpretability between the input lyrics and generated melodies so humans can understand their relationships. To solve this issue, in this article, we propose a model for melody generation from lyrics with local interpretability, which contains two significant contributions: (i) Mutual information between input lyrics and generated melody is exploited to instruct the training of the network, which avoids the loss of content consistency during the training stage. (ii) Transformer is explored to efficiently extract semantic features from lyrics sequences, which provides more interpretable correlations between different syllables in lyrics. Experiments on a large-scale dataset with paired lyrics-melodies demonstrate that the proposed approach can generate higher-quality melodies from lyrics compared with existing methods.
14
Sepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 98.1K citations
Generative Adversarial Networks
arXiv (Cornell University) · 2014 · 4.6K citations · Full text
Interpretable machine learning: Fundamental principles and 10 grand challenges
Cynthia Rudin, Chaofan Chen, Zhi Chen et al. · Statistics Surveys · 2022 · 844 citations · Full text
48th Annual Meeting of the Association for Computational Linguistics
R.A. Ittoo, Gosse Bouma · 2010 · 224 citations