Ajou University repository

Unsupervised Contrastive Learning of Sentence Embeddings Through Optimized Sample Construction and Knowledge Distillation
Citations

SCOPUS

0

Citation Export

Publication Year
2024-01-01
Journal
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Publisher
Springer Science and Business Media Deutschland GmbH
Citation
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol.14326 LNAI, pp.375-381
Keyword
Contrastive LearningDistillationSemantic SimilarityUnsupervised Sentence Embedding
Mesh Keyword
Augmentation methodsContrastive learningData augmentationData enhancementEmbeddingsNegative samplesPositive examplesSemantic similarityText similarityUnsupervised sentence embedding
All Science Classification Codes (ASJC)
Theoretical Computer ScienceComputer Science (all)
Abstract
Unsupervised contrastive learning of sentence embedding has been a recent focus of researchers. However, issues such as unreasonable division of positive and negative samples and poor data enhancement leading to text semantic changes still exist. We propose an optimized data augmentation method that combines contrastive learning’s data augmentation with unsupervised sentence pair modelling’s distillation. Our data augmentation uses in-sentence tokens for positive examples and text similarity for negative examples, while the distillation is conducted without supervised pairs. Experimental results on the STS task show that our method achieves a Spearman correlation of 81.03%, outperforming existing STS benchmarks.
Language
eng
URI
https://aurora.ajou.ac.kr/handle/2018.oak/37111
https://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=85177450839&origin=inward
DOI
https://doi.org/10.1007/978-981-99-7022-3_35
Journal URL
https://www.springer.com/series/558
Type
Conference
Show full item record

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.

Related Researcher

Chung, Tae-Sun Image
Chung, Tae-Sun정태선
Department of Software and Computer Engineering
Read More

Total Views & Downloads

File Download

  • There are no files associated with this item.