ZhichunRoad at Amazon KDD Cup 2022: MultiTask Pre-Training for E-Commerce Product Search

Xuange Cui, Wei Xiong, Songlin Wang
1/31/2023
cs.CL

No official code found. Best implementation: cuixuage/KDDCup2022-ESCI (22 stars, last commit 2022-08-23).

Abstract

In this paper, we propose a robust multilingual model to improve the quality of search results. Our model not only leverage the processed class-balanced dataset, but also benefit from multitask pre-training that leads to more general representations. In pre-training stage, we adopt mlm task, classification task and contrastive learning task to achieve considerably performance. In fine-tuning stage, we use confident learning, exponential moving average method (EMA), adversarial training (FGM) and regularized dropout strategy (R-Drop) to improve the model's generalization and robustness. Moreover, we use a multi-granular semantic unit to discover the queries and products textual metadata for enhancing the representation of the model. Our approach obtained competitive results and ranked top-8 in three tasks. We release the source code and pre-trained models associated with this work.

View on arXivView PDF

Code Implementations(1)

cuixuage/KDDCup2022-ESCICommunity100%

Oral presentation at KDD Cup Workshop @ KDD 2022

223Shell, PythonJul 30, 20224 years ago

Similar Papers

Enhancing Model Performance in Multilingual Information Retrieval with Comprehensive Data Engineering Techniques

88% match2023

HARGO: Heterogeneity-Aware Reward-Guided Optimization for RL Post-Training of LLMs on HPC Tasks

85% match2026

From Data Deluge to Data Curation: A Filtering-WoRA Paradigm for Efficient Text-based Person Search

85% match1 repo92024

m3BERT: A Modern, Multi-lingual, Matryoshka Bidirectional Encoder

85% match1 repo2026

Adapting English Quality Classifiers for Multilingual LLM Pretraining Data Selection

84% match2026

MIA 2022 Shared Task Submission: Leveraging Entity Representations, Dense-Sparse Hybrids, and Fusion-in-Decoder for Cross-Lingual Question Answering

84% match2022

Cite this paper

@article{cui2023zhichunroad,
  title  = {ZhichunRoad at Amazon KDD Cup 2022: MultiTask Pre-Training for E-Commerce Product Search},
  author = {Xuange Cui and Wei Xiong and Songlin Wang},
  year   = {2023},
  eprint = {2301.13455},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2301.13455v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2301.13455.svg)](https://codeofpaper.com/paper/2301.13455)

Discussion