M2E-try On Net: fashion from model to everyone

Zhonghua Wu, Qingyi Tao, Guosheng Lin, Jianfei Cai

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

11 Citations (Scopus)

Abstract

Most existing virtual try-on applications require clean clothes images. Instead, we present a novel virtual Try-On network, M2E-Try On Net, which transfers the clothes from a model image to a person image without the need of any clean product images. To obtain a realistic image of person wearing the desired model clothes, we aim to solve the following challenges: 1) non-rigid nature of clothes - we need to align poses between the model and the user; 2) richness in textures of fashion items - preserving the fine details and characteristics of the clothes is critical for photo-realistic transfer; 3) variation of identity appearances - it is required to fit the desired model clothes to the person identity seamlessly. To tackle these challenges, we introduce three key components, including the pose alignment network (PAN), the texture refinement network (TRN) and the fitting network (FTN). Since it is unlikely to gather image pairs of input person image and desired output image (i.e. person wearing the desired clothes), our framework is trained in a self-supervised manner to gradually transfer the poses and textures of the model's clothes to the desired appearance. In the experiments, we verify on the Deep Fashion dataset and MVC dataset that our method can generate photo-realistic images for the person to try-on the model clothes. Furthermore, we explore the model capability for different fashion items, including both upper and lower garments.

Original languageEnglish
Title of host publicationProceedings of the 27th ACM International Conference on Multimedia
EditorsGuillaume Gravier, Hayley Hung, Chong-Wah Ngo, Wei Tsang Ooi
Place of PublicationNew York NY USA
PublisherAssociation for Computing Machinery (ACM)
Pages293-301
Number of pages9
ISBN (Electronic)9781450368896, 9781450367936
DOIs
Publication statusPublished - 2019
EventACM International Conference on Multimedia 2019 - Nice, France
Duration: 21 Oct 201925 Oct 2019
Conference number: 27th
https://dl.acm.org/doi/proceedings/10.1145/3343031

Publication series

NameMM 2019 - Proceedings of the 27th ACM International Conference on Multimedia

Conference

ConferenceACM International Conference on Multimedia 2019
Abbreviated titleMM 2019
Country/TerritoryFrance
CityNice
Period21/10/1925/10/19
Internet address

Keywords

  • Hybrid Learning
  • Image Generation
  • Virtual Try-on

Cite this