[KDD 2020] Comprehensive Information Integration Modeling Framework for Video Titling - CrossMinds.ai
[KDD 2020] Comprehensive Information Integration Modeling Framework for Video Titling
Aug 13, 202021 views
Jingren Zhou
In e-commerce, consumer-generated videos, which in general deliver consumers’ individual preferences for the different aspects,of certain products, are massive in volume. To recommend these,videos to potential consumers more effectively, diverse and catchy,video titles are critical. However, consumer-generated videos seldom accompany appropriate titles. To bridge this gap, we integrate comprehensive sources of information, including the content,of consumer-generated videos, the narrative comment sentences,supplied by consumers, and the product attributes, in an end-toend modeling framework. Although automatic,video titling,is very,useful and demanding, it is much less addressed than video captioning. The latter focuses on generating sentences that describe,videos as a whole while our task requires the product-aware multigrained video analysis. To tackle this issue, the proposed method,consists of two processes,,i,.,e,., granular-level interaction modeling,and abstraction-level story-line summarization. Specifically, the,granular-level interaction modeling first utilizes temporal-spatial,landmark cues, descriptive words, and abstractive attributes to,builds three individual graphs and recognizes the intra-actions,in each graph through Graph Neural Networks (GNN). Then the,global-local aggregation module is proposed to model inter-actions,across graphs and aggregate heterogeneous graphs into a holistic graph representation. The abstraction-level story-line summarization further considers both frame-level video features and the,holistic graph to utilize the interactions between products and backgrounds, and generate the story-line topic of the video. We collect,a large-scale dataset accordingly from real-world data in Taobao, a,world-leading e-commerce platform, and will make the desensitized,version publicly available to nourish further development of the,research community,1,. Relatively extensive experiments on various,datasets demonstrate the efficacy of the proposed method.
Applied Research