Multi-label classification by formulating label-specific features from simultaneous instance level and feature level

Multi-label learning (MLL) trains a classification model from multiple labelled datasets, where each training instance is annotated with a set of class labels simultaneously. Following the binary relevance MLL paradigm, a recently effective spirit is to constructing specific features for each label, instead of training over the original feature space. Existing label-specific methods, however, only consider the information from instance distributions, making the reconstructed features poorly discriminative. In this paper, we propose the generation of Label-spEcific feaTures by simultaneously exploring insTance distributions and fEatuRe distributions, and suggest a new method named L etter . L etter reconstructs two subsets of new features from the instance level and feature level, respectively. More concretely, from the instance level, L etter incorporates a sparse constraint, and from the feature level, we cluster the original features to construct new features as an extension. The combination of these two new feature subsets is the final set of label-specific features. Extensive experiments on a total of 14 benchmark datasets verify the competitive performance of L etter against the existing state-of-the-art MLL methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic €32.70 /Month

Buy Now

Price includes VAT (France)

Instant access to the full article PDF.

Rent this article via DeepDyve

Similar content being viewed by others

Learning label-specific features with global and local label correlation for multi-label classification

Article 19 May 2022

Robust multi-label feature selection with shared label enhancement

Article 01 September 2022

Label distribution feature selection based on label-specific features

Article 11 July 2024

Explore related subjects

Notes

|⋅| returns the set cardinality. We employ the Euclidean as the metric in this paper.

References

  1. Boutell MR, Luo J, Shen X, Brown CM (2004) Learning multi-label scene classification. Pattern Recogn 37(9):1757–1771 ArticleGoogle Scholar
  2. Cesa-Bianchi N, Re M, Valentini G (2012) Synergy of multi-label hierarchical ensembles, data fusion, and cost-sensitive methods for gene functional inference. Mach Learn 88(1-2):209–241 ArticleMathSciNetGoogle Scholar
  3. Chang CC, Lin CJ (2011) Libsvm: a library for support vector machines. ACM Trans Intell Syst Technol 2(3):1–27 ArticleGoogle Scholar
  4. Chen Y, Zhang F, Zuo W (2018) Deep image annotation and classification by fusing multi-modal semantic topics. KSII Trans Internet Inf Syst 12(1):392–412 Google Scholar
  5. Cheng W, Hüllermeier E (2009) Combining instance-based learning and logistic regression for multilabel classification. Mach Learn 76(2-3):211–225 ArticleGoogle Scholar
  6. Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30 MathSciNetMATHGoogle Scholar
  7. Elisseeff A, Weston J (2002) A kernel method for multi-labelled classification. In: Advances in neural information processing systems, pp 681–687
  8. Fürnkranz J, Hüllermeier E, Mencía EL, Brinker K (2008) Multilabel classification via calibrated label ranking. Mach Learn 73(2):133–153 ArticleGoogle Scholar
  9. Gong T, Liu B, Chu Q, Yu N (2019) Using multi-label classification to improve object detection. Neurocomputing 370:174–185 ArticleGoogle Scholar
  10. Guo Y, Chung F, Li G, Wang J, Gee JC (2019) Leveraging label-specific discriminant mapping features for multi-label learning. ACM Trans Knowl Discov Data 13(2):24 ArticleGoogle Scholar
  11. He ZF, Yang M (2019) Sparse and low-rank representation for multi-label classification. Appl Intell 49(5):1708–1723 ArticleGoogle Scholar
  12. Huang J, Li G, Huang Q, Wu X (2015) Learning label specific features for multi-label classification. In: IEEE International conference on data mining, pp 181–190
  13. Huang J, Li G, Huang Q, Wu X (2016) Learning label-specific features and class-dependent labels for multi-label classification. IEEE Trans Knowl Data Eng 28(12):3309–3323 ArticleGoogle Scholar
  14. Huang J, Li G, Huang Q, Wu X (2017) Joint feature selection and classification for multilabel learning. IEEE Trans Cybern 48(3):876–889 ArticleGoogle Scholar
  15. Jabreel M, Moreno A (2019) A deep learning-based approach for multi-label emotion classification in tweets. Appl Sci 9(6): 1123 ArticleGoogle Scholar
  16. Li X, Ouyang J, Zhou X (2015) Centroid prior topic model for multi-label classification. Pattern Recogn Lett 62:8–13 ArticleGoogle Scholar
  17. Li X, Ouyang J, Zhou X (2015) Supervised topic models for multi-label classification. Neurocomputing 149:811–819 ArticleGoogle Scholar
  18. Li X, Ouyang J, Zhou X (2016) Labelset topic model for multi-label document classification. J Intell Inf Syst 46(4):83–97 ArticleGoogle Scholar
  19. Li X, Ouyang J, Zhou X, Lu Y, Liu Y (2015) Supervised labeled latent dirichlet allocation for document categorization. Appl Intell 42(3):581–593 ArticleGoogle Scholar
  20. Li X, Ouyang J, Zou Y (2018) Supervised topic models with weighted words: multi-label document classification. Front Inf Technol Electron Eng 19(4):513–523 ArticleGoogle Scholar
  21. Ma J, Zhang H, Chow TW (2019) Multilabel classification with label-specific features and classifiers: a coarse-and fine-tuned framework. IEEE Trans Cybern:1–15
  22. Read J, Pfahringer B, Holmes G, Frank E (2009) Classifier chains for multi-label classification. In: Machine learning and knowledge discovery in databases, pp 254–269
  23. Read J, Pfahringer B, Holmes G, Frank E (2011) Classifier chains for multi-label classification. Mach Learn 85(3):333 ArticleMathSciNetGoogle Scholar
  24. Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22(8):888–905 ArticleGoogle Scholar
  25. Trohidis K, Tsoumakas G, Kalliris G, Vlahavas I (2008) Multi-label classification of music into emotions. In: International society for music information retrieval, vol 8, pp 325–330
  26. Tsoumakas G, Katakis I, Vlahavas I (2009) Mining multi-label data. In: Data mining and knowledge discovery handbook, pp 667–685
  27. Wang T, Liu L, Liu N, Zhang H, Zhang L, Feng S (2020) A multi-label text classification method via dynamic semantic representation model and deep neural network. Appl Intell:1–13
  28. Wang Y, Rao Y, Zhan X, Chen H, Luo M, Yin J (2016) Sentiment and emotion classification over noisy labels. Knowl-Based Syst 111:207–216 ArticleGoogle Scholar
  29. Wei X, Yu Z, Zhang C, Hu Q (2018) Ensemble of label specific features for multi-label classification. In: IEEE International conference on multimedia and expo, pp 1–6
  30. Weng W, Chen YN, Chen CL, Wu SX, Liu JH (2020) Non-sparse label specific features selection for multi-label classification. Neurocomputing 377:85–94 ArticleGoogle Scholar
  31. Xu S, Yang X, Yu H, Yu DJ, Yang J, Tsang EC (2016) Multi-label learning with label-specific feature reduction. Knowl-Based Syst 104:52–61 ArticleGoogle Scholar
  32. Yu Q, Wang J, Zhang S, Gong Y, Zhao J (2017) Combining local and global hypotheses in deep neural network for multi-label image classification. Neurocomputing 235:38–45 ArticleGoogle Scholar
  33. Zhang J, Li C, Cao D, Lin Y, Su S, Dai L, Li S (2018) Multi-label learning with label-specific features by resolving label correlations. Knowl-Based Syst 159:148–157 ArticleGoogle Scholar
  34. Zhang JJ, Fang M, Li X (2015) Multi-label learning with discriminative features for each label. Neurocomputing 154:305–316 ArticleGoogle Scholar
  35. Zhang ML (2011) L ift : Multi-label learning with label-specific features. In: International joint conference on artificial intelligence, pp 1609–1614
  36. Zhang ML, Li YK, Liu XY, Geng X (2018) Binary relevance for multi-label learning: an overview. Front Comput Sci 12(2):191–202 ArticleGoogle Scholar
  37. Zhang ML, Wu L (2015) L ift : Multi-label learning with label-specific features. IEEE Trans Pattern Anal Mach Intell 37(11):107–120 ArticleGoogle Scholar
  38. Zhang ML, Zhou ZH (2007) Ml-knn: a lazy learning approach to multi-label learning. Pattern Recogn 40(7):2038–2048 ArticleGoogle Scholar
  39. Zhang ML, Zhou ZH (2014) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26(8):1819–1837 ArticleGoogle Scholar
  40. Zhang P, Gao W, Liu G (2018) Feature selection considering weighted relevancy. Appl Intell 48(12):4615–4625 ArticleGoogle Scholar

Acknowledgements

This work was supported by the National Natural Science Foundation of China (NSFC) No.51805203, the Science and Technology Development Plan of Jilin Province 20190201023JC, and the Development and Reform Commission of Jilin Province 2019C054-2.

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Jilin University, Jilin, China Yuanyuan Guan, Wenhui Li, Boxiang Zhang & Manglai Ji
  2. The Northeast Normal University, Jilin, China Bing Han
  1. Yuanyuan Guan