SSIG and IRISA at Multimodal Person Discovery

Cassio Elias dos Santos Jr. (1) , Guillaume Gravier (2) , William Robson Schwartz (1)

1 DCC - UFMG - Departamento de Ciência da Computação [Minas Gerais]
2 LinkMedia - Creating and exploiting explicit links between multimedia fragments

This paper describes our approach and results in the multi-modal person discovery in broadcast TV task at MediaEval 2015. We investigate two distinct aspects of multimodal person discovery. One refers to face clusters, which are considered to propagate names associated to faces in one shot to other faces that probably belong to the same person. The face clustering approach consists in calculating face similarities using partial least squares (PLS) and a simple hierarchical approach. The other aspect refers to tag propagation in a graph-based approach where nodes are speaking faces and edges link similar faces/speakers. The advantage of the graph-based tag propagation is to not rely on face/speaker clustering, which we believe can be errorprone.

Domaines: Multimédia [cs.MM]; Son [cs.SD]; Traitement du signal et de l'image [eess.SP]; Traitement du texte et du document; Vision par ordinateur et reconnaissance de formes [cs.CV]; Apprentissage [cs.LG]

Cassio Elias dos Santos Jr., Guillaume Gravier, William Robson Schwartz. SSIG and IRISA at Multimodal Person Discovery. Working Notes Proceedings of the MediaEval Workshop, 2015, Wurzen, Germany. ⟨hal-01196171⟩