Ulster University Logo

Integration of text and image analysis for flood event image recognition

Jing, Min, Bryan, Scotney, Coleman, Sonya, McGinnity, T.Martin, Zhang, Xiubo, Kelly, Stephen, Ahmad, Khurshid, Schlaf, Antje, Gr¨under-Fahrer, Sabine and Heyer, Gerhard (2016) Integration of text and image analysis for flood event image recognition. In: Signals and Systems Conference (ISSC), 2016 27th Irish. IEEE. 6 pp. [Conference contribution]

[img] Text - Published Version

DOI: 10.1109/ISSC.2016.7528454


Flood event monitoring plays an important role for emergency management. With the fast growth of social media, a large number of images and videos are uploaded and searched on the internet during disasters, which can be used as “sensors” for improving efficiency of emergency management. This work proposes a novel framework in which the rich information available from social media is incorporated with image analysis to enhance image retrieval for disaster management. The text associated with images of flooding events was used to extract prominent words associated with flooding. The image features are represented by a histogram of visual words obtained using the Bag-of-Words (BoW) model. The text and image analysis are integrated at the feature level, in which the text features are conjoined directly with image features. The proposed approach was evaluated based on two flood event corpuses obtained from the US Federal Emergency Management Agency media library and public Facebook pages and groups related to flood and flood aid (in German). The experimental results demonstrate the improved performance of image recognition after incorporating the text features, which suggests the potential to enhance the efficiency of emergency management.

Item Type:Conference contribution (Speech)
Keywords:flood event image recognition; social media analysis; multimodal data fusion; emergency management
Faculties and Schools:Faculty of Computing & Engineering
Faculty of Computing & Engineering > School of Computing and Information Engineering
Faculty of Computing & Engineering > School of Computing and Intelligent Systems
Research Institutes and Groups:Computer Science Research Institute > Intelligent Systems Research Centre
Computer Science Research Institute
Computer Science Research Institute > Information and Communication Engineering
ID Code:40045
Deposited By: Dr Min Jing
Deposited On:25 Apr 2018 11:34
Last Modified:25 Apr 2018 11:34

Repository Staff Only: item control page