English
Related papers

Related papers: Multimodal Transformer for Nursing Activity Recogn…

200 papers

Recognizing human activity plays a significant role in the advancements of human-interaction applications in healthcare, personal fitness, and smart devices. Many papers presented various techniques for human activity representation that…

Computer Vision and Pattern Recognition · Computer Science 2023-11-22 Reem Abdel-Salam , Rana Mostafa , Mayada Hadhood

In the human activity recognition research area, prior studies predominantly concentrate on leveraging advanced algorithms on public datasets to enhance recognition performance, little attention has been paid to executing real-time kitchen…

Signal Processing · Electrical Eng. & Systems 2024-09-11 Mengxi Liu , Sungho Suh , Juan Felipe Vargas , Bo Zhou , Agnes Grünerbl , Paul Lukowicz

In this study, we introduce a multi-modal approach that efficiently integrates multi-scale clinical and dermoscopy features within a single network, thereby substantially reducing model parameters. The proposed method includes three novel…

Image and Video Processing · Electrical Eng. & Systems 2024-03-31 Peng Tang , Tobias Lasser

With the development of the Internet of Things(IoT) and Artificial Intelligence(AI) technologies, human activity recognition has enabled various applications, such as smart homes and assisted living. In this paper, we target a new…

Machine Learning · Computer Science 2021-06-30 Rex Liu , Sarina A Fazio , Huanle Zhang , Albara Ah Ramli , Xin Liu , Jason Yeates Adams

In smart healthcare, Human Activity Recognition (HAR) is considered to be an efficient model in pervasive computation from sensor readings. The Ambient Assisted Living (AAL) in the home or community helps the people in providing independent…

Machine Learning · Computer Science 2021-11-22 Pankaj Khatiwada , Ayan Chatterjee , Matrika Subedi

In the field of medical imaging, AI-assisted techniques such as object detection, segmentation, and classification are widely employed to alleviate the workload of physicians and doctors. However, single-task models are predominantly used,…

Image and Video Processing · Electrical Eng. & Systems 2025-11-18 Fan Li , Arun Iyengar , Lanyu Xu

The increase in world elderly population has significantly underlined the need for continuous health care measurement, specifically in rehabilitation monitoring. The new technologies has enabled people to have in home healthcare services,…

Human-Computer Interaction · Computer Science 2020-10-20 Shayan Shokri , Shane Ward , Pierre-Amaury M. Anton , Paolo Siffredi , Guglielmo Papetti

The diagnosis of medical diseases faces challenges such as the misdiagnosis of small lesions. Deep learning, particularly multimodal approaches, has shown great potential in the field of medical disease diagnosis. However, the differences…

Computer Vision and Pattern Recognition · Computer Science 2025-08-07 Jianxun Yu , Ruiquan Ge , Zhipeng Wang , Cheng Yang , Chenyu Lin , Xianjun Fu , Jikui Liu , Ahmed Elazab , Changmiao Wang

Purpose: Automatic segmentation and classification of surgical activity is crucial for providing advanced support in computer-assisted interventions and autonomous functionalities in robot-assisted surgeries. Prior works have focused on…

Computer Vision and Pattern Recognition · Computer Science 2023-02-23 Sanat Ramesh , Diego Dall'Alba , Cristians Gonzalez , Tong Yu , Pietro Mascagni , Didier Mutter , Jacques Marescaux , Paolo Fiorini , Nicolas Padoy

The integration of multi-modal Magnetic Resonance Imaging (MRI) and clinical data holds great promise for enhancing the diagnosis of neurological disorders (NDs) in real-world clinical settings. Deep Learning (DL) has recently emerged as a…

Image and Video Processing · Electrical Eng. & Systems 2025-06-19 Wajih Hassan Raza , Aamir Bader Shah , Yu Wen , Yidan Shen , Juan Diego Martinez Lemus , Mya Caryn Schiess , Timothy Michael Ellmore , Renjie Hu , Xin Fu

This paper investigates the optimal selection and fusion of feature encoders across multiple modalities and combines these in one neural network to improve sentiment detection. We compare different fusion methods and examine the impact of…

Computation and Language · Computer Science 2024-06-04 Zehui Wu , Ziwei Gong , Jaywon Koo , Julia Hirschberg

Chest imaging plays an essential role in diagnosing and predicting patients with COVID-19 with evidence of worsening respiratory status. Many deep learning-based approaches for pneumonia recognition have been developed to enable…

Image and Video Processing · Electrical Eng. & Systems 2024-01-17 Shengchao Chen , Sufen Ren , Guanjun Wang , Mengxing Huang , Chenyang Xue

Physical rehabilitation exercises suggested by healthcare professionals can help recovery from various musculoskeletal disorders and prevent re-injury. However, patients' engagement tends to decrease over time without direct supervision,…

Human-Computer Interaction · Computer Science 2025-04-22 Aleksa Marusic , Sao Mai Nguyen , Adriana Tapus

In recent years, the widespread adoption of wearable devices has highlighted the growing importance of behavior analysis using IMU. While applications span diverse fields such as healthcare and robotics, recent studies have increasingly…

Computer Vision and Pattern Recognition · Computer Science 2025-06-05 Koki Matsuishi , Kosuke Ukita , Tsuyoshi Okita

Accurate classification of medical device risk levels is essential for regulatory oversight and clinical safety. We present a Transformer-based multimodal framework that integrates textual descriptions and visual information to predict…

Machine Learning · Computer Science 2025-05-02 Yu Han , Aaron Ceross , Jeroen H. M. Bergmann

Human-machine interaction has been around for several decades now, with new applications emerging every day. One of the major goals that remain to be achieved is designing an interaction similar to how a human interacts with another human.…

Human-Computer Interaction · Computer Science 2022-12-27 Tauheed Khan Mohd , Nicole Nguyen , Ahmad Y Javaid

Endowing the robotic systems with cognitive capabilities for recognizing daily activities of humans is an important challenge, which requires sophisticated and novel approaches. Most of the proposed approaches explore pattern recognition…

Robotics · Computer Science 2018-10-02 Hazem Abdelkawy , Naouel Ayari , Abdelghani Chibani , Yacine Amirat , Ferhat Attal

Devices and sensors for identification of fallers can be used to implement actions to prevent falls and to allow the elderly to live an independent life while reducing the long-term care costs. In this study we aimed to investigate the…

Computers and Society · Computer Science 2017-04-13 Moacir Ponti , Patricia Bet , Caroline Oliveira , Paula C. Castro

Breast cancer screening, primarily conducted through mammography, is often supplemented with ultrasound for women with dense breast tissue. However, existing deep learning models analyze each modality independently, missing opportunities to…

Image and Video Processing · Electrical Eng. & Systems 2023-11-16 Yiqiu Shen , Jungkyu Park , Frank Yeung , Eliana Goldberg , Laura Heacock , Farah Shamout , Krzysztof J. Geras

Surface material recognition is a key component in robotic perception and physical interaction, particularly when leveraging both tactile and visual sensory inputs. In this work, we propose Surformer v1, a transformer-based architecture…

Computer Vision and Pattern Recognition · Computer Science 2025-08-12 Manish Kansana , Elias Hossain , Shahram Rahimi , Noorbakhsh Amiri Golilarz