Annotation Performance for multi-channel time series HAR Dataset in Logistics

Abstract

This contribution proposes an approach for annotating human actions and their coarse semantic descriptions for multichannel time-series. For this purpose, a new dataset that consists of Optical Motion Capturing and IMU time-series data for industrial deployment is created and annotated by 6 individuals. The expenditure of time for labelling, both classes and semantic attributes, and the annotation consistency are examined. The initial annotations are revised by a single domain expert to measure its effect on the overall between-individual consistency. Consistency measurements by means of Cohen's κ are analysed. The results give insights on the effort for dataset creation in the field of Human Activity Recognition for industrial application. The Cohen's κ for consistency assessment was moderate and substantial for the initial annotation, and it increased slightly after revision.