---

Human Activity Recognition with Inertial Sensors

---

Latest updates August 2012 Current Developments section added. November 2011 Android application provided for free download in the Implementation section. October 2011 Implementation details added, open source OSGI bundle linked. September 2011 Porting to Android completed. October 2010 Results section and Inference Test added. September 2010 Movie for Ubicomp 2010 available here. September 2010 Slides and papers for ION-GNSS 2010 are available in the publications section. September 2010 This site is online. June 2010 Master thesis work completed and now available here.

General information about Human Motion Related Activity Recognition

Knowledge about the current motion related activity (e.g. Sitting, Standing, Walking, Running, Jumping, Falling and Lying) of a person is information that is required or useful for a number of applications. Technical advances in recent years have reduced prices for sensors capable of providing the necessary motion and pose related signals, in particular MEMS based inertial measurement units (IMUs). In addition to low cost sensors, unobtrusiveness is a requirement for an activity recognition system: We achieve this by mounting one IMU on the belt of the user. Our signal processing approach is a multi-tier one. We first compute features from the raw accelerations and turn rates and use these for classification with Bayesian Network techniques trained from a semi naturalistic, labelled data set.
In the following, you will find the links to download our Java and Android implementations.

Test our Activity Recognition

Results

	Sitting	Standing	Walking	Running	Jumping	Falling	Lying
RECALL	1	0.98	1	0.93	0.93	1	0.98
PRECISION	0.97	1	0.98	1	0.93	0.8	1

These results stem from a four-fold cross-validation of classification with a grid based filter, using a BN with learnt parameters and structure and a manually defined hidden Markov model. Features are computed at 4 Hz, with sliding windows and a recognition delay of 0.5 s taken into account. The data were recorded under semi-naturalistic conditions.

Implementation

The setup installed at DLR (as shown in the video below) is composed of a wired sensor and some computing unit, computing the Bayesian inference. The single components and versions are described shortly in the following:

• Sensor:
  The wired IMU used was the xsens MTx IMU.
  Initially, a C-written driver was used to feed the data to a socket, the Java implemented listened to.
  The current version has implemented the same driver library in Java.
• Java OSGI Implementation:
  This implementation was realised during the EU ICT-Persist project.
• Android Implementation:
  The next step has been the portation to smaller devices, like the Android Galaxy S2.
  The adaptation of the Java code above is straightforward. Download app for free here!
  The connection of the wired IMU, the necessary Android-kernel adaptation and modification of read/write access under Android are described in this Techblog.

Data Set

The data set used to achieve our results was collected from 16 male and female subjects aged between 23 and 50 and anotated manually by an observer. In total it contains about 4.5 hours of annoted activities Sitting, Standing, Walking, Running, Jumping, Falling and Lying.

We provide our labeled dataset - over 4 hours worth of recorded and labeled human activities - which we would like to share with the research community in a zip file about 300 MB here.

The archive contains 'readme' files with details of the data. In case you have any questions, just email us.

Videos and other media

• Ubicomp 2010 Video: Demonstrating activity recognition in three scenarios: Download as [MPG]
• Poster for WIT research day [], [Book of Abstracts]

Publications on Activity Recognition with Inertial Sensors

You can follow the ELIB link to our internal electronic library where you can download the paper as PDF and retrieve citation information, e.g. for BibTeX. Additional papers are available on our pages on related topics (links at the top left of this page).

• Frank, Korbinian (2011) Adaptive and Tractable Bayesian Context Inference for Resource Constrained Devices. PhD Thesis, October 2011. Department of Computing, Mathematics and Physics, Waterford Institute of Technology, Ireland. Supervised by Dr. Tom Pfeifer and Dr. Patrick Robertson. Examined by: Declon O'Sullivan, Trinity College Dublin. Publisher: multicon multimedia consulting, 22 Dec. 2011, Schöneiche, Germany. ISBN 978-3930736188.[bibtex]

• Frank, Korbinian and Vera Nadales, María Josefa and Robertson, Patrick and Pfeifer, Tom (2010) Bayesian Recognition of Motion Related Activities with Inertial Sensors. 12th ACM International Conference on Ubiquitous Computing, 26 - 29 Sep. 2010, Copenhagen, Danmark. ISBN 978-1-4503-0283-8.[bibtex]

• Frank, Korbinian and Vera Nadales, María Josefa and Robertson, Patrick and Angermann, Michael (2010) Reliable Real-Time Recognition of motion related human activities using MEMS inertial sensors. ION GNSS 2010, 21. Sep. - 24. Sep. 2010, Portland, Oregon, USA.
  This paper received the best presentation award in its session at ION. Download paper: [][bibtex] Download slides: []

• Vera Nadales, Maria Josefa (2010) Recognition of Human Motion Related Activities from Sensors. Master's Thesis, University of Malaga, Escuela Técnica Superior de Ingeniería de Telecomunicación. Download: [][bibtex]

• Frank, Korbinian and Roeckl, Matthias and Vera Nadales, Maria Josefa and Robertson, Patrick and Pfeifer, Tom (2010) Comparison of Exact Static and Dynamic Bayesian Context Inference Methods for Activity Recognition. In: Managing Ubiquitous Communications and Services. Seventh IEEE International Workshop, MUCS 2010, Mannheim, Germany, 29th March 2010. Proceedings (12), pp. 41-47. Schöneiche: multicon verlag. 7th International Workshop on Managing Ubiquitous Communications and Services (MUCS 2010) part of PerCom 2010, 29 Mar - 2 Apr 2010, Mannheim, Germany. ISBN 978-3-930736-15-7.[bibtex]

Current Work

Currently we are further developping our work to be even less obtrusive. There are several steps:

• Exchanging the wired one with a wireless IMU. This needs calibration of that IMU and the creation of a Java driver module.
• Transfer of computation processes to the IMU internal CPU. Reduce sending rate and computational demands of smartphone app.

Links

• Stephen Intille from MIT helped us with an accelerometer data set for activity recognition. See also more PlaceLab related activities..
• Wireless Sensor Networks Research Group of Bogazici University with a nice system for the identification of ADLs and in particular Falling for Ambient Assisted living (AAL).
• XSens MTx: the Inertial Measurement Unit (IMU) used for our project

Acknowledgments

The research leading to these results has received funding from the European Community’s Seventh Framework Programme [FP7/2007-2013] under grant agreement no. 215098 of the Persist (PERsonal Self-Improving SmarT spaces) Collaborative Project and from the Irish HEA through the PRTLI cycle 4 project ”Serving Society: Management of Future Communication Networks and Services”. We also want to thank all persons helping to record the data set.