Human Activity Recognition

<p align="center"> <img src="images/Project Logo.png" alt="HAR Logo" width="80px" height="80px"> </p> <h1 align="center"> Human Activity Recognition </h1> <h3 align="center"> A Comparative Study between Different Pre-processing Approaches and Classifiers </h3> </br> <p align="center"> <img src="images/Signal.gif" alt="Sample signal" width="70%" height="70%"> </p> <!-- TABLE OF CONTENTS --> <h2 id="table-of-contents"> :book: Table of Contents</h2> <details open="open"> <summary>Table of Contents</summary> <ol> <li><a href="#about-the-project"> ➤ About The Project</a></li> <li><a href="#prerequisites"> ➤ Prerequisites</a></li> <li><a href="#folder-structure"> ➤ Folder Structure</a></li> <li><a href="#dataset"> ➤ Dataset</a></li> <li><a href="#roadmap"> ➤ Roadmap</a></li> <li> <a href="#preprocessing"> ➤ Preprocessing</a> <ul> <li><a href="#preprocessed-data">Pre-processed data</a></li> <li><a href="#statistical-feature">Statistical feature</a></li> <li><a href="#topological-feature">Topological feature</a></li> </ul> </li> <!--<li><a href="#experiments">Experiments</a></li>--> <li><a href="#results-and-discussion"> ➤ Results and Discussion</a></li> <li><a href="#references"> ➤ References</a></li> <li><a href="#contributors"> ➤ Contributors</a></li> </ol> </details>

<!-- ABOUT THE PROJECT --> <h2 id="about-the-project"> :pencil: About The Project</h2> <p align="justify"> This project focuses on classifying human activities using data collected from accelerometer and gyroscope sensors on phones and watches. The raw sensor data will undergo preprocessing through two distinct methods: topological data analysis and statistical feature extraction from segmented time series. The aim is to compare and assess the performance of various classifiers, including Decision Tree, k-Nearest Neighbors, Random Forest, SVM, and CNN, trained on the two differently preprocessed datasets. </p> <p align="center"> <img src="images/WISDM Activities.png" alt="Table1: 18 Activities" width="70%" height="70%"> <!--figcaption>Caption goes here</figcaption--> </p>

<!-- PREREQUISITES --> <h2 id="prerequisites"> :fork_and_knife: Prerequisites</h2>

<br> <br>

<!--This project is written in Python programming language. <br>-->

The following open source packages are used in this project:

• Numpy
• Pandas
• Matplotlib
• Scikit-Learn
• Scikit-tda
• Giotto-tda
• TensorFlow
• Keras

<!-- :paw_prints:--> <!-- FOLDER STRUCTURE --> <h2 id="folder-structure"> :cactus: Folder Structure</h2>

code
.
│
├── data
│   ├── raw_data
│   │   ├── phone
│   │   │   ├── accel
│   │   │   └── gyro
│   │   ├── watch
│   │       ├── accel
│   │       └── gyro
│   │
│   ├── transformed_data
│   │   ├── phone
│   │   │   ├── accel
│   │   │   └── gyro
│   │   ├── watch
│   │       ├── accel
│   │       └── gyro
│   │
│   ├── feature_label_tables
│   │    ├── feature_phone_accel
│   │    ├── feature_phone_gyro
│   │    ├── feature_watch_accel
│   │    ├── feature_watch_gyro
│   │
│   ├── wisdm-dataset
│        ├── raw
│        │   ├── phone
│        │   ├── accel
│        │   └── gyro
│        ├── watch
│            ├── accel
│            └── gyro
│
├── CNN_Impersonal_TransformedData.ipynb
├── CNN_Personal_TransformedData.ipynb  
├── CNN_Impersonal_RawData.ipynb    
├── CNN_Personal_RawData.ipynb 
├── Classifier_SVM_Personal.ipynb
├── Classifier_SVM_Impersonal.ipynb
├── statistical_analysis_time_domain.py
├── Topological data analysis.ipynb  

<!-- DATASET --> <h2 id="dataset"> :floppy_disk: Dataset</h2> <p> The WISDM (Wireless Sensor Data Mining) dataset includes raw time-series data collected from accelerometer and gyroscope sensors of a smartphone and smartwatch with their corresponding labels for each activity. The sensor data was collected at a rate of 20 Hz (i.e., every 50ms). Weiss et.al., collected this dataset from 51 subjects who performed 18 different activities listed in Table 2, each for 3 minutes, while having the smartphone in their right pant pocket and wearing the smartwatch in their dominant hand. Each line of the time-series sensor file is considered as input. <p align="center"> <img src="images/Human Activity.gif" alt="Human Activity.gif" display="inline-block" width="60%" height="50%"> </p>

The WISDM dataset is publicly available. Please refer to the Link

The following table shows the 18 activities represented in data set.

</p> <p align="center"> <img src="images/Activity Table.png" alt="Table1: 18 Activities" width="45%" height="45%"> </p>

<!-- ROADMAP --> <h2 id="roadmap"> :dart: Roadmap</h2> <p align="justify"> Weiss et. al. has trained three models namely Decision Tree, k-Nearest Neighbors, and Random Forest for human activity classification by preprocessing the raw time series data using statistical feature extraction from segmented time series. The goals of this project include the following: <ol> <li> <p align="justify"> Train the same models - Decision Tree, k Nearest Neighbors, and Random Forest using the preprocessed data obtained from topological data analysis and compare the performance against the results obtained by Weiss et. al. </p> </li> <li> <p align="justify"> Train SVM and CNN using the preprocessed data generated by Weiss et. al. and evaluate the performance against their Decision Tree, k Nearest Neighbors, and Random Forest models. </p> </li> </ol> </p>

<!-- PREPROCESSING --> <h2 id="preprocessing"> :hammer: Preprocessing</h2> <p align="justify"> The WISDM (Wireless Sensor Data Mining) dataset includes raw time-series data collected from accelerometer and gyroscope sensors of a smartphone and smartwatch with their corresponding labels for each activity. The sensor data was collected at a rate of 20 Hz (i.e., every 50ms). Weiss et.al., collected this dataset from 51 subjects who performed 18 different activities listed in the previous table, each for 3 minutes, while having the smartphone in their right pant pocket and wearing the smartwatch in their dominant hand. <br> In this project we tried three different feature sets, extracted from the raw data, which are as follows: <ol> <li><b>Pre-processed data</b> generated by Weiss et. al.</li> <li><b>Statistical feature extraction</b></li> <li><b>Topological feature extraction</b></li> </ol>

All these three approaches used windowing technique to segment the raw time series and extract features from each segment.

<!-- PRE-PROCESSED DATA --> <h2 id="preprocessed-data"> :diamond_shape_with_a_dot_inside: Pre-processed data</h2> <p align="justify"> Weiss et.al used windowing technique with window size of 10 seconds to extract statistical features. They extracted 93 features out of which 43 were used to train their models. We also used the same 43 features to train our SVM and CNN. The 43 features are 1. average sensor value 2. standard deviation 3. absolute difference 4. average resultant acceleration 5. Binned distribution (10 equal sized bins per axis) and 5. time between peaks, for each axis. </p>

<!-- STATISTICAL FEATURE --> <h2 id="statistical-feature"> :large_orange_diamond: Statistical feature</h2> <p align="justify"> For this approach, we segmented the dataset using 10 second window size (200 datapoints) with no overlapping. We decided to keep the window size same as whatWeiss et.al. applied in their study, for the sake of comparison. After segmentation, for each segment we calculated eight statistical features, namely, ‘min’, ‘max’, ‘mean’, ‘standard deviation’, ‘median’, ‘variance’, ‘zero crossing’ and ‘mean crossing’, for each axes. The zero and mean crossing features are calculated by counting the rate of when a signal passes line y=0 (if we let y-axis to be the specific measurement and x-axis to represent time) and the frequency at which the signal passes the line y = mean(signal), respectively. However, these two features did not show a significant difference between different activities, so we decided to ignore them. </p>

<!-- TOPOLOGICAL FEATURE --> <h2 id="topological-feature">