Transbigdata
A Python package develop for transportation spatio-temporal big data processing, analysis and visualization.
Install / Use
/learn @ni1o1/TransbigdataREADME
English 中文版
TransBigData
<img src="https://github.com/ni1o1/transbigdata/raw/main/docs/source/_static/logo-wordmark-dark.png" style="width:550px">
Introduction
TransBigData is a Python package developed for transportation spatio-temporal big data processing, analysis and visualization. TransBigData provides fast and concise methods for processing common transportation spatio-temporal big data such as Taxi GPS data, bicycle sharing data and bus GPS data. TransBigData provides a variety of processing methods for each stage of transportation spatio-temporal big data analysis. The code with TransBigData is clean, efficient, flexible, and easy to use, allowing complex data tasks to be achieved with concise code.
For some specific types of data, TransBigData also provides targeted tools for specific needs, such as extraction of Origin and Destination(OD) of taxi trips from taxi GPS data and identification of arrival and departure information from bus GPS data. The latest stable release of the software can be installed via pip and full documentation
can be found at https://transbigdata.readthedocs.io/en/latest/. Introduction PPT can be found here and here(in Chinese)
Target Audience
The target audience of TransBigData includes:
- Data science researchers and data engineers in the field of transportation big data, smart transportation systems, and urban computing, particularly those who want to integrate innovative algorithms into intelligent trasnportation systems
- Government, enterprises, or other entities who expect efficient and reliable management decision support through transportation spatio-temporal data analysis.
Technical Features
- Provide a variety of processing methods for each stage of transportation spatio-temporal big data analysis.
- The code with
TransBigDatais clean, efficient, flexible, and easy to use, allowing complex data tasks to be achieved with concise code.
Main Functions
Currently, TransBigData mainly provides the following methods:
- Data Quality: Provides methods to quickly obtain the general information of the dataset, including the data amount the time period and the sampling interval.
- Data Preprocess: Provides methods to clean multiple types of data error.
- Data Gridding: Provides methods to generate multiple types of geographic grids (Rectangular grids, Hexagonal grids) in the research area. Provides fast algorithms to map GPS data to the generated grids.
- Data Aggregating: Provides methods to aggregate GPS data and OD data into geographic polygon.
- Data Visualization: Built-in visualization capabilities leverage the visualization package keplergl to interactively visualize data on Jupyter notebook with simple code.
- Trajectory Processing: Provides methods to process trajectory data, including generating trajectory linestring from GPS points, and trajectory densification, etc.
- Basemap Loading: Provides methods to display Mapbox basemap on matplotlib figures
Grid processing framework offered by TransBigData
Here is an overview of the gridding framework offered by TransBigData.
See This Example for further details.
Trajectory processing framework offered by TransBigData
Here is an overview of the Trajectory processing framework offered by TransBigData.
See This Example for further details.
Installation
TransBigData support Python >= 3.6
Using pypi 
TransBigData can be installed by using pip install. Before installing TransBigData, make sure that you have installed the available geopandas package. If you already have geopandas installed, run the following code directly from the command prompt to install TransBigData:
pip install transbigdata
Using conda-forge 
You can also install TransBigData by conda-forge, this will automaticaly solve the dependency, it can be installed with:
conda install -c conda-forge transbigdata
Contributing to TransBigData 
All contributions, bug reports, bug fixes, documentation improvements, enhancements and ideas are welcome. A detailed overview on how to contribute can be found in the contributing guide on GitHub.
Examples
Example of data visualization
Visualize trajectories (with keplergl)
Visualize data distribution (with keplergl)
Visualize OD (with keplergl)
Example of taxi GPS data processing
The following example shows how to use the TransBigData to perform data gridding, data aggregating and data visualization for taxi GPS data.
Read the data
import transbigdata as tbd
import pandas as pd
#Read taxi gps data
data = pd.read_csv('TaxiData-Sample.csv',header = None)
data.columns = ['VehicleNum','time','lon','lat','OpenStatus','Speed']
data
Data pre-processing
Define the study area and use the tbd.clean_outofbounds method to delete the data out of the study area
#Define the study area
bounds = [113.75, 22.4, 114.62, 22.86]
#Delete the data out of the study area
data = tbd.clean_outofbounds(data,bounds = bounds,col = ['lon','lat'])
Data gridding
The most basic way to express the data distribution is in the form of geograpic grids. TransBigData provides methods to generate multiple types of geographic
