Erlemar.github.io
Data science portfolio
Install / Use
/learn @Erlemar/Erlemar.github.ioREADME
Now I have a personal website!
Data science portfolio by Andrey Lukyanenko
This portfolio is a compilation of notebooks which I created for data analysis or for exploration of machine learning algorithms. A separate category is for separate projects.
Stand-alone projects.
Handwritten digit recognition
This is my own project using image recognition methods in practice. This is a site (also works on mobile) where user can draw a digit, and machine learning models (FNN and CNN) will try to recognize it. After than models can use the drawn digit for training to improve their accuracy. Live version is here. The code can be found here.
Chatbot in telegram
A conversational chatbot in telegram which was created for an honor assignment of nlp course by Higher School of Economics. The main functionality of the bot is to distinguish two types of questions (questions related to programming and others) and then either give an answer or talk using a conversational model.
Kaggle competitions.
Avito demand prediction
Avito demand prediction was a competition on Kaggle where we tried to predict something like demand based on ads content. This competition was very interesting because it had tabular data, texts and images. On the other hand this was the reason the competition was quite difficult. My team reached 131th place and got bronze medal! Here is a link to my solution.
Categorization of purchases
This was a Russian inclass Kaggle competition in the third session of ODS ml course. It sounded interesting and I took part in it reaching a 3rd place. Here is my kaggle kernel with a solution.
Kaggle kernels.
2018 Kaggle ML & DS Survey Challenge
Some time ago Kaggle launched a big online survey for kagglers and now this data is public. There were multiple choice questions and some forms for open answers. Survey received 23k+ respondents from 147 countries. As a result we have a big dataset with rich information on data scientists using Kaggle. In this kernel I compare DS in USA, Russia, India and other countries.
DonorsChoose.org Application Screening
DonorsChoose.org empowers public school teachers from across the country to request much-needed materials and experiences for their students. DonorsChoose.org receives hundreds of thousands of project proposals each year for classroom projects in need of funding. This is a competition on Kaggle where people can create a machine learning model to help this fund with auto-approving of applications. Prizes are given to the authors with the most upvoted kernels. Here is my kernel with extensive EDA, feature engineering and building models. This kernel got 2nd place by the number of votes and I won Google Pixelbook for it!
Avito Demand Prediction Challenge
Avito challenge is about predicting demand for an online advertisement based on its full description (title, description, images, etc.), its context (geographically where it was posted, similar ads already posted) and historical demand for similar ads in similar contexts. The competition is interesting due to many types of data in it which allows to build various models. Here is my kernel with EDA, creating features and building models.
Home Credit Default Risk
Home Credit Bank offers a challenge of credit scoring. There is a lot of data about applicants and their previous behavior. Here is my kernel.
Movie Review Sentiment Analysis
Some time ago Kaggle has launched several "remakes" of old competitions. It means that datasets are the same, but now we are offered an opportunity to simply explore the data and create kernels with new methods. One of these competitions is sentiment analysis of Rotten Tomatoes dataset with 5 classes (negative, somewhat negative, neutral, somewhat positive, positive). I have created a kernel with EDA and modern NN architecture: LSTM-CNN. Currently this kernel shows the 5th result of leaderboard.
Two Sigma: Using News to Predict Stock Movements
In this competition Reuters provide unique data, which can't be obtained outside of this competition. We can see a 10 years worth of news and market data on many companies. This competition is kernel-only, which means that everyone has the same amount of computational power for this competition. In my kernel I have analysed the data and showed trends of market data.
Santander Value Prediction Challenge
In this competition we got an anonymized dataset, later it was found that it had a certain structure. In my kernel I tried to analyze the data and created new features using NN model.
Google Analytics Customer Revenue Prediction
RStudio hosted this competition to prove that machine learning algorithms can impact business and help marketing. In my kernel I did an extensive EDA and build an interesting LGB model.
Data Science for Good: Center for Policing Equity
This dataset was provided by The Center for Policing Equity. They hope that kagglers will help to create better models, find some unique insights and improve geo-analytics. In my kernel I try to do such things.
Classification problems.
Titanic: Machine Learning from Disaster
Titanic: Machine Learning from Disaster is a knowledge competition on Kaggle. Many people started practicing in machine learning with this competition, so did I. This is a binary classification problem: based on information about Titanic passengers we predict whether they survived or not. General description and data are available on Kaggle. Titanic dataset provides interesting opportunities for feature engineering.
Ghouls, Goblins, and Ghosts... Boo!
Ghouls, Goblins, and Ghosts... Boo! is a knowledge competition on Kaggle. This is a multiple classification problem: based on information about monsters we predict their types. A fun competition for Halloween. General description and data are available on Kaggle. This dataset has little number of samples, so careful feature selection and model ensemble are necessary for high accuracy.
Otto Group Product Classification Challenge
Otto Group Product Classification Challenge is a knowledge competition on Kaggle. This is a multiple classification problem. Based on information about products we predict their category. General description and data are available on Kaggle. The data is obfuscated, so the main questionlies in the selection of the model for prediction.
Imbalanced classes
In real world it is common to meet data in which some classes are more common and others are rarer. In case of a serious disbalance prediction rare classes could be difficult using standard classification methods. In this notebook I analyse such a situation. I can't share the data, used in this analysis.
Bank card activations
Banks strive to increase the efficiency of their contacts with customers. One of the areas which require this is offering new products to existing clients (cross-selling). Instead of offering new products to all clients, it is a good idea to predict the probability of a positive response. Then the offers could be sent to those clients, for whom the probability of response is higher than some threshold value. In this notebook I try to solve this problem.
Regression problems.
House Prices: Advanced Regression Techniques
House Prices: Advanced Regression Techniques is a knowledge competition on Kaggle. This is a regression problem: based on information about houses we predict their prices. General description and data are available on [Kaggle](https://www.kaggle.c
