Paint2Code

Welcome to Paint2Code, a robust tool designed to transform your hand-drawn sketches into functional HTML code. This innovative project leverages advanced image recognition and machine learning algorithms to interpret drawings and convert them into clean, structured HTML elements.

Features

• Image to HTML Conversion: Upload your sketch and receive HTML code.
• Support for Multiple HTML Elements: Detects various shapes and interprets them as different HTML elements.
• Easy to Use Interface: User-friendly interface designed for both beginners and advanced users.
• Multiple Encoder Models: Utilize various encoder models to enhance accuracy and flexibility in interpreting sketches.
• Support for Multiple HTML Code Styles: Choose from different HTML coding styles to match your project requirements.

Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.

Prerequisites

What things you need to install the software and how to install them:

- Python 3.8 or higher
- matplotlib==3.8.0
- nltk==3.8.1
- numpy==1.26.4
- Pillow==10.3.0
- streamlit==1.33.0
- torch==2.2.1
- torchvision==0.17.1
- tqdm==4.65.0

Installing

A step-by-step series of examples that tell you how to get a development environment running:

1. Clone the repo:

   git clone https://github.com/nico1008/paint2code
   

2. Install the required packages:

   pip install -r requirements.txt
   

Usage

To use the original Paint2Code dataset, please visit pain2code Dataset to download it.

Model Weights

• CustomCNN: Download here
• MobileNetV3: Download here
• ResNet18: Download here

Data Placement

Place your training data in the .data/all_data folder.

Methods of Use

There are two methods to use this project: via Jupyter notebooks or Python scripts. I suggest using the Jupyter notebooks for better data visualisation.

Method 1: Using Jupyter Notebooks

1. Data Preparation: Open prepareData.ipynb and run all cells.
2. Model Training: Open the appropriate notebook for desired model and run all cells:
   • trainCustomCNN.ipynb for CustomCNN
   • trainMobileNet.ipynb for MobileNetV3
   • trainResNet18.ipynb for ResNet18
3. Model Evaluation: Open the corresponding evaluation notebook and run all cells:
   • evalCustomCNN.ipynb for CustomCNN
   • evalMobileNet.ipynb for MobileNetV3
   • evalResNet18.ipynb for ResNet18

Method 2: Using Python Scripts

1. Data Preparation:

   python prepareData.py
   

2. Model Training:

   python train.py
   

3. Model Evaluation:

   python eval.py
   

Web host

you can visit the paint2code website to use the model on any desired device.

Acknowledgments

I would like to thank the following resources, communities and people for their invaluable contributions and support:

• The OpenCV team for their robust computer vision library.
• The PyTorch community for their deep learning framework.
• KKopilka for major moral support and love.
• Tony Beltramelli project draws significant inspiration from Tony Beltramelli's pioneering Pix2Code modell.

Contact Me

If you have any questions, suggestions, or need further assistance, please feel free to reach out:

• Telegram: @nico_1008k
• GitHub: nico1008

---

I hope you find Paint2Code useful! If you have any questions or feedback, please feel free to reach out.