clearmail README

Introduction

clearmail is an open-source project that leverages AI to filter emails according to a set of simple rules you can write in english. The tool stars important emails and rejects or categorizes non-important emails according to preferences you can easily specify.

For maximum peace of mind, clearmail does not delete any emails. Instead, it only adds or removes labels from them, allowing you to review the AI's work. This project began as a weekend project and is still very much a work in progress!

How it works

1. At a Given Interval...

Clearmail operates on a configurable interval, determined by the refreshInterval setting in the config.yml file. This interval sets how often clearmail checks for new emails. When running in script mode, the process wakes up at this interval, checks for new emails since the last processed timestamp, and then goes back to sleep until the next interval.

2. Connecting to the Gmail via IMAP

Clearmail uses the IMAP protocol to connect to your Gmail account. It securely authenticates using the credentials provided in the .env file and establishes a connection to the server.

3. Searching for New Emails

Once connected, clearmail searches the inbox for any unread emails that have arrived since the last processed timestamp that are not STARRED.

4. Processing Each Email

For each new email identified, clearmail performs the following steps:

Analyzing the Email: The email's sender, subject, and body is analyzed using either the local LLM or OpenAI to determine if the email should be kept/starred or rejected/sorted according to predefined rules you specify in plain english in the config.yml file.

Sample Rules for Keeping Emails

rules:
  keep: |
    * Email is a direct reply to one of my sent emails
    * Email contains tracking information for a recent purchase
    * Subject: "Invoice" or "Receipt" (Transactional emails)

Example Rules for Rejecting Emails

rules:
  reject: |
    * Bulk emails that are not addressed to me specifically by name
    * Subject contains "Subscribe" or "Join now"
    * Email looks like a promotion

Categorizing or Moving the Email: If the email is worth reading according to your rules, it is left in the inbox and starred. If it's not, its either:
- Moved to the rejection folder (as named in rejectedFolderName), if the email is considered not important.
- Moved to a specific label like Social, if sortIntoCategoryFolders is enabled and the email matches one of the specified categories. You can specify any categories you want! For example:
```
categoryFolderNames:
  - News
  - Social Updates
  - Work
  - Family
  - Financial
```

5. Wrap Up

If any errors occur during the process, such as connection issues or errors in email analysis, clearmail logs these errors for debugging purposes.

Requirements

To use clearmail you will need:

A Gmail account
Node.js installed on your system

Note: this has only been tested for Mac.

Setup Instructions

Follow these steps to get clearmail up and running on your system:

Step 1: Gmail IMAP Access with App Password

To securely access your Gmail account using IMAP in applications like clearmail, especially when you have 2-Step Verification enabled, you'll need to create and use an app password. An app password is a 16-character code that allows less secure apps to access your Google Account. Here's a detailed guide on how to create and use app passwords for Gmail IMAP access:

Prerequisites

2-Step Verification: To create an app password, your Google Account must have 2-Step Verification enabled. This adds an additional layer of security to your account by requiring a second verification step during sign-in.

Creating an App Password

Go to Your Google Account:
- Navigate to Google Account settings.
Select Security:
- Find the "Security" tab on the left-hand side and click on it to access your security settings.
Access 2-Step Verification Settings:
- Under the "Signing in to Google" section, find and select "2-Step Verification." You may need to sign in to your account again for security purposes.
Open App Passwords Page:
- Scroll down to the bottom of the 2-Step Verification page, and you should see the "App passwords" option. Click on it to proceed.
- If you do not see this option, ensure that 2-Step Verification is indeed enabled and not set up exclusively for security keys. Note that app passwords may not be available for accounts managed by work, school, or other organizations, or for accounts with Advanced Protection enabled.
Generate a New App Password:
- Click on "Select app" and choose "Mail" as the application you want to generate the password for.
- Choose the device you are generating the password for (e.g., Windows Computer, iPhone, or other).
- Click on "Generate" to create your new app password.
Copy and Use the App Password:
- A 16-character code will be displayed on your screen. This is your app password, and you'll use it instead of your regular password for setting up IMAP access in clearmail.
- Follow any on-screen instructions to enter the app password into clearmail's configuration. Typically, you'll replace your regular password with this app password in the .env file where IMAP credentials are specified.

Step 2: Configure the YAML File

Navigate to the config.yml file in the clearmail directory. Customize these settings to match your email management preferences.

YAML File Options

The config.yml file contains several options to customize how clearmail works:

useLocalLLM: Determines whether to use a local language model or OpenAI for email analysis.
maxEmailChars: The maximum number of characters from an email body to feed to the AI for analysis.
maxEmailsToProcessAtOnce: Limits the number of emails processed in a single batch.
refreshInterval: How often, in seconds, to check for new emails.
timestampFilePath: The file path for storing the timestamp of the last processed email.
sortIntoCategoryFolders: Whether to sort emails into specified categories.
rejectedFolderName: The name of the folder where rejected emails are moved.
categoryFolderNames: A list of folder names for categorizing emails.
rules: Simple rules defining which emails to keep or reject.

Additional details are included as comments in config.yml.

Step 3: Configure .env File

To integrate your environment with clearmail, you'll need to configure the .env file by setting up various environment variables that the application requires to run. Copy the .env.example to .env and fill in the following:

.env File Configuration

OPENAI_API_KEY:
- Description: Optional. If you choose to not use a local LLM, fill in your OpenAI API key here.
IMAP_USER:
- Description: Your email address that you will use to access your Gmail account via IMAP.
IMAP_PASSWORD:
- Description: Use app password generated above.
IMAP_HOST:
- Description: The IMAP server address for Gmail.
- Default Value: imap.gmail.com. This is pre-set for Gmail accounts and typically does not need to be changed.
IMAP_PORT:
- Description: The port number used to connect to the IMAP server.
- Default Value: 993. This is the standard port for IMAP over SSL (IMAPS) and is used by Gmail.

Example .env File Content

OPENAI_API_KEY=your_openai_api_key_here
IMAP_USER=yourname@gmail.com
IMAP_PASSWORD=your_app_password_or_regular_password_here
IMAP_HOST=imap.gmail.com
IMAP_PORT=993

Step 4: Run the Process

Expanding on Step 4 to include instructions on setting up Node.js on your machine and ensuring you navigate to the correct folder to run clearmail:

Installing Node.js

Node.js is a JavaScript runtime built on Chrome's V8 JavaScript engine, and it's required to run clearmail. Here's how to install it:

Download Node.js: Visit the official Node.js website to download the installer for your operating system. It is recommended to download the LTS (Long Term Support) version for better stability.
Install Node.js:
- Windows & macOS: Run the downloaded installer and follow the on-screen instructions. The installer includes Node.js and npm (Node Package Manager).
- Linux: You can install Node.js via a package manager. Instructions for different distributions are available on the Node.js website under the Linux installations guide.
Verify Installation: Open a terminal or command prompt and type the following commands to verify that Node.js and npm are installed correctly:
```
node --version
npm --version
```
If the installation was successful, you should see the version numbers for both Node.js and npm.

Navigating to the clearmail Directory

Before running the clearmail process, make sure you are in the directory where clearmail is located:

Open a Terminal or Command Prompt: Use a terminal on Linux or macOS, or Command Prompt/Powershell on Windows.
Navigate to the clearmail Directory: Use the cd (change directory) command to navigate to the folder where you have clearmail installed. For example, if you have clearmail in a folder named "clearmail" on your desktop, the command might look like this:
- On Windows:
```
cd Desktop\clearmail
```
- On Linux or macOS:
```
cd ~/Desktop/clearmail
```

Running clearmail

Once Node.js is installed and you are in the correct directory, you can start clearmail by running the following

Clearmail

Install / Use

README