PsychicHttp - HTTP on your ESP 🧙🔮

PsychicHttp is a webserver library for ESP32 + Arduino framework which uses the ESP-IDF HTTP Server library under the hood. It is written in a similar style to the Arduino WebServer, ESPAsyncWebServer, and ArduinoMongoose libraries to make writing code simple and porting from those other libraries straightforward.

Discord: https://discord.gg/TAQrTR3f9C

Features

• Asynchronous approach (server runs in its own FreeRTOS thread)
• Handles all HTTP methods with lots of convenience functions:
  • GET/POST parameters
  • get/set headers
  • get/set cookies
  • basic key/value session data storage
  • authentication (basic and digest mode)
• HTTPS / SSL support
• Static fileserving (SPIFFS, LittleFS, etc.)
• Chunked response serving for large files
• File uploads (Basic + Multipart)
• Websocket support with onOpen, onFrame, and onClose callbacks
• EventSource / SSE support with onOpen, and onClose callbacks
• Request filters, including Client vs AP mode (ON_STA_FILTER / ON_AP_FILTER)
• TemplatePrinter class for dynamic variables at runtime

Differences from ESPAsyncWebserver

• No templating system (anyone actually use this?)
• No url rewriting (but you can use response->redirect)

Usage

Installation

Platformio

PlatformIO is an open source ecosystem for IoT development.

Add "PsychicHttp" to project using Project Configuration File platformio.ini and lib_deps option:

[env:myboard]
platform = espressif...
board = ...
framework = arduino

# using the latest stable version
lib_deps = hoeken/PsychicHttp

# or using GIT Url (the latest development version)
lib_deps = https://github.com/hoeken/PsychicHttp

Installation - Arduino

Open Tools -> Manage Libraries... and search for PsychicHttp.

Principles of Operation

Things to Note

• PsychicHttp is a fully asynchronous server and as such does not run on the loop thread.
• You should not use yield or delay or any function that uses them inside the callbacks.
• The server is smart enough to know when to close the connection and free resources.
• You can not send more than one response to a single request.

PsychicHttp

• Listens for connections.
• Wraps the incoming request into PsychicRequest.
• Keeps track of clients + calls optional callbacks on client open and close.
• Find the appropriate handler (if any) for a request and pass it on.

Request Life Cycle

• TCP connection is received by the server.
• HTTP request is wrapped inside PsychicRequest object + TCP Connection wrapped inside PsychicConnection object.
• When the request head is received, the server goes through all PsychicEndpoints and finds one that matches the url + method.
  • handler->filter() and handler->canHandle() are called on the handler to verify the handler should process the request.
  • handler->needsAuthentication() is called and sends an authorization response if required.
  • handler->handleRequest() is called to actually process the HTTP request.
• If the handler cannot process the request, the server will loop through any global handlers and call that handler if it passes filter(), canHandle(), and needsAuthentication().
• If no global handlers are called, the server.defaultEndpoint handler will be called.
• Each handler is responsible for processing the request and sending a response.
• When the response is sent, the client is closed and freed from the memory.
  • Unless its a special handler like websockets or eventsource.

Handlers

• PsychicHandler is used for processing and responding to specific HTTP requests.
• PsychicHandler instances can be attached to any endpoint or as global handlers.
• Setting a Filter to the PsychicHandler controls when to apply the handler, decision can be based on request method, url, request host/port/target host, the request client's localIP or remoteIP.
• Two filter callbacks are provided: ON_AP_FILTER to execute the rewrite when request is made to the AP interface, ON_STA_FILTER to execute the rewrite when request is made to the STA interface.
• The canHandle method is used for handler specific control on whether the requests can be handled. Decision can be based on request method, request url, request host/port/target host.
• Depending on how the handler is implemented, it may provide callbacks for adding your own custom processing code to the handler.
• Global Handlers are evaluated in the order they are attached to the server. The canHandle is called only if the Filter that was set to the Handler return true.
• The first global Handler that can handle the request is selected, no further processing of handlers is called.

Responses and how do they work

• The PsychicResponse objects are used to send the response data back to the client.
• Typically the response should be fully generated and sent from the callback.
• It may be possible to generate the response outside the callback, but it will be difficult.
  • The exceptions are websockets + eventsource where the response is sent, but the connection is maintained and new data can be sent/received outside the handler.

Porting From ESPAsyncWebserver

If you have existing code using ESPAsyncWebserver, you will feel right at home with PsychicHttp. Even if internally it is much different, the external interface is very similar. Some things are mostly cosmetic, like different class names and callback definitions. A few things might require a bit more in-depth approach. If you're porting your code and run into issues that aren't covered here, please post and issue.

Globals Stuff

• Change your #include to #include <PsychicHttp.h>
• Change your server instance: PsychicHttpServer server;
• Define websocket handler if you have one: PsychicWebSocketHandler websocketHandler;
• Define eventsource if you have one: PsychicEventSource eventSource;

setup() Stuff

• add your handlers and call server.begin()
• server has a configurable limit on .on() endpoints. change it with server.config.max_uri_handlers = 20; as needed.
• check your callback function definitions:
  • AsyncWebServerRequest -> PsychicRequest
  • no more onBody() event
    • for small bodies (server.maxRequestBodySize, default 16k) it will be automatically loaded and accessed by request->body()
    • for large bodies, use an upload handler and onUpload()
  • websocket callbacks are much different (and simpler!)
  • websocket / eventsource handlers get attached to url in server.on("/url", &handler) instead of passing url to handler constructor.
  • eventsource callbacks are onOpen and onClose now.
• HTTP_ANY is not supported by ESP-IDF, so we can't use it either.
• NO server.onFileUpload(onUpload); (you could attach an UploadHandler to the default endpoint i guess?)
• NO server.onRequestBody(onBody); (same)

Requests / Responses

• request->send is now response->send()
• if you create a response, call response->send() directly, not request->send(reply)
• request->headers() is not supported by ESP-IDF, you have to just check for the header you need.
• No AsyncCallbackJsonWebHandler (for now... can add if needed)
• No request->beginResponse(). Instanciate a PsychicResponse instead: PsychicResponse response(request);
• No PROGMEM suppport (its not relevant to ESP32: https://esp32.com/viewtopic.php?t=20595)
• No Stream response support just yet

Usage

Create the Server

Here is an example of the typical server setup:

#include <PsychicHttp.h>
PsychicHttpServer server;

void setup()
{
   //optional low level setup server config stuff here.
   //server.config is an ESP-IDF httpd_config struct
   //see: https://docs.espressif.com/projects/esp-idf/en/v4.4.6/esp32/api-reference/protocols/esp_http_server.html#_CPPv412httpd_config
   //increase maximum number of uri endpoint handlers (.on() calls)
   server.config.max_uri_handlers = 20; 

   //connect to wifi

   //call server methods to attach endpoints and handlers
   server.on(...);
   server.serveStatic(...);
   server.attachHandler(...);
}

Add Handlers

One major difference from ESPAsyncWebserver is that handlers can be attached to a specific url (endpoint) or as a global handler. The reason for this, is that attaching to a specific URL is more efficient and makes for cleaner code.

Endpoint Handlers

An endpoint is basically just the URL path (eg. /path/to/file) without any query string. The server.on(...) function is a convenience function for creating endpoints and attaching a handler to them. There are two main styles: attaching a basic WebRequest handler and attaching an external handler.

//creates a basic PsychicWebHandler that calls the request_callback callback
server.on("/url", HTTP_GET, request_callback);

//same as above, but defaults to HTTP_GET
server.on("/url", request_callback);

//attaches a websocket handler to /ws
PsychicWebSocketHandler websocketHandler;
server.on("/ws", &websocketHandler);

The server.on(...) returns a pointer to the endpoint, whic