deep-foundation

Deep Architecture

Gitpod usage

Start

Everything starts automatically. Just watch the terminal.

When all tasks are done, you can open Deep.Case App in browser: http://localhost:3007/ (ctrl/cmd + click by link)

It is also possible to open 3007 port manually, to do it open PORTS tab and select link for 3007 port.

Update

GitPod may fail to load prebuild, in that case it is recommended to use these sequence of actions in existing GitPod instance (use GitPod instance multiple times, remember GitPod still may fail):

1. Press CTRL+C in Gitpod Task 2: bash terminal. Or stop active npm run gitpod-start command.
2. Execute

npm run gitpod-update

1. Press ↑ in Gitpod Task 2: bash terminal and restart npm run gitpod-start command.

Codespaces usage

Everything starts automatically.

When all tasks are done, you can open Deep.Case App in browser: http://localhost:3007/ (ctrl/cmd + click by link)

You may need to make port 3007 public.

When all tasks are done, you can open Deep.Case App using PORTS tab:

Local usage

Installation

• Install nvm

  curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash
  export NVM_DIR="$([ -z "${XDG_CONFIG_HOME-}" ] && printf %s "${HOME}/.nvm" || printf %s "${XDG_CONFIG_HOME}/nvm")"
  [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"
  

• Install docker and docker-compose
  If you have Docker Desktop with docker compose (not docker-compose) you should enable this setting in Docker Desktop: Or use this command (if your are on linux):

  echo 'docker compose --compatibility "$@"' | sudo tee -a /usr/local/bin/docker-compose && sudo chmod +x /usr/local/bin/docker-compose
  

  Install docker on Ubuntu or use this commands:

  sudo apt update
  sudo apt install -y git curl docker.io docker-compose
  

  Optionally set up docker group and add user to this group (it may be done automatically on installation):

  sudo groupadd docker
  sudo usermod -aG docker $(whoami)
  

  These commands must be able to run without sudo, if it is not the case restart computer or relogin as user.

  docker run hello-world
  docker rm $(docker ps -a -q --filter "ancestor=hello-world")
  docker image rm hello-world
  

• Run this script to initialize and launch deep

  git clone https://github.com/deep-foundation/dev.git --depth=1
  cd dev
  nvm install && nvm use
  npm i -g npm@latest
  npm ci
  npm run packages | tee local.packages.log.txt
  npm run local | tee local.log.txt
  

  Run migrations while npm run local is executed (for example in separate terminal or screen).

  cd dev
  nvm use
  npm run local-migrate | tee local.migrations.log.txt
  

Launch/Restart

npm run local

Open

When all tasks are done, you can open Deep.Case App in browser: http://localhost:3007/ (ctrl/cmd + click by link)

Server usage

Preparation

Make sure you have virtual machine or server that is connected to the internet.

If you would like to use domain and SSL, make sure:

• DNS server of this domain configurated to point to the public IP of your server;
• HTTP (80) port for cerbot is accessable from the internet to be able to authenticate the domain ownership;
• HTTP (80) and HTTPS (443) ports are accessible from the internet to make nginx work correctly and make the Deep itself accessable.

If you would like to use IP without SSL, make sure:

• Ports 3006 and 3007 are open, for example, using the ufw package on Ubuntu:

  sudo ufw allow 3006
  sudo ufw allow 3007
  

Note that access via IP is not secure (there is no way to issue SSL certificate to IP address), that means it can be only used for development purposes with no sensetive data.

Install docker

sudo apt update
sudo apt install -y git curl docker.io docker-compose

Add docker group (optional)

Sometimes this action is not done on docker's installation via apt/apt-get or you may need to add access for docker to another user.

sudo groupadd docker
sudo usermod -aG docker $(whoami)

Check docker

These commands must be able to run without sudo, if it is not the case restart computer or relogin as user.

docker run hello-world
docker rm $(docker ps -a -q --filter "ancestor=hello-world")
docker image rm hello-world

Continue only if docker run hello-world works without sudo and errors.

After docker installation it may be required to move files to another drive (for example on azure VMs).

Install nvm

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash
export NVM_DIR="$([ -z "${XDG_CONFIG_HOME-}" ] && printf %s "${HOME}/.nvm" || printf %s "${XDG_CONFIG_HOME}/nvm")"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"

Install node

nvm install 18 && nvm alias default 18 && nvm use default

Update npm

npm i -g npm@latest

Installation

Configure deep (to support domain with SSL)

export GQL_SSL="1"
export PROTOCOL="https"
export DEEPCASE_HOST="chatgpt.deep.foundation"
export DEEPLINKS_HOST="deeplinks.chatgpt.deep.foundation"

Install and configure nginx (to support domain with SSL)

mkdir nginx
cd nginx
wget https://raw.githubusercontent.com/deep-foundation/dev/main/configure-nginx.js
npm i commander
node configure-nginx.js --configurations "$DEEPCASE_HOST 3007" "$DEEPLINKS_HOST 3006" --certbot-email drakonard@gmail.com
cd ..
rm -rf nginx

This command can be used to update the certificate once it expires (every 3 months). It may be a good idea to add it as sh script to cron.

Configure deep (to support IP without SSL)

Do not execute this step if you selected steps for domain with SSL before.

export GQL_SSL="0"
export PROTOCOL="http"
export HOST_IP="185.105.118.59"
export DEEPCASE_HOST="$HOST_IP:3007"
export DEEPLINKS_HOST="$HOST_IP:3006"

Install deeplinks global command

npm rm --unsafe-perm -g @deep-foundation/deeplinks
npm install --unsafe-perm -g @deep-foundation/deeplinks@latest

Make your Deep instance configurations (aka call-options.json)

export HASURA_ADMIN_SECRET=$(node -e "console.log(require('crypto').randomBytes(24).toString('hex'));")
export POSTGRES_PASSWORD=$(node -e "console.log(require('crypto').randomBytes(24).toString('hex'));")
export MINIO_ACCESS_KEY=$(node -e "console.log(require('crypto').randomBytes(24).toString('hex'));")
export MINIO_SECRET_KEY=$(node -e "console.log(require('crypto').randomBytes(24).toString('hex'));")
tee call-options.json << JSON
{
  "operation": "run",
  "envs": {
    "DEEPLINKS_PUBLIC_URL": "${PROTOCOL}://$DEEPLINKS_HOST",
    "NEXT_PUBLIC_DEEPLINKS_URL": "${PROTOCOL}://$DEEPLINKS_HOST",
    "NEXT_PUBLIC_GQL_PATH": "$DEEPLINKS_HOST/gql",
    "NEXT_PUBLIC_GQL_SSL": "${GQL_SSL}",
    "NEXT_PUBLIC_DEEPLINKS_SERVER": "${PROTOCOL}://$DEEPCASE_HOST",
    "NEXT_PUBLIC_ENGINES_ROUTE": "0",
    "NEXT_PUBLIC_DISABLE_CONNECTOR": "1",
    "JWT_SECRET": "'{\"type\":\"HS256\",\"key\":\"$(node -e "console.log(require('crypto').randomBytes(50).toString('base64'));")\"}'",
    "DEEPLINKS_HASURA_STORAGE_URL": "http://host.docker.internal:8000/",
    "HASURA_GRAPHQL_ADMIN_SECRET": "$HASURA_ADMIN_SECRET",
    "MIGRATIONS_HASURA_SECRET": "$HASURA_ADMIN_SECRET",
    "DEEPLINKS_HASURA_SECRET": "$HASURA_ADMIN_SECRET",
    "POSTGRES_PASSWORD": "$POSTGRES_PASSWORD",
    "HASURA_GRAPHQL_DATABASE_URL": "postgres://postgres:$POSTGRES_PASSWORD@postgres:5432/postgres",
    "POSTGRES_MIGRATIONS_SOURCE": "postgres://postgres:$POSTGRES_PASSWORD@host.docker.internal:5432/postgres?sslmode=disable",
    "RESTORE_VOLUME_FROM_SNAPSHOT": "0",
    "MANUAL_MIGRATIONS": "1",
    "MINIO_ROOT_USER": "$MINIO_ACCESS_KEY",
    "MINIO_ROOT_PASSWORD": "$MINIO_SECRET_KEY",
    "S3_ACCESS_KEY": "$MINIO_ACCESS_KEY",
    "S3_SECRET_KEY": "$MINIO_SECRET_KEY"
  }
}
JSON

Check that no other deep instances installed at the moment

docker ps -a

On fresh/clean server this list should be empty. If you have any previously installed deep instances, make sure the data is backed up.

Install your own deep instance

export DEEPLINKS_CALL_OPTIONS=$(cat call-options.json)
export DEBUG="deeplinks:engine:*,deeplinks:migrations:*"
deeplinks

Check that deep instance is installed correctly

docker ps -a

Make sure that all containers have status "Up".

Restart

Entire docker

sudo systemctl stop docker
sudo systemctl start docker

Single docker container

docker restart deep-links

Restore/Update

Update deeplinks (optional)

npm rm --unsafe-perm -g @deep-foundation/deeplinks
npm install --unsafe-perm -g @deep-foundation/deeplinks@lates