Horus
:dart: A gRPC-Node Distributed Tracing and Monitoring Tool.
Install / Use
/learn @oslabs-beta/HorusREADME
What is Horus?
<b>Horus</b> is a Distributed Tracing and Monitoring Tool for gRPC-Node Applications.
Our team aims to provide the user a seamless experience in adding tracing and monitoring functionality to their application with our NPM Packages!<br/><br/>
<b>Core Features</b> :zap:
- Horus traces, logs and monitors gRPC requests. Tracing and logging features include:
- Provides data on which gRPC method was called for a specific gRPC request.
- How long it took to complete the gRPC request.
- Date/Time of Completion.
- Traces any additional Intraservice Requests that were made in completing that request.
- Request data will be outputted in a text file (but users can also visualize the data with our Neo4J Integration.)
- Alerts user whenever gRPC request times go beyond a threshold of (+/-) 2 STD of its average request time using Slack Webhooks.
- We provide a mock-microservice which performs gRPC requests AND <b>intraservice gRPC requests</b> between 2 different services.
- Neo4j Integration to visualize data. <br/>
<b>The Mock-Microservice app. (in master branch) that we provide is a <b>bookstore app.</b> with the following services: </b> <br/>
- Books Service
- Customers Service <br/>
And yes, it does support intraservice gRPC requests! (Check out our setup tutorial in the table of contents below).
<br/>All modules can be found under the npm organization @horustracer. If you have any questions, please reach out to the team at: HorusTracerOfficial@gmail.com
<br/>Table of Contents
<br/> <br/>Installation
Installing Horus into your application is quick and easy. <br> <b>Follow the steps below</b> to set up the <b>ClientWrapper, ServerWrapper, and any intraservice requests</b> that you have.
For a more in depth guide to installing Horus, check the tutorial right below the installation section.
<b>NPM Installation: </b>
npm install @horustracer/ClientWrapper //(Mandatory)
npm install @horustracer/ServerWrapper //(Mandatory)
npm install @horustracer/Visualizer //(optional, for Neo4J integration to visualize request data.)
or
npm install @horustracer/ClientWrapper @horustracer/ServerWrapper @horustracer/Visualizer
<b>1. Setting up the ClientWrapper</b> <br/>
- Import the ClientWrapper from @horustracer/ClientWrapper into your stub (gRPC client) file.
- Initialize a new instance of the ClientWrapper, passing in the gRPC client, service and output text file name.
- Then export the ClientWrapper in the place of your previous stub.
const HorusClientWrapper = require('@horustracer/ClientWrapper');
const grpc = require("grpc");
const protoLoader = require("@grpc/proto-loader");
const path = require('path');
const PROTO_PATH = path.join(__dirname, "../protos/books.proto");
const packageDefinition = protoLoader.loadSync(PROTO_PATH, {
keepCase: true,
longs: String,
enums: String,
arrays: true
});
const BooksService = grpc.loadPackageDefinition(packageDefinition).BooksService;
const client = new BooksService (
"localhost:30043",
grpc.credentials.createInsecure()
);
//process.env.HORUS_DB utilizes environment variables. You can replace it with your MongoDB URI.
//process.env.SLACK_URL utilizes environment variables. You can replace it with your SLACK URL.
const ClientWrapper = new HorusClientWrapper(client, BooksService, 'books.txt', 'main', `${process.env.HORUS_DB}`, `${process.env.SLACK_URL}`);
module.exports = ClientWrapper;
<b>2. Setting up the ServerWrapper</b> <br/>
- Import the ServerWrapper into your server file.
- Rather than invoking the server.addService method, create a new instance of the ServerWrapper, passing in the server, proto, and methods as arguments.
const HorusServerWrapper = require('@horustracer/ServerWrapper);
const grpc = require('grpc');
const protoLoader = require("@grpc/proto-loader");
const path = require('path');
const controller = require("./booksController.js");
const PROTO_PATH = path.join(__dirname, '../protos/books.proto');
const ServerWrapper = new HorusServerWrapper(server, booksProto.BooksService.service, {
CreateBook: async (call, callback) => {
const book = call.request;
let result = await controller.CreateBook(book);
callback(null, result);
})
server.bind("127.0.0.1:30043", grpc.ServerCredentials.createInsecure());
server.start();
<b>3. Intraservice Requests</b> <br/>
- <b>If you don't have any intraservice requests, you're all done and can skip this part.</b>
- If you do, invoke the makeHandShakeWithServer method in your intraservice request's callback. The makeHandShakeWithServer method lives on the ClientWrapper. It's first argument is the ServerWrapper it's being invoked in and the second is the name of the intraservice request. (Remember because of part 1, your stub should be exporting the ClientWrapper rather than the gRPC client);
const HorusServerWrapper = require('@horustracer/ServerWrapper);
const booksStub = require('../stubs/booksStubs.js)
const CustomerServerWrapper = new HorusServerWrapper(server, customersProto.CustomersService.service, {
GetCustomer: async (call, callback) => {
const customerId = call.request;
const customer = await controller.GetCustomer(customerId);
const customersFavoriteBookId = {favBookId: customer.favBookId};
booksStub.GetBookById(customersFavoriteBookId, (error, response) => {
booksStub.makeHandShakeWithServer(CustomerServerWrapper, 'GetBookById');
callback(null, result);
});
})
Once you have completed this step for your intraservice requests, you're all done.
<br/> <br/>Branch Information
- Master - Includes Mock-Microservice, supports Intraservice Request.
- Staging - Staged version of master. Think of it like a draft of the master branch. .
- Package Branch - Includes source code for our 3 NPM packages.
- @horustracer/ClientWrapper
- @horustracer/ServerWrapper
- @horustracer/Visualizer
To iterate fork to your own repository and submit PRs.
Click here to see different ways of contributing.
<br/> <br/>Step-by-Step Tutorial For Horus
<b> Note: This does NOT include setting up the Mock Microservice Application. Check that section below, if you're interested in running the mock microservice application with the Horus Tool. </b> <br/>
Installing Horus into your application is quick and easy. Follow the steps below to set up the <b>ClientWrapper</b>, <b>ServerWrapper</b>, and any <b>intraservice handshake functions</b> you may need.
<br/><b>1. Setting up the ClientWrapper</b> <br/>
The Horus ClientWrapper does its magic by "wrapping" your preexisting gRPC stub (gRPC client). Think of it as a middleman which lives between you (the developer) and your stub. Horus uses a similar approach for the ServerWrapper, except it wraps your server methods rather than the gRPC server itself.
<br/> <p align="center"> <img src="./src/Assets/sketchWrapperNew.png"/> </p> <br/>Install the ClientWrapper by running npm install @horustracer/ClientWrapper. Then "require" in the ClientWrapper into your stub (gRPC client) file.
// run "npm install @horustracer/ClientWrapper" in your terminal
const HorusClientWrapper = require('@horustracer/ClientWrapper')
Now install and require in the grpc, @grpc.proto-loader, and path modules as you normally would. Using grpc.proto-loader, we are dynamically generated code from the proto files. This is opposed to static generation of code using the protoc compiler.
We installed the 'path' module to help us get the path of where the .proto file is located in your file system.
<br/>
// run "npm install grpc @grpc/proto-loader path"
const HorusClientWrapper = require('@horustracer/ClientWrapper');
const grpc = require("grpc");
const protoLoader = require("@grpc/proto-loader");
const path = require('path');
const PROTO_PATH = path.join(__dirname, "../protos/books.proto");
Next set up your package definition, service, and client as you normally would with gRPC.
<br/>
const HorusClientWrapper = require('@horustracer/ClientWrapper');
const grpc = require("grpc");
const protoLoader = require("@grpc/proto-loader");
const path = require('path');
const PROTO_PATH = path.join(__dirname, "../protos/books.proto");
const packageDefinition = protoLoader.loadSync(PROTO_PATH, {
keepCase: true,
longs: String,
enums: String,
arrays: true
});
const BooksService = grpc.loadPackageDefinition(packageDefinition).BooksService;
const client = new BooksService (
"localhost:30043",
grpc.credentials.createInsecure()
);
