Intellect

:Info: Intellect is a Domain-specific language and Rules Engine for Python.

:Author: Michael Joseph Walsh

1. What is Intellect

---

Intellect is a DSL ("Domain-Specific Language") and Rule Engine for Python I authored for expressing policies to orchestrate and control a dynamic network defense cyber-security platform being researched in The MITRE Corporation's Innovation Program.

The rules engine provides an intellect, a form of artificial intelligence, a faculty of reasoning and understanding objectively over a working memory. The memory retains knowledge relevant to the system, and a set of rules authored in the DSL that describe a necessary behavior to achieve some goal. Each rule has an optional condition, and a suite of one or more actions. These actions either further direct the behavior of the system, and/or further inform the system. The engine starts with some facts, truths known about past or present circumstances, and uses rules to infer more facts. These facts fire more rules, that infer more facts and so on.

For the platform in the Innovation Program, the network defender uses the DSL to confer policy, how the platform is to respond to network events mounted over covert network channels, but there are no direct ties written into the language nor the rule engine to cyber security and thus the system in its entirety can be more broadly used in other domains.

2. TODOS

---

There are number of improvements I would like to work for future releases:

• Add Support for Multiple Rule Conditions.
• Move to an ANTLR4 based parser. The ANTLR Runtime for Python dependency was left behind with the the release of ANTL4. Luckily, n ANTLR 4 runtime for both Python 2 and 3 target is being tackled by Eric Vergnaud and is taking shape <https://github.com/ericvergnaud/antlr4>_. The last I looked all execParser and execLexer tests passed. Woot!
• Support Python 3.

Please help support these efforts.

.. image:: https://github.com/nemonik/Intellect/raw/master/images/gittip.png :target: https://www.gittip.com/nemonik/

3. Intellect In The News

---

The September 2013 issue, Volume 37 of Elsevier's "Computers and Security" contains a journal article entitled "Active cyber defense with denial and deception: A cyber-wargame experiment <http://dx.doi.org/10.1016/j.cose.2013.03.015>_" describing a computer network security use case for Intellect.

4. Installation

---

• To install via setuptools <http://peak.telecommunity.com/DevCenter/setuptools>_ use easy_install -U Intellect
• To install via pip <http://www.pip-installer.org/en/latest/installing.html>_ use pip install Intellect
• To install via pypm <http://code.activestate.com/pypm/>_ use pypm install intellect
• Or download the latest source from Master <http://github.com/nemonik/Intellect/archives/master>_ or the most recent tagged release Tagged Releases <https://github.com/nemonik/Intellect/tags>_, unpack, and run python setup.py install

5. Dependencies

---

• ANTLR3 Python Runtime <https://github.com/antlr/antlr3/tree/master/runtime/Python>_ that will contrain you to Python 2.x. In the past I've noted here that "Python 3 at present is not supported, because ANTLR3 appears not to support Python 3." Well, that's not exactly true, there is a Python3 runtime, I just wasn't aware of it nor have I worked with it. It can be found here ANTLR3 Python3 Runtime <https://github.com/antlr/antlr3/tree/master/runtime/Python3>_
• Python itself, if you don't already have it. I've tested the code on Python 2.7.1 and 2.7.2., but will likely work on any and all Python 2.x versions.

6. Source Code Contributions

---

The source code is available under the BSD 4-clause license. If you have ideas, code, bug reports, or fixes you would like to contribute please do so.

Bugs and feature requests can be filed at Github <http://github.com/nemonik/Intellect>_.

7. Background

---

Many production rule system implementations have been open-sourced, such as JBoss Drools, Rools, Jess, Lisa, et cetera. If you're familiar with the Drools syntax, Intellect's syntax should look familiar. (I'm not saying it is based on it, because it is not entirely, but I found as I was working the syntax I would check with Drools and if made sense to push in the direction of Drools, this is what I did.) The aforementioned implementations are available for other languages for expressing production rules, but it is my belief that Python is under-represented, and as such it was my thought the language and rule engine could benefit from being open sourced, and so I put a request in.

The MITRE Corporation granted release August 4, 2011.

Thus, releasing the domain-specific language (DSL) and Rule Engine to Open Source in the hopes doing so will extend its use and increase its chances for possible adoption, while at the same time mature the project with more interested eyeballs being placed on it.

Starting out, it was initially assumed the aforementioned platform would be integrated with the best Open Source rules engine available for Python as there are countless implementation for Ruby, Java, and Perl, but surprisingly I found none fitting the project's needs. This led to the thought of inventing one; simply typing the keywords "python rules engine" into Google though will return to you the advice "to not invent yet another rules language", but instead you are advised to "just write your rules in Python, import them, and execute them." The basis for this advice can be coalesced down to doing so otherwise does not fit with the "Python Philosophy." At the time, I did not believe this to be true, nor fully contextualized, and yet admittedly, I had not yet authored a line of Python code (Yes, you're looking at my first Python program. So, please give me a break.) nor used ANTLR3 prior to this effort. Looking back, I firmly believe the act of inventing a rules engine and abstracting it behind a nomenclature that describes and illuminates a specific domain is the best way for in case of aforementioned platform the network defender to think about the problem. Like I said though the DSL and rules engine could be used for anything needing a "production rule system".

As there were no rules engines available for Python fitting the platforms needs, a policy language and naive forward chaining rules engine were built from scratch. The policy language's grammar is based on a subset of Python language syntax. The policy DSL is parsed and lexed with the help of the ANTLR3 Parse Generator and Runtime for Python.

8. Facts (Data being reasoned over)

---

The interpreter, the rules engine, and the remainder of the code such as objects for conferring discrete network conditions, referred to as "facts", are also authored in Python. Python's approach to the object-oriented programming paradigm, where objects consist of data fields and methods, did not easily lend itself to describing "facts". Because the data fields of a Python object referred to syntactically as "attributes" can and often are set on an instance of a class, they will not exist prior to a class's instantiation. In order for a rules engine to work, it must be able to fully introspect an object instance representing a condition. This proves to be very difficult unless the property decorator with its two attributes, "getter" and "setter", introduced in Python 2.6, are adopted and formally used for authoring these objects. Coincidentally, the use of the "Getter/Setter Pattern" used frequently in Java is singularly frowned upon in the Python developer community with the cheer of "Python is not Java".

So, you will need to author your facts as Python object's who attributes are formally denoted as properties like so for the attributes you would like to reason over::

class ClassA(object):
	'''
	An example fact
	'''

	def __init__(self, property0 = None, property1 = None):
		'''
		ClassA initializer
		'''
		self._property0 = property0

	@property
	def property0(self):
		return self._property0

	@property0.setter
	def property0(self, value):
		self._property0 = value

9. The Policy DSL

Example with policy files can be found at the path intellect/examples <https://github.com/nemonik/Intellect/tree/master/intellect/examples>. Policy files must follow the Policy grammar as define in intellect/grammar/Policy.g <https://raw.github.com/nemonik/Intellect/master/intellect/grammar/Policy.g>. The rest of this section documents the grammar of policy domain-specific language.

9.1 Import Statements (ImportStmts)

Import statements basically follow Python's with a few limitations. For example, The wild card form of import is not supported for the reasons elaborated here <http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html#importing>_ and follow the Python 2.7.2 grammar. ImportStmt statements exist only at the same level of ruleStmt statements as per the grammar, and are typically at the top of a policy file, but are not limited to. In fact, if you break up your policy across several files the last imported as class or module wins as the one being named.

.. _9.2:

9.2 Attribute Statements (attribute)

.. figure:: https://github.com/nemonik/Intellect/raw/master/images/attributeStmt.jpg

The syntax diagram for a attributeStmt.

attributeStmt statements are expressions used to create policy attributes, a form of globals, that are accessible from rules.

For example, a policy could be written::

import logging

first_sum = 0
second_sum = 0

rule "set both first_sum and second_sum to 1":
	agenda-group "test_d"
	then:
		attribute (first_sum, second_sum) = (1,1)
		l