Blazegraph TinkerPop3 Implementation (blazegraph-gremlin)

=======

Welcome to the Blazegraph/TinkerPop3 project. The TP3 implementation has some significant differences from the TP2 version. The data model has been changed to use RDF*, an RDF reification framework described here.

The concept behind blazegraph-gremlin is that property graph (PG) data can be loaded and accessed via the TinkerPop3 API, but underneath the hood the data will be stored as RDF using the PG data model described in this document. Once PG data has been loaded you can interact with it just like you would interact with ordinary RDF - you can run SPARQL queries or interact with the data via the SAIL API. It just works. The PG data model is also customizable via a round-tripping interface called the BlazeValueFactory, also described in detail in this document.

Some interesting features of the Blazegraph/TP3 implementation include:

• Listener API - subscribe to notifications about updates to the graph (adds and removes of vertices/edges/properties, commits, rollbacks, etc.)
• History API - capture full or partial history of edits to the graph.
• Built-in full text index and search API to find graph elements.
• Automatic SPARQL to PG translation - run a SPARQL query and get your results back in property graph form.
• Query management API - list and cancel running Sparql queries.
• Bulk Load API for fast setup of new graphs.
• Support for MVCC concurrency model for high-concurrency read access.

Maven Central Dependency

Blazegraph and the TinkerPop3 implementation are available on Maven Central.

   <dependency>
      <groupId>com.blazegraph</groupId>
      <artifactId>blazegraph-gremlin</artifactId>
      <version>1.0.0</version>
   </dependency>

##Javadocs Checkout the latest Javadocs.

Getting Started

To build blazegraph-gremlin:

> mvn clean install -Dtest=SampleCode

To import blazegraph-gremlin into Eclipse:

> mvn eclipse:eclipse

Then select "File-Import-Existing Projects Into Workspace" from the Eclipse menu and select the root directory of this project.

Continue reading this document and take a look at SampleCode.java provided in blazegraph-gremlin/src/test for information on how to get started writing your TP3 application with Blazegraph.

Gremlin Console

To install the blazegraph-gremlin plugin in the gremlin console:

gremlin> :install com.blazegraph blazegraph-gremlin 1.0.0
==>Loaded: [com.blazegraph, blazegraph-gremlin, 1.0.0] - restart the console to use [tinkerpop.blazegraph]

Once you restart the console, activate the blazegraph-gremlin plugin:

gremlin> :plugin use tinkerpop.blazegraph
==>tinkerpop.blazegraph activated

You can then open a BlazeGraph instance by specifying the location you would like to use for the persistent journal file:

gremlin> g = BlazeGraphFactory.open("/tmp/blazegraph.jnl")
BlazeGraph(TM) Graph Engine

                   Flexible
                   Reliable
                  Affordable
      Web-Scale Computing for the Enterprise
      
Copyright SYSTAP, LLC DBA Blazegraph 2006-2016.  All rights reserved.
==>blazegraphembedded[vertices:0 edges:0]

Blazegraph/TP3 Data Model

It's important to understand how Blazegraph organizes property graph data as RDF. Blazegraph uses the RDF* framework, which is an extension to RDF that provides for an easier RDF reification syntax. Reification is a means of using an RDF statement as an RDF value in other statements. The RDF* syntax for an RDF "statement as a value" is as follows:

# statement
:john :knows :mary . 
# "statement as value"
<<:john :knows :mary>> dc:source <http://johnknowsmary.com> .

Blazegraph uses the OpenRDF SAIL API and represents RDF* reified statements as bnodes in that API. This is important for understanding how to write SPARQL queries against TP3 graphs and how to interpret query results.

Property graph values must be converted into RDF values and vice versa. Blazegraph provides a BlazeValueFactory interface with a default implementation. You can extend this interface and provide your own value factory if you prefer a custom look for the RDF values in your property graph.

Blazegraph accepts user-supplied IDs (strings only) for vertices and edges. If no id is supplied a UUID will be generated. By default, TP3 ids and property keys are converted into URIs by prepending a blaze: namespace prefix. Property values are simply converted into datatyped literals.

Two fixed URIs are used and provided by the BlazeValueFactory to represent element labels (rdf:type by default) and property values for Cardinality.list vertex properties (rdf:value by default). These can also be overriden as desired.

Property graph elements are represented as follows in Blazegraph:

# BlazeVertex john = graph.addVertex(T.id, "john", T.label, "person");
blaze:john rdf:type blaze:person .

# BlazeEdge knows = graph.addEdge(john, mary, "knows", T.id, "k01");
blaze:john blaze:k01 blaze:mary .
<<blaze:john blaze:k01 blaze:mary>> rdf:type blaze:knows .

Vertices requires one statement, edges require two.

Edge properties are simply attached to the reified edge statement:

# BlazeProperty p = knows.property("acl", "private");
<<blaze:john blaze:k01 blaze:mary>> blaze:acl "private" .

Representation of vertex properties depends on the cardinality of the key. Cardinality.single and .set look the same, .list is represented differently. Vertices can mix and match cardinalities for different keys. All three cardinalities are supported, but Cardinality.set is the one most closely aligned with RDF and as such will provide the best performance of the three. User supplied ids are NOT supported for vertex properties.

Cardinality.set and Cardinality.single are modeled the same way:

# VertexProperty set = john.property(Cardinality.set, "age", 25, "acl", "public");
blaze:john blaze:age "25"^^xsd:int .
<<blaze:john blaze:age "25"^^xsd:int>> blaze:acl "public" .

Cardinality.list is modeled differently:

# VertexProperty list = john.property(Cardinality.list, "city", "salt lake city", "acl", "public");
blaze:john blaze:city "12765"^^bg:listIndex .
<<blaze:john blaze:city "12765"^^bg:listIndex>> rdf:value "salt lake city" .
<<blaze:john blaze:city "12765"^^bg:listIndex>> blaze:acl "public" .

Cardinality.list uses a specially datatyped and monotonically increasing internal identifier to represent the vertex property (the actual datatype is <http://www.blazegraph.com/rdf/datatype#listIndex>). This identifier serves to manage duplicate list values and ordering of list items. It's important to note this difference as different cardinalities will require different SPARQL queries.

Putting it all together: The Crew

Here is how the TinkerPop3 "Crew" dataset looks when loaded into Blazegraph. Human-friendly IDs have been assigned to vertices and edge UUIDs have been abbreviated to 5 characters for brevity.

blaze:tinkergraph rdf:type blaze:software ;
                  blaze:name "tinkergraph" .
            
blaze:gremlin rdf:type blaze:software ;
              blaze:name "gremlin" ;
              blaze:48f63 blaze:tinkergraph .
            
<<blaze:gremlin blaze:48f63 blaze:tinkergraph>> rdf:type blaze:traverses .                
            
blaze:daniel rdf:type blaze:person ;
             blaze:name "daniel" ;
             blaze:81056 blaze:tinkergraph ;
             blaze:e09ac blaze:gremlin ;
             blaze:location "spremberg" ;
             blaze:location "kaiserslautern" ;
             blaze:location "aachen" .
            
<<blaze:daniel blaze:81056 blaze:tinkergraph>>   rdf:type blaze:uses ;
                                                 blaze:skill "3"^^xsd:int .
<<blaze:daniel blaze:e09ac blaze:gremlin>>       rdf:type blaze:uses ;
                                                 blaze:skill "5"^^xsd:int .
<<blaze:daniel blaze:location "spremberg">>      blaze:startTime "1982"^^xsd:int ;
                                                 blaze:endTime "2005"^^xsd:int .
<<blaze:daniel blaze:location "kaiserslautern">> blaze:startTime "2005"^^xsd:int ;
                                                 blaze:endTime "2009"^^xsd:int .
<<blaze:daniel blaze:location "aachen">>         blaze:startTime "2009"^^xsd:int .
            
blaze:marko rdf:type blaze:person ;
            blaze:name "marko" ;
            blaze:42af2 blaze:gremlin ;
            blaze:4edec blaze:gremlin ;
            blaze:61d50 blaze:tinkergraph ;
            blaze:68c12 blaze:tinkergraph ;
            blaze:location "san diego" ;
            blaze:location "santa cruz" ;
            blaze:location "brussels" ;
            blaze:location "santa fe" .

<<blaze:marko blaze:42af2 blaze:gremlin>>     rdf:type blaze:develops ;
                                              blaze:since "2009"^^xsd:int .
<<blaze:marko blaze:4edec blaze:gremlin>>     rdf:type blaze:uses ;
                                              blaze:skill "4"^^xsd:int .
<<blaze:marko blaze:61d50 blaze:tinkergraph>> rdf:type blaze:develops ;
                                              blaze:since "2010"^^xsd:int .
<<blaze:marko blaze:68c12 blaze:tinkergraph>> rdf:type blaze:uses ;
                                              blaze:skill "5"^^xsd:int .
<<blaze:marko blaze:location "san diego">>    blaze:startTime "1997"^^xsd:int ;
                                              blaze:endTime "2001"^^xsd:int .
<<blaze:marko blaze:location "santa cruz">>   blaze:startTime "2001"^