BigGraph Developer Guide
BigGraph is mainly a big library providing services to handle most of the aspects of data storage, search, data model, collaboration, cypher queries, data worker runners, long running process runners and much more.
Secondly, BigGraph can also run by itself, featuring a Bolt Server to execute Cypher queries, the CypherLab and runners for Data Worker plugins, Long Running Process plugins and External Worker plugins. See the Architecture and Concepts page for details regarding what these mean.
BigGraph is NOT a transactional system. It's main purpose is to ingest, store and search through large amounts of data in a fast way. It does not have the notion of a transaction, commit or rollback.

Direct API

You can use the Direct API with RocksDB and Elasticsearch by including the following Maven dependencies (available in Maven Central).
1
<dependency>
2
<groupId>io.bigconnect</groupId>
3
<artifactId>core</artifactId>
4
<version>4.2.0</version>
5
</dependency>
6
<dependency>
7
<groupId>io.bigconnect</groupId>
8
<artifactId>storage-rocksdb</artifactId>
9
<version>4.2.0</version>
10
</dependency>
11
<dependency>
12
<groupId>io.bigconnect</groupId>
13
<artifactId>search-index</artifactId>
14
<version>4.2.0</version>
15
</dependency>
Copied!
The following example creates Graph instance and performs some basic operations.
1
import com.mware.ge.*;
2
import com.mware.core.model.schema.SchemaConstants;
3
import static com.mware.ge.rocksdb.RocksDBGraphConfiguration.*;
4
import static com.mware.ge.elasticsearch5.DefaultIndexSelectionStrategy.*;
5
import static com.mware.ge.values.storable.Values.*;
6
7
/* Create a Graph instance (connect to RocksDB and Elasticsearch) */
8
Map configMap = new HashMap();
9
configMap.put(STREAMING_PROPERTY_VALUE_DATA_FOLDER, "/tmp/biggraph/spv");
10
configMap.put(DATA_PATH, dataPath.toString(), "/tmp/biggraph/data/");
11
configMap.put(WAL_PATH, walPath.toString(), "/tmp/biggraph/data/wal");
12
configMap.put(SEARCH_INDEX_PROP_PREFIX, Elasticsearch5SearchIndex.class.getName());
13
configMap.put(SEARCH_INDEX_PROP_PREFIX + "." + CONFIG_INDEX_NAME, "biggraph");
14
configMap.put(SEARCH_INDEX_PROP_PREFIX + "." + CONFIG_INDEX_NAME, "biggraph");
15
configMap.put(SEARCH_INDEX_PROP_PREFIX + "." + CLUSTER_NAME, "biggraph");
16
configMap.put(SEARCH_INDEX_PROP_PREFIX + "." + ES_LOCATIONS, "localhost");
17
Graph graph = RocksDBGraph.create(configMap);
18
19
/* Create a vertex with a property */
20
Authorizations authorizations = graph.createAuthorizations();
21
Vertex v1 = graph.prepareVertex(Visibility.EMPTY, SchemaConstants.CONCEPT_TYPE_THING)
22
.setProperty("prop1", stringValue("value1"), null, Visibility.EMPTY)
23
.save(authorizations);
24
Vertex v2 = graph.prepareVertex(Visibility.EMPTY, SchemaConstants.CONCEPT_TYPE_THING)
25
.setProperty("prop1", stringValue("value2"), null, Visibility.EMPTY)
26
.save(authorizations);
27
graph.flush();
28
29
/* Create an edge between the two vertices and set a property on the edge */
30
Edge edge = graph.prepareEdge(v1, v2, "v1Tov2", Visibility.EMPTY)
31
.setProperty("prop2", stringValue("value3"), Visibility.EMPTY)
32
.save(authorizations);
33
getGraph().flush();
34
35
/* search for a vertex by on of its properties */
36
Query query = graph.query(AUTHORIZATIONS_A).has("prop1", stringValue("value1"));
37
try (QueryResultsIterable<Vertex> results = query.vertices()) {
38
long totalHits = results.getTotalHits();
39
results.forEach(v -> System.out.print(v.getId()));
40
} catch (IOException ex) {
41
ex.printStackTrace();
42
}
43
44
45
/* delete an element */
46
graph.deleteElement(v1, authorizations);
47
Copied!

Cypher

Cypher queries are processed by the Bolt server using a fast binary protocol. Clients connect to it using a specific port (10242 by default) and execute queries, in a similar way to a JDBC driver. Driver libraries are available for Java, PHP and NodeJS.
Below is a simple example on how to connect to a Bolt server and execute a simple Cypher query.
1
Config config = Config.builder()
2
.withoutEncryption()
3
.withTrustStrategy(Config.TrustStrategy.trustAllCertificates())
4
.withLogging(slf4j())
5
.build();
6
7
Driver driver = BigConnect.driver(URI.create("bolt://localhost:10242"), AuthTokens.basic( "admin", "admin"), config);
8
Session session = driver.session();
9
StatementResult r = session.run("match (d:document) return d.title");
10
while (r.hasNext()) {
11
System.out.println(r.next().asMap());
12
}
13
14
session.close();
15
driver.close();
Copied!
As you can see, it's quite similar with running SQL queries through a JDBC driver.
Careful planning of Cypher queries is essential, because they can seriosly damage the performance of your big graph. Avoid queries that make full scans over the entire database, that update the entire database or which need to process a large number of records.
BigGraph is a big data system and does not benefit from optimizations like local page caches, fast disk reads etc. It's a highly distributed system and scanning over huge datasets will require a significant amount of time.
The recommended way to use Cypher is to start from a single node (eg. match by id) and expand the graph starting from that node.

SQL

SQL queries are run against the datastore using Trino. They are usually faster than Cypher queries for specific tasks because Trino pushes down WHERE filters to Elasticsearch shards. The more Trino workers and Elasticsearch shards and replicas you have, the faster your queries will be.
Normally all SQL statements supported by Trino should work.
Inserts and deletes are not designed for large batches. If you plan to insert/delete data through SQL, you will find it to be slow. Please consider the Direct API for batch operations.
Last modified 5mo ago
Copy link