Skip to content

Apache Jena integration

This guide explains the functionalities of the jelly-jena module, which provides Jelly support for Apache Jena.

If you just want to add Jelly format support to Apache Jena / Apache Jena Fuseki, you can use the Jelly-JVM plugin JAR. See the dedicated guide for more information.

Base facilities

jelly-jena implements the eu.ostrzyciel.jelly.core.ConverterFactory trait in eu.ostrzyciel.jelly.convert.jena.JenaConverterFactory . This factory allows you to build encoders and decoders that convert between Jelly's RdfStreamFrames and Apache Jena's Triple and Quad objects. The eu.ostrzyciel.jelly.core.proto.v1.RdfStreamFrame class is an object representation of Jelly's binary format.

The module also implements the eu.ostrzyciel.jelly.core.IterableAdapter trait in eu.ostrzyciel.jelly.convert.jena.JenaIterableAdapter . This adapter provides extension methods for Apache Jena's Model, Dataset, Graph, and DatasetGraph classes to convert them into an iterable of triples (.asTriples), quads (.asQuads), or named graphs (.asGraphs). This is useful when working with Jelly on a lower level or when using the jelly-stream module.

Serialization and deserialization with RIOT

jelly-jena implements an RDF writer and reader for Apache Jena's RIOT library. This means you can use Jelly just like, for example, Turtle or RDF/XML. See the example below:

Example: JenaRiot.scala (click to expand)

Source code on GitHub

JenaRiot.scala
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
package eu.ostrzyciel.jelly.examples

import eu.ostrzyciel.jelly.convert.jena.riot.*
import eu.ostrzyciel.jelly.core.*
import org.apache.jena.rdf.model.ModelFactory
import org.apache.jena.riot.{RDFDataMgr, RDFFormat, RDFParser, RDFWriterRegistry, RIOT}

import java.io.{File, FileOutputStream}
import scala.util.Using

/**
 * Example of using Jelly's integration with Apache Jena's RIOT library for
 * writing and reading RDF graphs and datasets to/from disk.
 *
 * See also: https://jena.apache.org/documentation/io/
 */
object JenaRiot extends shared.Example:
  def main(args: Array[String]): Unit =
    // Load the RDF graph from an N-Triples file
    val model = RDFDataMgr.loadModel(File(getClass.getResource("/weather.nt").toURI).toURI.toString)

    // Print the size of the model
    println(s"Loaded an RDF graph from N-Triples with size: ${model.size}")

    Using.resource(new FileOutputStream("weather.jelly")) { out =>
      // Write the model to a Jelly file
      // Note: by default this will use the [[JellyFormat.JELLY_SMALL_STRICT]] format variant
      RDFDataMgr.write(out, model, JellyLanguage.JELLY)
      println("Saved the model to a Jelly file")
    }

    // Load the RDF graph from a Jelly file
    val model2 = RDFDataMgr.loadModel("weather.jelly", JellyLanguage.JELLY)

    // Print the size of the model
    println(s"Loaded an RDF graph from Jelly with size: ${model2.size}")



    // ---------------------------------
    println("\n")

    // Try the same with an RDF dataset and some different settings
    val dataset = RDFDataMgr.loadDataset(File(getClass.getResource("/weather-graphs.trig").toURI).toURI.toString)
    println(s"Loaded an RDF dataset from a Trig file with ${dataset.asDatasetGraph.size} named graphs and " +
      s"${dataset.asDatasetGraph.stream.count} quads")

    Using.resource(new FileOutputStream("weather-quads.jelly")) { out =>
      // Write the dataset to a Jelly file, using the "BIG" settings
      // (better compression for big files, more memory usage)
      RDFDataMgr.write(out, dataset, JellyFormat.JELLY_BIG_STRICT)
      println("Saved the dataset to a Jelly file")
    }

    // Load the RDF dataset from a Jelly file
    val dataset2 = RDFDataMgr.loadDataset("weather-quads.jelly", JellyLanguage.JELLY)
    println(s"Loaded an RDF dataset from Jelly with ${dataset2.asDatasetGraph.size} named graphs and " +
      s"${dataset2.asDatasetGraph.stream.count} quads")

    // ---------------------------------
    println("\n")

    // Custom Jelly format – change any settings you like
    val customFormat = new RDFFormat(
      JellyLanguage.JELLY,
      JellyFormatVariant(
        opt = JellyOptions.smallStrict
          .withMaxPrefixTableSize(0) // disable the prefix table
          .withStreamName("My weather stream"), // add metadata to the stream
        frameSize = 16 // make RdfStreamFrames with 16 rows each
      )
    )

    // Jena requires us to register the custom format – once for graphs and once for datasets,
    // as Jelly supports both.
    RDFWriterRegistry.register(customFormat, JellyGraphWriterFactory)
    RDFWriterRegistry.register(customFormat, JellyDatasetWriterFactory)

    Using.resource(new FileOutputStream("weather-quads-custom.jelly")) { out =>
      // Write the dataset to a Jelly file using the custom format
      RDFDataMgr.write(out, dataset, customFormat)
      println("Saved the dataset to a Jelly file with custom settings")
    }

    // Load the RDF dataset from a Jelly file with the custom format
    val dataset3 = RDFDataMgr.loadDataset("weather-quads-custom.jelly", JellyLanguage.JELLY)
    println(s"Loaded an RDF dataset from Jelly with custom settings with ${dataset3.asDatasetGraph.size} named graphs" +
      s" and ${dataset3.asDatasetGraph.stream.count} quads")

    // ---------------------------------
    println("\n")

    // By default, the parser has limits on for example the maximum size of the lookup tables.
    // The default supported options are [[JellyOptions.defaultSupportedOptions]].
    // You can change these limits by creating your own options object.
    val customOptions = JellyOptions.defaultSupportedOptions
      .withMaxNameTableSize(50) // set the maximum size of the name table to 100
    // Create a Context object with the custom options
    val parserContext = RIOT.getContext.copy()
      .set(JellyLanguage.SYMBOL_SUPPORTED_OPTIONS, customOptions)

    println("Trying to load the model with custom supported options...")
    val model3 = ModelFactory.createDefaultModel()
    try
      // The loading operation should fail because our allowed max name table size is too low
      RDFParser.create()
        .source("weather.jelly")
        .lang(JellyLanguage.JELLY)
        // Set the context object with the custom options
        .context(parserContext)
        .parse(model3)
    catch
      case e: RdfProtoDeserializationError =>
        // The stream uses a name table size of 128, which is larger than the maximum supported size of 50.
        // To read this stream, set maxNameTableSize to at least 128 in the supportedOptions for this decoder.
        println(s"Failed to load the model with custom options: ${e.getMessage}")

Usage notes:

  • eu.ostrzyciel.jelly.core.JellyOptions provides a few common presets for Jelly serialization options construct a JellyFormatVariant, as shown in the example above. You can also further customize the serialization options (e.g., dictionary size).
  • The RIOT writer (serializer) integration implements only the delimited variant of Jelly. It is used for writing Jelly to files on disk or sockets. Because of this, you cannot use RIOT to write non-delimited Jelly data (e.g., a single message to a Kafka stream). For this, you should use the jelly-stream module or the more low-level API: Low-level usage.
  • However, the RIOT parser (deserializer) integration will automatically detect if the parsed Jelly data is delimited or not. If it's non-delimited, the parser will assume that there is only one RdfStreamFrame in the file.
  • Jelly's parsers and writers are registered in the eu.ostrzyciel.jelly.convert.jena.riot.JellyLanguage object (source code). This registration should happen automatically when you include the jelly-jena module in your project, using Jena's component initialization mechanism.

Streaming serialization with RIOT

jelly-jena also implements a streaming writer (StreamRDF API in Jena). Using it is similar to the regular RIOT writer, with a slightly different setup:

Example: JenaRiotStreaming.scala (click to expand)

Source code on GitHub

JenaRiotStreaming.scala
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
package eu.ostrzyciel.jelly.examples

import eu.ostrzyciel.jelly.convert.jena.riot.*
import eu.ostrzyciel.jelly.core.JellyOptions
import eu.ostrzyciel.jelly.core.proto.v1.PhysicalStreamType
import org.apache.jena.graph.{NodeFactory, Triple}
import org.apache.jena.riot.system.{StreamRDFLib, StreamRDFWriter}
import org.apache.jena.riot.{RDFDataMgr, RDFParser, RIOT}

import java.io.{File, FileOutputStream}
import scala.util.Using

/**
 * Example of using Apache Jena's streaming IO API with Jelly.
 *
 * See also: https://jena.apache.org/documentation/io/streaming-io.html
 */
object JenaRiotStreaming extends shared.Example:
  def main(args: Array[String]): Unit =
    // Initialize a Jena StreamRDF to consume the statements
    val readerStream = StreamRDFLib.count()

    println("Reading a stream of triples from a Jelly file...")

    // Parse a Jelly file as a stream of triples
    val inputFileTriples = new File(getClass.getResource("/jelly/weather.jelly").toURI)
    RDFParser
      .source(inputFileTriples.toURI.toString)
      .lang(JellyLanguage.JELLY)
      .parse(readerStream)

    println(f"Read ${readerStream.countTriples()} triples")
    println()
    println("Reading a stream of quads from a Jelly file...")

    // Parse a different Jelly file as a stream of quads and send it to the same sink
    val inputFileQuads = new File(getClass.getResource("/jelly/weather-quads.jelly").toURI)
    RDFParser
      .source(inputFileQuads.toURI.toString)
      .lang(JellyLanguage.JELLY)
      .parse(readerStream)

    // Print the number of triples and quads
    //
    // The number of triples here is the sum of the triples from the first file and the triples
    // in the default graph of the second file. This is just how Jena handles it.
    println(f"Read ${readerStream.countTriples()} triples (in total)" +
      f" and ${readerStream.countQuads()} quads")

    // -------------------------------------
    println("\n")

    println("Writing a stream of 10 triples to a file...")

    // Try writing some triples to a file
    // We need to create an instance of RdfStreamOptions to pass to the writer:
    val options = JellyOptions.smallStrict
      // The stream writer does not know if we will be writing triples or quads – we
      // have to specify the physical stream type explicitly.
      .withPhysicalType(PhysicalStreamType.TRIPLES)
      .withStreamName("A stream of 10 triples")

    // To pass the options, we use Jena's Context mechanism
    val context = RIOT.getContext.copy()
      .set(JellyLanguage.SYMBOL_STREAM_OPTIONS, options)
      .set(JellyLanguage.SYMBOL_FRAME_SIZE, 128) // optional, default is 256

    Using.resource(new FileOutputStream("stream-riot.jelly")) { out =>
      // Create the writer – remember to pass the context!
      val writerStream = StreamRDFWriter.getWriterStream(out, JellyLanguage.JELLY, context)
      writerStream.start()

      for i <- 1 to 10 do
        writerStream.triple(Triple.create(
          NodeFactory.createBlankNode(),
          NodeFactory.createURI("https://example.org/p"),
          NodeFactory.createLiteralString(s"object $i")
        ))

      writerStream.finish()
    }

    println("Done writing triples")

    // Load the RDF graph that we just saved using normal RIOT API
    val model = RDFDataMgr.loadModel("stream-riot.jelly", JellyLanguage.JELLY)

    println("Loaded the stream from disk, contents:\n")
    model.write(System.out, "NT")

See also