Skip to content

Overview and supported features

What is Jelly and pyjelly?

Jelly is a high-performance serialization format for RDF knowledge graphs and knowledge graph streams. It's designed to be fast, compact, and flexible.

With Jelly, you can transmit both flat and structured streams of triples, quads, graphs, and datasets. Jelly works well in both batch and real-time settings – including files, sockets, or streaming protocols like Kafka or gRPC.

pyjelly is a Python implementation of the Jelly protocol. It provides:

  • Full support for reading and writing Jelly-encoded RDF data
  • Reading and writing SPARQL query results in Jelly-SPARQL
  • Seamless integration with RDFLib ("works just like Turtle")
  • Standalone generic API that doesn't need RDFLib or any other RDF library
  • Support for all Jelly stream types
  • Tools for working with delimited and non-delimited Jelly streams
  • Fine-grained control over serialization options and framing

Supported features

Jelly-RDF versions

pyjelly reads all versions of Jelly-RDF: 1.0, 1.1 and 1.2. It writes Jelly-RDF 1.2 by default, which is smaller and faster than Jelly-RDF 1.1.

Older Jelly parsers can't read Jelly-RDF 1.2. To write Jelly-RDF 1.1 for them, set the protocol version to 2:

from pyjelly.options import StreamParameters
from pyjelly.serialize.streams import SerializerOptions

options = SerializerOptions(params=StreamParameters(version=2))
graph.serialize("out.jelly", format="jelly", options=options)

Supported stream types

pyjelly writes the TRIPLES and QUADS physical stream types. It can also read GRAPHS, which only exists in Jelly-RDF 1.0 and 1.1.

A Jelly-RDF 1.2 stream is either one graph or dataset, or a sequence of RDF Messages. When you write a stream of graphs or datasets (e.g., with grouped_stream_to_file), each is an RDF Message. When you read such a stream with parse_jelly_grouped, you get one graph or dataset per message.

RDF 1.2

The generic API supports RDF 1.2: triple terms, as a Triple in the object of a statement, and directional language-tagged strings, as Literal("text", "en", direction="ltr"). RDFLib 7 doesn't support RDF 1.2 yet.

To declare that a stream uses RDF 1.2, set rdf_version=jelly.RDF_VERSION_1_2 in StreamParameters (jelly is pyjelly.jelly). GenericStatementSink.serialize() doesn't take options, so write the statements with flat_stream_to_file or grouped_stream_to_file:

from pyjelly import jelly
from pyjelly.integrations.generic.generic_sink import IRI, Literal, Triple
from pyjelly.integrations.generic.parse import parse_jelly_flat
from pyjelly.integrations.generic.serialize import flat_stream_to_file
from pyjelly.options import StreamParameters
from pyjelly.serialize.streams import SerializerOptions

alice = IRI("http://example.org/alice")


def generate_statements():
    # A triple term in the object of a statement
    yield Triple(
        alice,
        IRI("http://example.org/says"),
        Triple(alice, IRI("http://example.org/knows"), IRI("http://example.org/bob")),
    )
    # A language-tagged string with a base direction
    yield Triple(
        alice, IRI("http://example.org/name"), Literal("Alice", "en", direction="ltr")
    )


# Declare that the stream uses RDF 1.2
options = SerializerOptions(params=StreamParameters(rdf_version=jelly.RDF_VERSION_1_2))
with open("rdf12.jelly", "wb") as out_file:
    flat_stream_to_file(generate_statements(), out_file, options=options)

with open("rdf12.jelly", "rb") as in_file:
    for statement in parse_jelly_flat(in_file):
        print(statement)

SPARQL query results

pyjelly reads and writes the results of SELECT and ASK queries in Jelly-SPARQL, with RDFLib's Result.parse and Result.serialize, or with the generic API. See SPARQL query results.

Conformance to the Jelly specification

pyjelly is continuously tested for conformance to the Jelly specification. It passes the conformance tests of Jelly-RDF 1.2, and the tests of Jelly-RDF 1.1, except for writing the GRAPHS physical stream type. It also passes the conformance tests of Jelly-SPARQL.

See the conformance tests in the pyjelly repository.

Use cases

Use cases for pyjelly include:

  • Client-server communication – link your client app in Python to the server (e.g., Apache Jena, RDF4J) with Jelly to reduce latency and improve user experience.
  • Inter-service communication – use Jelly to efficiently exchange RDF data between microservices.
  • Data science workflows – use Jelly to read and write RDF data in data science pipelines, enabling efficient processing of large datasets.
    • pyjelly is fully streaming, so it can handle large datasets without loading everything into memory at once.
    • We are working on support for pandas and other data science libraries – stay tuned for updates!
  • Database dumps and bulk loads – quickly read and write large RDF datasets with Jelly, reducing storage space and improving database maintenance tasks.

Integrations with other libraries

pyjelly works with libraries that connect to RDFLib, see:

Pre-compiled binaries

pyjelly is written in pure Python, so you can run it on any platform that supports Python. We also provide pre-compiled binaries using mypyc for better performance. Supported platforms include Windows, macOS, and Linux (x86_64 and ARM64). When installing pyjelly, your package manager will automatically download the right binary, or, if your platform is not supported, it will automatically fall back to the pure Python implementation. You can find the full list of supported platforms on pypi.

In rudimentary benchmarks, the mypyc-compiled version of pyjelly is 20–80% faster than the pure Python version, depending on the workload.