Getting started
This guide walks you through installing and working with pyjelly and RDFLib.
Installation (with RDFLib)
Install pyjelly from PyPI:
Requirements
- Python 3.10 or newer
- Linux, macOS, or Windows
Usage with RDFLib
Once you install pyjelly, it integrates automatically with RDFLib through standard RDFLib API.
Serializing a graph
To serialize a graph to the Jelly format see:
from rdflib import Graph
g = Graph()
g.parse("https://www.w3.org/TR/vocab-ssn/integrated/examples/sunspots.ttl")
g.serialize(destination="sunspots.jelly", format="jelly")
print(f"Wrote {len(g)} triples to sunspots.jelly")
This creates a delimited Jelly stream using default options.
Including namespace declarations (prefixes)
By default, Jelly serializes only triples/quads. To also include namespace declarations (prefixes) in the output, enable the namespace_declarations option.
Prefixes bound in RDFLib's namespace manager will then be written into the Jelly stream and restored on parsing.
from rdflib import Graph, Literal, Namespace
from rdflib.namespace import FOAF
from pyjelly.options import StreamParameters
from pyjelly.serialize.streams import SerializerOptions
# Build a tiny graph and bind a prefix
EX = Namespace("http://example.org/")
g = Graph()
g.bind("ex", EX)
g.add((EX.alice, FOAF.name, Literal("Alice")))
# Enable namespace declarations in Jelly output
options = SerializerOptions(params=StreamParameters(namespace_declarations=True))
g.serialize("namespaces.jelly", format="jelly", options=options)
# Parse back and check that the prefix is there
g_new = Graph()
g_new.parse("namespaces.jelly", format="jelly")
print("Prefix ex:", dict(g_new.namespaces()).get("ex"))
Tip
For an existing graph you can (re)bind a prefix just before saving:
RDFLib binds about 30 common prefixes to every new graph, and they are all written as well. To write only the prefixes you bind yourself, create the graph with Graph(bind_namespaces="none").
Parsing a graph
To load RDF data from a .jelly file see:
from rdflib import Graph
g = Graph()
g.parse("sunspots.jelly", format="jelly")
print("Parsed triples:")
for s, p, o in g:
print(f"{s} {p} {o}")
RDFLib will reconstruct the graph from the Jelly file.
Parsing a stream of graphs
You can process a Jelly stream as a stream of graphs. A Jelly stream can hold a sequence of graphs, each in its own RDF Message – we can load each one as a separate RDFLib graph.
In this example, we use a dataset of weather measurements. We count the number of triples in each graph:
import gzip
import urllib.request
from itertools import islice
from pyjelly.integrations.rdflib.parse import parse_jelly_grouped
# Dataset: Katrina weather measurements (10k graphs)
# Documentation: https://w3id.org/riverbench/datasets/lod-katrina/dev
url = "https://w3id.org/riverbench/datasets/lod-katrina/dev/files/jelly_10K.jelly.gz"
# Load, uncompress .gz file, and pass to Jelly parser, all in a streaming manner
with (
urllib.request.urlopen(url) as response,
gzip.open(response) as jelly_stream,
):
graphs = parse_jelly_grouped(jelly_stream)
# Only the first 50 graphs, the rest will not be parsed
for i, graph in enumerate(islice(graphs, 50)):
print(f"Graph {i} in the stream has {len(graph)} triples")
Each iteration receives only one graph, allowing for processing large datasets efficiently, without exhausting memory.
Parsing a stream of triples
You can also process a Jelly stream as a flat stream of triples.
We look through a fragment of Denmark's OpenStreetMap to find all city names:
import gzip
import urllib.request
from pyjelly.integrations.rdflib.parse import parse_jelly_flat
from pyjelly.integrations.rdflib.statements import Triple
from rdflib import URIRef
# Dataset: OpenStreetMap data for Denmark (first 10k objects)
# Documentation: https://w3id.org/riverbench/datasets/osm2rdf-denmark/dev
url = (
"https://w3id.org/riverbench/datasets/osm2rdf-denmark/dev/files/jelly_10K.jelly.gz"
)
# We are looking for city names in the dataset
predicate_to_look_for = URIRef("https://www.openstreetmap.org/wiki/Key:addr:city")
city_names = set()
with (
urllib.request.urlopen(url) as response,
gzip.open(response) as jelly_stream,
):
for event in parse_jelly_flat(jelly_stream):
if isinstance(event, Triple): # we are only interested in triples
if event.p == predicate_to_look_for:
city_names.add(event.o)
print(f"Found {len(city_names)} unique city names in the dataset.")
print("Sample city names:")
for city in list(city_names)[:10]:
print(f"- {city}")
parse_jelly_flat returns a generator of stream events (i.e., statements parsed). This case allows you to efficiently process the file triple-by-triple and build custom aggregations from the stream.
Checking IRIs
By default, the IRIs in a Jelly file are not checked when it is read into RDFLib, which makes reading faster. To turn it on, set validate_iris=True, e.g., graph.parse("data.jelly", format="jelly", validate_iris=True). The parsing functions in pyjelly.integrations.rdflib.parse take the same argument.
Serializing a stream of graphs
If you have a generator object containing graphs, you can easily serialize it into the Jelly format:
from pyjelly.integrations.rdflib.serialize import grouped_stream_to_file
from rdflib import Graph, Literal, Namespace
import random
def generate_sample_graphs():
ex = Namespace("http://example.org/")
for _ in range(10):
g = Graph()
g.add((ex.sensor, ex.temperature, Literal(random.random())))
g.add((ex.sensor, ex.humidity, Literal(random.random())))
yield g
output_file_name = "output.jelly"
print(f"Streaming graphs into {output_file_name}…")
sample_graphs = generate_sample_graphs()
with open(output_file_name, "wb") as out_file:
grouped_stream_to_file(sample_graphs, out_file)
print("All done.")
This method allows for transmitting logically grouped data, preserving their original division: each graph is one RDF Message.
Serializing a stream of statements
If you have a generator object containing statements, you can easily serialize it into the Jelly format:
from pyjelly.integrations.rdflib.serialize import flat_stream_to_file
from rdflib import Literal, Namespace
import random
# example generator with triples statements
def generate_sample_triples():
ex = Namespace("http://example.org/")
for _ in range(10):
yield (ex.sensor, ex.temperature, Literal(random.random()))
output_file_name = "flat_output.jelly"
print(f"Streaming triples into {output_file_name}…")
sample_triples = generate_sample_triples()
with open(output_file_name, "wb") as out_file:
flat_stream_to_file(sample_triples, out_file)
print("All done.")
The flat method transmits the data as a continuous sequence of statements, keeping it simple and ordered.
For more control over framing and other options, see pyjelly.integrations.rdflib.serialize in the API reference.
File extension support
When parsing, you can generally omit the format="jelly" parameter if the file ends in .jelly – RDFLib will auto-detect the format:
from rdflib import Graph
import pyjelly.integrations.rdflib
g = Graph()
g.parse("sunspots.jelly")
print(f"Parsed {len(g)} triples")
Warning
Unfortunately, the way this is implemented in RDFLib is a bit wonky, so it will only work if you explicitly import pyjelly.integrations.rdflib, or you used format="jelly" in a serialize() or parse() call before.
This only works for parsing. serialize() does not look at the file extension, so g.serialize("out.jelly") writes Turtle. Always pass format="jelly" when serializing.