Skip to content

SPARQL query results

pyjelly reads and writes SPARQL query results in Jelly-SPARQL: the results of SELECT queries (solution sequences) and ASK queries (boolean results). Like Jelly-RDF, it is a compact binary format that can be written and read as a stream.

The media type of Jelly-SPARQL is application/x-jelly-sparql, and the file extension is .jellys.

Usage with RDFLib

With pyjelly installed, RDFLib reads and writes query results in the jelly format, like it does json or xml:

from rdflib import Graph
from rdflib.query import Result

graph = Graph()
graph.parse(
    data="""
        @prefix foaf: <http://xmlns.com/foaf/0.1/> .
        <http://example.org/alice> foaf:name "Alice" ; foaf:knows <http://example.org/bob> .
        <http://example.org/bob> foaf:name "Bob" .
    """,
    format="turtle",
)

# Run a query and write its results in Jelly-SPARQL
results = graph.query("""
    PREFIX foaf: <http://xmlns.com/foaf/0.1/>
    SELECT ?person ?name WHERE { ?person foaf:name ?name }
""")
results.serialize(destination="results.jellys", format="jelly")

# Read them back
with open("results.jellys", "rb") as file:
    parsed = Result.parse(file, format="jelly")

for row in parsed:
    print(f"{row.person} is called {row.name}")

# Results of ASK queries work the same way
answer = graph.query("ASK { ?person ?p 'Alice' }")
answer.serialize(destination="answer.jellys", format="jelly")
with open("answer.jellys", "rb") as file:
    print("Is anyone called Alice?", Result.parse(file, format="jelly").askAnswer)

Result.parse returns a JellySparqlResult (from pyjelly.integrations.rdflib.sparql), which is an RDFLib Result with more attributes:

  • truncated – True if the stream ended without indicating that the results are complete. pyjelly also issues a JellyTruncatedResultsWarning then. A stream cut in the middle of a frame can't be read at all, and raises JellyConformanceError.
  • error – why the results are incomplete, in a sequence of results. See: incomplete results.
  • links – the IRIs of links that describe the results, like head.link in SPARQL JSON results.

RDFLib does not support RDF 1.2 yet, so triple terms and literals with a base direction can be read only with the generic API.

Querying a SPARQL endpoint

RDFLib's SPARQLStore can ask a SPARQL endpoint for Jelly-SPARQL, if the endpoint offers it (e.g., Apache Jena Fuseki with the Jelly-JVM plugin):

from rdflib import Graph
from rdflib.plugins.stores.sparqlstore import SPARQLStore

store = SPARQLStore(
    "http://localhost:3030/dataset/sparql",
    returnFormat="application/x-jelly-sparql",
)
for row in Graph(store=store).query("SELECT * WHERE { ?s ?p ?o } LIMIT 10"):
    print(row)

Warning

Use the media type, application/x-jelly-sparql, as returnFormat, not jelly.

Large results

By default, Result.parse reads all rows at once. With Result.parse(file, format="jelly", lazy=True), the rows are read as the result is iterated, like RDFLib does with the results of its own queries. The stream must then stay open until you are done with the result.

When writing, the rows of an RDFLib result are written as the query makes them, so the first frames are written before the query is done. Each frame is flushed when it's written.

from rdflib import Literal, URIRef, Variable
from rdflib.query import Result

from pyjelly.errors import JellyIncompleteResultsError

# Many results, made one by one
results = Result("SELECT")
results.vars = [Variable("item"), Variable("number")]
results.bindings = (
    {
        Variable("item"): URIRef(f"http://example.org/item/{i}"),
        Variable("number"): Literal(i),
    }
    for i in range(100_000)
)
results.serialize(destination="many_results.jellys", format="jelly")

# Read the rows as they are needed, not all at once
with open("many_results.jellys", "rb") as file:
    parsed = Result.parse(file, format="jelly", lazy=True)
    try:
        total = sum(row.number.toPython() for row in parsed)
    except JellyIncompleteResultsError as error:
        # The writer said that the results are incomplete, e.g., the query failed
        print(f"Some results are missing: {error.error}")
        raise

print(f"The sum of the numbers is {total}")

Note

Iterate a lazy result with a for loop. list(result), len(result) and bool(result) read all of its rows first, as for any RDFLib result.

Incomplete results

A query can fail after some of its results were already written, for example in case of a timeout, or a broken backend. Jelly-SPARQL then ends the stream with an error, so that readers know that the results are incomplete. pyjelly does this for you: if getting a row fails while results are being written, the stream ends with the error, and the error is raised.

When reading such results, pyjelly raises JellyIncompleteResultsError after the rows that were read. Its error attribute is the message of the writer, and its result attribute contains the results read before the error:

from pyjelly.errors import JellyIncompleteResultsError

try:
    result = Result.parse(file, format="jelly")
except JellyIncompleteResultsError as error:
    print(f"The query failed: {error.error}")
    partial_result = error.result

If you write results that you read from Jelly-SPARQL, and they were incomplete or truncated, pyjelly writes them as incomplete too.

Options

To change how results are written, pass SparqlOptions to serialize:

from pyjelly.options import LookupPreset, SparqlOptions

options = SparqlOptions(
    # Sizes of the lookups, see below
    lookup_preset=LookupPreset(max_names=4096, max_prefixes=256, max_datatypes=32),
    # How many values (rows times variables) a frame may have, at most
    frame_values=1024,
)
results.serialize(destination="results.jellys", format="jelly", options=options)

The default lookup sizes are 8192 names, 1024 prefixes and 64 datatypes, and a frame has at most 4096 values by default. Readers accept lookups of up to 16384 names, 4096 prefixes and 256 datatypes.

Note

LookupPreset() without arguments has the smaller defaults of Jelly-RDF (4000 names, 150 prefixes and 32 datatypes), not the defaults of SparqlOptions. Give all three sizes when you make a LookupPreset for SPARQL results.

With delimited=False, the results are written as one frame without its size, e.g., to be used in Kafka or MQTT. Then the lookups must be large enough to store all terms of the result set. Files and HTTP responses must be delimited, which is the default. To send the frames of the results as separate messages, see writing frame by frame.

Sequences of results

A Jelly-SPARQL stream can also contain a sequence of results, e.g., the answers to a stream of queries, or to a query evaluated again and again. This is the PUNCTUATED stream type of Jelly-SPARQL. To write and read such a stream, use the functions in pyjelly.integrations.rdflib.sparql:

from pyjelly.integrations.rdflib.sparql import (
    parse_jelly_sparql_sequence,
    serialize_jelly_sparql_sequence,
)

with open("answers.jellys", "wb") as file:
    serialize_jelly_sparql_sequence([result_1, result_2, ask_result], file)

with open("answers.jellys", "rb") as file:
    for result in parse_jelly_sparql_sequence(file):
        print(result.type, len(result))

The lookups are shared by the results of the stream, so the terms they have in common are written once.

An incomplete result doesn't end a sequence, as there may be more results after it. parse_jelly_sparql_sequence doesn't raise an error for it, but sets its error attribute, so check it.

Generic API

The generic API supports all of RDF 1.2. A row is a tuple with one value per variable, or None if the variable is unbound:

from pyjelly.integrations.generic.generic_sink import IRI, Literal
from pyjelly.integrations.generic.sparql import (
    parse_jelly_sparql,
    serialize_jelly_sparql,
)

alice = IRI("http://example.org/alice")
bob = IRI("http://example.org/bob")

# The rows of the results: one value per variable, None if it is unbound
rows = [
    (alice, Literal("Alice"), bob),
    (bob, Literal("Bob"), None),
]
with open("results.jellys", "wb") as file:
    serialize_jelly_sparql(["person", "name", "knows"], rows, file)

with open("results.jellys", "rb") as file:
    results = parse_jelly_sparql(file)
    print("Variables:", results.variables)
    # The rows are read as they are iterated, so before the file is closed
    for person, name, knows in results:
        print(person, name, knows)

parse_jelly_sparql returns a SparqlResultSet. Its rows are read as you iterate it, so the input must stay open until then, and the rows can be iterated only once. If the results are incomplete, iterating the rows raises JellyIncompleteResultsError after the last row.

serialize_jelly_sparql_ask writes the result of an ASK query. parse_jelly_sparql_sequence and serialize_jelly_sparql_sequence read and write sequences of results.

Writing frame by frame

To write results frame by frame, e.g., to send each frame as a message over Kafka, use SparqlStream directly. It works with the terms of RDFLib (pyjelly.integrations.rdflib.sparql.sparql_stream()) or of the generic API (pyjelly.integrations.generic.sparql.sparql_stream()):

from pyjelly.integrations.generic.sparql import sparql_stream

stream = sparql_stream()
stream.start(["person", "name"])
for row in rows:
    # A frame, when it is full
    if frame := stream.row(row):
        send(frame.SerializeToString())
# The last frame of the results, which says they are complete
send(stream.end().SerializeToString())

If the results can't be completed, end them with stream.end("why") instead.

Conformance

pyjelly passes the conformance tests of Jelly-SPARQL, with the generic API. The RDFLib integration passes all tests except those that use RDF 1.2 terms.