diff --git a/.github/workflows/artifacts.yml b/.github/workflows/artifacts.yml index ba2fcc36..ddd934c9 100644 --- a/.github/workflows/artifacts.yml +++ b/.github/workflows/artifacts.yml @@ -17,7 +17,7 @@ jobs: with: python-version: 3.8 - run: python -m venv python/venv - - run: source python/venv/bin/activate && pip install --upgrade 'maturin~=0.12' sphinx + - run: source python/venv/bin/activate && pip install --upgrade maturin sphinx - run: source venv/bin/activate && maturin develop working-directory: ./python - run: source ../venv/bin/activate && sphinx-build -M doctest . build diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 66af914f..b29c0c3a 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -158,7 +158,7 @@ jobs: with: python-version: 3.8 - run: python -m venv python/venv - - run: source python/venv/bin/activate && pip install --upgrade 'maturin~=0.12' sphinx + - run: source python/venv/bin/activate && pip install --upgrade maturin sphinx - run: source venv/bin/activate && maturin develop working-directory: ./python - run: source ../venv/bin/activate && sphinx-build -M doctest . build diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 0f54330e..de818a59 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -90,7 +90,7 @@ jobs: with: python-version: "3.10" - run: python -m venv python/venv - - run: source python/venv/bin/activate && pip install --upgrade 'maturin~=0.12' sphinx + - run: source python/venv/bin/activate && pip install --upgrade maturin sphinx - run: source venv/bin/activate && maturin develop working-directory: ./python - run: source ../venv/bin/activate && python -m unittest diff --git a/python/docs/index.rst b/python/docs/index.rst index 9182c31b..3b29138c 100644 --- a/python/docs/index.rst +++ b/python/docs/index.rst @@ -59,9 +59,9 @@ Table of contents """"""""""""""""" .. toctree:: - :maxdepth: 2 model io store sparql + migration diff --git a/python/docs/migration.rst b/python/docs/migration.rst new file mode 100644 index 00000000..0ebc1cba --- /dev/null +++ b/python/docs/migration.rst @@ -0,0 +1,34 @@ +Migration Guide +=============== + +From 0.2 to 0.3 +""""""""""""""" + +* Python 3.6 and ``manylinux2010`` (`PEP 571 `_) support have been removed. The new minimal versions are Python 3.7 and ``manylinux2014`` (`PEP 599 `_). +* The on-disk storage system has been rebuilt on top of `RocksDB `_. + It is now implemented by the :py:class:`.Store` class that keeps the same API as the late :py:class:`.SledStore` class. + + To migrate you have to dump the store content using pyoxigraph **0.2** and the following code: + + .. code-block:: python + + from pyoxigraph import SledStore + store = SledStore('MY_STORAGE_PATH') + with open('temp_file.nq', 'wb') as fp: + store.dump(fp, "application/n-quads") + + And then upgrade to pyoxigraph **0.3** and run: + + .. code-block:: python + + from pyoxigraph import Store + store = Store('MY_NEW_STORAGE_PATH') + with open('temp_file.nq', 'rb') as fp: + store.bulk_load(fp, "application/n-quads") + +* The in-memory storage class :py:class:`.MemoryStore` has been merged into the :py:class:`.Store` class that provides the exact same API as the late :py:class:`.MemoryStore`. + On platforms other than Linux, a temporary directory is created when opening the :py:class:`.Store` and automatically removed when it is garbage collected. No data is written in this directory. +* :py:class:`.Store` operations are now transactional using the "repeatable read" isolation level: + the store only exposes changes that have been "committed" (i.e. no partial writes) + and the exposed state does not change for the complete duration of a read operation (e.g. a SPARQL query) or a read/write operation (e.g. a SPARQL update). +* `RDF-star `_ is now supported (including serialization formats and SPARQL-star). :py:class:`.Triple` can now be used in :py:attr:`.Triple.object`, :py:attr:`.Triple.object`, :py:attr:`.Quad.subject` and :py:attr:`.Quad.object`. diff --git a/python/docs/store.rst b/python/docs/store.rst index 32bff252..2fea9ab3 100644 --- a/python/docs/store.rst +++ b/python/docs/store.rst @@ -1,5 +1,5 @@ -Disk-based RDF Store -==================== +RDF Store +========= .. autoclass:: pyoxigraph.Store :members: diff --git a/python/src/store.rs b/python/src/store.rs index 31141a7e..14390102 100644 --- a/python/src/store.rs +++ b/python/src/store.rs @@ -12,7 +12,7 @@ use pyo3::prelude::*; use pyo3::{Py, PyRef}; use std::io::BufReader; -/// Disk-based RDF store. +/// RDF store. /// /// It encodes a `RDF dataset `_ and allows to query it using SPARQL. /// It is based on the `RocksDB `_ key-value database. @@ -21,7 +21,9 @@ use std::io::BufReader; /// been "committed" (i.e. no partial writes) and the exposed state does not change for the complete duration /// of a read operation (e.g. a SPARQL query) or a read/write operation (e.g. a SPARQL update). /// -/// :param path: the path of the directory in which the store should read and write its data. If the directory does not exist, it is created. If no directory is provided a temporary one is created and removed when the Python garbage collector removes the store. +/// :param path: the path of the directory in which the store should read and write its data. If the directory does not exist, it is created. +/// If no directory is provided a temporary one is created and removed when the Python garbage collector removes the store. +/// In this case the store data are kept in memory and never written on disk. /// :type path: str or None, optional. /// :raises IOError: if the target directory contains invalid data or could not be accessed. /// @@ -313,7 +315,7 @@ impl PyStore { /// Loads an RDF serialization into the store. /// - /// This function is designed to be as fast as possible on big files without transactional guarantees. + /// This function is designed to be as fast as possible on big files **without** transactional guarantees. /// If the file is invalid only a piece of it might be written to the store. /// /// The :py:func:`load` method is also available for loads with transactional guarantees.