Tpt
a84b898fda
|
12 months ago | |
---|---|---|
.. | ||
src | 12 months ago | |
templates | 1 year ago | |
Cargo.toml | 12 months ago | |
Dockerfile | 1 year ago | |
README.md | 12 months ago | |
logo.svg | 1 year ago |
README.md
Oxigraph CLI
Oxigraph CLI is a graph database implementing the SPARQL standard. It is packaged as a command line tool allowing to manipulate an RDF files, query them using SPARQL... It also allows to spawn a HTTP server on top of the database.
Oxigraph is in heavy development and SPARQL query evaluation has not been optimized yet.
Oxigraph provides different installation methods for Oxigraph CLI:
cargo install
(multiplatform)- A Docker image
- A Homebrew formula
- A conda-forge package
- Pre-built binaries
It is also usable as a Rust library and as a Python library.
Oxigraph implements the following specifications:
- SPARQL 1.1 Query, SPARQL 1.1 Update, and SPARQL 1.1 Federated Query.
- Turtle, TriG, N-Triples, N-Quads, and RDF XML RDF serialization formats for both data ingestion and retrieval using the Rio library.
- SPARQL Query Results XML Format, SPARQL 1.1 Query Results JSON Format and SPARQL 1.1 Query Results CSV and TSV Formats.
- SPARQL 1.1 Protocol and SPARQL 1.1 Graph Store HTTP Protocol.
A preliminary benchmark is provided.
Note that Oxigraph CLI was previously named Oxigraph Server before version 0.4. Older versions are available under this name.
Installation
You need to have a recent stable version of Rust and Cargo installed.
To download, build and install the latest released version run cargo install oxigraph-cli
.
There is no need to clone the git repository.
To compile the command line tool from source, clone this git repository including its submodules (git clone --recursive https://github.com/oxigraph/oxigraph.git
), and execute cargo build --release
in the cli
directory to compile the full binary after having downloaded its dependencies.
It will create a fat binary in target/release/oxigraph
.
Some build options (cargo features) are available:
rocksdb-pkg-config
: links against an already compiled rocksdb shared library found using pkg-config.native-tls
: Enables Oxigraph HTTP client for query federation using the host OS TLS stack (enabled by default).rustls-native
Enables Oxigraph HTTP client for query federation using Rustls and the native certificates.rustls-webpki
Enables Oxigraph HTTP client for query federation using Rustls and the Common CA Database certificates.
Usage
Run oxigraph serve --location my_data_storage_directory
to start the server where my_data_storage_directory
is the directory where you want Oxigraph data to be stored. It listens by default on localhost:7878
.
The server provides an HTML UI, based on YASGUI, with a form to execute SPARQL requests.
It provides the following REST actions:
/query
allows evaluating SPARQL queries against the server repository following the SPARQL 1.1 Protocol. For example:
This action supports content negotiation and could return Turtle, N-Triples, RDF XML, SPARQL Query Results XML Format and SPARQL Query Results JSON Format.curl -X POST -H 'Content-Type:application/sparql-query' \ --data 'SELECT * WHERE { ?s ?p ?o } LIMIT 10' http://localhost:7878/query
/update
allows to execute SPARQL updates against the server repository following the SPARQL 1.1 Protocol. For example:curl -X POST -H 'Content-Type: application/sparql-update' \ --data 'DELETE WHERE { <http://example.com/s> ?p ?o }' http://localhost:7878/update
/store
allows to retrieve and change the server content using the SPARQL 1.1 Graph Store HTTP Protocol. For example:
will add the N-Triples filecurl -f -X POST -H 'Content-Type:application/n-triples' \ -T MY_FILE.nt "http://localhost:7878/store?graph=http://example.com/g"
MY_FILE.nt
to the server dataset inside of thehttp://example.com/g
named graph. Turtle, N-Triples and RDF XML are supported. It is also possible toPOST
,PUT
andGET
the complete RDF dataset on the server using RDF dataset formats (TriG and N-Quads) against the/store
endpoint. For example:
will add the N-Quads filecurl -f -X POST -H 'Content-Type:application/n-quads' \ -T MY_FILE.nq http://localhost:7878/store
MY_FILE.nq
to the server dataset.
Use oxigraph --help
to see the possible options when starting the server.
It is also possible to load RDF data offline using bulk loading:
oxigraph load --location my_data_storage_directory --file my_file.nq
Using a Docker image
Display the help menu
docker run --rm ghcr.io/oxigraph/oxigraph --help
Run the Webserver
Expose the server on port 7878
of the host machine, and save data on the local ./data
folder
docker run --rm -v $PWD/data:/data -p 7878:7878 ghcr.io/oxigraph/oxigraph serve --location /data --bind 0.0.0.0:7878
You can then access it from your machine on port 7878
:
# Open the GUI in a browser
firefox http://localhost:7878
# Post some data
curl http://localhost:7878/store?default -H 'Content-Type: text/turtle' -T ./data.ttl
# Make a query
curl -X POST -H 'Accept: application/sparql-results+json' -H 'Content-Type: application/sparql-query' --data 'SELECT * WHERE { ?s ?p ?o } LIMIT 10' http://localhost:7878/query
# Make an UPDATE
curl -X POST -H 'Content-Type: application/sparql-update' --data 'DELETE WHERE { <http://example.com/s> ?p ?o }' http://localhost:7878/update
Run the Web server with basic authentication
It can be useful to make Oxigraph SPARQL endpoint available publicly, with a layer of authentication on /update
to be able to add data.
You can do so by using a nginx basic authentication in an additional docker container with docker-compose
. First create a nginx.conf
file:
daemon off;
events {
worker_connections 1024;
}
http {
server {
server_name localhost;
listen 7878;
rewrite ^/(.*) /$1 break;
proxy_ignore_client_abort on;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header Access-Control-Allow-Origin "*";
location ~ ^(/|/query)$ {
proxy_pass http://oxigraph:7878;
proxy_pass_request_headers on;
}
location ~ ^(/update|/store)$ {
auth_basic "Oxigraph Administrator's Area";
auth_basic_user_file /etc/nginx/.htpasswd;
proxy_pass http://oxigraph:7878;
proxy_pass_request_headers on;
}
}
}
Then a docker-compose.yml
in the same folder, you can change the default user and password in the environment
section:
version: "3"
services:
oxigraph:
image: ghcr.io/oxigraph/oxigraph:latest
## To build from local source code:
# build:
# context: .
# dockerfile: server/Dockerfile
volumes:
- ./data:/data
nginx-auth:
image: nginx:1.21.4
environment:
- OXIGRAPH_USER=oxigraph
- OXIGRAPH_PASSWORD=oxigraphy
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf
## For multiple users: uncomment this line to mount a pre-generated .htpasswd
# - ./.htpasswd:/etc/nginx/.htpasswd
ports:
- "7878:7878"
entrypoint: "bash -c 'echo -n $OXIGRAPH_USER: >> /etc/nginx/.htpasswd && echo $OXIGRAPH_PASSWORD | openssl passwd -stdin -apr1 >> /etc/nginx/.htpasswd && /docker-entrypoint.sh nginx'"
Once the docker-compose.yaml
and nginx.conf
are ready, start the Oxigraph server and nginx proxy for authentication on http://localhost:7878:
docker-compose up
Then it is possible to update the graph using basic authentication mechanisms. For example with curl
: change $OXIGRAPH_USER
and $OXIGRAPH_PASSWORD
, or set them as environment variables, then run this command to insert a simple triple:
curl -X POST -u $OXIGRAPH_USER:$OXIGRAPH_PASSWORD -H 'Content-Type: application/sparql-update' --data 'INSERT DATA { <http://example.com/s> <http://example.com/p> <http://example.com/o> }' http://localhost:7878/update
In case you want to have multiple users, you can comment the entrypoint:
line in the docker-compose.yml
file, uncomment the .htpasswd
volume, then generate each user in the .htpasswd
file with this command:
htpasswd -Bbn $OXIGRAPH_USER $OXIGRAPH_PASSWORD >> .htpasswd
Build the image
You could easily build your own Docker image by cloning this repository with its submodules, and going to the root folder:
git clone --recursive https://github.com/oxigraph/oxigraph.git
cd oxigraph
Then run this command to build the image locally:
docker build -t ghcr.io/oxigraph/oxigraph -f server/Dockerfile .
Homebrew
Oxigraph maintains a Homebrew formula in a custom tap.
To install Oxigraph server using Homebrew do:
brew tap oxigraph/oxigraph
brew install oxigraph
It installs the oxigraph
binary. See the usage documentation to know how to use it.
Systemd
It is possible to run Oxigraph in the background using systemd.
For that, you can use the following oxigraph.service
file (it might be inserted into /etc/systemd/system/
or $HOME/.config/systemd/user
):
[Unit]
Description=Oxigraph database server
After=network-online.target
Wants=network-online.target
[Service]
Type=notify
ExecStart=/PATH/TO/oxigraph serve --location /PATH/TO/OXIGRAPH/DATA
[Install]
WantedBy=multi-user.target
Migration guide
From 0.2 to 0.3
- The cli API has been completely rewritten. To start the server run
oxigraph serve --location MY_STORAGE
instead ofoxigraph --file MY_STORAGE
. - Fast data bulk loading is not supported using
oxigraph load --location MY_STORAGE --file MY_FILE
. The file format is guessed from the extension (.nt
,.ttl
,.nq
...). - RDF-star is now implemented.
- All operations are now transactional using the "repeatable read" isolation level: the store only exposes changes that have been "committed" (i.e. no partial writes) and the exposed state does not change for the complete duration of a read operation (e.g. a SPARQL query) or a read/write operation (e.g. a SPARQL update).
Help
Feel free to use GitHub discussions or the Gitter chat to ask questions or talk about Oxigraph. Bug reports are also very welcome.
If you need advanced support or are willing to pay to get some extra features, feel free to reach out to Tpt.
License
This project is licensed under either of
- Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
- MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)
at your option.
Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in Oxigraph by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.