Postlite is a network proxy to allow access to remote SQLite databases over the Postgres wire protocol. This allows GUI tools to be used on remote SQLite databases which can make administration easier.
The proxy works by translating Postgres frontend wire messages into SQLite transactions and converting results back into Postgres response wire messages. Many Postgres clients also inspect the pg_catalog
to determine system information so Postlite mirrors this catalog by using an attached in-memory database with virtual tables. The proxy also performs minor rewriting on these system queries to convert them to usable SQLite syntax.
Note: This software is in alpha. Please report bugs. Postlite doesn't alter your database unless you issue INSERT, UPDATE, DELETE commands so it's probably safe. If anything, the Postlite process may die but it shouldn't affect your database.
Supported clients
Postgres clients can be quite particular about how they initialize so not all clients may work. Below are the clients that are currently being tested. If you would like to see more clients supported or if you're having issues with existing clients, please submit an issue!
Usage
To use Postlite, execute the command with the directory that contains your SQLite databases:
$ postlite -data-dir /data
On another machine, you can connect via the regular Postgres port of 5432:
$ psql --host HOSTNAME my.db
This will connect you to a SQLite database at the path /data/my.db
.
Development
Postlite uses virtual tables to simulate the pg_catalog
so you will need to enable the vtable
tag when building:
$ go install -tags vtable ./cmd/postlite
Contribution Policy
Postlite is open to code contributions for bug fixes & documentation fixes only. Features carry a long-term maintenance burden so they will not be accepted at this time. Please submit an issue if you have a feature you'd like to request.
Possible to use postlite to query osQuery endpoints?
This is a very slick little utility - I have a similar need ... accessing osQuery endpoints. osQuery has its own, special, access points but it would much nicer to have a PostgreSQL wire compatible interface to osQuery endpoints.
Since osQuery is built on top of SQLite, does anyone know if a PostgreSQL wire compatible interface to osQuery endpoints is possible with
postlite
?Drop in replacement for Postgres?
This project looks very interesting and promising! Thanks for bringing this idea out.
I have a couple of questions after going through the README.
Sqlite does not perform well over NFS due to file locking issues that is present with NFS. Using postlite on the NFS server to mitigate that problem would be a humongous value addition from this project. Edit: I see that you also have worked on Litestream! You don't need me to tell you this :)
If it can perform reasonably well (not expecting production grade due to the backing store being sqlite), I can see this being used as a central database server (or even a DBaaS) for many homelabs too :)
add support for pg_range as virtual table
Hi and thanks for a great project. I tried using postlite/sqlite when running some integration tests locally. Sadly they fail because postlite does not support the pg_range catalog table. Would it be possible to add this feature?
I have this small issue using pgMaestro gui
received ssl request message: &pgproto3.SSLRequest{} received startup message: &pgproto3.StartupMessage{ProtocolVersion:0x30000, Parameters:map[string]string{"database":"data.db3", "user":"postgres"}} [recv] &pgproto3.Query{String:"SET DateStyle TO 'ISO, MDY'"} received query: "SET DateStyle TO 'ISO, MDY'" [recv] &pgproto3.Terminate{} connection closed: xxxx
Postico 2 not working
Postico 2 (
2.0beta (9418)
) does not work with postlite. The error is:no such function: current_schemas
The failing query:
Connecting with
psql
works fine.Improve startup state handling
It looks like a remote denial-of-service (via stack overflow) on the server can be triggered by continuously sending
SSLRequest
, sincehandleSSLRequestMessage
recursively callsserveConnStartup
and the call fromserveConnStartup
tohandleSSLRequestMessage
isn't a tail call.This PR introduces a
startupState
type for managing the lifecycle of the startup phase.Client (TablePlus) is not loading the database data
Using TablePlus client connecting does not show the databases or if directly input the database name it doesn't show any tables. PSQL command line does work fine.
Latest version of tableplus
[recv] &pgproto3.Query{String:"SELECT nspname FROM pg_catalog.pg_namespace;"} received query: "SELECT nspname FROM pg_catalog.pg_namespace;" [recv] &pgproto3.Query{String:"SELECT pg_catalog.pg_get_userbyid(p.proowner) as owner,p.oid AS oid,pg_get_function_identity_arguments(p.oid)AS args,n.nspname AS function_schema,p.proname AS function_name,CASE WHEN p.proisagg THEN'aggregate'WHEN p.proiswindow THEN'window'WHEN p.prorettype='pg_catalog.trigger'::pg_catalog.regtype THEN'trigger'ELSE'function'END AS function_type FROM pg_catalog.pg_proc p LEFT JOIN pg_catalog.pg_namespace n ON n.oid=p.pronamespace WHERE n.nspname<>'pg_catalog'AND n.nspname<>'information_schema';"} received query: "SELECT pg_catalog.pg_get_userbyid(p.proowner) as owner,p.oid AS oid,pg_get_function_identity_arguments(p.oid)AS args,n.nspname AS function_schema,p.proname AS function_name,CASE WHEN p.proisagg THEN'aggregate'WHEN p.proiswindow THEN'window'WHEN p.prorettype='pg_catalog.trigger'::pg_catalog.regtype THEN'trigger'ELSE'function'END AS function_type FROM pg_catalog.pg_proc p LEFT JOIN pg_catalog.pg_namespace n ON n.oid=p.pronamespace WHERE n.nspname<>'pg_catalog'AND n.nspname<>'information_schema';" [recv] &pgproto3.Query{String:"(SELECT table_name, table_schema, table_type FROM information_schema.tables) UNION (SELECT c.relname AS table_name, n.nspname AS table_schema, 'MATERIALIZED VIEW' FROM pg_catalog.pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace WHERE c.relkind = 'm');"} received query: "(SELECT table_name, table_schema, table_type FROM information_schema.tables) UNION (SELECT c.relname AS table_name, n.nspname AS table_schema, 'MATERIALIZED VIEW' FROM pg_catalog.pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace WHERE c.relkind = 'm');" [recv] &pgproto3.Query{String:"SELECT p.oid AS oid, p.relname AS table_name, n.nspname as table_schema FROM pg_class AS p JOIN pg_namespace AS n ON p.relnamespace=n.oid WHERE p.relkind='r';"} received query: "SELECT p.oid AS oid, p.relname AS table_name, n.nspname as table_schema FROM pg_class AS p JOIN pg_namespace AS n ON p.relnamespace=n.oid WHERE p.relkind='r';"
can't connect with dbeaver postgres - can connect with psql
can't connect with dbeaver postgres - can connect with psql
dbeaver config: localhost, 5432, nc_timebilling.db
error: can't connect with dbeaver postgres - no databases found on the server
works: $ psql --host localhost nc_timebilling.db
Build instructions
Hi,
I'm likely doing something wrong, but unable to build.
Any hints appreciated.
Thannks!