id,node_id,name,full_name,private,owner,html_url,description,fork,created_at,updated_at,pushed_at,homepage,size,stargazers_count,watchers_count,language,has_issues,has_projects,has_downloads,has_wiki,has_pages,forks_count,archived,disabled,open_issues_count,license,topics,forks,open_issues,watchers,default_branch,permissions,temp_clone_token,organization,network_count,subscribers_count,readme,readme_html,allow_forking,visibility,is_template,template_repository,web_commit_signoff_required,has_discussions
138669673,MDEwOlJlcG9zaXRvcnkxMzg2Njk2NzM=,datasette-vega,simonw/datasette-vega,0,9599,https://github.com/simonw/datasette-vega,Datasette plugin for visualizing data using Vega,0,2018-06-26T01:40:54Z,2021-12-10T22:20:46Z,2021-12-10T22:20:43Z,,59,42,42,JavaScript,1,1,1,1,0,2,0,0,31,apache-2.0,"[""datasette"", ""datasette-io"", ""datasette-plugin"", ""plugin"", ""react"", ""vega""]",2,31,42,master,"{""admin"": false, ""maintain"": false, ""push"": false, ""triage"": false, ""pull"": false}",,,2,2,"# datasette-vega
[](https://pypi.org/project/datasette-vega/)
[](https://github.com/simonw/datasette-vega/blob/master/LICENSE)
A [Datasette](https://github.com/simonw/datasette) plugin that provides tools
for generating charts using [Vega](https://vega.github.io/).
Try out the latest master build as a live demo at https://datasette-vega-latest.datasette.io/ or try the latest release installed as a plugin at https://fivethirtyeight.datasettes.com/
To add this to your Datasette installation, install the plugin like so:
pip install datasette-vega
The plugin will then add itself to every Datasette table view.
If you are publishing data using the `datasette publish` command, you can
include this plugin like so:
datasette publish now mydatabase.db --install=datasette-vega
","
datasette-vega
A Datasette plugin that provides tools
for generating charts using Vega.
To add this to your Datasette installation, install the plugin like so:
pip install datasette-vega
The plugin will then add itself to every Datasette table view.
If you are publishing data using the datasette publish command, you can
include this plugin like so:
datasette publish now mydatabase.db --install=datasette-vega
",1,public,0,,,
195145678,MDEwOlJlcG9zaXRvcnkxOTUxNDU2Nzg=,sqlite-diffable,simonw/sqlite-diffable,0,9599,https://github.com/simonw/sqlite-diffable,Tools for dumping/loading a SQLite database to diffable directory structure,0,2019-07-04T00:58:46Z,2022-07-12T17:00:19Z,2022-08-18T22:49:29Z,,30,42,42,Python,1,1,1,1,0,3,0,0,3,apache-2.0,"[""datasette-io"", ""datasette-tool"", ""sqlite""]",3,3,42,main,"{""admin"": false, ""maintain"": false, ""push"": false, ""triage"": false, ""pull"": false}",,,3,1,"# sqlite-diffable
[](https://pypi.org/project/sqlite-diffable/)
[](https://github.com/simonw/sqlite-diffable/releases)
[](https://github.com/simonw/sqlite-diffable/blob/main/LICENSE)
Tools for dumping/loading a SQLite database to diffable directory structure
## Installation
pip install sqlite-diffable
## Demo
The repository at [simonw/simonwillisonblog-backup](https://github.com/simonw/simonwillisonblog-backup) contains a backup of the database on my blog, https://simonwillison.net/ - created using this tool.
## Dumping a database
Given a SQLite database called `fixtures.db` containing a table `facetable`, the following will dump out that table to the `dump/` directory:
sqlite-diffable dump fixtures.db dump/ facetable
To dump out every table in that database, use `--all`:
sqlite-diffable dump fixtures.db dump/ --all
## Loading a database
To load a previously dumped database, run the following:
sqlite-diffable load restored.db dump/
This will show an error if any of the tables that are being restored already exist in the database file.
You can replace those tables (dropping them before restoring them) using the `--replace` option:
sqlite-diffable load restored.db dump/ --replace
## Converting to JSON objects
Table rows are stored in the `.ndjson` files as newline-delimited JSON arrays, like this:
```
[""a"", ""a"", ""a-a"", 63, null, 0.7364712141640124, ""$null""]
[""a"", ""b"", ""a-b"", 51, null, 0.6020187290499803, ""$null""]
```
Sometimes it can be more convenient to work with a list of JSON objects.
The `sqlite-diffable objects` command can read a `.ndjson` file and its accompanying `.metadata.json` file and output JSON objects to standard output:
sqlite-diffable objects fixtures.db dump/sortable.ndjson
The output of that command looks something like this:
```
{""pk1"": ""a"", ""pk2"": ""a"", ""content"": ""a-a"", ""sortable"": 63, ""sortable_with_nulls"": null, ""sortable_with_nulls_2"": 0.7364712141640124, ""text"": ""$null""}
{""pk1"": ""a"", ""pk2"": ""b"", ""content"": ""a-b"", ""sortable"": 51, ""sortable_with_nulls"": null, ""sortable_with_nulls_2"": 0.6020187290499803, ""text"": ""$null""}
```
Add `-o` to write that output to a file:
sqlite-diffable objects fixtures.db dump/sortable.ndjson -o output.txt
Add `--array` to output a JSON array of objects, as opposed to a newline-delimited file:
sqlite-diffable objects fixtures.db dump/sortable.ndjson --array
Output:
```
[
{""pk1"": ""a"", ""pk2"": ""a"", ""content"": ""a-a"", ""sortable"": 63, ""sortable_with_nulls"": null, ""sortable_with_nulls_2"": 0.7364712141640124, ""text"": ""$null""},
{""pk1"": ""a"", ""pk2"": ""b"", ""content"": ""a-b"", ""sortable"": 51, ""sortable_with_nulls"": null, ""sortable_with_nulls_2"": 0.6020187290499803, ""text"": ""$null""}
]
```
## Storage format
Each table is represented as two files. The first, `table_name.metadata.json`, contains metadata describing the structure of the table. For a table called `redirects_redirect` that file might look like this:
```json
{
""name"": ""redirects_redirect"",
""columns"": [
""id"",
""domain"",
""path"",
""target"",
""created""
],
""schema"": ""CREATE TABLE [redirects_redirect] (\n [id] INTEGER PRIMARY KEY,\n [domain] TEXT,\n [path] TEXT,\n [target] TEXT,\n [created] TEXT\n)""
}
```
It is an object with three keys: `name` is the name of the table, `columns` is an array of column strings and `schema` is the SQL schema text used for tha table.
The second file, `table_name.ndjson`, contains [newline-delimited JSON](http://ndjson.org/) for every row in the table. Each row is represented as a JSON array with items corresponding to each of the columns defined in the metadata.
That file for the `redirects_redirect.ndjson` table might look like this:
```
[1, ""feeds.simonwillison.net"", ""swn-everything"", ""https://simonwillison.net/atom/everything/"", ""2017-10-01T21:11:36.440537+00:00""]
[2, ""feeds.simonwillison.net"", ""swn-entries"", ""https://simonwillison.net/atom/entries/"", ""2017-10-01T21:12:32.478849+00:00""]
[3, ""feeds.simonwillison.net"", ""swn-links"", ""https://simonwillison.net/atom/links/"", ""2017-10-01T21:12:54.820729+00:00""]
```
","
sqlite-diffable
Tools for dumping/loading a SQLite database to diffable directory structure
Each table is represented as two files. The first, table_name.metadata.json, contains metadata describing the structure of the table. For a table called redirects_redirect that file might look like this:
It is an object with three keys: name is the name of the table, columns is an array of column strings and schema is the SQL schema text used for tha table.
The second file, table_name.ndjson, contains newline-delimited JSON for every row in the table. Each row is represented as a JSON array with items corresponding to each of the columns defined in the metadata.
That file for the redirects_redirect.ndjson table might look like this: