# repos

node_id: MDEwOlJlcG9zaXRvcnkyNDM4ODcwMzY=
name: datasette-configure-fts
full_name: simonw/datasette-configure-fts
html_url: https://github.com/simonw/datasette-configure-fts
description: Datasette plugin for enabling full-text search against selected table columns
created_at: 2020-02-29T01:50:57Z
updated_at: 2020-11-01T02:59:12Z
pushed_at: 2020-11-01T02:59:10Z
language: Python
license: apache-2.0
topics: ["datasette", "datasette-plugin", "datasette-io"]
default_branch: main
permissions: {"admin": false, "push": false, "pull": false}
readme:
  # datasette-configure-fts

  [![PyPI](https://img.shields.io/pypi/v/datasette-configure-fts.svg)](https://pypi.org/project/datasette-configure-fts/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-configure-fts?include_prereleases&label=changelog)](https://github.com/simonw/datasette-configure-fts/releases)
  [![Tests](https://github.com/simonw/datasette-configure-fts/workflows/Test/badge.svg)](https://github.com/simonw/datasette-configure-fts/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-configure-fts/blob/main/LICENSE)

  Datasette plugin for enabling full-text search against selected table columns

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-configure-fts

  ## Usage

  Having installed the plugin, visit `/-/configure-fts` on your Datasette instance to configure FTS for tables on attached writable databases.

  Any time you have permission to configure FTS for a table a menu item will appear in the table actions menu on the table page.

  By default only [the root actor](https://datasette.readthedocs.io/en/stable/authentication.html#using-the-root-actor) can access the page - so you'll need to run Datasette with the `--root` option and click on the link shown in the terminal to sign in and access the page.

  The `configure-fts` permission governs access. You can use permission plugins such as [datasette-permissions-sql](https://github.com/simonw/datasette-permissions-sql) to grant additional access to the write interface.

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-datasette-configure-fts" class="anchor" aria-hidden="true" href="#user-content-datasette-configure-fts"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-configure-fts</h1>
  <p><a href="https://pypi.org/project/datasette-configure-fts/" rel="nofollow"><img src="https://camo.githubusercontent.com/73d7c1152d58fc5ae85794eaf3165fd023270f7f118bde3acaa24522371419b5/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d636f6e6669677572652d6674732e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-configure-fts.svg" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-configure-fts/releases"><img src="https://camo.githubusercontent.com/cd5d1f0fe6b01840d00ffa3e5696a7827c30095262f3ea8c4447f0547e16dc17/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d636f6e6669677572652d6674733f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-configure-fts?include_prereleases&amp;label=changelog" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-configure-fts/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-configure-fts/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-configure-fts/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p>
  <p>Datasette plugin for enabling full-text search against selected table columns</p>
  <h2><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p>Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ datasette install datasette-configure-fts
  "><pre><code>$ datasette install datasette-configure-fts
  </code></pre></div>
  <h2><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p>Having installed the plugin, visit <code>/-/configure-fts</code> on your Datasette instance to configure FTS for tables on attached writable databases.</p>
  <p>Any time you have permission to configure FTS for a table a menu item will appear in the table actions menu on the table page.</p>
  <p>By default only <a href="https://datasette.readthedocs.io/en/stable/authentication.html#using-the-root-actor" rel="nofollow">the root actor</a> can access the page - so you'll need to run Datasette with the <code>--root</code> option and click on the link shown in the terminal to sign in and access the page.</p>
  <p>The <code>configure-fts</code> permission governs access. You can use permission plugins such as <a href="https://github.com/simonw/datasette-permissions-sql">datasette-permissions-sql</a> to grant additional access to the write interface.</p>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyNDU2NzA2NzA=
name: fec-to-sqlite
full_name: simonw/fec-to-sqlite
html_url: https://github.com/simonw/fec-to-sqlite
description: Save FEC campaign finance data to a SQLite database
created_at: 2020-03-07T16:52:49Z
updated_at: 2020-12-19T05:09:05Z
pushed_at: 2020-03-07T18:21:48Z
language: Python
license: apache-2.0
topics: ["sqlite", "fec", "datasette", "datasette-io", "datasette-tool"]
default_branch: master
permissions: {"admin": false, "push": false, "pull": false}
readme:
  # fec-to-sqlite

  [![PyPI](https://img.shields.io/pypi/v/fec-to-sqlite.svg)](https://pypi.org/project/fec-to-sqlite/)
  [![CircleCI](https://circleci.com/gh/simonw/fec-to-sqlite.svg?style=svg)](https://circleci.com/gh/simonw/fec-to-sqlite)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/fec-to-sqlite/blob/master/LICENSE)

  Create a SQLite database using FEC campaign contributions data.

  This tool builds on [fecfile](https://github.com/esonderegger/) by Evan Sonderegger.

  ## How to install

      $ pip install fec-to-sqlite

  ## Usage

      $ fec-to-sqlite filings filings.db 1146148

  This fetches the filing with ID `1146148` and stores it in tables in a SQLite database called `filings.db`. It will create any tables it needs.

  You can pass more than one filing ID, separated by spaces.

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-fec-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-fec-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>fec-to-sqlite</h1>
  <p><a href="https://pypi.org/project/fec-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/93308ffe0f5302a01fe685a905be46fc42abddee239c695af80456aabcd72e94/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6665632d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/fec-to-sqlite.svg" style="max-width:100%;"></a>
  <a href="https://circleci.com/gh/simonw/fec-to-sqlite" rel="nofollow"><img src="https://camo.githubusercontent.com/46d17ba2dacfcf6081f8db0a40f9dfd848e9bb2af54c6652e8358a01a88ba2cd/68747470733a2f2f636972636c6563692e636f6d2f67682f73696d6f6e772f6665632d746f2d73716c6974652e7376673f7374796c653d737667" alt="CircleCI" data-canonical-src="https://circleci.com/gh/simonw/fec-to-sqlite.svg?style=svg" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/fec-to-sqlite/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p>
  <p>Create a SQLite database using FEC campaign contributions data.</p>
  <p>This tool builds on <a href="https://github.com/esonderegger/">fecfile</a> by Evan Sonderegger.</p>
  <h2><a id="user-content-how-to-install" class="anchor" aria-hidden="true" href="#user-content-how-to-install"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>How to install</h2>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ pip install fec-to-sqlite
  "><pre><code>$ pip install fec-to-sqlite
  </code></pre></div>
  <h2><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ fec-to-sqlite filings filings.db 1146148
  "><pre><code>$ fec-to-sqlite filings filings.db 1146148
  </code></pre></div>
  <p>This fetches the filing with ID <code>1146148</code> and stores it in tables in a SQLite database called <code>filings.db</code>. It will create any tables it needs.</p>
  <p>You can pass more than one filing ID, separated by spaces.</p>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyNDU4NTY3MzE=
name: datasette-search-all
full_name: simonw/datasette-search-all
html_url: https://github.com/simonw/datasette-search-all
description: Datasette plugin for searching all searchable tables at once
created_at: 2020-03-08T17:21:54Z
updated_at: 2021-12-19T04:06:49Z
pushed_at: 2022-10-05T01:53:33Z
language: Python
license: apache-2.0
topics: ["datasette", "datasette-io", "datasette-plugin", "search"]
default_branch: main
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
visibility: public
readme:
  # datasette-search-all

  [![PyPI](https://img.shields.io/pypi/v/datasette-search-all.svg)](https://pypi.org/project/datasette-search-all/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-search-all?include_prereleases&label=changelog)](https://github.com/simonw/datasette-search-all/releases)
  [![Tests](https://github.com/simonw/datasette-search-all/workflows/Test/badge.svg)](https://github.com/simonw/datasette-search-all/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-search-all/blob/main/LICENSE)

  Datasette plugin for searching all searchable tables at once.

  ## Installation

  Install the plugin in the same Python environment as Datasette:

      pip install datasette-search-all

  ## Background

  See [datasette-search-all: a new plugin for searching multiple Datasette tables at once](https://simonwillison.net/2020/Mar/9/datasette-search-all/) for background on this project. You can try the plugin out at https://fara.datasettes.com/

  ## Usage

  This plugin only works if at least one of the tables connected to your Datasette instance has been configured for SQLite's full-text search.

  The [Datasette search documentation](https://docs.datasette.io/en/stable/full_text_search.html) includes details on how to enable full-text search for a table.

  You can also use the following tools:

  * [sqlite-utils](https://sqlite-utils.datasette.io/en/stable/cli.html#configuring-full-text-search) includes a command-line tool for enabling full-text search.
  * [datasette-enable-fts](https://github.com/simonw/datasette-enable-fts) is a Datasette plugin that adds a web interface for enabling search for specific columns.

  If the plugin detects at least one searchable table it will add a search form to the homepage.

  You can also navigate to `/-/search` on your Datasette instance to use the search interface directly.

  ## Screenshot

  ![Animated screenshot showing the plugin in action](https://raw.githubusercontent.com/simonw/datasette-search-all/main/animated-screenshot.gif)

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-datasette-search-all" class="anchor" aria-hidden="true" href="#user-content-datasette-search-all"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-search-all</h1>
  <p dir="auto"><a href="https://pypi.org/project/datasette-search-all/" rel="nofollow"><img src="https://camo.githubusercontent.com/2fe2b05de97f33998d29929414fb3878c21e05f5d6820e69a8c1b9aa1a6d7593/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d7365617263682d616c6c2e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-search-all.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-search-all/releases"><img src="https://camo.githubusercontent.com/e686df469f1732c1e8112cf75ac7f2388e793d4eaf381b965cf54876e5a6a75e/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d7365617263682d616c6c3f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-search-all?include_prereleases&amp;label=changelog" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-search-all/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-search-all/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-search-all/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width: 100%;"></a></p>
  <p dir="auto">Datasette plugin for searching all searchable tables at once.</p>
  <h2 dir="auto"><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p dir="auto">Install the plugin in the same Python environment as Datasette:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="pip install datasette-search-all"><pre class="notranslate"><code>pip install datasette-search-all
  </code></pre></div>
  <h2 dir="auto"><a id="user-content-background" class="anchor" aria-hidden="true" href="#user-content-background"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Background</h2>
  <p dir="auto">See <a href="https://simonwillison.net/2020/Mar/9/datasette-search-all/" rel="nofollow">datasette-search-all: a new plugin for searching multiple Datasette tables at once</a> for background on this project. You can try the plugin out at <a href="https://fara.datasettes.com/" rel="nofollow">https://fara.datasettes.com/</a></p>
  <h2 dir="auto"><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p dir="auto">This plugin only works if at least one of the tables connected to your Datasette instance has been configured for SQLite's full-text search.</p>
  <p dir="auto">The <a href="https://docs.datasette.io/en/stable/full_text_search.html" rel="nofollow">Datasette search documentation</a> includes details on how to enable full-text search for a table.</p>
  <p dir="auto">You can also use the following tools:</p>
  <ul dir="auto">
  <li><a href="https://sqlite-utils.datasette.io/en/stable/cli.html#configuring-full-text-search" rel="nofollow">sqlite-utils</a> includes a command-line tool for enabling full-text search.</li>
  <li><a href="https://github.com/simonw/datasette-enable-fts">datasette-enable-fts</a> is a Datasette plugin that adds a web interface for enabling search for specific columns.</li>
  </ul>
  <p dir="auto">If the plugin detects at least one searchable table it will add a search form to the homepage.</p>
  <p dir="auto">You can also navigate to <code>/-/search</code> on your Datasette instance to use the search interface directly.</p>
  <h2 dir="auto"><a id="user-content-screenshot" class="anchor" aria-hidden="true" href="#user-content-screenshot"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Screenshot</h2>
  <p dir="auto"><a target="_blank" rel="noopener noreferrer nofollow" href="https://raw.githubusercontent.com/simonw/datasette-search-all/main/animated-screenshot.gif"><img src="https://raw.githubusercontent.com/simonw/datasette-search-all/main/animated-screenshot.gif" alt="Animated screenshot showing the plugin in action" data-animated-image="" style="max-width: 100%;"></a></p>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyNDYxMDg1NjE=
name: datasette-column-inspect
full_name: simonw/datasette-column-inspect
html_url: https://github.com/simonw/datasette-column-inspect
description: Experimental plugin that adds a column inspector
created_at: 2020-03-09T18:11:00Z
updated_at: 2020-12-09T21:46:10Z
pushed_at: 2020-12-09T21:47:38Z
language: HTML
license: apache-2.0
topics: ["datasette", "datasette-plugin", "datasette-io"]
default_branch: main
permissions: {"admin": false, "push": false, "pull": false}
readme:
  # datasette-column-inspect

  [![PyPI](https://img.shields.io/pypi/v/datasette-column-inspect.svg)](https://pypi.org/project/datasette-column-inspect/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-column-inspect?include_prereleases&label=changelog)](https://github.com/simonw/datasette-column-inspect/releases)
  [![Tests](https://github.com/simonw/datasette-column-inspect/workflows/Test/badge.svg)](https://github.com/simonw/datasette-column-inspect/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-column-inspect/blob/main/LICENSE)

  Highly experimental Datasette plugin for inspecting columns.

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ pip install datasette-column-inspect

  ## Usage

  This plugin adds an icon to each column on the table page which opens an inspection side panel.

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-datasette-column-inspect" class="anchor" aria-hidden="true" href="#user-content-datasette-column-inspect"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-column-inspect</h1>
  <p><a href="https://pypi.org/project/datasette-column-inspect/" rel="nofollow"><img src="https://camo.githubusercontent.com/e5f8f5c2f4a6206e386bea29ca46270bc4bb3d12cc3a0126dc181f1cfde0e7ab/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d636f6c756d6e2d696e73706563742e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-column-inspect.svg" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-column-inspect/releases"><img src="https://camo.githubusercontent.com/cd438ac87792c14aa871310ac3c1e61a61d71e09601e83e7bc5fabeeb5418242/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d636f6c756d6e2d696e73706563743f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-column-inspect?include_prereleases&amp;label=changelog" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-column-inspect/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-column-inspect/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-column-inspect/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p>
  <p>Highly experimental Datasette plugin for inspecting columns.</p>
  <h2><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p>Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ pip install datasette-column-inspect
  "><pre><code>$ pip install datasette-column-inspect
  </code></pre></div>
  <h2><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p>This plugin adds an icon to each column on the table page which opens an inspection side panel.</p>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyNDc1Mjc0Mzg=
name: datasette-edit-schema
full_name: simonw/datasette-edit-schema
html_url: https://github.com/simonw/datasette-edit-schema
description: Datasette plugin for modifying table schemas
created_at: 2020-03-15T18:34:06Z
updated_at: 2022-07-01T22:20:25Z
pushed_at: 2022-08-22T22:45:58Z
language: JavaScript
license: apache-2.0
topics: ["datasette", "datasette-io", "datasette-plugin"]
default_branch: main
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
visibility: public
readme:
  # datasette-edit-schema

  [![PyPI](https://img.shields.io/pypi/v/datasette-edit-schema.svg)](https://pypi.org/project/datasette-edit-schema/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-edit-schema?include_prereleases&label=changelog)](https://github.com/simonw/datasette-edit-schema/releases)
  [![Tests](https://github.com/simonw/datasette-edit-schema/workflows/Test/badge.svg)](https://github.com/simonw/datasette-edit-schema/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-edit-schema/blob/master/LICENSE)

  Datasette plugin for modifying table schemas

  ## Features

  * Add new columns to a table
  * Rename columns in a table
  * Modify the type of columns in a table
  * Re-order the columns in a table
  * Rename a table
  * Delete a table

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ pip install datasette-edit-schema

  ## Usage

  Navigate to `/-/edit-schema/dbname/tablename` on your Datasette instance to edit a specific table.

  Use `/-/edit-schema/dbname` to create a new table in a specific database.

  By default only [the root actor](https://datasette.readthedocs.io/en/stable/authentication.html#using-the-root-actor) can access the page - so you'll need to run Datasette with the `--root` option and click on the link shown in the terminal to sign in and access the page.

  ## Permissions

  The `edit-schema` permission governs access. You can use permission plugins such as [datasette-permissions-sql](https://github.com/simonw/datasette-permissions-sql) to grant additional access to the write interface.

  These permission checks will call the `permission_allowed()` plugin hook with three arguments:

  - `action` will be the string `"edit-schema"`
  - `actor` will be the currently authenticated actor - usually a dictionary
  - `resource` will be the string name of the database

  ## Screenshot

  ![datasette-edit-schema interface](https://raw.githubusercontent.com/simonw/datasette-edit-schema/main/datasette-edit-schema.png)

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-edit-schema
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-datasette-edit-schema" class="anchor" aria-hidden="true" href="#user-content-datasette-edit-schema"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-edit-schema</h1>
  <p dir="auto"><a href="https://pypi.org/project/datasette-edit-schema/" rel="nofollow"><img src="https://camo.githubusercontent.com/c7772b0d1f8c2377409f509e097f39ac2540e5215589dc08e73e1d497cb7e41d/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d656469742d736368656d612e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-edit-schema.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-edit-schema/releases"><img src="https://camo.githubusercontent.com/e6d232795a6c369f9bf6a52f5656bbbc227f6a5d0e6ed99d8a02d703e74b07dc/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d656469742d736368656d613f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-edit-schema?include_prereleases&amp;label=changelog" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-edit-schema/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-edit-schema/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-edit-schema/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width: 100%;"></a></p>
  <p dir="auto">Datasette plugin for modifying table schemas</p>
  <h2 dir="auto"><a id="user-content-features" class="anchor" aria-hidden="true" href="#user-content-features"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Features</h2>
  <ul dir="auto">
  <li>Add new columns to a table</li>
  <li>Rename columns in a table</li>
  <li>Modify the type of columns in a table</li>
  <li>Re-order the columns in a table</li>
  <li>Rename a table</li>
  <li>Delete a table</li>
  </ul>
  <h2 dir="auto"><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p dir="auto">Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="$ pip install datasette-edit-schema"><pre class="notranslate"><code>$ pip install datasette-edit-schema
  </code></pre></div>
  <h2 dir="auto"><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p dir="auto">Navigate to <code>/-/edit-schema/dbname/tablename</code> on your Datasette instance to edit a specific table.</p>
  <p dir="auto">Use <code>/-/edit-schema/dbname</code> to create a new table in a specific database.</p>
  <p dir="auto">By default only <a href="https://datasette.readthedocs.io/en/stable/authentication.html#using-the-root-actor" rel="nofollow">the root actor</a> can access the page - so you'll need to run Datasette with the <code>--root</code> option and click on the link shown in the terminal to sign in and access the page.</p>
  <h2 dir="auto"><a id="user-content-permissions" class="anchor" aria-hidden="true" href="#user-content-permissions"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Permissions</h2>
  <p dir="auto">The <code>edit-schema</code> permission governs access. You can use permission plugins such as <a href="https://github.com/simonw/datasette-permissions-sql">datasette-permissions-sql</a> to grant additional access to the write interface.</p>
  <p dir="auto">These permission checks will call the <code>permission_allowed()</code> plugin hook with three arguments:</p>
  <ul dir="auto">
  <li><code>action</code> will be the string <code>"edit-schema"</code></li>
  <li><code>actor</code> will be the currently authenticated actor - usually a dictionary</li>
  <li><code>resource</code> will be the string name of the database</li>
  </ul>
  <h2 dir="auto"><a id="user-content-screenshot" class="anchor" aria-hidden="true" href="#user-content-screenshot"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Screenshot</h2>
  <p dir="auto"><a target="_blank" rel="noopener noreferrer" href="https://raw.githubusercontent.com/simonw/datasette-edit-schema/main/datasette-edit-schema.png"><img src="https://raw.githubusercontent.com/simonw/datasette-edit-schema/main/datasette-edit-schema.png" alt="datasette-edit-schema interface" style="max-width: 100%;"></a></p>
  <h2 dir="auto"><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p dir="auto">To set up this plugin locally, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="cd datasette-edit-schema
  python3 -mvenv venv
  source venv/bin/activate"><pre class="notranslate"><code>cd datasette-edit-schema
  python3 -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p dir="auto">Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="pipenv shell"><pre class="notranslate"><code>pipenv shell
  </code></pre></div>
  <p dir="auto">Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="pip install -e '.[test]'"><pre class="notranslate"><code>pip install -e '.[test]'
  </code></pre></div>
  <p dir="auto">To run the tests:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="pytest"><pre class="notranslate"><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyNDgzODUyOTk=
name: datasette-publish-fly
full_name: simonw/datasette-publish-fly
html_url: https://github.com/simonw/datasette-publish-fly
description: Datasette plugin for publishing data using Fly
created_at: 2020-03-19T01:47:01Z
updated_at: 2022-09-29T22:28:45Z
pushed_at: 2022-09-29T17:25:15Z
language: Python
license: apache-2.0
topics: ["datasette", "datasette-io", "datasette-plugin", "fly"]
default_branch: main
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
visibility: public
readme:
  # datasette-publish-fly

  [![PyPI](https://img.shields.io/pypi/v/datasette-publish-fly.svg)](https://pypi.org/project/datasette-publish-fly/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-publish-fly?include_prereleases&label=changelog)](https://github.com/simonw/datasette-publish-fly/releases)
  [![Tests](https://github.com/simonw/datasette-publish-fly/workflows/Test/badge.svg)](https://github.com/simonw/datasette-publish-fly/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-publish-fly/blob/main/LICENSE)

  [Datasette](https://datasette.io/) plugin for deploying Datasette instances to [Fly.io](https://fly.io/).

  Project background: [Using SQLite and Datasette with Fly Volumes](https://simonwillison.net/2022/Feb/15/fly-volumes/)

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-publish-fly

  ## Deploying read-only data

  First, install the `flyctl` command-line tool by [following their instructions](https://fly.io/docs/getting-started/installing-flyctl/).

  Run `flyctl auth signup` to create an account there, or `flyctl auth login` if you already have one.

  You can now use `datasette publish fly` to publish one or more SQLite database files:

      datasette publish fly my-database.db --app="my-data-app"

  The argument you pass to `--app` will be used for the URL of your application: `my-data-app.fly.dev`.

  To update an application, run the publish command passing the same application name to the `--app` option.

  Fly have [a free tier](https://fly.io/docs/about/pricing/#free-allowances), beyond which they will charge you monthly for each application you have live.  Details of their pricing can be [found on their site](https://fly.io/docs/pricing/).

  Your application will be deployed at `https://your-app-name.fly.io/` - be aware that it may take several minutes to start working the first time you deploy it.

  ## Using Fly volumes for writable databases

  Fly [Volumes](https://fly.io/docs/reference/volumes/) provide persistant disk storage for Fly applications. Volumes can be 1GB or more in size and the Fly free tier includes 3GB of volume space.

  Datasette plugins such as [datasette-uploads-csvs](https://datasette.io/plugins/datasette-upload-csvs) and [datasette-tiddlywiki](https://datasette.io/plugins/datasette-tiddlywiki) can be deployed to Fly and store their mutable data in a volume.

  > :warning: **You should only run a single instance of your application** if your database accepts writes. Fly has excellent support for running multiple instances in different geographical regions, but `datasette-publish-fly` with volumes is not yet compatible with that model. You should probably [use Fly PostgreSQL instead](https://fly.io/blog/globally-distributed-postgres/).

  Here's how to deploy `datasette-tiddlywiki` with authentication provided by `datasette-auth-passwords`.

  First, you'll need to create a root password hash to use to sign into the instance.

  You can do that by installing the plugin and running the `datasette hash-password` command, or by using [this hosted tool](https://datasette-auth-passwords-demo.datasette.io/-/password-tool).

  The hash should look like `pbkdf2_sha256$...` - you'll need this for the next step.

  In this example we're also deploying a read-only database called `content.db`.

  Pick a name for your new application, then run the following:

      datasette publish fly \
      content.db \
      --app your-application-name \
      --create-volume 1 \
      --create-db tiddlywiki \
      --install datasette-auth-passwords \
      --install datasette-tiddlywiki \
      --plugin-secret datasette-auth-passwords root_password_hash 'pbkdf2_sha256$...'

  This will create the new application, deploy the `content.db` read-only database, create a 1GB volume for that application, create a new database in that volume called `tiddlywiki.db`, then install the two plugins and configure the password you specified.

  ### Updating applications that use a volume

  Once you have deployed an application using a volume, you can update that application without needing the `--create-volume` or `--create-db` options. To add the [datasette-graphq](https://datasette.io/plugins/datasette-graphql) plugin to your deployed application you would run the following:

      datasette publish fly \
      content.db \
      --app your-application-name \
      --install datasette-auth-passwords \
      --install datasette-tiddlywiki \
      --install datasette-graphql \
      --plugin-secret datasette-auth-passwords root_password_hash 'pbkdf2_sha256$...' \

  Since the application name is the same you don't need the `--create-volume` or `--create-db` options - these are persisted automatically between deploys.

  You do need to specify the full list of plugins that you want to have installed, and any plugin secrets.

  You also need to include any read-only database files that are part of the instance - `content.db` in this example - otherwise the new deployment will not include them.

  ### Advanced volume usage

  `datasette publish fly` will add a volume called `datasette` to your Fly application. You can customize the name using the `--volume name custom_name` option.

  Fly can be used to scale applications to run multiple instances in multiple regions around the world. This works well with read-only Datasette but is not currently recommended using Datasette with volumes, since each Fly replica would need its own volume and data stored in one instance would not be visible in others.

  If you want to use multiple instances with volumes you will need to switch to using the `flyctl` command directly. The `--generate-dir` option, described below, can help with this.

  ## Generating without deploying

  Use the `--generate-dir` option to generate a directory that can be deployed to Fly rather than deploying directly:

      datasette publish fly my-database.db \
        --app="my-generated-app" \
        --generate-dir /tmp/deploy-this

  You can then manually deploy your generated application using the following:

      cd /tmp/deploy-this
      flyctl apps create my-generated-app
      flyctl deploy

  ## datasette publish fly --help

  <!-- [[[cog
  import cog
  from datasette import cli
  from click.testing import CliRunner
  runner = CliRunner()
  result = runner.invoke(cli.cli, ["publish", "fly", "--help"])
  help = result.output.replace("Usage: cli", "Usage: datasette")
  cog.out(
      "```\n{}```".format(help)
  )
  ]]] -->
  ```
  Usage: datasette publish fly [OPTIONS] [FILES]...

    Deploy an application to Fly that runs Datasette against the provided database
    files.

    Usage example:

        datasette publish fly my-database.db --app="my-data-app"

    Full documentation: https://datasette.io/plugins/datasette-publish-fly

  Options:
    -m, --metadata FILENAME         Path to JSON/YAML file containing metadata to
                                    publish
    --extra-options TEXT            Extra options to pass to datasette serve
    --branch TEXT                   Install datasette from a GitHub branch e.g.
                                    main
    --template-dir DIRECTORY        Path to directory containing custom templates
    --plugins-dir DIRECTORY         Path to directory containing custom plugins
    --static MOUNT:DIRECTORY        Serve static files from this directory at
                                    /MOUNT/...
    --install TEXT                  Additional packages (e.g. plugins) to install
    --plugin-secret <TEXT TEXT TEXT>...
                                    Secrets to pass to plugins, e.g. --plugin-
                                    secret datasette-auth-github client_id xxx
    --version-note TEXT             Additional note to show on /-/versions
    --secret TEXT                   Secret used for signing secure values, such as
                                    signed cookies
    --title TEXT                    Title for metadata
    --license TEXT                  License label for metadata
    --license_url TEXT              License URL for metadata
    --source TEXT                   Source label for metadata
    --source_url TEXT               Source URL for metadata
    --about TEXT                    About label for metadata
    --about_url TEXT                About URL for metadata
    --spatialite                    Enable SpatialLite extension
    --region TEXT                   Fly region to deploy to, e.g sjc - see
                                    https://fly.io/docs/reference/regions/
    --create-volume INTEGER RANGE   Create and attach volume of this size in GB
                                    [x>=1]
    --create-db TEXT                Names of read-write database files to create
    --volume-name TEXT              Volume name to use
    -a, --app TEXT                  Name of Fly app to deploy  [required]
    -o, --org TEXT                  Name of Fly org to deploy to
    --generate-dir DIRECTORY        Output generated application files and stop
                                    without deploying
    --show-files                    Output the generated Dockerfile, metadata.json
                                    and fly.toml
    --help                          Show this message and exit.
  ```
  <!-- [[[end]]] -->

  ## Development

  To contribute to this tool, first checkout the code. Then create a new virtual environment:

      cd datasette-publish-fly
      python -m venv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

  ### Integration tests

  The tests in `tests/test_integration.py` make actual calls to Fly to deploy a test application.

  These tests are skipped by default. If you have `flyctl` installed and configured, you can run the integration tests like this:

      pytest --integration -s

  The `-s` option here ensures that output from the deploys will be visible to you - otherwise it can look like the tests have hung.

  The tests will create applications on Fly that start with the prefix `publish-fly-temp-` and then delete them at the end of the run.

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-datasette-publish-fly" class="anchor" aria-hidden="true" href="#user-content-datasette-publish-fly"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-publish-fly</h1>
  <p dir="auto"><a href="https://pypi.org/project/datasette-publish-fly/" rel="nofollow"><img src="https://camo.githubusercontent.com/e734388aed6f472c37bdfa14e6ddace47ad6e959c87da3b6399a5313f3558110/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d7075626c6973682d666c792e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-publish-fly.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-publish-fly/releases"><img src="https://camo.githubusercontent.com/35efe08178a809f72d27ab8954426873c2c94d698c5ebd81d7eb284b421aa967/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d7075626c6973682d666c793f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-publish-fly?include_prereleases&amp;label=changelog" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-publish-fly/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-publish-fly/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-publish-fly/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width: 100%;"></a></p>
  <p dir="auto"><a href="https://datasette.io/" rel="nofollow">Datasette</a> plugin for deploying Datasette instances to <a href="https://fly.io/" rel="nofollow">Fly.io</a>.</p>
  <p dir="auto">Project background: <a href="https://simonwillison.net/2022/Feb/15/fly-volumes/" rel="nofollow">Using SQLite and Datasette with Fly Volumes</a></p>
  <h2 dir="auto"><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p dir="auto">Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="$ datasette install datasette-publish-fly"><pre class="notranslate"><code>$ datasette install datasette-publish-fly
  </code></pre></div>
  <h2 dir="auto"><a id="user-content-deploying-read-only-data" class="anchor" aria-hidden="true" href="#user-content-deploying-read-only-data"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Deploying read-only data</h2>
  <p dir="auto">First, install the <code>flyctl</code> command-line tool by <a href="https://fly.io/docs/getting-started/installing-flyctl/" rel="nofollow">following their instructions</a>.</p>
  <p dir="auto">Run <code>flyctl auth signup</code> to create an account there, or <code>flyctl auth login</code> if you already have one.</p>
  <p dir="auto">You can now use <code>datasette publish fly</code> to publish one or more SQLite database files:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="datasette publish fly my-database.db --app=&quot;my-data-app&quot;"><pre class="notranslate"><code>datasette publish fly my-database.db --app="my-data-app"
  </code></pre></div>
  <p dir="auto">The argument you pass to <code>--app</code> will be used for the URL of your application: <code>my-data-app.fly.dev</code>.</p>
  <p dir="auto">To update an application, run the publish command passing the same application name to the <code>--app</code> option.</p>
  <p dir="auto">Fly have <a href="https://fly.io/docs/about/pricing/#free-allowances" rel="nofollow">a free tier</a>, beyond which they will charge you monthly for each application you have live.  Details of their pricing can be <a href="https://fly.io/docs/pricing/" rel="nofollow">found on their site</a>.</p>
  <p dir="auto">Your application will be deployed at <code>https://your-app-name.fly.io/</code> - be aware that it may take several minutes to start working the first time you deploy it.</p>
  <h2 dir="auto"><a id="user-content-using-fly-volumes-for-writable-databases" class="anchor" aria-hidden="true" href="#user-content-using-fly-volumes-for-writable-databases"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Using Fly volumes for writable databases</h2>
  <p dir="auto">Fly <a href="https://fly.io/docs/reference/volumes/" rel="nofollow">Volumes</a> provide persistant disk storage for Fly applications. Volumes can be 1GB or more in size and the Fly free tier includes 3GB of volume space.</p>
  <p dir="auto">Datasette plugins such as <a href="https://datasette.io/plugins/datasette-upload-csvs" rel="nofollow">datasette-uploads-csvs</a> and <a href="https://datasette.io/plugins/datasette-tiddlywiki" rel="nofollow">datasette-tiddlywiki</a> can be deployed to Fly and store their mutable data in a volume.</p>
  <blockquote>
  <p dir="auto"><g-emoji class="g-emoji" alias="warning" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/26a0.png">⚠️</g-emoji> <strong>You should only run a single instance of your application</strong> if your database accepts writes. Fly has excellent support for running multiple instances in different geographical regions, but <code>datasette-publish-fly</code> with volumes is not yet compatible with that model. You should probably <a href="https://fly.io/blog/globally-distributed-postgres/" rel="nofollow">use Fly PostgreSQL instead</a>.</p>
  </blockquote>
  <p dir="auto">Here's how to deploy <code>datasette-tiddlywiki</code> with authentication provided by <code>datasette-auth-passwords</code>.</p>
  <p dir="auto">First, you'll need to create a root password hash to use to sign into the instance.</p>
  <p dir="auto">You can do that by installing the plugin and running the <code>datasette hash-password</code> command, or by using <a href="https://datasette-auth-passwords-demo.datasette.io/-/password-tool" rel="nofollow">this hosted tool</a>.</p>
  <p dir="auto">The hash should look like <code>pbkdf2_sha256$...</code> - you'll need this for the next step.</p>
  <p dir="auto">In this example we're also deploying a read-only database called <code>content.db</code>.</p>
  <p dir="auto">Pick a name for your new application, then run the following:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="datasette publish fly \
  content.db \
  --app your-application-name \
  --create-volume 1 \
  --create-db tiddlywiki \
  --install datasette-auth-passwords \
  --install datasette-tiddlywiki \
  --plugin-secret datasette-auth-passwords root_password_hash 'pbkdf2_sha256$...'"><pre class="notranslate"><code>datasette publish fly \
  content.db \
  --app your-application-name \
  --create-volume 1 \
  --create-db tiddlywiki \
  --install datasette-auth-passwords \
  --install datasette-tiddlywiki \
  --plugin-secret datasette-auth-passwords root_password_hash 'pbkdf2_sha256$...'
  </code></pre></div>
  <p dir="auto">This will create the new application, deploy the <code>content.db</code> read-only database, create a 1GB volume for that application, create a new database in that volume called <code>tiddlywiki.db</code>, then install the two plugins and configure the password you specified.</p>
  <h3 dir="auto"><a id="user-content-updating-applications-that-use-a-volume" class="anchor" aria-hidden="true" href="#user-content-updating-applications-that-use-a-volume"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Updating applications that use a volume</h3>
  <p dir="auto">Once you have deployed an application using a volume, you can update that application without needing the <code>--create-volume</code> or <code>--create-db</code> options. To add the <a href="https://datasette.io/plugins/datasette-graphql" rel="nofollow">datasette-graphq</a> plugin to your deployed application you would run the following:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="datasette publish fly \
  content.db \
  --app your-application-name \
  --install datasette-auth-passwords \
  --install datasette-tiddlywiki \
  --install datasette-graphql \
  --plugin-secret datasette-auth-passwords root_password_hash 'pbkdf2_sha256$...' \"><pre class="notranslate"><code>datasette publish fly \
  content.db \
  --app your-application-name \
  --install datasette-auth-passwords \
  --install datasette-tiddlywiki \
  --install datasette-graphql \
  --plugin-secret datasette-auth-passwords root_password_hash 'pbkdf2_sha256$...' \
  </code></pre></div>
  <p dir="auto">Since the application name is the same you don't need the <code>--create-volume</code> or <code>--create-db</code> options - these are persisted automatically between deploys.</p>
  <p dir="auto">You do need to specify the full list of plugins that you want to have installed, and any plugin secrets.</p>
  <p dir="auto">You also need to include any read-only database files that are part of the instance - <code>content.db</code> in this example - otherwise the new deployment will not include them.</p>
  <h3 dir="auto"><a id="user-content-advanced-volume-usage" class="anchor" aria-hidden="true" href="#user-content-advanced-volume-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Advanced volume usage</h3>
  <p dir="auto"><code>datasette publish fly</code> will add a volume called <code>datasette</code> to your Fly application. You can customize the name using the <code>--volume name custom_name</code> option.</p>
  <p dir="auto">Fly can be used to scale applications to run multiple instances in multiple regions around the world. This works well with read-only Datasette but is not currently recommended using Datasette with volumes, since each Fly replica would need its own volume and data stored in one instance would not be visible in others.</p>
  <p dir="auto">If you want to use multiple instances with volumes you will need to switch to using the <code>flyctl</code> command directly. The <code>--generate-dir</code> option, described below, can help with this.</p>
  <h2 dir="auto"><a id="user-content-generating-without-deploying" class="anchor" aria-hidden="true" href="#user-content-generating-without-deploying"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Generating without deploying</h2>
  <p dir="auto">Use the <code>--generate-dir</code> option to generate a directory that can be deployed to Fly rather than deploying directly:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="datasette publish fly my-database.db \
    --app=&quot;my-generated-app&quot; \
    --generate-dir /tmp/deploy-this"><pre class="notranslate"><code>datasette publish fly my-database.db \
    --app="my-generated-app" \
    --generate-dir /tmp/deploy-this
  </code></pre></div>
  <p dir="auto">You can then manually deploy your generated application using the following:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="cd /tmp/deploy-this
  flyctl apps create my-generated-app
  flyctl deploy"><pre class="notranslate"><code>cd /tmp/deploy-this
  flyctl apps create my-generated-app
  flyctl deploy
  </code></pre></div>
  <h2 dir="auto"><a id="user-content-datasette-publish-fly---help" class="anchor" aria-hidden="true" href="#user-content-datasette-publish-fly---help"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette publish fly --help</h2>

  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Usage: datasette publish fly [OPTIONS] [FILES]...

    Deploy an application to Fly that runs Datasette against the provided database
    files.

    Usage example:

        datasette publish fly my-database.db --app=&quot;my-data-app&quot;

    Full documentation: https://datasette.io/plugins/datasette-publish-fly

  Options:
    -m, --metadata FILENAME         Path to JSON/YAML file containing metadata to
                                    publish
    --extra-options TEXT            Extra options to pass to datasette serve
    --branch TEXT                   Install datasette from a GitHub branch e.g.
                                    main
    --template-dir DIRECTORY        Path to directory containing custom templates
    --plugins-dir DIRECTORY         Path to directory containing custom plugins
    --static MOUNT:DIRECTORY        Serve static files from this directory at
                                    /MOUNT/...
    --install TEXT                  Additional packages (e.g. plugins) to install
    --plugin-secret &lt;TEXT TEXT TEXT&gt;...
                                    Secrets to pass to plugins, e.g. --plugin-
                                    secret datasette-auth-github client_id xxx
    --version-note TEXT             Additional note to show on /-/versions
    --secret TEXT                   Secret used for signing secure values, such as
                                    signed cookies
    --title TEXT                    Title for metadata
    --license TEXT                  License label for metadata
    --license_url TEXT              License URL for metadata
    --source TEXT                   Source label for metadata
    --source_url TEXT               Source URL for metadata
    --about TEXT                    About label for metadata
    --about_url TEXT                About URL for metadata
    --spatialite                    Enable SpatialLite extension
    --region TEXT                   Fly region to deploy to, e.g sjc - see
                                    https://fly.io/docs/reference/regions/
    --create-volume INTEGER RANGE   Create and attach volume of this size in GB
                                    [x&gt;=1]
    --create-db TEXT                Names of read-write database files to create
    --volume-name TEXT              Volume name to use
    -a, --app TEXT                  Name of Fly app to deploy  [required]
    -o, --org TEXT                  Name of Fly org to deploy to
    --generate-dir DIRECTORY        Output generated application files and stop
                                    without deploying
    --show-files                    Output the generated Dockerfile, metadata.json
                                    and fly.toml
    --help                          Show this message and exit."><pre class="notranslate"><code>Usage: datasette publish fly [OPTIONS] [FILES]...

    Deploy an application to Fly that runs Datasette against the provided database
    files.

    Usage example:

        datasette publish fly my-database.db --app="my-data-app"

    Full documentation: https://datasette.io/plugins/datasette-publish-fly

  Options:
    -m, --metadata FILENAME         Path to JSON/YAML file containing metadata to
                                    publish
    --extra-options TEXT            Extra options to pass to datasette serve
    --branch TEXT                   Install datasette from a GitHub branch e.g.
                                    main
    --template-dir DIRECTORY        Path to directory containing custom templates
    --plugins-dir DIRECTORY         Path to directory containing custom plugins
    --static MOUNT:DIRECTORY        Serve static files from this directory at
                                    /MOUNT/...
    --install TEXT                  Additional packages (e.g. plugins) to install
    --plugin-secret &lt;TEXT TEXT TEXT&gt;...
                                    Secrets to pass to plugins, e.g. --plugin-
                                    secret datasette-auth-github client_id xxx
    --version-note TEXT             Additional note to show on /-/versions
    --secret TEXT                   Secret used for signing secure values, such as
                                    signed cookies
    --title TEXT                    Title for metadata
    --license TEXT                  License label for metadata
    --license_url TEXT              License URL for metadata
    --source TEXT                   Source label for metadata
    --source_url TEXT               Source URL for metadata
    --about TEXT                    About label for metadata
    --about_url TEXT                About URL for metadata
    --spatialite                    Enable SpatialLite extension
    --region TEXT                   Fly region to deploy to, e.g sjc - see
                                    https://fly.io/docs/reference/regions/
    --create-volume INTEGER RANGE   Create and attach volume of this size in GB
                                    [x&gt;=1]
    --create-db TEXT                Names of read-write database files to create
    --volume-name TEXT              Volume name to use
    -a, --app TEXT                  Name of Fly app to deploy  [required]
    -o, --org TEXT                  Name of Fly org to deploy to
    --generate-dir DIRECTORY        Output generated application files and stop
                                    without deploying
    --show-files                    Output the generated Dockerfile, metadata.json
                                    and fly.toml
    --help                          Show this message and exit.
  </code></pre></div>

  <h2 dir="auto"><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p dir="auto">To contribute to this tool, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="cd datasette-publish-fly
  python -m venv venv
  source venv/bin/activate"><pre class="notranslate"><code>cd datasette-publish-fly
  python -m venv venv
  source venv/bin/activate
  </code></pre></div>
  <p dir="auto">Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="pipenv shell"><pre class="notranslate"><code>pipenv shell
  </code></pre></div>
  <p dir="auto">Now install the dependencies and test dependencies:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="pip install -e '.[test]'"><pre class="notranslate"><code>pip install -e '.[test]'
  </code></pre></div>
  <p dir="auto">To run the tests:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="pytest"><pre class="notranslate"><code>pytest
  </code></pre></div>
  <h3 dir="auto"><a id="user-content-integration-tests" class="anchor" aria-hidden="true" href="#user-content-integration-tests"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Integration tests</h3>
  <p dir="auto">The tests in <code>tests/test_integration.py</code> make actual calls to Fly to deploy a test application.</p>
  <p dir="auto">These tests are skipped by default. If you have <code>flyctl</code> installed and configured, you can run the integration tests like this:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="pytest --integration -s"><pre class="notranslate"><code>pytest --integration -s
  </code></pre></div>
  <p dir="auto">The <code>-s</code> option here ensures that output from the deploys will be visible to you - otherwise it can look like the tests have hung.</p>
  <p dir="auto">The tests will create applications on Fly that start with the prefix <code>publish-fly-temp-</code> and then delete them at the end of the run.</p>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyNDg5MDM1NDQ=
name: hacker-news-to-sqlite
full_name: dogsheep/hacker-news-to-sqlite
html_url: https://github.com/dogsheep/hacker-news-to-sqlite
description: Create a SQLite database containing data pulled from Hacker News
created_at: 2020-03-21T04:02:05Z
updated_at: 2021-06-06T22:42:00Z
pushed_at: 2021-03-13T19:15:06Z
language: Python
license: apache-2.0
topics: ["hacker-news", "datasette", "dogsheep", "datasette-io", "datasette-tool"]
default_branch: main
permissions: {"admin": false, "push": false, "pull": false}
readme:
  # hacker-news-to-sqlite

  [![PyPI](https://img.shields.io/pypi/v/hacker-news-to-sqlite.svg)](https://pypi.org/project/hacker-news-to-sqlite/)
  [![Changelog](https://img.shields.io/github/v/release/dogsheep/hacker-news-to-sqlite?include_prereleases&label=changelog)](https://github.com/dogsheep/hacker-news-to-sqlite/releases)
  [![Tests](https://github.com/dogsheep/hacker-news-to-sqlite/workflows/Test/badge.svg)](https://github.com/dogsheep/hacker-news-to-sqlite/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/hacker-news-to-sqlite/blob/main/LICENSE)

  Create a SQLite database containing data fetched from [Hacker News](https://news.ycombinator.com/).

  ## How to install

      $ pip install hacker-news-to-sqlite

  ## Usage

      $ hacker-news-to-sqlite user hacker-news.db your-username
      Importing items:  37%|███████████                        | 845/2297 [05:09<11:02,  2.19it/s]

  Imports all of your Hacker News submissions and comments into a SQLite database called `hacker-news.db`.

      $ hacker-news-to-sqlite trees hacker-news.db 22640038 22643218

  Fetches the entire comments tree in which any of those content IDs appears.

  ## Browsing your data with Datasette

  You can use [Datasette](https://datasette.readthedocs.org/) to browse your data. Install Datasette like this:

      $ pip install datasette

  Now run it against your `hacker-news.db` file like so:

      $ datasette hacker-news.db

  Visit `https://localhost:8001/` to search and explore your data.

  You can improve the display of your data usinng the [datasette-render-timestamps](https://github.com/simonw/datasette-render-timestamps) and [datasette-render-html](https://github.com/simonw/datasette-render-html) plugins. Install them like this:

      $ pip install datasette-render-timestamps datasette-render-html

  Now save the following configuration in a file called `metadata.json`:

  ```json
  {
      "databases": {
          "hacker-news": {
              "tables": {
                  "items": {
                      "plugins": {
                          "datasette-render-html": {
                              "columns": [
                                  "text"
                              ]
                          },
                          "datasette-render-timestamps": {
                              "columns": [
                                  "time"
                              ]
                          }
                      }
                  },
                  "users": {
                      "plugins": {
                          "datasette-render-timestamps": {
                              "columns": [
                                  "created"
                              ]
                          }
                      }
                  }
              }
          }
      }
  }
  ```
  Run Datasette like this:

      $ datasette -m metadata.json hacker-news.db

  The timestamp columns will now be rendered as human-readable dates, and any HTML in your posts will be displayed as rendered HTML.

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-hacker-news-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-hacker-news-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>hacker-news-to-sqlite</h1>
  <p><a href="https://pypi.org/project/hacker-news-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/bf6d88d26ea4d8f1f396c1b1fc88ae74380de6a30f5a792c9f29664f21a219dd/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6861636b65722d6e6577732d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/hacker-news-to-sqlite.svg" style="max-width:100%;"></a>
  <a href="https://github.com/dogsheep/hacker-news-to-sqlite/releases"><img src="https://camo.githubusercontent.com/090ef507ee9402a786e40313d173f516f1dbdedf3df703402296594a4ebcad73/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f646f6773686565702f6861636b65722d6e6577732d746f2d73716c6974653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/dogsheep/hacker-news-to-sqlite?include_prereleases&amp;label=changelog" style="max-width:100%;"></a>
  <a href="https://github.com/dogsheep/hacker-news-to-sqlite/actions?query=workflow%3ATest"><img src="https://github.com/dogsheep/hacker-news-to-sqlite/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/hacker-news-to-sqlite/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p>
  <p>Create a SQLite database containing data fetched from <a href="https://news.ycombinator.com/" rel="nofollow">Hacker News</a>.</p>
  <h2><a id="user-content-how-to-install" class="anchor" aria-hidden="true" href="#user-content-how-to-install"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>How to install</h2>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ pip install hacker-news-to-sqlite
  "><pre><code>$ pip install hacker-news-to-sqlite
  </code></pre></div>
  <h2><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ hacker-news-to-sqlite user hacker-news.db your-username
  Importing items:  37%|███████████                        | 845/2297 [05:09&lt;11:02,  2.19it/s]
  "><pre><code>$ hacker-news-to-sqlite user hacker-news.db your-username
  Importing items:  37%|███████████                        | 845/2297 [05:09&lt;11:02,  2.19it/s]
  </code></pre></div>
  <p>Imports all of your Hacker News submissions and comments into a SQLite database called <code>hacker-news.db</code>.</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ hacker-news-to-sqlite trees hacker-news.db 22640038 22643218
  "><pre><code>$ hacker-news-to-sqlite trees hacker-news.db 22640038 22643218
  </code></pre></div>
  <p>Fetches the entire comments tree in which any of those content IDs appears.</p>
  <h2><a id="user-content-browsing-your-data-with-datasette" class="anchor" aria-hidden="true" href="#user-content-browsing-your-data-with-datasette"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Browsing your data with Datasette</h2>
  <p>You can use <a href="https://datasette.readthedocs.org/" rel="nofollow">Datasette</a> to browse your data. Install Datasette like this:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ pip install datasette
  "><pre><code>$ pip install datasette
  </code></pre></div>
  <p>Now run it against your <code>hacker-news.db</code> file like so:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ datasette hacker-news.db
  "><pre><code>$ datasette hacker-news.db
  </code></pre></div>
  <p>Visit <code>https://localhost:8001/</code> to search and explore your data.</p>
  <p>You can improve the display of your data usinng the <a href="https://github.com/simonw/datasette-render-timestamps">datasette-render-timestamps</a> and <a href="https://github.com/simonw/datasette-render-html">datasette-render-html</a> plugins. Install them like this:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ pip install datasette-render-timestamps datasette-render-html
  "><pre><code>$ pip install datasette-render-timestamps datasette-render-html
  </code></pre></div>
  <p>Now save the following configuration in a file called <code>metadata.json</code>:</p>
  <div class="highlight highlight-source-json position-relative" data-snippet-clipboard-copy-content="{
      &quot;databases&quot;: {
          &quot;hacker-news&quot;: {
              &quot;tables&quot;: {
                  &quot;items&quot;: {
                      &quot;plugins&quot;: {
                          &quot;datasette-render-html&quot;: {
                              &quot;columns&quot;: [
                                  &quot;text&quot;
                              ]
                          },
                          &quot;datasette-render-timestamps&quot;: {
                              &quot;columns&quot;: [
                                  &quot;time&quot;
                              ]
                          }
                      }
                  },
                  &quot;users&quot;: {
                      &quot;plugins&quot;: {
                          &quot;datasette-render-timestamps&quot;: {
                              &quot;columns&quot;: [
                                  &quot;created&quot;
                              ]
                          }
                      }
                  }
              }
          }
      }
  }
  "><pre>{
      <span class="pl-s"><span class="pl-pds">"</span>databases<span class="pl-pds">"</span></span>: {
          <span class="pl-s"><span class="pl-pds">"</span>hacker-news<span class="pl-pds">"</span></span>: {
              <span class="pl-s"><span class="pl-pds">"</span>tables<span class="pl-pds">"</span></span>: {
                  <span class="pl-s"><span class="pl-pds">"</span>items<span class="pl-pds">"</span></span>: {
                      <span class="pl-s"><span class="pl-pds">"</span>plugins<span class="pl-pds">"</span></span>: {
                          <span class="pl-s"><span class="pl-pds">"</span>datasette-render-html<span class="pl-pds">"</span></span>: {
                              <span class="pl-s"><span class="pl-pds">"</span>columns<span class="pl-pds">"</span></span>: [
                                  <span class="pl-s"><span class="pl-pds">"</span>text<span class="pl-pds">"</span></span>
                              ]
                          },
                          <span class="pl-s"><span class="pl-pds">"</span>datasette-render-timestamps<span class="pl-pds">"</span></span>: {
                              <span class="pl-s"><span class="pl-pds">"</span>columns<span class="pl-pds">"</span></span>: [
                                  <span class="pl-s"><span class="pl-pds">"</span>time<span class="pl-pds">"</span></span>
                              ]
                          }
                      }
                  },
                  <span class="pl-s"><span class="pl-pds">"</span>users<span class="pl-pds">"</span></span>: {
                      <span class="pl-s"><span class="pl-pds">"</span>plugins<span class="pl-pds">"</span></span>: {
                          <span class="pl-s"><span class="pl-pds">"</span>datasette-render-timestamps<span class="pl-pds">"</span></span>: {
                              <span class="pl-s"><span class="pl-pds">"</span>columns<span class="pl-pds">"</span></span>: [
                                  <span class="pl-s"><span class="pl-pds">"</span>created<span class="pl-pds">"</span></span>
                              ]
                          }
                      }
                  }
              }
          }
      }
  }</pre></div>
  <p>Run Datasette like this:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ datasette -m metadata.json hacker-news.db
  "><pre><code>$ datasette -m metadata.json hacker-news.db
  </code></pre></div>
  <p>The timestamp columns will now be rendered as human-readable dates, and any HTML in your posts will be displayed as rendered HTML.</p>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyNDg5OTk5OTQ=
name: datasette-show-errors
full_name: simonw/datasette-show-errors
html_url: https://github.com/simonw/datasette-show-errors
description: Datasette plugin for displaying error tracebacks
created_at: 2020-03-21T15:06:04Z
updated_at: 2020-09-24T00:17:29Z
pushed_at: 2020-09-01T00:32:23Z
language: Python
license: apache-2.0
topics: ["asgi", "datasette", "starlette", "datasette-plugin", "datasette-io"]
default_branch: master
permissions: {"admin": false, "push": false, "pull": false}
readme:
  # datasette-show-errors

  [![PyPI](https://img.shields.io/pypi/v/datasette-show-errors.svg)](https://pypi.org/project/datasette-show-errors/)
  [![CircleCI](https://circleci.com/gh/simonw/datasette-show-errors.svg?style=svg)](https://circleci.com/gh/simonw/datasette-show-errors)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-show-errors/blob/master/LICENSE)

  Datasette plugin for displaying error tracebacks.

  **This plugin does not work with current versions of Datasette.** See [issue #2](https://github.com/simonw/datasette-show-errors/issues/2).

  ## Installation

      pip install datasette-show-errors

  ## Usage

  Installing the plugin will cause any internal error to be displayed with a full traceback, rather than just a generic 500 page.

  Be careful not to use this in a context that might expose sensitive information.

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-datasette-show-errors" class="anchor" aria-hidden="true" href="#user-content-datasette-show-errors"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-show-errors</h1>
  <p><a href="https://pypi.org/project/datasette-show-errors/" rel="nofollow"><img src="https://camo.githubusercontent.com/b3e198ab4e1253d5eca382cb810057bf1be9b3a78db6f60f57c2a5a269d49304/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d73686f772d6572726f72732e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-show-errors.svg" style="max-width:100%;"></a>
  <a href="https://circleci.com/gh/simonw/datasette-show-errors" rel="nofollow"><img src="https://camo.githubusercontent.com/183abdd282beee4022d9443bd5c050fa0a2d8fd024156414206fa188872a7a8e/68747470733a2f2f636972636c6563692e636f6d2f67682f73696d6f6e772f6461746173657474652d73686f772d6572726f72732e7376673f7374796c653d737667" alt="CircleCI" data-canonical-src="https://circleci.com/gh/simonw/datasette-show-errors.svg?style=svg" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-show-errors/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p>
  <p>Datasette plugin for displaying error tracebacks.</p>
  <p><strong>This plugin does not work with current versions of Datasette.</strong> See <a href="https://github.com/simonw/datasette-show-errors/issues/2">issue #2</a>.</p>
  <h2><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pip install datasette-show-errors
  "><pre><code>pip install datasette-show-errors
  </code></pre></div>
  <h2><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p>Installing the plugin will cause any internal error to be displayed with a full traceback, rather than just a generic 500 page.</p>
  <p>Be careful not to use this in a context that might expose sensitive information.</p>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyNTM2MzI5NDg=
name: datasette-publish-vercel
full_name: simonw/datasette-publish-vercel
html_url: https://github.com/simonw/datasette-publish-vercel
description: Datasette plugin for publishing data using Vercel
created_at: 2020-04-06T22:47:13Z
updated_at: 2022-07-29T17:09:47Z
pushed_at: 2022-08-24T17:43:41Z
language: Python
license: apache-2.0
topics: ["datasette", "datasette-io", "datasette-plugin", "vercel", "zeit-now"]
default_branch: main
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
visibility: public
readme:
  # datasette-publish-vercel

  [![PyPI](https://img.shields.io/pypi/v/datasette-publish-vercel.svg)](https://pypi.org/project/datasette-publish-vercel/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-publish-vercel?include_prereleases&label=changelog)](https://github.com/simonw/datasette-publish-vercel/releases)
  [![Tests](https://github.com/simonw/datasette-publish-vercel/workflows/Test/badge.svg)](https://github.com/simonw/datasette-publish-vercel/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-publish-vercel/blob/main/LICENSE)

  Datasette plugin for publishing data using [Vercel](https://vercel.com/).

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-publish-vercel

  ## Usage

  First, install the Vercel CLI tool by [following their instructions](https://vercel.com/download).

  Run `vercel login` to login to (or create) an account.

  Now you can use `datasette publish vercel` to publish your data:

      datasette publish vercel my-database.db --project=my-database

  The `--project` argument is required - it specifies the project name that should be used for your deployment. This will be used as part of the deployment's URL.

  ### Other options

  * `--no-prod` deploys to the project without updating the "production" URL alias to point to that new deployment. Without that option all deploys go directly to production.
  * `--debug` enables the Vercel CLI debug output.
  * `--token` allows you to pass a Now authentication token, rather than needing to first run `now login` to configure the tool. Tokens can be created in the Vercel web dashboard under Account Settings -> Tokens.
  * `--public` runs `vercel --public` to publish the application source code at `/_src` e.g. https://datasette-public.now.sh/_src and make recent logs visible at `/_logs` e.g. https://datasette-public.now.sh/_logs
  * `--generate-dir` - by default this tool generates a new Vercel app in a temporary directory, deploys it and then deletes the directory. Use `--generate-dir=my-app` to output the generated application files to a new directory of your choice instead. You can then deploy it by running `vercel` in that directory.
  * `--setting default_page_size 10` - use this to set Datasette settings, as described in [the documentation](https://docs.datasette.io/en/stable/settings.html). This is a replacement for the unsupported `--extra-options` option.

  ### Full help

  **Warning:** Some of these options are not yet implemented by this plugin. In particular, the following do not yet work:

  * `--extra-options` - use `--setting` described above instead.
  * `--plugin-secret`
  * `--version-note`

  ```
  $ datasette publish vercel --help

  Usage: datasette publish vercel [OPTIONS] [FILES]...

    Publish to https://vercel.com/

  Options:
    -m, --metadata FILENAME         Path to JSON/YAML file containing metadata to publish
    --extra-options TEXT            Extra options to pass to datasette serve
    --branch TEXT                   Install datasette from a GitHub branch e.g. main
    --template-dir DIRECTORY        Path to directory containing custom templates
    --plugins-dir DIRECTORY         Path to directory containing custom plugins
    --static MOUNT:DIRECTORY        Serve static files from this directory at /MOUNT/...
    --install TEXT                  Additional packages (e.g. plugins) to install
    --plugin-secret <TEXT TEXT TEXT>...
                                    Secrets to pass to plugins, e.g. --plugin-secret
                                    datasette-auth-github client_id xxx
    --version-note TEXT             Additional note to show on /-/versions
    --secret TEXT                   Secret used for signing secure values, such as signed
                                    cookies
    --title TEXT                    Title for metadata
    --license TEXT                  License label for metadata
    --license_url TEXT              License URL for metadata
    --source TEXT                   Source label for metadata
    --source_url TEXT               Source URL for metadata
    --about TEXT                    About label for metadata
    --about_url TEXT                About URL for metadata
    --token TEXT                    Auth token to use for deploy
    --project PROJECT               Vercel project name to use  [required]
    --scope TEXT                    Optional Vercel scope (e.g. a team name)
    --no-prod                       Don't deploy directly to production
    --debug                         Enable Vercel CLI debug output
    --public                        Publish source with Vercel CLI --public
    --generate-dir DIRECTORY        Output generated application files and stop without
                                    deploying
    --generate-vercel-json          Output generated vercel.json file and stop without
                                    deploying
    --vercel-json FILENAME          Custom vercel.json file to use instead of generating
                                    one
    --setting SETTING...            Setting, see docs.datasette.io/en/stable/settings.html
    --crossdb                       Enable cross-database SQL queries
    --help                          Show this message and exit.
  ```
  ## Using a custom `vercel.json` file

  If you want to add additional redirects or similar to your Vercel configuration you may want to provide a custom `vercel.json` file.

  To do this, first generate a configuration file (without running a deploy) using the `--generate-vercel-json` option:

      datasette publish vercel my-database.db \
        --project=my-database \
        --generate-vercel-json > vercel.json

  You can now edit the `vercel.json` file that this creates to add your custom options.

  Then run the deploy using:

      datasette publish vercel my-database.db \
        --project=my-database \
        --vercel-json=vercel.json

  ## Setting a `DATASETTE_SECRET`

  Datasette uses [a secret string](https://docs.datasette.io/en/stable/settings.html#configuring-the-secret) for purposes such as signing authentication cookies. This secret is reset when the server restarts, which will sign out any users who are authenticated using a signed cookie.

  You can avoid this by generating a `DATASETTE_SECRET` secret string and setting that as a [Vercel environment variable](https://vercel.com/docs/concepts/projects/environment-variables). If you do this the secret will stay consistent and your users will not be signed out.

  ## Using this with GitHub Actions

  This plugin can be used together with [GitHub Actions](https://github.com/features/actions) to deploy Datasette instances automatically on new pushes to a repo, or on a schedule.

  The GitHub Actions runners already have the Vercel deployment tool installed. You'll need to create an API token for your account at [vercel.com/account/tokens](https://vercel.com/account/tokens), and store that as a secret in your GitHub repository called `VERCEL_TOKEN`.

  Make sure your workflow has installed `datasette` and `datasette-publish-vercel` using `pip`, then add the following step to your GitHub Actions workflow:
  ```
      - name: Deploy Datasette using Vercel
        env:
          VERCEL_TOKEN: ${{ secrets.VERCEL_TOKEN }}
        run: |-
          datasette publish vercel mydb.db \
            --token $VERCEL_TOKEN \
            --project my-vercel-project
  ```
  You can see a full example of a workflow that uses Vercel in this way [in the simonw/til repository](https://github.com/simonw/til/blob/12b3f0d3679320cbeafa5df164bbc08ba703625d/.github/workflows/build.yml).

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-datasette-publish-vercel" class="anchor" aria-hidden="true" href="#user-content-datasette-publish-vercel"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-publish-vercel</h1>
  <p dir="auto"><a href="https://pypi.org/project/datasette-publish-vercel/" rel="nofollow"><img src="https://camo.githubusercontent.com/54812f865fef02d66cf1d1155ea07750ad19a0a88dad2d4f1a86f4fbd3fba8a0/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d7075626c6973682d76657263656c2e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-publish-vercel.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-publish-vercel/releases"><img src="https://camo.githubusercontent.com/5ba538bc3912064a11f09b105ecd92083d1223f2f8a38034d17734d09a26f321/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d7075626c6973682d76657263656c3f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-publish-vercel?include_prereleases&amp;label=changelog" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-publish-vercel/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-publish-vercel/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-publish-vercel/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width: 100%;"></a></p>
  <p dir="auto">Datasette plugin for publishing data using <a href="https://vercel.com/" rel="nofollow">Vercel</a>.</p>
  <h2 dir="auto"><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p dir="auto">Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="$ datasette install datasette-publish-vercel"><pre class="notranslate"><code>$ datasette install datasette-publish-vercel
  </code></pre></div>
  <h2 dir="auto"><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p dir="auto">First, install the Vercel CLI tool by <a href="https://vercel.com/download" rel="nofollow">following their instructions</a>.</p>
  <p dir="auto">Run <code>vercel login</code> to login to (or create) an account.</p>
  <p dir="auto">Now you can use <code>datasette publish vercel</code> to publish your data:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="datasette publish vercel my-database.db --project=my-database"><pre class="notranslate"><code>datasette publish vercel my-database.db --project=my-database
  </code></pre></div>
  <p dir="auto">The <code>--project</code> argument is required - it specifies the project name that should be used for your deployment. This will be used as part of the deployment's URL.</p>
  <h3 dir="auto"><a id="user-content-other-options" class="anchor" aria-hidden="true" href="#user-content-other-options"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Other options</h3>
  <ul dir="auto">
  <li><code>--no-prod</code> deploys to the project without updating the "production" URL alias to point to that new deployment. Without that option all deploys go directly to production.</li>
  <li><code>--debug</code> enables the Vercel CLI debug output.</li>
  <li><code>--token</code> allows you to pass a Now authentication token, rather than needing to first run <code>now login</code> to configure the tool. Tokens can be created in the Vercel web dashboard under Account Settings -&gt; Tokens.</li>
  <li><code>--public</code> runs <code>vercel --public</code> to publish the application source code at <code>/_src</code> e.g. <a href="https://datasette-public.now.sh/_src" rel="nofollow">https://datasette-public.now.sh/_src</a> and make recent logs visible at <code>/_logs</code> e.g. <a href="https://datasette-public.now.sh/_logs" rel="nofollow">https://datasette-public.now.sh/_logs</a></li>
  <li><code>--generate-dir</code> - by default this tool generates a new Vercel app in a temporary directory, deploys it and then deletes the directory. Use <code>--generate-dir=my-app</code> to output the generated application files to a new directory of your choice instead. You can then deploy it by running <code>vercel</code> in that directory.</li>
  <li><code>--setting default_page_size 10</code> - use this to set Datasette settings, as described in <a href="https://docs.datasette.io/en/stable/settings.html" rel="nofollow">the documentation</a>. This is a replacement for the unsupported <code>--extra-options</code> option.</li>
  </ul>
  <h3 dir="auto"><a id="user-content-full-help" class="anchor" aria-hidden="true" href="#user-content-full-help"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Full help</h3>
  <p dir="auto"><strong>Warning:</strong> Some of these options are not yet implemented by this plugin. In particular, the following do not yet work:</p>
  <ul dir="auto">
  <li><code>--extra-options</code> - use <code>--setting</code> described above instead.</li>
  <li><code>--plugin-secret</code></li>
  <li><code>--version-note</code></li>
  </ul>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="$ datasette publish vercel --help

  Usage: datasette publish vercel [OPTIONS] [FILES]...

    Publish to https://vercel.com/

  Options:
    -m, --metadata FILENAME         Path to JSON/YAML file containing metadata to publish
    --extra-options TEXT            Extra options to pass to datasette serve
    --branch TEXT                   Install datasette from a GitHub branch e.g. main
    --template-dir DIRECTORY        Path to directory containing custom templates
    --plugins-dir DIRECTORY         Path to directory containing custom plugins
    --static MOUNT:DIRECTORY        Serve static files from this directory at /MOUNT/...
    --install TEXT                  Additional packages (e.g. plugins) to install
    --plugin-secret &lt;TEXT TEXT TEXT&gt;...
                                    Secrets to pass to plugins, e.g. --plugin-secret
                                    datasette-auth-github client_id xxx
    --version-note TEXT             Additional note to show on /-/versions
    --secret TEXT                   Secret used for signing secure values, such as signed
                                    cookies
    --title TEXT                    Title for metadata
    --license TEXT                  License label for metadata
    --license_url TEXT              License URL for metadata
    --source TEXT                   Source label for metadata
    --source_url TEXT               Source URL for metadata
    --about TEXT                    About label for metadata
    --about_url TEXT                About URL for metadata
    --token TEXT                    Auth token to use for deploy
    --project PROJECT               Vercel project name to use  [required]
    --scope TEXT                    Optional Vercel scope (e.g. a team name)
    --no-prod                       Don't deploy directly to production
    --debug                         Enable Vercel CLI debug output
    --public                        Publish source with Vercel CLI --public
    --generate-dir DIRECTORY        Output generated application files and stop without
                                    deploying
    --generate-vercel-json          Output generated vercel.json file and stop without
                                    deploying
    --vercel-json FILENAME          Custom vercel.json file to use instead of generating
                                    one
    --setting SETTING...            Setting, see docs.datasette.io/en/stable/settings.html
    --crossdb                       Enable cross-database SQL queries
    --help                          Show this message and exit."><pre class="notranslate"><code>$ datasette publish vercel --help

  Usage: datasette publish vercel [OPTIONS] [FILES]...

    Publish to https://vercel.com/

  Options:
    -m, --metadata FILENAME         Path to JSON/YAML file containing metadata to publish
    --extra-options TEXT            Extra options to pass to datasette serve
    --branch TEXT                   Install datasette from a GitHub branch e.g. main
    --template-dir DIRECTORY        Path to directory containing custom templates
    --plugins-dir DIRECTORY         Path to directory containing custom plugins
    --static MOUNT:DIRECTORY        Serve static files from this directory at /MOUNT/...
    --install TEXT                  Additional packages (e.g. plugins) to install
    --plugin-secret &lt;TEXT TEXT TEXT&gt;...
                                    Secrets to pass to plugins, e.g. --plugin-secret
                                    datasette-auth-github client_id xxx
    --version-note TEXT             Additional note to show on /-/versions
    --secret TEXT                   Secret used for signing secure values, such as signed
                                    cookies
    --title TEXT                    Title for metadata
    --license TEXT                  License label for metadata
    --license_url TEXT              License URL for metadata
    --source TEXT                   Source label for metadata
    --source_url TEXT               Source URL for metadata
    --about TEXT                    About label for metadata
    --about_url TEXT                About URL for metadata
    --token TEXT                    Auth token to use for deploy
    --project PROJECT               Vercel project name to use  [required]
    --scope TEXT                    Optional Vercel scope (e.g. a team name)
    --no-prod                       Don't deploy directly to production
    --debug                         Enable Vercel CLI debug output
    --public                        Publish source with Vercel CLI --public
    --generate-dir DIRECTORY        Output generated application files and stop without
                                    deploying
    --generate-vercel-json          Output generated vercel.json file and stop without
                                    deploying
    --vercel-json FILENAME          Custom vercel.json file to use instead of generating
                                    one
    --setting SETTING...            Setting, see docs.datasette.io/en/stable/settings.html
    --crossdb                       Enable cross-database SQL queries
    --help                          Show this message and exit.
  </code></pre></div>
  <h2 dir="auto"><a id="user-content-using-a-custom-verceljson-file" class="anchor" aria-hidden="true" href="#user-content-using-a-custom-verceljson-file"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Using a custom <code>vercel.json</code> file</h2>
  <p dir="auto">If you want to add additional redirects or similar to your Vercel configuration you may want to provide a custom <code>vercel.json</code> file.</p>
  <p dir="auto">To do this, first generate a configuration file (without running a deploy) using the <code>--generate-vercel-json</code> option:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="datasette publish vercel my-database.db \
    --project=my-database \
    --generate-vercel-json &gt; vercel.json"><pre class="notranslate"><code>datasette publish vercel my-database.db \
    --project=my-database \
    --generate-vercel-json &gt; vercel.json
  </code></pre></div>
  <p dir="auto">You can now edit the <code>vercel.json</code> file that this creates to add your custom options.</p>
  <p dir="auto">Then run the deploy using:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="datasette publish vercel my-database.db \
    --project=my-database \
    --vercel-json=vercel.json"><pre class="notranslate"><code>datasette publish vercel my-database.db \
    --project=my-database \
    --vercel-json=vercel.json
  </code></pre></div>
  <h2 dir="auto"><a id="user-content-setting-a-datasette_secret" class="anchor" aria-hidden="true" href="#user-content-setting-a-datasette_secret"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Setting a <code>DATASETTE_SECRET</code></h2>
  <p dir="auto">Datasette uses <a href="https://docs.datasette.io/en/stable/settings.html#configuring-the-secret" rel="nofollow">a secret string</a> for purposes such as signing authentication cookies. This secret is reset when the server restarts, which will sign out any users who are authenticated using a signed cookie.</p>
  <p dir="auto">You can avoid this by generating a <code>DATASETTE_SECRET</code> secret string and setting that as a <a href="https://vercel.com/docs/concepts/projects/environment-variables" rel="nofollow">Vercel environment variable</a>. If you do this the secret will stay consistent and your users will not be signed out.</p>
  <h2 dir="auto"><a id="user-content-using-this-with-github-actions" class="anchor" aria-hidden="true" href="#user-content-using-this-with-github-actions"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Using this with GitHub Actions</h2>
  <p dir="auto">This plugin can be used together with <a href="https://github.com/features/actions">GitHub Actions</a> to deploy Datasette instances automatically on new pushes to a repo, or on a schedule.</p>
  <p dir="auto">The GitHub Actions runners already have the Vercel deployment tool installed. You'll need to create an API token for your account at <a href="https://vercel.com/account/tokens" rel="nofollow">vercel.com/account/tokens</a>, and store that as a secret in your GitHub repository called <code>VERCEL_TOKEN</code>.</p>
  <p dir="auto">Make sure your workflow has installed <code>datasette</code> and <code>datasette-publish-vercel</code> using <code>pip</code>, then add the following step to your GitHub Actions workflow:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="    - name: Deploy Datasette using Vercel
        env:
          VERCEL_TOKEN: ${{ secrets.VERCEL_TOKEN }}
        run: |-
          datasette publish vercel mydb.db \
            --token $VERCEL_TOKEN \
            --project my-vercel-project"><pre class="notranslate"><code>    - name: Deploy Datasette using Vercel
        env:
          VERCEL_TOKEN: ${{ secrets.VERCEL_TOKEN }}
        run: |-
          datasette publish vercel mydb.db \
            --token $VERCEL_TOKEN \
            --project my-vercel-project
  </code></pre></div>
  <p dir="auto">You can see a full example of a workflow that uses Vercel in this way <a href="https://github.com/simonw/til/blob/12b3f0d3679320cbeafa5df164bbc08ba703625d/.github/workflows/build.yml">in the simonw/til repository</a>.</p>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyNTU0NjAzNDc=
name: datasette-clone
full_name: simonw/datasette-clone
html_url: https://github.com/simonw/datasette-clone
description: Create a local copy of database files from a Datasette instance
created_at: 2020-04-13T23:05:41Z
updated_at: 2021-06-08T15:33:21Z
pushed_at: 2021-02-22T19:32:36Z
language: Python
license: apache-2.0
topics: ["datasette", "datasette-io", "datasette-tool"]
default_branch: main
permissions: {"admin": false, "push": false, "pull": false}
readme:
  # datasette-clone

  [![PyPI](https://img.shields.io/pypi/v/datasette-clone.svg)](https://pypi.org/project/datasette-clone/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-clone?include_prereleases&label=changelog)](https://github.com/simonw/datasette-clone/releases)
  [![Tests](https://github.com/simonw/datasette-clone/workflows/Test/badge.svg)](https://github.com/simonw/datasette-clone/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-clone/blob/main/LICENSE)

  Create a local copy of database files from a Datasette instance.

  See [datasette-clone](https://simonwillison.net/2020/Apr/14/datasette-clone/) on my blog for background on this project.

  ## How to install

      $ pip install datasette-clone

  ## Usage

  This only works against Datasette instances running immutable databases (with the `-i` option). Databases published using the `datasette publish` command should be compatible with this tool.

  To download copies of all `.db` files from an instance, run:

      datasette-clone https://latest.datasette.io

  You can provide an optional second argument to specify a directory:

      datasette-clone https://latest.datasette.io /tmp/here-please

  The command stores its own copy of a `databases.json` manifest and uses it to only download databases that have changed the next time you run the command.

  It also stores a copy of the instance's `metadata.json` to ensure you have a copy of any source and licensing information for the downloaded databases.

  If your instance is protected by an API token, you can use `--token` to provide it:

      datasette-clone https://latest.datasette.io --token=xyz

  For verbose output showing what the tool is doing, use `-v`.

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-datasette-clone" class="anchor" aria-hidden="true" href="#user-content-datasette-clone"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-clone</h1>
  <p><a href="https://pypi.org/project/datasette-clone/" rel="nofollow"><img src="https://camo.githubusercontent.com/17cc348c1197fd918ac911e525efd416a5d4ca5d3f00729cbf85c930ac5dbde6/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d636c6f6e652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-clone.svg" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-clone/releases"><img src="https://camo.githubusercontent.com/9f8615567ed0f9b4d38d5f69dafcc8ae2fa97aa13895779172645064f16fcc9f/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d636c6f6e653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-clone?include_prereleases&amp;label=changelog" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-clone/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-clone/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-clone/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p>
  <p>Create a local copy of database files from a Datasette instance.</p>
  <p>See <a href="https://simonwillison.net/2020/Apr/14/datasette-clone/" rel="nofollow">datasette-clone</a> on my blog for background on this project.</p>
  <h2><a id="user-content-how-to-install" class="anchor" aria-hidden="true" href="#user-content-how-to-install"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>How to install</h2>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ pip install datasette-clone
  "><pre><code>$ pip install datasette-clone
  </code></pre></div>
  <h2><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p>This only works against Datasette instances running immutable databases (with the <code>-i</code> option). Databases published using the <code>datasette publish</code> command should be compatible with this tool.</p>
  <p>To download copies of all <code>.db</code> files from an instance, run:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="datasette-clone https://latest.datasette.io
  "><pre><code>datasette-clone https://latest.datasette.io
  </code></pre></div>
  <p>You can provide an optional second argument to specify a directory:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="datasette-clone https://latest.datasette.io /tmp/here-please
  "><pre><code>datasette-clone https://latest.datasette.io /tmp/here-please
  </code></pre></div>
  <p>The command stores its own copy of a <code>databases.json</code> manifest and uses it to only download databases that have changed the next time you run the command.</p>
  <p>It also stores a copy of the instance's <code>metadata.json</code> to ensure you have a copy of any source and licensing information for the downloaded databases.</p>
  <p>If your instance is protected by an API token, you can use <code>--token</code> to provide it:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="datasette-clone https://latest.datasette.io --token=xyz
  "><pre><code>datasette-clone https://latest.datasette.io --token=xyz
  </code></pre></div>
  <p>For verbose output showing what the tool is doing, use <code>-v</code>.</p>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyNTY4MzQ5MDc=
name: dogsheep-photos
full_name: dogsheep/dogsheep-photos
html_url: https://github.com/dogsheep/dogsheep-photos
description: Upload your photos to S3 and import metadata about them into a SQLite database
created_at: 2020-04-18T19:22:13Z
updated_at: 2021-11-04T20:45:03Z
pushed_at: 2021-11-04T20:45:00Z
language: Python
license: apache-2.0
topics: ["datasette", "datasette-io", "datasette-tool", "dogsheep", "sqlite"]
default_branch: master
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
visibility: public
readme:
  # dogsheep-photos

  [![PyPI](https://img.shields.io/pypi/v/dogsheep-photos.svg)](https://pypi.org/project/dogsheep-photos/)
  [![Changelog](https://img.shields.io/github/v/release/dogsheep/dogsheep-photos?include_prereleases&label=changelog)](https://github.com/dogsheep/dogsheep-photos/releases)
  [![CircleCI](https://circleci.com/gh/dogsheep/dogsheep-photos.svg?style=svg)](https://circleci.com/gh/dogsheep/dogsheep-photos)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/dogsheep/dogsheep-photos/blob/master/LICENSE)

  Save details of your photos to a SQLite database and upload them to S3.

  See [Using SQL to find my best photo of a pelican according to Apple Photos](https://simonwillison.net/2020/May/21/apple-photos-sqlite/) for background information on this project.

  ## What these tools do

  These tools are a work-in-progress mechanism for taking full ownership of your photos. The core idea is to help implement the following:

  * Every photo you have taken lives in a single, private Amazon S3 bucket
  * You have a single SQLite database file which stores metadata about those photos - potentially pulled from multiple different places. This may include EXIF data, Apple Photos, the results of running machine learning APIs against photos and much more besides.
  * You can then use [Datasette](https://github.com/simonw/datasette) to explore your own photos.

  I'm a heavy user of Apple Photos so the initial releases of this tool will have a bias towards that, but ideally I would like a subset of these tools to be useful to people no matter which core photo solution they are using.

  ## Installation

      $ pip install dogsheep-photos

  ## Authentication (if using S3)

  If you want to use S3 to store your photos, you will need to first create S3 credentials for a new, dedicated bucket.

  You may find the [s3-credentials tool](https://github.com/simonw/s3-credentials) useful for this.

  Run this command and paste in your credentials. You will need three values: the name of your S3 bucket, your Access key ID and your Secret access key.

      $ dogsheep-photos s3-auth

  This will create a file called `auth.json` in your current directory containing the required values. To save the file at a different path or filename, use the `--auth=myauth.json` option.

  ## Uploading photos

  Run this command to upload every photo in a specific directory to your S3 bucket:

      $ dogsheep-photos upload photos.db \
          ~/Pictures/Photos\ Library.photoslibrary/original

  The command will only upload photos that have not yet been uploaded, based on their sha256 hash.

  `photos.db` will be created with an `uploads` table containing details of which files were uploaded.

  To see what the command would do without uploading any files, use the `--dry-run` option.

  The sha256 hash of the photo contents will be used as the name of the file in the bucket, with an extension matching the type of file. This is an implementation of the [Content addressable storage](https://en.wikipedia.org/wiki/Content-addressable_storage) pattern.

  ## Importing Apple Photos metadata

  The `apple-photos` command imports metadata from your Apple Photos library.

      $ photo-to-sqlite apple-photos photos.db

  Imported metadata includes places, people, albums, quality scores and machine learning labels for the photo contents.

  ## Creating a subset database

  You can create a new, subset database of photos using the `create-subset` command.

  This is useful for creating a shareable SQLite database that only contains metadata for a selected set of photos.

  Since photo metadata contains latitude and longitude you may not want to share a database that includes photos taken at your home address.

  `create-subset` takes three arguments: an existing database file created using the `apple-photos` command, the name of the new, shareable database file you would like to create and a SQL query that returns the `sha256` hash values of the photos you would like to include in that database.

  For example, here's how to create a shareable database of just the photos that have been added to albums containing the word "Public":

      $ dogsheep-photos create-subset \
          photos.db \
          public.db \
          "select sha256 from apple_photos where albums like '%Public%'"

  ## Serving photos locally with datasette-media

  If you don't want to upload your photos to S3 but you still want to browse them using Datasette you can do so using the [datasette-media](https://github.com/simonw/datasette-media) plugin. This plugin adds the ability to serve images and other static files directly from disk, configured using a SQL query.

  To use it, first install Datasette and the plugin:

      $ pip install datasette datasette-media

  If any of your photos are `.HEIC` images taken by an iPhone you should also install the optional `pyheif` dependency:

      $ pip install pyheif

  Now create a `metadata.yaml` file configuring the plugin:

  ```yaml
  plugins:
    datasette-media:
      thumbnail:
        sql: |-
          select path as filepath, 200 as resize_height from apple_photos where uuid = :key
      large:
        sql: |-
          select path as filepath, 1024 as resize_height from apple_photos where uuid = :key
  ```
  This will configure two URL endpoints - one for 200 pixel high thumbnails and one for 1024 pixel high larger images.

  Create your `photos.db` database using the `apple-photos` command, then run Datasette like this:

      $ datasette -m metadata.yaml

  Your photos will be served on URLs that look like this:

      http://127.0.0.1:8001/-/media/thumbnail/F4469918-13F3-43D8-9EC1-734C0E6B60AD
      http://127.0.0.1:8001/-/media/large/F4469918-13F3-43D8-9EC1-734C0E6B60AD

  You can find the UUIDs for use in these URLs by running `select uuid from photos_with_apple_metadata`.

  ### Displaying images using datasette-json-html

  If you are using `datasette-media` to serve photos you can include images directly in Datasette query results using the [datasette-json-html](https://github.com/simonw/datasette-json-html) plugin.

  Run `pip install datasette-json-html` to install the plugin, then use queries like this to view your images:

  ```sql
  select
      json_object(
          'img_src',
          '/-/media/thumbnail/' || uuid
      ) as photo,
      uuid,
      date
  from
      apple_photos
  order by
      date desc
  limit 10;
  ```
  The `photo` column returned by this query should render as image tags that display the correct images.

  ### Displaying images using custom template pages

  Datasette's [custom pages](https://datasette.readthedocs.io/en/stable/custom_templates.html#custom-pages) feature lets you create custom pages for a Datasette instance by dropping HTML templates into a `templates/pages` directory and then running Datasette using `datasette --template-dir=templates/`.

  You can combine that ability with the [datasette-template-sql](https://github.com/simonw/datasette-template-sql) plugin to create custom template pages that directly display photos served by `datasette-media`.

  Install the plugin using `pip install datasette-template-sql`.

  Create a `templates/pages` folder and add the following files:

  `recent-photos.html`
  ```html+jinja
  <h1>Recent photos</h1>

  <div>
  {% for photo in sql("select * from apple_photos order by date desc limit 20") %}
      <img src="/-/media/photo/{{ photo['uuid'] }}">
  {% endfor %}
  </div>
  ```
  `random-photos.html`
  ```html+jinja
  <h1>Random photos</h1>

  <div>
  {% for photo in sql("with foo as (select * from apple_photos order by date desc limit 5000) select * from foo order by random() limit 20") %}
      <img src="/-/media/photo/{{ photo['uuid'] }}">
  {% endfor %}
  </div>
  ```
  Now run Datasette like this:

      $ datasette photos.db -m metadata.yaml --template-dir=templates/

  Visiting `http://localhost:8001/recent-photos` will display 20 recent photos. Visiting `http://localhost:8001/random-photos` will display 20 photos randomly selected from your 5,000 most recent.

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-dogsheep-photos" class="anchor" aria-hidden="true" href="#user-content-dogsheep-photos"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>dogsheep-photos</h1>
  <p><a href="https://pypi.org/project/dogsheep-photos/" rel="nofollow"><img src="https://camo.githubusercontent.com/c21d10e4250454707420755a0d4b90c97709771d795ffa55b495c98956a3938d/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f646f6773686565702d70686f746f732e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/dogsheep-photos.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/dogsheep/dogsheep-photos/releases"><img src="https://camo.githubusercontent.com/c7bead2f1c989034f62914278a736eecdfbab79ce4d9b3ab804d1692e230d79d/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f646f6773686565702f646f6773686565702d70686f746f733f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/dogsheep/dogsheep-photos?include_prereleases&amp;label=changelog" style="max-width: 100%;"></a>
  <a href="https://circleci.com/gh/dogsheep/dogsheep-photos" rel="nofollow"><img src="https://camo.githubusercontent.com/3aefe86d4a3560c1ca080cf5b7e4dcc5788b239df8b7a3fed40e622b834c22e8/68747470733a2f2f636972636c6563692e636f6d2f67682f646f6773686565702f646f6773686565702d70686f746f732e7376673f7374796c653d737667" alt="CircleCI" data-canonical-src="https://circleci.com/gh/dogsheep/dogsheep-photos.svg?style=svg" style="max-width: 100%;"></a>
  <a href="https://github.com/dogsheep/dogsheep-photos/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width: 100%;"></a></p>
  <p>Save details of your photos to a SQLite database and upload them to S3.</p>
  <p>See <a href="https://simonwillison.net/2020/May/21/apple-photos-sqlite/" rel="nofollow">Using SQL to find my best photo of a pelican according to Apple Photos</a> for background information on this project.</p>
  <h2><a id="user-content-what-these-tools-do" class="anchor" aria-hidden="true" href="#user-content-what-these-tools-do"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>What these tools do</h2>
  <p>These tools are a work-in-progress mechanism for taking full ownership of your photos. The core idea is to help implement the following:</p>
  <ul>
  <li>Every photo you have taken lives in a single, private Amazon S3 bucket</li>
  <li>You have a single SQLite database file which stores metadata about those photos - potentially pulled from multiple different places. This may include EXIF data, Apple Photos, the results of running machine learning APIs against photos and much more besides.</li>
  <li>You can then use <a href="https://github.com/simonw/datasette">Datasette</a> to explore your own photos.</li>
  </ul>
  <p>I'm a heavy user of Apple Photos so the initial releases of this tool will have a bias towards that, but ideally I would like a subset of these tools to be useful to people no matter which core photo solution they are using.</p>
  <h2><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="$ pip install dogsheep-photos
  "><pre><code>$ pip install dogsheep-photos
  </code></pre></div>
  <h2><a id="user-content-authentication-if-using-s3" class="anchor" aria-hidden="true" href="#user-content-authentication-if-using-s3"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Authentication (if using S3)</h2>
  <p>If you want to use S3 to store your photos, you will need to first create S3 credentials for a new, dedicated bucket.</p>
  <p>You may find the <a href="https://github.com/simonw/s3-credentials">s3-credentials tool</a> useful for this.</p>
  <p>Run this command and paste in your credentials. You will need three values: the name of your S3 bucket, your Access key ID and your Secret access key.</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="$ dogsheep-photos s3-auth
  "><pre><code>$ dogsheep-photos s3-auth
  </code></pre></div>
  <p>This will create a file called <code>auth.json</code> in your current directory containing the required values. To save the file at a different path or filename, use the <code>--auth=myauth.json</code> option.</p>
  <h2><a id="user-content-uploading-photos" class="anchor" aria-hidden="true" href="#user-content-uploading-photos"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Uploading photos</h2>
  <p>Run this command to upload every photo in a specific directory to your S3 bucket:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="$ dogsheep-photos upload photos.db \
      ~/Pictures/Photos\ Library.photoslibrary/original
  "><pre><code>$ dogsheep-photos upload photos.db \
      ~/Pictures/Photos\ Library.photoslibrary/original
  </code></pre></div>
  <p>The command will only upload photos that have not yet been uploaded, based on their sha256 hash.</p>
  <p><code>photos.db</code> will be created with an <code>uploads</code> table containing details of which files were uploaded.</p>
  <p>To see what the command would do without uploading any files, use the <code>--dry-run</code> option.</p>
  <p>The sha256 hash of the photo contents will be used as the name of the file in the bucket, with an extension matching the type of file. This is an implementation of the <a href="https://en.wikipedia.org/wiki/Content-addressable_storage" rel="nofollow">Content addressable storage</a> pattern.</p>
  <h2><a id="user-content-importing-apple-photos-metadata" class="anchor" aria-hidden="true" href="#user-content-importing-apple-photos-metadata"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Importing Apple Photos metadata</h2>
  <p>The <code>apple-photos</code> command imports metadata from your Apple Photos library.</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="$ photo-to-sqlite apple-photos photos.db
  "><pre><code>$ photo-to-sqlite apple-photos photos.db
  </code></pre></div>
  <p>Imported metadata includes places, people, albums, quality scores and machine learning labels for the photo contents.</p>
  <h2><a id="user-content-creating-a-subset-database" class="anchor" aria-hidden="true" href="#user-content-creating-a-subset-database"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Creating a subset database</h2>
  <p>You can create a new, subset database of photos using the <code>create-subset</code> command.</p>
  <p>This is useful for creating a shareable SQLite database that only contains metadata for a selected set of photos.</p>
  <p>Since photo metadata contains latitude and longitude you may not want to share a database that includes photos taken at your home address.</p>
  <p><code>create-subset</code> takes three arguments: an existing database file created using the <code>apple-photos</code> command, the name of the new, shareable database file you would like to create and a SQL query that returns the <code>sha256</code> hash values of the photos you would like to include in that database.</p>
  <p>For example, here's how to create a shareable database of just the photos that have been added to albums containing the word "Public":</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="$ dogsheep-photos create-subset \
      photos.db \
      public.db \
      &quot;select sha256 from apple_photos where albums like '%Public%'&quot;
  "><pre><code>$ dogsheep-photos create-subset \
      photos.db \
      public.db \
      "select sha256 from apple_photos where albums like '%Public%'"
  </code></pre></div>
  <h2><a id="user-content-serving-photos-locally-with-datasette-media" class="anchor" aria-hidden="true" href="#user-content-serving-photos-locally-with-datasette-media"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Serving photos locally with datasette-media</h2>
  <p>If you don't want to upload your photos to S3 but you still want to browse them using Datasette you can do so using the <a href="https://github.com/simonw/datasette-media">datasette-media</a> plugin. This plugin adds the ability to serve images and other static files directly from disk, configured using a SQL query.</p>
  <p>To use it, first install Datasette and the plugin:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="$ pip install datasette datasette-media
  "><pre><code>$ pip install datasette datasette-media
  </code></pre></div>
  <p>If any of your photos are <code>.HEIC</code> images taken by an iPhone you should also install the optional <code>pyheif</code> dependency:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="$ pip install pyheif
  "><pre><code>$ pip install pyheif
  </code></pre></div>
  <p>Now create a <code>metadata.yaml</code> file configuring the plugin:</p>
  <div class="highlight highlight-source-yaml position-relative overflow-auto" data-snippet-clipboard-copy-content="plugins:
    datasette-media:
      thumbnail:
        sql: |-
          select path as filepath, 200 as resize_height from apple_photos where uuid = :key
      large:
        sql: |-
          select path as filepath, 1024 as resize_height from apple_photos where uuid = :key
  "><pre><span class="pl-ent">plugins</span>:
    <span class="pl-ent">datasette-media</span>:
      <span class="pl-ent">thumbnail</span>:
        <span class="pl-ent">sql</span>: <span class="pl-s">|-</span>
  <span class="pl-s">        select path as filepath, 200 as resize_height from apple_photos where uuid = :key</span>
  <span class="pl-s"></span>    <span class="pl-ent">large</span>:
        <span class="pl-ent">sql</span>: <span class="pl-s">|-</span>
  <span class="pl-s">        select path as filepath, 1024 as resize_height from apple_photos where uuid = :key</span></pre></div>
  <p>This will configure two URL endpoints - one for 200 pixel high thumbnails and one for 1024 pixel high larger images.</p>
  <p>Create your <code>photos.db</code> database using the <code>apple-photos</code> command, then run Datasette like this:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="$ datasette -m metadata.yaml
  "><pre><code>$ datasette -m metadata.yaml
  </code></pre></div>
  <p>Your photos will be served on URLs that look like this:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="http://127.0.0.1:8001/-/media/thumbnail/F4469918-13F3-43D8-9EC1-734C0E6B60AD
  http://127.0.0.1:8001/-/media/large/F4469918-13F3-43D8-9EC1-734C0E6B60AD
  "><pre><code>http://127.0.0.1:8001/-/media/thumbnail/F4469918-13F3-43D8-9EC1-734C0E6B60AD
  http://127.0.0.1:8001/-/media/large/F4469918-13F3-43D8-9EC1-734C0E6B60AD
  </code></pre></div>
  <p>You can find the UUIDs for use in these URLs by running <code>select uuid from photos_with_apple_metadata</code>.</p>
  <h3><a id="user-content-displaying-images-using-datasette-json-html" class="anchor" aria-hidden="true" href="#user-content-displaying-images-using-datasette-json-html"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Displaying images using datasette-json-html</h3>
  <p>If you are using <code>datasette-media</code> to serve photos you can include images directly in Datasette query results using the <a href="https://github.com/simonw/datasette-json-html">datasette-json-html</a> plugin.</p>
  <p>Run <code>pip install datasette-json-html</code> to install the plugin, then use queries like this to view your images:</p>
  <div class="highlight highlight-source-sql position-relative overflow-auto" data-snippet-clipboard-copy-content="select
      json_object(
          'img_src',
          '/-/media/thumbnail/' || uuid
      ) as photo,
      uuid,
      date
  from
      apple_photos
  order by
      date desc
  limit 10;
  "><pre><span class="pl-k">select</span>
      json_object(
          <span class="pl-s"><span class="pl-pds">'</span>img_src<span class="pl-pds">'</span></span>,
          <span class="pl-s"><span class="pl-pds">'</span>/-/media/thumbnail/<span class="pl-pds">'</span></span> <span class="pl-k">||</span> uuid
      ) <span class="pl-k">as</span> photo,
      uuid,
      <span class="pl-k">date</span>
  <span class="pl-k">from</span>
      apple_photos
  <span class="pl-k">order by</span>
      <span class="pl-k">date</span> <span class="pl-k">desc</span>
  <span class="pl-k">limit</span> <span class="pl-c1">10</span>;</pre></div>
  <p>The <code>photo</code> column returned by this query should render as image tags that display the correct images.</p>
  <h3><a id="user-content-displaying-images-using-custom-template-pages" class="anchor" aria-hidden="true" href="#user-content-displaying-images-using-custom-template-pages"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Displaying images using custom template pages</h3>
  <p>Datasette's <a href="https://datasette.readthedocs.io/en/stable/custom_templates.html#custom-pages" rel="nofollow">custom pages</a> feature lets you create custom pages for a Datasette instance by dropping HTML templates into a <code>templates/pages</code> directory and then running Datasette using <code>datasette --template-dir=templates/</code>.</p>
  <p>You can combine that ability with the <a href="https://github.com/simonw/datasette-template-sql">datasette-template-sql</a> plugin to create custom template pages that directly display photos served by <code>datasette-media</code>.</p>
  <p>Install the plugin using <code>pip install datasette-template-sql</code>.</p>
  <p>Create a <code>templates/pages</code> folder and add the following files:</p>
  <p><code>recent-photos.html</code></p>
  <div class="highlight highlight-text-html-django position-relative overflow-auto" data-snippet-clipboard-copy-content="&lt;h1&gt;Recent photos&lt;/h1&gt;

  &lt;div&gt;
  {% for photo in sql(&quot;select * from apple_photos order by date desc limit 20&quot;) %}
      &lt;img src=&quot;/-/media/photo/{{ photo['uuid'] }}&quot;&gt;
  {% endfor %}
  &lt;/div&gt;
  "><pre>&lt;<span class="pl-ent">h1</span>&gt;Recent photos&lt;/<span class="pl-ent">h1</span>&gt;

  &lt;<span class="pl-ent">div</span>&gt;
  <span class="pl-e">{%</span> <span class="pl-k">for</span> <span class="pl-s">photo</span> <span class="pl-k">in</span> <span class="pl-s">sql</span>(<span class="pl-s">"select * from apple_photos order by date desc limit 20"</span>) <span class="pl-e">%}</span>
      &lt;<span class="pl-ent">img</span> <span class="pl-e">src</span>=<span class="pl-s"><span class="pl-pds">"</span>/-/media/photo/{{ photo['uuid'] }}<span class="pl-pds">"</span></span>&gt;
  <span class="pl-e">{%</span> <span class="pl-k">endfor</span> <span class="pl-e">%}</span>
  &lt;/<span class="pl-ent">div</span>&gt;</pre></div>
  <p><code>random-photos.html</code></p>
  <div class="highlight highlight-text-html-django position-relative overflow-auto" data-snippet-clipboard-copy-content="&lt;h1&gt;Random photos&lt;/h1&gt;

  &lt;div&gt;
  {% for photo in sql(&quot;with foo as (select * from apple_photos order by date desc limit 5000) select * from foo order by random() limit 20&quot;) %}
      &lt;img src=&quot;/-/media/photo/{{ photo['uuid'] }}&quot;&gt;
  {% endfor %}
  &lt;/div&gt;
  "><pre>&lt;<span class="pl-ent">h1</span>&gt;Random photos&lt;/<span class="pl-ent">h1</span>&gt;

  &lt;<span class="pl-ent">div</span>&gt;
  <span class="pl-e">{%</span> <span class="pl-k">for</span> <span class="pl-s">photo</span> <span class="pl-k">in</span> <span class="pl-s">sql</span>(<span class="pl-s">"with foo as (select * from apple_photos order by date desc limit 5000) select * from foo order by random() limit 20"</span>) <span class="pl-e">%}</span>
      &lt;<span class="pl-ent">img</span> <span class="pl-e">src</span>=<span class="pl-s"><span class="pl-pds">"</span>/-/media/photo/{{ photo['uuid'] }}<span class="pl-pds">"</span></span>&gt;
  <span class="pl-e">{%</span> <span class="pl-k">endfor</span> <span class="pl-e">%}</span>
  &lt;/<span class="pl-ent">div</span>&gt;</pre></div>
  <p>Now run Datasette like this:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="$ datasette photos.db -m metadata.yaml --template-dir=templates/
  "><pre><code>$ datasette photos.db -m metadata.yaml --template-dir=templates/
  </code></pre></div>
  <p>Visiting <code>http://localhost:8001/recent-photos</code> will display 20 recent photos. Visiting <code>http://localhost:8001/random-photos</code> will display 20 photos randomly selected from your 5,000 most recent.</p>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyNjE2MzQ4MDc=
name: datasette-media
full_name: simonw/datasette-media
html_url: https://github.com/simonw/datasette-media
description: Datasette plugin for serving media based on a SQL query
created_at: 2020-05-06T02:42:57Z
updated_at: 2021-05-03T05:04:39Z
pushed_at: 2020-07-30T23:39:29Z
language: Python
license: apache-2.0
topics: ["datasette", "datasette-plugin", "datasette-io"]
default_branch: main
permissions: {"admin": false, "push": false, "pull": false}
readme:
  # datasette-media

  [![PyPI](https://img.shields.io/pypi/v/datasette-media.svg)](https://pypi.org/project/datasette-media/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-media?include_prereleases&label=changelog)](https://github.com/simonw/datasette-media/releases)
  [![CircleCI](https://circleci.com/gh/simonw/datasette-media.svg?style=svg)](https://circleci.com/gh/simonw/datasette-media)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-media/blob/master/LICENSE)

  Datasette plugin for serving media based on a SQL query.

  Use this when you have a database table containing references to files on disk - or binary content stored in BLOB columns - that you would like to be able to serve to your users.

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ pip install datasette-media

  ### HEIC image support

  Modern iPhones save their photos using the [HEIC image format](https://en.wikipedia.org/wiki/High_Efficiency_Image_File_Format). Processing these images requires an additional dependency, [pyheif](https://pypi.org/project/pyheif/). You can include this dependency by running:

      $ pip install datasette-media[heif]

  ## Usage

  You can use this plugin to configure Datasette to serve static media based on SQL queries to an underlying database table.

  Media will be served from URLs that start with `/-/media/`. The full URL to each media asset will look like this:

      /-/media/type-of-media/media-key

  `type-of-media` will correspond to a configured SQL query, and might be something like `photo`. `media-key` will be an identifier that is used as part of the underlying SQL query to find which file should be served.

  ### Serving static files from disk

  The following ``metadata.json`` configuration will cause this plugin to serve files from disk, based on queries to a database table called `apple_photos`.

  ```json
  {
      "plugins": {
          "datasette-media": {
              "photo": {
                  "sql": "select filepath from apple_photos where uuid=:key"
              }
          }
      }
  }
  ```

  A request to `/-/media/photo/CF972D33-5324-44F2-8DAE-22CB3182CD31` will execute the following SQL query:

  ```sql
  select filepath from apple_photos where uuid=:key
  ```

  The value from the URL -  in this case `CF972D33-5324-44F2-8DAE-22CB3182CD31` - will be passed as the `:key` parameter to the query.

  The query returns a `filepath` value that has been read from the table. The plugin will then read that file from disk and serve it in response to the request.

  SQL queries default to running against the first connected database. You can specify a different database to execute the query against using `"database": "name_of_db"`. To execute against `photos.db`, use this:

  ```json
  {
      "plugins": {
          "datasette-media": {
              "photo": {
                  "sql": "select filepath from apple_photos where uuid=:key",
                  "database": "photos"
              }
          }
      }
  }
  ```

  See [dogsheep-photos](https://github.com/dogsheep/dogsheep-photos) for an example of an application that can benefit from this plugin.

  ### Serving binary content from BLOB columns

  If your SQL query returns a `content` column, this will be served directly to the user:

  ```json
  {
      "plugins": {
          "datasette-media": {
              "photo": {
                  "sql": "select thumbnail as content from photos where uuid=:key",
                  "database": "thumbs"
              }
          }
      }
  }
  ```

  You can also return a `content_type` column which will be used as the `Content-Type` header served to the user:

  ```json
  {
      "plugins": {
          "datasette-media": {
              "photo": {
                  "sql": "select body as content, 'text/html;charset=utf-8' as content_type from documents where id=:key",
                  "database": "documents"
              }
          }
      }
  }
  ```

  If you do not specify a `content_type` the default of `application/octet-stream` will be used.

  ### Serving content proxied from a URL

  To serve content that is itself fetched from elsewhere, return a `content_url` column. This can be particularly useful when combined with the ability to resize images (described in the next section).

  ```json
  {
      "plugins": {
          "datasette-media": {
              "photos": {
                  "sql": "select photo_url as content_url from photos where id=:key",
                  "database": "photos",
                  "enable_transform": true
              }
          }
      }
  }
  ```

  Now you can access resized versions of images from that URL like so:

      /-/media/photos/13?w=200

  ### Setting a download file name

  The `content_filename` column can be returned to force browsers to download the content using a specific file name.

  ```json
  {
      "plugins": {
          "datasette-media": {
              "hello": {
                  "sql": "select 'Hello ' || :key as content, 'hello.txt' as content_filename"
              }
          }
      }
  }
  ```

  Visiting `/-/media/hello/Groot` will cause your browser to download a file called `hello.txt` containing the text `Hello Groot`.

  ### Resizing or transforming images

  Your SQL query can specify that an image should be resized and/or converted to another format by returning additional columns. All three are optional.

  * `resize_width` - the width to resize the image to
  * `resize_width` - the height to resize the image to
  * `output_format` - the output format to use (e.g. `jpeg` or `png`) - any output format [supported by Pillow](https://pillow.readthedocs.io/en/stable/handbook/image-file-formats.html) is allowed here.

  If you specify one but not the other of `resize_width` or `resize_height` the unspecified one will be calculated automatically to maintain the aspect ratio of the image.

  Here's an example configuration that will resize all images to be JPEGs that are 200 pixels in height:

  ```json
  {
      "plugins": {
          "datasette-media": {
              "photo": {
                  "sql": "select filepath, 200 as resize_height, 'jpeg' as output_format from apple_photos where uuid=:key",
                  "database": "photos"
              }
          }
      }
  }
  ```

  If you enable the `enable_transform` configuration option you can instead specify transform parameters at runtime using querystring parameters. For example:

  - `/-/media/photo/CF972D33?w=200` to resize to a fixed width
  - `/-/media/photo/CF972D33?h=200` to resize to a fixed height
  - `/-/media/photo/CF972D33?format=jpeg` to convert to JPEG

  That option is added like so:

  ```json
  {
      "plugins": {
          "datasette-media": {
              "photo": {
                  "sql": "select filepath from apple_photos where uuid=:key",
                  "database": "photos",
                  "enable_transform": true
              }
          }
      }
  }
  ```

  The maximum allowed height or width is 4000 pixels. You can change this limit using the `"max_width_height"` option:

  ```json
  {
      "plugins": {
          "datasette-media": {
              "photo": {
                  "sql": "select filepath from apple_photos where uuid=:key",
                  "database": "photos",
                  "enable_transform": true,
                  "max_width_height": 1000
              }
          }
      }
  }
  ```

  ## Configuration

  In addition to the different named content types, the following special plugin configuration setting is available:

  - `transform_threads` - number of threads to use for running transformations (e.g. resizing). Defaults to 4.

  This can be used like this:

  ```json
  {
      "plugins": {
          "datasette-media": {
              "photo": {
                  "sql": "select filepath from apple_photos where uuid=:key",
                  "database": "photos"
              },
              "transform_threads": 8
          }
      }
  }
  ```

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-datasette-media" class="anchor" aria-hidden="true" href="#user-content-datasette-media"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-media</h1>
  <p><a href="https://pypi.org/project/datasette-media/" rel="nofollow"><img src="https://camo.githubusercontent.com/e54db370ebd4d8d68a1054cdb8186da706ee7e8e725a8543b6681404e49bff00/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d6d656469612e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-media.svg" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-media/releases"><img src="https://camo.githubusercontent.com/45b31e5abfda86f1f9f9396856b3e703513fbfd980f2070daf1fa3fcb898214b/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d6d656469613f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-media?include_prereleases&amp;label=changelog" style="max-width:100%;"></a>
  <a href="https://circleci.com/gh/simonw/datasette-media" rel="nofollow"><img src="https://camo.githubusercontent.com/3f836c0fc5a36c2afcce223b886456440cf30d4f4cff3c009a40c730fcc0a66d/68747470733a2f2f636972636c6563692e636f6d2f67682f73696d6f6e772f6461746173657474652d6d656469612e7376673f7374796c653d737667" alt="CircleCI" data-canonical-src="https://circleci.com/gh/simonw/datasette-media.svg?style=svg" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-media/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p>
  <p>Datasette plugin for serving media based on a SQL query.</p>
  <p>Use this when you have a database table containing references to files on disk - or binary content stored in BLOB columns - that you would like to be able to serve to your users.</p>
  <h2><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p>Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ pip install datasette-media
  "><pre><code>$ pip install datasette-media
  </code></pre></div>
  <h3><a id="user-content-heic-image-support" class="anchor" aria-hidden="true" href="#user-content-heic-image-support"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>HEIC image support</h3>
  <p>Modern iPhones save their photos using the <a href="https://en.wikipedia.org/wiki/High_Efficiency_Image_File_Format" rel="nofollow">HEIC image format</a>. Processing these images requires an additional dependency, <a href="https://pypi.org/project/pyheif/" rel="nofollow">pyheif</a>. You can include this dependency by running:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ pip install datasette-media[heif]
  "><pre><code>$ pip install datasette-media[heif]
  </code></pre></div>
  <h2><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p>You can use this plugin to configure Datasette to serve static media based on SQL queries to an underlying database table.</p>
  <p>Media will be served from URLs that start with <code>/-/media/</code>. The full URL to each media asset will look like this:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="/-/media/type-of-media/media-key
  "><pre><code>/-/media/type-of-media/media-key
  </code></pre></div>
  <p><code>type-of-media</code> will correspond to a configured SQL query, and might be something like <code>photo</code>. <code>media-key</code> will be an identifier that is used as part of the underlying SQL query to find which file should be served.</p>
  <h3><a id="user-content-serving-static-files-from-disk" class="anchor" aria-hidden="true" href="#user-content-serving-static-files-from-disk"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Serving static files from disk</h3>
  <p>The following <code>metadata.json</code> configuration will cause this plugin to serve files from disk, based on queries to a database table called <code>apple_photos</code>.</p>
  <div class="highlight highlight-source-json position-relative" data-snippet-clipboard-copy-content="{
      &quot;plugins&quot;: {
          &quot;datasette-media&quot;: {
              &quot;photo&quot;: {
                  &quot;sql&quot;: &quot;select filepath from apple_photos where uuid=:key&quot;
              }
          }
      }
  }
  "><pre>{
      <span class="pl-s"><span class="pl-pds">"</span>plugins<span class="pl-pds">"</span></span>: {
          <span class="pl-s"><span class="pl-pds">"</span>datasette-media<span class="pl-pds">"</span></span>: {
              <span class="pl-s"><span class="pl-pds">"</span>photo<span class="pl-pds">"</span></span>: {
                  <span class="pl-s"><span class="pl-pds">"</span>sql<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>select filepath from apple_photos where uuid=:key<span class="pl-pds">"</span></span>
              }
          }
      }
  }</pre></div>
  <p>A request to <code>/-/media/photo/CF972D33-5324-44F2-8DAE-22CB3182CD31</code> will execute the following SQL query:</p>
  <div class="highlight highlight-source-sql position-relative" data-snippet-clipboard-copy-content="select filepath from apple_photos where uuid=:key
  "><pre><span class="pl-k">select</span> filepath <span class="pl-k">from</span> apple_photos <span class="pl-k">where</span> uuid<span class="pl-k">=</span>:key</pre></div>
  <p>The value from the URL -  in this case <code>CF972D33-5324-44F2-8DAE-22CB3182CD31</code> - will be passed as the <code>:key</code> parameter to the query.</p>
  <p>The query returns a <code>filepath</code> value that has been read from the table. The plugin will then read that file from disk and serve it in response to the request.</p>
  <p>SQL queries default to running against the first connected database. You can specify a different database to execute the query against using <code>"database": "name_of_db"</code>. To execute against <code>photos.db</code>, use this:</p>
  <div class="highlight highlight-source-json position-relative" data-snippet-clipboard-copy-content="{
      &quot;plugins&quot;: {
          &quot;datasette-media&quot;: {
              &quot;photo&quot;: {
                  &quot;sql&quot;: &quot;select filepath from apple_photos where uuid=:key&quot;,
                  &quot;database&quot;: &quot;photos&quot;
              }
          }
      }
  }
  "><pre>{
      <span class="pl-s"><span class="pl-pds">"</span>plugins<span class="pl-pds">"</span></span>: {
          <span class="pl-s"><span class="pl-pds">"</span>datasette-media<span class="pl-pds">"</span></span>: {
              <span class="pl-s"><span class="pl-pds">"</span>photo<span class="pl-pds">"</span></span>: {
                  <span class="pl-s"><span class="pl-pds">"</span>sql<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>select filepath from apple_photos where uuid=:key<span class="pl-pds">"</span></span>,
                  <span class="pl-s"><span class="pl-pds">"</span>database<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>photos<span class="pl-pds">"</span></span>
              }
          }
      }
  }</pre></div>
  <p>See <a href="https://github.com/dogsheep/dogsheep-photos">dogsheep-photos</a> for an example of an application that can benefit from this plugin.</p>
  <h3><a id="user-content-serving-binary-content-from-blob-columns" class="anchor" aria-hidden="true" href="#user-content-serving-binary-content-from-blob-columns"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Serving binary content from BLOB columns</h3>
  <p>If your SQL query returns a <code>content</code> column, this will be served directly to the user:</p>
  <div class="highlight highlight-source-json position-relative" data-snippet-clipboard-copy-content="{
      &quot;plugins&quot;: {
          &quot;datasette-media&quot;: {
              &quot;photo&quot;: {
                  &quot;sql&quot;: &quot;select thumbnail as content from photos where uuid=:key&quot;,
                  &quot;database&quot;: &quot;thumbs&quot;
              }
          }
      }
  }
  "><pre>{
      <span class="pl-s"><span class="pl-pds">"</span>plugins<span class="pl-pds">"</span></span>: {
          <span class="pl-s"><span class="pl-pds">"</span>datasette-media<span class="pl-pds">"</span></span>: {
              <span class="pl-s"><span class="pl-pds">"</span>photo<span class="pl-pds">"</span></span>: {
                  <span class="pl-s"><span class="pl-pds">"</span>sql<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>select thumbnail as content from photos where uuid=:key<span class="pl-pds">"</span></span>,
                  <span class="pl-s"><span class="pl-pds">"</span>database<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>thumbs<span class="pl-pds">"</span></span>
              }
          }
      }
  }</pre></div>
  <p>You can also return a <code>content_type</code> column which will be used as the <code>Content-Type</code> header served to the user:</p>
  <div class="highlight highlight-source-json position-relative" data-snippet-clipboard-copy-content="{
      &quot;plugins&quot;: {
          &quot;datasette-media&quot;: {
              &quot;photo&quot;: {
                  &quot;sql&quot;: &quot;select body as content, 'text/html;charset=utf-8' as content_type from documents where id=:key&quot;,
                  &quot;database&quot;: &quot;documents&quot;
              }
          }
      }
  }
  "><pre>{
      <span class="pl-s"><span class="pl-pds">"</span>plugins<span class="pl-pds">"</span></span>: {
          <span class="pl-s"><span class="pl-pds">"</span>datasette-media<span class="pl-pds">"</span></span>: {
              <span class="pl-s"><span class="pl-pds">"</span>photo<span class="pl-pds">"</span></span>: {
                  <span class="pl-s"><span class="pl-pds">"</span>sql<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>select body as content, 'text/html;charset=utf-8' as content_type from documents where id=:key<span class="pl-pds">"</span></span>,
                  <span class="pl-s"><span class="pl-pds">"</span>database<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>documents<span class="pl-pds">"</span></span>
              }
          }
      }
  }</pre></div>
  <p>If you do not specify a <code>content_type</code> the default of <code>application/octet-stream</code> will be used.</p>
  <h3><a id="user-content-serving-content-proxied-from-a-url" class="anchor" aria-hidden="true" href="#user-content-serving-content-proxied-from-a-url"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Serving content proxied from a URL</h3>
  <p>To serve content that is itself fetched from elsewhere, return a <code>content_url</code> column. This can be particularly useful when combined with the ability to resize images (described in the next section).</p>
  <div class="highlight highlight-source-json position-relative" data-snippet-clipboard-copy-content="{
      &quot;plugins&quot;: {
          &quot;datasette-media&quot;: {
              &quot;photos&quot;: {
                  &quot;sql&quot;: &quot;select photo_url as content_url from photos where id=:key&quot;,
                  &quot;database&quot;: &quot;photos&quot;,
                  &quot;enable_transform&quot;: true
              }
          }
      }
  }
  "><pre>{
      <span class="pl-s"><span class="pl-pds">"</span>plugins<span class="pl-pds">"</span></span>: {
          <span class="pl-s"><span class="pl-pds">"</span>datasette-media<span class="pl-pds">"</span></span>: {
              <span class="pl-s"><span class="pl-pds">"</span>photos<span class="pl-pds">"</span></span>: {
                  <span class="pl-s"><span class="pl-pds">"</span>sql<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>select photo_url as content_url from photos where id=:key<span class="pl-pds">"</span></span>,
                  <span class="pl-s"><span class="pl-pds">"</span>database<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>photos<span class="pl-pds">"</span></span>,
                  <span class="pl-s"><span class="pl-pds">"</span>enable_transform<span class="pl-pds">"</span></span>: <span class="pl-c1">true</span>
              }
          }
      }
  }</pre></div>
  <p>Now you can access resized versions of images from that URL like so:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="/-/media/photos/13?w=200
  "><pre><code>/-/media/photos/13?w=200
  </code></pre></div>
  <h3><a id="user-content-setting-a-download-file-name" class="anchor" aria-hidden="true" href="#user-content-setting-a-download-file-name"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Setting a download file name</h3>
  <p>The <code>content_filename</code> column can be returned to force browsers to download the content using a specific file name.</p>
  <div class="highlight highlight-source-json position-relative" data-snippet-clipboard-copy-content="{
      &quot;plugins&quot;: {
          &quot;datasette-media&quot;: {
              &quot;hello&quot;: {
                  &quot;sql&quot;: &quot;select 'Hello ' || :key as content, 'hello.txt' as content_filename&quot;
              }
          }
      }
  }
  "><pre>{
      <span class="pl-s"><span class="pl-pds">"</span>plugins<span class="pl-pds">"</span></span>: {
          <span class="pl-s"><span class="pl-pds">"</span>datasette-media<span class="pl-pds">"</span></span>: {
              <span class="pl-s"><span class="pl-pds">"</span>hello<span class="pl-pds">"</span></span>: {
                  <span class="pl-s"><span class="pl-pds">"</span>sql<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>select 'Hello ' || :key as content, 'hello.txt' as content_filename<span class="pl-pds">"</span></span>
              }
          }
      }
  }</pre></div>
  <p>Visiting <code>/-/media/hello/Groot</code> will cause your browser to download a file called <code>hello.txt</code> containing the text <code>Hello Groot</code>.</p>
  <h3><a id="user-content-resizing-or-transforming-images" class="anchor" aria-hidden="true" href="#user-content-resizing-or-transforming-images"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Resizing or transforming images</h3>
  <p>Your SQL query can specify that an image should be resized and/or converted to another format by returning additional columns. All three are optional.</p>
  <ul>
  <li><code>resize_width</code> - the width to resize the image to</li>
  <li><code>resize_width</code> - the height to resize the image to</li>
  <li><code>output_format</code> - the output format to use (e.g. <code>jpeg</code> or <code>png</code>) - any output format <a href="https://pillow.readthedocs.io/en/stable/handbook/image-file-formats.html" rel="nofollow">supported by Pillow</a> is allowed here.</li>
  </ul>
  <p>If you specify one but not the other of <code>resize_width</code> or <code>resize_height</code> the unspecified one will be calculated automatically to maintain the aspect ratio of the image.</p>
  <p>Here's an example configuration that will resize all images to be JPEGs that are 200 pixels in height:</p>
  <div class="highlight highlight-source-json position-relative" data-snippet-clipboard-copy-content="{
      &quot;plugins&quot;: {
          &quot;datasette-media&quot;: {
              &quot;photo&quot;: {
                  &quot;sql&quot;: &quot;select filepath, 200 as resize_height, 'jpeg' as output_format from apple_photos where uuid=:key&quot;,
                  &quot;database&quot;: &quot;photos&quot;
              }
          }
      }
  }
  "><pre>{
      <span class="pl-s"><span class="pl-pds">"</span>plugins<span class="pl-pds">"</span></span>: {
          <span class="pl-s"><span class="pl-pds">"</span>datasette-media<span class="pl-pds">"</span></span>: {
              <span class="pl-s"><span class="pl-pds">"</span>photo<span class="pl-pds">"</span></span>: {
                  <span class="pl-s"><span class="pl-pds">"</span>sql<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>select filepath, 200 as resize_height, 'jpeg' as output_format from apple_photos where uuid=:key<span class="pl-pds">"</span></span>,
                  <span class="pl-s"><span class="pl-pds">"</span>database<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>photos<span class="pl-pds">"</span></span>
              }
          }
      }
  }</pre></div>
  <p>If you enable the <code>enable_transform</code> configuration option you can instead specify transform parameters at runtime using querystring parameters. For example:</p>
  <ul>
  <li><code>/-/media/photo/CF972D33?w=200</code> to resize to a fixed width</li>
  <li><code>/-/media/photo/CF972D33?h=200</code> to resize to a fixed height</li>
  <li><code>/-/media/photo/CF972D33?format=jpeg</code> to convert to JPEG</li>
  </ul>
  <p>That option is added like so:</p>
  <div class="highlight highlight-source-json position-relative" data-snippet-clipboard-copy-content="{
      &quot;plugins&quot;: {
          &quot;datasette-media&quot;: {
              &quot;photo&quot;: {
                  &quot;sql&quot;: &quot;select filepath from apple_photos where uuid=:key&quot;,
                  &quot;database&quot;: &quot;photos&quot;,
                  &quot;enable_transform&quot;: true
              }
          }
      }
  }
  "><pre>{
      <span class="pl-s"><span class="pl-pds">"</span>plugins<span class="pl-pds">"</span></span>: {
          <span class="pl-s"><span class="pl-pds">"</span>datasette-media<span class="pl-pds">"</span></span>: {
              <span class="pl-s"><span class="pl-pds">"</span>photo<span class="pl-pds">"</span></span>: {
                  <span class="pl-s"><span class="pl-pds">"</span>sql<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>select filepath from apple_photos where uuid=:key<span class="pl-pds">"</span></span>,
                  <span class="pl-s"><span class="pl-pds">"</span>database<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>photos<span class="pl-pds">"</span></span>,
                  <span class="pl-s"><span class="pl-pds">"</span>enable_transform<span class="pl-pds">"</span></span>: <span class="pl-c1">true</span>
              }
          }
      }
  }</pre></div>
  <p>The maximum allowed height or width is 4000 pixels. You can change this limit using the <code>"max_width_height"</code> option:</p>
  <div class="highlight highlight-source-json position-relative" data-snippet-clipboard-copy-content="{
      &quot;plugins&quot;: {
          &quot;datasette-media&quot;: {
              &quot;photo&quot;: {
                  &quot;sql&quot;: &quot;select filepath from apple_photos where uuid=:key&quot;,
                  &quot;database&quot;: &quot;photos&quot;,
                  &quot;enable_transform&quot;: true,
                  &quot;max_width_height&quot;: 1000
              }
          }
      }
  }
  "><pre>{
      <span class="pl-s"><span class="pl-pds">"</span>plugins<span class="pl-pds">"</span></span>: {
          <span class="pl-s"><span class="pl-pds">"</span>datasette-media<span class="pl-pds">"</span></span>: {
              <span class="pl-s"><span class="pl-pds">"</span>photo<span class="pl-pds">"</span></span>: {
                  <span class="pl-s"><span class="pl-pds">"</span>sql<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>select filepath from apple_photos where uuid=:key<span class="pl-pds">"</span></span>,
                  <span class="pl-s"><span class="pl-pds">"</span>database<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>photos<span class="pl-pds">"</span></span>,
                  <span class="pl-s"><span class="pl-pds">"</span>enable_transform<span class="pl-pds">"</span></span>: <span class="pl-c1">true</span>,
                  <span class="pl-s"><span class="pl-pds">"</span>max_width_height<span class="pl-pds">"</span></span>: <span class="pl-c1">1000</span>
              }
          }
      }
  }</pre></div>
  <h2><a id="user-content-configuration" class="anchor" aria-hidden="true" href="#user-content-configuration"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Configuration</h2>
  <p>In addition to the different named content types, the following special plugin configuration setting is available:</p>
  <ul>
  <li><code>transform_threads</code> - number of threads to use for running transformations (e.g. resizing). Defaults to 4.</li>
  </ul>
  <p>This can be used like this:</p>
  <div class="highlight highlight-source-json position-relative" data-snippet-clipboard-copy-content="{
      &quot;plugins&quot;: {
          &quot;datasette-media&quot;: {
              &quot;photo&quot;: {
                  &quot;sql&quot;: &quot;select filepath from apple_photos where uuid=:key&quot;,
                  &quot;database&quot;: &quot;photos&quot;
              },
              &quot;transform_threads&quot;: 8
          }
      }
  }
  "><pre>{
      <span class="pl-s"><span class="pl-pds">"</span>plugins<span class="pl-pds">"</span></span>: {
          <span class="pl-s"><span class="pl-pds">"</span>datasette-media<span class="pl-pds">"</span></span>: {
              <span class="pl-s"><span class="pl-pds">"</span>photo<span class="pl-pds">"</span></span>: {
                  <span class="pl-s"><span class="pl-pds">"</span>sql<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>select filepath from apple_photos where uuid=:key<span class="pl-pds">"</span></span>,
                  <span class="pl-s"><span class="pl-pds">"</span>database<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>photos<span class="pl-pds">"</span></span>
              },
              <span class="pl-s"><span class="pl-pds">"</span>transform_threads<span class="pl-pds">"</span></span>: <span class="pl-c1">8</span>
          }
      }
  }</pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyNzE0MDg4OTU=
name: datasette-permissions-sql
full_name: simonw/datasette-permissions-sql
html_url: https://github.com/simonw/datasette-permissions-sql
description: Datasette plugin for configuring permission checks using SQL queries
created_at: 2020-06-10T23:48:13Z
updated_at: 2020-06-12T07:06:12Z
pushed_at: 2020-06-12T07:06:15Z
language: Python
license: apache-2.0
topics: ["datasette", "datasette-plugin", "datasette-io"]
default_branch: master
permissions: {"admin": false, "push": false, "pull": false}
readme:
  # datasette-permissions-sql

  [![PyPI](https://img.shields.io/pypi/v/datasette-permissions-sql.svg)](https://pypi.org/project/datasette-permissions-sql/)
  [![CircleCI](https://circleci.com/gh/simonw/datasette-permissions-sql.svg?style=svg)](https://circleci.com/gh/simonw/datasette-permissions-sql)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-permissions-sql/blob/master/LICENSE)

  Datasette plugin for configuring permission checks using SQL queries

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ pip install datasette-permissions-sql

  ## Usage

  First, read up on how Datasette's [authentication and permissions system](https://datasette.readthedocs.io/en/latest/authentication.html) works.

  This plugin lets you define rules containing SQL queries that are executed to see if the currently authenticated actor has permission to perform certain actions.

  Consider a canned query which authenticated users should only be able to execute if a row in the `users` table says that they are a member of staff.

  That `users` table in the `mydatabase.db` database could look like this:

  | id | username | is_staff |
  |--|--------|--------|
  | 1 | cleopaws | 0 |
  | 2 | simon | 1 |

  Authenticated users have an `actor` that looks like this:

  ```json
  {
      "id": 2,
      "username": "simon"
  }
  ```

  To configure the canned query to only be executable by staff users, add the following to your `metadata.json`:

  ```json
  {
      "plugins": {
          "datasette-permissions-sql": [
              {
                  "action": "view-query",
                  "resource": ["mydatabase", "promote_to_staff"],
                  "sql": "SELECT * FROM users WHERE is_staff = 1 AND id = :actor_id"
              }
          ]
      },
      "databases": {
          "mydatabase": {
              "queries": {
                  "promote_to_staff": {
                      "sql": "UPDATE users SET is is_staff=1 WHERE id=:id",
                      "write": true
                  }
              }
          }
      }
  }
  ```

  The `"datasette-permissions-sql"` key is a list of rules. Each of those rules has the following shape:

  ```json
  {
      "action": "name-of-action",
      "resource": ["resource identifier to run this on"],
      "sql": "SQL query to execute",
      "database": "mydatabase"
  }
  ```

  Both `"action"` and `"resource"` are optional. If present, the SQL query will only be executed on permission checks that match the action and, if present, the resource indicators.

  `"database"` is also optional: it specifies the named database that the query should be executed against. If it is not present the first connected database will be used.

  The Datasette documentation includes a [list of built-in permissions](https://datasette.readthedocs.io/en/stable/authentication.html#built-in-permissions) that you might want to use here.

  ### The SQL query

  If the SQL query returns any rows the action will be allowed. If it returns no rows, the plugin hook will return `False` and deny access to that action.

  The SQL query is called with a number of named parameters. You can use any of these as part of the query.

  The list of parameters is as follows:

  * `action` - the action, e.g. `"view-database"`
  * `resource_1` - the first component of the resource, if one was passed
  * `resource_2` - the second component of the resource, if available
  * `actor_*` - a parameter for every key on the actor. Usually `actor_id` is present.

  If any rows are returned, the permission check passes. If no rows are returned the check fails.

  Another example table, this time granting explicit access to individual tables. Consider a table called `table_access` that looks like this:

  | user_id | database | table |
  | - | - | - |
  | 1 | mydb | dogs |
  | 2 | mydb | dogs |
  | 1 | mydb | cats |

  The following SQL query would grant access to the `dogs` ttable in the `mydb.db` database to users 1 and 2 - but would forbid access for user 2 to the `cats` table:

  ```sql
  SELECT
      *
  FROM
      table_access
  WHERE
      user_id = :actor_id
      AND "database" = :resource_1
      AND "table" = :resource_2
  ```
  In a `metadata.yaml` configuration file that would look like this:

  ```yaml
  databases:
    mydb:
      allow_sql: {}
  plugins:
    datasette-permissions-sql:
    - action: view-table
      sql: |-
        SELECT
          *
        FROM
          table_access
        WHERE
          user_id = :actor_id
          AND "database" = :resource_1
          AND "table" = :resource_2
  ```
  We're using `allow_sql: {}` here to disable arbitrary SQL queries. This prevents users from running `select * from cats` directly to work around the permissions limits.

  ### Fallback mode

  The default behaviour of this plugin is to take full control of specified permissions. The SQL query will directly control if the user is allowed or denied access to the permission.

  This means that the default policy for each permission (which in Datasette core is "allow" for `view-database` and friends) will be ignored. It also means that any other `permission_allowed` plugins will not get their turn once this plugin has executed.

  You can change this on a per-rule basis using ``"fallback": true``:

  ```json
  {
      "action": "view-table",
      "resource": ["mydatabase", "mytable"],
      "sql": "select * from admins where user_id = :actor_id",
      "fallback": true
  }
  ```

  When running in fallback mode, a query result returning no rows will cause the plugin hook to return ``None`` - which means "I have no opinion on this permission, fall back to other plugins or the default".

  In this mode you can still return `False` (for "deny access") by returning a single row with a single value of `-1`. For example:

  ```json
  {
      "action": "view-table",
      "resource": ["mydatabase", "mytable"],
      "sql": "select -1 from banned where user_id = :actor_id",
      "fallback": true
  }
  ```

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-datasette-permissions-sql" class="anchor" aria-hidden="true" href="#user-content-datasette-permissions-sql"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-permissions-sql</h1>
  <p><a href="https://pypi.org/project/datasette-permissions-sql/" rel="nofollow"><img src="https://camo.githubusercontent.com/898a57540c5f63707e7d55e28f18598cd101ae1b7cbad1ff71ead7f4a12174f4/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d7065726d697373696f6e732d73716c2e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-permissions-sql.svg" style="max-width:100%;"></a>
  <a href="https://circleci.com/gh/simonw/datasette-permissions-sql" rel="nofollow"><img src="https://camo.githubusercontent.com/587e99cf6d88c2bcfe530b5748d7f0f2ed638809bec3aa914244b7bf2ddcdc82/68747470733a2f2f636972636c6563692e636f6d2f67682f73696d6f6e772f6461746173657474652d7065726d697373696f6e732d73716c2e7376673f7374796c653d737667" alt="CircleCI" data-canonical-src="https://circleci.com/gh/simonw/datasette-permissions-sql.svg?style=svg" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-permissions-sql/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p>
  <p>Datasette plugin for configuring permission checks using SQL queries</p>
  <h2><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p>Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ pip install datasette-permissions-sql
  "><pre><code>$ pip install datasette-permissions-sql
  </code></pre></div>
  <h2><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p>First, read up on how Datasette's <a href="https://datasette.readthedocs.io/en/latest/authentication.html" rel="nofollow">authentication and permissions system</a> works.</p>
  <p>This plugin lets you define rules containing SQL queries that are executed to see if the currently authenticated actor has permission to perform certain actions.</p>
  <p>Consider a canned query which authenticated users should only be able to execute if a row in the <code>users</code> table says that they are a member of staff.</p>
  <p>That <code>users</code> table in the <code>mydatabase.db</code> database could look like this:</p>
  <table>
  <thead>
  <tr>
  <th>id</th>
  <th>username</th>
  <th>is_staff</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td>1</td>
  <td>cleopaws</td>
  <td>0</td>
  </tr>
  <tr>
  <td>2</td>
  <td>simon</td>
  <td>1</td>
  </tr>
  </tbody>
  </table>
  <p>Authenticated users have an <code>actor</code> that looks like this:</p>
  <div class="highlight highlight-source-json position-relative" data-snippet-clipboard-copy-content="{
      &quot;id&quot;: 2,
      &quot;username&quot;: &quot;simon&quot;
  }
  "><pre>{
      <span class="pl-s"><span class="pl-pds">"</span>id<span class="pl-pds">"</span></span>: <span class="pl-c1">2</span>,
      <span class="pl-s"><span class="pl-pds">"</span>username<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>simon<span class="pl-pds">"</span></span>
  }</pre></div>
  <p>To configure the canned query to only be executable by staff users, add the following to your <code>metadata.json</code>:</p>
  <div class="highlight highlight-source-json position-relative" data-snippet-clipboard-copy-content="{
      &quot;plugins&quot;: {
          &quot;datasette-permissions-sql&quot;: [
              {
                  &quot;action&quot;: &quot;view-query&quot;,
                  &quot;resource&quot;: [&quot;mydatabase&quot;, &quot;promote_to_staff&quot;],
                  &quot;sql&quot;: &quot;SELECT * FROM users WHERE is_staff = 1 AND id = :actor_id&quot;
              }
          ]
      },
      &quot;databases&quot;: {
          &quot;mydatabase&quot;: {
              &quot;queries&quot;: {
                  &quot;promote_to_staff&quot;: {
                      &quot;sql&quot;: &quot;UPDATE users SET is is_staff=1 WHERE id=:id&quot;,
                      &quot;write&quot;: true
                  }
              }
          }
      }
  }
  "><pre>{
      <span class="pl-s"><span class="pl-pds">"</span>plugins<span class="pl-pds">"</span></span>: {
          <span class="pl-s"><span class="pl-pds">"</span>datasette-permissions-sql<span class="pl-pds">"</span></span>: [
              {
                  <span class="pl-s"><span class="pl-pds">"</span>action<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>view-query<span class="pl-pds">"</span></span>,
                  <span class="pl-s"><span class="pl-pds">"</span>resource<span class="pl-pds">"</span></span>: [<span class="pl-s"><span class="pl-pds">"</span>mydatabase<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>promote_to_staff<span class="pl-pds">"</span></span>],
                  <span class="pl-s"><span class="pl-pds">"</span>sql<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>SELECT * FROM users WHERE is_staff = 1 AND id = :actor_id<span class="pl-pds">"</span></span>
              }
          ]
      },
      <span class="pl-s"><span class="pl-pds">"</span>databases<span class="pl-pds">"</span></span>: {
          <span class="pl-s"><span class="pl-pds">"</span>mydatabase<span class="pl-pds">"</span></span>: {
              <span class="pl-s"><span class="pl-pds">"</span>queries<span class="pl-pds">"</span></span>: {
                  <span class="pl-s"><span class="pl-pds">"</span>promote_to_staff<span class="pl-pds">"</span></span>: {
                      <span class="pl-s"><span class="pl-pds">"</span>sql<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>UPDATE users SET is is_staff=1 WHERE id=:id<span class="pl-pds">"</span></span>,
                      <span class="pl-s"><span class="pl-pds">"</span>write<span class="pl-pds">"</span></span>: <span class="pl-c1">true</span>
                  }
              }
          }
      }
  }</pre></div>
  <p>The <code>"datasette-permissions-sql"</code> key is a list of rules. Each of those rules has the following shape:</p>
  <div class="highlight highlight-source-json position-relative" data-snippet-clipboard-copy-content="{
      &quot;action&quot;: &quot;name-of-action&quot;,
      &quot;resource&quot;: [&quot;resource identifier to run this on&quot;],
      &quot;sql&quot;: &quot;SQL query to execute&quot;,
      &quot;database&quot;: &quot;mydatabase&quot;
  }
  "><pre>{
      <span class="pl-s"><span class="pl-pds">"</span>action<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>name-of-action<span class="pl-pds">"</span></span>,
      <span class="pl-s"><span class="pl-pds">"</span>resource<span class="pl-pds">"</span></span>: [<span class="pl-s"><span class="pl-pds">"</span>resource identifier to run this on<span class="pl-pds">"</span></span>],
      <span class="pl-s"><span class="pl-pds">"</span>sql<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>SQL query to execute<span class="pl-pds">"</span></span>,
      <span class="pl-s"><span class="pl-pds">"</span>database<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>mydatabase<span class="pl-pds">"</span></span>
  }</pre></div>
  <p>Both <code>"action"</code> and <code>"resource"</code> are optional. If present, the SQL query will only be executed on permission checks that match the action and, if present, the resource indicators.</p>
  <p><code>"database"</code> is also optional: it specifies the named database that the query should be executed against. If it is not present the first connected database will be used.</p>
  <p>The Datasette documentation includes a <a href="https://datasette.readthedocs.io/en/stable/authentication.html#built-in-permissions" rel="nofollow">list of built-in permissions</a> that you might want to use here.</p>
  <h3><a id="user-content-the-sql-query" class="anchor" aria-hidden="true" href="#user-content-the-sql-query"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>The SQL query</h3>
  <p>If the SQL query returns any rows the action will be allowed. If it returns no rows, the plugin hook will return <code>False</code> and deny access to that action.</p>
  <p>The SQL query is called with a number of named parameters. You can use any of these as part of the query.</p>
  <p>The list of parameters is as follows:</p>
  <ul>
  <li><code>action</code> - the action, e.g. <code>"view-database"</code></li>
  <li><code>resource_1</code> - the first component of the resource, if one was passed</li>
  <li><code>resource_2</code> - the second component of the resource, if available</li>
  <li><code>actor_*</code> - a parameter for every key on the actor. Usually <code>actor_id</code> is present.</li>
  </ul>
  <p>If any rows are returned, the permission check passes. If no rows are returned the check fails.</p>
  <p>Another example table, this time granting explicit access to individual tables. Consider a table called <code>table_access</code> that looks like this:</p>
  <table>
  <thead>
  <tr>
  <th>user_id</th>
  <th>database</th>
  <th>table</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td>1</td>
  <td>mydb</td>
  <td>dogs</td>
  </tr>
  <tr>
  <td>2</td>
  <td>mydb</td>
  <td>dogs</td>
  </tr>
  <tr>
  <td>1</td>
  <td>mydb</td>
  <td>cats</td>
  </tr>
  </tbody>
  </table>
  <p>The following SQL query would grant access to the <code>dogs</code> ttable in the <code>mydb.db</code> database to users 1 and 2 - but would forbid access for user 2 to the <code>cats</code> table:</p>
  <div class="highlight highlight-source-sql position-relative" data-snippet-clipboard-copy-content="SELECT
      *
  FROM
      table_access
  WHERE
      user_id = :actor_id
      AND &quot;database&quot; = :resource_1
      AND &quot;table&quot; = :resource_2
  "><pre><span class="pl-k">SELECT</span>
      <span class="pl-k">*</span>
  <span class="pl-k">FROM</span>
      table_access
  <span class="pl-k">WHERE</span>
      user_id <span class="pl-k">=</span> :actor_id
      <span class="pl-k">AND</span> <span class="pl-s"><span class="pl-pds">"</span>database<span class="pl-pds">"</span></span> <span class="pl-k">=</span> :resource_1
      <span class="pl-k">AND</span> <span class="pl-s"><span class="pl-pds">"</span>table<span class="pl-pds">"</span></span> <span class="pl-k">=</span> :resource_2</pre></div>
  <p>In a <code>metadata.yaml</code> configuration file that would look like this:</p>
  <div class="highlight highlight-source-yaml position-relative" data-snippet-clipboard-copy-content="databases:
    mydb:
      allow_sql: {}
  plugins:
    datasette-permissions-sql:
    - action: view-table
      sql: |-
        SELECT
          *
        FROM
          table_access
        WHERE
          user_id = :actor_id
          AND &quot;database&quot; = :resource_1
          AND &quot;table&quot; = :resource_2
  "><pre><span class="pl-ent">databases</span>:
    <span class="pl-ent">mydb</span>:
      <span class="pl-ent">allow_sql</span>: <span class="pl-s">{}</span>
  <span class="pl-ent">plugins</span>:
    <span class="pl-ent">datasette-permissions-sql</span>:
    - <span class="pl-ent">action</span>: <span class="pl-s">view-table</span>
      <span class="pl-ent">sql</span>: <span class="pl-s">|-</span>
  <span class="pl-s">      SELECT</span>
  <span class="pl-s">        *</span>
  <span class="pl-s">      FROM</span>
  <span class="pl-s">        table_access</span>
  <span class="pl-s">      WHERE</span>
  <span class="pl-s">        user_id = :actor_id</span>
  <span class="pl-s">        AND "database" = :resource_1</span>
  <span class="pl-s">        AND "table" = :resource_2</span></pre></div>
  <p>We're using <code>allow_sql: {}</code> here to disable arbitrary SQL queries. This prevents users from running <code>select * from cats</code> directly to work around the permissions limits.</p>
  <h3><a id="user-content-fallback-mode" class="anchor" aria-hidden="true" href="#user-content-fallback-mode"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Fallback mode</h3>
  <p>The default behaviour of this plugin is to take full control of specified permissions. The SQL query will directly control if the user is allowed or denied access to the permission.</p>
  <p>This means that the default policy for each permission (which in Datasette core is "allow" for <code>view-database</code> and friends) will be ignored. It also means that any other <code>permission_allowed</code> plugins will not get their turn once this plugin has executed.</p>
  <p>You can change this on a per-rule basis using <code>"fallback": true</code>:</p>
  <div class="highlight highlight-source-json position-relative" data-snippet-clipboard-copy-content="{
      &quot;action&quot;: &quot;view-table&quot;,
      &quot;resource&quot;: [&quot;mydatabase&quot;, &quot;mytable&quot;],
      &quot;sql&quot;: &quot;select * from admins where user_id = :actor_id&quot;,
      &quot;fallback&quot;: true
  }
  "><pre>{
      <span class="pl-s"><span class="pl-pds">"</span>action<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>view-table<span class="pl-pds">"</span></span>,
      <span class="pl-s"><span class="pl-pds">"</span>resource<span class="pl-pds">"</span></span>: [<span class="pl-s"><span class="pl-pds">"</span>mydatabase<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>mytable<span class="pl-pds">"</span></span>],
      <span class="pl-s"><span class="pl-pds">"</span>sql<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>select * from admins where user_id = :actor_id<span class="pl-pds">"</span></span>,
      <span class="pl-s"><span class="pl-pds">"</span>fallback<span class="pl-pds">"</span></span>: <span class="pl-c1">true</span>
  }</pre></div>
  <p>When running in fallback mode, a query result returning no rows will cause the plugin hook to return <code>None</code> - which means "I have no opinion on this permission, fall back to other plugins or the default".</p>
  <p>In this mode you can still return <code>False</code> (for "deny access") by returning a single row with a single value of <code>-1</code>. For example:</p>
  <div class="highlight highlight-source-json position-relative" data-snippet-clipboard-copy-content="{
      &quot;action&quot;: &quot;view-table&quot;,
      &quot;resource&quot;: [&quot;mydatabase&quot;, &quot;mytable&quot;],
      &quot;sql&quot;: &quot;select -1 from banned where user_id = :actor_id&quot;,
      &quot;fallback&quot;: true
  }
  "><pre>{
      <span class="pl-s"><span class="pl-pds">"</span>action<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>view-table<span class="pl-pds">"</span></span>,
      <span class="pl-s"><span class="pl-pds">"</span>resource<span class="pl-pds">"</span></span>: [<span class="pl-s"><span class="pl-pds">"</span>mydatabase<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>mytable<span class="pl-pds">"</span></span>],
      <span class="pl-s"><span class="pl-pds">"</span>sql<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>select -1 from banned where user_id = :actor_id<span class="pl-pds">"</span></span>,
      <span class="pl-s"><span class="pl-pds">"</span>fallback<span class="pl-pds">"</span></span>: <span class="pl-c1">true</span>
  }</pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyNzE2NjUzMzY=
name: datasette-auth-tokens
full_name: simonw/datasette-auth-tokens
html_url: https://github.com/simonw/datasette-auth-tokens
description: Datasette plugin for authenticating access using API tokens
created_at: 2020-06-11T23:23:30Z
updated_at: 2021-10-15T00:52:53Z
pushed_at: 2021-10-15T00:54:20Z
language: Python
license: apache-2.0
topics: ["datasette", "datasette-io", "datasette-plugin"]
default_branch: main
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
visibility: public
readme:
  # datasette-auth-tokens

  [![PyPI](https://img.shields.io/pypi/v/datasette-auth-tokens.svg)](https://pypi.org/project/datasette-auth-tokens/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-auth-tokens?include_prereleases&label=changelog)](https://github.com/simonw/datasette-auth-tokens/releases)
  [![Tests](https://github.com/simonw/datasette-auth-tokens/workflows/Test/badge.svg)](https://github.com/simonw/datasette-auth-tokens/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-auth-tokens/blob/main/LICENSE)

  Datasette plugin for authenticating access using API tokens

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ pip install datasette-auth-tokens

  ## Hard-coded tokens

  Read about Datasette's [authentication and permissions system](https://datasette.readthedocs.io/en/latest/authentication.html).

  This plugin lets you configure secret API tokens which can be used to make authenticated requests to Datasette.

  First, create a random API token. A useful recipe for doing that is the following:

      $ python -c 'import secrets; print(secrets.token_hex(32))'
      5f9a486dd807de632200b17508c75002bb66ca6fde1993db1de6cbd446362589

  Decide on the actor that this token should represent, for example:

  ```json
  {
      "bot_id": "my-bot"
  }
  ```

  You can then use `"allow"` blocks to provide that token with permission to access specific actions. To enable access to a configured writable SQL query you could use this in your `metadata.json`:

  ```json
  {
      "plugins": {
          "datasette-auth-tokens": {
              "tokens": [
                  {
                      "token": {
                          "$env": "BOT_TOKEN"
                      },
                      "actor": {
                          "bot_id": "my-bot"
                      }
                  }
              ]
          }
      },
      "databases": {
          ":memory:": {
              "queries": {
                  "show_version": {
                      "sql": "select sqlite_version()",
                      "allow": {
                          "bot_id": "my-bot"
                      }
                  }
              }
          }
      }
  }
  ```
  This uses Datasette's [secret configuration values mechanism](https://datasette.readthedocs.io/en/stable/plugins.html#secret-configuration-values) to allow the secret token to be passed as an environment variable.

  Run Datasette like this:

      BOT_TOKEN="this-is-the-secret-token" \
          datasette -m metadata.json

  You can now run authenticated API queries like this:

      $ curl -H 'Authorization: Bearer this-is-the-secret-token' \
        'http://127.0.0.1:8001/:memory:/show_version.json?_shape=array'
      [{"sqlite_version()": "3.31.1"}]

  Additionally you can allow passing the token as a query string parameter, although that's disabled by default given the security implications of URLs with secret tokens included. This may be useful to easily allow embedding data between different services.

  Simply enable it using the `param` config value:

  ```json
  {
      "plugins": {
          "datasette-auth-tokens": {
              "tokens": [
                  {
                      "token": {
                          "$env": "BOT_TOKEN"
                      },
                      "actor": {
                          "bot_id": "my-bot"
                      },
                  }
              ],
              "param": "_auth_token"
          }
      },
      "databases": {
          ":memory:": {
              "queries": {
                  "show_version": {
                      "sql": "select sqlite_version()",
                      "allow": {
                          "bot_id": "my-bot"
                      }
                  }
              }
          }
      }
  }
  ```

  You can now run authenticated API queries like this:

      $ curl http://127.0.0.1:8001/:memory:/show_version.json?_shape=array&_auth_token=this-is-the-secret-token
      [{"sqlite_version()": "3.31.1"}]

  ## Tokens from your database

  As an alternative (or in addition) to the hard-coded list of tokens you can store tokens in a database table and configure the plugin to access them using a SQL query.

  Your query needs to take a `:token_id` parameter and return at least two columns: one called `token_secret` and one called `actor_*` - usually `actor_id`. Further `actor_` prefixed columns can be returned to provide more details for the authenticated actor.

  Here's a simple example of a configuration query:

  ```sql
  select actor_id, actor_name, token_secret from tokens where token_id = :token_id
  ```

  This can run against a table like this one:

  | token_id | token_secret | actor_id | actor_name |
  | -------- | ------------ | -------- | ---------- |
  | 1        | bd3c94f51fcd | 78       | Cleopaws   |
  | 2        | 86681b4d6f66 | 32       | Pancakes   |

  The tokens are formed as the token ID, then a hyphen, then the token secret. For example:

  - `1-bd3c94f51fcd`
  - `2-86681b4d6f66`

  The SQL query will be executed with the portion before the hyphen as the `:token_id` parameter.

  The `token_secret` value returned by the query will be compared to the portion of the token after the hyphen to check if the token is valid.

  Columns with a prefix of `actor_` will be used to populate the actor dictionary. In the above example, a token of `2-86681b4d6f66` will become an actor dictionary of `{"id": 32, "name": "Pancakes"}`.

  To configure this, use a `"query"` block in your plugin configuration like this:

  ```json
  {
      "plugins": {
          "datasette-auth-tokens": {
              "query": {
                  "sql": "select actor_id, actor_name, token_secret from tokens where token_id = :token_id",
                  "database": "tokens"
              }
          }
      },
      "databases": {
          "tokens": {
              "allow": {}
          }
      }
  }
  ```
  The `"sql"` key here contains the SQL query. The `"database"` key has the name of the attached database file that the query should be executed against - in this case it would execute against `tokens.db`.

  ### Securing your tokens

  Anyone with access to your Datasette instance can use it to read the `token_secret` column in your tokens table. This probably isn't what you want!

  To avoid this, you should lock down access to that table. The configuration example above shows how to do this using an `"allow": {}` block. Consult Datasette's [Permissions documentation](https://datasette.readthedocs.io/en/stable/authentication.html#permissions) for more information about how to lock down this kind of access.

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-datasette-auth-tokens" class="anchor" aria-hidden="true" href="#user-content-datasette-auth-tokens"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-auth-tokens</h1>
  <p><a href="https://pypi.org/project/datasette-auth-tokens/" rel="nofollow"><img src="https://camo.githubusercontent.com/8372a0259329c822611c82d155c9e1e29a243723aa17d0061b64bc06216c8e50/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d617574682d746f6b656e732e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-auth-tokens.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-auth-tokens/releases"><img src="https://camo.githubusercontent.com/5c2dba28aad8f4910893ff5a064ae0c826b40c16aae67c5573c17f60c3b5ff97/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d617574682d746f6b656e733f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-auth-tokens?include_prereleases&amp;label=changelog" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-auth-tokens/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-auth-tokens/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-auth-tokens/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width: 100%;"></a></p>
  <p>Datasette plugin for authenticating access using API tokens</p>
  <h2><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p>Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="$ pip install datasette-auth-tokens
  "><pre><code>$ pip install datasette-auth-tokens
  </code></pre></div>
  <h2><a id="user-content-hard-coded-tokens" class="anchor" aria-hidden="true" href="#user-content-hard-coded-tokens"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Hard-coded tokens</h2>
  <p>Read about Datasette's <a href="https://datasette.readthedocs.io/en/latest/authentication.html" rel="nofollow">authentication and permissions system</a>.</p>
  <p>This plugin lets you configure secret API tokens which can be used to make authenticated requests to Datasette.</p>
  <p>First, create a random API token. A useful recipe for doing that is the following:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="$ python -c 'import secrets; print(secrets.token_hex(32))'
  5f9a486dd807de632200b17508c75002bb66ca6fde1993db1de6cbd446362589
  "><pre><code>$ python -c 'import secrets; print(secrets.token_hex(32))'
  5f9a486dd807de632200b17508c75002bb66ca6fde1993db1de6cbd446362589
  </code></pre></div>
  <p>Decide on the actor that this token should represent, for example:</p>
  <div class="highlight highlight-source-json position-relative overflow-auto" data-snippet-clipboard-copy-content="{
      &quot;bot_id&quot;: &quot;my-bot&quot;
  }
  "><pre>{
      <span class="pl-s"><span class="pl-pds">"</span>bot_id<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>my-bot<span class="pl-pds">"</span></span>
  }</pre></div>
  <p>You can then use <code>"allow"</code> blocks to provide that token with permission to access specific actions. To enable access to a configured writable SQL query you could use this in your <code>metadata.json</code>:</p>
  <div class="highlight highlight-source-json position-relative overflow-auto" data-snippet-clipboard-copy-content="{
      &quot;plugins&quot;: {
          &quot;datasette-auth-tokens&quot;: {
              &quot;tokens&quot;: [
                  {
                      &quot;token&quot;: {
                          &quot;$env&quot;: &quot;BOT_TOKEN&quot;
                      },
                      &quot;actor&quot;: {
                          &quot;bot_id&quot;: &quot;my-bot&quot;
                      }
                  }
              ]
          }
      },
      &quot;databases&quot;: {
          &quot;:memory:&quot;: {
              &quot;queries&quot;: {
                  &quot;show_version&quot;: {
                      &quot;sql&quot;: &quot;select sqlite_version()&quot;,
                      &quot;allow&quot;: {
                          &quot;bot_id&quot;: &quot;my-bot&quot;
                      }
                  }
              }
          }
      }
  }
  "><pre>{
      <span class="pl-s"><span class="pl-pds">"</span>plugins<span class="pl-pds">"</span></span>: {
          <span class="pl-s"><span class="pl-pds">"</span>datasette-auth-tokens<span class="pl-pds">"</span></span>: {
              <span class="pl-s"><span class="pl-pds">"</span>tokens<span class="pl-pds">"</span></span>: [
                  {
                      <span class="pl-s"><span class="pl-pds">"</span>token<span class="pl-pds">"</span></span>: {
                          <span class="pl-s"><span class="pl-pds">"</span>$env<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>BOT_TOKEN<span class="pl-pds">"</span></span>
                      },
                      <span class="pl-s"><span class="pl-pds">"</span>actor<span class="pl-pds">"</span></span>: {
                          <span class="pl-s"><span class="pl-pds">"</span>bot_id<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>my-bot<span class="pl-pds">"</span></span>
                      }
                  }
              ]
          }
      },
      <span class="pl-s"><span class="pl-pds">"</span>databases<span class="pl-pds">"</span></span>: {
          <span class="pl-s"><span class="pl-pds">"</span>:memory:<span class="pl-pds">"</span></span>: {
              <span class="pl-s"><span class="pl-pds">"</span>queries<span class="pl-pds">"</span></span>: {
                  <span class="pl-s"><span class="pl-pds">"</span>show_version<span class="pl-pds">"</span></span>: {
                      <span class="pl-s"><span class="pl-pds">"</span>sql<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>select sqlite_version()<span class="pl-pds">"</span></span>,
                      <span class="pl-s"><span class="pl-pds">"</span>allow<span class="pl-pds">"</span></span>: {
                          <span class="pl-s"><span class="pl-pds">"</span>bot_id<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>my-bot<span class="pl-pds">"</span></span>
                      }
                  }
              }
          }
      }
  }</pre></div>
  <p>This uses Datasette's <a href="https://datasette.readthedocs.io/en/stable/plugins.html#secret-configuration-values" rel="nofollow">secret configuration values mechanism</a> to allow the secret token to be passed as an environment variable.</p>
  <p>Run Datasette like this:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="BOT_TOKEN=&quot;this-is-the-secret-token&quot; \
      datasette -m metadata.json
  "><pre><code>BOT_TOKEN="this-is-the-secret-token" \
      datasette -m metadata.json
  </code></pre></div>
  <p>You can now run authenticated API queries like this:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="$ curl -H 'Authorization: Bearer this-is-the-secret-token' \
    'http://127.0.0.1:8001/:memory:/show_version.json?_shape=array'
  [{&quot;sqlite_version()&quot;: &quot;3.31.1&quot;}]
  "><pre><code>$ curl -H 'Authorization: Bearer this-is-the-secret-token' \
    'http://127.0.0.1:8001/:memory:/show_version.json?_shape=array'
  [{"sqlite_version()": "3.31.1"}]
  </code></pre></div>
  <p>Additionally you can allow passing the token as a query string parameter, although that's disabled by default given the security implications of URLs with secret tokens included. This may be useful to easily allow embedding data between different services.</p>
  <p>Simply enable it using the <code>param</code> config value:</p>
  <div class="highlight highlight-source-json position-relative overflow-auto" data-snippet-clipboard-copy-content="{
      &quot;plugins&quot;: {
          &quot;datasette-auth-tokens&quot;: {
              &quot;tokens&quot;: [
                  {
                      &quot;token&quot;: {
                          &quot;$env&quot;: &quot;BOT_TOKEN&quot;
                      },
                      &quot;actor&quot;: {
                          &quot;bot_id&quot;: &quot;my-bot&quot;
                      },
                  }
              ],
              &quot;param&quot;: &quot;_auth_token&quot;
          }
      },
      &quot;databases&quot;: {
          &quot;:memory:&quot;: {
              &quot;queries&quot;: {
                  &quot;show_version&quot;: {
                      &quot;sql&quot;: &quot;select sqlite_version()&quot;,
                      &quot;allow&quot;: {
                          &quot;bot_id&quot;: &quot;my-bot&quot;
                      }
                  }
              }
          }
      }
  }
  "><pre>{
      <span class="pl-s"><span class="pl-pds">"</span>plugins<span class="pl-pds">"</span></span>: {
          <span class="pl-s"><span class="pl-pds">"</span>datasette-auth-tokens<span class="pl-pds">"</span></span>: {
              <span class="pl-s"><span class="pl-pds">"</span>tokens<span class="pl-pds">"</span></span>: [
                  {
                      <span class="pl-s"><span class="pl-pds">"</span>token<span class="pl-pds">"</span></span>: {
                          <span class="pl-s"><span class="pl-pds">"</span>$env<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>BOT_TOKEN<span class="pl-pds">"</span></span>
                      },
                      <span class="pl-s"><span class="pl-pds">"</span>actor<span class="pl-pds">"</span></span>: {
                          <span class="pl-s"><span class="pl-pds">"</span>bot_id<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>my-bot<span class="pl-pds">"</span></span>
                      },
                  }
              ],
              <span class="pl-s"><span class="pl-pds">"</span>param<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>_auth_token<span class="pl-pds">"</span></span>
          }
      },
      <span class="pl-s"><span class="pl-pds">"</span>databases<span class="pl-pds">"</span></span>: {
          <span class="pl-s"><span class="pl-pds">"</span>:memory:<span class="pl-pds">"</span></span>: {
              <span class="pl-s"><span class="pl-pds">"</span>queries<span class="pl-pds">"</span></span>: {
                  <span class="pl-s"><span class="pl-pds">"</span>show_version<span class="pl-pds">"</span></span>: {
                      <span class="pl-s"><span class="pl-pds">"</span>sql<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>select sqlite_version()<span class="pl-pds">"</span></span>,
                      <span class="pl-s"><span class="pl-pds">"</span>allow<span class="pl-pds">"</span></span>: {
                          <span class="pl-s"><span class="pl-pds">"</span>bot_id<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>my-bot<span class="pl-pds">"</span></span>
                      }
                  }
              }
          }
      }
  }</pre></div>
  <p>You can now run authenticated API queries like this:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="$ curl http://127.0.0.1:8001/:memory:/show_version.json?_shape=array&amp;_auth_token=this-is-the-secret-token
  [{&quot;sqlite_version()&quot;: &quot;3.31.1&quot;}]
  "><pre><code>$ curl http://127.0.0.1:8001/:memory:/show_version.json?_shape=array&amp;_auth_token=this-is-the-secret-token
  [{"sqlite_version()": "3.31.1"}]
  </code></pre></div>
  <h2><a id="user-content-tokens-from-your-database" class="anchor" aria-hidden="true" href="#user-content-tokens-from-your-database"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Tokens from your database</h2>
  <p>As an alternative (or in addition) to the hard-coded list of tokens you can store tokens in a database table and configure the plugin to access them using a SQL query.</p>
  <p>Your query needs to take a <code>:token_id</code> parameter and return at least two columns: one called <code>token_secret</code> and one called <code>actor_*</code> - usually <code>actor_id</code>. Further <code>actor_</code> prefixed columns can be returned to provide more details for the authenticated actor.</p>
  <p>Here's a simple example of a configuration query:</p>
  <div class="highlight highlight-source-sql position-relative overflow-auto" data-snippet-clipboard-copy-content="select actor_id, actor_name, token_secret from tokens where token_id = :token_id
  "><pre><span class="pl-k">select</span> actor_id, actor_name, token_secret <span class="pl-k">from</span> tokens <span class="pl-k">where</span> token_id <span class="pl-k">=</span> :token_id</pre></div>
  <p>This can run against a table like this one:</p>
  <table>
  <thead>
  <tr>
  <th>token_id</th>
  <th>token_secret</th>
  <th>actor_id</th>
  <th>actor_name</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td>1</td>
  <td>bd3c94f51fcd</td>
  <td>78</td>
  <td>Cleopaws</td>
  </tr>
  <tr>
  <td>2</td>
  <td>86681b4d6f66</td>
  <td>32</td>
  <td>Pancakes</td>
  </tr>
  </tbody>
  </table>
  <p>The tokens are formed as the token ID, then a hyphen, then the token secret. For example:</p>
  <ul>
  <li><code>1-bd3c94f51fcd</code></li>
  <li><code>2-86681b4d6f66</code></li>
  </ul>
  <p>The SQL query will be executed with the portion before the hyphen as the <code>:token_id</code> parameter.</p>
  <p>The <code>token_secret</code> value returned by the query will be compared to the portion of the token after the hyphen to check if the token is valid.</p>
  <p>Columns with a prefix of <code>actor_</code> will be used to populate the actor dictionary. In the above example, a token of <code>2-86681b4d6f66</code> will become an actor dictionary of <code>{"id": 32, "name": "Pancakes"}</code>.</p>
  <p>To configure this, use a <code>"query"</code> block in your plugin configuration like this:</p>
  <div class="highlight highlight-source-json position-relative overflow-auto" data-snippet-clipboard-copy-content="{
      &quot;plugins&quot;: {
          &quot;datasette-auth-tokens&quot;: {
              &quot;query&quot;: {
                  &quot;sql&quot;: &quot;select actor_id, actor_name, token_secret from tokens where token_id = :token_id&quot;,
                  &quot;database&quot;: &quot;tokens&quot;
              }
          }
      },
      &quot;databases&quot;: {
          &quot;tokens&quot;: {
              &quot;allow&quot;: {}
          }
      }
  }
  "><pre>{
      <span class="pl-s"><span class="pl-pds">"</span>plugins<span class="pl-pds">"</span></span>: {
          <span class="pl-s"><span class="pl-pds">"</span>datasette-auth-tokens<span class="pl-pds">"</span></span>: {
              <span class="pl-s"><span class="pl-pds">"</span>query<span class="pl-pds">"</span></span>: {
                  <span class="pl-s"><span class="pl-pds">"</span>sql<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>select actor_id, actor_name, token_secret from tokens where token_id = :token_id<span class="pl-pds">"</span></span>,
                  <span class="pl-s"><span class="pl-pds">"</span>database<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>tokens<span class="pl-pds">"</span></span>
              }
          }
      },
      <span class="pl-s"><span class="pl-pds">"</span>databases<span class="pl-pds">"</span></span>: {
          <span class="pl-s"><span class="pl-pds">"</span>tokens<span class="pl-pds">"</span></span>: {
              <span class="pl-s"><span class="pl-pds">"</span>allow<span class="pl-pds">"</span></span>: {}
          }
      }
  }</pre></div>
  <p>The <code>"sql"</code> key here contains the SQL query. The <code>"database"</code> key has the name of the attached database file that the query should be executed against - in this case it would execute against <code>tokens.db</code>.</p>
  <h3><a id="user-content-securing-your-tokens" class="anchor" aria-hidden="true" href="#user-content-securing-your-tokens"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Securing your tokens</h3>
  <p>Anyone with access to your Datasette instance can use it to read the <code>token_secret</code> column in your tokens table. This probably isn't what you want!</p>
  <p>To avoid this, you should lock down access to that table. The configuration example above shows how to do this using an <code>"allow": {}</code> block. Consult Datasette's <a href="https://datasette.readthedocs.io/en/stable/authentication.html#permissions" rel="nofollow">Permissions documentation</a> for more information about how to lock down this kind of access.</p>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyNzIwOTg0ODY=
name: datasette-psutil
full_name: simonw/datasette-psutil
html_url: https://github.com/simonw/datasette-psutil
description: Datasette plugin adding a /-/psutil debugging endpoint
created_at: 2020-06-13T22:57:07Z
updated_at: 2022-03-07T15:36:30Z
pushed_at: 2022-03-07T15:35:57Z
homepage: https://datasette.io/plugins/datasette-psutil
language: Python
license: apache-2.0
topics: ["datasette", "datasette-io", "datasette-plugin", "psutil"]
default_branch: main
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
visibility: public
readme:
  # datasette-psutil

  [![PyPI](https://img.shields.io/pypi/v/datasette-psutil.svg)](https://pypi.org/project/datasette-psutil/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-psutil?include_prereleases&label=changelog)](https://github.com/simonw/datasette-psutil/releases)
  [![Tests](https://github.com/simonw/datasette-psutil/workflows/Test/badge.svg)](https://github.com/simonw/datasette-psutil/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-psutil/blob/main/LICENSE)

  Datasette plugin adding a `/-/psutil` debugging endpoint

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ pip install datasette-psutil

  ## Usage

  Visit `/-/psutil` on your Datasette instance to see various information provided by [psutil](https://psutil.readthedocs.io/).

  ## Demo

  https://latest-with-plugins.datasette.io/-/psutil is a live demo of this plugin, hosted on Google Cloud Run.

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-datasette-psutil" class="anchor" aria-hidden="true" href="#user-content-datasette-psutil"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-psutil</h1>
  <p dir="auto"><a href="https://pypi.org/project/datasette-psutil/" rel="nofollow"><img src="https://camo.githubusercontent.com/be1734fd4b22f0108384342af796a3468669cbb4d82f0e79b28aeb60f5634904/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d70737574696c2e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-psutil.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-psutil/releases"><img src="https://camo.githubusercontent.com/8c025e6a2847072141ce3a8022ce39878eb8adc71fdeed5dbf93a6fa755f3785/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d70737574696c3f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-psutil?include_prereleases&amp;label=changelog" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-psutil/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-psutil/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-psutil/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width: 100%;"></a></p>
  <p dir="auto">Datasette plugin adding a <code>/-/psutil</code> debugging endpoint</p>
  <h2 dir="auto"><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p dir="auto">Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="$ pip install datasette-psutil"><pre><code>$ pip install datasette-psutil
  </code></pre></div>
  <h2 dir="auto"><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p dir="auto">Visit <code>/-/psutil</code> on your Datasette instance to see various information provided by <a href="https://psutil.readthedocs.io/" rel="nofollow">psutil</a>.</p>
  <h2 dir="auto"><a id="user-content-demo" class="anchor" aria-hidden="true" href="#user-content-demo"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Demo</h2>
  <p dir="auto"><a href="https://latest-with-plugins.datasette.io/-/psutil" rel="nofollow">https://latest-with-plugins.datasette.io/-/psutil</a> is a live demo of this plugin, hosted on Google Cloud Run.</p>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyNzM2MDk4Nzk=
name: datasette-saved-queries
full_name: simonw/datasette-saved-queries
html_url: https://github.com/simonw/datasette-saved-queries
description: Datasette plugin that lets users save and execute queries
created_at: 2020-06-20T00:20:42Z
updated_at: 2020-09-24T05:08:37Z
pushed_at: 2020-08-15T23:38:46Z
language: Python
topics: ["datasette", "datasette-plugin", "datasette-io"]
default_branch: main
permissions: {"admin": false, "push": false, "pull": false}
readme:
  # datasette-saved-queries

  [![PyPI](https://img.shields.io/pypi/v/datasette-saved-queries.svg)](https://pypi.org/project/datasette-saved-queries/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-saved-queries?label=changelog)](https://github.com/simonw/datasette-saved-queries/releases)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-saved-queries/blob/master/LICENSE)

  Datasette plugin that lets users save and execute queries

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ pip install datasette-saved-queries

  ## Usage

  When the plugin is installed Datasette will automatically create a `saved_queries` table in the first connected database when it starts up.

  It also creates a `save_query` writable canned query which you can use to save new queries.

  Queries that you save will be added to the query list on the database page.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-saved-queries
      python -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-datasette-saved-queries" class="anchor" aria-hidden="true" href="#user-content-datasette-saved-queries"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-saved-queries</h1>
  <p><a href="https://pypi.org/project/datasette-saved-queries/" rel="nofollow"><img src="https://camo.githubusercontent.com/1be37b6bd17b348b570a8c0e73486d46c1b1a3a2f3f0e88d360ea41dd85b6cdb/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d73617665642d717565726965732e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-saved-queries.svg" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-saved-queries/releases"><img src="https://camo.githubusercontent.com/5e0c781e8fb49e961b9a45d559f6d6e77aa1269954ba59f648e7855e44dd4fe8/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d73617665642d717565726965733f6c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-saved-queries?label=changelog" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-saved-queries/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p>
  <p>Datasette plugin that lets users save and execute queries</p>
  <h2><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p>Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ pip install datasette-saved-queries
  "><pre><code>$ pip install datasette-saved-queries
  </code></pre></div>
  <h2><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p>When the plugin is installed Datasette will automatically create a <code>saved_queries</code> table in the first connected database when it starts up.</p>
  <p>It also creates a <code>save_query</code> writable canned query which you can use to save new queries.</p>
  <p>Queries that you save will be added to the query list on the database page.</p>
  <h2><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p>To set up this plugin locally, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="cd datasette-saved-queries
  python -mvenv venv
  source venv/bin/activate
  "><pre><code>cd datasette-saved-queries
  python -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p>Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pipenv shell
  "><pre><code>pipenv shell
  </code></pre></div>
  <p>Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pip install -e '.[test]'
  "><pre><code>pip install -e '.[test]'
  </code></pre></div>
  <p>To run the tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pytest
  "><pre><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyNzQyNjQ0ODQ=
name: sqlite-generate
full_name: simonw/sqlite-generate
html_url: https://github.com/simonw/sqlite-generate
description: Tool for generating demo SQLite databases
created_at: 2020-06-22T23:36:44Z
updated_at: 2021-02-27T15:25:26Z
pushed_at: 2021-02-27T15:25:24Z
homepage: https://sqlite-generate-demo.datasette.io/
language: Python
license: apache-2.0
topics: ["sqlite", "datasette-io", "datasette-tool"]
default_branch: main
permissions: {"admin": false, "push": false, "pull": false}
readme:
  # sqlite-generate

  [![PyPI](https://img.shields.io/pypi/v/sqlite-generate.svg)](https://pypi.org/project/sqlite-generate/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/sqlite-generate?label=changelog)](https://github.com/simonw/sqlite-generate/releases)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/sqlite-generate/blob/master/LICENSE)

  Tool for generating demo SQLite databases

  ## Installation

  Install this plugin using `pip`:

      $ pip install sqlite-generate

  ## Demo

  You can see a demo of the database generated using this command running in [Datasette](https://github.com/simonw/datasette) at https://sqlite-generate-demo.datasette.io/

  The demo is generated using the following command:

      sqlite-generate demo.db --seed seed --fts --columns=10 --fks=0,3 --pks=0,2

  ## Usage

  To generate a SQLite database file called `data.db` with 10 randomly named tables in it, run the following:

      sqlite-generate data.db

  You can use the `--tables` option to generate a different number of tables:

      sqlite-generate data.db --tables 20

  You can run the command against the same database file multiple times to keep adding new tables, using different settings for each batch of generated tables.

  By default each table will contain a random number of rows between 0 and 200. You can customize this with the `--rows` option:

      sqlite-generate data.db --rows 20

  This will insert 20 rows into each table.

      sqlite-generate data.db --rows 500,2000

  This inserts a random number of rows between 500 and 2000 into each table.

  Each table will have 5 columns. You can change this using `--columns`:

      sqlite-generate data.db --columns 10

  `--columns` can also accept a range:

      sqlite-generate data.db --columns 5,15

  You can control the random number seed used with the `--seed` option. This will result in the exact same database file being created by multiple runs of the tool:

      sqlite-generate data.db --seed=myseed

  By default each table will contain between 0 and 2 foreign key columns to other tables. You can control this using the `--fks` option, with either a single number or a range:

      sqlite-generate data.db --columns=20 --fks=5,15

  Each table will have a single primary key column called `id`. You can use the `--pks=` option to change the number of primary key columns on each table. Drop it to 0 to generate [rowid tables](https://www.sqlite.org/rowidtable.html). Increase it above 1 to generate tables with compound primary keys. Or use a range to get a random selection of different primary key layouts:

      sqlite-generate data.db --pks=0,2

  To configure [SQLite full-text search](https://www.sqlite.org/fts5.html) for all columns of type text, use `--fts`:

      sqlite-generate data.db --fts

  This will use FTS5 by default. To use [FTS4](https://www.sqlite.org/fts3.html) instead, use `--fts4`.

  ## Development

  To contribute to this tool, first checkout the code. Then create a new virtual environment:

      cd sqlite-generate
      python -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-sqlite-generate" class="anchor" aria-hidden="true" href="#user-content-sqlite-generate"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>sqlite-generate</h1>
  <p><a href="https://pypi.org/project/sqlite-generate/" rel="nofollow"><img src="https://camo.githubusercontent.com/f0bc12d9a036f8faadbe40bbc37caa416eadf33d6694322ca385c16f2302b575/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f73716c6974652d67656e65726174652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/sqlite-generate.svg" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/sqlite-generate/releases"><img src="https://camo.githubusercontent.com/8acbae82ad62477a2630aee86e3c2c6c498a06acc6fd6d6c08e51779ea3905de/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f73716c6974652d67656e65726174653f6c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/sqlite-generate?label=changelog" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/sqlite-generate/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p>
  <p>Tool for generating demo SQLite databases</p>
  <h2><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p>Install this plugin using <code>pip</code>:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ pip install sqlite-generate
  "><pre><code>$ pip install sqlite-generate
  </code></pre></div>
  <h2><a id="user-content-demo" class="anchor" aria-hidden="true" href="#user-content-demo"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Demo</h2>
  <p>You can see a demo of the database generated using this command running in <a href="https://github.com/simonw/datasette">Datasette</a> at <a href="https://sqlite-generate-demo.datasette.io/" rel="nofollow">https://sqlite-generate-demo.datasette.io/</a></p>
  <p>The demo is generated using the following command:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="sqlite-generate demo.db --seed seed --fts --columns=10 --fks=0,3 --pks=0,2
  "><pre><code>sqlite-generate demo.db --seed seed --fts --columns=10 --fks=0,3 --pks=0,2
  </code></pre></div>
  <h2><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p>To generate a SQLite database file called <code>data.db</code> with 10 randomly named tables in it, run the following:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="sqlite-generate data.db
  "><pre><code>sqlite-generate data.db
  </code></pre></div>
  <p>You can use the <code>--tables</code> option to generate a different number of tables:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="sqlite-generate data.db --tables 20
  "><pre><code>sqlite-generate data.db --tables 20
  </code></pre></div>
  <p>You can run the command against the same database file multiple times to keep adding new tables, using different settings for each batch of generated tables.</p>
  <p>By default each table will contain a random number of rows between 0 and 200. You can customize this with the <code>--rows</code> option:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="sqlite-generate data.db --rows 20
  "><pre><code>sqlite-generate data.db --rows 20
  </code></pre></div>
  <p>This will insert 20 rows into each table.</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="sqlite-generate data.db --rows 500,2000
  "><pre><code>sqlite-generate data.db --rows 500,2000
  </code></pre></div>
  <p>This inserts a random number of rows between 500 and 2000 into each table.</p>
  <p>Each table will have 5 columns. You can change this using <code>--columns</code>:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="sqlite-generate data.db --columns 10
  "><pre><code>sqlite-generate data.db --columns 10
  </code></pre></div>
  <p><code>--columns</code> can also accept a range:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="sqlite-generate data.db --columns 5,15
  "><pre><code>sqlite-generate data.db --columns 5,15
  </code></pre></div>
  <p>You can control the random number seed used with the <code>--seed</code> option. This will result in the exact same database file being created by multiple runs of the tool:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="sqlite-generate data.db --seed=myseed
  "><pre><code>sqlite-generate data.db --seed=myseed
  </code></pre></div>
  <p>By default each table will contain between 0 and 2 foreign key columns to other tables. You can control this using the <code>--fks</code> option, with either a single number or a range:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="sqlite-generate data.db --columns=20 --fks=5,15
  "><pre><code>sqlite-generate data.db --columns=20 --fks=5,15
  </code></pre></div>
  <p>Each table will have a single primary key column called <code>id</code>. You can use the <code>--pks=</code> option to change the number of primary key columns on each table. Drop it to 0 to generate <a href="https://www.sqlite.org/rowidtable.html" rel="nofollow">rowid tables</a>. Increase it above 1 to generate tables with compound primary keys. Or use a range to get a random selection of different primary key layouts:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="sqlite-generate data.db --pks=0,2
  "><pre><code>sqlite-generate data.db --pks=0,2
  </code></pre></div>
  <p>To configure <a href="https://www.sqlite.org/fts5.html" rel="nofollow">SQLite full-text search</a> for all columns of type text, use <code>--fts</code>:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="sqlite-generate data.db --fts
  "><pre><code>sqlite-generate data.db --fts
  </code></pre></div>
  <p>This will use FTS5 by default. To use <a href="https://www.sqlite.org/fts3.html" rel="nofollow">FTS4</a> instead, use <code>--fts4</code>.</p>
  <h2><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p>To contribute to this tool, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="cd sqlite-generate
  python -mvenv venv
  source venv/bin/activate
  "><pre><code>cd sqlite-generate
  python -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p>Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pipenv shell
  "><pre><code>pipenv shell
  </code></pre></div>
  <p>Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pip install -e '.[test]'
  "><pre><code>pip install -e '.[test]'
  </code></pre></div>
  <p>To run the tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pytest
  "><pre><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyNzQyOTM1OTc=
name: datasette-block-robots
full_name: simonw/datasette-block-robots
html_url: https://github.com/simonw/datasette-block-robots
description: Datasette plugin that blocks robots and crawlers using robots.txt
created_at: 2020-06-23T02:52:23Z
updated_at: 2022-08-30T16:13:40Z
pushed_at: 2022-08-30T16:25:38Z
homepage: https://datasette.io/plugins/datasette-block-robots
language: Python
topics: ["datasette", "datasette-io", "datasette-plugin", "robots-txt"]
default_branch: main
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
visibility: public
readme:
  # datasette-block-robots

  [![PyPI](https://img.shields.io/pypi/v/datasette-block-robots.svg)](https://pypi.org/project/datasette-block-robots/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-block-robots?label=changelog)](https://github.com/simonw/datasette-block-robots/releases)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-block-robots/blob/master/LICENSE)

  Datasette plugin that blocks robots and crawlers using robots.txt

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ pip install datasette-block-robots

  ## Usage

  Having installed the plugin, `/robots.txt` on your Datasette instance will return the following:

      User-agent: *
      Disallow: /

  This will request all robots and crawlers not to visit any of the pages on your site.

  Here's a demo of the plugin in action: https://sqlite-generate-demo.datasette.io/robots.txt

  ## Configuration

  By default the plugin will block all access to the site, using `Disallow: /`.

  If you want the index page to be indexed by search engines without crawling the database, table or row pages themselves, you can use the following:

  ```json
  {
      "plugins": {
          "datasette-block-robots": {
              "allow_only_index": true
          }
      }
  }
  ```
  This will return a `/robots.txt` like so:

      User-agent: *
      Disallow: /db1
      Disallow: /db2

  With a `Disallow` line for every attached database.

  To block access to specific areas of the site using custom paths, add this to your `metadata.json` configuration file:

  ```json
  {
      "plugins": {
          "datasette-block-robots": {
              "disallow": ["/mydatabase/mytable"]
          }
      }
  }
  ```
  This will result in a `/robots.txt` that looks like this:

      User-agent: *
      Disallow: /mydatabase/mytable

  Alternatively you can set the full contents of the `robots.txt` file using the `literal` configuration option. Here's how to do that if you are using YAML rather than JSON and have a `metadata.yml` file:

  ```yaml
  plugins:
      datasette-block-robots:
          literal: |-
              User-agent: *
              Disallow: /
              User-agent: Bingbot
              User-agent: Googlebot
              Disallow:
  ```
  This example would block all crawlers with the exception of Googlebot and Bingbot, which are allowed to crawl the entire site.

  ## Extending this with other plugins

  This plugin adds a new [plugin hook](https://docs.datasette.io/en/stable/plugin_hooks.html) to Datasete called `block_robots_extra_lines()` which can be used by other plugins to add their own additional lines to the `robots.txt` file.

  The hook can optionally accept these parameters:

  - `datasette`: The current [Datasette instance](https://docs.datasette.io/en/stable/internals.html#datasette-class). You can use this to execute SQL queries or read plugin configuration settings.
  - `request`: The [Request object](https://docs.datasette.io/en/stable/internals.html#request-object) representing the incoming request to `/robots.txt`.

  The hook should return a list of strings, each representing a line to be added to the `robots.txt` file.

  It can also return an `async def` function, which will be awaited and used to generate a list of lines. Use this option if you need to make `await` calls inside you hook implementation.

  This example uses the hook to add a `Sitemap: http://example.com/sitemap.xml` line to the `robots.txt` file:

  ```python
  from datasette import hookimpl

  @hookimpl
  def block_robots_extra_lines(datasette, request):
      return [
          "Sitemap: {}".format(datasette.absolute_url(request, "/sitemap.xml")),
      ]
  ```
  This example blocks access to paths based on a database query:

  ```python
  @hookimpl
  def block_robots_extra_lines(datasette):
      async def inner():
          db = datasette.get_database()
          result = await db.execute("select path from mytable")
          return [
              "Disallow: /{}".format(row["path"]) for row in result
          ]
      return inner
  ```
  [datasette-sitemap](https://datasette.io/plugins/datasette-sitemap) is an example of a plugin that uses this hook.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-block-robots
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-datasette-block-robots" class="anchor" aria-hidden="true" href="#user-content-datasette-block-robots"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-block-robots</h1>
  <p dir="auto"><a href="https://pypi.org/project/datasette-block-robots/" rel="nofollow"><img src="https://camo.githubusercontent.com/f4e772c8056a8f8ea71ddee7622d691ff2292eff02e39a1937dd0acb53ba13f5/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d626c6f636b2d726f626f74732e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-block-robots.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-block-robots/releases"><img src="https://camo.githubusercontent.com/bc5b8190e1a2e9d22d6eb0392d076535f3bb7ddf1b4e6f12dc0a2f5a607c948f/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d626c6f636b2d726f626f74733f6c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-block-robots?label=changelog" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-block-robots/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width: 100%;"></a></p>
  <p dir="auto">Datasette plugin that blocks robots and crawlers using robots.txt</p>
  <h2 dir="auto"><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p dir="auto">Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="$ pip install datasette-block-robots"><pre class="notranslate"><code>$ pip install datasette-block-robots
  </code></pre></div>
  <h2 dir="auto"><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p dir="auto">Having installed the plugin, <code>/robots.txt</code> on your Datasette instance will return the following:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="User-agent: *
  Disallow: /"><pre class="notranslate"><code>User-agent: *
  Disallow: /
  </code></pre></div>
  <p dir="auto">This will request all robots and crawlers not to visit any of the pages on your site.</p>
  <p dir="auto">Here's a demo of the plugin in action: <a href="https://sqlite-generate-demo.datasette.io/robots.txt" rel="nofollow">https://sqlite-generate-demo.datasette.io/robots.txt</a></p>
  <h2 dir="auto"><a id="user-content-configuration" class="anchor" aria-hidden="true" href="#user-content-configuration"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Configuration</h2>
  <p dir="auto">By default the plugin will block all access to the site, using <code>Disallow: /</code>.</p>
  <p dir="auto">If you want the index page to be indexed by search engines without crawling the database, table or row pages themselves, you can use the following:</p>
  <div class="highlight highlight-source-json notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="{
      &quot;plugins&quot;: {
          &quot;datasette-block-robots&quot;: {
              &quot;allow_only_index&quot;: true
          }
      }
  }"><pre>{
      <span class="pl-ent">"plugins"</span>: {
          <span class="pl-ent">"datasette-block-robots"</span>: {
              <span class="pl-ent">"allow_only_index"</span>: <span class="pl-c1">true</span>
          }
      }
  }</pre></div>
  <p dir="auto">This will return a <code>/robots.txt</code> like so:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="User-agent: *
  Disallow: /db1
  Disallow: /db2"><pre class="notranslate"><code>User-agent: *
  Disallow: /db1
  Disallow: /db2
  </code></pre></div>
  <p dir="auto">With a <code>Disallow</code> line for every attached database.</p>
  <p dir="auto">To block access to specific areas of the site using custom paths, add this to your <code>metadata.json</code> configuration file:</p>
  <div class="highlight highlight-source-json notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="{
      &quot;plugins&quot;: {
          &quot;datasette-block-robots&quot;: {
              &quot;disallow&quot;: [&quot;/mydatabase/mytable&quot;]
          }
      }
  }"><pre>{
      <span class="pl-ent">"plugins"</span>: {
          <span class="pl-ent">"datasette-block-robots"</span>: {
              <span class="pl-ent">"disallow"</span>: [<span class="pl-s"><span class="pl-pds">"</span>/mydatabase/mytable<span class="pl-pds">"</span></span>]
          }
      }
  }</pre></div>
  <p dir="auto">This will result in a <code>/robots.txt</code> that looks like this:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="User-agent: *
  Disallow: /mydatabase/mytable"><pre class="notranslate"><code>User-agent: *
  Disallow: /mydatabase/mytable
  </code></pre></div>
  <p dir="auto">Alternatively you can set the full contents of the <code>robots.txt</code> file using the <code>literal</code> configuration option. Here's how to do that if you are using YAML rather than JSON and have a <code>metadata.yml</code> file:</p>
  <div class="highlight highlight-source-yaml notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="plugins:
      datasette-block-robots:
          literal: |-
              User-agent: *
              Disallow: /
              User-agent: Bingbot
              User-agent: Googlebot
              Disallow:"><pre><span class="pl-ent">plugins</span>:
      <span class="pl-ent">datasette-block-robots</span>:
          <span class="pl-ent">literal</span>: <span class="pl-s">|-</span>
  <span class="pl-s">            User-agent: *</span>
  <span class="pl-s">            Disallow: /</span>
  <span class="pl-s">            User-agent: Bingbot</span>
  <span class="pl-s">            User-agent: Googlebot</span>
  <span class="pl-s">            Disallow:</span></pre></div>
  <p dir="auto">This example would block all crawlers with the exception of Googlebot and Bingbot, which are allowed to crawl the entire site.</p>
  <h2 dir="auto"><a id="user-content-extending-this-with-other-plugins" class="anchor" aria-hidden="true" href="#user-content-extending-this-with-other-plugins"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Extending this with other plugins</h2>
  <p dir="auto">This plugin adds a new <a href="https://docs.datasette.io/en/stable/plugin_hooks.html" rel="nofollow">plugin hook</a> to Datasete called <code>block_robots_extra_lines()</code> which can be used by other plugins to add their own additional lines to the <code>robots.txt</code> file.</p>
  <p dir="auto">The hook can optionally accept these parameters:</p>
  <ul dir="auto">
  <li><code>datasette</code>: The current <a href="https://docs.datasette.io/en/stable/internals.html#datasette-class" rel="nofollow">Datasette instance</a>. You can use this to execute SQL queries or read plugin configuration settings.</li>
  <li><code>request</code>: The <a href="https://docs.datasette.io/en/stable/internals.html#request-object" rel="nofollow">Request object</a> representing the incoming request to <code>/robots.txt</code>.</li>
  </ul>
  <p dir="auto">The hook should return a list of strings, each representing a line to be added to the <code>robots.txt</code> file.</p>
  <p dir="auto">It can also return an <code>async def</code> function, which will be awaited and used to generate a list of lines. Use this option if you need to make <code>await</code> calls inside you hook implementation.</p>
  <p dir="auto">This example uses the hook to add a <code>Sitemap: http://example.com/sitemap.xml</code> line to the <code>robots.txt</code> file:</p>
  <div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="from datasette import hookimpl

  @hookimpl
  def block_robots_extra_lines(datasette, request):
      return [
          &quot;Sitemap: {}&quot;.format(datasette.absolute_url(request, &quot;/sitemap.xml&quot;)),
      ]"><pre><span class="pl-k">from</span> <span class="pl-s1">datasette</span> <span class="pl-k">import</span> <span class="pl-s1">hookimpl</span>

  <span class="pl-en">@<span class="pl-s1">hookimpl</span></span>
  <span class="pl-k">def</span> <span class="pl-en">block_robots_extra_lines</span>(<span class="pl-s1">datasette</span>, <span class="pl-s1">request</span>):
      <span class="pl-k">return</span> [
          <span class="pl-s">"Sitemap: {}"</span>.<span class="pl-en">format</span>(<span class="pl-s1">datasette</span>.<span class="pl-en">absolute_url</span>(<span class="pl-s1">request</span>, <span class="pl-s">"/sitemap.xml"</span>)),
      ]</pre></div>
  <p dir="auto">This example blocks access to paths based on a database query:</p>
  <div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="@hookimpl
  def block_robots_extra_lines(datasette):
      async def inner():
          db = datasette.get_database()
          result = await db.execute(&quot;select path from mytable&quot;)
          return [
              &quot;Disallow: /{}&quot;.format(row[&quot;path&quot;]) for row in result
          ]
      return inner"><pre><span class="pl-en">@<span class="pl-s1">hookimpl</span></span>
  <span class="pl-k">def</span> <span class="pl-en">block_robots_extra_lines</span>(<span class="pl-s1">datasette</span>):
      <span class="pl-k">async</span> <span class="pl-k">def</span> <span class="pl-en">inner</span>():
          <span class="pl-s1">db</span> <span class="pl-c1">=</span> <span class="pl-s1">datasette</span>.<span class="pl-en">get_database</span>()
          <span class="pl-s1">result</span> <span class="pl-c1">=</span> <span class="pl-k">await</span> <span class="pl-s1">db</span>.<span class="pl-en">execute</span>(<span class="pl-s">"select path from mytable"</span>)
          <span class="pl-k">return</span> [
              <span class="pl-s">"Disallow: /{}"</span>.<span class="pl-en">format</span>(<span class="pl-s1">row</span>[<span class="pl-s">"path"</span>]) <span class="pl-k">for</span> <span class="pl-s1">row</span> <span class="pl-c1">in</span> <span class="pl-s1">result</span>
          ]
      <span class="pl-k">return</span> <span class="pl-s1">inner</span></pre></div>
  <p dir="auto"><a href="https://datasette.io/plugins/datasette-sitemap" rel="nofollow">datasette-sitemap</a> is an example of a plugin that uses this hook.</p>
  <h2 dir="auto"><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p dir="auto">To set up this plugin locally, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="cd datasette-block-robots
  python3 -mvenv venv
  source venv/bin/activate"><pre class="notranslate"><code>cd datasette-block-robots
  python3 -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p dir="auto">Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="pipenv shell"><pre class="notranslate"><code>pipenv shell
  </code></pre></div>
  <p dir="auto">Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="pip install -e '.[test]'"><pre class="notranslate"><code>pip install -e '.[test]'
  </code></pre></div>
  <p dir="auto">To run the tests:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="pytest"><pre class="notranslate"><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyNzU2MTU5NDc=
name: datasette-glitch
full_name: simonw/datasette-glitch
html_url: https://github.com/simonw/datasette-glitch
description: Utilities to help run Datasette on Glitch
created_at: 2020-06-28T15:41:25Z
updated_at: 2020-07-01T22:48:35Z
pushed_at: 2020-07-01T22:49:22Z
language: Python
topics: ["glitch", "datasette", "datasette-plugin", "datasette-io"]
default_branch: main
permissions: {"admin": false, "push": false, "pull": false}
readme:
  # datasette-glitch

  [![PyPI](https://img.shields.io/pypi/v/datasette-glitch.svg)](https://pypi.org/project/datasette-glitch/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-glitch?label=changelog)](https://github.com/simonw/datasette-glitch/releases)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-glitch/blob/master/LICENSE)

  Utilities to help run Datasette on Glitch

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ pip install datasette-glitch

  ## Usage

  This plugin outputs a special link which will sign you into Datasette as the root user.

  Click Tools -> Logs in the Glitch editor interface after your app starts to see the link.

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-datasette-glitch" class="anchor" aria-hidden="true" href="#user-content-datasette-glitch"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-glitch</h1>
  <p><a href="https://pypi.org/project/datasette-glitch/" rel="nofollow"><img src="https://camo.githubusercontent.com/da8d1e55f6fa2edbfe632b0cb5b8b97a65d75b3e74cfe2c32992720b136f7fe3/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d676c697463682e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-glitch.svg" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-glitch/releases"><img src="https://camo.githubusercontent.com/8609928bfae188a41e14d2e647a1805211a59fce8f113e4f3bc7a5e8dbcd44b6/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d676c697463683f6c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-glitch?label=changelog" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-glitch/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p>
  <p>Utilities to help run Datasette on Glitch</p>
  <h2><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p>Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ pip install datasette-glitch
  "><pre><code>$ pip install datasette-glitch
  </code></pre></div>
  <h2><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p>This plugin outputs a special link which will sign you into Datasette as the root user.</p>
  <p>Click Tools -&gt; Logs in the Glitch editor interface after your app starts to see the link.</p>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyNzU2MjQzNDY=
name: datasette-init
full_name: simonw/datasette-init
html_url: https://github.com/simonw/datasette-init
description: Ensure specific tables and views exist on startup
created_at: 2020-06-28T16:26:29Z
updated_at: 2021-06-14T19:43:55Z
pushed_at: 2020-07-01T22:47:09Z
language: Python
topics: []
default_branch: main
permissions: {"admin": false, "push": false, "pull": false}
readme:
  # datasette-init

  [![PyPI](https://img.shields.io/pypi/v/datasette-init.svg)](https://pypi.org/project/datasette-init/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-init?label=changelog)](https://github.com/simonw/datasette-init/releases)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-init/blob/master/LICENSE)

  Ensure specific tables and views exist on startup

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ pip install datasette-init

  ## Usage

  This plugin is configured using `metadata.json` (or `metadata.yaml`). 

  ### Creating tables

  Add a block like this that specifies the tables you would like to ensure exist:

  ```json
  {
    "plugins": {
      "datasette-init": {
        "my_database": {
          "tables": {
            "dogs": {
              "columns": {
                "id": "integer",
                "name": "text",
                "age": "integer",
                "weight": "float"
              },
              "pk": "id"
            }
          }
        }
      }
    }
  }
  ```

  Any tables that do not yet exist will be created when Datasette first starts.

  Valid column types are `"integer"`, `"text"`, `"float"` and `"blob"`.

  The `"pk"` is optional, and is used to define the primary key. To define a compound primary key (across more than one column) use a list of column names here:
  ```json
      "pk": ["id1", "id2"]
  ```

  ### Creating views

  The plugin can also be used to create views:

  ```json
  {
    "plugins": {
      "datasette-init": {
        "my_database": {
          "views": {
            "my_view": "select 1 + 1"
          }
        }
      }
    }
  }
  ```

  Each view in the ``"views"`` block will be created when the Database first starts. If a view with the same name already exists it will be replaced with the new definition.


  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-init
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-datasette-init" class="anchor" aria-hidden="true" href="#user-content-datasette-init"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-init</h1>
  <p><a href="https://pypi.org/project/datasette-init/" rel="nofollow"><img src="https://camo.githubusercontent.com/83fefe5c408de5ede9ce3a712c8133a53de0da3cea828b513827c7d14c4c3cb2/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d696e69742e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-init.svg" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-init/releases"><img src="https://camo.githubusercontent.com/c60721d7193f69d423216b711462556b27b48be0c5b33e74b1831a2daea05bdc/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d696e69743f6c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-init?label=changelog" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-init/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p>
  <p>Ensure specific tables and views exist on startup</p>
  <h2><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p>Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ pip install datasette-init
  "><pre><code>$ pip install datasette-init
  </code></pre></div>
  <h2><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p>This plugin is configured using <code>metadata.json</code> (or <code>metadata.yaml</code>).</p>
  <h3><a id="user-content-creating-tables" class="anchor" aria-hidden="true" href="#user-content-creating-tables"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Creating tables</h3>
  <p>Add a block like this that specifies the tables you would like to ensure exist:</p>
  <div class="highlight highlight-source-json position-relative" data-snippet-clipboard-copy-content="{
    &quot;plugins&quot;: {
      &quot;datasette-init&quot;: {
        &quot;my_database&quot;: {
          &quot;tables&quot;: {
            &quot;dogs&quot;: {
              &quot;columns&quot;: {
                &quot;id&quot;: &quot;integer&quot;,
                &quot;name&quot;: &quot;text&quot;,
                &quot;age&quot;: &quot;integer&quot;,
                &quot;weight&quot;: &quot;float&quot;
              },
              &quot;pk&quot;: &quot;id&quot;
            }
          }
        }
      }
    }
  }
  "><pre>{
    <span class="pl-s"><span class="pl-pds">"</span>plugins<span class="pl-pds">"</span></span>: {
      <span class="pl-s"><span class="pl-pds">"</span>datasette-init<span class="pl-pds">"</span></span>: {
        <span class="pl-s"><span class="pl-pds">"</span>my_database<span class="pl-pds">"</span></span>: {
          <span class="pl-s"><span class="pl-pds">"</span>tables<span class="pl-pds">"</span></span>: {
            <span class="pl-s"><span class="pl-pds">"</span>dogs<span class="pl-pds">"</span></span>: {
              <span class="pl-s"><span class="pl-pds">"</span>columns<span class="pl-pds">"</span></span>: {
                <span class="pl-s"><span class="pl-pds">"</span>id<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>integer<span class="pl-pds">"</span></span>,
                <span class="pl-s"><span class="pl-pds">"</span>name<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>text<span class="pl-pds">"</span></span>,
                <span class="pl-s"><span class="pl-pds">"</span>age<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>integer<span class="pl-pds">"</span></span>,
                <span class="pl-s"><span class="pl-pds">"</span>weight<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>float<span class="pl-pds">"</span></span>
              },
              <span class="pl-s"><span class="pl-pds">"</span>pk<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>id<span class="pl-pds">"</span></span>
            }
          }
        }
      }
    }
  }</pre></div>
  <p>Any tables that do not yet exist will be created when Datasette first starts.</p>
  <p>Valid column types are <code>"integer"</code>, <code>"text"</code>, <code>"float"</code> and <code>"blob"</code>.</p>
  <p>The <code>"pk"</code> is optional, and is used to define the primary key. To define a compound primary key (across more than one column) use a list of column names here:</p>
  <div class="highlight highlight-source-json position-relative" data-snippet-clipboard-copy-content="    &quot;pk&quot;: [&quot;id1&quot;, &quot;id2&quot;]
  "><pre>    <span class="pl-s"><span class="pl-pds">"</span>pk<span class="pl-pds">"</span></span>: [<span class="pl-s"><span class="pl-pds">"</span>id1<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>id2<span class="pl-pds">"</span></span>]</pre></div>
  <h3><a id="user-content-creating-views" class="anchor" aria-hidden="true" href="#user-content-creating-views"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Creating views</h3>
  <p>The plugin can also be used to create views:</p>
  <div class="highlight highlight-source-json position-relative" data-snippet-clipboard-copy-content="{
    &quot;plugins&quot;: {
      &quot;datasette-init&quot;: {
        &quot;my_database&quot;: {
          &quot;views&quot;: {
            &quot;my_view&quot;: &quot;select 1 + 1&quot;
          }
        }
      }
    }
  }
  "><pre>{
    <span class="pl-s"><span class="pl-pds">"</span>plugins<span class="pl-pds">"</span></span>: {
      <span class="pl-s"><span class="pl-pds">"</span>datasette-init<span class="pl-pds">"</span></span>: {
        <span class="pl-s"><span class="pl-pds">"</span>my_database<span class="pl-pds">"</span></span>: {
          <span class="pl-s"><span class="pl-pds">"</span>views<span class="pl-pds">"</span></span>: {
            <span class="pl-s"><span class="pl-pds">"</span>my_view<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>select 1 + 1<span class="pl-pds">"</span></span>
          }
        }
      }
    }
  }</pre></div>
  <p>Each view in the <code>"views"</code> block will be created when the Database first starts. If a view with the same name already exists it will be replaced with the new definition.</p>
  <h2><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p>To set up this plugin locally, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="cd datasette-init
  python3 -mvenv venv
  source venv/bin/activate
  "><pre><code>cd datasette-init
  python3 -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p>Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pipenv shell
  "><pre><code>pipenv shell
  </code></pre></div>
  <p>Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pip install -e '.[test]'
  "><pre><code>pip install -e '.[test]'
  </code></pre></div>
  <p>To run the tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pytest
  "><pre><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyNzU3MTEyNTQ=
name: datasette-write
full_name: simonw/datasette-write
html_url: https://github.com/simonw/datasette-write
description: Datasette plugin providing a UI for executing SQL writes against the database
created_at: 2020-06-29T02:27:31Z
updated_at: 2021-09-11T06:00:31Z
pushed_at: 2021-09-11T06:03:07Z
homepage: https://datasette.io/plugins/datasette-write
language: Python
topics: ["datasette-io", "datasette-plugin"]
default_branch: main
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
readme:
  # datasette-write

  [![PyPI](https://img.shields.io/pypi/v/datasette-write.svg)](https://pypi.org/project/datasette-write/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-write?label=changelog)](https://github.com/simonw/datasette-write/releases)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-write/blob/master/LICENSE)

  Datasette plugin providing a UI for writing to a database

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ pip install datasette-write

  ## Usage

  Having installed the plugin, visit `/-/write` on your Datasette instance to submit SQL queries that will be executed against a write connection to the specified database.

  By default only the `root` user can access the page - so you'll need to run Datasette with the `--root` option and click on the link shown in the terminal to sign in and access the page.

  The `datasette-write` permission governs access. You can use permission plugins such as [datasette-permissions-sql](https://github.com/simonw/datasette-permissions-sql) to grant additional access to the write interface.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-write
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-datasette-write" class="anchor" aria-hidden="true" href="#user-content-datasette-write"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-write</h1>
  <p><a href="https://pypi.org/project/datasette-write/" rel="nofollow"><img src="https://camo.githubusercontent.com/af6b19f94bfb6ec8ed636dff5653009dc4f78e69fd40954cb91643aa191fbf8c/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d77726974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-write.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-write/releases"><img src="https://camo.githubusercontent.com/e1043914eb7cc84d8e2e2f9dbd17ba4051783e6f7da84c9bdce83a3a53e0b9e4/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d77726974653f6c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-write?label=changelog" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-write/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width: 100%;"></a></p>
  <p>Datasette plugin providing a UI for writing to a database</p>
  <h2><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p>Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ pip install datasette-write
  "><pre><code>$ pip install datasette-write
  </code></pre></div>
  <h2><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p>Having installed the plugin, visit <code>/-/write</code> on your Datasette instance to submit SQL queries that will be executed against a write connection to the specified database.</p>
  <p>By default only the <code>root</code> user can access the page - so you'll need to run Datasette with the <code>--root</code> option and click on the link shown in the terminal to sign in and access the page.</p>
  <p>The <code>datasette-write</code> permission governs access. You can use permission plugins such as <a href="https://github.com/simonw/datasette-permissions-sql">datasette-permissions-sql</a> to grant additional access to the write interface.</p>
  <h2><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p>To set up this plugin locally, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="cd datasette-write
  python3 -mvenv venv
  source venv/bin/activate
  "><pre><code>cd datasette-write
  python3 -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p>Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pipenv shell
  "><pre><code>pipenv shell
  </code></pre></div>
  <p>Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pip install -e '.[test]'
  "><pre><code>pip install -e '.[test]'
  </code></pre></div>
  <p>To run the tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pytest
  "><pre><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyNzkzNTcxMjM=
name: datasette-auth-passwords
full_name: simonw/datasette-auth-passwords
html_url: https://github.com/simonw/datasette-auth-passwords
description: Datasette plugin for authentication using passwords
created_at: 2020-07-13T16:34:39Z
updated_at: 2022-02-10T22:07:52Z
pushed_at: 2022-03-22T01:49:50Z
homepage: https://datasette-auth-passwords-demo.datasette.io
language: Python
topics: ["datasette", "datasette-io", "datasette-plugin"]
default_branch: main
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
visibility: public
readme:
  # datasette-auth-passwords

  [![PyPI](https://img.shields.io/pypi/v/datasette-auth-passwords.svg)](https://pypi.org/project/datasette-auth-passwords/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-auth-passwords?label=changelog)](https://github.com/simonw/datasette-auth-passwords/releases)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-auth-passwords/blob/master/LICENSE)

  Datasette plugin for authenticating access using passwords

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-auth-passwords

  ## Demo

  A demo of this plugin is running at https://datasette-auth-passwords-demo.datasette.io/

  The demo is configured to show the `public.db` database to everyone, but the `private.db` database only to logged in users.

  You can log in at https://datasette-auth-passwords-demo.datasette.io/-/login with username `root` and password `password!`.

  ## Usage

  This plugin works based on a list of username/password accounts that are hard-coded into the plugin configuration.

  First, you'll need to create a password hash. There are three ways to do that:

  - Install the plugin, then use the interactive tool located at `/-/password-tool`
  - Use the hosted version of that tool at https://datasette-auth-passwords-demo.datasette.io/-/password-tool
  - Use the `datasette hash-password` command, described below

  Now add the following to your `metadata.json`:

  ```json
  {
      "plugins": {
          "datasette-auth-passwords": {
              "someusername_password_hash": {
                  "$env": "PASSWORD_HASH_1"
              }
          }
      }
  }
  ```

  The password hash can now be specified in an environment variable when you run Datasette. You can do that like so:

      PASSWORD_HASH_1='pbkdf2_sha256$...' \
          datasette -m metadata.json

  Be sure to use single quotes here otherwise the `$` symbols in the password hash may be incorrectly interpreted by your shell.

  You will now be able to log in to your instance using the form at `/-/login` with `someusername` as the username and the password that you used to create your hash as the password.

  You can include as many accounts as you like in the configuration, each with different usernames.

  ### datasette hash-password

  The plugin exposes a new CLI command, `datasette hash-password`. You can run this without arguments to interactively create a new password hash:
  ```
  % datasette hash-password
  Password: 
  Repeat for confirmation: 
  pbkdf2_sha256$260000$1513...
  ```
  Or if you want to use it as part of a script, you can add the `--no-confirm` option to generate a hash directly from a value passed to standard input:
  ```
  % echo 'my password' | datasette hash-password --no-confirm
  pbkdf2_sha256$260000$daa...
  ```
  ### Specifying actors

  By default, a logged in user will result in an [actor block](https://datasette.readthedocs.io/en/stable/authentication.html#actors) that just contains their username:

  ```json
  {
      "id": "someusername"
  }
  ```

  You can customize the actor that will be used for a username by including an `"actors"` configuration block, like this:

  ```json
  {
      "plugins": {
          "datasette-auth-passwords": {
              "someusername_password_hash": {
                  "$env": "PASSWORD_HASH_1"
              },
              "actors": {
                  "someusername": {
                      "id": "someusername",
                      "name": "Some user"
                  }
              }
          }
      }
  }
  ```
  ### HTTP Basic authentication option

  This plugin defaults to implementing login using an HTML form that sets a signed authentication cookie.

  You can alternatively configure it to use [HTTP Basic authentication](https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication#basic_authentication_scheme) instead.

  Do this by adding `"http_basic_auth": true` to the `datasette-auth-passwords` block in your plugin configuration.

  This option introduces the following behaviour:

  - Account usernames and passwords are configured in the same way as form-based authentication
  - Every page within Datasette - even pages that normally do not use authentication, such as static assets - will display a browser login prompt
  - Users will be unable to log out without closing their browser entirely

  There is a demo of this mode at https://datasette-auth-passwords-http-basic-demo.datasette.io/ - sign in with username `root` and password `password!`

  ### Using with datasette publish

  If you are publishing data using a [datasette publish](https://datasette.readthedocs.io/en/stable/publish.html#datasette-publish) command you can use the `--plugin-secret` option to securely configure your password hashes (see [secret configuration values](https://datasette.readthedocs.io/en/stable/plugins.html#secret-configuration-values)).

  You would run the command something like this:

      datasette publish cloudrun mydatabase.db \
          --install datasette-auth-passwords \
          --plugin-secret datasette-auth-passwords root_password_hash 'pbkdf2_sha256$...' \
          --service datasette-auth-passwords-demo

  This will allow you to log in as username `root` using the password that you used to create the hash.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-auth-passwords
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-datasette-auth-passwords" class="anchor" aria-hidden="true" href="#user-content-datasette-auth-passwords"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-auth-passwords</h1>
  <p dir="auto"><a href="https://pypi.org/project/datasette-auth-passwords/" rel="nofollow"><img src="https://camo.githubusercontent.com/6069887e91e8732eb602a4f9f0294b4fb71295c8f3fd82133d21bdb062694a96/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d617574682d70617373776f7264732e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-auth-passwords.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-auth-passwords/releases"><img src="https://camo.githubusercontent.com/826bb037366364dfa68ab57355193eb9ded8ecb7d137c62e66d8b9502fe9039f/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d617574682d70617373776f7264733f6c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-auth-passwords?label=changelog" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-auth-passwords/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width: 100%;"></a></p>
  <p dir="auto">Datasette plugin for authenticating access using passwords</p>
  <h2 dir="auto"><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p dir="auto">Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="$ datasette install datasette-auth-passwords"><pre><code>$ datasette install datasette-auth-passwords
  </code></pre></div>
  <h2 dir="auto"><a id="user-content-demo" class="anchor" aria-hidden="true" href="#user-content-demo"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Demo</h2>
  <p dir="auto">A demo of this plugin is running at <a href="https://datasette-auth-passwords-demo.datasette.io/" rel="nofollow">https://datasette-auth-passwords-demo.datasette.io/</a></p>
  <p dir="auto">The demo is configured to show the <code>public.db</code> database to everyone, but the <code>private.db</code> database only to logged in users.</p>
  <p dir="auto">You can log in at <a href="https://datasette-auth-passwords-demo.datasette.io/-/login" rel="nofollow">https://datasette-auth-passwords-demo.datasette.io/-/login</a> with username <code>root</code> and password <code>password!</code>.</p>
  <h2 dir="auto"><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p dir="auto">This plugin works based on a list of username/password accounts that are hard-coded into the plugin configuration.</p>
  <p dir="auto">First, you'll need to create a password hash. There are three ways to do that:</p>
  <ul dir="auto">
  <li>Install the plugin, then use the interactive tool located at <code>/-/password-tool</code></li>
  <li>Use the hosted version of that tool at <a href="https://datasette-auth-passwords-demo.datasette.io/-/password-tool" rel="nofollow">https://datasette-auth-passwords-demo.datasette.io/-/password-tool</a></li>
  <li>Use the <code>datasette hash-password</code> command, described below</li>
  </ul>
  <p dir="auto">Now add the following to your <code>metadata.json</code>:</p>
  <div class="highlight highlight-source-json position-relative overflow-auto" data-snippet-clipboard-copy-content="{
      &quot;plugins&quot;: {
          &quot;datasette-auth-passwords&quot;: {
              &quot;someusername_password_hash&quot;: {
                  &quot;$env&quot;: &quot;PASSWORD_HASH_1&quot;
              }
          }
      }
  }"><pre>{
      <span class="pl-ent">"plugins"</span>: {
          <span class="pl-ent">"datasette-auth-passwords"</span>: {
              <span class="pl-ent">"someusername_password_hash"</span>: {
                  <span class="pl-ent">"$env"</span>: <span class="pl-s"><span class="pl-pds">"</span>PASSWORD_HASH_1<span class="pl-pds">"</span></span>
              }
          }
      }
  }</pre></div>
  <p dir="auto">The password hash can now be specified in an environment variable when you run Datasette. You can do that like so:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="PASSWORD_HASH_1='pbkdf2_sha256$...' \
      datasette -m metadata.json"><pre><code>PASSWORD_HASH_1='pbkdf2_sha256$...' \
      datasette -m metadata.json
  </code></pre></div>
  <p dir="auto">Be sure to use single quotes here otherwise the <code>$</code> symbols in the password hash may be incorrectly interpreted by your shell.</p>
  <p dir="auto">You will now be able to log in to your instance using the form at <code>/-/login</code> with <code>someusername</code> as the username and the password that you used to create your hash as the password.</p>
  <p dir="auto">You can include as many accounts as you like in the configuration, each with different usernames.</p>
  <h3 dir="auto"><a id="user-content-datasette-hash-password" class="anchor" aria-hidden="true" href="#user-content-datasette-hash-password"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette hash-password</h3>
  <p dir="auto">The plugin exposes a new CLI command, <code>datasette hash-password</code>. You can run this without arguments to interactively create a new password hash:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="% datasette hash-password
  Password: 
  Repeat for confirmation: 
  pbkdf2_sha256$260000$1513..."><pre><code>% datasette hash-password
  Password: 
  Repeat for confirmation: 
  pbkdf2_sha256$260000$1513...
  </code></pre></div>
  <p dir="auto">Or if you want to use it as part of a script, you can add the <code>--no-confirm</code> option to generate a hash directly from a value passed to standard input:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="% echo 'my password' | datasette hash-password --no-confirm
  pbkdf2_sha256$260000$daa..."><pre><code>% echo 'my password' | datasette hash-password --no-confirm
  pbkdf2_sha256$260000$daa...
  </code></pre></div>
  <h3 dir="auto"><a id="user-content-specifying-actors" class="anchor" aria-hidden="true" href="#user-content-specifying-actors"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Specifying actors</h3>
  <p dir="auto">By default, a logged in user will result in an <a href="https://datasette.readthedocs.io/en/stable/authentication.html#actors" rel="nofollow">actor block</a> that just contains their username:</p>
  <div class="highlight highlight-source-json position-relative overflow-auto" data-snippet-clipboard-copy-content="{
      &quot;id&quot;: &quot;someusername&quot;
  }"><pre>{
      <span class="pl-ent">"id"</span>: <span class="pl-s"><span class="pl-pds">"</span>someusername<span class="pl-pds">"</span></span>
  }</pre></div>
  <p dir="auto">You can customize the actor that will be used for a username by including an <code>"actors"</code> configuration block, like this:</p>
  <div class="highlight highlight-source-json position-relative overflow-auto" data-snippet-clipboard-copy-content="{
      &quot;plugins&quot;: {
          &quot;datasette-auth-passwords&quot;: {
              &quot;someusername_password_hash&quot;: {
                  &quot;$env&quot;: &quot;PASSWORD_HASH_1&quot;
              },
              &quot;actors&quot;: {
                  &quot;someusername&quot;: {
                      &quot;id&quot;: &quot;someusername&quot;,
                      &quot;name&quot;: &quot;Some user&quot;
                  }
              }
          }
      }
  }"><pre>{
      <span class="pl-ent">"plugins"</span>: {
          <span class="pl-ent">"datasette-auth-passwords"</span>: {
              <span class="pl-ent">"someusername_password_hash"</span>: {
                  <span class="pl-ent">"$env"</span>: <span class="pl-s"><span class="pl-pds">"</span>PASSWORD_HASH_1<span class="pl-pds">"</span></span>
              },
              <span class="pl-ent">"actors"</span>: {
                  <span class="pl-ent">"someusername"</span>: {
                      <span class="pl-ent">"id"</span>: <span class="pl-s"><span class="pl-pds">"</span>someusername<span class="pl-pds">"</span></span>,
                      <span class="pl-ent">"name"</span>: <span class="pl-s"><span class="pl-pds">"</span>Some user<span class="pl-pds">"</span></span>
                  }
              }
          }
      }
  }</pre></div>
  <h3 dir="auto"><a id="user-content-http-basic-authentication-option" class="anchor" aria-hidden="true" href="#user-content-http-basic-authentication-option"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>HTTP Basic authentication option</h3>
  <p dir="auto">This plugin defaults to implementing login using an HTML form that sets a signed authentication cookie.</p>
  <p dir="auto">You can alternatively configure it to use <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication#basic_authentication_scheme" rel="nofollow">HTTP Basic authentication</a> instead.</p>
  <p dir="auto">Do this by adding <code>"http_basic_auth": true</code> to the <code>datasette-auth-passwords</code> block in your plugin configuration.</p>
  <p dir="auto">This option introduces the following behaviour:</p>
  <ul dir="auto">
  <li>Account usernames and passwords are configured in the same way as form-based authentication</li>
  <li>Every page within Datasette - even pages that normally do not use authentication, such as static assets - will display a browser login prompt</li>
  <li>Users will be unable to log out without closing their browser entirely</li>
  </ul>
  <p dir="auto">There is a demo of this mode at <a href="https://datasette-auth-passwords-http-basic-demo.datasette.io/" rel="nofollow">https://datasette-auth-passwords-http-basic-demo.datasette.io/</a> - sign in with username <code>root</code> and password <code>password!</code></p>
  <h3 dir="auto"><a id="user-content-using-with-datasette-publish" class="anchor" aria-hidden="true" href="#user-content-using-with-datasette-publish"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Using with datasette publish</h3>
  <p dir="auto">If you are publishing data using a <a href="https://datasette.readthedocs.io/en/stable/publish.html#datasette-publish" rel="nofollow">datasette publish</a> command you can use the <code>--plugin-secret</code> option to securely configure your password hashes (see <a href="https://datasette.readthedocs.io/en/stable/plugins.html#secret-configuration-values" rel="nofollow">secret configuration values</a>).</p>
  <p dir="auto">You would run the command something like this:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="datasette publish cloudrun mydatabase.db \
      --install datasette-auth-passwords \
      --plugin-secret datasette-auth-passwords root_password_hash 'pbkdf2_sha256$...' \
      --service datasette-auth-passwords-demo"><pre><code>datasette publish cloudrun mydatabase.db \
      --install datasette-auth-passwords \
      --plugin-secret datasette-auth-passwords root_password_hash 'pbkdf2_sha256$...' \
      --service datasette-auth-passwords-demo
  </code></pre></div>
  <p dir="auto">This will allow you to log in as username <code>root</code> using the password that you used to create the hash.</p>
  <h2 dir="auto"><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p dir="auto">To set up this plugin locally, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="cd datasette-auth-passwords
  python3 -mvenv venv
  source venv/bin/activate"><pre><code>cd datasette-auth-passwords
  python3 -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p dir="auto">Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pipenv shell"><pre><code>pipenv shell
  </code></pre></div>
  <p dir="auto">Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pip install -e '.[test]'"><pre><code>pip install -e '.[test]'
  </code></pre></div>
  <p dir="auto">To run the tests:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pytest"><pre><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyODA1MDAwMjc=
name: datasette-insert
full_name: simonw/datasette-insert
html_url: https://github.com/simonw/datasette-insert
description: Datasette plugin for inserting and updating data
created_at: 2020-07-17T18:40:34Z
updated_at: 2022-06-27T02:54:14Z
pushed_at: 2022-07-22T17:52:23Z
language: Python
topics: ["datasette", "datasette-io", "datasette-plugin"]
default_branch: main
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
visibility: public
readme:
  # datasette-insert

  [![PyPI](https://img.shields.io/pypi/v/datasette-insert.svg)](https://pypi.org/project/datasette-insert/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-insert?include_prereleases&label=changelog)](https://github.com/simonw/datasette-insert/releases)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-insert/blob/master/LICENSE)

  Datasette plugin for inserting and updating data

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ pip install datasette-insert

  This plugin should always be deployed with additional configuration to prevent unauthenticated access, see notes below.

  If you are trying it out on your own local machine, you can `pip install` the [datasette-insert-unsafe](https://github.com/simonw/datasette-insert-unsafe) plugin to allow access without needing to set up authentication or permissions separately.

  ## Inserting data and creating tables

  Start datasette and make sure it has a writable SQLite database attached to it. If you have not yet created a database file you can use this:

      datasette data.db --create

  The `--create` option will create a new empty `data.db` database file if it does not already exist.

  The plugin adds an endpoint that allows data to be inserted or updated and tables to be created by POSTing JSON data to the following URL:

      /-/insert/name-of-database/name-of-table

  The JSON should look like this:

  ```json
  [
      {
          "id": 1,
          "name": "Cleopaws",
          "age": 5
      },
      {
          "id": 2,
          "name": "Pancakes",
          "age": 5
      }
  ]
  ```

  The first time data is posted to the URL a table of that name will be created if it does not aready exist, with the desired columns.

  You can specify which column should be used as the primary key using the `?pk=` URL argument.

  Here's how to POST to a database and create a new table using the Python `requests` library:

  ```python
  import requests

  requests.post("http://localhost:8001/-/insert/data/dogs?pk=id", json=[
      {
          "id": 1,
          "name": "Cleopaws",
          "age": 5
      },
      {
          "id": 2,
          "name": "Pancakes",
          "age": 4
      }
  ])
  ```
  And here's how to do the same thing using `curl`:

  ```
  curl --request POST \
    --data '[
        {
          "id": 1,
          "name": "Cleopaws",
          "age": 5
        },
        {
          "id": 2,
          "name": "Pancakes",
          "age": 4
        }
      ]' \
      'http://localhost:8001/-/insert/data/dogs?pk=id'
  ```
  Or by piping in JSON like so:

      cat dogs.json | curl --request POST -d @- \
          'http://localhost:8001/-/insert/data/dogs?pk=id'

  ### Inserting a single row

  If you are inserting a single row you can optionally send it as a dictionary rather than a list with a single item:

  ```
  curl --request POST \
    --data '{
        "id": 1,
        "name": "Cleopaws",
        "age": 5
      }' \
      'http://localhost:8001/-/insert/data/dogs?pk=id'
  ```

  ### Automatically adding new columns

  If you send data to an existing table with keys that are not reflected by the existing columns, you will get an HTTP 400 error with a JSON response like this:

  ```json
  {
      "status": 400,
      "error": "Unknown keys: 'foo'",
      "error_code": "unknown_keys"
  }
  ```

  If you add `?alter=1` to the URL you are posting to any missing columns will be automatically added:

  ```
  curl --request POST \
    --data '[
        {
          "id": 3,
          "name": "Boris",
          "age": 1,
          "breed": "Husky"
        }
      ]' \
      'http://localhost:8001/-/insert/data/dogs?alter=1'
  ```

  ## Upserting data

  An "upsert" operation can be used to partially update a record. With upserts you can send a subset of the keys and, if the ID matches the specified primary key, they will be used to update an existing record.

  Upserts can be sent to the `/-/upsert` API endpoint.

  This example will update the dog with ID=1's age from 5 to 7:
  ```
  curl --request POST \
    --data '{
        "id": 1,
        "age": 7
      }' \
      'http://localhost:3322/-/upsert/data/dogs?pk=id'
  ```
  Like the `/-/insert` endpoint, the `/-/upsert` endpoint can accept an array of objects too. It also supports the `?alter=1` option.

  ## Permissions and authentication

  This plugin defaults to denying all access, to help ensure people don't accidentally deploy it on the open internet in an unsafe configuration.

  You can read about [Datasette's approach to authentication](https://datasette.readthedocs.io/en/stable/authentication.html) in the Datasette manual.

  You can install the `datasette-insert-unsafe` plugin to run in unsafe mode, where all access is allowed by default.

  I recommend using this plugin in conjunction with [datasette-auth-tokens](https://github.com/simonw/datasette-auth-tokens), which provides a mechanism for making authenticated calls using API tokens.

  You can then use ["allow" blocks](https://datasette.readthedocs.io/en/stable/authentication.html#defining-permissions-with-allow-blocks) in the `datasette-insert` plugin configuration to specify which authenticated tokens are allowed to make use of the API.

  Here's an example `metadata.json` file which restricts access to the `/-/insert` API to an API token defined in an `INSERT_TOKEN` environment variable:

  ```json
  {
      "plugins": {
          "datasette-insert": {
              "allow": {
                  "bot": "insert-bot"
              }
          },
          "datasette-auth-tokens": {
              "tokens": [
                  {
                      "token": {
                          "$env": "INSERT_TOKEN"
                      },
                      "actor": {
                          "bot": "insert-bot"
                      }
                  }
              ]
          }
      }
  }
  ```
  With this configuration in place you can start Datasette like this:

      INSERT_TOKEN=abc123 datasette data.db -m metadata.json

  You can now send data to the API using `curl` like this:

  ```
  curl --request POST \
    -H "Authorization: Bearer abc123" \
    --data '[
        {
          "id": 3,
          "name": "Boris",
          "age": 1,
          "breed": "Husky"
        }
      ]' \
      'http://localhost:8001/-/insert/data/dogs'
  ```

  Or using the Python `requests` library like so:

  ```python
  requests.post(
      "http://localhost:8001/-/insert/data/dogs",
      json={"id": 1, "name": "Cleopaws", "age": 5},
      headers={"Authorization": "bearer abc123"},
  )
  ```

  ### Finely grained permissions

  Using an `"allow"` block as described above grants full permission to the features enabled by the API.

  The API implements several new Datasett permissions, which other plugins can use to make more finely grained decisions.

  The full set of permissions are as follows:

  - `insert:all` - all permissions - this is used by the `"allow"` block described above. Argument: `database_name`
  - `insert:insert-update` - the ability to insert data into an existing table, or to update data by its primary key. Arguments: `(database_name, table_name)`
  - `insert:create-table` - the ability to create a new table. Argument: `database_name`
  - `insert:alter-table` - the ability to add columns to an existing table (using `?alter=1`). Arguments: `(database_name, table_name)`

  You can use plugins like [datasette-permissions-sql](https://github.com/simonw/datasette-permissions-sql) to hook into these more detailed permissions for finely grained control over what actions each authenticated actor can take.

  Plugins that implement the [permission_allowed()](https://datasette.readthedocs.io/en/stable/plugin_hooks.html#plugin-hook-permission-allowed) plugin hook can take full control over these permission decisions.

  ## CORS

  If you start Datasette with the `datasette --cors` option the following HTTP headers will be added to resources served by this plugin:

      Access-Control-Allow-Origin: *
      Access-Control-Allow-Headers: content-type,authorization
      Access-Control-Allow-Methods: POST

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-insert
      python3 -m venv venv
      source venv/bin/activate

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-datasette-insert" class="anchor" aria-hidden="true" href="#user-content-datasette-insert"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-insert</h1>
  <p dir="auto"><a href="https://pypi.org/project/datasette-insert/" rel="nofollow"><img src="https://camo.githubusercontent.com/f8375e92990e2f5fdcf4657451dccba02132abfd34e39bb49773201264951b85/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d696e736572742e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-insert.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-insert/releases"><img src="https://camo.githubusercontent.com/035155855612d32c93ac562f5316c97a15edc01cc0ef35ad4535951368e9ff42/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d696e736572743f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-insert?include_prereleases&amp;label=changelog" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-insert/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width: 100%;"></a></p>
  <p dir="auto">Datasette plugin for inserting and updating data</p>
  <h2 dir="auto"><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p dir="auto">Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="$ pip install datasette-insert"><pre class="notranslate"><code>$ pip install datasette-insert
  </code></pre></div>
  <p dir="auto">This plugin should always be deployed with additional configuration to prevent unauthenticated access, see notes below.</p>
  <p dir="auto">If you are trying it out on your own local machine, you can <code>pip install</code> the <a href="https://github.com/simonw/datasette-insert-unsafe">datasette-insert-unsafe</a> plugin to allow access without needing to set up authentication or permissions separately.</p>
  <h2 dir="auto"><a id="user-content-inserting-data-and-creating-tables" class="anchor" aria-hidden="true" href="#user-content-inserting-data-and-creating-tables"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Inserting data and creating tables</h2>
  <p dir="auto">Start datasette and make sure it has a writable SQLite database attached to it. If you have not yet created a database file you can use this:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="datasette data.db --create"><pre class="notranslate"><code>datasette data.db --create
  </code></pre></div>
  <p dir="auto">The <code>--create</code> option will create a new empty <code>data.db</code> database file if it does not already exist.</p>
  <p dir="auto">The plugin adds an endpoint that allows data to be inserted or updated and tables to be created by POSTing JSON data to the following URL:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="/-/insert/name-of-database/name-of-table"><pre class="notranslate"><code>/-/insert/name-of-database/name-of-table
  </code></pre></div>
  <p dir="auto">The JSON should look like this:</p>
  <div class="highlight highlight-source-json notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="[
      {
          &quot;id&quot;: 1,
          &quot;name&quot;: &quot;Cleopaws&quot;,
          &quot;age&quot;: 5
      },
      {
          &quot;id&quot;: 2,
          &quot;name&quot;: &quot;Pancakes&quot;,
          &quot;age&quot;: 5
      }
  ]"><pre>[
      {
          <span class="pl-ent">"id"</span>: <span class="pl-c1">1</span>,
          <span class="pl-ent">"name"</span>: <span class="pl-s"><span class="pl-pds">"</span>Cleopaws<span class="pl-pds">"</span></span>,
          <span class="pl-ent">"age"</span>: <span class="pl-c1">5</span>
      },
      {
          <span class="pl-ent">"id"</span>: <span class="pl-c1">2</span>,
          <span class="pl-ent">"name"</span>: <span class="pl-s"><span class="pl-pds">"</span>Pancakes<span class="pl-pds">"</span></span>,
          <span class="pl-ent">"age"</span>: <span class="pl-c1">5</span>
      }
  ]</pre></div>
  <p dir="auto">The first time data is posted to the URL a table of that name will be created if it does not aready exist, with the desired columns.</p>
  <p dir="auto">You can specify which column should be used as the primary key using the <code>?pk=</code> URL argument.</p>
  <p dir="auto">Here's how to POST to a database and create a new table using the Python <code>requests</code> library:</p>
  <div class="highlight highlight-source-python notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import requests

  requests.post(&quot;http://localhost:8001/-/insert/data/dogs?pk=id&quot;, json=[
      {
          &quot;id&quot;: 1,
          &quot;name&quot;: &quot;Cleopaws&quot;,
          &quot;age&quot;: 5
      },
      {
          &quot;id&quot;: 2,
          &quot;name&quot;: &quot;Pancakes&quot;,
          &quot;age&quot;: 4
      }
  ])"><pre><span class="pl-k">import</span> <span class="pl-s1">requests</span>

  <span class="pl-s1">requests</span>.<span class="pl-en">post</span>(<span class="pl-s">"http://localhost:8001/-/insert/data/dogs?pk=id"</span>, <span class="pl-s1">json</span><span class="pl-c1">=</span>[
      {
          <span class="pl-s">"id"</span>: <span class="pl-c1">1</span>,
          <span class="pl-s">"name"</span>: <span class="pl-s">"Cleopaws"</span>,
          <span class="pl-s">"age"</span>: <span class="pl-c1">5</span>
      },
      {
          <span class="pl-s">"id"</span>: <span class="pl-c1">2</span>,
          <span class="pl-s">"name"</span>: <span class="pl-s">"Pancakes"</span>,
          <span class="pl-s">"age"</span>: <span class="pl-c1">4</span>
      }
  ])</pre></div>
  <p dir="auto">And here's how to do the same thing using <code>curl</code>:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="curl --request POST \
    --data '[
        {
          &quot;id&quot;: 1,
          &quot;name&quot;: &quot;Cleopaws&quot;,
          &quot;age&quot;: 5
        },
        {
          &quot;id&quot;: 2,
          &quot;name&quot;: &quot;Pancakes&quot;,
          &quot;age&quot;: 4
        }
      ]' \
      'http://localhost:8001/-/insert/data/dogs?pk=id'"><pre class="notranslate"><code>curl --request POST \
    --data '[
        {
          "id": 1,
          "name": "Cleopaws",
          "age": 5
        },
        {
          "id": 2,
          "name": "Pancakes",
          "age": 4
        }
      ]' \
      'http://localhost:8001/-/insert/data/dogs?pk=id'
  </code></pre></div>
  <p dir="auto">Or by piping in JSON like so:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="cat dogs.json | curl --request POST -d @- \
      'http://localhost:8001/-/insert/data/dogs?pk=id'"><pre class="notranslate"><code>cat dogs.json | curl --request POST -d @- \
      'http://localhost:8001/-/insert/data/dogs?pk=id'
  </code></pre></div>
  <h3 dir="auto"><a id="user-content-inserting-a-single-row" class="anchor" aria-hidden="true" href="#user-content-inserting-a-single-row"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Inserting a single row</h3>
  <p dir="auto">If you are inserting a single row you can optionally send it as a dictionary rather than a list with a single item:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="curl --request POST \
    --data '{
        &quot;id&quot;: 1,
        &quot;name&quot;: &quot;Cleopaws&quot;,
        &quot;age&quot;: 5
      }' \
      'http://localhost:8001/-/insert/data/dogs?pk=id'"><pre class="notranslate"><code>curl --request POST \
    --data '{
        "id": 1,
        "name": "Cleopaws",
        "age": 5
      }' \
      'http://localhost:8001/-/insert/data/dogs?pk=id'
  </code></pre></div>
  <h3 dir="auto"><a id="user-content-automatically-adding-new-columns" class="anchor" aria-hidden="true" href="#user-content-automatically-adding-new-columns"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Automatically adding new columns</h3>
  <p dir="auto">If you send data to an existing table with keys that are not reflected by the existing columns, you will get an HTTP 400 error with a JSON response like this:</p>
  <div class="highlight highlight-source-json notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="{
      &quot;status&quot;: 400,
      &quot;error&quot;: &quot;Unknown keys: 'foo'&quot;,
      &quot;error_code&quot;: &quot;unknown_keys&quot;
  }"><pre>{
      <span class="pl-ent">"status"</span>: <span class="pl-c1">400</span>,
      <span class="pl-ent">"error"</span>: <span class="pl-s"><span class="pl-pds">"</span>Unknown keys: 'foo'<span class="pl-pds">"</span></span>,
      <span class="pl-ent">"error_code"</span>: <span class="pl-s"><span class="pl-pds">"</span>unknown_keys<span class="pl-pds">"</span></span>
  }</pre></div>
  <p dir="auto">If you add <code>?alter=1</code> to the URL you are posting to any missing columns will be automatically added:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="curl --request POST \
    --data '[
        {
          &quot;id&quot;: 3,
          &quot;name&quot;: &quot;Boris&quot;,
          &quot;age&quot;: 1,
          &quot;breed&quot;: &quot;Husky&quot;
        }
      ]' \
      'http://localhost:8001/-/insert/data/dogs?alter=1'"><pre class="notranslate"><code>curl --request POST \
    --data '[
        {
          "id": 3,
          "name": "Boris",
          "age": 1,
          "breed": "Husky"
        }
      ]' \
      'http://localhost:8001/-/insert/data/dogs?alter=1'
  </code></pre></div>
  <h2 dir="auto"><a id="user-content-upserting-data" class="anchor" aria-hidden="true" href="#user-content-upserting-data"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Upserting data</h2>
  <p dir="auto">An "upsert" operation can be used to partially update a record. With upserts you can send a subset of the keys and, if the ID matches the specified primary key, they will be used to update an existing record.</p>
  <p dir="auto">Upserts can be sent to the <code>/-/upsert</code> API endpoint.</p>
  <p dir="auto">This example will update the dog with ID=1's age from 5 to 7:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="curl --request POST \
    --data '{
        &quot;id&quot;: 1,
        &quot;age&quot;: 7
      }' \
      'http://localhost:3322/-/upsert/data/dogs?pk=id'"><pre class="notranslate"><code>curl --request POST \
    --data '{
        "id": 1,
        "age": 7
      }' \
      'http://localhost:3322/-/upsert/data/dogs?pk=id'
  </code></pre></div>
  <p dir="auto">Like the <code>/-/insert</code> endpoint, the <code>/-/upsert</code> endpoint can accept an array of objects too. It also supports the <code>?alter=1</code> option.</p>
  <h2 dir="auto"><a id="user-content-permissions-and-authentication" class="anchor" aria-hidden="true" href="#user-content-permissions-and-authentication"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Permissions and authentication</h2>
  <p dir="auto">This plugin defaults to denying all access, to help ensure people don't accidentally deploy it on the open internet in an unsafe configuration.</p>
  <p dir="auto">You can read about <a href="https://datasette.readthedocs.io/en/stable/authentication.html" rel="nofollow">Datasette's approach to authentication</a> in the Datasette manual.</p>
  <p dir="auto">You can install the <code>datasette-insert-unsafe</code> plugin to run in unsafe mode, where all access is allowed by default.</p>
  <p dir="auto">I recommend using this plugin in conjunction with <a href="https://github.com/simonw/datasette-auth-tokens">datasette-auth-tokens</a>, which provides a mechanism for making authenticated calls using API tokens.</p>
  <p dir="auto">You can then use <a href="https://datasette.readthedocs.io/en/stable/authentication.html#defining-permissions-with-allow-blocks" rel="nofollow">"allow" blocks</a> in the <code>datasette-insert</code> plugin configuration to specify which authenticated tokens are allowed to make use of the API.</p>
  <p dir="auto">Here's an example <code>metadata.json</code> file which restricts access to the <code>/-/insert</code> API to an API token defined in an <code>INSERT_TOKEN</code> environment variable:</p>
  <div class="highlight highlight-source-json notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="{
      &quot;plugins&quot;: {
          &quot;datasette-insert&quot;: {
              &quot;allow&quot;: {
                  &quot;bot&quot;: &quot;insert-bot&quot;
              }
          },
          &quot;datasette-auth-tokens&quot;: {
              &quot;tokens&quot;: [
                  {
                      &quot;token&quot;: {
                          &quot;$env&quot;: &quot;INSERT_TOKEN&quot;
                      },
                      &quot;actor&quot;: {
                          &quot;bot&quot;: &quot;insert-bot&quot;
                      }
                  }
              ]
          }
      }
  }"><pre>{
      <span class="pl-ent">"plugins"</span>: {
          <span class="pl-ent">"datasette-insert"</span>: {
              <span class="pl-ent">"allow"</span>: {
                  <span class="pl-ent">"bot"</span>: <span class="pl-s"><span class="pl-pds">"</span>insert-bot<span class="pl-pds">"</span></span>
              }
          },
          <span class="pl-ent">"datasette-auth-tokens"</span>: {
              <span class="pl-ent">"tokens"</span>: [
                  {
                      <span class="pl-ent">"token"</span>: {
                          <span class="pl-ent">"$env"</span>: <span class="pl-s"><span class="pl-pds">"</span>INSERT_TOKEN<span class="pl-pds">"</span></span>
                      },
                      <span class="pl-ent">"actor"</span>: {
                          <span class="pl-ent">"bot"</span>: <span class="pl-s"><span class="pl-pds">"</span>insert-bot<span class="pl-pds">"</span></span>
                      }
                  }
              ]
          }
      }
  }</pre></div>
  <p dir="auto">With this configuration in place you can start Datasette like this:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="INSERT_TOKEN=abc123 datasette data.db -m metadata.json"><pre class="notranslate"><code>INSERT_TOKEN=abc123 datasette data.db -m metadata.json
  </code></pre></div>
  <p dir="auto">You can now send data to the API using <code>curl</code> like this:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="curl --request POST \
    -H &quot;Authorization: Bearer abc123&quot; \
    --data '[
        {
          &quot;id&quot;: 3,
          &quot;name&quot;: &quot;Boris&quot;,
          &quot;age&quot;: 1,
          &quot;breed&quot;: &quot;Husky&quot;
        }
      ]' \
      'http://localhost:8001/-/insert/data/dogs'"><pre class="notranslate"><code>curl --request POST \
    -H "Authorization: Bearer abc123" \
    --data '[
        {
          "id": 3,
          "name": "Boris",
          "age": 1,
          "breed": "Husky"
        }
      ]' \
      'http://localhost:8001/-/insert/data/dogs'
  </code></pre></div>
  <p dir="auto">Or using the Python <code>requests</code> library like so:</p>
  <div class="highlight highlight-source-python notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="requests.post(
      &quot;http://localhost:8001/-/insert/data/dogs&quot;,
      json={&quot;id&quot;: 1, &quot;name&quot;: &quot;Cleopaws&quot;, &quot;age&quot;: 5},
      headers={&quot;Authorization&quot;: &quot;bearer abc123&quot;},
  )"><pre><span class="pl-s1">requests</span>.<span class="pl-en">post</span>(
      <span class="pl-s">"http://localhost:8001/-/insert/data/dogs"</span>,
      <span class="pl-s1">json</span><span class="pl-c1">=</span>{<span class="pl-s">"id"</span>: <span class="pl-c1">1</span>, <span class="pl-s">"name"</span>: <span class="pl-s">"Cleopaws"</span>, <span class="pl-s">"age"</span>: <span class="pl-c1">5</span>},
      <span class="pl-s1">headers</span><span class="pl-c1">=</span>{<span class="pl-s">"Authorization"</span>: <span class="pl-s">"bearer abc123"</span>},
  )</pre></div>
  <h3 dir="auto"><a id="user-content-finely-grained-permissions" class="anchor" aria-hidden="true" href="#user-content-finely-grained-permissions"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Finely grained permissions</h3>
  <p dir="auto">Using an <code>"allow"</code> block as described above grants full permission to the features enabled by the API.</p>
  <p dir="auto">The API implements several new Datasett permissions, which other plugins can use to make more finely grained decisions.</p>
  <p dir="auto">The full set of permissions are as follows:</p>
  <ul dir="auto">
  <li><code>insert:all</code> - all permissions - this is used by the <code>"allow"</code> block described above. Argument: <code>database_name</code></li>
  <li><code>insert:insert-update</code> - the ability to insert data into an existing table, or to update data by its primary key. Arguments: <code>(database_name, table_name)</code></li>
  <li><code>insert:create-table</code> - the ability to create a new table. Argument: <code>database_name</code></li>
  <li><code>insert:alter-table</code> - the ability to add columns to an existing table (using <code>?alter=1</code>). Arguments: <code>(database_name, table_name)</code></li>
  </ul>
  <p dir="auto">You can use plugins like <a href="https://github.com/simonw/datasette-permissions-sql">datasette-permissions-sql</a> to hook into these more detailed permissions for finely grained control over what actions each authenticated actor can take.</p>
  <p dir="auto">Plugins that implement the <a href="https://datasette.readthedocs.io/en/stable/plugin_hooks.html#plugin-hook-permission-allowed" rel="nofollow">permission_allowed()</a> plugin hook can take full control over these permission decisions.</p>
  <h2 dir="auto"><a id="user-content-cors" class="anchor" aria-hidden="true" href="#user-content-cors"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>CORS</h2>
  <p dir="auto">If you start Datasette with the <code>datasette --cors</code> option the following HTTP headers will be added to resources served by this plugin:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Access-Control-Allow-Origin: *
  Access-Control-Allow-Headers: content-type,authorization
  Access-Control-Allow-Methods: POST"><pre class="notranslate"><code>Access-Control-Allow-Origin: *
  Access-Control-Allow-Headers: content-type,authorization
  Access-Control-Allow-Methods: POST
  </code></pre></div>
  <h2 dir="auto"><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p dir="auto">To set up this plugin locally, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="cd datasette-insert
  python3 -m venv venv
  source venv/bin/activate"><pre class="notranslate"><code>cd datasette-insert
  python3 -m venv venv
  source venv/bin/activate
  </code></pre></div>
  <p dir="auto">Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="pip install -e '.[test]'"><pre class="notranslate"><code>pip install -e '.[test]'
  </code></pre></div>
  <p dir="auto">To run the tests:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="pytest"><pre class="notranslate"><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyODE0ODEzNDc=
name: datasette-copyable
full_name: simonw/datasette-copyable
html_url: https://github.com/simonw/datasette-copyable
description: Datasette plugin for outputting tables in formats suitable for copy and paste
created_at: 2020-07-21T19:04:08Z
updated_at: 2022-03-26T20:02:45Z
pushed_at: 2022-03-26T20:02:42Z
language: Python
topics: ["datasette", "datasette-io", "datasette-plugin"]
default_branch: main
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
visibility: public
readme:
  # datasette-copyable

  [![PyPI](https://img.shields.io/pypi/v/datasette-copyable.svg)](https://pypi.org/project/datasette-copyable/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-copyable?include_prereleases&label=changelog)](https://github.com/simonw/datasette-copyable/releases)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-copyable/blob/master/LICENSE)

  Datasette plugin for outputting tables in formats suitable for copy and paste

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ pip install datasette-copyable

  ## Demo

  You can try this plugin on [fivethirtyeight.datasettes.com](https://fivethirtyeight.datasettes.com/) - browse for tables or queries there and look for the "copyable" link. Here's an example for a table of [airline safety data](https://fivethirtyeight.datasettes.com/fivethirtyeight/airline-safety~2Fairline-safety.copyable).

  ## Usage

  This plugin adds a `.copyable` output extension to every table, view and query.

  Navigating to this page will show an interface allowing you to select a format for copying and pasting the demo. The default is TSV, which is suitable for copying into Google Sheets or Excel.

  You can add `?_raw=1` to get back just the raw data.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-copyable
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-datasette-copyable" class="anchor" aria-hidden="true" href="#user-content-datasette-copyable"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-copyable</h1>
  <p dir="auto"><a href="https://pypi.org/project/datasette-copyable/" rel="nofollow"><img src="https://camo.githubusercontent.com/b2f6ef49042754455e8b1b28b9f6798720434389c3de9356c1acb9838fae10a8/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d636f707961626c652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-copyable.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-copyable/releases"><img src="https://camo.githubusercontent.com/07698f7d0b34e7b0f7e80f7889418d5c5078a25d72584e1f4e3926d0253b4a32/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d636f707961626c653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-copyable?include_prereleases&amp;label=changelog" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-copyable/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width: 100%;"></a></p>
  <p dir="auto">Datasette plugin for outputting tables in formats suitable for copy and paste</p>
  <h2 dir="auto"><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p dir="auto">Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="$ pip install datasette-copyable"><pre><code>$ pip install datasette-copyable
  </code></pre></div>
  <h2 dir="auto"><a id="user-content-demo" class="anchor" aria-hidden="true" href="#user-content-demo"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Demo</h2>
  <p dir="auto">You can try this plugin on <a href="https://fivethirtyeight.datasettes.com/" rel="nofollow">fivethirtyeight.datasettes.com</a> - browse for tables or queries there and look for the "copyable" link. Here's an example for a table of <a href="https://fivethirtyeight.datasettes.com/fivethirtyeight/airline-safety~2Fairline-safety.copyable" rel="nofollow">airline safety data</a>.</p>
  <h2 dir="auto"><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p dir="auto">This plugin adds a <code>.copyable</code> output extension to every table, view and query.</p>
  <p dir="auto">Navigating to this page will show an interface allowing you to select a format for copying and pasting the demo. The default is TSV, which is suitable for copying into Google Sheets or Excel.</p>
  <p dir="auto">You can add <code>?_raw=1</code> to get back just the raw data.</p>
  <h2 dir="auto"><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p dir="auto">To set up this plugin locally, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="cd datasette-copyable
  python3 -mvenv venv
  source venv/bin/activate"><pre><code>cd datasette-copyable
  python3 -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p dir="auto">Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pipenv shell"><pre><code>pipenv shell
  </code></pre></div>
  <p dir="auto">Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pip install -e '.[test]'"><pre><code>pip install -e '.[test]'
  </code></pre></div>
  <p dir="auto">To run the tests:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pytest"><pre><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyODQzODMyNjU=
name: datasette-graphql
full_name: simonw/datasette-graphql
html_url: https://github.com/simonw/datasette-graphql
description: Datasette plugin providing an automatic GraphQL API for your SQLite databases
created_at: 2020-08-02T03:31:58Z
updated_at: 2022-07-17T02:00:26Z
pushed_at: 2022-07-18T21:13:34Z
homepage: https://datasette-graphql-demo.datasette.io/
language: Python
topics: ["datasette", "datasette-io", "datasette-plugin", "graphql", "sqlite"]
default_branch: main
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
visibility: public
readme:
  # datasette-graphql

  [![PyPI](https://img.shields.io/pypi/v/datasette-graphql.svg)](https://pypi.org/project/datasette-graphql/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-graphql?include_prereleases&label=changelog)](https://github.com/simonw/datasette-graphql/releases)
  [![Tests](https://github.com/simonw/datasette-graphql/workflows/Test/badge.svg)](https://github.com/simonw/datasette-graphql/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-graphql/blob/main/LICENSE)

  **Datasette plugin providing an automatic GraphQL API for your SQLite databases**

  Read more about this project: [GraphQL in Datasette with the new datasette-graphql plugin](https://simonwillison.net/2020/Aug/7/datasette-graphql/)

  Try out a live demo at [datasette-graphql-demo.datasette.io/graphql](https://datasette-graphql-demo.datasette.io/graphql?query=%7B%0A%20%20repos(first%3A10%2C%20search%3A%20%22sql%22%2C%20sort_desc%3A%20created_at)%20%7B%0A%20%20%20%20totalCount%0A%20%20%20%20pageInfo%20%7B%0A%20%20%20%20%20%20endCursor%0A%20%20%20%20%20%20hasNextPage%0A%20%20%20%20%7D%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20description_%0A%20%20%20%20%09stargazers_count%0A%20%20%20%20%20%20created_at%0A%20%20%20%20%20%20owner%20%7B%0A%20%20%20%20%20%20%20%20name%0A%20%20%20%20%20%20%20%20html_url%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A)

  <!-- toc -->

  - [Installation](#installation)
  - [Usage](#usage)
    * [Querying for tables and columns](#querying-for-tables-and-columns)
    * [Fetching a single record](#fetching-a-single-record)
    * [Accessing nested objects](#accessing-nested-objects)
    * [Accessing related objects](#accessing-related-objects)
    * [Filtering tables](#filtering-tables)
    * [Sorting](#sorting)
    * [Pagination](#pagination)
    * [Search](#search)
    * [Columns containing JSON strings](#columns-containing-json-strings)
    * [Auto camelCase](#auto-camelcase)
    * [CORS](#cors)
    * [Execution limits](#execution-limits)
  - [The graphql() template function](#the-graphql-template-function)
  - [Adding custom fields with plugins](#adding-custom-fields-with-plugins)
  - [Development](#development)

  <!-- tocstop -->

  ![GraphiQL animated demo](https://static.simonwillison.net/static/2020/graphiql.gif)

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-graphql

  ## Usage

  This plugin sets up `/graphql` as a GraphQL endpoint for the first attached database.

  If you have multiple attached databases each will get its own endpoint at `/graphql/name_of_database`.

  The automatically generated GraphQL schema is available at `/graphql/name_of_database.graphql` - here's [an example](https://datasette-graphql-demo.datasette.io/graphql/github.graphql).

  ### Querying for tables and columns

  Individual tables (and SQL views) can be queried like this:

  ```graphql
  {
    repos {
      nodes {
        id
        full_name
        description_
      }
    }
  }
  ```
  [Try this query](https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20repos%20%7B%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20id%0A%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20description_%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A)

  In this example query the underlying database table is called `repos` and its columns include `id`, `full_name` and `description`. Since `description` is a reserved word the query needs to ask for `description_` instead.

  ### Fetching a single record

  If you only want to fetch a single record - for example if you want to fetch a row by its primary key - you can use the `tablename_row` field:

  ```graphql
  {
    repos_row(id: 107914493) {
      id
      full_name
      description_
    }
  }
  ```
  [Try this query](https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20repos_row%28id%3A%20107914493%29%20%7B%0A%20%20%20%20id%0A%20%20%20%20full_name%0A%20%20%20%20description_%0A%20%20%7D%0A%7D%0A)

  The `tablename_row` field accepts the primary key column (or columns) as arguments. It also supports the same `filter:`, `search:`, `sort:` and `sort_desc:` arguments as the `tablename` field, described below.

  ### Accessing nested objects

  If a column is a foreign key to another table, you can request columns from the table pointed to by that foreign key using a nested query like this:

  ```graphql
  {
    repos {
      nodes {
        id
        full_name
        owner {
          id
          login
        }
      }
    }
  }
  ```
  [Try this query](https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20repos%20%7B%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20id%0A%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20owner%20%7B%0A%20%20%20%20%20%20%20%20id%0A%20%20%20%20%20%20%20%20login%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A)

  ### Accessing related objects

  If another table has a foreign key back to the table you are accessing, you can fetch rows from that related table.

  Consider a `users` table which is related to `repos` - a repo has a foreign key back to the user that owns the repository. The `users` object type will have a `repos_by_owner_list` field which can be used to access those related repos:

  ```graphql
  {
    users(first: 1, search: "simonw") {
      nodes {
        name
        repos_by_owner_list(first: 5) {
          totalCount
          nodes {
            full_name
          }
        }
      }
    }
  }
  ```
  [Try this query](https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20users%28first%3A%201%2C%20search%3A%20%22simonw%22%29%20%7B%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20name%0A%20%20%20%20%20%20repos_by_owner_list%28first%3A%205%29%20%7B%0A%20%20%20%20%20%20%20%20totalCount%0A%20%20%20%20%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A)


  ### Filtering tables

  You can filter the rows returned for a specific table using the `filter:` argument. This accepts a filter object mapping columns to operations. For example, to return just repositories with the Apache 2 license and more than 10 stars:

  ```graphql
  {
    repos(filter: {license: {eq: "apache-2.0"}, stargazers_count: {gt: 10}}) {
      nodes {
        full_name
        stargazers_count
        license {
          key
        }
      }
    }
  }
  ```
  [Try this query](https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20repos%28filter%3A%20%7Blicense%3A%20%7Beq%3A%20%22apache-2.0%22%7D%2C%20stargazers_count%3A%20%7Bgt%3A%2010%7D%7D%29%20%7B%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20stargazers_count%0A%20%20%20%20%20%20license%20%7B%0A%20%20%20%20%20%20%20%20key%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A)

  See [table filters examples](https://github.com/simonw/datasette-graphql/blob/main/examples/filters.md) for more operations, and [column filter arguments](https://docs.datasette.io/en/stable/json_api.html#column-filter-arguments) in the Datasette documentation for details of how those operations work.

  These same filters can be used on nested relationships, like so:

  ```graphql
  {
    users_row(id: 9599) {
      name
      repos_by_owner_list(filter: {name: {startswith: "datasette-"}}) {
        totalCount
        nodes {
          full_name
        }
      }
    }
  }
  ```
  [Try this query](https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20users_row%28id%3A%209599%29%20%7B%0A%20%20%20%20name%0A%20%20%20%20repos_by_owner_list%28filter%3A%20%7Bname%3A%20%7Bstartswith%3A%20%22datasette-%22%7D%7D%29%20%7B%0A%20%20%20%20%20%20totalCount%0A%20%20%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A)


  The `where:` argument can be used as an alternative to `filter:` when the thing you are expressing is too complex to be modeled using a filter expression. It accepts a string fragment of SQL that will be included in the `WHERE` clause of the SQL query.

  ```graphql
  {
    repos(where: "name='sqlite-utils' or name like 'datasette-%'") {
      totalCount
      nodes {
        full_name
      }
    }
  }
  ```
  [Try this query](https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20repos%28where%3A%20%22name%3D%27sqlite-utils%27%20or%20name%20like%20%27datasette-%25%27%22%29%20%7B%0A%20%20%20%20totalCount%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20full_name%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A)

  ### Sorting

  You can set a sort order for results from a table using the `sort:` or `sort_desc:` arguments. The value for this argument should be the name of the column you wish to sort (or sort-descending) by.

  ```graphql
  {
    repos(sort_desc: stargazers_count) {
      nodes {
        full_name
        stargazers_count
      }
    }
  }
  ```
  [Try this query](https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20repos%28sort_desc%3A%20stargazers_count%29%20%7B%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20stargazers_count%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A)

  ### Pagination

  By default the first 10 rows will be returned. You can control this using the `first:` argument.

  ```graphql
  {
    repos(first: 20) {
      totalCount
      pageInfo {
        hasNextPage
        endCursor
      }
      nodes {
        full_name
        stargazers_count
        license {
          key
        }
      }
    }
  }
  ```
  [Try this query](https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20repos%28first%3A%2020%29%20%7B%0A%20%20%20%20totalCount%0A%20%20%20%20pageInfo%20%7B%0A%20%20%20%20%20%20hasNextPage%0A%20%20%20%20%20%20endCursor%0A%20%20%20%20%7D%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20stargazers_count%0A%20%20%20%20%20%20license%20%7B%0A%20%20%20%20%20%20%20%20key%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A)

  The `totalCount` field returns the total number of records that match the query.

  Requesting the `pageInfo.endCursor` field provides you with the value you need to request the next page. You can pass this to the `after:` argument to request the next page.

  ```graphql
  {
    repos(first: 20, after: "134874019") {
      totalCount
      pageInfo {
        hasNextPage
        endCursor
      }
      nodes {
        full_name
        stargazers_count
        license {
          key
        }
      }
    }
  }
  ```
  [Try this query](https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20repos%28first%3A%2020%2C%20after%3A%20%22134874019%22%29%20%7B%0A%20%20%20%20totalCount%0A%20%20%20%20pageInfo%20%7B%0A%20%20%20%20%20%20hasNextPage%0A%20%20%20%20%20%20endCursor%0A%20%20%20%20%7D%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20stargazers_count%0A%20%20%20%20%20%20license%20%7B%0A%20%20%20%20%20%20%20%20key%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A)

  The `hasNextPage` field tells you if there are any more records.

  ### Search

  If a table has been configured to use SQLite full-text search you can execute searches against it using the `search:` argument:

  ```graphql
  {
    repos(search: "datasette") {
      totalCount
      pageInfo {
        hasNextPage
        endCursor
      }
      nodes {
        full_name
        description_
      }
    }
  }
  ```
  [Try this query](https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20repos%28search%3A%20%22datasette%22%29%20%7B%0A%20%20%20%20totalCount%0A%20%20%20%20pageInfo%20%7B%0A%20%20%20%20%20%20hasNextPage%0A%20%20%20%20%20%20endCursor%0A%20%20%20%20%7D%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20description_%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A)

  The [sqlite-utils](https://sqlite-utils.datasette.io/) Python library and CLI tool can be used to add full-text search to an existing database table.

  ### Columns containing JSON strings

  If your table has a column that contains data encoded as JSON, `datasette-graphql` will make that column available as an encoded JSON string. Clients calling your API will need to parse the string as JSON in order to access the data.

  You can return the data as a nested structure by configuring that column to be treated as a JSON column. The [plugin configuration](https://docs.datasette.io/en/stable/plugins.html#plugin-configuration) for that in `metadata.json` looks like this:

  ```json
  {
      "databases": {
          "test": {
              "tables": {
                  "repos": {
                      "plugins": {
                          "datasette-graphql": {
                              "json_columns": [
                                  "tags"
                              ]
                          }
                      }
                  }
              }
          }
      }
  }
  ```

  ### Auto camelCase

  The names of your columns and tables default to being matched by their representations in GraphQL.

  If you have tables with `names_like_this` you may want to work with them in GraphQL using `namesLikeThis`, for consistency with GraphQL and JavaScript conventions.

  You can turn on automatic camelCase using the `"auto_camelcase"` plugin configuration setting in `metadata.json`, like this:

  ```json
  {
      "plugins": {
          "datasette-graphql": {
              "auto_camelcase": true
          }
      }
  }
  ```

  ### CORS

  This plugin obeys the `--cors` option passed to the `datasette` command-line tool. If you pass `--cors` it adds the following CORS HTTP headers to allow JavaScript running on other domains to access the GraphQL API:

      access-control-allow-headers: content-type
      access-control-allow-method: POST
      access-control-allow-origin: *

  ### Execution limits

  The plugin implements two limits by default:

  - The total time spent executing all of the underlying SQL queries that make up the GraphQL execution must not exceed 1000ms (one second)
  - The total number of SQL table queries executed as a result of nested GraphQL fields must not exceed 100

  These limits can be customized using the `num_queries_limit` and `time_limit_ms` plugin configuration settings, for example in `metadata.json`:

  ```json
  {
      "plugins": {
          "datasette-graphql": {
              "num_queries_limit": 200,
              "time_limit_ms": 5000
          }
      }
  }
  ```
  Setting these to `0` will disable the limit checks entirely.

  ## The graphql() template function

  The plugin also makes a Jinja template function available called `graphql()`. You can use that function in your Datasette [custom templates](https://docs.datasette.io/en/stable/custom_templates.html#custom-templates) like so:

  ```html+jinja
  {% set users = graphql("""
  {
      users {
          nodes {
              name
              points
              score
          }
      }
  }
  """)["users"] %}
  {% for user in users.nodes %}
      <p>{{ user.name }} - points: {{ user.points }}, score = {{ user.score }}</p>
  {% endfor %}
  ```

  The function executes a GraphQL query against the generated schema and returns the results. You can assign those results to a variable in your template and then loop through and display them.

  By default the query will be run against the first attached database. You can use the optional second argument to the function to specify a different database - for example, to run against an attached `github.db` database you would do this:

  ```html+jinja
  {% set user = graphql("""
  {
      users_row(id:9599) {
          name
          login
          avatar_url
      }
  }
  """, "github")["users_row"] %}

  <h1>Hello, {{ user.name }}</h1>
  ```

  You can use [GraphQL variables](https://graphql.org/learn/queries/#variables) in these template calls by passing them to the `variables=` argument:

  ```html+jinja
  {% set user = graphql("""
  query ($id: Int) {
      users_row(id: $id) {
          name
          login
          avatar_url
      }
  }
  """, database="github", variables={"id": 9599})["users_row"] %}

  <h1>Hello, {{ user.name }}</h1>
  ```
  ## Adding custom fields with plugins

  `datasette-graphql` adds a new [plugin hook](https://docs.datasette.io/en/stable/writing_plugins.html) to Datasette which can be used to add custom fields to your GraphQL schema.

  The plugin hook looks like this:

  ```python
  @hookimpl
  def graphql_extra_fields(datasette, database):
      "A list of (name, field_type) tuples to include in the GraphQL schema"
  ```

  You can use this hook to return a list of tuples describing additional fields that should be exposed in your schema. Each tuple should consist of a string naming the new field, plus a [Graphene Field object](https://docs.graphene-python.org/en/latest/types/objecttypes/) that specifies the schema and provides a `resolver` function.

  This example implementation uses `pkg_resources` to return a list of currently installed Python packages:

  ```python
  import graphene
  from datasette import hookimpl
  import pkg_resources


  @hookimpl
  def graphql_extra_fields():
      class Package(graphene.ObjectType):
          "An installed package"
          name = graphene.String()
          version = graphene.String()

      def resolve_packages(root, info):
          return [
              {"name": d.project_name, "version": d.version}
              for d in pkg_resources.working_set
          ]

      return [
          (
              "packages",
              graphene.Field(
                  graphene.List(Package),
                  description="List of installed packages",
                  resolver=resolve_packages,
              ),
          ),
      ]
  ```

  With this plugin installed, the following GraphQL query can be used to retrieve a list of installed packages:

  ```graphql
  {
    packages {
      name
      version
    }
  }
  ```

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-graphql
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-datasette-graphql" class="anchor" aria-hidden="true" href="#user-content-datasette-graphql"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-graphql</h1>
  <p dir="auto"><a href="https://pypi.org/project/datasette-graphql/" rel="nofollow"><img src="https://camo.githubusercontent.com/4f158d7a41cfbd69895c661789cfefe5a60276cfc5beb38a22b93f2f212f045b/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d6772617068716c2e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-graphql.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-graphql/releases"><img src="https://camo.githubusercontent.com/e6f6b7bc6bee887ca763326a05e8c863827a25a8305f3029d2f71222c4168c08/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d6772617068716c3f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-graphql?include_prereleases&amp;label=changelog" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-graphql/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-graphql/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-graphql/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width: 100%;"></a></p>
  <p dir="auto"><strong>Datasette plugin providing an automatic GraphQL API for your SQLite databases</strong></p>
  <p dir="auto">Read more about this project: <a href="https://simonwillison.net/2020/Aug/7/datasette-graphql/" rel="nofollow">GraphQL in Datasette with the new datasette-graphql plugin</a></p>
  <p dir="auto">Try out a live demo at <a href="https://datasette-graphql-demo.datasette.io/graphql?query=%7B%0A%20%20repos(first%3A10%2C%20search%3A%20%22sql%22%2C%20sort_desc%3A%20created_at)%20%7B%0A%20%20%20%20totalCount%0A%20%20%20%20pageInfo%20%7B%0A%20%20%20%20%20%20endCursor%0A%20%20%20%20%20%20hasNextPage%0A%20%20%20%20%7D%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20description_%0A%20%20%20%20%09stargazers_count%0A%20%20%20%20%20%20created_at%0A%20%20%20%20%20%20owner%20%7B%0A%20%20%20%20%20%20%20%20name%0A%20%20%20%20%20%20%20%20html_url%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A" rel="nofollow">datasette-graphql-demo.datasette.io/graphql</a></p>

  <ul dir="auto">
  <li><a href="#user-content-installation">Installation</a></li>
  <li><a href="#user-content-usage">Usage</a>
  <ul dir="auto">
  <li><a href="#user-content-querying-for-tables-and-columns">Querying for tables and columns</a></li>
  <li><a href="#user-content-fetching-a-single-record">Fetching a single record</a></li>
  <li><a href="#user-content-accessing-nested-objects">Accessing nested objects</a></li>
  <li><a href="#user-content-accessing-related-objects">Accessing related objects</a></li>
  <li><a href="#user-content-filtering-tables">Filtering tables</a></li>
  <li><a href="#user-content-sorting">Sorting</a></li>
  <li><a href="#user-content-pagination">Pagination</a></li>
  <li><a href="#user-content-search">Search</a></li>
  <li><a href="#user-content-columns-containing-json-strings">Columns containing JSON strings</a></li>
  <li><a href="#user-content-auto-camelcase">Auto camelCase</a></li>
  <li><a href="#user-content-cors">CORS</a></li>
  <li><a href="#user-content-execution-limits">Execution limits</a></li>
  </ul>
  </li>
  <li><a href="#user-content-the-graphql-template-function">The graphql() template function</a></li>
  <li><a href="#user-content-adding-custom-fields-with-plugins">Adding custom fields with plugins</a></li>
  <li><a href="#user-content-development">Development</a></li>
  </ul>

  <p dir="auto"><a target="_blank" rel="noopener noreferrer" href="https://camo.githubusercontent.com/55315121dd4aa360e7be736ada3e0bf144de0274e5a01a6430747ed2c304cb00/68747470733a2f2f7374617469632e73696d6f6e77696c6c69736f6e2e6e65742f7374617469632f323032302f677261706869716c2e676966"><img src="https://camo.githubusercontent.com/55315121dd4aa360e7be736ada3e0bf144de0274e5a01a6430747ed2c304cb00/68747470733a2f2f7374617469632e73696d6f6e77696c6c69736f6e2e6e65742f7374617469632f323032302f677261706869716c2e676966" alt="GraphiQL animated demo" data-animated-image="" data-canonical-src="https://static.simonwillison.net/static/2020/graphiql.gif" style="max-width: 100%;"></a></p>
  <h2 dir="auto"><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p dir="auto">Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="$ datasette install datasette-graphql"><pre class="notranslate"><code>$ datasette install datasette-graphql
  </code></pre></div>
  <h2 dir="auto"><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p dir="auto">This plugin sets up <code>/graphql</code> as a GraphQL endpoint for the first attached database.</p>
  <p dir="auto">If you have multiple attached databases each will get its own endpoint at <code>/graphql/name_of_database</code>.</p>
  <p dir="auto">The automatically generated GraphQL schema is available at <code>/graphql/name_of_database.graphql</code> - here's <a href="https://datasette-graphql-demo.datasette.io/graphql/github.graphql" rel="nofollow">an example</a>.</p>
  <h3 dir="auto"><a id="user-content-querying-for-tables-and-columns" class="anchor" aria-hidden="true" href="#user-content-querying-for-tables-and-columns"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Querying for tables and columns</h3>
  <p dir="auto">Individual tables (and SQL views) can be queried like this:</p>
  <div class="highlight highlight-source-graphql notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="{
    repos {
      nodes {
        id
        full_name
        description_
      }
    }
  }"><pre>{
    <span class="pl-v">repos</span> {
      <span class="pl-v">nodes</span> {
        <span class="pl-v">id</span>
        <span class="pl-v">full_name</span>
        <span class="pl-v">description_</span>
      }
    }
  }</pre></div>
  <p dir="auto"><a href="https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20repos%20%7B%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20id%0A%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20description_%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A" rel="nofollow">Try this query</a></p>
  <p dir="auto">In this example query the underlying database table is called <code>repos</code> and its columns include <code>id</code>, <code>full_name</code> and <code>description</code>. Since <code>description</code> is a reserved word the query needs to ask for <code>description_</code> instead.</p>
  <h3 dir="auto"><a id="user-content-fetching-a-single-record" class="anchor" aria-hidden="true" href="#user-content-fetching-a-single-record"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Fetching a single record</h3>
  <p dir="auto">If you only want to fetch a single record - for example if you want to fetch a row by its primary key - you can use the <code>tablename_row</code> field:</p>
  <div class="highlight highlight-source-graphql notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="{
    repos_row(id: 107914493) {
      id
      full_name
      description_
    }
  }"><pre>{
    <span class="pl-v">repos_row</span>(<span class="pl-v">id</span>: <span class="pl-c1">107914493</span>) {
      <span class="pl-v">id</span>
      <span class="pl-v">full_name</span>
      <span class="pl-v">description_</span>
    }
  }</pre></div>
  <p dir="auto"><a href="https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20repos_row%28id%3A%20107914493%29%20%7B%0A%20%20%20%20id%0A%20%20%20%20full_name%0A%20%20%20%20description_%0A%20%20%7D%0A%7D%0A" rel="nofollow">Try this query</a></p>
  <p dir="auto">The <code>tablename_row</code> field accepts the primary key column (or columns) as arguments. It also supports the same <code>filter:</code>, <code>search:</code>, <code>sort:</code> and <code>sort_desc:</code> arguments as the <code>tablename</code> field, described below.</p>
  <h3 dir="auto"><a id="user-content-accessing-nested-objects" class="anchor" aria-hidden="true" href="#user-content-accessing-nested-objects"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Accessing nested objects</h3>
  <p dir="auto">If a column is a foreign key to another table, you can request columns from the table pointed to by that foreign key using a nested query like this:</p>
  <div class="highlight highlight-source-graphql notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="{
    repos {
      nodes {
        id
        full_name
        owner {
          id
          login
        }
      }
    }
  }"><pre>{
    <span class="pl-v">repos</span> {
      <span class="pl-v">nodes</span> {
        <span class="pl-v">id</span>
        <span class="pl-v">full_name</span>
        <span class="pl-v">owner</span> {
          <span class="pl-v">id</span>
          <span class="pl-v">login</span>
        }
      }
    }
  }</pre></div>
  <p dir="auto"><a href="https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20repos%20%7B%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20id%0A%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20owner%20%7B%0A%20%20%20%20%20%20%20%20id%0A%20%20%20%20%20%20%20%20login%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A" rel="nofollow">Try this query</a></p>
  <h3 dir="auto"><a id="user-content-accessing-related-objects" class="anchor" aria-hidden="true" href="#user-content-accessing-related-objects"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Accessing related objects</h3>
  <p dir="auto">If another table has a foreign key back to the table you are accessing, you can fetch rows from that related table.</p>
  <p dir="auto">Consider a <code>users</code> table which is related to <code>repos</code> - a repo has a foreign key back to the user that owns the repository. The <code>users</code> object type will have a <code>repos_by_owner_list</code> field which can be used to access those related repos:</p>
  <div class="highlight highlight-source-graphql notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="{
    users(first: 1, search: &quot;simonw&quot;) {
      nodes {
        name
        repos_by_owner_list(first: 5) {
          totalCount
          nodes {
            full_name
          }
        }
      }
    }
  }"><pre>{
    <span class="pl-v">users</span>(<span class="pl-v">first</span>: <span class="pl-c1">1</span>, <span class="pl-v">search</span>: <span class="pl-s"><span class="pl-pds">"</span>simonw<span class="pl-pds">"</span></span>) {
      <span class="pl-v">nodes</span> {
        <span class="pl-v">name</span>
        <span class="pl-v">repos_by_owner_list</span>(<span class="pl-v">first</span>: <span class="pl-c1">5</span>) {
          <span class="pl-v">totalCount</span>
          <span class="pl-v">nodes</span> {
            <span class="pl-v">full_name</span>
          }
        }
      }
    }
  }</pre></div>
  <p dir="auto"><a href="https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20users%28first%3A%201%2C%20search%3A%20%22simonw%22%29%20%7B%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20name%0A%20%20%20%20%20%20repos_by_owner_list%28first%3A%205%29%20%7B%0A%20%20%20%20%20%20%20%20totalCount%0A%20%20%20%20%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A" rel="nofollow">Try this query</a></p>
  <h3 dir="auto"><a id="user-content-filtering-tables" class="anchor" aria-hidden="true" href="#user-content-filtering-tables"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Filtering tables</h3>
  <p dir="auto">You can filter the rows returned for a specific table using the <code>filter:</code> argument. This accepts a filter object mapping columns to operations. For example, to return just repositories with the Apache 2 license and more than 10 stars:</p>
  <div class="highlight highlight-source-graphql notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="{
    repos(filter: {license: {eq: &quot;apache-2.0&quot;}, stargazers_count: {gt: 10}}) {
      nodes {
        full_name
        stargazers_count
        license {
          key
        }
      }
    }
  }"><pre>{
    <span class="pl-v">repos</span>(<span class="pl-v">filter</span>: {<span class="pl-c1"><span class="pl-s">license</span></span>: {<span class="pl-c1"><span class="pl-s">eq</span></span>: <span class="pl-s"><span class="pl-pds">"</span>apache-2.0<span class="pl-pds">"</span></span>}, <span class="pl-c1"><span class="pl-s">stargazers_count</span></span>: {<span class="pl-c1"><span class="pl-s">gt</span></span>: <span class="pl-c1">10</span>}}) {
      <span class="pl-v">nodes</span> {
        <span class="pl-v">full_name</span>
        <span class="pl-v">stargazers_count</span>
        <span class="pl-v">license</span> {
          <span class="pl-v">key</span>
        }
      }
    }
  }</pre></div>
  <p dir="auto"><a href="https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20repos%28filter%3A%20%7Blicense%3A%20%7Beq%3A%20%22apache-2.0%22%7D%2C%20stargazers_count%3A%20%7Bgt%3A%2010%7D%7D%29%20%7B%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20stargazers_count%0A%20%20%20%20%20%20license%20%7B%0A%20%20%20%20%20%20%20%20key%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A" rel="nofollow">Try this query</a></p>
  <p dir="auto">See <a href="https://github.com/simonw/datasette-graphql/blob/main/examples/filters.md">table filters examples</a> for more operations, and <a href="https://docs.datasette.io/en/stable/json_api.html#column-filter-arguments" rel="nofollow">column filter arguments</a> in the Datasette documentation for details of how those operations work.</p>
  <p dir="auto">These same filters can be used on nested relationships, like so:</p>
  <div class="highlight highlight-source-graphql notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="{
    users_row(id: 9599) {
      name
      repos_by_owner_list(filter: {name: {startswith: &quot;datasette-&quot;}}) {
        totalCount
        nodes {
          full_name
        }
      }
    }
  }"><pre>{
    <span class="pl-v">users_row</span>(<span class="pl-v">id</span>: <span class="pl-c1">9599</span>) {
      <span class="pl-v">name</span>
      <span class="pl-v">repos_by_owner_list</span>(<span class="pl-v">filter</span>: {<span class="pl-c1"><span class="pl-s">name</span></span>: {<span class="pl-c1"><span class="pl-s">startswith</span></span>: <span class="pl-s"><span class="pl-pds">"</span>datasette-<span class="pl-pds">"</span></span>}}) {
        <span class="pl-v">totalCount</span>
        <span class="pl-v">nodes</span> {
          <span class="pl-v">full_name</span>
        }
      }
    }
  }</pre></div>
  <p dir="auto"><a href="https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20users_row%28id%3A%209599%29%20%7B%0A%20%20%20%20name%0A%20%20%20%20repos_by_owner_list%28filter%3A%20%7Bname%3A%20%7Bstartswith%3A%20%22datasette-%22%7D%7D%29%20%7B%0A%20%20%20%20%20%20totalCount%0A%20%20%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A" rel="nofollow">Try this query</a></p>
  <p dir="auto">The <code>where:</code> argument can be used as an alternative to <code>filter:</code> when the thing you are expressing is too complex to be modeled using a filter expression. It accepts a string fragment of SQL that will be included in the <code>WHERE</code> clause of the SQL query.</p>
  <div class="highlight highlight-source-graphql notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="{
    repos(where: &quot;name='sqlite-utils' or name like 'datasette-%'&quot;) {
      totalCount
      nodes {
        full_name
      }
    }
  }"><pre>{
    <span class="pl-v">repos</span>(<span class="pl-v">where</span>: <span class="pl-s"><span class="pl-pds">"</span>name='sqlite-utils' or name like 'datasette-%'<span class="pl-pds">"</span></span>) {
      <span class="pl-v">totalCount</span>
      <span class="pl-v">nodes</span> {
        <span class="pl-v">full_name</span>
      }
    }
  }</pre></div>
  <p dir="auto"><a href="https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20repos%28where%3A%20%22name%3D%27sqlite-utils%27%20or%20name%20like%20%27datasette-%25%27%22%29%20%7B%0A%20%20%20%20totalCount%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20full_name%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A" rel="nofollow">Try this query</a></p>
  <h3 dir="auto"><a id="user-content-sorting" class="anchor" aria-hidden="true" href="#user-content-sorting"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Sorting</h3>
  <p dir="auto">You can set a sort order for results from a table using the <code>sort:</code> or <code>sort_desc:</code> arguments. The value for this argument should be the name of the column you wish to sort (or sort-descending) by.</p>
  <div class="highlight highlight-source-graphql notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="{
    repos(sort_desc: stargazers_count) {
      nodes {
        full_name
        stargazers_count
      }
    }
  }"><pre>{
    <span class="pl-v">repos</span>(<span class="pl-v">sort_desc</span>:<span class="pl-c1"> stargazers_count</span>) {
      <span class="pl-v">nodes</span> {
        <span class="pl-v">full_name</span>
        <span class="pl-v">stargazers_count</span>
      }
    }
  }</pre></div>
  <p dir="auto"><a href="https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20repos%28sort_desc%3A%20stargazers_count%29%20%7B%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20stargazers_count%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A" rel="nofollow">Try this query</a></p>
  <h3 dir="auto"><a id="user-content-pagination" class="anchor" aria-hidden="true" href="#user-content-pagination"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Pagination</h3>
  <p dir="auto">By default the first 10 rows will be returned. You can control this using the <code>first:</code> argument.</p>
  <div class="highlight highlight-source-graphql notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="{
    repos(first: 20) {
      totalCount
      pageInfo {
        hasNextPage
        endCursor
      }
      nodes {
        full_name
        stargazers_count
        license {
          key
        }
      }
    }
  }"><pre>{
    <span class="pl-v">repos</span>(<span class="pl-v">first</span>: <span class="pl-c1">20</span>) {
      <span class="pl-v">totalCount</span>
      <span class="pl-v">pageInfo</span> {
        <span class="pl-v">hasNextPage</span>
        <span class="pl-v">endCursor</span>
      }
      <span class="pl-v">nodes</span> {
        <span class="pl-v">full_name</span>
        <span class="pl-v">stargazers_count</span>
        <span class="pl-v">license</span> {
          <span class="pl-v">key</span>
        }
      }
    }
  }</pre></div>
  <p dir="auto"><a href="https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20repos%28first%3A%2020%29%20%7B%0A%20%20%20%20totalCount%0A%20%20%20%20pageInfo%20%7B%0A%20%20%20%20%20%20hasNextPage%0A%20%20%20%20%20%20endCursor%0A%20%20%20%20%7D%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20stargazers_count%0A%20%20%20%20%20%20license%20%7B%0A%20%20%20%20%20%20%20%20key%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A" rel="nofollow">Try this query</a></p>
  <p dir="auto">The <code>totalCount</code> field returns the total number of records that match the query.</p>
  <p dir="auto">Requesting the <code>pageInfo.endCursor</code> field provides you with the value you need to request the next page. You can pass this to the <code>after:</code> argument to request the next page.</p>
  <div class="highlight highlight-source-graphql notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="{
    repos(first: 20, after: &quot;134874019&quot;) {
      totalCount
      pageInfo {
        hasNextPage
        endCursor
      }
      nodes {
        full_name
        stargazers_count
        license {
          key
        }
      }
    }
  }"><pre>{
    <span class="pl-v">repos</span>(<span class="pl-v">first</span>: <span class="pl-c1">20</span>, <span class="pl-v">after</span>: <span class="pl-s"><span class="pl-pds">"</span>134874019<span class="pl-pds">"</span></span>) {
      <span class="pl-v">totalCount</span>
      <span class="pl-v">pageInfo</span> {
        <span class="pl-v">hasNextPage</span>
        <span class="pl-v">endCursor</span>
      }
      <span class="pl-v">nodes</span> {
        <span class="pl-v">full_name</span>
        <span class="pl-v">stargazers_count</span>
        <span class="pl-v">license</span> {
          <span class="pl-v">key</span>
        }
      }
    }
  }</pre></div>
  <p dir="auto"><a href="https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20repos%28first%3A%2020%2C%20after%3A%20%22134874019%22%29%20%7B%0A%20%20%20%20totalCount%0A%20%20%20%20pageInfo%20%7B%0A%20%20%20%20%20%20hasNextPage%0A%20%20%20%20%20%20endCursor%0A%20%20%20%20%7D%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20stargazers_count%0A%20%20%20%20%20%20license%20%7B%0A%20%20%20%20%20%20%20%20key%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A" rel="nofollow">Try this query</a></p>
  <p dir="auto">The <code>hasNextPage</code> field tells you if there are any more records.</p>
  <h3 dir="auto"><a id="user-content-search" class="anchor" aria-hidden="true" href="#user-content-search"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Search</h3>
  <p dir="auto">If a table has been configured to use SQLite full-text search you can execute searches against it using the <code>search:</code> argument:</p>
  <div class="highlight highlight-source-graphql notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="{
    repos(search: &quot;datasette&quot;) {
      totalCount
      pageInfo {
        hasNextPage
        endCursor
      }
      nodes {
        full_name
        description_
      }
    }
  }"><pre>{
    <span class="pl-v">repos</span>(<span class="pl-v">search</span>: <span class="pl-s"><span class="pl-pds">"</span>datasette<span class="pl-pds">"</span></span>) {
      <span class="pl-v">totalCount</span>
      <span class="pl-v">pageInfo</span> {
        <span class="pl-v">hasNextPage</span>
        <span class="pl-v">endCursor</span>
      }
      <span class="pl-v">nodes</span> {
        <span class="pl-v">full_name</span>
        <span class="pl-v">description_</span>
      }
    }
  }</pre></div>
  <p dir="auto"><a href="https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20repos%28search%3A%20%22datasette%22%29%20%7B%0A%20%20%20%20totalCount%0A%20%20%20%20pageInfo%20%7B%0A%20%20%20%20%20%20hasNextPage%0A%20%20%20%20%20%20endCursor%0A%20%20%20%20%7D%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20description_%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A" rel="nofollow">Try this query</a></p>
  <p dir="auto">The <a href="https://sqlite-utils.datasette.io/" rel="nofollow">sqlite-utils</a> Python library and CLI tool can be used to add full-text search to an existing database table.</p>
  <h3 dir="auto"><a id="user-content-columns-containing-json-strings" class="anchor" aria-hidden="true" href="#user-content-columns-containing-json-strings"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Columns containing JSON strings</h3>
  <p dir="auto">If your table has a column that contains data encoded as JSON, <code>datasette-graphql</code> will make that column available as an encoded JSON string. Clients calling your API will need to parse the string as JSON in order to access the data.</p>
  <p dir="auto">You can return the data as a nested structure by configuring that column to be treated as a JSON column. The <a href="https://docs.datasette.io/en/stable/plugins.html#plugin-configuration" rel="nofollow">plugin configuration</a> for that in <code>metadata.json</code> looks like this:</p>
  <div class="highlight highlight-source-json notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="{
      &quot;databases&quot;: {
          &quot;test&quot;: {
              &quot;tables&quot;: {
                  &quot;repos&quot;: {
                      &quot;plugins&quot;: {
                          &quot;datasette-graphql&quot;: {
                              &quot;json_columns&quot;: [
                                  &quot;tags&quot;
                              ]
                          }
                      }
                  }
              }
          }
      }
  }"><pre>{
      <span class="pl-ent">"databases"</span>: {
          <span class="pl-ent">"test"</span>: {
              <span class="pl-ent">"tables"</span>: {
                  <span class="pl-ent">"repos"</span>: {
                      <span class="pl-ent">"plugins"</span>: {
                          <span class="pl-ent">"datasette-graphql"</span>: {
                              <span class="pl-ent">"json_columns"</span>: [
                                  <span class="pl-s"><span class="pl-pds">"</span>tags<span class="pl-pds">"</span></span>
                              ]
                          }
                      }
                  }
              }
          }
      }
  }</pre></div>
  <h3 dir="auto"><a id="user-content-auto-camelcase" class="anchor" aria-hidden="true" href="#user-content-auto-camelcase"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Auto camelCase</h3>
  <p dir="auto">The names of your columns and tables default to being matched by their representations in GraphQL.</p>
  <p dir="auto">If you have tables with <code>names_like_this</code> you may want to work with them in GraphQL using <code>namesLikeThis</code>, for consistency with GraphQL and JavaScript conventions.</p>
  <p dir="auto">You can turn on automatic camelCase using the <code>"auto_camelcase"</code> plugin configuration setting in <code>metadata.json</code>, like this:</p>
  <div class="highlight highlight-source-json notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="{
      &quot;plugins&quot;: {
          &quot;datasette-graphql&quot;: {
              &quot;auto_camelcase&quot;: true
          }
      }
  }"><pre>{
      <span class="pl-ent">"plugins"</span>: {
          <span class="pl-ent">"datasette-graphql"</span>: {
              <span class="pl-ent">"auto_camelcase"</span>: <span class="pl-c1">true</span>
          }
      }
  }</pre></div>
  <h3 dir="auto"><a id="user-content-cors" class="anchor" aria-hidden="true" href="#user-content-cors"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>CORS</h3>
  <p dir="auto">This plugin obeys the <code>--cors</code> option passed to the <code>datasette</code> command-line tool. If you pass <code>--cors</code> it adds the following CORS HTTP headers to allow JavaScript running on other domains to access the GraphQL API:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="access-control-allow-headers: content-type
  access-control-allow-method: POST
  access-control-allow-origin: *"><pre class="notranslate"><code>access-control-allow-headers: content-type
  access-control-allow-method: POST
  access-control-allow-origin: *
  </code></pre></div>
  <h3 dir="auto"><a id="user-content-execution-limits" class="anchor" aria-hidden="true" href="#user-content-execution-limits"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Execution limits</h3>
  <p dir="auto">The plugin implements two limits by default:</p>
  <ul dir="auto">
  <li>The total time spent executing all of the underlying SQL queries that make up the GraphQL execution must not exceed 1000ms (one second)</li>
  <li>The total number of SQL table queries executed as a result of nested GraphQL fields must not exceed 100</li>
  </ul>
  <p dir="auto">These limits can be customized using the <code>num_queries_limit</code> and <code>time_limit_ms</code> plugin configuration settings, for example in <code>metadata.json</code>:</p>
  <div class="highlight highlight-source-json notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="{
      &quot;plugins&quot;: {
          &quot;datasette-graphql&quot;: {
              &quot;num_queries_limit&quot;: 200,
              &quot;time_limit_ms&quot;: 5000
          }
      }
  }"><pre>{
      <span class="pl-ent">"plugins"</span>: {
          <span class="pl-ent">"datasette-graphql"</span>: {
              <span class="pl-ent">"num_queries_limit"</span>: <span class="pl-c1">200</span>,
              <span class="pl-ent">"time_limit_ms"</span>: <span class="pl-c1">5000</span>
          }
      }
  }</pre></div>
  <p dir="auto">Setting these to <code>0</code> will disable the limit checks entirely.</p>
  <h2 dir="auto"><a id="user-content-the-graphql-template-function" class="anchor" aria-hidden="true" href="#user-content-the-graphql-template-function"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>The graphql() template function</h2>
  <p dir="auto">The plugin also makes a Jinja template function available called <code>graphql()</code>. You can use that function in your Datasette <a href="https://docs.datasette.io/en/stable/custom_templates.html#custom-templates" rel="nofollow">custom templates</a> like so:</p>
  <div class="highlight highlight-text-html-django notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="{% set users = graphql(&quot;&quot;&quot;
  {
      users {
          nodes {
              name
              points
              score
          }
      }
  }
  &quot;&quot;&quot;)[&quot;users&quot;] %}
  {% for user in users.nodes %}
      &lt;p&gt;{{ user.name }} - points: {{ user.points }}, score = {{ user.score }}&lt;/p&gt;
  {% endfor %}"><pre><span class="pl-e">{%</span> <span class="pl-s">set</span> <span class="pl-s">users</span> = <span class="pl-s">graphql</span>(<span class="pl-s">"""</span>
  <span class="pl-s">{</span>
  <span class="pl-s">    users {</span>
  <span class="pl-s">        nodes {</span>
  <span class="pl-s">            name</span>
  <span class="pl-s">            points</span>
  <span class="pl-s">            score</span>
  <span class="pl-s">        }</span>
  <span class="pl-s">    }</span>
  <span class="pl-s">}</span>
  <span class="pl-s">"""</span>)[<span class="pl-s">"users"</span>] <span class="pl-e">%}</span>
  <span class="pl-e">{%</span> <span class="pl-k">for</span> <span class="pl-s">user</span> <span class="pl-k">in</span> <span class="pl-s">users</span>.<span class="pl-s">nodes</span> <span class="pl-e">%}</span>
      &lt;<span class="pl-ent">p</span>&gt;{{ user.name }} - points: {{ user.points }}, score = {{ user.score }}&lt;/<span class="pl-ent">p</span>&gt;
  <span class="pl-e">{%</span> <span class="pl-k">endfor</span> <span class="pl-e">%}</span></pre></div>
  <p dir="auto">The function executes a GraphQL query against the generated schema and returns the results. You can assign those results to a variable in your template and then loop through and display them.</p>
  <p dir="auto">By default the query will be run against the first attached database. You can use the optional second argument to the function to specify a different database - for example, to run against an attached <code>github.db</code> database you would do this:</p>
  <div class="highlight highlight-text-html-django notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="{% set user = graphql(&quot;&quot;&quot;
  {
      users_row(id:9599) {
          name
          login
          avatar_url
      }
  }
  &quot;&quot;&quot;, &quot;github&quot;)[&quot;users_row&quot;] %}

  &lt;h1&gt;Hello, {{ user.name }}&lt;/h1&gt;"><pre><span class="pl-e">{%</span> <span class="pl-s">set</span> <span class="pl-s">user</span> = <span class="pl-s">graphql</span>(<span class="pl-s">"""</span>
  <span class="pl-s">{</span>
  <span class="pl-s">    users_row(id:9599) {</span>
  <span class="pl-s">        name</span>
  <span class="pl-s">        login</span>
  <span class="pl-s">        avatar_url</span>
  <span class="pl-s">    }</span>
  <span class="pl-s">}</span>
  <span class="pl-s">"""</span>, <span class="pl-s">"github"</span>)[<span class="pl-s">"users_row"</span>] <span class="pl-e">%}</span>

  &lt;<span class="pl-ent">h1</span>&gt;Hello, {{ user.name }}&lt;/<span class="pl-ent">h1</span>&gt;</pre></div>
  <p dir="auto">You can use <a href="https://graphql.org/learn/queries/#variables" rel="nofollow">GraphQL variables</a> in these template calls by passing them to the <code>variables=</code> argument:</p>
  <div class="highlight highlight-text-html-django notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="{% set user = graphql(&quot;&quot;&quot;
  query ($id: Int) {
      users_row(id: $id) {
          name
          login
          avatar_url
      }
  }
  &quot;&quot;&quot;, database=&quot;github&quot;, variables={&quot;id&quot;: 9599})[&quot;users_row&quot;] %}

  &lt;h1&gt;Hello, {{ user.name }}&lt;/h1&gt;"><pre><span class="pl-e">{%</span> <span class="pl-s">set</span> <span class="pl-s">user</span> = <span class="pl-s">graphql</span>(<span class="pl-s">"""</span>
  <span class="pl-s">query ($id: Int) {</span>
  <span class="pl-s">    users_row(id: $id) {</span>
  <span class="pl-s">        name</span>
  <span class="pl-s">        login</span>
  <span class="pl-s">        avatar_url</span>
  <span class="pl-s">    }</span>
  <span class="pl-s">}</span>
  <span class="pl-s">"""</span>, <span class="pl-s">database</span>=<span class="pl-s">"github"</span>, <span class="pl-s">variables</span>={<span class="pl-s">"id"</span>: 9599})[<span class="pl-s">"users_row"</span>] <span class="pl-e">%}</span>

  &lt;<span class="pl-ent">h1</span>&gt;Hello, {{ user.name }}&lt;/<span class="pl-ent">h1</span>&gt;</pre></div>
  <h2 dir="auto"><a id="user-content-adding-custom-fields-with-plugins" class="anchor" aria-hidden="true" href="#user-content-adding-custom-fields-with-plugins"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Adding custom fields with plugins</h2>
  <p dir="auto"><code>datasette-graphql</code> adds a new <a href="https://docs.datasette.io/en/stable/writing_plugins.html" rel="nofollow">plugin hook</a> to Datasette which can be used to add custom fields to your GraphQL schema.</p>
  <p dir="auto">The plugin hook looks like this:</p>
  <div class="highlight highlight-source-python notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="@hookimpl
  def graphql_extra_fields(datasette, database):
      &quot;A list of (name, field_type) tuples to include in the GraphQL schema&quot;"><pre><span class="pl-en">@<span class="pl-s1">hookimpl</span></span>
  <span class="pl-k">def</span> <span class="pl-en">graphql_extra_fields</span>(<span class="pl-s1">datasette</span>, <span class="pl-s1">database</span>):
      <span class="pl-s">"A list of (name, field_type) tuples to include in the GraphQL schema"</span></pre></div>
  <p dir="auto">You can use this hook to return a list of tuples describing additional fields that should be exposed in your schema. Each tuple should consist of a string naming the new field, plus a <a href="https://docs.graphene-python.org/en/latest/types/objecttypes/" rel="nofollow">Graphene Field object</a> that specifies the schema and provides a <code>resolver</code> function.</p>
  <p dir="auto">This example implementation uses <code>pkg_resources</code> to return a list of currently installed Python packages:</p>
  <div class="highlight highlight-source-python notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import graphene
  from datasette import hookimpl
  import pkg_resources


  @hookimpl
  def graphql_extra_fields():
      class Package(graphene.ObjectType):
          &quot;An installed package&quot;
          name = graphene.String()
          version = graphene.String()

      def resolve_packages(root, info):
          return [
              {&quot;name&quot;: d.project_name, &quot;version&quot;: d.version}
              for d in pkg_resources.working_set
          ]

      return [
          (
              &quot;packages&quot;,
              graphene.Field(
                  graphene.List(Package),
                  description=&quot;List of installed packages&quot;,
                  resolver=resolve_packages,
              ),
          ),
      ]"><pre><span class="pl-k">import</span> <span class="pl-s1">graphene</span>
  <span class="pl-k">from</span> <span class="pl-s1">datasette</span> <span class="pl-k">import</span> <span class="pl-s1">hookimpl</span>
  <span class="pl-k">import</span> <span class="pl-s1">pkg_resources</span>


  <span class="pl-en">@<span class="pl-s1">hookimpl</span></span>
  <span class="pl-k">def</span> <span class="pl-en">graphql_extra_fields</span>():
      <span class="pl-k">class</span> <span class="pl-v">Package</span>(<span class="pl-s1">graphene</span>.<span class="pl-v">ObjectType</span>):
          <span class="pl-s">"An installed package"</span>
          <span class="pl-s1">name</span> <span class="pl-c1">=</span> <span class="pl-s1">graphene</span>.<span class="pl-v">String</span>()
          <span class="pl-s1">version</span> <span class="pl-c1">=</span> <span class="pl-s1">graphene</span>.<span class="pl-v">String</span>()

      <span class="pl-k">def</span> <span class="pl-en">resolve_packages</span>(<span class="pl-s1">root</span>, <span class="pl-s1">info</span>):
          <span class="pl-k">return</span> [
              {<span class="pl-s">"name"</span>: <span class="pl-s1">d</span>.<span class="pl-s1">project_name</span>, <span class="pl-s">"version"</span>: <span class="pl-s1">d</span>.<span class="pl-s1">version</span>}
              <span class="pl-k">for</span> <span class="pl-s1">d</span> <span class="pl-c1">in</span> <span class="pl-s1">pkg_resources</span>.<span class="pl-s1">working_set</span>
          ]

      <span class="pl-k">return</span> [
          (
              <span class="pl-s">"packages"</span>,
              <span class="pl-s1">graphene</span>.<span class="pl-v">Field</span>(
                  <span class="pl-s1">graphene</span>.<span class="pl-v">List</span>(<span class="pl-v">Package</span>),
                  <span class="pl-s1">description</span><span class="pl-c1">=</span><span class="pl-s">"List of installed packages"</span>,
                  <span class="pl-s1">resolver</span><span class="pl-c1">=</span><span class="pl-s1">resolve_packages</span>,
              ),
          ),
      ]</pre></div>
  <p dir="auto">With this plugin installed, the following GraphQL query can be used to retrieve a list of installed packages:</p>
  <div class="highlight highlight-source-graphql notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="{
    packages {
      name
      version
    }
  }"><pre>{
    <span class="pl-v">packages</span> {
      <span class="pl-v">name</span>
      <span class="pl-v">version</span>
    }
  }</pre></div>
  <h2 dir="auto"><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p dir="auto">To set up this plugin locally, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="cd datasette-graphql
  python3 -mvenv venv
  source venv/bin/activate"><pre class="notranslate"><code>cd datasette-graphql
  python3 -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p dir="auto">Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="pipenv shell"><pre class="notranslate"><code>pipenv shell
  </code></pre></div>
  <p dir="auto">Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="pip install -e '.[test]'"><pre class="notranslate"><code>pip install -e '.[test]'
  </code></pre></div>
  <p dir="auto">To run the tests:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="pytest"><pre class="notranslate"><code>pytest
  </code></pre></div>
  </article></div>


