# repos

node_id: MDEwOlJlcG9zaXRvcnkyODg2Mjk3NjY=
name: datasette-schema-versions
full_name: simonw/datasette-schema-versions
html_url: https://github.com/simonw/datasette-schema-versions
description: Datasette plugin that shows the schema version of every attached database
created_at: 2020-08-19T04:04:39Z
updated_at: 2021-09-11T02:42:37Z
pushed_at: 2021-09-11T02:44:32Z
language: Python
topics: ["datasette", "datasette-io", "datasette-plugin"]
default_branch: main
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
readme:
  # datasette-schema-versions

  [![PyPI](https://img.shields.io/pypi/v/datasette-schema-versions.svg)](https://pypi.org/project/datasette-schema-versions/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-schema-versions?include_prereleases&label=changelog)](https://github.com/simonw/datasette-schema-versions/releases)
  [![Tests](https://github.com/simonw/datasette-schema-versions/workflows/Test/badge.svg)](https://github.com/simonw/datasette-schema-versions/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-schema-versions/blob/main/LICENSE)

  Datasette plugin that shows the schema version of every attached database

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-schema-versions

  ## Usage

  Visit `/-/schema-versions` on your Datasette instance to see a numeric version for the schema for each of your databases.

  Any changes you make to the schema will increase this version number.

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-datasette-schema-versions" class="anchor" aria-hidden="true" href="#user-content-datasette-schema-versions"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-schema-versions</h1>
  <p><a href="https://pypi.org/project/datasette-schema-versions/" rel="nofollow"><img src="https://camo.githubusercontent.com/d76a12bb768f47749ba883bf02f411fdf2c82b5312cfbdb50ea78a89a7216a86/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d736368656d612d76657273696f6e732e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-schema-versions.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-schema-versions/releases"><img src="https://camo.githubusercontent.com/0e3dace5580b3d4eea8e0a4f8daec3103eadf0be90444db5b77bdb9afcb2b5c8/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d736368656d612d76657273696f6e733f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-schema-versions?include_prereleases&amp;label=changelog" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-schema-versions/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-schema-versions/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-schema-versions/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width: 100%;"></a></p>
  <p>Datasette plugin that shows the schema version of every attached database</p>
  <h2><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p>Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ datasette install datasette-schema-versions
  "><pre><code>$ datasette install datasette-schema-versions
  </code></pre></div>
  <h2><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p>Visit <code>/-/schema-versions</code> on your Datasette instance to see a numeric version for the schema for each of your databases.</p>
  <p>Any changes you make to the schema will increase this version number.</p>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyOTEzMzkwODY=
name: airtable-export
full_name: simonw/airtable-export
html_url: https://github.com/simonw/airtable-export
description: Export Airtable data to YAML, JSON or SQLite files on disk
created_at: 2020-08-29T19:51:37Z
updated_at: 2021-06-08T17:30:30Z
pushed_at: 2021-04-09T23:41:52Z
homepage: https://datasette.io/tools/airtable-export
language: Python
license: apache-2.0
topics: ["yaml", "airtable", "airtable-api", "datasette-io", "datasette-tool"]
default_branch: main
permissions: {"admin": false, "push": false, "pull": false}
readme:
  # airtable-export

  [![PyPI](https://img.shields.io/pypi/v/airtable-export.svg)](https://pypi.org/project/airtable-export/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/airtable-export?include_prereleases&label=changelog)](https://github.com/simonw/airtable-export/releases)
  [![Tests](https://github.com/simonw/airtable-export/workflows/Test/badge.svg)](https://github.com/simonw/airtable-export/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/airtable-export/blob/master/LICENSE)

  Export Airtable data to files on disk

  ## Installation

  Install this tool using `pip`:

      $ pip install airtable-export

  ## Usage

  You will need to know the following information:

  - Your Airtable base ID - this is a string starting with `app...`
  - Your Airtable API key - this is a string starting with `key...`
  - The names of each of the tables that you wish to export

  You can export all of your data to a folder called `export/` by running the following:

      airtable-export export base_id table1 table2 --key=key

  This example would create two files: `export/table1.yml` and `export/table2.yml`.

  Rather than passing the API key using the `--key` option you can set it as an environment variable called `AIRTABLE_KEY`.

  ## Export options

  By default the tool exports your data as YAML.

  You can also export as JSON or as [newline delimited JSON](http://ndjson.org/) using the `--json` or `--ndjson` options:

      airtable-export export base_id table1 table2 --key=key --ndjson

  You can pass multiple format options at once. This command will create a `.json`, `.yml` and `.ndjson` file for each exported table:

      airtable-export export base_id table1 table2 \
          --key=key --ndjson --yaml --json

  ### SQLite database export

  You can export tables to a SQLite database file using the `--sqlite database.db` option:

      airtable-export export base_id table1 table2 \
          --key=key --sqlite database.db

  This can be combined with other format options. If you only specify `--sqlite` the export directory argument will be ignored.

  The SQLite database will have a table created for each table you export. Those tables will have a primary key column called `airtable_id`.

  If you run this command against an existing SQLite database records with matching primary keys will be over-written by new records from the export.

  ## Request options

  By default the tool uses [python-httpx](https://www.python-httpx.org)'s default configurations.

  You can override the `user-agent` using the `--user-agent` option:

      airtable-export export base_id table1 table2 --key=key --user-agent "Airtable Export Robot"

  You can override the [timeout during a network read operation](https://www.python-httpx.org/advanced/#fine-tuning-the-configuration) using the `--http-read-timeout` option. If not set, this defaults to 5s.

      airtable-export export base_id table1 table2 --key=key --http-read-timeout 60

  ## Running this using GitHub Actions

  [GitHub Actions](https://github.com/features/actions) is GitHub's workflow automation product. You can use it to run `airtable-export` in order to back up your Airtable data to a GitHub repository. Doing this gives you a visible commit history of changes you make to your Airtable data - like [this one](https://github.com/natbat/rockybeaches/commits/main/airtable).

  To run this for your own Airtable database you'll first need to add the following secrets to your GitHub repository:

  <dl>
    <dt>AIRTABLE_BASE_ID</dt>
    <dd>The base ID, a string beginning `app...`</dd>
    <dt>AIRTABLE_KEY</dt>
    <dd>Your Airtable API key</dd>
    <dt>AIRTABLE_TABLES</dt>
    <dd>A space separated list of the Airtable tables that you want to backup. If any of these contain spaces you will need to enclose them in single quotes, e.g. <samp>'My table with spaces in the name' OtherTableWithNoSpaces</samp></dd>
  </dl>

  Once you have set those secrets, add the following as a file called `.github/workflows/backup-airtable.yml`:
  ```yaml
  name: Backup Airtable

  on:
    workflow_dispatch:
    schedule:
    - cron: '32 0 * * *'

  jobs:
    build:
      runs-on: ubuntu-latest
      steps:
      - name: Check out repo
        uses: actions/checkout@v2
      - name: Set up Python
        uses: actions/setup-python@v2
        with:
          python-version: 3.8
      - uses: actions/cache@v2
        name: Configure pip caching
        with:
          path: ~/.cache/pip
          key: ${{ runner.os }}-pip-
          restore-keys: |
            ${{ runner.os }}-pip-
      - name: Install airtable-export
        run: |
          pip install airtable-export
      - name: Backup Airtable to backups/
        env:
          AIRTABLE_BASE_ID: ${{ secrets.AIRTABLE_BASE_ID }}
          AIRTABLE_KEY: ${{ secrets.AIRTABLE_KEY }}
          AIRTABLE_TABLES: ${{ secrets.AIRTABLE_TABLES }}
        run: |-
          airtable-export backups $AIRTABLE_BASE_ID $AIRTABLE_TABLES -v
      - name: Commit and push if it changed
        run: |-
          git config user.name "Automated"
          git config user.email "actions@users.noreply.github.com"
          git add -A
          timestamp=$(date -u)
          git commit -m "Latest data: ${timestamp}" || exit 0
          git push
  ```
  This will run once a day (at 32 minutes past midnight UTC) and will also run if you manually click the "Run workflow" button, see [GitHub Actions: Manual triggers with workflow_dispatch](https://github.blog/changelog/2020-07-06-github-actions-manual-triggers-with-workflow_dispatch/).

  ## Development

  To contribute to this tool, first checkout the code. Then create a new virtual environment:

      cd airtable-export
      python -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-airtable-export" class="anchor" aria-hidden="true" href="#user-content-airtable-export"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>airtable-export</h1>
  <p><a href="https://pypi.org/project/airtable-export/" rel="nofollow"><img src="https://camo.githubusercontent.com/2635595699cd87784148506d703996615d75d6e8d6d0eba3928a886a5ebac963/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6169727461626c652d6578706f72742e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/airtable-export.svg" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/airtable-export/releases"><img src="https://camo.githubusercontent.com/361aa58985cc1f5841b899de85bedd6e7ced2a253334b4dc390bd1762b2f8be5/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6169727461626c652d6578706f72743f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/airtable-export?include_prereleases&amp;label=changelog" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/airtable-export/actions?query=workflow%3ATest"><img src="https://github.com/simonw/airtable-export/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/airtable-export/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p>
  <p>Export Airtable data to files on disk</p>
  <h2><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p>Install this tool using <code>pip</code>:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ pip install airtable-export
  "><pre><code>$ pip install airtable-export
  </code></pre></div>
  <h2><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p>You will need to know the following information:</p>
  <ul>
  <li>Your Airtable base ID - this is a string starting with <code>app...</code></li>
  <li>Your Airtable API key - this is a string starting with <code>key...</code></li>
  <li>The names of each of the tables that you wish to export</li>
  </ul>
  <p>You can export all of your data to a folder called <code>export/</code> by running the following:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="airtable-export export base_id table1 table2 --key=key
  "><pre><code>airtable-export export base_id table1 table2 --key=key
  </code></pre></div>
  <p>This example would create two files: <code>export/table1.yml</code> and <code>export/table2.yml</code>.</p>
  <p>Rather than passing the API key using the <code>--key</code> option you can set it as an environment variable called <code>AIRTABLE_KEY</code>.</p>
  <h2><a id="user-content-export-options" class="anchor" aria-hidden="true" href="#user-content-export-options"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Export options</h2>
  <p>By default the tool exports your data as YAML.</p>
  <p>You can also export as JSON or as <a href="http://ndjson.org/" rel="nofollow">newline delimited JSON</a> using the <code>--json</code> or <code>--ndjson</code> options:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="airtable-export export base_id table1 table2 --key=key --ndjson
  "><pre><code>airtable-export export base_id table1 table2 --key=key --ndjson
  </code></pre></div>
  <p>You can pass multiple format options at once. This command will create a <code>.json</code>, <code>.yml</code> and <code>.ndjson</code> file for each exported table:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="airtable-export export base_id table1 table2 \
      --key=key --ndjson --yaml --json
  "><pre><code>airtable-export export base_id table1 table2 \
      --key=key --ndjson --yaml --json
  </code></pre></div>
  <h3><a id="user-content-sqlite-database-export" class="anchor" aria-hidden="true" href="#user-content-sqlite-database-export"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>SQLite database export</h3>
  <p>You can export tables to a SQLite database file using the <code>--sqlite database.db</code> option:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="airtable-export export base_id table1 table2 \
      --key=key --sqlite database.db
  "><pre><code>airtable-export export base_id table1 table2 \
      --key=key --sqlite database.db
  </code></pre></div>
  <p>This can be combined with other format options. If you only specify <code>--sqlite</code> the export directory argument will be ignored.</p>
  <p>The SQLite database will have a table created for each table you export. Those tables will have a primary key column called <code>airtable_id</code>.</p>
  <p>If you run this command against an existing SQLite database records with matching primary keys will be over-written by new records from the export.</p>
  <h2><a id="user-content-request-options" class="anchor" aria-hidden="true" href="#user-content-request-options"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Request options</h2>
  <p>By default the tool uses <a href="https://www.python-httpx.org" rel="nofollow">python-httpx</a>'s default configurations.</p>
  <p>You can override the <code>user-agent</code> using the <code>--user-agent</code> option:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="airtable-export export base_id table1 table2 --key=key --user-agent &quot;Airtable Export Robot&quot;
  "><pre><code>airtable-export export base_id table1 table2 --key=key --user-agent "Airtable Export Robot"
  </code></pre></div>
  <p>You can override the <a href="https://www.python-httpx.org/advanced/#fine-tuning-the-configuration" rel="nofollow">timeout during a network read operation</a> using the <code>--http-read-timeout</code> option. If not set, this defaults to 5s.</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="airtable-export export base_id table1 table2 --key=key --http-read-timeout 60
  "><pre><code>airtable-export export base_id table1 table2 --key=key --http-read-timeout 60
  </code></pre></div>
  <h2><a id="user-content-running-this-using-github-actions" class="anchor" aria-hidden="true" href="#user-content-running-this-using-github-actions"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Running this using GitHub Actions</h2>
  <p><a href="https://github.com/features/actions">GitHub Actions</a> is GitHub's workflow automation product. You can use it to run <code>airtable-export</code> in order to back up your Airtable data to a GitHub repository. Doing this gives you a visible commit history of changes you make to your Airtable data - like <a href="https://github.com/natbat/rockybeaches/commits/main/airtable">this one</a>.</p>
  <p>To run this for your own Airtable database you'll first need to add the following secrets to your GitHub repository:</p>
  <dl>
    <dt>AIRTABLE_BASE_ID</dt>
    <dd>The base ID, a string beginning `app...`</dd>
    <dt>AIRTABLE_KEY</dt>
    <dd>Your Airtable API key</dd>
    <dt>AIRTABLE_TABLES</dt>
    <dd>A space separated list of the Airtable tables that you want to backup. If any of these contain spaces you will need to enclose them in single quotes, e.g. <samp>'My table with spaces in the name' OtherTableWithNoSpaces</samp></dd>
  </dl>
  <p>Once you have set those secrets, add the following as a file called <code>.github/workflows/backup-airtable.yml</code>:</p>
  <div class="highlight highlight-source-yaml position-relative" data-snippet-clipboard-copy-content="name: Backup Airtable

  on:
    workflow_dispatch:
    schedule:
    - cron: '32 0 * * *'

  jobs:
    build:
      runs-on: ubuntu-latest
      steps:
      - name: Check out repo
        uses: actions/checkout@v2
      - name: Set up Python
        uses: actions/setup-python@v2
        with:
          python-version: 3.8
      - uses: actions/cache@v2
        name: Configure pip caching
        with:
          path: ~/.cache/pip
          key: ${{ runner.os }}-pip-
          restore-keys: |
            ${{ runner.os }}-pip-
      - name: Install airtable-export
        run: |
          pip install airtable-export
      - name: Backup Airtable to backups/
        env:
          AIRTABLE_BASE_ID: ${{ secrets.AIRTABLE_BASE_ID }}
          AIRTABLE_KEY: ${{ secrets.AIRTABLE_KEY }}
          AIRTABLE_TABLES: ${{ secrets.AIRTABLE_TABLES }}
        run: |-
          airtable-export backups $AIRTABLE_BASE_ID $AIRTABLE_TABLES -v
      - name: Commit and push if it changed
        run: |-
          git config user.name &quot;Automated&quot;
          git config user.email &quot;actions@users.noreply.github.com&quot;
          git add -A
          timestamp=$(date -u)
          git commit -m &quot;Latest data: ${timestamp}&quot; || exit 0
          git push
  "><pre><span class="pl-ent">name</span>: <span class="pl-s">Backup Airtable</span>

  <span class="pl-ent">on</span>:
    <span class="pl-ent">workflow_dispatch</span>:
    <span class="pl-ent">schedule</span>:
    - <span class="pl-ent">cron</span>: <span class="pl-s"><span class="pl-pds">'</span>32 0 * * *<span class="pl-pds">'</span></span>

  <span class="pl-ent">jobs</span>:
    <span class="pl-ent">build</span>:
      <span class="pl-ent">runs-on</span>: <span class="pl-s">ubuntu-latest</span>
      <span class="pl-ent">steps</span>:
      - <span class="pl-ent">name</span>: <span class="pl-s">Check out repo</span>
        <span class="pl-ent">uses</span>: <span class="pl-s">actions/checkout@v2</span>
      - <span class="pl-ent">name</span>: <span class="pl-s">Set up Python</span>
        <span class="pl-ent">uses</span>: <span class="pl-s">actions/setup-python@v2</span>
        <span class="pl-ent">with</span>:
          <span class="pl-ent">python-version</span>: <span class="pl-c1">3.8</span>
      - <span class="pl-ent">uses</span>: <span class="pl-s">actions/cache@v2</span>
        <span class="pl-ent">name</span>: <span class="pl-s">Configure pip caching</span>
        <span class="pl-ent">with</span>:
          <span class="pl-ent">path</span>: <span class="pl-s">~/.cache/pip</span>
          <span class="pl-ent">key</span>: <span class="pl-s">${{ runner.os }}-pip-</span>
          <span class="pl-ent">restore-keys</span>: <span class="pl-s">|</span>
  <span class="pl-s">          ${{ runner.os }}-pip-</span>
  <span class="pl-s"></span>    - <span class="pl-ent">name</span>: <span class="pl-s">Install airtable-export</span>
        <span class="pl-ent">run</span>: <span class="pl-s">|</span>
  <span class="pl-s">        pip install airtable-export</span>
  <span class="pl-s"></span>    - <span class="pl-ent">name</span>: <span class="pl-s">Backup Airtable to backups/</span>
        <span class="pl-ent">env</span>:
          <span class="pl-ent">AIRTABLE_BASE_ID</span>: <span class="pl-s">${{ secrets.AIRTABLE_BASE_ID }}</span>
          <span class="pl-ent">AIRTABLE_KEY</span>: <span class="pl-s">${{ secrets.AIRTABLE_KEY }}</span>
          <span class="pl-ent">AIRTABLE_TABLES</span>: <span class="pl-s">${{ secrets.AIRTABLE_TABLES }}</span>
        <span class="pl-ent">run</span>: <span class="pl-s">|-</span>
  <span class="pl-s">        airtable-export backups $AIRTABLE_BASE_ID $AIRTABLE_TABLES -v</span>
  <span class="pl-s"></span>    - <span class="pl-ent">name</span>: <span class="pl-s">Commit and push if it changed</span>
        <span class="pl-ent">run</span>: <span class="pl-s">|-</span>
  <span class="pl-s">        git config user.name "Automated"</span>
  <span class="pl-s">        git config user.email "actions@users.noreply.github.com"</span>
  <span class="pl-s">        git add -A</span>
  <span class="pl-s">        timestamp=$(date -u)</span>
  <span class="pl-s">        git commit -m "Latest data: ${timestamp}" || exit 0</span>
  <span class="pl-s">        git push</span></pre></div>
  <p>This will run once a day (at 32 minutes past midnight UTC) and will also run if you manually click the "Run workflow" button, see <a href="https://github.blog/changelog/2020-07-06-github-actions-manual-triggers-with-workflow_dispatch/" rel="nofollow">GitHub Actions: Manual triggers with workflow_dispatch</a>.</p>
  <h2><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p>To contribute to this tool, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="cd airtable-export
  python -mvenv venv
  source venv/bin/activate
  "><pre><code>cd airtable-export
  python -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p>Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pipenv shell
  "><pre><code>pipenv shell
  </code></pre></div>
  <p>Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pip install -e '.[test]'
  "><pre><code>pip install -e '.[test]'
  </code></pre></div>
  <p>To run the tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pytest
  "><pre><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyOTEzNTkzNTg=
name: datasette-yaml
full_name: simonw/datasette-yaml
html_url: https://github.com/simonw/datasette-yaml
description: Export Datasette records as YAML
created_at: 2020-08-29T22:32:15Z
updated_at: 2020-12-28T03:20:36Z
pushed_at: 2021-05-13T08:59:53Z
language: Python
topics: ["yaml", "datasette", "datasette-plugin", "datasette-io"]
default_branch: main
permissions: {"admin": false, "push": false, "pull": false}
readme:
  # datasette-yaml

  [![PyPI](https://img.shields.io/pypi/v/datasette-yaml.svg)](https://pypi.org/project/datasette-yaml/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-yaml?include_prereleases&label=changelog)](https://github.com/simonw/datasette-yaml/releases)
  [![Tests](https://github.com/simonw/datasette-yaml/workflows/Test/badge.svg)](https://github.com/simonw/datasette-yaml/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-yaml/blob/main/LICENSE)

  Export Datasette records as YAML

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-yaml

  ## Usage

  Having installed this plugin, every table and query will gain a new `.yaml` export link.

  You can also construct these URLs directly: `/dbname/tablename.yaml`

  ## Demo

  The plugin is running on [covid-19.datasettes.com](https://covid-19.datasettes.co/) - for example [/covid/latest_ny_times_counties_with_populations.yaml](https://covid-19.datasettes.com/covid/latest_ny_times_counties_with_populations.yaml)

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-yaml
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-datasette-yaml" class="anchor" aria-hidden="true" href="#user-content-datasette-yaml"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-yaml</h1>
  <p><a href="https://pypi.org/project/datasette-yaml/" rel="nofollow"><img src="https://camo.githubusercontent.com/46f8a5a11934b044a19fda3d282e8ec32c2f77cc4e73c7dedc7cb6508e72ec3f/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d79616d6c2e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-yaml.svg" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-yaml/releases"><img src="https://camo.githubusercontent.com/cfbcacc1ce66a9127dd962a62d240197dca4b780bf0a70e3a9e8c9707779f41c/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d79616d6c3f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-yaml?include_prereleases&amp;label=changelog" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-yaml/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-yaml/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-yaml/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p>
  <p>Export Datasette records as YAML</p>
  <h2><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p>Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ datasette install datasette-yaml
  "><pre><code>$ datasette install datasette-yaml
  </code></pre></div>
  <h2><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p>Having installed this plugin, every table and query will gain a new <code>.yaml</code> export link.</p>
  <p>You can also construct these URLs directly: <code>/dbname/tablename.yaml</code></p>
  <h2><a id="user-content-demo" class="anchor" aria-hidden="true" href="#user-content-demo"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Demo</h2>
  <p>The plugin is running on <a href="https://covid-19.datasettes.co/" rel="nofollow">covid-19.datasettes.com</a> - for example <a href="https://covid-19.datasettes.com/covid/latest_ny_times_counties_with_populations.yaml" rel="nofollow">/covid/latest_ny_times_counties_with_populations.yaml</a></p>
  <h2><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p>To set up this plugin locally, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="cd datasette-yaml
  python3 -mvenv venv
  source venv/bin/activate
  "><pre><code>cd datasette-yaml
  python3 -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p>Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pipenv shell
  "><pre><code>pipenv shell
  </code></pre></div>
  <p>Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pip install -e '.[test]'
  "><pre><code>pip install -e '.[test]'
  </code></pre></div>
  <p>To run the tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pytest
  "><pre><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyOTMxNjQ0NDc=
name: datasette-backup
full_name: simonw/datasette-backup
html_url: https://github.com/simonw/datasette-backup
description: Plugin adding backup options to Datasette
created_at: 2020-09-05T22:33:29Z
updated_at: 2020-09-24T00:16:59Z
pushed_at: 2020-09-07T02:27:30Z
language: Python
topics: ["datasette", "datasette-plugin", "datasette-io"]
default_branch: main
permissions: {"admin": false, "push": false, "pull": false}
readme:
  # datasette-backup

  [![PyPI](https://img.shields.io/pypi/v/datasette-backup.svg)](https://pypi.org/project/datasette-backup/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-backup?include_prereleases&label=changelog)](https://github.com/simonw/datasette-backup/releases)
  [![Tests](https://github.com/simonw/datasette-backup/workflows/Test/badge.svg)](https://github.com/simonw/datasette-backup/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-backup/blob/main/LICENSE)

  Plugin adding backup options to Datasette

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-backup

  ## Usage

  Once installed, you can download a SQL backup of any of your databases from:

      /-/backup/dbname.sql

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-backup
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-datasette-backup" class="anchor" aria-hidden="true" href="#user-content-datasette-backup"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-backup</h1>
  <p><a href="https://pypi.org/project/datasette-backup/" rel="nofollow"><img src="https://camo.githubusercontent.com/1f11324a2eb1c6968d27d839ddad8abfe01ae68a37d8d6b808ecfca2eab6af14/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d6261636b75702e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-backup.svg" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-backup/releases"><img src="https://camo.githubusercontent.com/b87d1e53482d2b02fca6a2b3536e7b81b061039c4eb05a353ca71e4bfad2b5e3/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d6261636b75703f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-backup?include_prereleases&amp;label=changelog" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-backup/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-backup/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-backup/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p>
  <p>Plugin adding backup options to Datasette</p>
  <h2><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p>Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ datasette install datasette-backup
  "><pre><code>$ datasette install datasette-backup
  </code></pre></div>
  <h2><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p>Once installed, you can download a SQL backup of any of your databases from:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="/-/backup/dbname.sql
  "><pre><code>/-/backup/dbname.sql
  </code></pre></div>
  <h2><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p>To set up this plugin locally, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="cd datasette-backup
  python3 -mvenv venv
  source venv/bin/activate
  "><pre><code>cd datasette-backup
  python3 -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p>Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pipenv shell
  "><pre><code>pipenv shell
  </code></pre></div>
  <p>Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pip install -e '.[test]'
  "><pre><code>pip install -e '.[test]'
  </code></pre></div>
  <p>To run the tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pytest
  "><pre><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyOTMzNjE1MTQ=
name: geocode-sqlite
full_name: eyeseast/geocode-sqlite
html_url: https://github.com/eyeseast/geocode-sqlite
description: Geocode rows in a SQLite database table
created_at: 2020-09-06T21:05:39Z
updated_at: 2022-11-02T19:19:56Z
pushed_at: 2022-11-07T17:31:05Z
language: Python
license: apache-2.0
topics: []
default_branch: main
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
visibility: public
readme:
  # geocode-sqlite

  [![PyPI](https://img.shields.io/pypi/v/geocode-sqlite.svg)](https://pypi.org/project/geocode-sqlite/)
  [![Changelog](https://img.shields.io/github/v/release/eyeseast/geocode-sqlite?include_prereleases&label=changelog)](https://github.com/eyeseast/geocode-sqlite/releases)
  [![Tests](https://github.com/eyeseast/geocode-sqlite/workflows/Test/badge.svg)](https://github.com/eyeseast/geocode-sqlite/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/eyeseast/geocode-sqlite/blob/master/LICENSE)

  Geocode rows from a SQLite table

  ## Installation

  Install this tool using `pip` or `pipx`:

  ```sh
  # install inside a virtualenv
  pip install geocode-sqlite

  # install globally
  pipx install geocode-sqlite
  ```

  ## Usage

  Let's say you have a spreadsheet with addresses in it, and you'd like to map those locations.
  First, create a SQLite database and insert rows from that spreadsheet using `sqlite-utils`.

  ```sh
  sqlite-utils insert data.db data data.csv --csv
  ```

  Now, geocode it using OpenStreetMap's Nominatim geocoder.

  ```sh
  geocode-sqlite nominatim data.db data \
   --location="{address}, {city}, {state} {zip}" \
   --delay=1 \
   --user-agent="this-is-me"
  ```

  In the command above, you're using Nominatim, which is free and only asks for a unique user agent (`--user-agent`).

  This will connect to a database (`data.db`) and read all rows from the table `data` (skipping any that already
  have both a `latitude` and `longitude` column filled).

  You're also telling the geocoder how to extract a location query (`--location`) from a row of data, using Python's
  built-in string formatting, and setting a rate limit (`--delay`) of one request per second.

  For each row where geocoding succeeds, `latitude` and `longitude` will be populated. If you hit an error, or a rate limit,
  run the same query and pick up where you left off.

  The resulting table layout can be visualized with [datasette-cluster-map](https://datasette.io/plugins/datasette-cluster-map).

  Under the hood, this package uses the excellent [geopy](https://geopy.readthedocs.io/en/latest/) library, which is stable and thoroughly road-tested. If you need help understanding a particular geocoder's options, consult [geopy's documentation](https://geopy.readthedocs.io/en/latest/#module-geopy.geocoders).

  ### Supported Geocoders

  The CLI currently supports these geocoders:

  - `bing`
  - `googlev3`
  - `mapquest` (and `open-mapquest`)
  - `mapbox`
  - `nominatim`
  - `opencage`

  #### Adding new geocoders

  1. Open an issue with the name of the geocoding service as the ticket title ([example](https://github.com/eyeseast/geocode-sqlite/issues/35)). Put any noteworthy implementation details in the ticket body, like where to get an API key if one is required.
  2. Fork the repo and add a geocoder.
  3. Add an example to the `Makefile`. Add tests if there's new shared functionality.

  ### Common arguments and options

  Each geocoder needs to know where to find the data it's working with. These are the first two arguments:

  - `database`: a path to a SQLite file, which must already exist
  - `table`: the name of a table, in that database, which exists and has data to geocode

  From there, we have a set of options passed to every geocoder:

  - `location`: a [string format](https://docs.python.org/3/library/stdtypes.html#str.format) that will be expanded with each row to build a full query, to be geocoded
  - `delay`: a delay between each call (some services require this)
  - `latitude`: latitude column name
  - `longitude`: longitude column name
  - `geojson`: store results as GeoJSON, instead of in latitude and longitude columns
  - `spatialite`: store results in a SpatiaLite geometry column, instead of in latitude and longitude columns
  - `raw`: store raw geocoding results in a JSON column

  Each geocoder takes additional, specific arguments beyond these, such as API keys. Again, [geopy's documentation](https://geopy.readthedocs.io/en/latest/#module-geopy.geocoders) is an excellent resource.

  ## Using SpatiaLite

  The `--spatialite` flag will store results in a [geometry column](https://www.gaia-gis.it/gaia-sins/spatialite-cookbook-5/cookbook_topics.adminstration.html#topic_TABLE_to_SpatialTable), instead of `latitude` and `longitude` columns. This is useful if you're doing other GIS operations, such as using a [spatial index](https://www.gaia-gis.it/fossil/libspatialite/wiki?name=SpatialIndex). See the [SpatiaLite cookbook](https://www.gaia-gis.it/gaia-sins/spatialite-cookbook-5/index.html) and [functions list](https://www.gaia-gis.it/gaia-sins/spatialite-sql-latest.html) for more of what's possible.

  ## Capturing additional geocoding data

  Geocoding services typically return more data than just coordinates. This might include accuracy, normalized addresses or other context. This can be captured using the `--raw` flag. By default, this will add a `raw` column and store the full geocoding response as JSON. If you want to rename that column, pass a value, like `--raw custom_raw`.

  The shape of this response object will vary between services. You can query specific values using [SQLite's built-in JSON functions](https://www.sqlite.org/json1.html). For example, this will work with Google's geocoder:

  ```sql
  select
    json_extract(raw, '$.formatted_address') as address,
    json_extract(raw, '$.geometry.location_type') as location_type
  from
    innout_test
  ```

  Check each geocoding service's documentation for what's included in the response.

  ## Python API

  The command line interface aims to support the most common options for each geocoder. For more fine-grained control, use the Python API.

  As with the CLI, this assumes you already have a SQLite database and a table of location data.

  ```python
  from geocode_sqlite import geocode_table
  from geopy.geocoders import Nominatim

  # create a geocoder instance, with some extra options
  nominatim = Nominatim(user_agent="this-is-me", domain="nominatim.local.dev", scheme="http")

  # assuming our database is in the same directory
  count = geocode_table("data.db", "data", query_template="{address}, {city}, {state} {zip}")

  # when it's done
  print(f"Geocoded {count} rows")
  ```

  Any [geopy geocoder](https://geopy.readthedocs.io/en/latest/#module-geopy.geocoders) can be used with the Python API.

  ## Development

  To contribute to this tool, first checkout the code. Then create a new virtual environment:

  ```sh
  cd geocode-sqlite
  python -m venv .venv
  source .venv/bin/activate
  ```

  Or if you are using `pipenv`:

  ```sh
  pipenv shell
  ```

  Now install the dependencies and tests:

  ```sh
  pip install -e '.[test]'
  ```

  To run the tests:

  ```sh
  pytest
  ```

  Please remember that this library is mainly glue code between other well-tested projects, specifically: [click](https://click.palletsprojects.com/), [geopy](https://geopy.readthedocs.io/en/stable/) and [sqlite-utils](https://sqlite-utils.datasette.io/en/stable/). Tests should focus on making sure those parts fit together correctly. We can assume the parts themselves already work.

  To that end, there is a test geocoder included: `geocode_sqlite.testing.DummyGeocoder`. That geocoder works with an included dataset of In-N-Out Burger locations provided by [AllThePlaces](https://www.alltheplaces.xyz/). It works like a normal GeoPy geocoder, except it will only return results for In-N-Out locations using the included database.

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-geocode-sqlite" class="anchor" aria-hidden="true" href="#user-content-geocode-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>geocode-sqlite</h1>
  <p dir="auto"><a href="https://pypi.org/project/geocode-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/48afed6b156b122a781142db699a225016ec438b4f64f9534d5f852433332a50/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f67656f636f64652d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/geocode-sqlite.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/eyeseast/geocode-sqlite/releases"><img src="https://camo.githubusercontent.com/cca0e9a2e0f5dbbfb761aeb9803bac6602b9831541a309b6c5a73ebec690b35e/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f65796573656173742f67656f636f64652d73716c6974653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/eyeseast/geocode-sqlite?include_prereleases&amp;label=changelog" style="max-width: 100%;"></a>
  <a href="https://github.com/eyeseast/geocode-sqlite/actions?query=workflow%3ATest"><img src="https://github.com/eyeseast/geocode-sqlite/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a>
  <a href="https://github.com/eyeseast/geocode-sqlite/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width: 100%;"></a></p>
  <p dir="auto">Geocode rows from a SQLite table</p>
  <h2 dir="auto"><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p dir="auto">Install this tool using <code>pip</code> or <code>pipx</code>:</p>
  <div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="# install inside a virtualenv
  pip install geocode-sqlite

  # install globally
  pipx install geocode-sqlite"><pre><span class="pl-c"><span class="pl-c">#</span> install inside a virtualenv</span>
  pip install geocode-sqlite

  <span class="pl-c"><span class="pl-c">#</span> install globally</span>
  pipx install geocode-sqlite</pre></div>
  <h2 dir="auto"><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p dir="auto">Let's say you have a spreadsheet with addresses in it, and you'd like to map those locations.
  First, create a SQLite database and insert rows from that spreadsheet using <code>sqlite-utils</code>.</p>
  <div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="sqlite-utils insert data.db data data.csv --csv"><pre>sqlite-utils insert data.db data data.csv --csv</pre></div>
  <p dir="auto">Now, geocode it using OpenStreetMap's Nominatim geocoder.</p>
  <div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="geocode-sqlite nominatim data.db data \
   --location=&quot;{address}, {city}, {state} {zip}&quot; \
   --delay=1 \
   --user-agent=&quot;this-is-me&quot;"><pre>geocode-sqlite nominatim data.db data \
   --location=<span class="pl-s"><span class="pl-pds">"</span>{address}, {city}, {state} {zip}<span class="pl-pds">"</span></span> \
   --delay=1 \
   --user-agent=<span class="pl-s"><span class="pl-pds">"</span>this-is-me<span class="pl-pds">"</span></span></pre></div>
  <p dir="auto">In the command above, you're using Nominatim, which is free and only asks for a unique user agent (<code>--user-agent</code>).</p>
  <p dir="auto">This will connect to a database (<code>data.db</code>) and read all rows from the table <code>data</code> (skipping any that already
  have both a <code>latitude</code> and <code>longitude</code> column filled).</p>
  <p dir="auto">You're also telling the geocoder how to extract a location query (<code>--location</code>) from a row of data, using Python's
  built-in string formatting, and setting a rate limit (<code>--delay</code>) of one request per second.</p>
  <p dir="auto">For each row where geocoding succeeds, <code>latitude</code> and <code>longitude</code> will be populated. If you hit an error, or a rate limit,
  run the same query and pick up where you left off.</p>
  <p dir="auto">The resulting table layout can be visualized with <a href="https://datasette.io/plugins/datasette-cluster-map" rel="nofollow">datasette-cluster-map</a>.</p>
  <p dir="auto">Under the hood, this package uses the excellent <a href="https://geopy.readthedocs.io/en/latest/" rel="nofollow">geopy</a> library, which is stable and thoroughly road-tested. If you need help understanding a particular geocoder's options, consult <a href="https://geopy.readthedocs.io/en/latest/#module-geopy.geocoders" rel="nofollow">geopy's documentation</a>.</p>
  <h3 dir="auto"><a id="user-content-supported-geocoders" class="anchor" aria-hidden="true" href="#user-content-supported-geocoders"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Supported Geocoders</h3>
  <p dir="auto">The CLI currently supports these geocoders:</p>
  <ul dir="auto">
  <li><code>bing</code></li>
  <li><code>googlev3</code></li>
  <li><code>mapquest</code> (and <code>open-mapquest</code>)</li>
  <li><code>mapbox</code></li>
  <li><code>nominatim</code></li>
  <li><code>opencage</code></li>
  </ul>
  <h4 dir="auto"><a id="user-content-adding-new-geocoders" class="anchor" aria-hidden="true" href="#user-content-adding-new-geocoders"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Adding new geocoders</h4>
  <ol dir="auto">
  <li>Open an issue with the name of the geocoding service as the ticket title (<a href="https://github.com/eyeseast/geocode-sqlite/issues/35" data-hovercard-type="issue" data-hovercard-url="/eyeseast/geocode-sqlite/issues/35/hovercard">example</a>). Put any noteworthy implementation details in the ticket body, like where to get an API key if one is required.</li>
  <li>Fork the repo and add a geocoder.</li>
  <li>Add an example to the <code>Makefile</code>. Add tests if there's new shared functionality.</li>
  </ol>
  <h3 dir="auto"><a id="user-content-common-arguments-and-options" class="anchor" aria-hidden="true" href="#user-content-common-arguments-and-options"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Common arguments and options</h3>
  <p dir="auto">Each geocoder needs to know where to find the data it's working with. These are the first two arguments:</p>
  <ul dir="auto">
  <li><code>database</code>: a path to a SQLite file, which must already exist</li>
  <li><code>table</code>: the name of a table, in that database, which exists and has data to geocode</li>
  </ul>
  <p dir="auto">From there, we have a set of options passed to every geocoder:</p>
  <ul dir="auto">
  <li><code>location</code>: a <a href="https://docs.python.org/3/library/stdtypes.html#str.format" rel="nofollow">string format</a> that will be expanded with each row to build a full query, to be geocoded</li>
  <li><code>delay</code>: a delay between each call (some services require this)</li>
  <li><code>latitude</code>: latitude column name</li>
  <li><code>longitude</code>: longitude column name</li>
  <li><code>geojson</code>: store results as GeoJSON, instead of in latitude and longitude columns</li>
  <li><code>spatialite</code>: store results in a SpatiaLite geometry column, instead of in latitude and longitude columns</li>
  <li><code>raw</code>: store raw geocoding results in a JSON column</li>
  </ul>
  <p dir="auto">Each geocoder takes additional, specific arguments beyond these, such as API keys. Again, <a href="https://geopy.readthedocs.io/en/latest/#module-geopy.geocoders" rel="nofollow">geopy's documentation</a> is an excellent resource.</p>
  <h2 dir="auto"><a id="user-content-using-spatialite" class="anchor" aria-hidden="true" href="#user-content-using-spatialite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Using SpatiaLite</h2>
  <p dir="auto">The <code>--spatialite</code> flag will store results in a <a href="https://www.gaia-gis.it/gaia-sins/spatialite-cookbook-5/cookbook_topics.adminstration.html#topic_TABLE_to_SpatialTable" rel="nofollow">geometry column</a>, instead of <code>latitude</code> and <code>longitude</code> columns. This is useful if you're doing other GIS operations, such as using a <a href="https://www.gaia-gis.it/fossil/libspatialite/wiki?name=SpatialIndex" rel="nofollow">spatial index</a>. See the <a href="https://www.gaia-gis.it/gaia-sins/spatialite-cookbook-5/index.html" rel="nofollow">SpatiaLite cookbook</a> and <a href="https://www.gaia-gis.it/gaia-sins/spatialite-sql-latest.html" rel="nofollow">functions list</a> for more of what's possible.</p>
  <h2 dir="auto"><a id="user-content-capturing-additional-geocoding-data" class="anchor" aria-hidden="true" href="#user-content-capturing-additional-geocoding-data"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Capturing additional geocoding data</h2>
  <p dir="auto">Geocoding services typically return more data than just coordinates. This might include accuracy, normalized addresses or other context. This can be captured using the <code>--raw</code> flag. By default, this will add a <code>raw</code> column and store the full geocoding response as JSON. If you want to rename that column, pass a value, like <code>--raw custom_raw</code>.</p>
  <p dir="auto">The shape of this response object will vary between services. You can query specific values using <a href="https://www.sqlite.org/json1.html" rel="nofollow">SQLite's built-in JSON functions</a>. For example, this will work with Google's geocoder:</p>
  <div class="highlight highlight-source-sql notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="select
    json_extract(raw, '$.formatted_address') as address,
    json_extract(raw, '$.geometry.location_type') as location_type
  from
    innout_test"><pre><span class="pl-k">select</span>
    json_extract(raw, <span class="pl-s"><span class="pl-pds">'</span>$.formatted_address<span class="pl-pds">'</span></span>) <span class="pl-k">as</span> address,
    json_extract(raw, <span class="pl-s"><span class="pl-pds">'</span>$.geometry.location_type<span class="pl-pds">'</span></span>) <span class="pl-k">as</span> location_type
  <span class="pl-k">from</span>
    innout_test</pre></div>
  <p dir="auto">Check each geocoding service's documentation for what's included in the response.</p>
  <h2 dir="auto"><a id="user-content-python-api" class="anchor" aria-hidden="true" href="#user-content-python-api"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Python API</h2>
  <p dir="auto">The command line interface aims to support the most common options for each geocoder. For more fine-grained control, use the Python API.</p>
  <p dir="auto">As with the CLI, this assumes you already have a SQLite database and a table of location data.</p>
  <div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="from geocode_sqlite import geocode_table
  from geopy.geocoders import Nominatim

  # create a geocoder instance, with some extra options
  nominatim = Nominatim(user_agent=&quot;this-is-me&quot;, domain=&quot;nominatim.local.dev&quot;, scheme=&quot;http&quot;)

  # assuming our database is in the same directory
  count = geocode_table(&quot;data.db&quot;, &quot;data&quot;, query_template=&quot;{address}, {city}, {state} {zip}&quot;)

  # when it's done
  print(f&quot;Geocoded {count} rows&quot;)"><pre><span class="pl-k">from</span> <span class="pl-s1">geocode_sqlite</span> <span class="pl-k">import</span> <span class="pl-s1">geocode_table</span>
  <span class="pl-k">from</span> <span class="pl-s1">geopy</span>.<span class="pl-s1">geocoders</span> <span class="pl-k">import</span> <span class="pl-v">Nominatim</span>

  <span class="pl-c"># create a geocoder instance, with some extra options</span>
  <span class="pl-s1">nominatim</span> <span class="pl-c1">=</span> <span class="pl-v">Nominatim</span>(<span class="pl-s1">user_agent</span><span class="pl-c1">=</span><span class="pl-s">"this-is-me"</span>, <span class="pl-s1">domain</span><span class="pl-c1">=</span><span class="pl-s">"nominatim.local.dev"</span>, <span class="pl-s1">scheme</span><span class="pl-c1">=</span><span class="pl-s">"http"</span>)

  <span class="pl-c"># assuming our database is in the same directory</span>
  <span class="pl-s1">count</span> <span class="pl-c1">=</span> <span class="pl-en">geocode_table</span>(<span class="pl-s">"data.db"</span>, <span class="pl-s">"data"</span>, <span class="pl-s1">query_template</span><span class="pl-c1">=</span><span class="pl-s">"{address}, {city}, {state} {zip}"</span>)

  <span class="pl-c"># when it's done</span>
  <span class="pl-en">print</span>(<span class="pl-s">f"Geocoded <span class="pl-s1"><span class="pl-kos">{</span><span class="pl-s1">count</span><span class="pl-kos">}</span></span> rows"</span>)</pre></div>
  <p dir="auto">Any <a href="https://geopy.readthedocs.io/en/latest/#module-geopy.geocoders" rel="nofollow">geopy geocoder</a> can be used with the Python API.</p>
  <h2 dir="auto"><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p dir="auto">To contribute to this tool, first checkout the code. Then create a new virtual environment:</p>
  <div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="cd geocode-sqlite
  python -m venv .venv
  source .venv/bin/activate"><pre><span class="pl-c1">cd</span> geocode-sqlite
  python -m venv .venv
  <span class="pl-c1">source</span> .venv/bin/activate</pre></div>
  <p dir="auto">Or if you are using <code>pipenv</code>:</p>
  <div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="pipenv shell"><pre>pipenv shell</pre></div>
  <p dir="auto">Now install the dependencies and tests:</p>
  <div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="pip install -e '.[test]'"><pre>pip install -e <span class="pl-s"><span class="pl-pds">'</span>.[test]<span class="pl-pds">'</span></span></pre></div>
  <p dir="auto">To run the tests:</p>
  <div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="pytest"><pre>pytest</pre></div>
  <p dir="auto">Please remember that this library is mainly glue code between other well-tested projects, specifically: <a href="https://click.palletsprojects.com/" rel="nofollow">click</a>, <a href="https://geopy.readthedocs.io/en/stable/" rel="nofollow">geopy</a> and <a href="https://sqlite-utils.datasette.io/en/stable/" rel="nofollow">sqlite-utils</a>. Tests should focus on making sure those parts fit together correctly. We can assume the parts themselves already work.</p>
  <p dir="auto">To that end, there is a test geocoder included: <code>geocode_sqlite.testing.DummyGeocoder</code>. That geocoder works with an included dataset of In-N-Out Burger locations provided by <a href="https://www.alltheplaces.xyz/" rel="nofollow">AllThePlaces</a>. It works like a normal GeoPy geocoder, except it will only return results for In-N-Out locations using the included database.</p>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyOTQ3MDYyNjc=
name: datasette-seaborn
full_name: simonw/datasette-seaborn
html_url: https://github.com/simonw/datasette-seaborn
description: Statistical visualizations for Datasette using Seaborn
created_at: 2020-09-11T13:43:08Z
updated_at: 2022-03-22T01:49:39Z
pushed_at: 2022-03-22T01:49:36Z
homepage: https://datasette-seaborn-demo.datasette.io/
language: Python
topics: ["datasette", "datasette-io", "datasette-plugin", "seaborn", "visualization"]
default_branch: main
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
visibility: public
readme:
  # datasette-seaborn

  [![PyPI](https://img.shields.io/pypi/v/datasette-seaborn.svg)](https://pypi.org/project/datasette-seaborn/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-seaborn?include_prereleases&label=changelog)](https://github.com/simonw/datasette-seaborn/releases)
  [![Tests](https://github.com/simonw/datasette-seaborn/workflows/Test/badge.svg)](https://github.com/simonw/datasette-seaborn/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-seaborn/blob/main/LICENSE)

  Statistical visualizations for Datasette using Seaborn

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-seaborn

  ## Usage

  Navigate to the new `.seaborn` extension for any Datasette table.

  The `_seaborn` argument specifies a method on `sns` to execute, e.g. `?_seaborn=relplot`.

  Extra arguments to those methods can be specified using e.g. `&_seaborn_x=column_name`.

  ## Configuration

  The plugin implements a default rendering time limit of five seconds. You can customize this limit using the `render_time_limit` setting, which accepts a floating point number of seconds. Add this to your `metadata.json`:

  ```json
  {
      "plugins": {
          "datasette-seaborn": {
              "render_time_limit": 1.0
          }
      }
  }
  ```

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-seaborn
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-datasette-seaborn" class="anchor" aria-hidden="true" href="#user-content-datasette-seaborn"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-seaborn</h1>
  <p dir="auto"><a href="https://pypi.org/project/datasette-seaborn/" rel="nofollow"><img src="https://camo.githubusercontent.com/c0a14e4147c52777b94b74dd7e3fc2c3e58dc5f896dadaf7128ab52b1facd581/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d736561626f726e2e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-seaborn.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-seaborn/releases"><img src="https://camo.githubusercontent.com/33834b7d5f3a4df257e0c20b9d8c6729f1b720d257bd6bc33c0cbf60a91382c7/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d736561626f726e3f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-seaborn?include_prereleases&amp;label=changelog" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-seaborn/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-seaborn/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-seaborn/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width: 100%;"></a></p>
  <p dir="auto">Statistical visualizations for Datasette using Seaborn</p>
  <h2 dir="auto"><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p dir="auto">Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="$ datasette install datasette-seaborn"><pre><code>$ datasette install datasette-seaborn
  </code></pre></div>
  <h2 dir="auto"><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p dir="auto">Navigate to the new <code>.seaborn</code> extension for any Datasette table.</p>
  <p dir="auto">The <code>_seaborn</code> argument specifies a method on <code>sns</code> to execute, e.g. <code>?_seaborn=relplot</code>.</p>
  <p dir="auto">Extra arguments to those methods can be specified using e.g. <code>&amp;_seaborn_x=column_name</code>.</p>
  <h2 dir="auto"><a id="user-content-configuration" class="anchor" aria-hidden="true" href="#user-content-configuration"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Configuration</h2>
  <p dir="auto">The plugin implements a default rendering time limit of five seconds. You can customize this limit using the <code>render_time_limit</code> setting, which accepts a floating point number of seconds. Add this to your <code>metadata.json</code>:</p>
  <div class="highlight highlight-source-json position-relative overflow-auto" data-snippet-clipboard-copy-content="{
      &quot;plugins&quot;: {
          &quot;datasette-seaborn&quot;: {
              &quot;render_time_limit&quot;: 1.0
          }
      }
  }"><pre>{
      <span class="pl-ent">"plugins"</span>: {
          <span class="pl-ent">"datasette-seaborn"</span>: {
              <span class="pl-ent">"render_time_limit"</span>: <span class="pl-c1">1.0</span>
          }
      }
  }</pre></div>
  <h2 dir="auto"><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p dir="auto">To set up this plugin locally, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="cd datasette-seaborn
  python3 -mvenv venv
  source venv/bin/activate"><pre><code>cd datasette-seaborn
  python3 -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p dir="auto">Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pipenv shell"><pre><code>pipenv shell
  </code></pre></div>
  <p dir="auto">Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pip install -e '.[test]'"><pre><code>pip install -e '.[test]'
  </code></pre></div>
  <p dir="auto">To run the tests:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pytest"><pre><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyOTkxNDM4NDk=
name: datasette-dateutil
full_name: simonw/datasette-dateutil
html_url: https://github.com/simonw/datasette-dateutil
description: dateutil functions for Datasette
created_at: 2020-09-28T00:14:20Z
updated_at: 2022-03-01T00:09:57Z
pushed_at: 2022-03-01T01:40:21Z
language: Python
topics: ["datasette", "datasette-io", "datasette-plugin", "dateutil"]
default_branch: main
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
visibility: public
readme:
  # datasette-dateutil

  [![PyPI](https://img.shields.io/pypi/v/datasette-dateutil.svg)](https://pypi.org/project/datasette-dateutil/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-dateutil?include_prereleases&label=changelog)](https://github.com/simonw/datasette-dateutil/releases)
  [![Tests](https://github.com/simonw/datasette-dateutil/workflows/Test/badge.svg)](https://github.com/simonw/datasette-dateutil/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-dateutil/blob/main/LICENSE)

  dateutil functions for Datasette

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-dateutil

  ## Usage

  This function adds custom SQL functions that expose functionality from the [dateutil](https://dateutil.readthedocs.io/) Python library.

  Once installed, the following SQL functions become available:

  ### Parsing date strings

  - `dateutil_parse(text)` - returns an ISO8601 date string parsed from the text, or `null` if the input could not be parsed. `dateutil_parse("10 october 2020 3pm")` returns `2020-10-10T15:00:00`.
  - `dateutil_parse_fuzzy(text)` - same as `dateutil_parse()` but this also works against strings that contain a date somewhere within them - that date will be returned, or `null` if no dates could be found. `dateutil_parse_fuzzy("This is due 10 september")` returns `2020-09-10T00:00:00` (but will start returning the 2021 version of that if the year is 2021).

  The `dateutil_parse()` and `dateutil_parse_fuzzy()` functions both follow the American convention of assuming that `1/2/2020` lists the month first, evaluating this example to the 2nd of January.

  If you want to assume that the day comes first, use these two functions instead:

  - `dateutil_parse_dayfirst(text)`
  - `dateutil_parse_fuzzy_dayfirst(text)`

  Here's a query demonstrating these functions:

  ```sql
  select
    dateutil_parse("10 october 2020 3pm"),
    dateutil_parse_fuzzy("This is due 10 september"),
    dateutil_parse("1/2/2020"),
    dateutil_parse("2020-03-04"),
    dateutil_parse_dayfirst("2020-03-04");
  ```

  [Try that query](https://latest-with-plugins.datasette.io/fixtures?sql=select%0D%0A++dateutil_parse%28%2210+october+2020+3pm%22%29%2C%0D%0A++dateutil_parse_fuzzy%28%22This+is+due+10+september%22%29%2C%0D%0A++dateutil_parse%28%221%2F2%2F2020%22%29%2C%0D%0A++dateutil_parse%28%222020-03-04%22%29%2C%0D%0A++dateutil_parse_dayfirst%28%222020-03-04%22%29%3B)

  ### Optional default dates

  The `dateutil_parse()`, `dateutil_parse_fuzzy()`, `dateutil_parse_dayfirst()` and `dateutil_parse_fuzzy_dayfirst()` functions all accept an optional second argument specifying a "default" datetime to consider if some of the details are missing. For example, the following:
  ```sql
  select dateutil_parse('1st october', '1985-01-01')
  ```
  Will return `1985-10-01T00:00:00` - the missing year is replaced with the year from the default date.

  [Example query demonstrating the default date argument](https://latest-with-plugins.datasette.io/fixtures?sql=with+times+as+%28%0D%0A++select%0D%0A++++datetime%28%27now%27%29+as+t%0D%0A++union%0D%0A++select%0D%0A++++datetime%28%27now%27%2C+%27-1+year%27%29%0D%0A++union%0D%0A++select%0D%0A++++datetime%28%27now%27%2C+%27-3+years%27%29%0D%0A%29%0D%0Aselect+t%2C+dateutil_parse_fuzzy%28%22This+is+due+10+september%22%2C+t%29+from+times)

  ### Calculating Easter

  - `dateutil_easter(year)` - returns the date for Easter in that year, for example `dateutil_easter("2020")` returns `2020-04-12`.

  [Example Easter query](https://latest-with-plugins.datasette.io/fixtures?sql=select%0D%0A++dateutil_easter%282019%29%2C%0D%0A++dateutil_easter%282020%29%2C%0D%0A++dateutil_easter%282021%29)

  ### JSON arrays of dates

  Several functions return JSON arrays of date strings. These can be used with SQLite's `json_each()` function to perform joins against dates from a specific date range or recurrence rule.

  These functions can return up to 10,000 results. They will return an error if more than 10,000 dates would be returned - this is to protect against denial of service attacks.

  - `dateutil_dates_between('1 january 2020', '5 jan 2020')` - given two dates (in any format that can be handled by `dateutil_parse()`) this function returns a JSON string containing the dates between those two days, inclusive. This example returns `["2020-01-01", "2020-01-02", "2020-01-03", "2020-01-04", "2020-01-05"]`.
  - `dateutil_dates_between('1 january 2020', '5 jan 2020', 0)` - set the optional third argument to `0` to specify that you would like this to be exclusive of the last day. This example returns `["2020-01-01", "2020-01-02", "2020-01-03", "2020-01-04"]`.

  [Try these queries](https://latest-with-plugins.datasette.io/fixtures?sql=select%0D%0A++dateutil_dates_between%28%271+january+2020%27%2C+%275+jan+2020%27%29%2C%0D%0A++dateutil_dates_between%28%271+january+2020%27%2C+%275+jan+2020%27%2C+0%29)

  The `dateutil_rrule()` and `dateutil_rrule_date()` functions accept the iCalendar standard ``rrule` format - see [the dateutil documentation](https://dateutil.readthedocs.io/en/stable/rrule.html#rrulestr-examples) for more examples.

  This format lets you specify recurrence rules such as "the next four last mondays of the month".

  - `dateutil_rrule(rrule, optional_dtsart)` - given an rrule returns a JSON array of ISO datetimes. The second argument is optional and will be treated as the start date for the rule.
  - `dateutil_rrule_date(rrule, optional_dtsart)` - same as `dateutil_rrule()` but returns ISO dates.

  Example query:

  ```sql
  select
    dateutil_rrule('FREQ=HOURLY;COUNT=5'),
    dateutil_rrule_date(
      'FREQ=DAILY;COUNT=3',
      '1st jan 2020'
    );
  ```
  [Try the rrule example query](https://latest-with-plugins.datasette.io/fixtures?sql=select%0D%0A++dateutil_rrule('FREQ%3DHOURLY%3BCOUNT%3D5')%2C%0D%0A++dateutil_rrule_date(%0D%0A++++'FREQ%3DDAILY%3BCOUNT%3D3'%2C%0D%0A++++'1st+jan+2020'%0D%0A++)%3B)

  ### Joining data using json_each()

  SQLite's [json_each() function](https://www.sqlite.org/json1.html#jeach) can be used to turn a JSON array of dates into a table that can be joined against other data. Here's a query that returns a table showing every day in January 2019:

  ```sql
  select
    value as date
  from
    json_each(
      dateutil_dates_between('1 Jan 2019', '31 Jan 2019')
    )
  ```
  [Try that query](https://latest-with-plugins.datasette.io/fixtures?sql=select%0D%0A++value+as+date%0D%0Afrom%0D%0A++json_each%28%0D%0A++++dateutil_dates_between%28%271+Jan+2019%27%2C+%2731+Jan+2019%27%29%0D%0A++%29)

  You can run joins against this table by assigning it a name using SQLite's [support for Common Table Expressions (CTEs)](https://sqlite.org/lang_with.html).

  This example query uses `substr(created, 0, 11)` to retrieve the date portion of the `created` column in the [facetable demo table](https://latest-with-plugins.datasette.io/fixtures/facetable), then joins that against the table of days in January to calculate the count of rows created on each day. The `LEFT JOIN` against `days_in_january` ensures that days which had no created records are still returned in the results, with a count of 0.

  ```sql
  with created_dates as (
    select
      substr(created, 0, 11) as date
    from
      facetable
  ),
  days_in_january as (
    select
      value as date
    from
      json_each(
        dateutil_dates_between('1 Jan 2019', '31 Jan 2019')
      )
  )
  select
    days_in_january.date,
    count(created_dates.date) as total
  from
    days_in_january
    left join created_dates on days_in_january.date = created_dates.date
  group by
    days_in_january.date;
  ```
  [Try that query](https://latest-with-plugins.datasette.io/fixtures?sql=with+created_dates+as+%28%0D%0A++select%0D%0A++++substr%28created%2C+0%2C+11%29+as+date%0D%0A++from%0D%0A++++facetable%0D%0A%29%2C%0D%0Adays_in_january+as+%28%0D%0A++select%0D%0A++++value+as+date%0D%0A++from%0D%0A++++json_each%28%0D%0A++++++dateutil_dates_between%28%271+Jan+2019%27%2C+%2731+Jan+2019%27%29%0D%0A++++%29%0D%0A%29%0D%0Aselect%0D%0A++days_in_january.date%2C%0D%0A++count%28created_dates.date%29+as+total%0D%0Afrom%0D%0A++days_in_january%0D%0A++left+join+created_dates+on+days_in_january.date+%3D+created_dates.date%0D%0Agroup+by%0D%0A++days_in_january.date%3B#g.mark=bar&g.x_column=date&g.x_type=ordinal&g.y_column=total&g.y_type=quantitative) with a bar chart rendered using the [datasette-vega](https://github.com/simonw/datasette-vega) plugin.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-dateutil
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-datasette-dateutil" class="anchor" aria-hidden="true" href="#user-content-datasette-dateutil"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-dateutil</h1>
  <p dir="auto"><a href="https://pypi.org/project/datasette-dateutil/" rel="nofollow"><img src="https://camo.githubusercontent.com/0e859dccf125298272af85fddd25900a6c9a11a9ebba16d8cc93782759d53e21/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d646174657574696c2e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-dateutil.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-dateutil/releases"><img src="https://camo.githubusercontent.com/084ae9754f14a59ff18c2c74d621cab421866c6af7f4c99c9c94588cdf6cac2a/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d646174657574696c3f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-dateutil?include_prereleases&amp;label=changelog" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-dateutil/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-dateutil/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-dateutil/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width: 100%;"></a></p>
  <p dir="auto">dateutil functions for Datasette</p>
  <h2 dir="auto"><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p dir="auto">Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="$ datasette install datasette-dateutil"><pre><code>$ datasette install datasette-dateutil
  </code></pre></div>
  <h2 dir="auto"><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p dir="auto">This function adds custom SQL functions that expose functionality from the <a href="https://dateutil.readthedocs.io/" rel="nofollow">dateutil</a> Python library.</p>
  <p dir="auto">Once installed, the following SQL functions become available:</p>
  <h3 dir="auto"><a id="user-content-parsing-date-strings" class="anchor" aria-hidden="true" href="#user-content-parsing-date-strings"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Parsing date strings</h3>
  <ul dir="auto">
  <li><code>dateutil_parse(text)</code> - returns an ISO8601 date string parsed from the text, or <code>null</code> if the input could not be parsed. <code>dateutil_parse("10 october 2020 3pm")</code> returns <code>2020-10-10T15:00:00</code>.</li>
  <li><code>dateutil_parse_fuzzy(text)</code> - same as <code>dateutil_parse()</code> but this also works against strings that contain a date somewhere within them - that date will be returned, or <code>null</code> if no dates could be found. <code>dateutil_parse_fuzzy("This is due 10 september")</code> returns <code>2020-09-10T00:00:00</code> (but will start returning the 2021 version of that if the year is 2021).</li>
  </ul>
  <p dir="auto">The <code>dateutil_parse()</code> and <code>dateutil_parse_fuzzy()</code> functions both follow the American convention of assuming that <code>1/2/2020</code> lists the month first, evaluating this example to the 2nd of January.</p>
  <p dir="auto">If you want to assume that the day comes first, use these two functions instead:</p>
  <ul dir="auto">
  <li><code>dateutil_parse_dayfirst(text)</code></li>
  <li><code>dateutil_parse_fuzzy_dayfirst(text)</code></li>
  </ul>
  <p dir="auto">Here's a query demonstrating these functions:</p>
  <div class="highlight highlight-source-sql position-relative overflow-auto" data-snippet-clipboard-copy-content="select
    dateutil_parse(&quot;10 october 2020 3pm&quot;),
    dateutil_parse_fuzzy(&quot;This is due 10 september&quot;),
    dateutil_parse(&quot;1/2/2020&quot;),
    dateutil_parse(&quot;2020-03-04&quot;),
    dateutil_parse_dayfirst(&quot;2020-03-04&quot;);"><pre><span class="pl-k">select</span>
    dateutil_parse(<span class="pl-s"><span class="pl-pds">"</span>10 october 2020 3pm<span class="pl-pds">"</span></span>),
    dateutil_parse_fuzzy(<span class="pl-s"><span class="pl-pds">"</span>This is due 10 september<span class="pl-pds">"</span></span>),
    dateutil_parse(<span class="pl-s"><span class="pl-pds">"</span>1/2/2020<span class="pl-pds">"</span></span>),
    dateutil_parse(<span class="pl-s"><span class="pl-pds">"</span>2020-03-04<span class="pl-pds">"</span></span>),
    dateutil_parse_dayfirst(<span class="pl-s"><span class="pl-pds">"</span>2020-03-04<span class="pl-pds">"</span></span>);</pre></div>
  <p dir="auto"><a href="https://latest-with-plugins.datasette.io/fixtures?sql=select%0D%0A++dateutil_parse%28%2210+october+2020+3pm%22%29%2C%0D%0A++dateutil_parse_fuzzy%28%22This+is+due+10+september%22%29%2C%0D%0A++dateutil_parse%28%221%2F2%2F2020%22%29%2C%0D%0A++dateutil_parse%28%222020-03-04%22%29%2C%0D%0A++dateutil_parse_dayfirst%28%222020-03-04%22%29%3B" rel="nofollow">Try that query</a></p>
  <h3 dir="auto"><a id="user-content-optional-default-dates" class="anchor" aria-hidden="true" href="#user-content-optional-default-dates"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Optional default dates</h3>
  <p dir="auto">The <code>dateutil_parse()</code>, <code>dateutil_parse_fuzzy()</code>, <code>dateutil_parse_dayfirst()</code> and <code>dateutil_parse_fuzzy_dayfirst()</code> functions all accept an optional second argument specifying a "default" datetime to consider if some of the details are missing. For example, the following:</p>
  <div class="highlight highlight-source-sql position-relative overflow-auto" data-snippet-clipboard-copy-content="select dateutil_parse('1st october', '1985-01-01')"><pre><span class="pl-k">select</span> dateutil_parse(<span class="pl-s"><span class="pl-pds">'</span>1st october<span class="pl-pds">'</span></span>, <span class="pl-s"><span class="pl-pds">'</span>1985-01-01<span class="pl-pds">'</span></span>)</pre></div>
  <p dir="auto">Will return <code>1985-10-01T00:00:00</code> - the missing year is replaced with the year from the default date.</p>
  <p dir="auto"><a href="https://latest-with-plugins.datasette.io/fixtures?sql=with+times+as+%28%0D%0A++select%0D%0A++++datetime%28%27now%27%29+as+t%0D%0A++union%0D%0A++select%0D%0A++++datetime%28%27now%27%2C+%27-1+year%27%29%0D%0A++union%0D%0A++select%0D%0A++++datetime%28%27now%27%2C+%27-3+years%27%29%0D%0A%29%0D%0Aselect+t%2C+dateutil_parse_fuzzy%28%22This+is+due+10+september%22%2C+t%29+from+times" rel="nofollow">Example query demonstrating the default date argument</a></p>
  <h3 dir="auto"><a id="user-content-calculating-easter" class="anchor" aria-hidden="true" href="#user-content-calculating-easter"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Calculating Easter</h3>
  <ul dir="auto">
  <li><code>dateutil_easter(year)</code> - returns the date for Easter in that year, for example <code>dateutil_easter("2020")</code> returns <code>2020-04-12</code>.</li>
  </ul>
  <p dir="auto"><a href="https://latest-with-plugins.datasette.io/fixtures?sql=select%0D%0A++dateutil_easter%282019%29%2C%0D%0A++dateutil_easter%282020%29%2C%0D%0A++dateutil_easter%282021%29" rel="nofollow">Example Easter query</a></p>
  <h3 dir="auto"><a id="user-content-json-arrays-of-dates" class="anchor" aria-hidden="true" href="#user-content-json-arrays-of-dates"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>JSON arrays of dates</h3>
  <p dir="auto">Several functions return JSON arrays of date strings. These can be used with SQLite's <code>json_each()</code> function to perform joins against dates from a specific date range or recurrence rule.</p>
  <p dir="auto">These functions can return up to 10,000 results. They will return an error if more than 10,000 dates would be returned - this is to protect against denial of service attacks.</p>
  <ul dir="auto">
  <li><code>dateutil_dates_between('1 january 2020', '5 jan 2020')</code> - given two dates (in any format that can be handled by <code>dateutil_parse()</code>) this function returns a JSON string containing the dates between those two days, inclusive. This example returns <code>["2020-01-01", "2020-01-02", "2020-01-03", "2020-01-04", "2020-01-05"]</code>.</li>
  <li><code>dateutil_dates_between('1 january 2020', '5 jan 2020', 0)</code> - set the optional third argument to <code>0</code> to specify that you would like this to be exclusive of the last day. This example returns <code>["2020-01-01", "2020-01-02", "2020-01-03", "2020-01-04"]</code>.</li>
  </ul>
  <p dir="auto"><a href="https://latest-with-plugins.datasette.io/fixtures?sql=select%0D%0A++dateutil_dates_between%28%271+january+2020%27%2C+%275+jan+2020%27%29%2C%0D%0A++dateutil_dates_between%28%271+january+2020%27%2C+%275+jan+2020%27%2C+0%29" rel="nofollow">Try these queries</a></p>
  <p dir="auto">The <code>dateutil_rrule()</code> and <code>dateutil_rrule_date()</code> functions accept the iCalendar standard ``rrule` format - see <a href="https://dateutil.readthedocs.io/en/stable/rrule.html#rrulestr-examples" rel="nofollow">the dateutil documentation</a> for more examples.</p>
  <p dir="auto">This format lets you specify recurrence rules such as "the next four last mondays of the month".</p>
  <ul dir="auto">
  <li><code>dateutil_rrule(rrule, optional_dtsart)</code> - given an rrule returns a JSON array of ISO datetimes. The second argument is optional and will be treated as the start date for the rule.</li>
  <li><code>dateutil_rrule_date(rrule, optional_dtsart)</code> - same as <code>dateutil_rrule()</code> but returns ISO dates.</li>
  </ul>
  <p dir="auto">Example query:</p>
  <div class="highlight highlight-source-sql position-relative overflow-auto" data-snippet-clipboard-copy-content="select
    dateutil_rrule('FREQ=HOURLY;COUNT=5'),
    dateutil_rrule_date(
      'FREQ=DAILY;COUNT=3',
      '1st jan 2020'
    );"><pre><span class="pl-k">select</span>
    dateutil_rrule(<span class="pl-s"><span class="pl-pds">'</span>FREQ=HOURLY;COUNT=5<span class="pl-pds">'</span></span>),
    dateutil_rrule_date(
      <span class="pl-s"><span class="pl-pds">'</span>FREQ=DAILY;COUNT=3<span class="pl-pds">'</span></span>,
      <span class="pl-s"><span class="pl-pds">'</span>1st jan 2020<span class="pl-pds">'</span></span>
    );</pre></div>
  <p dir="auto"><a href="https://latest-with-plugins.datasette.io/fixtures?sql=select%0D%0A++dateutil_rrule('FREQ%3DHOURLY%3BCOUNT%3D5')%2C%0D%0A++dateutil_rrule_date(%0D%0A++++'FREQ%3DDAILY%3BCOUNT%3D3'%2C%0D%0A++++'1st+jan+2020'%0D%0A++)%3B" rel="nofollow">Try the rrule example query</a></p>
  <h3 dir="auto"><a id="user-content-joining-data-using-json_each" class="anchor" aria-hidden="true" href="#user-content-joining-data-using-json_each"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Joining data using json_each()</h3>
  <p dir="auto">SQLite's <a href="https://www.sqlite.org/json1.html#jeach" rel="nofollow">json_each() function</a> can be used to turn a JSON array of dates into a table that can be joined against other data. Here's a query that returns a table showing every day in January 2019:</p>
  <div class="highlight highlight-source-sql position-relative overflow-auto" data-snippet-clipboard-copy-content="select
    value as date
  from
    json_each(
      dateutil_dates_between('1 Jan 2019', '31 Jan 2019')
    )"><pre><span class="pl-k">select</span>
    value <span class="pl-k">as</span> <span class="pl-k">date</span>
  <span class="pl-k">from</span>
    json_each(
      dateutil_dates_between(<span class="pl-s"><span class="pl-pds">'</span>1 Jan 2019<span class="pl-pds">'</span></span>, <span class="pl-s"><span class="pl-pds">'</span>31 Jan 2019<span class="pl-pds">'</span></span>)
    )</pre></div>
  <p dir="auto"><a href="https://latest-with-plugins.datasette.io/fixtures?sql=select%0D%0A++value+as+date%0D%0Afrom%0D%0A++json_each%28%0D%0A++++dateutil_dates_between%28%271+Jan+2019%27%2C+%2731+Jan+2019%27%29%0D%0A++%29" rel="nofollow">Try that query</a></p>
  <p dir="auto">You can run joins against this table by assigning it a name using SQLite's <a href="https://sqlite.org/lang_with.html" rel="nofollow">support for Common Table Expressions (CTEs)</a>.</p>
  <p dir="auto">This example query uses <code>substr(created, 0, 11)</code> to retrieve the date portion of the <code>created</code> column in the <a href="https://latest-with-plugins.datasette.io/fixtures/facetable" rel="nofollow">facetable demo table</a>, then joins that against the table of days in January to calculate the count of rows created on each day. The <code>LEFT JOIN</code> against <code>days_in_january</code> ensures that days which had no created records are still returned in the results, with a count of 0.</p>
  <div class="highlight highlight-source-sql position-relative overflow-auto" data-snippet-clipboard-copy-content="with created_dates as (
    select
      substr(created, 0, 11) as date
    from
      facetable
  ),
  days_in_january as (
    select
      value as date
    from
      json_each(
        dateutil_dates_between('1 Jan 2019', '31 Jan 2019')
      )
  )
  select
    days_in_january.date,
    count(created_dates.date) as total
  from
    days_in_january
    left join created_dates on days_in_january.date = created_dates.date
  group by
    days_in_january.date;"><pre>with created_dates <span class="pl-k">as</span> (
    <span class="pl-k">select</span>
      substr(created, <span class="pl-c1">0</span>, <span class="pl-c1">11</span>) <span class="pl-k">as</span> <span class="pl-k">date</span>
    <span class="pl-k">from</span>
      facetable
  ),
  days_in_january <span class="pl-k">as</span> (
    <span class="pl-k">select</span>
      value <span class="pl-k">as</span> <span class="pl-k">date</span>
    <span class="pl-k">from</span>
      json_each(
        dateutil_dates_between(<span class="pl-s"><span class="pl-pds">'</span>1 Jan 2019<span class="pl-pds">'</span></span>, <span class="pl-s"><span class="pl-pds">'</span>31 Jan 2019<span class="pl-pds">'</span></span>)
      )
  )
  <span class="pl-k">select</span>
    <span class="pl-c1">days_in_january</span>.<span class="pl-c1">date</span>,
    <span class="pl-c1">count</span>(<span class="pl-c1">created_dates</span>.<span class="pl-c1">date</span>) <span class="pl-k">as</span> total
  <span class="pl-k">from</span>
    days_in_january
    <span class="pl-k">left join</span> created_dates <span class="pl-k">on</span> <span class="pl-c1">days_in_january</span>.<span class="pl-c1">date</span> <span class="pl-k">=</span> <span class="pl-c1">created_dates</span>.<span class="pl-c1">date</span>
  <span class="pl-k">group by</span>
    <span class="pl-c1">days_in_january</span>.<span class="pl-c1">date</span>;</pre></div>
  <p dir="auto"><a href="https://latest-with-plugins.datasette.io/fixtures?sql=with+created_dates+as+%28%0D%0A++select%0D%0A++++substr%28created%2C+0%2C+11%29+as+date%0D%0A++from%0D%0A++++facetable%0D%0A%29%2C%0D%0Adays_in_january+as+%28%0D%0A++select%0D%0A++++value+as+date%0D%0A++from%0D%0A++++json_each%28%0D%0A++++++dateutil_dates_between%28%271+Jan+2019%27%2C+%2731+Jan+2019%27%29%0D%0A++++%29%0D%0A%29%0D%0Aselect%0D%0A++days_in_january.date%2C%0D%0A++count%28created_dates.date%29+as+total%0D%0Afrom%0D%0A++days_in_january%0D%0A++left+join+created_dates+on+days_in_january.date+%3D+created_dates.date%0D%0Agroup+by%0D%0A++days_in_january.date%3B#g.mark=bar&amp;g.x_column=date&amp;g.x_type=ordinal&amp;g.y_column=total&amp;g.y_type=quantitative" rel="nofollow">Try that query</a> with a bar chart rendered using the <a href="https://github.com/simonw/datasette-vega">datasette-vega</a> plugin.</p>
  <h2 dir="auto"><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p dir="auto">To set up this plugin locally, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="cd datasette-dateutil
  python3 -mvenv venv
  source venv/bin/activate"><pre><code>cd datasette-dateutil
  python3 -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p dir="auto">Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pipenv shell"><pre><code>pipenv shell
  </code></pre></div>
  <p dir="auto">Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pip install -e '.[test]'"><pre><code>pip install -e '.[test]'
  </code></pre></div>
  <p dir="auto">To run the tests:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pytest"><pre><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkyOTkxOTgzNjk=
name: datasette-import-table
full_name: simonw/datasette-import-table
html_url: https://github.com/simonw/datasette-import-table
description: Datasette plugin for importing tables from other Datasette instances
created_at: 2020-09-28T05:30:07Z
updated_at: 2022-06-09T15:27:33Z
pushed_at: 2022-06-09T16:40:22Z
language: Python
topics: ["datasette", "datasette-io", "datasette-plugin"]
default_branch: main
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
visibility: public
readme:
  # datasette-import-table

  [![PyPI](https://img.shields.io/pypi/v/datasette-import-table.svg)](https://pypi.org/project/datasette-import-table/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-import-table?include_prereleases&label=changelog)](https://github.com/simonw/datasette-import-table/releases)
  [![Tests](https://github.com/simonw/datasette-import-table/workflows/Test/badge.svg)](https://github.com/simonw/datasette-import-table/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-import-table/blob/main/LICENSE)

  Datasette plugin for importing tables from other Datasette instances

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-import-table

  ## Usage

  Visit `/-/import-table` for the interface. Paste in the URL to a table page on another Datasette instance and click the button to import that table.

  By default only [the root actor](https://datasette.readthedocs.io/en/stable/authentication.html#using-the-root-actor) can access the page - so you'll need to run Datasette with the `--root` option and click on the link shown in the terminal to sign in and access the page.

  The `import-table` permission governs access. You can use permission plugins such as [datasette-permissions-sql](https://github.com/simonw/datasette-permissions-sql) to grant additional access to the write interface.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-import-table
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-datasette-import-table" class="anchor" aria-hidden="true" href="#user-content-datasette-import-table"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-import-table</h1>
  <p dir="auto"><a href="https://pypi.org/project/datasette-import-table/" rel="nofollow"><img src="https://camo.githubusercontent.com/5ca70be082ec2a51a5a9ac69089d124ec92434735a9d91f42d65e08acca81519/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d696d706f72742d7461626c652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-import-table.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-import-table/releases"><img src="https://camo.githubusercontent.com/dd97f8f33028724e99b858c1b6f002b6d4463796b64d0bf405b316b690270127/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d696d706f72742d7461626c653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-import-table?include_prereleases&amp;label=changelog" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-import-table/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-import-table/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-import-table/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width: 100%;"></a></p>
  <p dir="auto">Datasette plugin for importing tables from other Datasette instances</p>
  <h2 dir="auto"><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p dir="auto">Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="$ datasette install datasette-import-table"><pre class="notranslate"><code>$ datasette install datasette-import-table
  </code></pre></div>
  <h2 dir="auto"><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p dir="auto">Visit <code>/-/import-table</code> for the interface. Paste in the URL to a table page on another Datasette instance and click the button to import that table.</p>
  <p dir="auto">By default only <a href="https://datasette.readthedocs.io/en/stable/authentication.html#using-the-root-actor" rel="nofollow">the root actor</a> can access the page - so you'll need to run Datasette with the <code>--root</code> option and click on the link shown in the terminal to sign in and access the page.</p>
  <p dir="auto">The <code>import-table</code> permission governs access. You can use permission plugins such as <a href="https://github.com/simonw/datasette-permissions-sql">datasette-permissions-sql</a> to grant additional access to the write interface.</p>
  <h2 dir="auto"><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p dir="auto">To set up this plugin locally, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="cd datasette-import-table
  python3 -mvenv venv
  source venv/bin/activate"><pre class="notranslate"><code>cd datasette-import-table
  python3 -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p dir="auto">Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="pipenv shell"><pre class="notranslate"><code>pipenv shell
  </code></pre></div>
  <p dir="auto">Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="pip install -e '.[test]'"><pre class="notranslate"><code>pip install -e '.[test]'
  </code></pre></div>
  <p dir="auto">To run the tests:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="pytest"><pre class="notranslate"><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkzMDMyMTgzNjk=
name: evernote-to-sqlite
full_name: dogsheep/evernote-to-sqlite
html_url: https://github.com/dogsheep/evernote-to-sqlite
description: Tools for converting Evernote content to SQLite
created_at: 2020-10-11T21:45:49Z
updated_at: 2021-08-26T19:01:54Z
pushed_at: 2021-08-26T19:02:47Z
language: Python
license: apache-2.0
topics: ["datasette-io", "datasette-tool", "dogsheep", "evernote", "sqlite"]
default_branch: main
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
readme:
  # evernote-to-sqlite

  [![PyPI](https://img.shields.io/pypi/v/evernote-to-sqlite.svg)](https://pypi.org/project/evernote-to-sqlite/)
  [![Changelog](https://img.shields.io/github/v/release/dogsheep/evernote-to-sqlite?include_prereleases&label=changelog)](https://github.com/dogsheep/evernote-to-sqlite/releases)
  [![Tests](https://github.com/dogsheep/evernote-to-sqlite/workflows/Test/badge.svg)](https://github.com/dogsheep/evernote-to-sqlite/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/dogsheep/evernote-to-sqlite/blob/master/LICENSE)

  Tools for converting Evernote content to SQLite. See [Building an Evernote to SQLite exporter](https://simonwillison.net/2020/Oct/16/building-evernote-sqlite-exporter/) for background on this project.

  ## Installation

  Install this tool using `pip`:

      $ pip install evernote-to-sqlite

  ## Usage

  Currently the only available command is `evernote-to-sqlite enex`, which converts Evernote's ENEX export files into a SQLite database.

  You can create [an ENEX export](https://help.evernote.com/hc/en-us/articles/209005557-Export-notes-and-notebooks-as-ENEX-or-HTML) in the Evernote desktop application by selecting some notes (or all of your notes) and using the `File -> Export Notes...` menu option.

  This used to be able to export everything in one go, but it looks like more recent Evernote versions only allow exporting up to fifty notes at a time, or let you export an entire notebook by right-clicking on the notebook and selecting "Export notebook...".

  You can convert that file to SQLite like so:

      $ evernote-to-sqlite enex evernote.db MyNotes.enex

  This will display a progress bar and create a SQLite database file called `evernote.db`.

  ### Limitations

  Unfortunately the ENEX export format does not include a unique identifier for each note. This means you cannot use this tool to re-import notes after they have been updated - you should consider this tool to be a one-time transformation of an ENEX file into an equivalent SQLite database.

  ENEX exports also do not include details of which notebook a note belongs to.

  ## Development

  To contribute to this tool, first checkout the code. Then create a new virtual environment:

      cd evernote-to-sqlite
      python -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-evernote-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-evernote-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>evernote-to-sqlite</h1>
  <p><a href="https://pypi.org/project/evernote-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/0035842afd045b61cc5c6ee9c8394c181ac4daf3530bf212349f37a05bebcdbb/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f657665726e6f74652d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/evernote-to-sqlite.svg" style="max-width:100%;"></a>
  <a href="https://github.com/dogsheep/evernote-to-sqlite/releases"><img src="https://camo.githubusercontent.com/b51eee78bbc51296102af124839ed665bf3f17755f4dbff95c4bc8ed0713d763/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f646f6773686565702f657665726e6f74652d746f2d73716c6974653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/dogsheep/evernote-to-sqlite?include_prereleases&amp;label=changelog" style="max-width:100%;"></a>
  <a href="https://github.com/dogsheep/evernote-to-sqlite/actions?query=workflow%3ATest"><img src="https://github.com/dogsheep/evernote-to-sqlite/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a>
  <a href="https://github.com/dogsheep/evernote-to-sqlite/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p>
  <p>Tools for converting Evernote content to SQLite. See <a href="https://simonwillison.net/2020/Oct/16/building-evernote-sqlite-exporter/" rel="nofollow">Building an Evernote to SQLite exporter</a> for background on this project.</p>
  <h2><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p>Install this tool using <code>pip</code>:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ pip install evernote-to-sqlite
  "><pre><code>$ pip install evernote-to-sqlite
  </code></pre></div>
  <h2><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p>Currently the only available command is <code>evernote-to-sqlite enex</code>, which converts Evernote's ENEX export files into a SQLite database.</p>
  <p>You can create <a href="https://help.evernote.com/hc/en-us/articles/209005557-Export-notes-and-notebooks-as-ENEX-or-HTML" rel="nofollow">an ENEX export</a> in the Evernote desktop application by selecting some notes (or all of your notes) and using the <code>File -&gt; Export Notes...</code> menu option.</p>
  <p>This used to be able to export everything in one go, but it looks like more recent Evernote versions only allow exporting up to fifty notes at a time, or let you export an entire notebook by right-clicking on the notebook and selecting "Export notebook...".</p>
  <p>You can convert that file to SQLite like so:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ evernote-to-sqlite enex evernote.db MyNotes.enex
  "><pre><code>$ evernote-to-sqlite enex evernote.db MyNotes.enex
  </code></pre></div>
  <p>This will display a progress bar and create a SQLite database file called <code>evernote.db</code>.</p>
  <h3><a id="user-content-limitations" class="anchor" aria-hidden="true" href="#user-content-limitations"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Limitations</h3>
  <p>Unfortunately the ENEX export format does not include a unique identifier for each note. This means you cannot use this tool to re-import notes after they have been updated - you should consider this tool to be a one-time transformation of an ENEX file into an equivalent SQLite database.</p>
  <p>ENEX exports also do not include details of which notebook a note belongs to.</p>
  <h2><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p>To contribute to this tool, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="cd evernote-to-sqlite
  python -mvenv venv
  source venv/bin/activate
  "><pre><code>cd evernote-to-sqlite
  python -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p>Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pipenv shell
  "><pre><code>pipenv shell
  </code></pre></div>
  <p>Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pip install -e '.[test]'
  "><pre><code>pip install -e '.[test]'
  </code></pre></div>
  <p>To run the tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pytest
  "><pre><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkzMDUxOTk2NjE=
name: sphinx-to-sqlite
full_name: simonw/sphinx-to-sqlite
html_url: https://github.com/simonw/sphinx-to-sqlite
description: Create a SQLite database from Sphinx documentation
created_at: 2020-10-18T21:26:55Z
updated_at: 2020-12-19T05:08:12Z
pushed_at: 2020-10-22T04:55:45Z
language: Python
license: apache-2.0
topics: ["sqlite", "sphinx", "datasette-io", "datasette-tool"]
default_branch: main
permissions: {"admin": false, "push": false, "pull": false}
readme:
  # sphinx-to-sqlite

  [![PyPI](https://img.shields.io/pypi/v/sphinx-to-sqlite.svg)](https://pypi.org/project/sphinx-to-sqlite/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/sphinx-to-sqlite?include_prereleases&label=changelog)](https://github.com/simonw/sphinx-to-sqlite/releases)
  [![Tests](https://github.com/simonw/sphinx-to-sqlite/workflows/Test/badge.svg)](https://github.com/simonw/sphinx-to-sqlite/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/sphinx-to-sqlite/blob/master/LICENSE)

  Create a SQLite database from Sphinx documentation.

  ## Demo

  You can see the results of running this tool against the [Datasette documentation](https://docs.datasette.io/) at https://latest-docs.datasette.io/docs/sections

  ## Installation

  Install this tool using `pip`:

      $ pip install sphinx-to-sqlite

  ## Usage

  First run `sphinx-build` with the `-b xml` option to create XML files in your `_build/` directory.

  Then run:

      $ sphinx-to-sqlite docs.db path/to/_build

  To build the SQLite database.

  ## Development

  To contribute to this tool, first checkout the code. Then create a new virtual environment:

      cd sphinx-to-sqlite
      python -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-sphinx-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-sphinx-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>sphinx-to-sqlite</h1>
  <p><a href="https://pypi.org/project/sphinx-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/877d22c6402df75a56257c7d5a426d35ee787ab139307e4a7060c7d706b4c6cd/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f737068696e782d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/sphinx-to-sqlite.svg" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/sphinx-to-sqlite/releases"><img src="https://camo.githubusercontent.com/69509856ca4dd51d3c9e67530adbd0f1f662719a608252a8d9cb5bdc3b90590b/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f737068696e782d746f2d73716c6974653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/sphinx-to-sqlite?include_prereleases&amp;label=changelog" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/sphinx-to-sqlite/actions?query=workflow%3ATest"><img src="https://github.com/simonw/sphinx-to-sqlite/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/sphinx-to-sqlite/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p>
  <p>Create a SQLite database from Sphinx documentation.</p>
  <h2><a id="user-content-demo" class="anchor" aria-hidden="true" href="#user-content-demo"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Demo</h2>
  <p>You can see the results of running this tool against the <a href="https://docs.datasette.io/" rel="nofollow">Datasette documentation</a> at <a href="https://latest-docs.datasette.io/docs/sections" rel="nofollow">https://latest-docs.datasette.io/docs/sections</a></p>
  <h2><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p>Install this tool using <code>pip</code>:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ pip install sphinx-to-sqlite
  "><pre><code>$ pip install sphinx-to-sqlite
  </code></pre></div>
  <h2><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p>First run <code>sphinx-build</code> with the <code>-b xml</code> option to create XML files in your <code>_build/</code> directory.</p>
  <p>Then run:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ sphinx-to-sqlite docs.db path/to/_build
  "><pre><code>$ sphinx-to-sqlite docs.db path/to/_build
  </code></pre></div>
  <p>To build the SQLite database.</p>
  <h2><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p>To contribute to this tool, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="cd sphinx-to-sqlite
  python -mvenv venv
  source venv/bin/activate
  "><pre><code>cd sphinx-to-sqlite
  python -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p>Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pipenv shell
  "><pre><code>pipenv shell
  </code></pre></div>
  <p>Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pip install -e '.[test]'
  "><pre><code>pip install -e '.[test]'
  </code></pre></div>
  <p>To run the tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pytest
  "><pre><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkzMDg5MzAxMTg=
name: datasette-edit-templates
full_name: simonw/datasette-edit-templates
html_url: https://github.com/simonw/datasette-edit-templates
description: Plugin allowing Datasette templates to be edited within Datasette
created_at: 2020-10-31T16:58:29Z
updated_at: 2022-09-14T20:59:49Z
pushed_at: 2022-10-27T23:00:04Z
language: Python
topics: []
default_branch: main
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
visibility: public
readme:
  # datasette-edit-templates

  [![PyPI](https://img.shields.io/pypi/v/datasette-edit-templates.svg)](https://pypi.org/project/datasette-edit-templates/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-edit-templates?include_prereleases&label=changelog)](https://github.com/simonw/datasette-edit-templates/releases)
  [![Tests](https://github.com/simonw/datasette-edit-templates/workflows/Test/badge.svg)](https://github.com/simonw/datasette-edit-templates/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-edit-templates/blob/main/LICENSE)

  Plugin allowing Datasette templates to be edited within Datasette.

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-edit-templates

  ## Usage

  Once installed, sign in as the root user using `datasette mydb.db --root`.

  On startup. a `_templates_` table will be created in the database you are running Datasette against.

  Use the app menu to navigate to the `/-/edit-templates` page, and edit templates there.

  Changes should become visible instantly, and will be persisted to your database.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-edit-templates
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-datasette-edit-templates" class="anchor" aria-hidden="true" href="#user-content-datasette-edit-templates"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-edit-templates</h1>
  <p dir="auto"><a href="https://pypi.org/project/datasette-edit-templates/" rel="nofollow"><img src="https://camo.githubusercontent.com/8bd45e7c1cf6353064781737c607c810b3f222dbac8997eadeb42439d58217e1/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d656469742d74656d706c617465732e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-edit-templates.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-edit-templates/releases"><img src="https://camo.githubusercontent.com/d5f67a37f11adcc81a796bf7521637fc57bfe0eacb3c03983481bcb964ad4f86/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d656469742d74656d706c617465733f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-edit-templates?include_prereleases&amp;label=changelog" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-edit-templates/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-edit-templates/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-edit-templates/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width: 100%;"></a></p>
  <p dir="auto">Plugin allowing Datasette templates to be edited within Datasette.</p>
  <h2 dir="auto"><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p dir="auto">Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="$ datasette install datasette-edit-templates"><pre class="notranslate"><code>$ datasette install datasette-edit-templates
  </code></pre></div>
  <h2 dir="auto"><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p dir="auto">Once installed, sign in as the root user using <code>datasette mydb.db --root</code>.</p>
  <p dir="auto">On startup. a <code>_templates_</code> table will be created in the database you are running Datasette against.</p>
  <p dir="auto">Use the app menu to navigate to the <code>/-/edit-templates</code> page, and edit templates there.</p>
  <p dir="auto">Changes should become visible instantly, and will be persisted to your database.</p>
  <h2 dir="auto"><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p dir="auto">To set up this plugin locally, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="cd datasette-edit-templates
  python3 -mvenv venv
  source venv/bin/activate"><pre class="notranslate"><code>cd datasette-edit-templates
  python3 -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p dir="auto">Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="pipenv shell"><pre class="notranslate"><code>pipenv shell
  </code></pre></div>
  <p dir="auto">Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="pip install -e '.[test]'"><pre class="notranslate"><code>pip install -e '.[test]'
  </code></pre></div>
  <p dir="auto">To run the tests:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="pytest"><pre class="notranslate"><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkzMTU3OTYwMTU=
name: datasette-ripgrep
full_name: simonw/datasette-ripgrep
html_url: https://github.com/simonw/datasette-ripgrep
description: Web interface for searching your code using ripgrep, built as a Datasette plugin
created_at: 2020-11-25T01:26:36Z
updated_at: 2022-04-24T03:48:42Z
pushed_at: 2022-06-30T22:45:03Z
homepage: https://ripgrep.datasette.io
language: Python
license: apache-2.0
topics: ["codesearch", "datasette", "datasette-io", "datasette-plugin", "ripgrep"]
default_branch: main
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
visibility: public
readme:
  # datasette-ripgrep

  [![PyPI](https://img.shields.io/pypi/v/datasette-ripgrep.svg)](https://pypi.org/project/datasette-ripgrep/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-ripgrep?include_prereleases&label=changelog)](https://github.com/simonw/datasette-ripgrep/releases)
  [![Tests](https://github.com/simonw/datasette-ripgrep/workflows/Test/badge.svg)](https://github.com/simonw/datasette-ripgrep/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-ripgrep/blob/main/LICENSE)

  Web interface for searching your code using [ripgrep](https://github.com/BurntSushi/ripgrep), built as a [Datasette](https://datasette.io/) plugin

  For background on this project see [datasette-ripgrep: deploy a regular expression search engine for your source code](https://simonwillison.net/2020/Nov/28/datasette-ripgrep/).

  ## Demo

  Try this plugin out at https://ripgrep.datasette.io/-/ripgrep - where you can run regular expression searches across the source code of Datasette and all of the `datasette-*` plugins belonging to the [simonw GitHub user](https://github.com/simonw).

  Some example searches:

  - [with.\*AsyncClient](https://ripgrep.datasette.io/-/ripgrep?pattern=with.*AsyncClient) - regular expression search for `with.*AsyncClient`
  - [.plugin_config, literal=on](https://ripgrep.datasette.io/-/ripgrep?pattern=.plugin_config\(&literal=on) - a non-regular expression search for `.plugin_config(`
  - [with.\*AsyncClient glob=datasette/\*\*](https://ripgrep.datasette.io/-/ripgrep?pattern=with.*AsyncClient&glob=datasette%2F%2A%2A) - search for that pattern only within the `datasette/` top folder
  - ["sqlite-utils\[">\] glob=setup.py](https://ripgrep.datasette.io/-/ripgrep?pattern=%22sqlite-utils%5B%22%3E%5D&glob=setup.py) - a regular expression search for packages that depend on either `sqlite-utils` or `sqlite-utils>=some-version`
  - [test glob=!\*.html](https://ripgrep.datasette.io/-/ripgrep?pattern=test&glob=%21*.html) - search for the string `test` but exclude results in HTML files

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-ripgrep

  The `rg` executable needs to be [installed](https://github.com/BurntSushi/ripgrep/blob/master/README.md#installation) such that it can be run by this tool.

  ## Usage

  This plugin requires configuration: it needs to a `path` setting so that it knows where to run searches.

  Create a `metadata.json` file that looks like this:

  ```json
  {
      "plugins": {
          "datasette-ripgrep": {
              "path": "/path/to/your/files"
          }
      }
  }
  ```

  Now run Datasette using `datasette -m metadata.json`. The plugin will add an interface at `/-/ripgrep` for running searches.

  ## Plugin configuration

  The `"path"` configuration is required. Optional extra configuration options are:

  - `time_limit` - floating point number. The `rg` process will be terminated if it takes longer than this limit. The default is one second, `1.0`.
  - `max_lines` - integer. The `rg` process will be terminated if it returns more than this number of lines. The default is `2000`.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-ripgrep
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-datasette-ripgrep" class="anchor" aria-hidden="true" href="#user-content-datasette-ripgrep"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-ripgrep</h1>
  <p dir="auto"><a href="https://pypi.org/project/datasette-ripgrep/" rel="nofollow"><img src="https://camo.githubusercontent.com/ebcbf381887ee3c4ed1c9da54876d414e77b4a1c7eb1056f8ea8c7b4a24e8156/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d726970677265702e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-ripgrep.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-ripgrep/releases"><img src="https://camo.githubusercontent.com/355fe4e16e5f39a0efea997bbb0d4b0eead22e1975695ca00351fa8f14deb370/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d726970677265703f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-ripgrep?include_prereleases&amp;label=changelog" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-ripgrep/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-ripgrep/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-ripgrep/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width: 100%;"></a></p>
  <p dir="auto">Web interface for searching your code using <a href="https://github.com/BurntSushi/ripgrep">ripgrep</a>, built as a <a href="https://datasette.io/" rel="nofollow">Datasette</a> plugin</p>
  <p dir="auto">For background on this project see <a href="https://simonwillison.net/2020/Nov/28/datasette-ripgrep/" rel="nofollow">datasette-ripgrep: deploy a regular expression search engine for your source code</a>.</p>
  <h2 dir="auto"><a id="user-content-demo" class="anchor" aria-hidden="true" href="#user-content-demo"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Demo</h2>
  <p dir="auto">Try this plugin out at <a href="https://ripgrep.datasette.io/-/ripgrep" rel="nofollow">https://ripgrep.datasette.io/-/ripgrep</a> - where you can run regular expression searches across the source code of Datasette and all of the <code>datasette-*</code> plugins belonging to the <a href="https://github.com/simonw">simonw GitHub user</a>.</p>
  <p dir="auto">Some example searches:</p>
  <ul dir="auto">
  <li><a href="https://ripgrep.datasette.io/-/ripgrep?pattern=with.*AsyncClient" rel="nofollow">with.*AsyncClient</a> - regular expression search for <code>with.*AsyncClient</code></li>
  <li><a href="https://ripgrep.datasette.io/-/ripgrep?pattern=.plugin_config(&amp;literal=on" rel="nofollow">.plugin_config, literal=on</a> - a non-regular expression search for <code>.plugin_config(</code></li>
  <li><a href="https://ripgrep.datasette.io/-/ripgrep?pattern=with.*AsyncClient&amp;glob=datasette%2F%2A%2A" rel="nofollow">with.*AsyncClient glob=datasette/**</a> - search for that pattern only within the <code>datasette/</code> top folder</li>
  <li><a href="https://ripgrep.datasette.io/-/ripgrep?pattern=%22sqlite-utils%5B%22%3E%5D&amp;glob=setup.py" rel="nofollow">"sqlite-utils["&gt;] glob=setup.py</a> - a regular expression search for packages that depend on either <code>sqlite-utils</code> or <code>sqlite-utils&gt;=some-version</code></li>
  <li><a href="https://ripgrep.datasette.io/-/ripgrep?pattern=test&amp;glob=%21*.html" rel="nofollow">test glob=!*.html</a> - search for the string <code>test</code> but exclude results in HTML files</li>
  </ul>
  <h2 dir="auto"><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p dir="auto">Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="$ datasette install datasette-ripgrep"><pre class="notranslate"><code>$ datasette install datasette-ripgrep
  </code></pre></div>
  <p dir="auto">The <code>rg</code> executable needs to be <a href="https://github.com/BurntSushi/ripgrep/blob/master/README.md#installation">installed</a> such that it can be run by this tool.</p>
  <h2 dir="auto"><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p dir="auto">This plugin requires configuration: it needs to a <code>path</code> setting so that it knows where to run searches.</p>
  <p dir="auto">Create a <code>metadata.json</code> file that looks like this:</p>
  <div class="highlight highlight-source-json notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="{
      &quot;plugins&quot;: {
          &quot;datasette-ripgrep&quot;: {
              &quot;path&quot;: &quot;/path/to/your/files&quot;
          }
      }
  }"><pre>{
      <span class="pl-ent">"plugins"</span>: {
          <span class="pl-ent">"datasette-ripgrep"</span>: {
              <span class="pl-ent">"path"</span>: <span class="pl-s"><span class="pl-pds">"</span>/path/to/your/files<span class="pl-pds">"</span></span>
          }
      }
  }</pre></div>
  <p dir="auto">Now run Datasette using <code>datasette -m metadata.json</code>. The plugin will add an interface at <code>/-/ripgrep</code> for running searches.</p>
  <h2 dir="auto"><a id="user-content-plugin-configuration" class="anchor" aria-hidden="true" href="#user-content-plugin-configuration"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Plugin configuration</h2>
  <p dir="auto">The <code>"path"</code> configuration is required. Optional extra configuration options are:</p>
  <ul dir="auto">
  <li><code>time_limit</code> - floating point number. The <code>rg</code> process will be terminated if it takes longer than this limit. The default is one second, <code>1.0</code>.</li>
  <li><code>max_lines</code> - integer. The <code>rg</code> process will be terminated if it returns more than this number of lines. The default is <code>2000</code>.</li>
  </ul>
  <h2 dir="auto"><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p dir="auto">To set up this plugin locally, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="cd datasette-ripgrep
  python3 -mvenv venv
  source venv/bin/activate"><pre class="notranslate"><code>cd datasette-ripgrep
  python3 -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p dir="auto">Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="pipenv shell"><pre class="notranslate"><code>pipenv shell
  </code></pre></div>
  <p dir="auto">Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="pip install -e '.[test]'"><pre class="notranslate"><code>pip install -e '.[test]'
  </code></pre></div>
  <p dir="auto">To run the tests:</p>
  <div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="pytest"><pre class="notranslate"><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkzMjcwODcyMDc=
name: datasette-css-properties
full_name: simonw/datasette-css-properties
html_url: https://github.com/simonw/datasette-css-properties
description: Experimental Datasette output plugin using CSS properties
created_at: 2021-01-05T18:38:07Z
updated_at: 2021-01-12T17:43:11Z
pushed_at: 2021-01-07T22:07:19Z
language: Python
topics: ["datasette-plugin", "datasette-io"]
default_branch: main
permissions: {"admin": false, "push": false, "pull": false}
readme:
  # datasette-css-properties

  [![PyPI](https://img.shields.io/pypi/v/datasette-css-properties.svg)](https://pypi.org/project/datasette-css-properties/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-css-properties?include_prereleases&label=changelog)](https://github.com/simonw/datasette-css-properties/releases)
  [![Tests](https://github.com/simonw/datasette-css-properties/workflows/Test/badge.svg)](https://github.com/simonw/datasette-css-properties/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-css-properties/blob/main/LICENSE)

  Extremely experimental Datasette output plugin using CSS properties, inspired by [Custom Properties as State](https://css-tricks.com/custom-properties-as-state/) by Chris Coyier.

  More about this project: [APIs from CSS without JavaScript: the datasette-css-properties plugin](https://simonwillison.net/2021/Jan/7/css-apis-no-javascript/)

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-css-properties

  ## Usage

  Once installed, this plugin adds a `.css` output format to every query result. This will return the first row in the query as a valid CSS file, defining each column as a custom property:

  Example: https://latest-with-plugins.datasette.io/fixtures/roadside_attractions.css produces:

  ```css
  :root {
    --pk: '1';
    --name: 'The Mystery Spot';
    --address: '465 Mystery Spot Road, Santa Cruz, CA 95065';
    --latitude: '37.0167';
    --longitude: '-122.0024';
  }
  ```

  If you link this stylesheet to your page you can then do things like this;

  ```html
  <link rel="stylesheet" href="https://latest-with-plugins.datasette.io/fixtures/roadside_attractions.css">
  <style>
  .attraction-name:after { content: var(--name); }
  </style>
  <p class="attraction-name">Attraction name: </p>
  ```

  Values will be quoted as CSS strings by default. If you want to return a "raw" value without the quotes - for example to set a CSS property that is numeric or a color, you can specify that column name using the `?_raw=column-name` parameter. This can be passed multiple times.

  Consider [this example query](https://latest-with-plugins.datasette.io/github?sql=select%0D%0A++%27%23%27+||+substr(sha%2C+0%2C+6)+as+[custom-bg]%0D%0Afrom%0D%0A++commits%0D%0Aorder+by%0D%0A++author_date+desc%0D%0Alimit%0D%0A++1%3B):

  ```sql
  select
    '#' || substr(sha, 0, 6) as [custom-bg]
  from
    commits
  order by
    author_date desc
  limit
    1;
  ```

  This returns the first 6 characters of the most recently authored commit with a `#` prefix. The `.css` [output rendered version](https://latest-with-plugins.datasette.io/github.css?sql=select%0D%0A++%27%23%27+||+substr(sha%2C+0%2C+6)+as+[custom-bg]%0D%0Afrom%0D%0A++commits%0D%0Aorder+by%0D%0A++author_date+desc%0D%0Alimit%0D%0A++1%3B) looks like this:

  ```css
  :root {
    --custom-bg: '#97fb1';
  }
  ```

  Adding `?_raw=custom-bg` to the URL produces [this instead](https://latest-with-plugins.datasette.io/github.css?sql=select%0D%0A++%27%23%27+||+substr(sha%2C+0%2C+6)+as+[custom-bg]%0D%0Afrom%0D%0A++commits%0D%0Aorder+by%0D%0A++author_date+desc%0D%0Alimit%0D%0A++1%3B&_raw=custom-bg):

  ```css
  :root {
    --custom-bg: #97fb1;
  }
  ```

  This can then be used as a color value like so:

  ```css
  h1 {
      background-color: var(--custom-bg);
  }
  ```

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-css-properties
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-datasette-css-properties" class="anchor" aria-hidden="true" href="#user-content-datasette-css-properties"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-css-properties</h1>
  <p><a href="https://pypi.org/project/datasette-css-properties/" rel="nofollow"><img src="https://camo.githubusercontent.com/73ef89bdb78bfff22274d72faf1843c5862cc6072d2df7743d43361fb6f5cace/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d6373732d70726f706572746965732e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-css-properties.svg" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-css-properties/releases"><img src="https://camo.githubusercontent.com/a67e245842d1c8a894df0cbab28ce5643293cfbe70b75afedcea53403d0750b2/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d6373732d70726f706572746965733f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-css-properties?include_prereleases&amp;label=changelog" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-css-properties/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-css-properties/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-css-properties/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p>
  <p>Extremely experimental Datasette output plugin using CSS properties, inspired by <a href="https://css-tricks.com/custom-properties-as-state/" rel="nofollow">Custom Properties as State</a> by Chris Coyier.</p>
  <p>More about this project: <a href="https://simonwillison.net/2021/Jan/7/css-apis-no-javascript/" rel="nofollow">APIs from CSS without JavaScript: the datasette-css-properties plugin</a></p>
  <h2><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p>Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ datasette install datasette-css-properties
  "><pre><code>$ datasette install datasette-css-properties
  </code></pre></div>
  <h2><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p>Once installed, this plugin adds a <code>.css</code> output format to every query result. This will return the first row in the query as a valid CSS file, defining each column as a custom property:</p>
  <p>Example: <a href="https://latest-with-plugins.datasette.io/fixtures/roadside_attractions.css" rel="nofollow">https://latest-with-plugins.datasette.io/fixtures/roadside_attractions.css</a> produces:</p>
  <div class="highlight highlight-source-css position-relative" data-snippet-clipboard-copy-content=":root {
    --pk: '1';
    --name: 'The Mystery Spot';
    --address: '465 Mystery Spot Road, Santa Cruz, CA 95065';
    --latitude: '37.0167';
    --longitude: '-122.0024';
  }
  "><pre><span class="pl-kos">:</span><span class="pl-c1">root</span> {
    <span class="pl-c1">--pk</span><span class="pl-kos">:</span> <span class="pl-s">'1'</span>;
    <span class="pl-c1">--name</span><span class="pl-kos">:</span> <span class="pl-s">'The Mystery Spot'</span>;
    <span class="pl-c1">--address</span><span class="pl-kos">:</span> <span class="pl-s">'465 Mystery Spot Road, Santa Cruz, CA 95065'</span>;
    <span class="pl-c1">--latitude</span><span class="pl-kos">:</span> <span class="pl-s">'37.0167'</span>;
    <span class="pl-c1">--longitude</span><span class="pl-kos">:</span> <span class="pl-s">'-122.0024'</span>;
  }</pre></div>
  <p>If you link this stylesheet to your page you can then do things like this;</p>
  <div class="highlight highlight-text-html-basic position-relative" data-snippet-clipboard-copy-content="&lt;link rel=&quot;stylesheet&quot; href=&quot;https://latest-with-plugins.datasette.io/fixtures/roadside_attractions.css&quot;&gt;
  &lt;style&gt;
  .attraction-name:after { content: var(--name); }
  &lt;/style&gt;
  &lt;p class=&quot;attraction-name&quot;&gt;Attraction name: &lt;/p&gt;
  "><pre><span class="pl-kos">&lt;</span><span class="pl-ent">link</span> <span class="pl-c1">rel</span>="<span class="pl-s">stylesheet</span>" <span class="pl-c1">href</span>="<span class="pl-s">https://latest-with-plugins.datasette.io/fixtures/roadside_attractions.css</span>"<span class="pl-kos">&gt;</span>
  <span class="pl-kos">&lt;</span><span class="pl-ent">style</span><span class="pl-kos">&gt;</span>
  .<span class="pl-c1">attraction-name</span><span class="pl-kos">:</span><span class="pl-c1">after</span> { <span class="pl-c1">content</span><span class="pl-kos">:</span> <span class="pl-en">var</span>(<span class="pl-s1">--name</span>); }
  <span class="pl-kos">&lt;/</span><span class="pl-ent">style</span><span class="pl-kos">&gt;</span>
  <span class="pl-kos">&lt;</span><span class="pl-ent">p</span> <span class="pl-c1">class</span>="<span class="pl-s">attraction-name</span>"<span class="pl-kos">&gt;</span>Attraction name: <span class="pl-kos">&lt;/</span><span class="pl-ent">p</span><span class="pl-kos">&gt;</span></pre></div>
  <p>Values will be quoted as CSS strings by default. If you want to return a "raw" value without the quotes - for example to set a CSS property that is numeric or a color, you can specify that column name using the <code>?_raw=column-name</code> parameter. This can be passed multiple times.</p>
  <p>Consider <a href="https://latest-with-plugins.datasette.io/github?sql=select%0D%0A++%27%23%27+%7C%7C+substr(sha%2C+0%2C+6)+as+%5Bcustom-bg%5D%0D%0Afrom%0D%0A++commits%0D%0Aorder+by%0D%0A++author_date+desc%0D%0Alimit%0D%0A++1%3B" rel="nofollow">this example query</a>:</p>
  <div class="highlight highlight-source-sql position-relative" data-snippet-clipboard-copy-content="select
    '#' || substr(sha, 0, 6) as [custom-bg]
  from
    commits
  order by
    author_date desc
  limit
    1;
  "><pre><span class="pl-k">select</span>
    <span class="pl-s"><span class="pl-pds">'</span>#<span class="pl-pds">'</span></span> <span class="pl-k">||</span> substr(sha, <span class="pl-c1">0</span>, <span class="pl-c1">6</span>) <span class="pl-k">as</span> [custom<span class="pl-k">-</span>bg]
  <span class="pl-k">from</span>
    commits
  <span class="pl-k">order by</span>
    author_date <span class="pl-k">desc</span>
  <span class="pl-k">limit</span>
    <span class="pl-c1">1</span>;</pre></div>
  <p>This returns the first 6 characters of the most recently authored commit with a <code>#</code> prefix. The <code>.css</code> <a href="https://latest-with-plugins.datasette.io/github.css?sql=select%0D%0A++%27%23%27+%7C%7C+substr(sha%2C+0%2C+6)+as+%5Bcustom-bg%5D%0D%0Afrom%0D%0A++commits%0D%0Aorder+by%0D%0A++author_date+desc%0D%0Alimit%0D%0A++1%3B" rel="nofollow">output rendered version</a> looks like this:</p>
  <div class="highlight highlight-source-css position-relative" data-snippet-clipboard-copy-content=":root {
    --custom-bg: '#97fb1';
  }
  "><pre><span class="pl-kos">:</span><span class="pl-c1">root</span> {
    <span class="pl-c1">--custom-bg</span><span class="pl-kos">:</span> <span class="pl-s">'#97fb1'</span>;
  }</pre></div>
  <p>Adding <code>?_raw=custom-bg</code> to the URL produces <a href="https://latest-with-plugins.datasette.io/github.css?sql=select%0D%0A++%27%23%27+%7C%7C+substr(sha%2C+0%2C+6)+as+%5Bcustom-bg%5D%0D%0Afrom%0D%0A++commits%0D%0Aorder+by%0D%0A++author_date+desc%0D%0Alimit%0D%0A++1%3B&amp;_raw=custom-bg" rel="nofollow">this instead</a>:</p>
  <div class="highlight highlight-source-css position-relative" data-snippet-clipboard-copy-content=":root {
    --custom-bg: #97fb1;
  }
  "><pre><span class="pl-kos">:</span><span class="pl-c1">root</span> {
    <span class="pl-c1">--custom-bg</span><span class="pl-kos">:</span> <span class="pl-pds"><span class="pl-kos">#</span>97fb1</span>;
  }</pre></div>
  <p>This can then be used as a color value like so:</p>
  <div class="highlight highlight-source-css position-relative" data-snippet-clipboard-copy-content="h1 {
      background-color: var(--custom-bg);
  }
  "><pre><span class="pl-ent">h1</span> {
      <span class="pl-c1">background-color</span><span class="pl-kos">:</span> <span class="pl-en">var</span>(<span class="pl-s1">--custom-bg</span>);
  }</pre></div>
  <h2><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p>To set up this plugin locally, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="cd datasette-css-properties
  python3 -mvenv venv
  source venv/bin/activate
  "><pre><code>cd datasette-css-properties
  python3 -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p>Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pipenv shell
  "><pre><code>pipenv shell
  </code></pre></div>
  <p>Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pip install -e '.[test]'
  "><pre><code>pip install -e '.[test]'
  </code></pre></div>
  <p>To run the tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pytest
  "><pre><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkzMjcyMzYxMTk=
name: datasette-export-notebook
full_name: simonw/datasette-export-notebook
html_url: https://github.com/simonw/datasette-export-notebook
description: Datasette plugin providing instructions for exporting data to Jupyter or Observable
created_at: 2021-01-06T07:37:00Z
updated_at: 2021-12-23T23:19:42Z
pushed_at: 2021-12-23T23:19:38Z
language: Python
topics: ["datasette-io", "datasette-plugin"]
default_branch: main
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
visibility: public
readme:
  # datasette-export-notebook

  [![PyPI](https://img.shields.io/pypi/v/datasette-export-notebook.svg)](https://pypi.org/project/datasette-export-notebook/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-export-notebook?include_prereleases&label=changelog)](https://github.com/simonw/datasette-export-notebook/releases)
  [![Tests](https://github.com/simonw/datasette-export-notebook/workflows/Test/badge.svg)](https://github.com/simonw/datasette-export-notebook/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-export-notebook/blob/main/LICENSE)

  Datasette plugin providing instructions for exporting data to a [Jupyter](https://jupyter.org/) or [Observable](https://observablehq.com/) notebook.

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-export-notebook

  ## Usage

  Once installed, the plugin will add a `.Notebook` export option to every table and query. Clicking on this link will show instructions for exporting the data to Jupyter or Observable.

  ## Demo

  You can see this plugin in action on the [latest-with-plugins.datasette.io](https://latest-with-plugins.datasette.io/) Datasette instance - for example on [/github/commits.Notebook](https://latest-with-plugins.datasette.io/github/commits.Notebook).

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-export-notebook
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-datasette-export-notebook" class="anchor" aria-hidden="true" href="#user-content-datasette-export-notebook"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-export-notebook</h1>
  <p dir="auto"><a href="https://pypi.org/project/datasette-export-notebook/" rel="nofollow"><img src="https://camo.githubusercontent.com/55cb2eea5bd120b89a8a3ee593e89d0b0f6342d404d471abec751dbde396b747/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d6578706f72742d6e6f7465626f6f6b2e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-export-notebook.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-export-notebook/releases"><img src="https://camo.githubusercontent.com/228563530ce305c8c67655839cea391aefa48f9b89ce07ba82d77f40b1c07a21/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d6578706f72742d6e6f7465626f6f6b3f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-export-notebook?include_prereleases&amp;label=changelog" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-export-notebook/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-export-notebook/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-export-notebook/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width: 100%;"></a></p>
  <p dir="auto">Datasette plugin providing instructions for exporting data to a <a href="https://jupyter.org/" rel="nofollow">Jupyter</a> or <a href="https://observablehq.com/" rel="nofollow">Observable</a> notebook.</p>
  <h2 dir="auto"><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p dir="auto">Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="$ datasette install datasette-export-notebook"><pre><code>$ datasette install datasette-export-notebook
  </code></pre></div>
  <h2 dir="auto"><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p dir="auto">Once installed, the plugin will add a <code>.Notebook</code> export option to every table and query. Clicking on this link will show instructions for exporting the data to Jupyter or Observable.</p>
  <h2 dir="auto"><a id="user-content-demo" class="anchor" aria-hidden="true" href="#user-content-demo"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Demo</h2>
  <p dir="auto">You can see this plugin in action on the <a href="https://latest-with-plugins.datasette.io/" rel="nofollow">latest-with-plugins.datasette.io</a> Datasette instance - for example on <a href="https://latest-with-plugins.datasette.io/github/commits.Notebook" rel="nofollow">/github/commits.Notebook</a>.</p>
  <h2 dir="auto"><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p dir="auto">To set up this plugin locally, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="cd datasette-export-notebook
  python3 -mvenv venv
  source venv/bin/activate"><pre><code>cd datasette-export-notebook
  python3 -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p dir="auto">Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pipenv shell"><pre><code>pipenv shell
  </code></pre></div>
  <p dir="auto">Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pip install -e '.[test]'"><pre><code>pip install -e '.[test]'
  </code></pre></div>
  <p dir="auto">To run the tests:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pytest"><pre><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkzMzAzMDQ2Mjg=
name: datasette-query-history
full_name: bretwalker/datasette-query-history
html_url: https://github.com/bretwalker/datasette-query-history
created_at: 2021-01-17T03:13:34Z
updated_at: 2021-04-18T03:06:36Z
pushed_at: 2021-01-17T23:11:37Z
language: JavaScript
license: apache-2.0
topics: []
default_branch: main
permissions: {"admin": false, "push": false, "pull": false}
readme:
  # datasette-query-history

  [![PyPI](https://img.shields.io/pypi/v/datasette-query-history.svg)](https://pypi.org/project/datasette-query-history/)
  [![Changelog](https://img.shields.io/github/v/release/bretwalker/datasette-query-history?include_prereleases&label=changelog)](https://github.com/bretwalker/datasette-query-history/releases)
  [![Tests](https://github.com/bretwalker/datasette-query-history/workflows/Test/badge.svg)](https://github.com/bretwalker/datasette-query-history/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/bretwalker/datasette-query-history/blob/main/LICENSE)

  Datasette plugin that keeps a list of the queries you've run and lets you rerun them.

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-query-history

  ## Usage

  Click the `Query History` button on the SQL editor page to see previous queries.  
  Click the ⬆︎ button to replace the current query with a previous query.  
  Click the `Clear Query History` button to clear the list previous queries.

  <img src="https://raw.githubusercontent.com/bretwalker/datasette-query-history/main/docs/datasette-query-history-example1.png" width="350px" alt="Screenshot of plugin">

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-query-history
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-datasette-query-history" class="anchor" aria-hidden="true" href="#user-content-datasette-query-history"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-query-history</h1>
  <p><a href="https://pypi.org/project/datasette-query-history/" rel="nofollow"><img src="https://camo.githubusercontent.com/d74ad3a4fc71b2fea733dcc8f934843e15f176363d7016e72f0f2e2a159148df/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d71756572792d686973746f72792e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-query-history.svg" style="max-width:100%;"></a>
  <a href="https://github.com/bretwalker/datasette-query-history/releases"><img src="https://camo.githubusercontent.com/9b546caa8588b66ba6dea138d26a307e46e9e28bc6de43117a4bc7afec6dc7fb/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f6272657477616c6b65722f6461746173657474652d71756572792d686973746f72793f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/bretwalker/datasette-query-history?include_prereleases&amp;label=changelog" style="max-width:100%;"></a>
  <a href="https://github.com/bretwalker/datasette-query-history/actions?query=workflow%3ATest"><img src="https://github.com/bretwalker/datasette-query-history/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a>
  <a href="https://github.com/bretwalker/datasette-query-history/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p>
  <p>Datasette plugin that keeps a list of the queries you've run and lets you rerun them.</p>
  <h2><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p>Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ datasette install datasette-query-history
  "><pre><code>$ datasette install datasette-query-history
  </code></pre></div>
  <h2><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p>Click the <code>Query History</code> button on the SQL editor page to see previous queries.<br>
  Click the ⬆︎ button to replace the current query with a previous query.<br>
  Click the <code>Clear Query History</code> button to clear the list previous queries.</p>
  <p><a target="_blank" rel="noopener noreferrer" href="https://raw.githubusercontent.com/bretwalker/datasette-query-history/main/docs/datasette-query-history-example1.png"><img src="https://raw.githubusercontent.com/bretwalker/datasette-query-history/main/docs/datasette-query-history-example1.png" width="350px" alt="Screenshot of plugin" style="max-width:100%;"></a></p>
  <h2><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p>To set up this plugin locally, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="cd datasette-query-history
  python3 -mvenv venv
  source venv/bin/activate
  "><pre><code>cd datasette-query-history
  python3 -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p>Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pipenv shell
  "><pre><code>pipenv shell
  </code></pre></div>
  <p>Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pip install -e '.[test]'
  "><pre><code>pip install -e '.[test]'
  </code></pre></div>
  <p>To run the tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pytest
  "><pre><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkzMzExNTE3MDg=
name: datasette-leaflet-freedraw
full_name: simonw/datasette-leaflet-freedraw
html_url: https://github.com/simonw/datasette-leaflet-freedraw
description: Draw polygons on maps in Datasette
created_at: 2021-01-20T00:55:03Z
updated_at: 2021-12-17T22:07:50Z
pushed_at: 2022-02-03T20:24:37Z
language: Python
topics: ["datasette", "datasette-io", "datasette-plugin", "leafletjs"]
default_branch: main
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
visibility: public
readme:
  # datasette-leaflet-freedraw

  [![PyPI](https://img.shields.io/pypi/v/datasette-leaflet-freedraw.svg)](https://pypi.org/project/datasette-leaflet-freedraw/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-leaflet-freedraw?include_prereleases&label=changelog)](https://github.com/simonw/datasette-leaflet-freedraw/releases)
  [![Tests](https://github.com/simonw/datasette-leaflet-freedraw/workflows/Test/badge.svg)](https://github.com/simonw/datasette-leaflet-freedraw/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-leaflet-freedraw/blob/main/LICENSE)

  Draw polygons on maps in Datasette

  Project background: [Drawing shapes on a map to query a SpatiaLite database](https://simonwillison.net/2021/Jan/24/drawing-shapes-spatialite/).

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-leaflet-freedraw

  ## Usage

  If a table has a SpatiaLite `geometry` column, the plugin will add a map interface to the table page allowing users to draw a shape on the map to find rows with a geometry that intersects that shape.

  The plugin can also work with arbitrary SQL queries. There it looks for input fields with a name of `freedraw` or that ends in `_freedraw` and replaces them with a map interface.

  The map interface uses the [FreeDraw](https://freedraw.herokuapp.com/) Leaflet plugin.

  ## Demo

  You can try out this plugin to run searches against the GreenInfo Network California Protected Areas Database. Here's [an example query](https://calands.datasettes.com/calands?sql=select%0D%0A++AsGeoJSON%28geometry%29%2C+*%0D%0Afrom%0D%0A++CPAD_2020a_SuperUnits%0D%0Awhere%0D%0A++PARK_NAME+like+%27%25mini%25%27+and%0D%0A++Intersects%28GeomFromGeoJSON%28%3Afreedraw%29%2C+geometry%29+%3D+1%0D%0A++and+CPAD_2020a_SuperUnits.rowid+in+%28%0D%0A++++select%0D%0A++++++rowid%0D%0A++++from%0D%0A++++++SpatialIndex%0D%0A++++where%0D%0A++++++f_table_name+%3D+%27CPAD_2020a_SuperUnits%27%0D%0A++++++and+search_frame+%3D+GeomFromGeoJSON%28%3Afreedraw%29%0D%0A++%29&freedraw=%7B%22type%22%3A%22MultiPolygon%22%2C%22coordinates%22%3A%5B%5B%5B%5B-122.42202758789064%2C37.82280243352759%5D%2C%5B-122.39868164062501%2C37.823887203271454%5D%2C%5B-122.38220214843751%2C37.81846319511331%5D%2C%5B-122.35061645507814%2C37.77071473849611%5D%2C%5B-122.34924316406251%2C37.74465712069939%5D%2C%5B-122.37258911132814%2C37.703380457832374%5D%2C%5B-122.39044189453125%2C37.690340943717715%5D%2C%5B-122.41241455078126%2C37.680559803205135%5D%2C%5B-122.44262695312501%2C37.67295135774715%5D%2C%5B-122.47283935546876%2C37.67295135774715%5D%2C%5B-122.52502441406251%2C37.68382032669382%5D%2C%5B-122.53463745117189%2C37.6892542140253%5D%2C%5B-122.54699707031251%2C37.690340943717715%5D%2C%5B-122.55798339843751%2C37.72945260537781%5D%2C%5B-122.54287719726564%2C37.77831314799672%5D%2C%5B-122.49893188476564%2C37.81303878836991%5D%2C%5B-122.46185302734376%2C37.82822612280363%5D%2C%5B-122.42889404296876%2C37.82822612280363%5D%2C%5B-122.42202758789064%2C37.82280243352759%5D%5D%5D%5D%7D) showing mini parks in San Francisco:

  ```sql
  select
    AsGeoJSON(geometry), *
  from
    CPAD_2020a_SuperUnits
  where
    PARK_NAME like '%mini%' and
    Intersects(GeomFromGeoJSON(:freedraw), geometry) = 1
    and CPAD_2020a_SuperUnits.rowid in (
      select
        rowid
      from
        SpatialIndex
      where
        f_table_name = 'CPAD_2020a_SuperUnits'
        and search_frame = GeomFromGeoJSON(:freedraw)
    )
  ```

  ![Screenshot of the plugin in action](https://static.simonwillison.net/static/2021/datasette-leaflet-freedraw.png)

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-leaflet-freedraw
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-datasette-leaflet-freedraw" class="anchor" aria-hidden="true" href="#user-content-datasette-leaflet-freedraw"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-leaflet-freedraw</h1>
  <p dir="auto"><a href="https://pypi.org/project/datasette-leaflet-freedraw/" rel="nofollow"><img src="https://camo.githubusercontent.com/43771dd02f1327b1f2ae2bdd743e01865626d143ca9e98a37700a8747300465f/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d6c6561666c65742d66726565647261772e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-leaflet-freedraw.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-leaflet-freedraw/releases"><img src="https://camo.githubusercontent.com/8c5d6738c95e5fb3628c688c1eb79e970c822e9880493b52f0a62089e1b3e40c/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d6c6561666c65742d66726565647261773f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-leaflet-freedraw?include_prereleases&amp;label=changelog" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-leaflet-freedraw/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-leaflet-freedraw/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-leaflet-freedraw/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width: 100%;"></a></p>
  <p dir="auto">Draw polygons on maps in Datasette</p>
  <p dir="auto">Project background: <a href="https://simonwillison.net/2021/Jan/24/drawing-shapes-spatialite/" rel="nofollow">Drawing shapes on a map to query a SpatiaLite database</a>.</p>
  <h2 dir="auto"><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p dir="auto">Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="$ datasette install datasette-leaflet-freedraw"><pre><code>$ datasette install datasette-leaflet-freedraw
  </code></pre></div>
  <h2 dir="auto"><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p dir="auto">If a table has a SpatiaLite <code>geometry</code> column, the plugin will add a map interface to the table page allowing users to draw a shape on the map to find rows with a geometry that intersects that shape.</p>
  <p dir="auto">The plugin can also work with arbitrary SQL queries. There it looks for input fields with a name of <code>freedraw</code> or that ends in <code>_freedraw</code> and replaces them with a map interface.</p>
  <p dir="auto">The map interface uses the <a href="https://freedraw.herokuapp.com/" rel="nofollow">FreeDraw</a> Leaflet plugin.</p>
  <h2 dir="auto"><a id="user-content-demo" class="anchor" aria-hidden="true" href="#user-content-demo"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Demo</h2>
  <p dir="auto">You can try out this plugin to run searches against the GreenInfo Network California Protected Areas Database. Here's <a href="https://calands.datasettes.com/calands?sql=select%0D%0A++AsGeoJSON%28geometry%29%2C+*%0D%0Afrom%0D%0A++CPAD_2020a_SuperUnits%0D%0Awhere%0D%0A++PARK_NAME+like+%27%25mini%25%27+and%0D%0A++Intersects%28GeomFromGeoJSON%28%3Afreedraw%29%2C+geometry%29+%3D+1%0D%0A++and+CPAD_2020a_SuperUnits.rowid+in+%28%0D%0A++++select%0D%0A++++++rowid%0D%0A++++from%0D%0A++++++SpatialIndex%0D%0A++++where%0D%0A++++++f_table_name+%3D+%27CPAD_2020a_SuperUnits%27%0D%0A++++++and+search_frame+%3D+GeomFromGeoJSON%28%3Afreedraw%29%0D%0A++%29&amp;freedraw=%7B%22type%22%3A%22MultiPolygon%22%2C%22coordinates%22%3A%5B%5B%5B%5B-122.42202758789064%2C37.82280243352759%5D%2C%5B-122.39868164062501%2C37.823887203271454%5D%2C%5B-122.38220214843751%2C37.81846319511331%5D%2C%5B-122.35061645507814%2C37.77071473849611%5D%2C%5B-122.34924316406251%2C37.74465712069939%5D%2C%5B-122.37258911132814%2C37.703380457832374%5D%2C%5B-122.39044189453125%2C37.690340943717715%5D%2C%5B-122.41241455078126%2C37.680559803205135%5D%2C%5B-122.44262695312501%2C37.67295135774715%5D%2C%5B-122.47283935546876%2C37.67295135774715%5D%2C%5B-122.52502441406251%2C37.68382032669382%5D%2C%5B-122.53463745117189%2C37.6892542140253%5D%2C%5B-122.54699707031251%2C37.690340943717715%5D%2C%5B-122.55798339843751%2C37.72945260537781%5D%2C%5B-122.54287719726564%2C37.77831314799672%5D%2C%5B-122.49893188476564%2C37.81303878836991%5D%2C%5B-122.46185302734376%2C37.82822612280363%5D%2C%5B-122.42889404296876%2C37.82822612280363%5D%2C%5B-122.42202758789064%2C37.82280243352759%5D%5D%5D%5D%7D" rel="nofollow">an example query</a> showing mini parks in San Francisco:</p>
  <div class="highlight highlight-source-sql position-relative overflow-auto" data-snippet-clipboard-copy-content="select
    AsGeoJSON(geometry), *
  from
    CPAD_2020a_SuperUnits
  where
    PARK_NAME like '%mini%' and
    Intersects(GeomFromGeoJSON(:freedraw), geometry) = 1
    and CPAD_2020a_SuperUnits.rowid in (
      select
        rowid
      from
        SpatialIndex
      where
        f_table_name = 'CPAD_2020a_SuperUnits'
        and search_frame = GeomFromGeoJSON(:freedraw)
    )"><pre><span class="pl-k">select</span>
    AsGeoJSON(geometry), <span class="pl-k">*</span>
  <span class="pl-k">from</span>
    CPAD_2020a_SuperUnits
  <span class="pl-k">where</span>
    PARK_NAME <span class="pl-k">like</span> <span class="pl-s"><span class="pl-pds">'</span>%mini%<span class="pl-pds">'</span></span> <span class="pl-k">and</span>
    Intersects(GeomFromGeoJSON(:freedraw), geometry) <span class="pl-k">=</span> <span class="pl-c1">1</span>
    <span class="pl-k">and</span> <span class="pl-c1">CPAD_2020a_SuperUnits</span>.<span class="pl-c1">rowid</span> <span class="pl-k">in</span> (
      <span class="pl-k">select</span>
        rowid
      <span class="pl-k">from</span>
        SpatialIndex
      <span class="pl-k">where</span>
        f_table_name <span class="pl-k">=</span> <span class="pl-s"><span class="pl-pds">'</span>CPAD_2020a_SuperUnits<span class="pl-pds">'</span></span>
        <span class="pl-k">and</span> search_frame <span class="pl-k">=</span> GeomFromGeoJSON(:freedraw)
    )</pre></div>
  <p dir="auto"><a target="_blank" rel="noopener noreferrer" href="https://camo.githubusercontent.com/56a2ed7e65e28e726d4c3abfbe56bd7960524e6af6512fc2cee083bdd7fcd863/68747470733a2f2f7374617469632e73696d6f6e77696c6c69736f6e2e6e65742f7374617469632f323032312f6461746173657474652d6c6561666c65742d66726565647261772e706e67"><img src="https://camo.githubusercontent.com/56a2ed7e65e28e726d4c3abfbe56bd7960524e6af6512fc2cee083bdd7fcd863/68747470733a2f2f7374617469632e73696d6f6e77696c6c69736f6e2e6e65742f7374617469632f323032312f6461746173657474652d6c6561666c65742d66726565647261772e706e67" alt="Screenshot of the plugin in action" data-canonical-src="https://static.simonwillison.net/static/2021/datasette-leaflet-freedraw.png" style="max-width: 100%;"></a></p>
  <h2 dir="auto"><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p dir="auto">To set up this plugin locally, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="cd datasette-leaflet-freedraw
  python3 -mvenv venv
  source venv/bin/activate"><pre><code>cd datasette-leaflet-freedraw
  python3 -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p dir="auto">Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pipenv shell"><pre><code>pipenv shell
  </code></pre></div>
  <p dir="auto">Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pip install -e '.[test]'"><pre><code>pip install -e '.[test]'
  </code></pre></div>
  <p dir="auto">To run the tests:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pytest"><pre><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkzMzE3MjA4MjQ=
name: datasette-leaflet
full_name: simonw/datasette-leaflet
html_url: https://github.com/simonw/datasette-leaflet
description: Datasette plugin adding the Leaflet JavaScript library
created_at: 2021-01-21T18:41:19Z
updated_at: 2021-04-20T16:27:35Z
pushed_at: 2021-02-01T22:20:28Z
language: JavaScript
topics: ["datasette", "datasette-plugin", "datasette-io"]
default_branch: main
permissions: {"admin": false, "push": false, "pull": false}
readme:
  # datasette-leaflet

  [![PyPI](https://img.shields.io/pypi/v/datasette-leaflet.svg)](https://pypi.org/project/datasette-leaflet/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-leaflet?include_prereleases&label=changelog)](https://github.com/simonw/datasette-leaflet/releases)
  [![Tests](https://github.com/simonw/datasette-leaflet/workflows/Test/badge.svg)](https://github.com/simonw/datasette-leaflet/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-leaflet/blob/main/LICENSE)

  Datasette plugin adding the [Leaflet](https://leafletjs.com/) JavaScript library.

  A growing number of Datasette plugins depend on the Leaflet JavaScript mapping library. They each have their own way of loading Leaflet, which could result in loading it multiple times (with multiple versions) if more than one plugin is installed.

  This library is intended to solve this problem, by providing a single plugin they can all depend on that loads Leaflet in a reusable way.

  Plugins that use this:

  - [datasette-leaflet-freedraw](https://datasette.io/plugins/datasette-leaflet-freedraw)
  - [datasette-leaflet-geojson](https://datasette.io/plugins/datasette-leaflet-geojson)
  - [datasette-cluster-map](https://datasette.io/plugins/datasette-cluster-map)

  ## Installation

  You can install this plugin like so:

      datasette install datasette-leaflet

  Usually this plugin will be a dependency of other plugins, so it should be installed automatically when you install them.

  ## Usage

  The plugin makes `leaflet.js` and `leaflet.css` available as static files. It provides two custom template variables with the URLs of those two files.

  - `{{ datasette_leaflet_url }}` is the URL to the JavaScript
  - `{{ datasette_leaflet_css_url }}` is the URL to the CSS

  These URLs are also made available as global JavaScript constants:

  - `datasette.leaflet.JAVASCRIPT_URL`
  - `datasette.leaflet.CSS_URL`

  The JavaScript is packaed as a [JavaScript module](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules). You can dynamically import the JavaScript from a custom template like this:

  ```html+jinja
  <script type="module">
  import('{{ datasette_leaflet_url }}')
    .then((leaflet) => {
      /* Use leaflet here */
    });
  </script>
  ```

  You can load the CSS like this:

  ```html+jinja
  <link rel="stylesheet" href="{{ datasette_leaflet_css_url }}">
  ```

  Or dynamically like this:

  ```html+jinja
  <script>
  let link = document.createElement('link');
  link.rel = 'stylesheet';
  link.href = '{{ datasette_leaflet_css_url }}';
  document.head.appendChild(link);
  </script>
  ```

  Here's a full example that loads the JavaScript and CSS and renders a map:

  ```html+jinja
  <script type="module">
  window.DATASETTE_CLUSTER_MAP_TILE_LAYER = "https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png";
  window.DATASETTE_CLUSTER_MAP_TILE_LAYER_OPTIONS = {"maxZoom": 19, "detectRetina": true, "attribution": "&copy; <a href=\"https://www.openstreetmap.org/copyright\">OpenStreetMap</a> contributors"};
  let link = document.createElement('link');
  link.rel = 'stylesheet';
  link.href = '{{ datasette_leaflet_css_url }}';
  document.head.appendChild(link);
  import('{{ datasette_leaflet_url }}')
    .then((leaflet) => {
      let div = document.createElement('div');
      div.style.height = '400px';
      document.querySelector('.content').appendChild(div);
      let tiles = leaflet.tileLayer(
          window.DATASETTE_CLUSTER_MAP_TILE_LAYER,
          window.DATASETTE_CLUSTER_MAP_TILE_LAYER_OPTIONS
      );
      let map = leaflet.map(div, {
          center: leaflet.latLng(0, 0),
          zoom: 2,
          layers: [tiles]
      });
    });
  </script>
  ```

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-datasette-leaflet" class="anchor" aria-hidden="true" href="#user-content-datasette-leaflet"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-leaflet</h1>
  <p><a href="https://pypi.org/project/datasette-leaflet/" rel="nofollow"><img src="https://camo.githubusercontent.com/50da3783cf231c9646e762bf20cfd464779db8b979de8627d491b9b4589e37c6/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d6c6561666c65742e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-leaflet.svg" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-leaflet/releases"><img src="https://camo.githubusercontent.com/8b54c9ff200877b23b90a28f91e1d361b5881fd92bf524df8b412176ce98a505/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d6c6561666c65743f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-leaflet?include_prereleases&amp;label=changelog" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-leaflet/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-leaflet/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-leaflet/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p>
  <p>Datasette plugin adding the <a href="https://leafletjs.com/" rel="nofollow">Leaflet</a> JavaScript library.</p>
  <p>A growing number of Datasette plugins depend on the Leaflet JavaScript mapping library. They each have their own way of loading Leaflet, which could result in loading it multiple times (with multiple versions) if more than one plugin is installed.</p>
  <p>This library is intended to solve this problem, by providing a single plugin they can all depend on that loads Leaflet in a reusable way.</p>
  <p>Plugins that use this:</p>
  <ul>
  <li><a href="https://datasette.io/plugins/datasette-leaflet-freedraw" rel="nofollow">datasette-leaflet-freedraw</a></li>
  <li><a href="https://datasette.io/plugins/datasette-leaflet-geojson" rel="nofollow">datasette-leaflet-geojson</a></li>
  <li><a href="https://datasette.io/plugins/datasette-cluster-map" rel="nofollow">datasette-cluster-map</a></li>
  </ul>
  <h2><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p>You can install this plugin like so:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="datasette install datasette-leaflet
  "><pre><code>datasette install datasette-leaflet
  </code></pre></div>
  <p>Usually this plugin will be a dependency of other plugins, so it should be installed automatically when you install them.</p>
  <h2><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p>The plugin makes <code>leaflet.js</code> and <code>leaflet.css</code> available as static files. It provides two custom template variables with the URLs of those two files.</p>
  <ul>
  <li><code>{{ datasette_leaflet_url }}</code> is the URL to the JavaScript</li>
  <li><code>{{ datasette_leaflet_css_url }}</code> is the URL to the CSS</li>
  </ul>
  <p>These URLs are also made available as global JavaScript constants:</p>
  <ul>
  <li><code>datasette.leaflet.JAVASCRIPT_URL</code></li>
  <li><code>datasette.leaflet.CSS_URL</code></li>
  </ul>
  <p>The JavaScript is packaed as a <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules" rel="nofollow">JavaScript module</a>. You can dynamically import the JavaScript from a custom template like this:</p>
  <div class="highlight highlight-text-html-django position-relative" data-snippet-clipboard-copy-content="&lt;script type=&quot;module&quot;&gt;
  import('{{ datasette_leaflet_url }}')
    .then((leaflet) =&gt; {
      /* Use leaflet here */
    });
  &lt;/script&gt;
  "><pre>&lt;<span class="pl-ent">script</span> <span class="pl-e">type</span>=<span class="pl-s"><span class="pl-pds">"</span>module<span class="pl-pds">"</span></span>&gt;<span class="pl-s1"></span>
  <span class="pl-s1"><span class="pl-k">import</span>(<span class="pl-s"><span class="pl-pds">'</span>{{ datasette_leaflet_url }}<span class="pl-pds">'</span></span>)</span>
  <span class="pl-s1">  .<span class="pl-c1">then</span>((<span class="pl-smi">leaflet</span>) <span class="pl-k">=&gt;</span> {</span>
  <span class="pl-s1">    <span class="pl-c"><span class="pl-c">/*</span> Use leaflet here <span class="pl-c">*/</span></span></span>
  <span class="pl-s1">  });</span>
  <span class="pl-s1"></span>&lt;/<span class="pl-ent">script</span>&gt;</pre></div>
  <p>You can load the CSS like this:</p>
  <div class="highlight highlight-text-html-django position-relative" data-snippet-clipboard-copy-content="&lt;link rel=&quot;stylesheet&quot; href=&quot;{{ datasette_leaflet_css_url }}&quot;&gt;
  "><pre>&lt;<span class="pl-ent">link</span> <span class="pl-e">rel</span>=<span class="pl-s"><span class="pl-pds">"</span>stylesheet<span class="pl-pds">"</span></span> <span class="pl-e">href</span>=<span class="pl-s"><span class="pl-pds">"</span>{{ datasette_leaflet_css_url }}<span class="pl-pds">"</span></span>&gt;</pre></div>
  <p>Or dynamically like this:</p>
  <div class="highlight highlight-text-html-django position-relative" data-snippet-clipboard-copy-content="&lt;script&gt;
  let link = document.createElement('link');
  link.rel = 'stylesheet';
  link.href = '{{ datasette_leaflet_css_url }}';
  document.head.appendChild(link);
  &lt;/script&gt;
  "><pre>&lt;<span class="pl-ent">script</span>&gt;<span class="pl-s1"></span>
  <span class="pl-s1"><span class="pl-k">let</span> link <span class="pl-k">=</span> <span class="pl-c1">document</span>.<span class="pl-c1">createElement</span>(<span class="pl-s"><span class="pl-pds">'</span>link<span class="pl-pds">'</span></span>);</span>
  <span class="pl-s1"><span class="pl-smi">link</span>.<span class="pl-c1">rel</span> <span class="pl-k">=</span> <span class="pl-s"><span class="pl-pds">'</span>stylesheet<span class="pl-pds">'</span></span>;</span>
  <span class="pl-s1"><span class="pl-smi">link</span>.<span class="pl-c1">href</span> <span class="pl-k">=</span> <span class="pl-s"><span class="pl-pds">'</span>{{ datasette_leaflet_css_url }}<span class="pl-pds">'</span></span>;</span>
  <span class="pl-s1"><span class="pl-c1">document</span>.<span class="pl-smi">head</span>.<span class="pl-c1">appendChild</span>(link);</span>
  <span class="pl-s1"></span>&lt;/<span class="pl-ent">script</span>&gt;</pre></div>
  <p>Here's a full example that loads the JavaScript and CSS and renders a map:</p>
  <div class="highlight highlight-text-html-django position-relative" data-snippet-clipboard-copy-content="&lt;script type=&quot;module&quot;&gt;
  window.DATASETTE_CLUSTER_MAP_TILE_LAYER = &quot;https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png&quot;;
  window.DATASETTE_CLUSTER_MAP_TILE_LAYER_OPTIONS = {&quot;maxZoom&quot;: 19, &quot;detectRetina&quot;: true, &quot;attribution&quot;: &quot;&amp;copy; &lt;a href=\&quot;https://www.openstreetmap.org/copyright\&quot;&gt;OpenStreetMap&lt;/a&gt; contributors&quot;};
  let link = document.createElement('link');
  link.rel = 'stylesheet';
  link.href = '{{ datasette_leaflet_css_url }}';
  document.head.appendChild(link);
  import('{{ datasette_leaflet_url }}')
    .then((leaflet) =&gt; {
      let div = document.createElement('div');
      div.style.height = '400px';
      document.querySelector('.content').appendChild(div);
      let tiles = leaflet.tileLayer(
          window.DATASETTE_CLUSTER_MAP_TILE_LAYER,
          window.DATASETTE_CLUSTER_MAP_TILE_LAYER_OPTIONS
      );
      let map = leaflet.map(div, {
          center: leaflet.latLng(0, 0),
          zoom: 2,
          layers: [tiles]
      });
    });
  &lt;/script&gt;
  "><pre>&lt;<span class="pl-ent">script</span> <span class="pl-e">type</span>=<span class="pl-s"><span class="pl-pds">"</span>module<span class="pl-pds">"</span></span>&gt;<span class="pl-s1"></span>
  <span class="pl-s1"><span class="pl-c1">window</span>.<span class="pl-c1">DATASETTE_CLUSTER_MAP_TILE_LAYER</span> <span class="pl-k">=</span> <span class="pl-s"><span class="pl-pds">"</span>https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png<span class="pl-pds">"</span></span>;</span>
  <span class="pl-s1"><span class="pl-c1">window</span>.<span class="pl-c1">DATASETTE_CLUSTER_MAP_TILE_LAYER_OPTIONS</span> <span class="pl-k">=</span> {<span class="pl-s"><span class="pl-pds">"</span>maxZoom<span class="pl-pds">"</span></span><span class="pl-k">:</span> <span class="pl-c1">19</span>, <span class="pl-s"><span class="pl-pds">"</span>detectRetina<span class="pl-pds">"</span></span><span class="pl-k">:</span> <span class="pl-c1">true</span>, <span class="pl-s"><span class="pl-pds">"</span>attribution<span class="pl-pds">"</span></span><span class="pl-k">:</span> <span class="pl-s"><span class="pl-pds">"</span>&amp;copy; &lt;a href=<span class="pl-cce">\"</span>https://www.openstreetmap.org/copyright<span class="pl-cce">\"</span>&gt;OpenStreetMap&lt;/a&gt; contributors<span class="pl-pds">"</span></span>};</span>
  <span class="pl-s1"><span class="pl-k">let</span> link <span class="pl-k">=</span> <span class="pl-c1">document</span>.<span class="pl-c1">createElement</span>(<span class="pl-s"><span class="pl-pds">'</span>link<span class="pl-pds">'</span></span>);</span>
  <span class="pl-s1"><span class="pl-smi">link</span>.<span class="pl-c1">rel</span> <span class="pl-k">=</span> <span class="pl-s"><span class="pl-pds">'</span>stylesheet<span class="pl-pds">'</span></span>;</span>
  <span class="pl-s1"><span class="pl-smi">link</span>.<span class="pl-c1">href</span> <span class="pl-k">=</span> <span class="pl-s"><span class="pl-pds">'</span>{{ datasette_leaflet_css_url }}<span class="pl-pds">'</span></span>;</span>
  <span class="pl-s1"><span class="pl-c1">document</span>.<span class="pl-smi">head</span>.<span class="pl-c1">appendChild</span>(link);</span>
  <span class="pl-s1"><span class="pl-k">import</span>(<span class="pl-s"><span class="pl-pds">'</span>{{ datasette_leaflet_url }}<span class="pl-pds">'</span></span>)</span>
  <span class="pl-s1">  .<span class="pl-c1">then</span>((<span class="pl-smi">leaflet</span>) <span class="pl-k">=&gt;</span> {</span>
  <span class="pl-s1">    <span class="pl-k">let</span> div <span class="pl-k">=</span> <span class="pl-c1">document</span>.<span class="pl-c1">createElement</span>(<span class="pl-s"><span class="pl-pds">'</span>div<span class="pl-pds">'</span></span>);</span>
  <span class="pl-s1">    <span class="pl-smi">div</span>.<span class="pl-c1">style</span>.<span class="pl-c1">height</span> <span class="pl-k">=</span> <span class="pl-s"><span class="pl-pds">'</span>400px<span class="pl-pds">'</span></span>;</span>
  <span class="pl-s1">    <span class="pl-c1">document</span>.<span class="pl-c1">querySelector</span>(<span class="pl-s"><span class="pl-pds">'</span>.content<span class="pl-pds">'</span></span>).<span class="pl-c1">appendChild</span>(div);</span>
  <span class="pl-s1">    <span class="pl-k">let</span> tiles <span class="pl-k">=</span> <span class="pl-smi">leaflet</span>.<span class="pl-en">tileLayer</span>(</span>
  <span class="pl-s1">        <span class="pl-c1">window</span>.<span class="pl-c1">DATASETTE_CLUSTER_MAP_TILE_LAYER</span>,</span>
  <span class="pl-s1">        <span class="pl-c1">window</span>.<span class="pl-c1">DATASETTE_CLUSTER_MAP_TILE_LAYER_OPTIONS</span></span>
  <span class="pl-s1">    );</span>
  <span class="pl-s1">    <span class="pl-k">let</span> map <span class="pl-k">=</span> <span class="pl-smi">leaflet</span>.<span class="pl-en">map</span>(div, {</span>
  <span class="pl-s1">        center<span class="pl-k">:</span> <span class="pl-smi">leaflet</span>.<span class="pl-en">latLng</span>(<span class="pl-c1">0</span>, <span class="pl-c1">0</span>),</span>
  <span class="pl-s1">        zoom<span class="pl-k">:</span> <span class="pl-c1">2</span>,</span>
  <span class="pl-s1">        layers<span class="pl-k">:</span> [tiles]</span>
  <span class="pl-s1">    });</span>
  <span class="pl-s1">  });</span>
  <span class="pl-s1"></span>&lt;/<span class="pl-ent">script</span>&gt;</pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkzMzUxMzcxMDg=
name: datasette-basemap
full_name: simonw/datasette-basemap
html_url: https://github.com/simonw/datasette-basemap
description: A basemap for Datasette and datasette-leaflet
created_at: 2021-02-02T01:49:09Z
updated_at: 2021-02-03T21:56:20Z
pushed_at: 2021-02-03T21:56:18Z
homepage: https://datasette.io/plugins/datasette-basemap
language: Python
topics: ["openstreetmap", "mbtiles", "datasette", "datasette-plugin", "datasette-io"]
default_branch: main
permissions: {"admin": false, "push": false, "pull": false}
readme:
  # datasette-basemap

  [![PyPI](https://img.shields.io/pypi/v/datasette-basemap.svg)](https://pypi.org/project/datasette-basemap/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-basemap?include_prereleases&label=changelog)](https://github.com/simonw/datasette-basemap/releases)
  [![Tests](https://github.com/simonw/datasette-basemap/workflows/Test/badge.svg)](https://github.com/simonw/datasette-basemap/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-basemap/blob/main/LICENSE)

  A basemap for Datasette and datasette-leaflet

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-basemap

  ## Usage

  This plugin will make a `basemap` database available containing OpenStreetMap tiles for zoom levels 0-6 in the [mbtiles](https://github.com/mapbox/mbtiles-spec) format. It is designed for use with the [datasette-tiles](https://datasette.io/plugins/datasette-tiles) tile server plugin.

  ## Demo

  You can preview this map at https://datasette-tiles-demo.datasette.io/-/tiles/basemap and browse the database directly at https://datasette-tiles-demo.datasette.io/basemap

  ## License

  The data bundled with this package is © OpenStreetMap contributors, licensed under the [Open Data Commons Open Database License](https://opendatacommons.org/licenses/odbl/). See [this page](https://www.openstreetmap.org/copyright) for more details.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-basemap
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-datasette-basemap" class="anchor" aria-hidden="true" href="#user-content-datasette-basemap"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-basemap</h1>
  <p><a href="https://pypi.org/project/datasette-basemap/" rel="nofollow"><img src="https://camo.githubusercontent.com/9e2030246117aed0c545320f2597b7cfefec254930d354c648b4a5ef1f695e4d/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d626173656d61702e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-basemap.svg" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-basemap/releases"><img src="https://camo.githubusercontent.com/7280ebff6265b28b5c3d37fe294e2bef782ad2cc64e0ca8281b119555e6ac485/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d626173656d61703f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-basemap?include_prereleases&amp;label=changelog" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-basemap/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-basemap/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-basemap/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p>
  <p>A basemap for Datasette and datasette-leaflet</p>
  <h2><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p>Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ datasette install datasette-basemap
  "><pre><code>$ datasette install datasette-basemap
  </code></pre></div>
  <h2><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p>This plugin will make a <code>basemap</code> database available containing OpenStreetMap tiles for zoom levels 0-6 in the <a href="https://github.com/mapbox/mbtiles-spec">mbtiles</a> format. It is designed for use with the <a href="https://datasette.io/plugins/datasette-tiles" rel="nofollow">datasette-tiles</a> tile server plugin.</p>
  <h2><a id="user-content-demo" class="anchor" aria-hidden="true" href="#user-content-demo"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Demo</h2>
  <p>You can preview this map at <a href="https://datasette-tiles-demo.datasette.io/-/tiles/basemap" rel="nofollow">https://datasette-tiles-demo.datasette.io/-/tiles/basemap</a> and browse the database directly at <a href="https://datasette-tiles-demo.datasette.io/basemap" rel="nofollow">https://datasette-tiles-demo.datasette.io/basemap</a></p>
  <h2><a id="user-content-license" class="anchor" aria-hidden="true" href="#user-content-license"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>License</h2>
  <p>The data bundled with this package is © OpenStreetMap contributors, licensed under the <a href="https://opendatacommons.org/licenses/odbl/" rel="nofollow">Open Data Commons Open Database License</a>. See <a href="https://www.openstreetmap.org/copyright" rel="nofollow">this page</a> for more details.</p>
  <h2><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p>To set up this plugin locally, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="cd datasette-basemap
  python3 -mvenv venv
  source venv/bin/activate
  "><pre><code>cd datasette-basemap
  python3 -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p>Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pipenv shell
  "><pre><code>pipenv shell
  </code></pre></div>
  <p>Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pip install -e '.[test]'
  "><pre><code>pip install -e '.[test]'
  </code></pre></div>
  <p>To run the tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pytest
  "><pre><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkzMzUxNzU2Mzc=
name: datasette-tiles
full_name: simonw/datasette-tiles
html_url: https://github.com/simonw/datasette-tiles
description: Mapping tile server for Datasette, serving tiles from MBTiles packages
created_at: 2021-02-02T05:11:12Z
updated_at: 2022-03-22T01:52:30Z
pushed_at: 2022-03-22T01:52:27Z
homepage: https://datasette.io/plugins/datasette-tiles
language: Python
topics: ["datasette", "datasette-io", "datasette-plugin", "mbtiles"]
default_branch: main
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
visibility: public
readme:
  # datasette-tiles

  [![PyPI](https://img.shields.io/pypi/v/datasette-tiles.svg)](https://pypi.org/project/datasette-tiles/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-tiles?include_prereleases&label=changelog)](https://github.com/simonw/datasette-tiles/releases)
  [![Tests](https://github.com/simonw/datasette-tiles/workflows/Test/badge.svg)](https://github.com/simonw/datasette-tiles/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-tiles/blob/main/LICENSE)

  Datasette plugin for serving MBTiles map tiles

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-tiles

  ## Demo

  You can try this plugin out at https://datasette-tiles-demo.datasette.io/-/tiles

  ## Usage

  This plugin scans all database files connected to Datasette to see if any of them are valid MBTiles databases.

  It can then serve tiles from those databases at the following URL:

      /-/tiles/db-name/zoom/x/y.png

  An example map for each database demonstrating the configured minimum and maximum zoom for that database can be found at `/-/tiles/db-name` - this can also be accessed via the table and database action menus for that database.

  Visit `/-/tiles` for an index page of attached valid databases.

  You can install the [datasette-basemap](https://datasette.io/plugins/datasette-basemap) plugin to get a `basemap` default set of tiles, handling zoom levels 0 to 6 using OpenStreetMap.

  ### Tile coordinate systems

  There are two tile coordinate systems in common use for online maps. The first is used by OpenStreetMap and Google Maps, the second is from a specification called [Tile Map Service](https://en.wikipedia.org/wiki/Tile_Map_Service), or TMS.

  Both systems use three components: `z/x/y` - where `z` is the zoom level, `x` is the column and `y` is the row.

  The difference is in the way the `y` value is counted. OpenStreetMap has y=0 at the top. TMS has y=0 at the bottom.

  An illustrative example: at zoom level 2 the map is divided into 16 total tiles. The OpenStreetMap scheme numbers them like so:

      0/0  1/0  2/0  3/0
      0/1  1/1  2/1  3/1
      0/2  1/2  2/2  3/2
      0/3  1/3  2/3  3/3

  The TMS scheme looks like this:

      0/3  1/3  2/3  3/3
      0/2  1/2  2/2  3/2
      0/1  1/1  2/1  3/1
      0/0  1/0  2/0  3/0

  `datasette-tiles` can serve tiles using either of these standards. For the OpenStreetMap / Google Maps 0-at-the-top system, use the following URL:

      /-/tiles/database-name/{z}/{x}/{y}.png

  For the TMS 0-at-the-bottom system, use this:

      /-/tiles-tms/database-name/{z}/{x}/{y}.png

  ### Configuring a Leaflet tile layer

  The following JavaScript will configure a [Leaflet TileLayer](https://leafletjs.com/reference-1.7.1.html#tilelayer) for use with this plugin:

  ```javascript
  var tiles = leaflet.tileLayer("/-/tiles/basemap/{z}/{x}/{y}.png", {
    minZoom: 0,
    maxZoom: 6,
    attribution: "\u00a9 OpenStreetMap contributors"
  });
  ```

  ### Tile stacks

  `datasette-tiles` can be configured to serve tiles from multiple attached MBTiles files, searching each database in order for a tile and falling back to the next in line if that tile is not found.

  For a demo of this in action, visit https://datasette-tiles-demo.datasette.io/-/tiles-stack and zoom in on Japan. It should start showing [Stamen's Toner map](maps.stamen.com) of Japan once you get to zoom level 6 and 7.

  The `/-/tiles-stack/{z}/{x}/{y}.png` endpoint provides this feature.

  If you start Datasette like this:

      datasette world.mbtiles country.mbtiles city1.mbtiles city2.mbtiles

  Any requests for a tile from the `/-/tiles-stack` path will first check the `city2` database, than `city1`, then `country`, then `world`.

  If you have the [datasette-basemap](https://datasette.io/plugins/datasette-basemap) plugin installed it will be given special treatment: the `basemap` database will always be the last database checked for a tile.

  Rather than rely on the order in which databases were attached, you can instead configure an explicit order using the `tiles-stack-order` plugin setting. Add the following to your `metadata.json` file:

  ```json
  {
      "plugins": {
          "datasette-tiles": {
              "tiles-stack-order": ["world", "country"]
          }
      }
  }
  ```

  You can then run Datasette like this:

      datasette -m metadata.json country.mbtiles world.mbtiles

  This endpoint serves tiles using the OpenStreetMap / Google Maps coordinate system. To load tiles using the TMS coordinate system use this endpoint instead:

      /-/tiles-stack-tms/{z}/{x}/{y}.png

  ### Retina tiles

  Retina (double resolution) tiles are supported by `datasette-tiles` if the MBTiles database file contains 512x512 tile images as opposed to the default of 256x256. JavaScript libraries such as Leaflet will serve these tiles with a fixed 256x256 size, which will cause them to be displayed correctly by capable operating systems.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-tiles
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-datasette-tiles" class="anchor" aria-hidden="true" href="#user-content-datasette-tiles"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-tiles</h1>
  <p dir="auto"><a href="https://pypi.org/project/datasette-tiles/" rel="nofollow"><img src="https://camo.githubusercontent.com/72c38bbf12ff995549cf94a6c7ca06d205a19cf3543af89763d47186340b37a0/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d74696c65732e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-tiles.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-tiles/releases"><img src="https://camo.githubusercontent.com/6aa9bdcd93e33cd40ad13d0d00224d316b3743f498bd502f2f751ab7315618e2/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d74696c65733f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-tiles?include_prereleases&amp;label=changelog" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-tiles/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-tiles/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-tiles/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width: 100%;"></a></p>
  <p dir="auto">Datasette plugin for serving MBTiles map tiles</p>
  <h2 dir="auto"><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p dir="auto">Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="$ datasette install datasette-tiles"><pre><code>$ datasette install datasette-tiles
  </code></pre></div>
  <h2 dir="auto"><a id="user-content-demo" class="anchor" aria-hidden="true" href="#user-content-demo"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Demo</h2>
  <p dir="auto">You can try this plugin out at <a href="https://datasette-tiles-demo.datasette.io/-/tiles" rel="nofollow">https://datasette-tiles-demo.datasette.io/-/tiles</a></p>
  <h2 dir="auto"><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p dir="auto">This plugin scans all database files connected to Datasette to see if any of them are valid MBTiles databases.</p>
  <p dir="auto">It can then serve tiles from those databases at the following URL:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="/-/tiles/db-name/zoom/x/y.png"><pre><code>/-/tiles/db-name/zoom/x/y.png
  </code></pre></div>
  <p dir="auto">An example map for each database demonstrating the configured minimum and maximum zoom for that database can be found at <code>/-/tiles/db-name</code> - this can also be accessed via the table and database action menus for that database.</p>
  <p dir="auto">Visit <code>/-/tiles</code> for an index page of attached valid databases.</p>
  <p dir="auto">You can install the <a href="https://datasette.io/plugins/datasette-basemap" rel="nofollow">datasette-basemap</a> plugin to get a <code>basemap</code> default set of tiles, handling zoom levels 0 to 6 using OpenStreetMap.</p>
  <h3 dir="auto"><a id="user-content-tile-coordinate-systems" class="anchor" aria-hidden="true" href="#user-content-tile-coordinate-systems"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Tile coordinate systems</h3>
  <p dir="auto">There are two tile coordinate systems in common use for online maps. The first is used by OpenStreetMap and Google Maps, the second is from a specification called <a href="https://en.wikipedia.org/wiki/Tile_Map_Service" rel="nofollow">Tile Map Service</a>, or TMS.</p>
  <p dir="auto">Both systems use three components: <code>z/x/y</code> - where <code>z</code> is the zoom level, <code>x</code> is the column and <code>y</code> is the row.</p>
  <p dir="auto">The difference is in the way the <code>y</code> value is counted. OpenStreetMap has y=0 at the top. TMS has y=0 at the bottom.</p>
  <p dir="auto">An illustrative example: at zoom level 2 the map is divided into 16 total tiles. The OpenStreetMap scheme numbers them like so:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="0/0  1/0  2/0  3/0
  0/1  1/1  2/1  3/1
  0/2  1/2  2/2  3/2
  0/3  1/3  2/3  3/3"><pre><code>0/0  1/0  2/0  3/0
  0/1  1/1  2/1  3/1
  0/2  1/2  2/2  3/2
  0/3  1/3  2/3  3/3
  </code></pre></div>
  <p dir="auto">The TMS scheme looks like this:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="0/3  1/3  2/3  3/3
  0/2  1/2  2/2  3/2
  0/1  1/1  2/1  3/1
  0/0  1/0  2/0  3/0"><pre><code>0/3  1/3  2/3  3/3
  0/2  1/2  2/2  3/2
  0/1  1/1  2/1  3/1
  0/0  1/0  2/0  3/0
  </code></pre></div>
  <p dir="auto"><code>datasette-tiles</code> can serve tiles using either of these standards. For the OpenStreetMap / Google Maps 0-at-the-top system, use the following URL:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="/-/tiles/database-name/{z}/{x}/{y}.png"><pre><code>/-/tiles/database-name/{z}/{x}/{y}.png
  </code></pre></div>
  <p dir="auto">For the TMS 0-at-the-bottom system, use this:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="/-/tiles-tms/database-name/{z}/{x}/{y}.png"><pre><code>/-/tiles-tms/database-name/{z}/{x}/{y}.png
  </code></pre></div>
  <h3 dir="auto"><a id="user-content-configuring-a-leaflet-tile-layer" class="anchor" aria-hidden="true" href="#user-content-configuring-a-leaflet-tile-layer"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Configuring a Leaflet tile layer</h3>
  <p dir="auto">The following JavaScript will configure a <a href="https://leafletjs.com/reference-1.7.1.html#tilelayer" rel="nofollow">Leaflet TileLayer</a> for use with this plugin:</p>
  <div class="highlight highlight-source-js position-relative overflow-auto" data-snippet-clipboard-copy-content="var tiles = leaflet.tileLayer(&quot;/-/tiles/basemap/{z}/{x}/{y}.png&quot;, {
    minZoom: 0,
    maxZoom: 6,
    attribution: &quot;\u00a9 OpenStreetMap contributors&quot;
  });"><pre><span class="pl-k">var</span> <span class="pl-s1">tiles</span> <span class="pl-c1">=</span> <span class="pl-s1">leaflet</span><span class="pl-kos">.</span><span class="pl-en">tileLayer</span><span class="pl-kos">(</span><span class="pl-s">"/-/tiles/basemap/{z}/{x}/{y}.png"</span><span class="pl-kos">,</span> <span class="pl-kos">{</span>
    <span class="pl-c1">minZoom</span>: <span class="pl-c1">0</span><span class="pl-kos">,</span>
    <span class="pl-c1">maxZoom</span>: <span class="pl-c1">6</span><span class="pl-kos">,</span>
    <span class="pl-c1">attribution</span>: <span class="pl-s">"\u00a9 OpenStreetMap contributors"</span>
  <span class="pl-kos">}</span><span class="pl-kos">)</span><span class="pl-kos">;</span></pre></div>
  <h3 dir="auto"><a id="user-content-tile-stacks" class="anchor" aria-hidden="true" href="#user-content-tile-stacks"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Tile stacks</h3>
  <p dir="auto"><code>datasette-tiles</code> can be configured to serve tiles from multiple attached MBTiles files, searching each database in order for a tile and falling back to the next in line if that tile is not found.</p>
  <p dir="auto">For a demo of this in action, visit <a href="https://datasette-tiles-demo.datasette.io/-/tiles-stack" rel="nofollow">https://datasette-tiles-demo.datasette.io/-/tiles-stack</a> and zoom in on Japan. It should start showing <a href="maps.stamen.com">Stamen's Toner map</a> of Japan once you get to zoom level 6 and 7.</p>
  <p dir="auto">The <code>/-/tiles-stack/{z}/{x}/{y}.png</code> endpoint provides this feature.</p>
  <p dir="auto">If you start Datasette like this:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="datasette world.mbtiles country.mbtiles city1.mbtiles city2.mbtiles"><pre><code>datasette world.mbtiles country.mbtiles city1.mbtiles city2.mbtiles
  </code></pre></div>
  <p dir="auto">Any requests for a tile from the <code>/-/tiles-stack</code> path will first check the <code>city2</code> database, than <code>city1</code>, then <code>country</code>, then <code>world</code>.</p>
  <p dir="auto">If you have the <a href="https://datasette.io/plugins/datasette-basemap" rel="nofollow">datasette-basemap</a> plugin installed it will be given special treatment: the <code>basemap</code> database will always be the last database checked for a tile.</p>
  <p dir="auto">Rather than rely on the order in which databases were attached, you can instead configure an explicit order using the <code>tiles-stack-order</code> plugin setting. Add the following to your <code>metadata.json</code> file:</p>
  <div class="highlight highlight-source-json position-relative overflow-auto" data-snippet-clipboard-copy-content="{
      &quot;plugins&quot;: {
          &quot;datasette-tiles&quot;: {
              &quot;tiles-stack-order&quot;: [&quot;world&quot;, &quot;country&quot;]
          }
      }
  }"><pre>{
      <span class="pl-ent">"plugins"</span>: {
          <span class="pl-ent">"datasette-tiles"</span>: {
              <span class="pl-ent">"tiles-stack-order"</span>: [<span class="pl-s"><span class="pl-pds">"</span>world<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>country<span class="pl-pds">"</span></span>]
          }
      }
  }</pre></div>
  <p dir="auto">You can then run Datasette like this:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="datasette -m metadata.json country.mbtiles world.mbtiles"><pre><code>datasette -m metadata.json country.mbtiles world.mbtiles
  </code></pre></div>
  <p dir="auto">This endpoint serves tiles using the OpenStreetMap / Google Maps coordinate system. To load tiles using the TMS coordinate system use this endpoint instead:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="/-/tiles-stack-tms/{z}/{x}/{y}.png"><pre><code>/-/tiles-stack-tms/{z}/{x}/{y}.png
  </code></pre></div>
  <h3 dir="auto"><a id="user-content-retina-tiles" class="anchor" aria-hidden="true" href="#user-content-retina-tiles"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Retina tiles</h3>
  <p dir="auto">Retina (double resolution) tiles are supported by <code>datasette-tiles</code> if the MBTiles database file contains 512x512 tile images as opposed to the default of 256x256. JavaScript libraries such as Leaflet will serve these tiles with a fixed 256x256 size, which will cause them to be displayed correctly by capable operating systems.</p>
  <h2 dir="auto"><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p dir="auto">To set up this plugin locally, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="cd datasette-tiles
  python3 -mvenv venv
  source venv/bin/activate"><pre><code>cd datasette-tiles
  python3 -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p dir="auto">Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pipenv shell"><pre><code>pipenv shell
  </code></pre></div>
  <p dir="auto">Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pip install -e '.[test]'"><pre><code>pip install -e '.[test]'
  </code></pre></div>
  <p dir="auto">To run the tests:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pytest"><pre><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkzMzUzNzIwNTA=
name: download-tiles
full_name: simonw/download-tiles
html_url: https://github.com/simonw/download-tiles
description: Download map tiles and store them in an MBTiles database
created_at: 2021-02-02T17:37:49Z
updated_at: 2021-05-29T07:22:58Z
pushed_at: 2021-02-16T04:19:59Z
homepage: https://datasette.io/tools/download-tiles
language: Python
license: apache-2.0
topics: ["openstreetmap", "mbtiles", "datasette-io", "datasette-tool"]
default_branch: main
permissions: {"admin": false, "push": false, "pull": false}
readme:
  # download-tiles

  [![PyPI](https://img.shields.io/pypi/v/download-tiles.svg)](https://pypi.org/project/download-tiles/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/download-tiles?include_prereleases&label=changelog)](https://github.com/simonw/download-tiles/releases)
  [![Tests](https://github.com/simonw/download-tiles/workflows/Test/badge.svg)](https://github.com/simonw/download-tiles/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/download-tiles/blob/master/LICENSE)

  Download map tiles and store them in an MBTiles database

  ## Installation

  Install this tool using `pip`:

      $ pip install download-tiles

  ## Usage

  This tool downloads tiles from a specified [TMS (Tile Map Server)](https://wiki.openstreetmap.org/wiki/TMS) server for a specified bounding box and range of zoom levels and stores those tiles in a MBTiles SQLite database. It is a command-line wrapper around the [Landez](https://github.com/makinacorpus/landez) Python libary.

  **Please use this tool responsibly**. Consult the usage policies of the tile servers you are interacting with, for example the [OpenStreetMap Tile Usage Policy](https://operations.osmfoundation.org/policies/tiles/).

  Running the following will download zoom levels 0-3 of OpenStreetMap, 85 tiles total, and store them in a SQLite database called `world.mbtiles`:

      download-tiles world.mbtiles

  You can customize which tile and zoom levels are downloaded using command options:

  `--zoom-levels=0-3` or `-z=0-3`

  The different zoom levels to download. Specify a single number, e.g. `15`, or a range of numbers e.g. `0-4`. Be careful with this setting as you can easily go over the limits requested by the underlying tile server.

  `--bbox=3.9,-6.3,14.5,10.2` or `-b=3.9,-6.3,14.5,10.2`

  The bounding box to fetch. Should be specified as `min-lon,min-lat,max-lon,max-lat`. You can use [bboxfinder.com](http://bboxfinder.com/) to find these for different areas.

  `--city=london` or `--country=madagascar`

  These options can be used instead of `--bbox`. The city or country specified will be looked up using the [Nominatum API](https://nominatim.org/release-docs/latest/api/Search/) and used to derive a bounding box.

  `--show-bbox`

  Use this option to output the bounding box that was retrieved for the `--city` or `--country` without downloading any tiles.

  `--name=Name`

  A name for this tile collection, used for the `name` field in the `metadata` table. If not specified a UUID will be used, or if you used `--city` or `--country` the name will be set to the full name of that place.

  `--attribution="Attribution string"`

  Attribution string to bake into the `metadata` table. This will default to `© OpenStreetMap contributors` unless you use `--tiles-url` to specify an alternative tile server, in which case you should specify a custom attribution string.

  You can use the `--attribution=osm` shortcut to specify the `© OpenStreetMap contributors` value without having to type it out in full.

  `--tiles-url=https://...`

  The tile server URL to use. This should include `{z}` and `{x}` and `{y}` specifiers, and can optionally include `{s}` for subdomains.

  The default URL used here is for OpenStreetMap, `http://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png`

  `--tiles-subdomains=a,b,c`

  A comma-separated list of subdomains to use for the `{s}` parameter.

  `--verbose`

  Use this option to turn on verbose logging.

  `--cache-dir=/tmp/tiles`

  Provide a directory to cache downloaded tiles between runs. This can be useful if you are worried you might not have used the correct options for the bounding box or zoom levels.

  ## Development

  To contribute to this tool, first checkout the code. Then create a new virtual environment:

      cd download-tiles
      python -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-download-tiles" class="anchor" aria-hidden="true" href="#user-content-download-tiles"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>download-tiles</h1>
  <p><a href="https://pypi.org/project/download-tiles/" rel="nofollow"><img src="https://camo.githubusercontent.com/82c46313526394b774727137f12ac6fbf6606364edbee19ebd99c951953b04b5/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f646f776e6c6f61642d74696c65732e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/download-tiles.svg" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/download-tiles/releases"><img src="https://camo.githubusercontent.com/32d363e282d2f95ba1b135630d328b8b61459c3e97b2b4e2fe1ec629be13d80e/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f646f776e6c6f61642d74696c65733f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/download-tiles?include_prereleases&amp;label=changelog" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/download-tiles/actions?query=workflow%3ATest"><img src="https://github.com/simonw/download-tiles/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/download-tiles/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p>
  <p>Download map tiles and store them in an MBTiles database</p>
  <h2><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p>Install this tool using <code>pip</code>:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ pip install download-tiles
  "><pre><code>$ pip install download-tiles
  </code></pre></div>
  <h2><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p>This tool downloads tiles from a specified <a href="https://wiki.openstreetmap.org/wiki/TMS" rel="nofollow">TMS (Tile Map Server)</a> server for a specified bounding box and range of zoom levels and stores those tiles in a MBTiles SQLite database. It is a command-line wrapper around the <a href="https://github.com/makinacorpus/landez">Landez</a> Python libary.</p>
  <p><strong>Please use this tool responsibly</strong>. Consult the usage policies of the tile servers you are interacting with, for example the <a href="https://operations.osmfoundation.org/policies/tiles/" rel="nofollow">OpenStreetMap Tile Usage Policy</a>.</p>
  <p>Running the following will download zoom levels 0-3 of OpenStreetMap, 85 tiles total, and store them in a SQLite database called <code>world.mbtiles</code>:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="download-tiles world.mbtiles
  "><pre><code>download-tiles world.mbtiles
  </code></pre></div>
  <p>You can customize which tile and zoom levels are downloaded using command options:</p>
  <p><code>--zoom-levels=0-3</code> or <code>-z=0-3</code></p>
  <p>The different zoom levels to download. Specify a single number, e.g. <code>15</code>, or a range of numbers e.g. <code>0-4</code>. Be careful with this setting as you can easily go over the limits requested by the underlying tile server.</p>
  <p><code>--bbox=3.9,-6.3,14.5,10.2</code> or <code>-b=3.9,-6.3,14.5,10.2</code></p>
  <p>The bounding box to fetch. Should be specified as <code>min-lon,min-lat,max-lon,max-lat</code>. You can use <a href="http://bboxfinder.com/" rel="nofollow">bboxfinder.com</a> to find these for different areas.</p>
  <p><code>--city=london</code> or <code>--country=madagascar</code></p>
  <p>These options can be used instead of <code>--bbox</code>. The city or country specified will be looked up using the <a href="https://nominatim.org/release-docs/latest/api/Search/" rel="nofollow">Nominatum API</a> and used to derive a bounding box.</p>
  <p><code>--show-bbox</code></p>
  <p>Use this option to output the bounding box that was retrieved for the <code>--city</code> or <code>--country</code> without downloading any tiles.</p>
  <p><code>--name=Name</code></p>
  <p>A name for this tile collection, used for the <code>name</code> field in the <code>metadata</code> table. If not specified a UUID will be used, or if you used <code>--city</code> or <code>--country</code> the name will be set to the full name of that place.</p>
  <p><code>--attribution="Attribution string"</code></p>
  <p>Attribution string to bake into the <code>metadata</code> table. This will default to <code>© OpenStreetMap contributors</code> unless you use <code>--tiles-url</code> to specify an alternative tile server, in which case you should specify a custom attribution string.</p>
  <p>You can use the <code>--attribution=osm</code> shortcut to specify the <code>© OpenStreetMap contributors</code> value without having to type it out in full.</p>
  <p><code>--tiles-url=https://...</code></p>
  <p>The tile server URL to use. This should include <code>{z}</code> and <code>{x}</code> and <code>{y}</code> specifiers, and can optionally include <code>{s}</code> for subdomains.</p>
  <p>The default URL used here is for OpenStreetMap, <code>http://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png</code></p>
  <p><code>--tiles-subdomains=a,b,c</code></p>
  <p>A comma-separated list of subdomains to use for the <code>{s}</code> parameter.</p>
  <p><code>--verbose</code></p>
  <p>Use this option to turn on verbose logging.</p>
  <p><code>--cache-dir=/tmp/tiles</code></p>
  <p>Provide a directory to cache downloaded tiles between runs. This can be useful if you are worried you might not have used the correct options for the bounding box or zoom levels.</p>
  <h2><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p>To contribute to this tool, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="cd download-tiles
  python -mvenv venv
  source venv/bin/activate
  "><pre><code>cd download-tiles
  python -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p>Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pipenv shell
  "><pre><code>pipenv shell
  </code></pre></div>
  <p>Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pip install -e '.[test]'
  "><pre><code>pip install -e '.[test]'
  </code></pre></div>
  <p>To run the tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pytest
  "><pre><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkzNDIxMjY2MTA=
name: datasette-block
full_name: simonw/datasette-block
html_url: https://github.com/simonw/datasette-block
description: Block all access to specific path prefixes
created_at: 2021-02-25T04:51:08Z
updated_at: 2021-02-25T08:18:28Z
pushed_at: 2021-02-25T05:03:45Z
homepage: https://datasette.io/plugins/datasette-block
language: Python
topics: ["datasette", "datasette-plugin", "datasette-io"]
default_branch: main
permissions: {"admin": false, "push": false, "pull": false}
readme:
  # datasette-block

  [![PyPI](https://img.shields.io/pypi/v/datasette-block.svg)](https://pypi.org/project/datasette-block/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-block?include_prereleases&label=changelog)](https://github.com/simonw/datasette-block/releases)
  [![Tests](https://github.com/simonw/datasette-block/workflows/Test/badge.svg)](https://github.com/simonw/datasette-block/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-block/blob/main/LICENSE)

  Block all access to specific path prefixes

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-block

  ## Configuration

  Add the following to `metadata.json` to block specific path prefixes:

  ```json
  {
      "plugins": {
          "datasette-block": {
              "prefixes": ["/all/"]
          }
      }
  }
  ```

  This will cause a 403 error to be returned for any path beginning with `/all/`.

  This blocking happens as an ASGI wrapper around Datasette.

  ## Why would you need this?

  You almost always would not. I use it with `datasette-ripgrep` to block access to static assets for unauthenticated users.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-block
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-datasette-block" class="anchor" aria-hidden="true" href="#user-content-datasette-block"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-block</h1>
  <p><a href="https://pypi.org/project/datasette-block/" rel="nofollow"><img src="https://camo.githubusercontent.com/eedcb03a8e4f528e044cee8db9b02e79782b8a9bccf66bd239701d9d7a991984/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d626c6f636b2e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-block.svg" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-block/releases"><img src="https://camo.githubusercontent.com/05cef47f53da7aaccdd3a18a809f166b816ac100f83757b2488082897269f5b4/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d626c6f636b3f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-block?include_prereleases&amp;label=changelog" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-block/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-block/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-block/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p>
  <p>Block all access to specific path prefixes</p>
  <h2><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p>Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ datasette install datasette-block
  "><pre><code>$ datasette install datasette-block
  </code></pre></div>
  <h2><a id="user-content-configuration" class="anchor" aria-hidden="true" href="#user-content-configuration"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Configuration</h2>
  <p>Add the following to <code>metadata.json</code> to block specific path prefixes:</p>
  <div class="highlight highlight-source-json position-relative" data-snippet-clipboard-copy-content="{
      &quot;plugins&quot;: {
          &quot;datasette-block&quot;: {
              &quot;prefixes&quot;: [&quot;/all/&quot;]
          }
      }
  }
  "><pre>{
      <span class="pl-s"><span class="pl-pds">"</span>plugins<span class="pl-pds">"</span></span>: {
          <span class="pl-s"><span class="pl-pds">"</span>datasette-block<span class="pl-pds">"</span></span>: {
              <span class="pl-s"><span class="pl-pds">"</span>prefixes<span class="pl-pds">"</span></span>: [<span class="pl-s"><span class="pl-pds">"</span>/all/<span class="pl-pds">"</span></span>]
          }
      }
  }</pre></div>
  <p>This will cause a 403 error to be returned for any path beginning with <code>/all/</code>.</p>
  <p>This blocking happens as an ASGI wrapper around Datasette.</p>
  <h2><a id="user-content-why-would-you-need-this" class="anchor" aria-hidden="true" href="#user-content-why-would-you-need-this"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Why would you need this?</h2>
  <p>You almost always would not. I use it with <code>datasette-ripgrep</code> to block access to static assets for unauthenticated users.</p>
  <h2><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p>To set up this plugin locally, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="cd datasette-block
  python3 -mvenv venv
  source venv/bin/activate
  "><pre><code>cd datasette-block
  python3 -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p>Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pipenv shell
  "><pre><code>pipenv shell
  </code></pre></div>
  <p>Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pip install -e '.[test]'
  "><pre><code>pip install -e '.[test]'
  </code></pre></div>
  <p>To run the tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pytest
  "><pre><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkzNDY1OTc1NTc=
name: tableau-to-sqlite
full_name: simonw/tableau-to-sqlite
html_url: https://github.com/simonw/tableau-to-sqlite
description: Fetch data from Tableau into a SQLite database
created_at: 2021-03-11T06:12:02Z
updated_at: 2021-06-10T04:40:44Z
pushed_at: 2021-04-29T16:11:03Z
language: Python
license: apache-2.0
topics: ["datasette-io", "datasette-tool"]
default_branch: main
permissions: {"admin": false, "push": false, "pull": false}
readme:
  # tableau-to-sqlite

  [![PyPI](https://img.shields.io/pypi/v/tableau-to-sqlite.svg)](https://pypi.org/project/tableau-to-sqlite/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/tableau-to-sqlite?include_prereleases&label=changelog)](https://github.com/simonw/tableau-to-sqlite/releases)
  [![Tests](https://github.com/simonw/tableau-to-sqlite/workflows/Test/badge.svg)](https://github.com/simonw/tableau-to-sqlite/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/tableau-to-sqlite/blob/master/LICENSE)

  Fetch data from Tableau into a SQLite database. A wrapper around [TableauScraper](https://github.com/bertrandmartel/tableau-scraping/).

  ## Installation

  Install this tool using `pip`:

      $ pip install tableau-to-sqlite

  ## Usage

  If you have the URL to a Tableau dashboard like this:

  https://results.mo.gov/t/COVID19/views/VaccinationsDashboard/Vaccinations

  You can pass that directly to the tool:

      tableau-to-sqlite tableau.db \
        https://results.mo.gov/t/COVID19/views/VaccinationsDashboard/Vaccinations

  This will create a SQLite database called `tableau.db` containing one table for each of the worksheets in that dashboard.

  If the dashboard is hosted on https://public.tableau.com/ you can instead provide the view name. This will be two strings separated by a `/` symbol - something like this:

      OregonCOVID-19VaccineProviderEnrollment/COVID-19VaccineProviderEnrollment

  Now run the tool like this:

      tableau-to-sqlite tableau.db \
          OregonCOVID-19VaccineProviderEnrollment/COVID-19VaccineProviderEnrollment

  ## Get the data as JSON or CSV

  If you're building a [git scraper](https://simonwillison.net/2020/Oct/9/git-scraping/) you may want to convert the data gathered by this tool to CSV or JSON to check into your repository.

  You can do that using [sqlite-utils](https://sqlite-utils.datasette.io/). Install it using `pip`:

      pip install sqlite-utils

  You can dump out a table as JSON like so:

      sqlite-utils rows tableau.db \
         'Admin Site and County Map Site No Info' > tableau.json

  Or as CSV like this:

      sqlite-utils rows tableau.db --csv \
         'Admin Site and County Map Site No Info' > tableau.csv

  ## Development

  To contribute to this tool, first checkout the code. Then create a new virtual environment:

      cd tableau-to-sqlite
      python -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-tableau-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-tableau-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>tableau-to-sqlite</h1>
  <p><a href="https://pypi.org/project/tableau-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/bdda6efc6980f655c99f0475b322f27bbc413e7e13e847e55e697dd135d10601/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f7461626c6561752d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/tableau-to-sqlite.svg" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/tableau-to-sqlite/releases"><img src="https://camo.githubusercontent.com/fd18da3a8541734c627a9be68134477abe876a8695165916ff7c8c5bbf7269eb/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f7461626c6561752d746f2d73716c6974653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/tableau-to-sqlite?include_prereleases&amp;label=changelog" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/tableau-to-sqlite/actions?query=workflow%3ATest"><img src="https://github.com/simonw/tableau-to-sqlite/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/tableau-to-sqlite/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p>
  <p>Fetch data from Tableau into a SQLite database. A wrapper around <a href="https://github.com/bertrandmartel/tableau-scraping/">TableauScraper</a>.</p>
  <h2><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p>Install this tool using <code>pip</code>:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ pip install tableau-to-sqlite
  "><pre><code>$ pip install tableau-to-sqlite
  </code></pre></div>
  <h2><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p>If you have the URL to a Tableau dashboard like this:</p>
  <p><a href="https://results.mo.gov/t/COVID19/views/VaccinationsDashboard/Vaccinations" rel="nofollow">https://results.mo.gov/t/COVID19/views/VaccinationsDashboard/Vaccinations</a></p>
  <p>You can pass that directly to the tool:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="tableau-to-sqlite tableau.db \
    https://results.mo.gov/t/COVID19/views/VaccinationsDashboard/Vaccinations
  "><pre><code>tableau-to-sqlite tableau.db \
    https://results.mo.gov/t/COVID19/views/VaccinationsDashboard/Vaccinations
  </code></pre></div>
  <p>This will create a SQLite database called <code>tableau.db</code> containing one table for each of the worksheets in that dashboard.</p>
  <p>If the dashboard is hosted on <a href="https://public.tableau.com/" rel="nofollow">https://public.tableau.com/</a> you can instead provide the view name. This will be two strings separated by a <code>/</code> symbol - something like this:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="OregonCOVID-19VaccineProviderEnrollment/COVID-19VaccineProviderEnrollment
  "><pre><code>OregonCOVID-19VaccineProviderEnrollment/COVID-19VaccineProviderEnrollment
  </code></pre></div>
  <p>Now run the tool like this:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="tableau-to-sqlite tableau.db \
      OregonCOVID-19VaccineProviderEnrollment/COVID-19VaccineProviderEnrollment
  "><pre><code>tableau-to-sqlite tableau.db \
      OregonCOVID-19VaccineProviderEnrollment/COVID-19VaccineProviderEnrollment
  </code></pre></div>
  <h2><a id="user-content-get-the-data-as-json-or-csv" class="anchor" aria-hidden="true" href="#user-content-get-the-data-as-json-or-csv"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Get the data as JSON or CSV</h2>
  <p>If you're building a <a href="https://simonwillison.net/2020/Oct/9/git-scraping/" rel="nofollow">git scraper</a> you may want to convert the data gathered by this tool to CSV or JSON to check into your repository.</p>
  <p>You can do that using <a href="https://sqlite-utils.datasette.io/" rel="nofollow">sqlite-utils</a>. Install it using <code>pip</code>:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pip install sqlite-utils
  "><pre><code>pip install sqlite-utils
  </code></pre></div>
  <p>You can dump out a table as JSON like so:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="sqlite-utils rows tableau.db \
     'Admin Site and County Map Site No Info' &gt; tableau.json
  "><pre><code>sqlite-utils rows tableau.db \
     'Admin Site and County Map Site No Info' &gt; tableau.json
  </code></pre></div>
  <p>Or as CSV like this:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="sqlite-utils rows tableau.db --csv \
     'Admin Site and County Map Site No Info' &gt; tableau.csv
  "><pre><code>sqlite-utils rows tableau.db --csv \
     'Admin Site and County Map Site No Info' &gt; tableau.csv
  </code></pre></div>
  <h2><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p>To contribute to this tool, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="cd tableau-to-sqlite
  python -mvenv venv
  source venv/bin/activate
  "><pre><code>cd tableau-to-sqlite
  python -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p>Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pipenv shell
  "><pre><code>pipenv shell
  </code></pre></div>
  <p>Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pip install -e '.[test]'
  "><pre><code>pip install -e '.[test]'
  </code></pre></div>
  <p>To run the tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pytest
  "><pre><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkzNDcyNjM3MjI=
name: django-sql-dashboard
full_name: simonw/django-sql-dashboard
html_url: https://github.com/simonw/django-sql-dashboard
description: Django app for building dashboards using raw SQL queries
created_at: 2021-03-13T03:38:23Z
updated_at: 2022-04-19T01:13:12Z
pushed_at: 2022-04-20T00:27:39Z
homepage: https://django-sql-dashboard.datasette.io/
language: Python
license: apache-2.0
topics: ["dashboards", "datasette-io", "datasette-tool", "django", "sql"]
default_branch: main
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
visibility: public
readme:
  # django-sql-dashboard

  [![PyPI](https://img.shields.io/pypi/v/django-sql-dashboard.svg)](https://pypi.org/project/django-sql-dashboard/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/django-sql-dashboard?include_prereleases&label=changelog)](https://github.com/simonw/django-sql-dashboard/releases)
  [![Tests](https://github.com/simonw/django-sql-dashboard/workflows/Test/badge.svg)](https://github.com/simonw/django-sql-dashboard/actions?query=workflow%3ATest)
  [![Documentation Status](https://readthedocs.org/projects/django-sql-dashboard/badge/?version=latest)](http://django-sql-dashboard.datasette.io/en/latest/?badge=latest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/django-sql-dashboard/blob/main/LICENSE)

  Django SQL Dashboard provides an authenticated interface for executing read-only SQL queries directly against your PostgreSQL database, bringing a useful subset of [Datasette](https://datasette.io/) to Django.

  Applications include ad-hoc analysis and debugging, plus the creation of reporting dashboards that can be shared with team members or published online.

  See my blog for [more about this project](https://simonwillison.net/2021/May/10/django-sql-dashboard/), including [a video demo](https://www.youtube.com/watch?v=ausrmMZkPEY).

  Features include:

  - Safely run read-only one or more SQL queries against your database and view the results in your browser
  - Bookmark queries and share those links with other members of your team
  - Create [saved dashboards](https://django-sql-dashboard.datasette.io/en/latest/saved-dashboards.html) from your queries, with full control over who can view and edit them
  - [Named parameters](https://django-sql-dashboard.datasette.io/en/latest/sql.html#sql-parameters) such as `select * from entries where id = %(id)s` will be turned into form fields, allowing quick creation of interactive dashboards
  - Produce [bar charts](https://django-sql-dashboard.datasette.io/en/latest/widgets.html#bar-label-bar-quantity), [progress bars](https://django-sql-dashboard.datasette.io/en/latest/widgets.html#total-count-completed-count) and more from SQL queries, with the ability to easily create new [custom dashboard widgets](https://django-sql-dashboard.datasette.io/en/latest/widgets.html#custom-widgets) using the Django template system
  - Write SQL queries that safely construct and render [markdown](https://django-sql-dashboard.datasette.io/en/latest/widgets.html#markdown) and [HTML](https://django-sql-dashboard.datasette.io/en/latest/widgets.html#html)
  - Export the full results of a SQL query as a downloadable CSV or TSV file, using a combination of Django's [streaming HTTP response](https://docs.djangoproject.com/en/3.2/ref/request-response/#django.http.StreamingHttpResponse) mechanism and PostgreSQL [server-side cursors](https://www.psycopg.org/docs/usage.html#server-side-cursors) to efficiently stream large amounts of data without running out of resources
  - Copy and paste the results of SQL queries directly into tools such as Google Sheets or Excel
  - Uses Django's authentication system, so dashboard accounts can be granted using Django's Admin tools

  ## Documentation

  Full documentation is at [django-sql-dashboard.datasette.io](https://django-sql-dashboard.datasette.io/)

  ## Screenshot

  <img width="1018" alt="Screenshot showing a SQL query that produces a table and one that produces a bar chart" src="https://user-images.githubusercontent.com/9599/124050883-42ad2300-d9d0-11eb-83e6-44ad85f7ef64.png">

  ## Alternatives

  - [django-sql-explorer](https://github.com/groveco/django-sql-explorer) provides a related set of functionality that also works against database backends other than PostgreSQL

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-django-sql-dashboard" class="anchor" aria-hidden="true" href="#user-content-django-sql-dashboard"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>django-sql-dashboard</h1>
  <p dir="auto"><a href="https://pypi.org/project/django-sql-dashboard/" rel="nofollow"><img src="https://camo.githubusercontent.com/87a7771e261b30e7b0da09a83f2d6120ce484b068a0d844a41d2f945141194d1/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f646a616e676f2d73716c2d64617368626f6172642e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/django-sql-dashboard.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/django-sql-dashboard/releases"><img src="https://camo.githubusercontent.com/f3c931ccf5487f5df160f339f767aba9ac45e15a5f4e8beb37fd6e33cbc239c0/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f646a616e676f2d73716c2d64617368626f6172643f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/django-sql-dashboard?include_prereleases&amp;label=changelog" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/django-sql-dashboard/actions?query=workflow%3ATest"><img src="https://github.com/simonw/django-sql-dashboard/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a>
  <a href="http://django-sql-dashboard.datasette.io/en/latest/?badge=latest" rel="nofollow"><img src="https://camo.githubusercontent.com/e5bd1be998e2bf35ee16299f10c1520b531a957a3b725b6b663e40d82d3c8425/68747470733a2f2f72656164746865646f63732e6f72672f70726f6a656374732f646a616e676f2d73716c2d64617368626f6172642f62616467652f3f76657273696f6e3d6c6174657374" alt="Documentation Status" data-canonical-src="https://readthedocs.org/projects/django-sql-dashboard/badge/?version=latest" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/django-sql-dashboard/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width: 100%;"></a></p>
  <p dir="auto">Django SQL Dashboard provides an authenticated interface for executing read-only SQL queries directly against your PostgreSQL database, bringing a useful subset of <a href="https://datasette.io/" rel="nofollow">Datasette</a> to Django.</p>
  <p dir="auto">Applications include ad-hoc analysis and debugging, plus the creation of reporting dashboards that can be shared with team members or published online.</p>
  <p dir="auto">See my blog for <a href="https://simonwillison.net/2021/May/10/django-sql-dashboard/" rel="nofollow">more about this project</a>, including <a href="https://www.youtube.com/watch?v=ausrmMZkPEY" rel="nofollow">a video demo</a>.</p>
  <p dir="auto">Features include:</p>
  <ul dir="auto">
  <li>Safely run read-only one or more SQL queries against your database and view the results in your browser</li>
  <li>Bookmark queries and share those links with other members of your team</li>
  <li>Create <a href="https://django-sql-dashboard.datasette.io/en/latest/saved-dashboards.html" rel="nofollow">saved dashboards</a> from your queries, with full control over who can view and edit them</li>
  <li><a href="https://django-sql-dashboard.datasette.io/en/latest/sql.html#sql-parameters" rel="nofollow">Named parameters</a> such as <code>select * from entries where id = %(id)s</code> will be turned into form fields, allowing quick creation of interactive dashboards</li>
  <li>Produce <a href="https://django-sql-dashboard.datasette.io/en/latest/widgets.html#bar-label-bar-quantity" rel="nofollow">bar charts</a>, <a href="https://django-sql-dashboard.datasette.io/en/latest/widgets.html#total-count-completed-count" rel="nofollow">progress bars</a> and more from SQL queries, with the ability to easily create new <a href="https://django-sql-dashboard.datasette.io/en/latest/widgets.html#custom-widgets" rel="nofollow">custom dashboard widgets</a> using the Django template system</li>
  <li>Write SQL queries that safely construct and render <a href="https://django-sql-dashboard.datasette.io/en/latest/widgets.html#markdown" rel="nofollow">markdown</a> and <a href="https://django-sql-dashboard.datasette.io/en/latest/widgets.html#html" rel="nofollow">HTML</a></li>
  <li>Export the full results of a SQL query as a downloadable CSV or TSV file, using a combination of Django's <a href="https://docs.djangoproject.com/en/3.2/ref/request-response/#django.http.StreamingHttpResponse" rel="nofollow">streaming HTTP response</a> mechanism and PostgreSQL <a href="https://www.psycopg.org/docs/usage.html#server-side-cursors" rel="nofollow">server-side cursors</a> to efficiently stream large amounts of data without running out of resources</li>
  <li>Copy and paste the results of SQL queries directly into tools such as Google Sheets or Excel</li>
  <li>Uses Django's authentication system, so dashboard accounts can be granted using Django's Admin tools</li>
  </ul>
  <h2 dir="auto"><a id="user-content-documentation" class="anchor" aria-hidden="true" href="#user-content-documentation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Documentation</h2>
  <p dir="auto">Full documentation is at <a href="https://django-sql-dashboard.datasette.io/" rel="nofollow">django-sql-dashboard.datasette.io</a></p>
  <h2 dir="auto"><a id="user-content-screenshot" class="anchor" aria-hidden="true" href="#user-content-screenshot"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Screenshot</h2>
  <p><a target="_blank" rel="noopener noreferrer" href="https://user-images.githubusercontent.com/9599/124050883-42ad2300-d9d0-11eb-83e6-44ad85f7ef64.png"><img width="1018" alt="Screenshot showing a SQL query that produces a table and one that produces a bar chart" src="https://user-images.githubusercontent.com/9599/124050883-42ad2300-d9d0-11eb-83e6-44ad85f7ef64.png" style="max-width: 100%;"></a></p>
  <h2 dir="auto"><a id="user-content-alternatives" class="anchor" aria-hidden="true" href="#user-content-alternatives"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Alternatives</h2>
  <ul dir="auto">
  <li><a href="https://github.com/groveco/django-sql-explorer">django-sql-explorer</a> provides a related set of functionality that also works against database backends other than PostgreSQL</li>
  </ul>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkzNjEwMTQyNzM=
name: datasette-dashboards
full_name: rclement/datasette-dashboards
html_url: https://github.com/rclement/datasette-dashboards
description: Datasette plugin providing data dashboards from metadata
created_at: 2021-04-23T21:56:48Z
updated_at: 2022-09-21T13:03:39Z
pushed_at: 2022-10-07T07:18:03Z
homepage: https://datasette-dashboards-demo.vercel.app
language: Python
license: apache-2.0
default_branch: master
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
visibility: public
topics:
  ["dashboards", "data-visualization", "datasette", "datasette-io", "datasette-plugin", "sql", "vega-lite"]

readme:
  # datasette-dashboards

  > Datasette plugin providing data dashboards from metadata

  [![PyPI](https://img.shields.io/pypi/v/datasette-dashboards.svg)](https://pypi.org/project/datasette-dashboards/)
  [![CI/CD](https://github.com/rclement/datasette-dashboards/actions/workflows/ci-cd.yml/badge.svg)](https://github.com/rclement/datasette-dashboards/actions/workflows/ci-cd.yml)
  [![Coverage Status](https://img.shields.io/codecov/c/github/rclement/datasette-dashboards)](https://codecov.io/gh/rclement/datasette-dashboards)
  [![License](https://img.shields.io/github/license/rclement/datasette-dashboards)](https://github.com/rclement/datasette-dashboards/blob/master/LICENSE)

  Try out a live demo at [https://datasette-dashboards-demo.vercel.app](https://datasette-dashboards-demo.vercel.app/-/dashboards)

  **WARNING**: this plugin is still experimental and not ready for production.
  Some breaking changes might happen between releases before reaching a stable version.
  Use it at your own risks!

  ![Datasette Dashboards Demo](https://raw.githubusercontent.com/rclement/datasette-dashboards/master/demo/datasette-dashboards-demo.png)

  ## Installation

  Install this plugin in the same environment as Datasette:

  ```bash
  $ datasette install datasette-dashboards
  ```

  ## Usage

  Define dashboards within `metadata.yml` / `metadata.json`:

  ```yaml
  plugins:
    datasette-dashboards:
      my-dashboard:
        title: My Dashboard
        description: Showing some nice metrics
        layout:
          - [analysis-note, events-count]
          - [analysis-note, events-source]
        filters:
          date_start:
            name: Date Start
            type: date
            default: "2021-01-01"
          date_end:
            name: Date End
            type: date
        charts:
          analysis-note:
            library: markdown
            display: |-
              # Analysis notes
              > A quick rundown of events statistics and KPIs

          events-count:
            title: Total number of events
            db: jobs
            query: SELECT count(*) as count FROM events
            library: metric
            display:
              field: count
              prefix:
              suffix:

          events-source:
            title: Number of events by source
            db: jobs
            query: SELECT source, count(*) as count FROM events WHERE TRUE [[ AND date >= date(:date_start) ]] [[ AND date <= date(:date_end) ]] GROUP BY source ORDER BY count DESC
            library: vega
            display:
              mark: { type: bar, tooltip: true }
              encoding:
                color: { field: source, type: nominal }
                theta: { field: count, type: quantitative }
  ```

  A new menu entry is now available, pointing at `/-/dashboards` to access all defined dashboards.

  ### Properties

  Dashboard properties:

  | Property      | Type     | Description           |
  | ------------- | -------- | --------------------- |
  | `title`       | `string` | Dashboard title       |
  | `description` | `string` | Dashboard description |
  | `layout`      | `array`  | Dashboard layout      |
  | `filters`     | `object` | Dashboard filters     |

  Dashboard filters:

  | Property  | Type               | Description                            |
  | --------- | ------------------ | -------------------------------------- |
  | `name`    | `string`           | Filter display name                    |
  | `type`    | `string`           | Filter type (`text`, `date`, `number`) |
  | `default` | `string`, `number` | (optional) Filter default value        |
  | `min`     | `number`           | (optional) Filter minimum value        |
  | `max`     | `number`           | (optional) Filter maximum value        |
  | `step`    | `number`           | (optional) Filter stepping value       |

  Common chart properties for all chart types:

  | Property  | Type     | Description                                              |
  | --------- | -------- | -------------------------------------------------------- |
  | `title`   | `string` | Chart title                                              |
  | `db`      | `string` | Database name against which to run the query             |
  | `query`   | `string` | SQL query to run and extract data from                   |
  | `library` | `string` | One of supported libraries: `vega`, `markdown`, `metric` |
  | `display` | `object` | Chart display specification (depend on the used library) |

  To define SQL queries using dashboard filters:

  ```sql
  SELECT * FROM mytable [[ WHERE col >= :my_filter ]]
  ```

  ```sql
  SELECT * FROM mytable WHERE TRUE [[ AND col1 = :my_filter_1 ]] [[ AND col2 = :my_filter_2 ]]
  ```

  #### Vega properties

  Available configuration for `vega` charts:

  | Property  | Type     | Description               |
  | --------- | -------- | ------------------------- |
  | `library` | `string` | Must be set to `vega`     |
  | `display` | `object` | Vega specification object |

  Notes about the `display` property:

  - Requires a valid [Vega specification object](https://vega.github.io/vega-lite/docs/)
  - Some fields are pre-defined: `$schema`, `title`, `width`, `view`, `config`, `data`
  - All fields are passed along as-is (overriding pre-defined fields if any)
  - Only `mark` and `encoding` fields are required as the bare-minimum

  #### Markdown properties

  Available configuration for `markdown` chart:

  | Property  | Type     | Description                                       |
  | --------- | -------- | ------------------------------------------------- |
  | `library` | `string` | Must be set to `markdown`                         |
  | `display` | `string` | Multi-line string containing the Markdown content |

  Note :

  - Some common properties do not apply and can be omitted: `title`, `db`, `query`
  - Markdown rendering is done by [`datasette-render-markdown`](https://datasette.io/plugins/datasette-render-markdown)
  - To configure Markdown rendering, extensions can be enabled in [metadata](https://datasette.io/plugins/datasette-render-markdown#user-content-markdown-extensions)

  #### Metric properties

  Available configuration for `metric` chart:

  | Property         | Type     | Description                               |
  | ---------------- | -------- | ----------------------------------------- |
  | `library`        | `string` | Must be set to `metric`                   |
  | `display.field`  | `string` | Numerical field to be displayed as metric |
  | `display.prefix` | `string` | Prefix to be displayed before metric      |
  | `display.suffix` | `string` | Prefix to be displayed after metric       |

  Note:

  - The `display.field` must reference a single-numerical value from the SQL query
    (e.g. numerical `number` field in `SELECT count(*) as number FROM events`)

  ### Dashboard layout

  The default dashboard layout will present two charts per row (one per row on mobile).
  To make use of custom dashboard layout using [CSS Grid Layout](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Grid_Layout),
  define the `layout` array property as a grid / matrix:

  - Each entry represents a row of charts
  - Each column is referring a chart by its property name

  ## Development

  To set up this plugin locally, first checkout the code.
  Then create a new virtual environment and the required dependencies:

  ```bash
  pipenv install -d
  pipenv shell
  ```

  To run the tests:

  ```bash
  pytest
  ```

  ## Demo

  With the developmnent environment setup, you can run the demo locally:

  ```bash
  datasette --metadata demo/metadata.yml demo/jobs.db
  ```

  ## License

  Licensed under Apache License, Version 2.0

  Copyright (c) 2021 - present Romain Clement

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-datasette-dashboards" class="anchor" aria-hidden="true" href="#user-content-datasette-dashboards"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-dashboards</h1>
  <blockquote>
  <p dir="auto">Datasette plugin providing data dashboards from metadata</p>
  </blockquote>
  <p dir="auto"><a href="https://pypi.org/project/datasette-dashboards/" rel="nofollow"><img src="https://camo.githubusercontent.com/f362a9e19c9666117e279326df7065185827fbec6acab7c6ae6c5e7e379191db/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d64617368626f617264732e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-dashboards.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/rclement/datasette-dashboards/actions/workflows/ci-cd.yml"><img src="https://github.com/rclement/datasette-dashboards/actions/workflows/ci-cd.yml/badge.svg" alt="CI/CD" style="max-width: 100%;"></a>
  <a href="https://codecov.io/gh/rclement/datasette-dashboards" rel="nofollow"><img src="https://camo.githubusercontent.com/12d74ac8b571a59d446ce62428195bfc76d2027e8394fb3cb1e8584bdf025c9f/68747470733a2f2f696d672e736869656c64732e696f2f636f6465636f762f632f6769746875622f72636c656d656e742f6461746173657474652d64617368626f61726473" alt="Coverage Status" data-canonical-src="https://img.shields.io/codecov/c/github/rclement/datasette-dashboards" style="max-width: 100%;"></a>
  <a href="https://github.com/rclement/datasette-dashboards/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/210d7ef998772287f630b1a00a40abc4c413c989b0c8757f93ff9c674d46f5c7/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f6c6963656e73652f72636c656d656e742f6461746173657474652d64617368626f61726473" alt="License" data-canonical-src="https://img.shields.io/github/license/rclement/datasette-dashboards" style="max-width: 100%;"></a></p>
  <p dir="auto">Try out a live demo at <a href="https://datasette-dashboards-demo.vercel.app/-/dashboards" rel="nofollow">https://datasette-dashboards-demo.vercel.app</a></p>
  <p dir="auto"><strong>WARNING</strong>: this plugin is still experimental and not ready for production.
  Some breaking changes might happen between releases before reaching a stable version.
  Use it at your own risks!</p>
  <p dir="auto"><a target="_blank" rel="noopener noreferrer nofollow" href="https://raw.githubusercontent.com/rclement/datasette-dashboards/master/demo/datasette-dashboards-demo.png"><img src="https://raw.githubusercontent.com/rclement/datasette-dashboards/master/demo/datasette-dashboards-demo.png" alt="Datasette Dashboards Demo" style="max-width: 100%;"></a></p>
  <h2 dir="auto"><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p dir="auto">Install this plugin in the same environment as Datasette:</p>
  <div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="$ datasette install datasette-dashboards"><pre>$ datasette install datasette-dashboards</pre></div>
  <h2 dir="auto"><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p dir="auto">Define dashboards within <code>metadata.yml</code> / <code>metadata.json</code>:</p>
  <div class="highlight highlight-source-yaml notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="plugins:
    datasette-dashboards:
      my-dashboard:
        title: My Dashboard
        description: Showing some nice metrics
        layout:
          - [analysis-note, events-count]
          - [analysis-note, events-source]
        filters:
          date_start:
            name: Date Start
            type: date
            default: &quot;2021-01-01&quot;
          date_end:
            name: Date End
            type: date
        charts:
          analysis-note:
            library: markdown
            display: |-
              # Analysis notes
              &gt; A quick rundown of events statistics and KPIs

          events-count:
            title: Total number of events
            db: jobs
            query: SELECT count(*) as count FROM events
            library: metric
            display:
              field: count
              prefix:
              suffix:

          events-source:
            title: Number of events by source
            db: jobs
            query: SELECT source, count(*) as count FROM events WHERE TRUE [[ AND date &gt;= date(:date_start) ]] [[ AND date &lt;= date(:date_end) ]] GROUP BY source ORDER BY count DESC
            library: vega
            display:
              mark: { type: bar, tooltip: true }
              encoding:
                color: { field: source, type: nominal }
                theta: { field: count, type: quantitative }"><pre><span class="pl-ent">plugins</span>:
    <span class="pl-ent">datasette-dashboards</span>:
      <span class="pl-ent">my-dashboard</span>:
        <span class="pl-ent">title</span>: <span class="pl-s">My Dashboard</span>
        <span class="pl-ent">description</span>: <span class="pl-s">Showing some nice metrics</span>
        <span class="pl-ent">layout</span>:
          - <span class="pl-s">[analysis-note, events-count]</span>
          - <span class="pl-s">[analysis-note, events-source]</span>
        <span class="pl-ent">filters</span>:
          <span class="pl-ent">date_start</span>:
            <span class="pl-ent">name</span>: <span class="pl-s">Date Start</span>
            <span class="pl-ent">type</span>: <span class="pl-s">date</span>
            <span class="pl-ent">default</span>: <span class="pl-s"><span class="pl-pds">"</span>2021-01-01<span class="pl-pds">"</span></span>
          <span class="pl-ent">date_end</span>:
            <span class="pl-ent">name</span>: <span class="pl-s">Date End</span>
            <span class="pl-ent">type</span>: <span class="pl-s">date</span>
        <span class="pl-ent">charts</span>:
          <span class="pl-ent">analysis-note</span>:
            <span class="pl-ent">library</span>: <span class="pl-s">markdown</span>
            <span class="pl-ent">display</span>: <span class="pl-s">|-</span>
  <span class="pl-s">            # Analysis notes</span>
  <span class="pl-s">            &gt; A quick rundown of events statistics and KPIs</span>
  <span class="pl-s"></span>
  <span class="pl-s"></span>        <span class="pl-ent">events-count</span>:
            <span class="pl-ent">title</span>: <span class="pl-s">Total number of events</span>
            <span class="pl-ent">db</span>: <span class="pl-s">jobs</span>
            <span class="pl-ent">query</span>: <span class="pl-s">SELECT count(*) as count FROM events</span>
            <span class="pl-ent">library</span>: <span class="pl-s">metric</span>
            <span class="pl-ent">display</span>:
              <span class="pl-ent">field</span>: <span class="pl-s">count</span>
              <span class="pl-ent">prefix</span>:
              <span class="pl-ent">suffix</span>:

          <span class="pl-ent">events-source</span>:
            <span class="pl-ent">title</span>: <span class="pl-s">Number of events by source</span>
            <span class="pl-ent">db</span>: <span class="pl-s">jobs</span>
            <span class="pl-ent">query</span>: <span class="pl-s">SELECT source, count(*) as count FROM events WHERE TRUE [[ AND date &gt;= date(:date_start) ]] [[ AND date &lt;= date(:date_end) ]] GROUP BY source ORDER BY count DESC</span>
            <span class="pl-ent">library</span>: <span class="pl-s">vega</span>
            <span class="pl-ent">display</span>:
              <span class="pl-ent">mark</span>: <span class="pl-s">{ type: bar, tooltip: true }</span>
              <span class="pl-ent">encoding</span>:
                <span class="pl-ent">color</span>: <span class="pl-s">{ field: source, type: nominal }</span>
                <span class="pl-ent">theta</span>: <span class="pl-s">{ field: count, type: quantitative }</span></pre></div>
  <p dir="auto">A new menu entry is now available, pointing at <code>/-/dashboards</code> to access all defined dashboards.</p>
  <h3 dir="auto"><a id="user-content-properties" class="anchor" aria-hidden="true" href="#user-content-properties"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Properties</h3>
  <p dir="auto">Dashboard properties:</p>
  <table>
  <thead>
  <tr>
  <th>Property</th>
  <th>Type</th>
  <th>Description</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td><code>title</code></td>
  <td><code>string</code></td>
  <td>Dashboard title</td>
  </tr>
  <tr>
  <td><code>description</code></td>
  <td><code>string</code></td>
  <td>Dashboard description</td>
  </tr>
  <tr>
  <td><code>layout</code></td>
  <td><code>array</code></td>
  <td>Dashboard layout</td>
  </tr>
  <tr>
  <td><code>filters</code></td>
  <td><code>object</code></td>
  <td>Dashboard filters</td>
  </tr>
  </tbody>
  </table>
  <p dir="auto">Dashboard filters:</p>
  <table>
  <thead>
  <tr>
  <th>Property</th>
  <th>Type</th>
  <th>Description</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td><code>name</code></td>
  <td><code>string</code></td>
  <td>Filter display name</td>
  </tr>
  <tr>
  <td><code>type</code></td>
  <td><code>string</code></td>
  <td>Filter type (<code>text</code>, <code>date</code>, <code>number</code>)</td>
  </tr>
  <tr>
  <td><code>default</code></td>
  <td><code>string</code>, <code>number</code></td>
  <td>(optional) Filter default value</td>
  </tr>
  <tr>
  <td><code>min</code></td>
  <td><code>number</code></td>
  <td>(optional) Filter minimum value</td>
  </tr>
  <tr>
  <td><code>max</code></td>
  <td><code>number</code></td>
  <td>(optional) Filter maximum value</td>
  </tr>
  <tr>
  <td><code>step</code></td>
  <td><code>number</code></td>
  <td>(optional) Filter stepping value</td>
  </tr>
  </tbody>
  </table>
  <p dir="auto">Common chart properties for all chart types:</p>
  <table>
  <thead>
  <tr>
  <th>Property</th>
  <th>Type</th>
  <th>Description</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td><code>title</code></td>
  <td><code>string</code></td>
  <td>Chart title</td>
  </tr>
  <tr>
  <td><code>db</code></td>
  <td><code>string</code></td>
  <td>Database name against which to run the query</td>
  </tr>
  <tr>
  <td><code>query</code></td>
  <td><code>string</code></td>
  <td>SQL query to run and extract data from</td>
  </tr>
  <tr>
  <td><code>library</code></td>
  <td><code>string</code></td>
  <td>One of supported libraries: <code>vega</code>, <code>markdown</code>, <code>metric</code></td>
  </tr>
  <tr>
  <td><code>display</code></td>
  <td><code>object</code></td>
  <td>Chart display specification (depend on the used library)</td>
  </tr>
  </tbody>
  </table>
  <p dir="auto">To define SQL queries using dashboard filters:</p>
  <div class="highlight highlight-source-sql notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="SELECT * FROM mytable [[ WHERE col &gt;= :my_filter ]]"><pre><span class="pl-k">SELECT</span> <span class="pl-k">*</span> <span class="pl-k">FROM</span> mytable [[ <span class="pl-k">WHERE</span> col <span class="pl-k">&gt;=</span> :my_filter ]]</pre></div>
  <div class="highlight highlight-source-sql notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="SELECT * FROM mytable WHERE TRUE [[ AND col1 = :my_filter_1 ]] [[ AND col2 = :my_filter_2 ]]"><pre><span class="pl-k">SELECT</span> <span class="pl-k">*</span> <span class="pl-k">FROM</span> mytable <span class="pl-k">WHERE</span> TRUE [[ <span class="pl-k">AND</span> col1 <span class="pl-k">=</span> :my_filter_1 ]] [[ <span class="pl-k">AND</span> col2 <span class="pl-k">=</span> :my_filter_2 ]]</pre></div>
  <h4 dir="auto"><a id="user-content-vega-properties" class="anchor" aria-hidden="true" href="#user-content-vega-properties"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Vega properties</h4>
  <p dir="auto">Available configuration for <code>vega</code> charts:</p>
  <table>
  <thead>
  <tr>
  <th>Property</th>
  <th>Type</th>
  <th>Description</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td><code>library</code></td>
  <td><code>string</code></td>
  <td>Must be set to <code>vega</code></td>
  </tr>
  <tr>
  <td><code>display</code></td>
  <td><code>object</code></td>
  <td>Vega specification object</td>
  </tr>
  </tbody>
  </table>
  <p dir="auto">Notes about the <code>display</code> property:</p>
  <ul dir="auto">
  <li>Requires a valid <a href="https://vega.github.io/vega-lite/docs/" rel="nofollow">Vega specification object</a></li>
  <li>Some fields are pre-defined: <code>$schema</code>, <code>title</code>, <code>width</code>, <code>view</code>, <code>config</code>, <code>data</code></li>
  <li>All fields are passed along as-is (overriding pre-defined fields if any)</li>
  <li>Only <code>mark</code> and <code>encoding</code> fields are required as the bare-minimum</li>
  </ul>
  <h4 dir="auto"><a id="user-content-markdown-properties" class="anchor" aria-hidden="true" href="#user-content-markdown-properties"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Markdown properties</h4>
  <p dir="auto">Available configuration for <code>markdown</code> chart:</p>
  <table>
  <thead>
  <tr>
  <th>Property</th>
  <th>Type</th>
  <th>Description</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td><code>library</code></td>
  <td><code>string</code></td>
  <td>Must be set to <code>markdown</code></td>
  </tr>
  <tr>
  <td><code>display</code></td>
  <td><code>string</code></td>
  <td>Multi-line string containing the Markdown content</td>
  </tr>
  </tbody>
  </table>
  <p dir="auto">Note :</p>
  <ul dir="auto">
  <li>Some common properties do not apply and can be omitted: <code>title</code>, <code>db</code>, <code>query</code></li>
  <li>Markdown rendering is done by <a href="https://datasette.io/plugins/datasette-render-markdown" rel="nofollow"><code>datasette-render-markdown</code></a></li>
  <li>To configure Markdown rendering, extensions can be enabled in <a href="https://datasette.io/plugins/datasette-render-markdown#user-content-markdown-extensions" rel="nofollow">metadata</a></li>
  </ul>
  <h4 dir="auto"><a id="user-content-metric-properties" class="anchor" aria-hidden="true" href="#user-content-metric-properties"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Metric properties</h4>
  <p dir="auto">Available configuration for <code>metric</code> chart:</p>
  <table>
  <thead>
  <tr>
  <th>Property</th>
  <th>Type</th>
  <th>Description</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td><code>library</code></td>
  <td><code>string</code></td>
  <td>Must be set to <code>metric</code></td>
  </tr>
  <tr>
  <td><code>display.field</code></td>
  <td><code>string</code></td>
  <td>Numerical field to be displayed as metric</td>
  </tr>
  <tr>
  <td><code>display.prefix</code></td>
  <td><code>string</code></td>
  <td>Prefix to be displayed before metric</td>
  </tr>
  <tr>
  <td><code>display.suffix</code></td>
  <td><code>string</code></td>
  <td>Prefix to be displayed after metric</td>
  </tr>
  </tbody>
  </table>
  <p dir="auto">Note:</p>
  <ul dir="auto">
  <li>The <code>display.field</code> must reference a single-numerical value from the SQL query
  (e.g. numerical <code>number</code> field in <code>SELECT count(*) as number FROM events</code>)</li>
  </ul>
  <h3 dir="auto"><a id="user-content-dashboard-layout" class="anchor" aria-hidden="true" href="#user-content-dashboard-layout"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Dashboard layout</h3>
  <p dir="auto">The default dashboard layout will present two charts per row (one per row on mobile).
  To make use of custom dashboard layout using <a href="https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Grid_Layout" rel="nofollow">CSS Grid Layout</a>,
  define the <code>layout</code> array property as a grid / matrix:</p>
  <ul dir="auto">
  <li>Each entry represents a row of charts</li>
  <li>Each column is referring a chart by its property name</li>
  </ul>
  <h2 dir="auto"><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p dir="auto">To set up this plugin locally, first checkout the code.
  Then create a new virtual environment and the required dependencies:</p>
  <div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="pipenv install -d
  pipenv shell"><pre>pipenv install -d
  pipenv shell</pre></div>
  <p dir="auto">To run the tests:</p>
  <div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="pytest"><pre>pytest</pre></div>
  <h2 dir="auto"><a id="user-content-demo" class="anchor" aria-hidden="true" href="#user-content-demo"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Demo</h2>
  <p dir="auto">With the developmnent environment setup, you can run the demo locally:</p>
  <div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="datasette --metadata demo/metadata.yml demo/jobs.db"><pre>datasette --metadata demo/metadata.yml demo/jobs.db</pre></div>
  <h2 dir="auto"><a id="user-content-license" class="anchor" aria-hidden="true" href="#user-content-license"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>License</h2>
  <p dir="auto">Licensed under Apache License, Version 2.0</p>
  <p dir="auto">Copyright (c) 2021 - present Romain Clement</p>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkzNzQ4NDYzMTE=
name: datasette-geojson
full_name: eyeseast/datasette-geojson
html_url: https://github.com/eyeseast/datasette-geojson
description: Add GeoJSON output to Datasette queries
created_at: 2021-06-08T01:33:19Z
updated_at: 2022-02-16T19:59:42Z
pushed_at: 2022-02-16T20:02:49Z
language: Python
topics: ["datasette-io", "datasette-plugin", "geojson", "gis", "sqlite"]
default_branch: main
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
visibility: public
readme:
  # datasette-geojson

  [![PyPI](https://img.shields.io/pypi/v/datasette-geojson.svg)](https://pypi.org/project/datasette-geojson/)
  [![Changelog](https://img.shields.io/github/v/release/eyeseast/datasette-geojson?include_prereleases&label=changelog)](https://github.com/eyeseast/datasette-geojson/releases)
  [![Tests](https://github.com/eyeseast/datasette-geojson/workflows/Test/badge.svg)](https://github.com/eyeseast/datasette-geojson/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/eyeseast/datasette-geojson/blob/main/LICENSE)

  Add GeoJSON as an output option for datasette queries.

  ## Installation

  Install this plugin in the same environment as Datasette.

      datasette install datasette-geojson

  ## Usage

  To render GeoJSON, add a `.geojson` extension to any query URL that includes a `geometry` column. That column should be a valid [GeoJSON geometry](https://datatracker.ietf.org/doc/html/rfc7946#section-3.1).

  For example, you might use [geojson-to-sqlite](https://pypi.org/project/geojson-to-sqlite/) or [shapefile-to-sqlite](https://pypi.org/project/shapefile-to-sqlite/) to load [neighborhood boundaries](https://bostonopendata-boston.opendata.arcgis.com/datasets/3525b0ee6e6b427f9aab5d0a1d0a1a28_0/explore) into a SQLite database.

  ```sh
  wget -O neighborhoods.geojson https://opendata.arcgis.com/datasets/3525b0ee6e6b427f9aab5d0a1d0a1a28_0.geojson
  geojson-to-sqlite boston.db neighborhoods neighborhoods.geojson --spatial-index # create a spatial index
  datasette serve boston.db --load-extension spatialite
  ```

  If you're using Spatialite, the geometry column will be in a binary format. If not, make sure the `geometry` column is a well-formed [GeoJSON geometry](https://datatracker.ietf.org/doc/html/rfc7946#section-3.1). If you used `geojson-to-sqlite` or `shapefile-to-sqlite`, you should be all set.

  Run this query in Datasette and you'll see a link to download GeoJSON:

  ```sql
  select
    rowid,
    OBJECTID,
    Name,
    Acres,
    Neighborhood_ID,
    SqMiles,
    ShapeSTArea,
    ShapeSTLength,
    geometry
  from
    neighborhoods
  order by
    rowid
  limit
    101
  ```

  Note that the geometry column needs to be explicitly _named_ `geometry` or you won't get the option to export GeoJSON. If you want to use a different column, rename it with `AS`: `SELECT other AS geometry FROM my_table`.

  ![export geojson](img/export-options.png)

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-geojson
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-datasette-geojson" class="anchor" aria-hidden="true" href="#user-content-datasette-geojson"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-geojson</h1>
  <p dir="auto"><a href="https://pypi.org/project/datasette-geojson/" rel="nofollow"><img src="https://camo.githubusercontent.com/526d2a27eb7e19691a8f966dba069780f3fe109d6ac662d72901ff561f31b133/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d67656f6a736f6e2e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-geojson.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/eyeseast/datasette-geojson/releases"><img src="https://camo.githubusercontent.com/367e4f55f2b376367d7fbc8ab6ec5d765b3b04590149b3a8f60794cd39a5b54d/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f65796573656173742f6461746173657474652d67656f6a736f6e3f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/eyeseast/datasette-geojson?include_prereleases&amp;label=changelog" style="max-width: 100%;"></a>
  <a href="https://github.com/eyeseast/datasette-geojson/actions?query=workflow%3ATest"><img src="https://github.com/eyeseast/datasette-geojson/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a>
  <a href="https://github.com/eyeseast/datasette-geojson/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width: 100%;"></a></p>
  <p dir="auto">Add GeoJSON as an output option for datasette queries.</p>
  <h2 dir="auto"><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p dir="auto">Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="datasette install datasette-geojson"><pre><code>datasette install datasette-geojson
  </code></pre></div>
  <h2 dir="auto"><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p dir="auto">To render GeoJSON, add a <code>.geojson</code> extension to any query URL that includes a <code>geometry</code> column. That column should be a valid <a href="https://datatracker.ietf.org/doc/html/rfc7946#section-3.1" rel="nofollow">GeoJSON geometry</a>.</p>
  <p dir="auto">For example, you might use <a href="https://pypi.org/project/geojson-to-sqlite/" rel="nofollow">geojson-to-sqlite</a> or <a href="https://pypi.org/project/shapefile-to-sqlite/" rel="nofollow">shapefile-to-sqlite</a> to load <a href="https://bostonopendata-boston.opendata.arcgis.com/datasets/3525b0ee6e6b427f9aab5d0a1d0a1a28_0/explore" rel="nofollow">neighborhood boundaries</a> into a SQLite database.</p>
  <div class="highlight highlight-source-shell position-relative overflow-auto" data-snippet-clipboard-copy-content="wget -O neighborhoods.geojson https://opendata.arcgis.com/datasets/3525b0ee6e6b427f9aab5d0a1d0a1a28_0.geojson
  geojson-to-sqlite boston.db neighborhoods neighborhoods.geojson --spatial-index # create a spatial index
  datasette serve boston.db --load-extension spatialite"><pre>wget -O neighborhoods.geojson https://opendata.arcgis.com/datasets/3525b0ee6e6b427f9aab5d0a1d0a1a28_0.geojson
  geojson-to-sqlite boston.db neighborhoods neighborhoods.geojson --spatial-index <span class="pl-c"><span class="pl-c">#</span> create a spatial index</span>
  datasette serve boston.db --load-extension spatialite</pre></div>
  <p dir="auto">If you're using Spatialite, the geometry column will be in a binary format. If not, make sure the <code>geometry</code> column is a well-formed <a href="https://datatracker.ietf.org/doc/html/rfc7946#section-3.1" rel="nofollow">GeoJSON geometry</a>. If you used <code>geojson-to-sqlite</code> or <code>shapefile-to-sqlite</code>, you should be all set.</p>
  <p dir="auto">Run this query in Datasette and you'll see a link to download GeoJSON:</p>
  <div class="highlight highlight-source-sql position-relative overflow-auto" data-snippet-clipboard-copy-content="select
    rowid,
    OBJECTID,
    Name,
    Acres,
    Neighborhood_ID,
    SqMiles,
    ShapeSTArea,
    ShapeSTLength,
    geometry
  from
    neighborhoods
  order by
    rowid
  limit
    101"><pre><span class="pl-k">select</span>
    rowid,
    OBJECTID,
    Name,
    Acres,
    Neighborhood_ID,
    SqMiles,
    ShapeSTArea,
    ShapeSTLength,
    geometry
  <span class="pl-k">from</span>
    neighborhoods
  <span class="pl-k">order by</span>
    rowid
  <span class="pl-k">limit</span>
    <span class="pl-c1">101</span></pre></div>
  <p dir="auto">Note that the geometry column needs to be explicitly <em>named</em> <code>geometry</code> or you won't get the option to export GeoJSON. If you want to use a different column, rename it with <code>AS</code>: <code>SELECT other AS geometry FROM my_table</code>.</p>
  <p dir="auto"><a target="_blank" rel="noopener noreferrer" href="img/export-options.png"><img src="img/export-options.png" alt="export geojson" style="max-width: 100%;"></a></p>
  <h2 dir="auto"><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p dir="auto">To set up this plugin locally, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="cd datasette-geojson
  python3 -mvenv venv
  source venv/bin/activate"><pre><code>cd datasette-geojson
  python3 -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p dir="auto">Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pipenv shell"><pre><code>pipenv shell
  </code></pre></div>
  <p dir="auto">Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pip install -e '.[test]'"><pre><code>pip install -e '.[test]'
  </code></pre></div>
  <p dir="auto">To run the tests:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pytest"><pre><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkzNzU1NDY2NzU=
name: datasette-placekey
full_name: simonw/datasette-placekey
html_url: https://github.com/simonw/datasette-placekey
description: SQL functions for working with placekeys
created_at: 2021-06-10T02:31:27Z
updated_at: 2021-06-10T02:33:22Z
pushed_at: 2021-06-10T02:32:42Z
homepage: https://datasette.io/plugins/datasette-placekey
language: Python
topics: ["datasette", "datasette-plugin", "datasette-io", "placekey"]
default_branch: main
permissions: {"admin": false, "push": false, "pull": false}
readme:
  # datasette-placekey

  [![PyPI](https://img.shields.io/pypi/v/datasette-placekey.svg)](https://pypi.org/project/datasette-placekey/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-placekey?include_prereleases&label=changelog)](https://github.com/simonw/datasette-placekey/releases)
  [![Tests](https://github.com/simonw/datasette-placekey/workflows/Test/badge.svg)](https://github.com/simonw/datasette-placekey/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-placekey/blob/main/LICENSE)

  SQL functions for working with [placekeys](https://www.placekey.io/).

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-placekey

  ## Usage

  The following SQL functions are exposed - [documentation here](https://placekey.github.io/placekey-py/placekey.html#module-placekey.placekey).

  ```sql
  select
    geo_to_placekey(33.0896104,129.7900839),
    placekey_to_geo('@6nh-nhh-kvf'),
    placekey_to_geo_latitude('@6nh-nhh-kvf'),
    placekey_to_geo_longitude('@6nh-nhh-kvf'),
    placekey_to_h3('@6nh-nhh-kvf'),
    h3_to_placekey('8a30d94e4c87fff'),
    placekey_to_geojson('@6nh-nhh-kvf'),
    placekey_to_wkt('@6nh-nhh-kvf'),
    placekey_format_is_valid('@6nh-nhh-kvf');
  ```

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-placekey
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-datasette-placekey" class="anchor" aria-hidden="true" href="#user-content-datasette-placekey"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-placekey</h1>
  <p><a href="https://pypi.org/project/datasette-placekey/" rel="nofollow"><img src="https://camo.githubusercontent.com/a570339480e04ec2eedd623479156b479b30fafcd8fa2ff60250a22ee4e77ed2/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d706c6163656b65792e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-placekey.svg" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-placekey/releases"><img src="https://camo.githubusercontent.com/120db4c1d380188526a62cbe06a80176d5be627da0b962b724e32422291df31c/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d706c6163656b65793f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-placekey?include_prereleases&amp;label=changelog" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-placekey/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-placekey/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a>
  <a href="https://github.com/simonw/datasette-placekey/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p>
  <p>SQL functions for working with <a href="https://www.placekey.io/" rel="nofollow">placekeys</a>.</p>
  <h2><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p>Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ datasette install datasette-placekey
  "><pre><code>$ datasette install datasette-placekey
  </code></pre></div>
  <h2><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p>The following SQL functions are exposed - <a href="https://placekey.github.io/placekey-py/placekey.html#module-placekey.placekey" rel="nofollow">documentation here</a>.</p>
  <div class="highlight highlight-source-sql position-relative" data-snippet-clipboard-copy-content="select
    geo_to_placekey(33.0896104,129.7900839),
    placekey_to_geo('@6nh-nhh-kvf'),
    placekey_to_geo_latitude('@6nh-nhh-kvf'),
    placekey_to_geo_longitude('@6nh-nhh-kvf'),
    placekey_to_h3('@6nh-nhh-kvf'),
    h3_to_placekey('8a30d94e4c87fff'),
    placekey_to_geojson('@6nh-nhh-kvf'),
    placekey_to_wkt('@6nh-nhh-kvf'),
    placekey_format_is_valid('@6nh-nhh-kvf');
  "><pre><span class="pl-k">select</span>
    geo_to_placekey(<span class="pl-c1">33</span>.<span class="pl-c1">0896104</span>,<span class="pl-c1">129</span>.<span class="pl-c1">7900839</span>),
    placekey_to_geo(<span class="pl-s"><span class="pl-pds">'</span>@6nh-nhh-kvf<span class="pl-pds">'</span></span>),
    placekey_to_geo_latitude(<span class="pl-s"><span class="pl-pds">'</span>@6nh-nhh-kvf<span class="pl-pds">'</span></span>),
    placekey_to_geo_longitude(<span class="pl-s"><span class="pl-pds">'</span>@6nh-nhh-kvf<span class="pl-pds">'</span></span>),
    placekey_to_h3(<span class="pl-s"><span class="pl-pds">'</span>@6nh-nhh-kvf<span class="pl-pds">'</span></span>),
    h3_to_placekey(<span class="pl-s"><span class="pl-pds">'</span>8a30d94e4c87fff<span class="pl-pds">'</span></span>),
    placekey_to_geojson(<span class="pl-s"><span class="pl-pds">'</span>@6nh-nhh-kvf<span class="pl-pds">'</span></span>),
    placekey_to_wkt(<span class="pl-s"><span class="pl-pds">'</span>@6nh-nhh-kvf<span class="pl-pds">'</span></span>),
    placekey_format_is_valid(<span class="pl-s"><span class="pl-pds">'</span>@6nh-nhh-kvf<span class="pl-pds">'</span></span>);</pre></div>
  <h2><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p>To set up this plugin locally, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="cd datasette-placekey
  python3 -mvenv venv
  source venv/bin/activate
  "><pre><code>cd datasette-placekey
  python3 -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p>Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pipenv shell
  "><pre><code>pipenv shell
  </code></pre></div>
  <p>Now install the dependencies and tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pip install -e '.[test]'
  "><pre><code>pip install -e '.[test]'
  </code></pre></div>
  <p>To run the tests:</p>
  <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="pytest
  "><pre><code>pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkzODI5ODY1NjQ=
name: datasette-geojson-map
full_name: eyeseast/datasette-geojson-map
html_url: https://github.com/eyeseast/datasette-geojson-map
description: Render a map for any query with a geometry column
created_at: 2021-07-05T01:54:13Z
updated_at: 2022-03-04T00:16:17Z
pushed_at: 2022-04-27T20:39:47Z
language: Python
topics: ["datasette-io", "datasette-plugin", "geojson", "gis", "leafletjs", "mapping"]
default_branch: main
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
visibility: public
readme:
  # datasette-geojson-map

  [![PyPI](https://img.shields.io/pypi/v/datasette-geojson-map.svg)](https://pypi.org/project/datasette-geojson-map/)
  [![Changelog](https://img.shields.io/github/v/release/eyeseast/datasette-geojson-map?include_prereleases&label=changelog)](https://github.com/eyeseast/datasette-geojson-map/releases)
  [![Tests](https://github.com/eyeseast/datasette-geojson-map/workflows/Test/badge.svg)](https://github.com/eyeseast/datasette-geojson-map/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/eyeseast/datasette-geojson-map/blob/main/LICENSE)

  Render a map for any query with a geometry column

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-geojson-map

  ## Usage

  Start by loading a GIS file.

  For example, you might use [geojson-to-sqlite](https://pypi.org/project/geojson-to-sqlite/) or [shapefile-to-sqlite](https://pypi.org/project/shapefile-to-sqlite/) to load [neighborhood boundaries](https://bostonopendata-boston.opendata.arcgis.com/datasets/3525b0ee6e6b427f9aab5d0a1d0a1a28_0/explore) into a SQLite database.

  ```sh
  wget -O neighborhoods.geojson https://opendata.arcgis.com/datasets/3525b0ee6e6b427f9aab5d0a1d0a1a28_0.geojson
  geojson-to-sqlite boston.db neighborhoods neighborhoods.geojson
  ```

  (The command above uses Spatialite, but that's not required.)

  Start up `datasette` and navigate to the `neighborhoods` table.

  ```sh
  datasette serve boston.db

  # in another terminal tab
  open http://localhost:8001/boston/neighborhoods
  ```

  You should see a map centered on Boston with each neighborhood outlined. Clicking a boundary will bring up a popup with details on that feature.

  ![Boston neighbhorhoods map](img/boston-neighborhoods-map.png)

  This plugin relies on (and will install) [datasette-geojson](https://github.com/eyeseast/datasette-geojson). Any query that includes a `geometry` column will produce a map of the results. This also includes single row views.

  Run the included `demo` project to see it live.

  ## Configuration

  This project uses the same map configuration as [datasette-cluster-map](https://github.com/simonw/datasette-cluster-map). Here's how you would use [Stamen's terrain tiles](http://maps.stamen.com/terrain/#12/37.7706/-122.3782):

  ```yaml
  plugins:
    datasette-geojson-map:
      tile_layer: https://stamen-tiles-{s}.a.ssl.fastly.net/terrain/{z}/{x}/{y}.{ext}
      tile_layer_options:
        attribution: >-
          Map tiles by <a href="http://stamen.com">Stamen Design</a>, 
          under <a href="http://creativecommons.org/licenses/by/3.0">CC BY 3.0</a>. 
          Data by <a href="http://openstreetmap.org">OpenStreetMap</a>, 
          under <a href="http://www.openstreetmap.org/copyright">ODbL</a>.
        subdomains: abcd
        minZoom: 1
        maxZoom: 16
        ext: jpg
  ```

  Options:

  - `tile_layer`: Use a URL template that can be passed to a [Leaflet Tilelayer](https://leafletjs.com/reference-1.7.1.html#tilelayer)
  - `tile_layer_options`: All options will be passed to the tile layer. See [Leaflet documentation](https://leafletjs.com/reference-1.7.1.html#tilelayer) for more on possible values here.

  ## Styling map features

  Map features can be styled using the [simplestyle-spec](https://github.com/mapbox/simplestyle-spec). This requires setting specific fields on returned rows. Here's an example:

  ```sql
  SELECT Name, geometry, "#ff0000" as fill, "#0000ff" as stroke, 0.2 as stroke-width,  from neighborhoods
  ```

  That will render a neighborhood map where each polygon is filled in red, outlined in blue and lines are 0.2 pixels wide.

  A more useful approach would use the `CASE` statement to color features based on data:

  ```sql
  SELECT
    Name,
    geometry,
    CASE
      Name
      WHEN "Roslindale" THEN "#ff0000"
      WHEN "Dorchester" THEN "#0000ff"
      ELSE "#dddddd"
    END fill
  FROM
    neighborhoods
  ```

  This will fill Roslindale in red, Dorchester in blue and all other neighborhoods in gray.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-geojson-map
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-datasette-geojson-map" class="anchor" aria-hidden="true" href="#user-content-datasette-geojson-map"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-geojson-map</h1>
  <p dir="auto"><a href="https://pypi.org/project/datasette-geojson-map/" rel="nofollow"><img src="https://camo.githubusercontent.com/1534879e09b364c34ab31432e489c7be09c28cdc94c68a85f1ffefc9b2a0f8df/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d67656f6a736f6e2d6d61702e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-geojson-map.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/eyeseast/datasette-geojson-map/releases"><img src="https://camo.githubusercontent.com/d45ff62474954cad3f625bc2d1e121eec6e3b2797d55917af6419fb819b45cef/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f65796573656173742f6461746173657474652d67656f6a736f6e2d6d61703f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/eyeseast/datasette-geojson-map?include_prereleases&amp;label=changelog" style="max-width: 100%;"></a>
  <a href="https://github.com/eyeseast/datasette-geojson-map/actions?query=workflow%3ATest"><img src="https://github.com/eyeseast/datasette-geojson-map/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a>
  <a href="https://github.com/eyeseast/datasette-geojson-map/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width: 100%;"></a></p>
  <p dir="auto">Render a map for any query with a geometry column</p>
  <h2 dir="auto"><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p dir="auto">Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="$ datasette install datasette-geojson-map"><pre class="notranslate"><code class="notranslate">$ datasette install datasette-geojson-map
  </code></pre></div>
  <h2 dir="auto"><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p dir="auto">Start by loading a GIS file.</p>
  <p dir="auto">For example, you might use <a href="https://pypi.org/project/geojson-to-sqlite/" rel="nofollow">geojson-to-sqlite</a> or <a href="https://pypi.org/project/shapefile-to-sqlite/" rel="nofollow">shapefile-to-sqlite</a> to load <a href="https://bostonopendata-boston.opendata.arcgis.com/datasets/3525b0ee6e6b427f9aab5d0a1d0a1a28_0/explore" rel="nofollow">neighborhood boundaries</a> into a SQLite database.</p>
  <div class="highlight highlight-source-shell position-relative overflow-auto" data-snippet-clipboard-copy-content="wget -O neighborhoods.geojson https://opendata.arcgis.com/datasets/3525b0ee6e6b427f9aab5d0a1d0a1a28_0.geojson
  geojson-to-sqlite boston.db neighborhoods neighborhoods.geojson"><pre>wget -O neighborhoods.geojson https://opendata.arcgis.com/datasets/3525b0ee6e6b427f9aab5d0a1d0a1a28_0.geojson
  geojson-to-sqlite boston.db neighborhoods neighborhoods.geojson</pre></div>
  <p dir="auto">(The command above uses Spatialite, but that's not required.)</p>
  <p dir="auto">Start up <code>datasette</code> and navigate to the <code>neighborhoods</code> table.</p>
  <div class="highlight highlight-source-shell position-relative overflow-auto" data-snippet-clipboard-copy-content="datasette serve boston.db

  # in another terminal tab
  open http://localhost:8001/boston/neighborhoods"><pre>datasette serve boston.db

  <span class="pl-c"><span class="pl-c">#</span> in another terminal tab</span>
  open http://localhost:8001/boston/neighborhoods</pre></div>
  <p dir="auto">You should see a map centered on Boston with each neighborhood outlined. Clicking a boundary will bring up a popup with details on that feature.</p>
  <p dir="auto"><a target="_blank" rel="noopener noreferrer" href="img/boston-neighborhoods-map.png"><img src="img/boston-neighborhoods-map.png" alt="Boston neighbhorhoods map" style="max-width: 100%;"></a></p>
  <p dir="auto">This plugin relies on (and will install) <a href="https://github.com/eyeseast/datasette-geojson">datasette-geojson</a>. Any query that includes a <code>geometry</code> column will produce a map of the results. This also includes single row views.</p>
  <p dir="auto">Run the included <code>demo</code> project to see it live.</p>
  <h2 dir="auto"><a id="user-content-configuration" class="anchor" aria-hidden="true" href="#user-content-configuration"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Configuration</h2>
  <p dir="auto">This project uses the same map configuration as <a href="https://github.com/simonw/datasette-cluster-map">datasette-cluster-map</a>. Here's how you would use <a href="http://maps.stamen.com/terrain/#12/37.7706/-122.3782" rel="nofollow">Stamen's terrain tiles</a>:</p>
  <div class="highlight highlight-source-yaml position-relative overflow-auto" data-snippet-clipboard-copy-content="plugins:
    datasette-geojson-map:
      tile_layer: https://stamen-tiles-{s}.a.ssl.fastly.net/terrain/{z}/{x}/{y}.{ext}
      tile_layer_options:
        attribution: &gt;-
          Map tiles by &lt;a href=&quot;http://stamen.com&quot;&gt;Stamen Design&lt;/a&gt;, 
          under &lt;a href=&quot;http://creativecommons.org/licenses/by/3.0&quot;&gt;CC BY 3.0&lt;/a&gt;. 
          Data by &lt;a href=&quot;http://openstreetmap.org&quot;&gt;OpenStreetMap&lt;/a&gt;, 
          under &lt;a href=&quot;http://www.openstreetmap.org/copyright&quot;&gt;ODbL&lt;/a&gt;.
        subdomains: abcd
        minZoom: 1
        maxZoom: 16
        ext: jpg"><pre><span class="pl-ent">plugins</span>:
    <span class="pl-ent">datasette-geojson-map</span>:
      <span class="pl-ent">tile_layer</span>: <span class="pl-s">https://stamen-tiles-{s}.a.ssl.fastly.net/terrain/{z}/{x}/{y}.{ext}</span>
      <span class="pl-ent">tile_layer_options</span>:
        <span class="pl-ent">attribution</span>: <span class="pl-s">&gt;-</span>
  <span class="pl-s">        Map tiles by &lt;a href="http://stamen.com"&gt;Stamen Design&lt;/a&gt;, </span>
  <span class="pl-s">        under &lt;a href="http://creativecommons.org/licenses/by/3.0"&gt;CC BY 3.0&lt;/a&gt;. </span>
  <span class="pl-s">        Data by &lt;a href="http://openstreetmap.org"&gt;OpenStreetMap&lt;/a&gt;, </span>
  <span class="pl-s">        under &lt;a href="http://www.openstreetmap.org/copyright"&gt;ODbL&lt;/a&gt;.</span>
  <span class="pl-s"></span>      <span class="pl-ent">subdomains</span>: <span class="pl-s">abcd</span>
        <span class="pl-ent">minZoom</span>: <span class="pl-c1">1</span>
        <span class="pl-ent">maxZoom</span>: <span class="pl-c1">16</span>
        <span class="pl-ent">ext</span>: <span class="pl-s">jpg</span></pre></div>
  <p dir="auto">Options:</p>
  <ul dir="auto">
  <li><code>tile_layer</code>: Use a URL template that can be passed to a <a href="https://leafletjs.com/reference-1.7.1.html#tilelayer" rel="nofollow">Leaflet Tilelayer</a></li>
  <li><code>tile_layer_options</code>: All options will be passed to the tile layer. See <a href="https://leafletjs.com/reference-1.7.1.html#tilelayer" rel="nofollow">Leaflet documentation</a> for more on possible values here.</li>
  </ul>
  <h2 dir="auto"><a id="user-content-styling-map-features" class="anchor" aria-hidden="true" href="#user-content-styling-map-features"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Styling map features</h2>
  <p dir="auto">Map features can be styled using the <a href="https://github.com/mapbox/simplestyle-spec">simplestyle-spec</a>. This requires setting specific fields on returned rows. Here's an example:</p>
  <div class="highlight highlight-source-sql position-relative overflow-auto" data-snippet-clipboard-copy-content="SELECT Name, geometry, &quot;#ff0000&quot; as fill, &quot;#0000ff&quot; as stroke, 0.2 as stroke-width,  from neighborhoods"><pre><span class="pl-k">SELECT</span> Name, geometry, <span class="pl-s"><span class="pl-pds">"</span>#ff0000<span class="pl-pds">"</span></span> <span class="pl-k">as</span> fill, <span class="pl-s"><span class="pl-pds">"</span>#0000ff<span class="pl-pds">"</span></span> <span class="pl-k">as</span> stroke, <span class="pl-c1">0</span>.<span class="pl-c1">2</span> <span class="pl-k">as</span> stroke<span class="pl-k">-</span>width,  <span class="pl-k">from</span> neighborhoods</pre></div>
  <p dir="auto">That will render a neighborhood map where each polygon is filled in red, outlined in blue and lines are 0.2 pixels wide.</p>
  <p dir="auto">A more useful approach would use the <code>CASE</code> statement to color features based on data:</p>
  <div class="highlight highlight-source-sql position-relative overflow-auto" data-snippet-clipboard-copy-content="SELECT
    Name,
    geometry,
    CASE
      Name
      WHEN &quot;Roslindale&quot; THEN &quot;#ff0000&quot;
      WHEN &quot;Dorchester&quot; THEN &quot;#0000ff&quot;
      ELSE &quot;#dddddd&quot;
    END fill
  FROM
    neighborhoods"><pre><span class="pl-k">SELECT</span>
    Name,
    geometry,
    CASE
      Name
      WHEN <span class="pl-s"><span class="pl-pds">"</span>Roslindale<span class="pl-pds">"</span></span> THEN <span class="pl-s"><span class="pl-pds">"</span>#ff0000<span class="pl-pds">"</span></span>
      WHEN <span class="pl-s"><span class="pl-pds">"</span>Dorchester<span class="pl-pds">"</span></span> THEN <span class="pl-s"><span class="pl-pds">"</span>#0000ff<span class="pl-pds">"</span></span>
      ELSE <span class="pl-s"><span class="pl-pds">"</span>#dddddd<span class="pl-pds">"</span></span>
    END fill
  <span class="pl-k">FROM</span>
    neighborhoods</pre></div>
  <p dir="auto">This will fill Roslindale in red, Dorchester in blue and all other neighborhoods in gray.</p>
  <h2 dir="auto"><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p dir="auto">To set up this plugin locally, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="cd datasette-geojson-map
  python3 -mvenv venv
  source venv/bin/activate"><pre class="notranslate"><code class="notranslate">cd datasette-geojson-map
  python3 -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p dir="auto">Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pipenv shell"><pre class="notranslate"><code class="notranslate">pipenv shell
  </code></pre></div>
  <p dir="auto">Now install the dependencies and test dependencies:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pip install -e '.[test]'"><pre class="notranslate"><code class="notranslate">pip install -e '.[test]'
  </code></pre></div>
  <p dir="auto">To run the tests:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pytest"><pre class="notranslate"><code class="notranslate">pytest
  </code></pre></div>
  </article></div>


node_id: MDEwOlJlcG9zaXRvcnkzOTA1MzU1MDA=
name: datasette-remote-metadata
full_name: simonw/datasette-remote-metadata
html_url: https://github.com/simonw/datasette-remote-metadata
description: Periodically refresh Datasette metadata from a remote URL
created_at: 2021-07-28T23:17:19Z
updated_at: 2021-12-13T19:40:51Z
pushed_at: 2021-12-13T19:40:48Z
language: Python
license: apache-2.0
topics: []
default_branch: main
permissions: {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false}
visibility: public
readme:
  # datasette-remote-metadata

  [![PyPI](https://img.shields.io/pypi/v/datasette-remote-metadata.svg)](https://pypi.org/project/datasette-remote-metadata/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-remote-metadata?include_prereleases&label=changelog)](https://github.com/simonw/datasette-remote-metadata/releases)
  [![Tests](https://github.com/simonw/datasette-remote-metadata/workflows/Test/badge.svg)](https://github.com/simonw/datasette-remote-metadata/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-remote-metadata/blob/main/LICENSE)

  Periodically refresh Datasette metadata from a remote URL

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-remote-metadata

  ## Usage

  Add the following to your `metadata.json`:

  ```json
  {
      "plugins": {
          "datasette-remote-metadata": {
              "url": "https://example.com/remote-metadata.yml"
          }
      }
  }
  ```
  The plugin will fetch the specified metadata  from that URL at startup and combine it with any existing metadata. You can use a URL to either a JSON file or a YAML file.

  It will periodically refresh that metadata - by default every 30 seconds, unless you specify an alternative `"ttl"` value in the plugin configuration.

  ## Configuration

  Available configuration options are as follows:

  - `"url"` - the URL to retrieve remote metadata from. Can link to a JSON or a YAML file.
  - `"ttl"` - integer value in secords: how frequently should the script check for fresh metadata. Defaults to 30 seconds.
  - `"headers"` - a dictionary of additional request headers to send.
  - `"cachebust"` - if true, a random `?0.29508` value will be added to the query string of the remote metadata to bust any intermediary caches.

  This example `metadata.json` configuration refreshes every 10 seconds, uses cache busting and sends an `Authorization: Bearer xyz` header with the request:

  ```json
  {
      "plugins": {
          "datasette-remote-metadata": {
              "url": "https://example.com/remote-metadata.yml",
              "ttl": 10,
              "cachebust": true,
              "headers": {
                  "Authorization": "Bearer xyz"
              }
          }
      }
  }
  ```
  This example if you are using `metadata.yaml` for configuration:
  ```yaml
  plugins:
    datasette-remote-metadata:
      url: https://example.com/remote-metadata.yml
      ttl: 10
      cachebust: true
      headers:
        Authorization: Bearer xyz
  ```

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-remote-metadata
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

readme_html:
  <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-datasette-remote-metadata" class="anchor" aria-hidden="true" href="#user-content-datasette-remote-metadata"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-remote-metadata</h1>
  <p dir="auto"><a href="https://pypi.org/project/datasette-remote-metadata/" rel="nofollow"><img src="https://camo.githubusercontent.com/88715eda95713112087b759d9348f413cf7421760dbf9dc75a891f603077bb02/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d72656d6f74652d6d657461646174612e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-remote-metadata.svg" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-remote-metadata/releases"><img src="https://camo.githubusercontent.com/81743aaae3ae2fa6d977f38bae18d79b191a53725378e4a945833f29a7d75bb1/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d72656d6f74652d6d657461646174613f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-remote-metadata?include_prereleases&amp;label=changelog" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-remote-metadata/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-remote-metadata/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a>
  <a href="https://github.com/simonw/datasette-remote-metadata/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width: 100%;"></a></p>
  <p dir="auto">Periodically refresh Datasette metadata from a remote URL</p>
  <h2 dir="auto"><a id="user-content-installation" class="anchor" aria-hidden="true" href="#user-content-installation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Installation</h2>
  <p dir="auto">Install this plugin in the same environment as Datasette.</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="$ datasette install datasette-remote-metadata"><pre><code>$ datasette install datasette-remote-metadata
  </code></pre></div>
  <h2 dir="auto"><a id="user-content-usage" class="anchor" aria-hidden="true" href="#user-content-usage"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Usage</h2>
  <p dir="auto">Add the following to your <code>metadata.json</code>:</p>
  <div class="highlight highlight-source-json position-relative overflow-auto" data-snippet-clipboard-copy-content="{
      &quot;plugins&quot;: {
          &quot;datasette-remote-metadata&quot;: {
              &quot;url&quot;: &quot;https://example.com/remote-metadata.yml&quot;
          }
      }
  }"><pre>{
      <span class="pl-ent">"plugins"</span>: {
          <span class="pl-ent">"datasette-remote-metadata"</span>: {
              <span class="pl-ent">"url"</span>: <span class="pl-s"><span class="pl-pds">"</span>https://example.com/remote-metadata.yml<span class="pl-pds">"</span></span>
          }
      }
  }</pre></div>
  <p dir="auto">The plugin will fetch the specified metadata  from that URL at startup and combine it with any existing metadata. You can use a URL to either a JSON file or a YAML file.</p>
  <p dir="auto">It will periodically refresh that metadata - by default every 30 seconds, unless you specify an alternative <code>"ttl"</code> value in the plugin configuration.</p>
  <h2 dir="auto"><a id="user-content-configuration" class="anchor" aria-hidden="true" href="#user-content-configuration"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Configuration</h2>
  <p dir="auto">Available configuration options are as follows:</p>
  <ul dir="auto">
  <li><code>"url"</code> - the URL to retrieve remote metadata from. Can link to a JSON or a YAML file.</li>
  <li><code>"ttl"</code> - integer value in secords: how frequently should the script check for fresh metadata. Defaults to 30 seconds.</li>
  <li><code>"headers"</code> - a dictionary of additional request headers to send.</li>
  <li><code>"cachebust"</code> - if true, a random <code>?0.29508</code> value will be added to the query string of the remote metadata to bust any intermediary caches.</li>
  </ul>
  <p dir="auto">This example <code>metadata.json</code> configuration refreshes every 10 seconds, uses cache busting and sends an <code>Authorization: Bearer xyz</code> header with the request:</p>
  <div class="highlight highlight-source-json position-relative overflow-auto" data-snippet-clipboard-copy-content="{
      &quot;plugins&quot;: {
          &quot;datasette-remote-metadata&quot;: {
              &quot;url&quot;: &quot;https://example.com/remote-metadata.yml&quot;,
              &quot;ttl&quot;: 10,
              &quot;cachebust&quot;: true,
              &quot;headers&quot;: {
                  &quot;Authorization&quot;: &quot;Bearer xyz&quot;
              }
          }
      }
  }"><pre>{
      <span class="pl-ent">"plugins"</span>: {
          <span class="pl-ent">"datasette-remote-metadata"</span>: {
              <span class="pl-ent">"url"</span>: <span class="pl-s"><span class="pl-pds">"</span>https://example.com/remote-metadata.yml<span class="pl-pds">"</span></span>,
              <span class="pl-ent">"ttl"</span>: <span class="pl-c1">10</span>,
              <span class="pl-ent">"cachebust"</span>: <span class="pl-c1">true</span>,
              <span class="pl-ent">"headers"</span>: {
                  <span class="pl-ent">"Authorization"</span>: <span class="pl-s"><span class="pl-pds">"</span>Bearer xyz<span class="pl-pds">"</span></span>
              }
          }
      }
  }</pre></div>
  <p dir="auto">This example if you are using <code>metadata.yaml</code> for configuration:</p>
  <div class="highlight highlight-source-yaml position-relative overflow-auto" data-snippet-clipboard-copy-content="plugins:
    datasette-remote-metadata:
      url: https://example.com/remote-metadata.yml
      ttl: 10
      cachebust: true
      headers:
        Authorization: Bearer xyz"><pre><span class="pl-ent">plugins</span>:
    <span class="pl-ent">datasette-remote-metadata</span>:
      <span class="pl-ent">url</span>: <span class="pl-s">https://example.com/remote-metadata.yml</span>
      <span class="pl-ent">ttl</span>: <span class="pl-c1">10</span>
      <span class="pl-ent">cachebust</span>: <span class="pl-c1">true</span>
      <span class="pl-ent">headers</span>:
        <span class="pl-ent">Authorization</span>: <span class="pl-s">Bearer xyz</span></pre></div>
  <h2 dir="auto"><a id="user-content-development" class="anchor" aria-hidden="true" href="#user-content-development"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Development</h2>
  <p dir="auto">To set up this plugin locally, first checkout the code. Then create a new virtual environment:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="cd datasette-remote-metadata
  python3 -mvenv venv
  source venv/bin/activate"><pre><code>cd datasette-remote-metadata
  python3 -mvenv venv
  source venv/bin/activate
  </code></pre></div>
  <p dir="auto">Or if you are using <code>pipenv</code>:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pipenv shell"><pre><code>pipenv shell
  </code></pre></div>
  <p dir="auto">Now install the dependencies and test dependencies:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pip install -e '.[test]'"><pre><code>pip install -e '.[test]'
  </code></pre></div>
  <p dir="auto">To run the tests:</p>
  <div class="snippet-clipboard-content position-relative overflow-auto" data-snippet-clipboard-copy-content="pytest"><pre><code>pytest
  </code></pre></div>
  </article></div>


