# releases

html_url: https://github.com/simonw/shot-scraper/releases/tag/0.1
node_id: RE_kwDOG-A5W84DqR9l
tag_name: 0.1
target_commitish: main
name: 0.1
created_at: 2022-03-09T19:13:44Z
published_at: 2022-03-09T19:16:43Z
body:
  - Switched from npm Playwright to [Python Playwright](https://pypi.org/project/playwright/). [#3](https://github.com/simonw/shot-scraper/issues/3)
  - New `shot-scraper install` command for installing the browser needed by Playwright. [#6](https://github.com/simonw/shot-scraper/issues/6)
  - New `shot-scraper shot URL` command (also the default if you just run `shot-scraper ...`) which takes a single screenshot. [#5](https://github.com/simonw/shot-scraper/issues/5)
  - `shot-scraper multi shots.yml` command now executes the YAML file with a list of shots in it.


html_url: https://github.com/simonw/shot-scraper/releases/tag/0.2
node_id: RE_kwDOG-A5W84DqUf7
tag_name: 0.2
target_commitish: main
name: 0.2
created_at: 2022-03-09T21:05:31Z
published_at: 2022-03-09T21:05:41Z
body:
  - `shot-scraper --selector SELECTOR` option to specify an element on the page using a CSS selector and take a screenshot of just that element. [#8](https://github.com/simonw/shot-scraper/issues/8)
  - `selector: ...` key in YAML file to specify an element by CSS selector.
  - `--javascript SCRIPT` option to specify custom JavaScript to be executed after the page has loaded but before the screenshot is taken. [#12](https://github.com/simonw/shot-scraper/issues/12)
  - `javascript:` key in YAML to specify JavaScript to execute.
  - `--width` and `--height` options to set the width and height of the browser window used for the screenshot. If a height is specified, the resulting screenshot will be that height rather than being the full height of the page. [#13](https://github.com/simonw/shot-scraper/issues/13)
  - Equivalent `width:` and `height:` keys in the YAML configuration.


html_url: https://github.com/simonw/shot-scraper/releases/tag/0.3
node_id: RE_kwDOG-A5W84DqYBD
tag_name: 0.3
target_commitish: main
name: 0.3
created_at: 2022-03-09T22:22:57Z
published_at: 2022-03-09T22:24:53Z
body:
  - Added a live demo, in the [shot-scraper-demo](https://github.com/simonw/shot-scraper-demo) repository. [#14](https://github.com/simonw/shot-scraper/issuse/14)
  - New `--quality 80` option for outputting smaller JPEG images with the specified quality. [#15](https://github.com/simonw/shot-scraper/issuse/15)
  - New `--wait 2000` option for waiting the specified number of milliseconds before taking the screenshot. [#16](https://github.com/simonw/shot-scraper/issuse/16)


html_url: https://github.com/simonw/shot-scraper/releases/tag/0.4
node_id: RE_kwDOG-A5W84Dqx7v
tag_name: 0.4
target_commitish: main
name: 0.4
created_at: 2022-03-10T22:33:48Z
published_at: 2022-03-10T22:33:51Z
body:
  - Added `shot-scraper accessibility URL` command, which dumps out a JSON copy of the Chromium accessibility tree for the page. [#22](https://github.com/simonw/shot-scraper/issues/22)
  - Fixed error in the `--help` output for the `shot-scraper multi` command.


html_url: https://github.com/eyeseast/geocode-sqlite/releases/tag/v0.6.0
node_id: RE_kwDOEXxXas4DrEJU
tag_name: v0.6.0
target_commitish: main
name: GeoJSON and SpatiaLite support
created_at: 2022-03-11T16:57:57Z
published_at: 2022-03-11T16:58:49Z
body:
  ## What's Changed
  * GeoJSON and SpatiaLite support by @eyeseast in https://github.com/eyeseast/geocode-sqlite/pull/23


  **Full Changelog**: https://github.com/eyeseast/geocode-sqlite/compare/v0.5.2...v0.6.0


html_url: https://github.com/simonw/sqlite-utils/releases/tag/3.25.1
node_id: RE_kwDOCGYnMM4DrGEG
tag_name: 3.25.1
target_commitish: main
name: 3.25.1
created_at: 2022-03-11T18:34:44Z
published_at: 2022-03-11T18:35:41Z
body:
  - Improved display of type information and parameters in the [API reference documentation](https://sqlite-utils.datasette.io/en/stable/reference.html). [#413](https://github.com/simonw/sqlite-utils/issues/413)


html_url: https://github.com/simonw/shot-scraper/releases/tag/0.5
node_id: RE_kwDOG-A5W84DrS4K
tag_name: 0.5
target_commitish: main
name: 0.5
created_at: 2022-03-12T19:10:11Z
published_at: 2022-03-12T19:17:00Z
body:
  - New `shot-scraper pdf` command for creating a PDF export of a web page. [#24](https://github.com/simonw/shot-scraper/issues/24)
  - `shot-scraper accessibility --javascript` option for executing custom JavaScript before taking the accessibility snapshot. [#23](https://github.com/simonw/shot-scraper/issues/23)
  - `shot-scraper accessibility -o filename.json` option. [#25](https://github.com/simonw/shot-scraper/issues/25)
  - README demos section now links to `@newshomepages` Twitter bot by [@palewire](https://github.com/palewire)
  - README now includes tips on executing JavaScript. [#20](https://github.com/simonw/shot-scraper/issues/20)
  - README now includes the `--help` output of the various commands.


html_url: https://github.com/simonw/shot-scraper/releases/tag/0.6
node_id: RE_kwDOG-A5W84DrTu_
tag_name: 0.6
target_commitish: main
name: 0.6
created_at: 2022-03-12T21:29:07Z
published_at: 2022-03-12T21:30:32Z
body:
  - Now supports taking screenshots of **pages that require authentication**. [#18](https://github.com/simonw/shot-scraper/issues/18)

    The following command will open a browser window for the specified website, wait for you to manually authenticate and hit `<enter>` in the terminal, and then write the resulting authentication context out to `auth.json`:

        shot-scraper auth https://github.com/ auth.json`

    You can then take authenticated screenshots like this:

        shot-scraper https://github.com/notifications \
          --auth auth.json -o notifications.png

    The `-a/--auth` option is also supported by the `multi`, `pdf` and `accessibility` commands.

  - The `shot-scraper` command can now open a browser in which you can **interact with a page** before the screenshot is taken: [#31](https://github.com/simonw/shot-scraper/issues/31)

        shot-scraper https://simonwillison.net/ \
          -o after-interaction.png \
          --height 800 --interactive

    This will output:

        Hit <enter> to take the shot and close the browser window:
          # And after you hit <enter>...
        Screenshot of 'https://simonwillison.net/' written to 'after-interaction.png'

  - You can now pass **multiple CSS selectors** in order to take a screenshot of the smallest area that encompasses all of the content referenced by those selectors: [#21](https://github.com/simonw/shot-scraper/issues/21)

        shot-scraper https://simonwillison.net/ \
          -s '#bighead' -s .overband \
          -o bighead-multi-selector.png

    Add `--padding 20` to include an additional 20px of padding around the specified area.

    The YAML format used by `snap-shotter multi` also now supports multiple CSS selectors, which look like this:

    ```yaml
    - output: bighead-multi-selector.png
      url: https://simonwillison.net/
      selectors:
      - "#bighead"
      - .overband
      padding: 20
    ```

  - Scripted tests can now be run using `tests/run_examples.sh` [#29](https://github.com/simonw/shot-scraper/issues/29)


html_url: https://github.com/simonw/shot-scraper/releases/tag/0.7
node_id: RE_kwDOG-A5W84DrWLv
tag_name: 0.7
target_commitish: main
name: 0.7
created_at: 2022-03-13T04:46:58Z
published_at: 2022-03-13T04:49:36Z
body:
  - The `shot-scraper shot` and `shot-scraper pdf` commands both now default to writing a file to disk if no filename is specified, using a name derived from the URL. If you want to write the PNG or PDF content to standard output you can do so using `-o -`. [#32](https://github.com/simonw/shot-scraper/issues/32)
  - New `--retina` flag for `shot-scraper shot` and `shot-scraper multi` which causes the screenshot to be taken with a device scale factor of 2. [#33](https://github.com/simonw/shot-scraper/issues/33)
  - `shot-scraper shot --devtools` option opens an interactive browser window with the browser developer tools enabled. [#34](https://github.com/simonw/shot-scraper/issues/34)


html_url: https://github.com/simonw/shot-scraper/releases/tag/0.8
node_id: RE_kwDOG-A5W84DrZlH
tag_name: 0.8
target_commitish: main
name: 0.8
created_at: 2022-03-13T17:02:55Z
published_at: 2022-03-13T17:05:21Z
body:
  - `shot-scraper` can now take screenshots of local files on disk: [#35](https://github.com/simonw/shot-scraper/issues/35)

        shot-scraper index.html -o index.png

  - If you call `shot-scraper` on a URL with no protocol, `http://` will be assumed. Redirects will be followed:

        shot-scraper datasette.io -o datasette.png


html_url: https://github.com/simonw/shot-scraper/releases/tag/0.9
node_id: RE_kwDOG-A5W84DrblZ
tag_name: 0.9
target_commitish: main
name: 0.9
created_at: 2022-03-13T23:59:19Z
published_at: 2022-03-14T00:05:56Z
body:
  - New [shot-scraper javascript](https://github.com/simonw/shot-scraper/blob/0.9/README.md#scraping-pages-using-javascript) command for executing JavaScript against a web page and returning the result to the console as JSON: [#38](https://github.com/simonw/shot-scraper/issues/38)

        % shot-scraper javascript datasette.io document.title
        "Datasette: An open source multi-tool for exploring and publishing data"

    This can be used for web scraping and data extraction. Any JavaScript errors will cause the command to return an exit code of 1, so this can also be used to run tests against a website from within a continuous integration environment such as GitHub Actions.

  - The `shot-scraper pdf` and `shot-scraper accessibility` commands can both now be used with local files in addition to URLs. [#37](https://github.com/simonw/shot-scraper/issues/37)
  - The `output:` key is no longer required in YAML shot configuration: if omitted, an automatic filename will be used instead. [#40](https://github.com/simonw/shot-scraper/issues/40)
  - An empty YAML file no longer produces an error. [#41](https://github.com/simonw/shot-scraper/issues/41)


html_url: https://github.com/simonw/datasette-publish-vercel/releases/tag/0.12.1
node_id: RE_kwDODx4htM4DsSms
tag_name: 0.12.1
target_commitish: main
name: 0.12.1
created_at: 2022-03-15T23:08:52Z
published_at: 2022-03-15T23:10:09Z
body:
  - Fixed bug where the `invoke_startup()` method was not being called on startup, which caused Datasette plugins using [the startup() plugin hook](https://docs.datasette.io/en/stable/plugin_hooks.html#startup-datasette) to break. [#54](https://github.com/simonw/datasette-publish-vercel/issues/54)


html_url: https://github.com/eyeseast/geocode-sqlite/releases/tag/v0.6.1
node_id: RE_kwDOEXxXas4DsUY9
tag_name: v0.6.1
target_commitish: main
name: v0.6.1
created_at: 2022-03-16T01:38:54Z
published_at: 2022-03-16T01:40:34Z
body:
  ## What's Changed
  * if no bbox options are passed, `value` is None by @noslouch in https://github.com/eyeseast/geocode-sqlite/pull/30
  * Pass --api-key to mapbox test by @eyeseast in https://github.com/eyeseast/geocode-sqlite/pull/33

  ## New Contributors
  * @noslouch made their first contribution in https://github.com/eyeseast/geocode-sqlite/pull/30

  **Full Changelog**: https://github.com/eyeseast/geocode-sqlite/compare/v0.6.0...v0.6.1


html_url: https://github.com/simonw/datasette-hashed-urls/releases/tag/0.2
node_id: RE_kwDOHAjKFc4DsW0m
tag_name: 0.2
target_commitish: main
name: 0.2
created_at: 2022-03-16T04:35:16Z
published_at: 2022-03-16T04:35:26Z
body:
  - The URL for a cached database file is now `/fixtures-aa7318b` - previously it used an underscore. This ensures compatibility with existing installations that had used Datasette's now deprecated [hashed URL mode](https://docs.datasette.io/en/0.60.2/performance.html#hashed-url-mode). [#3](https://github.com/simonw/datasette-hashed-urls/issues/3)
  - The `max-age` in the cache header can nowbe configured using the `max_age` plugin configuration setting. [#6](https://github.com/simonw/datasette-hashed-urls/issues/6)
  - Cache header is now `cache-control: max-age=31536000, public` - which has been tested with the cache provided by [Vercel](https://vercel.com/). [#4](https://github.com/simonw/datasette-hashed-urls/issues/4)


html_url: https://github.com/simonw/datasette/releases/tag/0.61a0
node_id: RE_kwDOBm6k_c4DtgJY
tag_name: 0.61a0
target_commitish: main
name: 0.61a0
created_at: 2022-03-20T01:14:40Z
published_at: 2022-03-20T01:16:41Z
body:
  -   Removed hashed URL mode from Datasette. The new `datasette-hashed-urls` plugin can be used to achieve the same result, see [datasette-hashed-urls](https://docs.datasette.io/en/latest/performance.html#performance-hashed-urls) for details. ([#1661](https://github.com/simonw/datasette/issues/1661))
  -   Databases can now have a custom path within the Datasette instance that is indpendent of the database name, using the `db.route` property. ([#1668](https://github.com/simonw/datasette/issues/1668))
  -   URLs within Datasette now use a different encoding scheme for tables or databases that include "special" characters outside of the range of `a-zA-Z0-9_-`. This scheme is explained here: [Tilde encoding](https://docs.datasette.io/en/latest/internals.html#internals-tilde-encoding). ([#1657](https://github.com/simonw/datasette/issues/1657))
  -   Table and row HTML pages now include a `<link rel="alternate" type="application/json+datasette" href="...">` element and return a `Link: URL; rel="alternate"; type="application/json+datasette"` HTTP header pointing to the JSON version of those pages. ([#1533](https://github.com/simonw/datasette/issues/1533))
  -   `Access-Control-Expose-Headers: Link` is now added to the CORS headers, allowing remote JavaScript to access that header.
  -   Canned queries are now shown at the top of the database page, directly below the SQL editor. Previously they were shown at the bottom, below the list of tables. ([#1612](https://github.com/simonw/datasette/issues/1612))
  -   Datasette now has a default favicon. ([#1603](https://github.com/simonw/datasette/issues/1603))
  -   `sqlite_stat` tables are now hidden by default. ([#1587](https://github.com/simonw/datasette/issues/1587))
  -   SpatiaLite tables `data_licenses`, `KNN` and `KNN2` are now hidden by default. ([#1601](https://github.com/simonw/datasette/issues/1601))
  -   Python 3.6 is no longer supported. ([#1577](https://github.com/simonw/datasette/issues/1577))
  -   Tests now run against Python 3.11-dev. ([#1621](https://github.com/simonw/datasette/issues/1621))
  -   Fixed bug where [custom pages](https://docs.datasette.io/en/latest/custom_templates.html#custom-pages) did not work on Windows. Thanks, Robert Christie. ([#1545](https://github.com/simonw/datasette/issues/1545))
  -   SQL query tracing mechanism now works for queries executed in `asyncio` sub-tasks, such as those created by `asyncio.gather()`. ([#1576](https://github.com/simonw/datasette/issues/1576))
  -   [datasette.tracer](https://docs.datasette.io/en/latest/internals.html#internals-tracer) mechanism is now documented.
  -   Common Datasette symbols can now be imported directly from the top-level `datasette` package, see [Import shortcuts](https://docs.datasette.io/en/latest/internals.html#internals-shortcuts). Those symbols are `Response`, `Forbidden`, `NotFound`, `hookimpl`, `actor_matches_allow`. ([#957](https://github.com/simonw/datasette/issues/957))
  -   `/-/versions` page now returns additional details for libraries used by SpatiaLite. ([#1607](https://github.com/simonw/datasette/issues/1607))
  -   Documentation now links to the [Datasette Tutorials](https://datasette.io/tutorials).
  -   Datasette will now also look for SpatiaLite in `/opt/homebrew` - thanks, Dan Peterson. ([#1649](https://github.com/simonw/datasette/pull/1649))
  -   Datasette is now covered by a [Code of Conduct](https://github.com/simonw/datasette/blob/main/CODE_OF_CONDUCT.md). ([#1654](https://github.com/simonw/datasette/issues/1654))


html_url: https://github.com/simonw/datasette-publish-vercel/releases/tag/0.13
node_id: RE_kwDODx4htM4DtgXq
tag_name: 0.13
target_commitish: main
name: 0.13
created_at: 2022-03-20T02:00:38Z
published_at: 2022-03-20T02:01:52Z
body:
  - New `--crossdb` option for publishing a Datasette instance that has [cross-database queries](https://docs.datasette.io/en/stable/sql_queries.html) enabled. [#55](https://github.com/simonw/datasette-publish-vercel/issues/55)


html_url: https://github.com/simonw/datasette/releases/tag/0.61
node_id: RE_kwDOBm6k_c4Du1X7
tag_name: 0.61
target_commitish: main
name: 0.61
created_at: 2022-03-23T18:12:26Z
published_at: 2022-03-23T18:14:36Z
body:
  In preparation for Datasette 1.0, this release includes two potentially backwards-incompatible changes. Hashed URL mode has been moved to a separate plugin, and the way Datasette generates URLs to databases and tables with special characters in their name such as `/` and `.` has changed.

  Datasette also now requires Python 3.7 or higher.

  See also the [annotated release notes](https://simonwillison.net/2022/Mar/24/datasette-061/).

  -   URLs within Datasette now use a different encoding scheme for tables or databases that include "special" characters outside of the range of `a-zA-Z0-9_-`. This scheme is explained here: [Tilde encoding](https://docs.datasette.io/en/stable/internals.html#internals-tilde-encoding). ([#1657](https://github.com/simonw/datasette/issues/1657))
  -   Removed hashed URL mode from Datasette. The new `datasette-hashed-urls` plugin can be used to achieve the same result, see [datasette-hashed-urls](https://docs.datasette.io/en/stable/performance.html#performance-hashed-urls) for details. ([#1661](https://github.com/simonw/datasette/issues/1661))
  -   Databases can now have a custom path within the Datasette instance that is independent of the database name, using the `db.route` property. ([#1668](https://github.com/simonw/datasette/issues/1668))
  -   Datasette is now covered by a [Code of Conduct](https://github.com/simonw/datasette/blob/main/CODE_OF_CONDUCT.md). ([#1654](https://github.com/simonw/datasette/issues/1654))
  -   Python 3.6 is no longer supported. ([#1577](https://github.com/simonw/datasette/issues/1577))
  -   Tests now run against Python 3.11-dev. ([#1621](https://github.com/simonw/datasette/issues/1621))
  -   New [datasette.ensure_permissions(actor, permissions)](https://docs.datasette.io/en/stable/internals.html#datasette-ensure-permissions) internal method for checking multiple permissions at once. ([#1675](https://github.com/simonw/datasette/issues/1675))
  -   New [datasette.check_visibility(actor, action, resource=None)](https://docs.datasette.io/en/stable/internals.html#datasette-check-visibilty) internal method for checking if a user can see a resource that would otherwise be invisible to unauthenticated users. ([#1678](https://github.com/simonw/datasette/issues/1678))
  -   Table and row HTML pages now include a `<link rel="alternate" type="application/json+datasette" href="...">` element and return a `Link: URL; rel="alternate"; type="application/json+datasette"` HTTP header pointing to the JSON version of those pages. ([#1533](https://github.com/simonw/datasette/issues/1533))
  -   `Access-Control-Expose-Headers: Link` is now added to the CORS headers, allowing remote JavaScript to access that header.
  -   Canned queries are now shown at the top of the database page, directly below the SQL editor. Previously they were shown at the bottom, below the list of tables. ([#1612](https://github.com/simonw/datasette/issues/1612))
  -   Datasette now has a default favicon. ([#1603](https://github.com/simonw/datasette/issues/1603))
  -   `sqlite_stat` tables are now hidden by default. ([#1587](https://github.com/simonw/datasette/issues/1587))
  -   SpatiaLite tables `data_licenses`, `KNN` and `KNN2` are now hidden by default. ([#1601](https://github.com/simonw/datasette/issues/1601))
  -   SQL query tracing mechanism now works for queries executed in `asyncio` sub-tasks, such as those created by `asyncio.gather()`. ([#1576](https://github.com/simonw/datasette/issues/1576))
  -   [datasette.tracer](https://docs.datasette.io/en/stable/internals.html#internals-tracer) mechanism is now documented.
  -   Common Datasette symbols can now be imported directly from the top-level `datasette` package, see [Import shortcuts](https://docs.datasette.io/en/stable/internals.html#internals-shortcuts). Those symbols are `Response`, `Forbidden`, `NotFound`, `hookimpl`, `actor_matches_allow`. ([#957](https://github.com/simonw/datasette/issues/957))
  -   `/-/versions` page now returns additional details for libraries used by SpatiaLite. ([#1607](https://github.com/simonw/datasette/issues/1607))
  -   Documentation now links to the [Datasette Tutorials](https://datasette.io/tutorials).
  -   Datasette will now also look for SpatiaLite in `/opt/homebrew` - thanks, Dan Peterson. ([#1649](https://github.com/simonw/datasette/pull/1649))
  -   Fixed bug where [custom pages](https://docs.datasette.io/en/stable/custom_templates.html#custom-pages) did not work on Windows. Thanks, Robert Christie. ([#1545](https://github.com/simonw/datasette/issues/1545))
  -   Fixed error caused when a table had a column named `n`. ([#1228](https://github.com/simonw/datasette/issues/1228))


html_url: https://github.com/simonw/datasette/releases/tag/0.61.1
node_id: RE_kwDOBm6k_c4Du4TC
tag_name: 0.61.1
target_commitish: main
name: 0.61.1
created_at: 2022-03-23T18:54:10Z
published_at: 2022-03-23T20:31:09Z
body:
  - Fixed a bug where databases with a different route from their name (as used by the [datasette-hashed-urls plugin](https://datasette.io/plugins/datasette-hashed-urls)) returned errors when executing custom SQL queries. ([#1682](https://github.com/simonw/datasette/issues/1682))


html_url: https://github.com/simonw/datasette-hashed-urls/releases/tag/0.3
node_id: RE_kwDOHAjKFc4Du4jE
tag_name: 0.3
target_commitish: main
name: 0.3
created_at: 2022-03-23T20:40:29Z
published_at: 2022-03-23T20:42:49Z
body:
  - Now depends on [Datasette 0.61.1](https://docs.datasette.io/en/stable/changelog.html#v0-61-1)
  - Fixes several bugs: [#9](https://github.com/simonw/datasette-hashed-urls/issues/9), [#10](https://github.com/simonw/datasette-hashed-urls/issues/10), [#12](https://github.com/simonw/datasette-hashed-urls/issues/12), [#13](https://github.com/simonw/datasette-hashed-urls/issues/13) - thanks for the QA, Forest Gregg.


html_url: https://github.com/simonw/datasette-hashed-urls/releases/tag/0.3.1
node_id: RE_kwDOHAjKFc4Du9om
tag_name: 0.3.1
target_commitish: main
name: 0.3.1
created_at: 2022-03-24T03:22:55Z
published_at: 2022-03-24T03:23:43Z
body:
  - Fixed a bug where the necessary CORS headers were not included with the redirects. [#14](https://github.com/simonw/datasette-hashed-urls/issues/14)


html_url: https://github.com/simonw/datasette-hashed-urls/releases/tag/0.4
node_id: RE_kwDOHAjKFc4DvQrA
tag_name: 0.4
target_commitish: main
name: 0.4
created_at: 2022-03-24T17:54:52Z
published_at: 2022-03-24T20:52:04Z
body:
  - If run with Datasette's `--crossdb` option to enable [cross-database queries](https://docs.datasette.io/en/stable/sql_queries.html#cross-database-queries) the `/_memory` database now gains a hash that is the combined hash of all of the attached immutable databases. [Example query](https://datasette-hashed-urls.vercel.app/_memory?sql=select+*+from+fixtures.facetable+union+all+select+*+from+fixtures2.facetable). [#15](https://github.com/simonw/datasette-hashed-urls/issues/15)


html_url: https://github.com/simonw/datasette-packages/releases/tag/0.1
node_id: RE_kwDOHENadM4DvpDN
tag_name: 0.1
target_commitish: main
name: 0.1
created_at: 2022-03-25T22:04:04Z
published_at: 2022-03-25T22:05:51Z
body:
  - Initial release. `/-/packages` shows a list of installed Python packages, `/-/packages.json` returns that same list as JSON.


html_url: https://github.com/simonw/datasette-packages/releases/tag/0.1.1
node_id: RE_kwDOHENadM4DvpKE
tag_name: 0.1.1
target_commitish: main
name: 0.1.1
created_at: 2022-03-25T22:16:17Z
published_at: 2022-03-25T22:17:21Z
body:
  - Fixed bug where plugin would raise a 500 error when installed with some other plugins. [#2](https://github.com/simonw/datasette-packages/issues/2)


html_url: https://github.com/hydrosquall/datasette-nteract-data-explorer/releases/tag/0.1.0
node_id: RE_kwDOHEflbM4DvxfT
tag_name: 0.1.0
target_commitish: main
name: 0.1.0 - Initial Release
created_at: 2022-03-27T00:22:35Z
published_at: 2022-03-27T00:33:21Z
body:
  First release! May not work in browsers that don't use ESModules. Feedback and bug reports are welcome.

  **Full Changelog**: https://github.com/hydrosquall/datasette-nteract-data-explorer/commits/0.1.0


html_url: https://github.com/hydrosquall/datasette-nteract-data-explorer/releases/tag/0.2.0
node_id: RE_kwDOHEflbM4DvxuL
tag_name: 0.2.0
target_commitish: main
name: 0.2.0 - With Statics
created_at: 2022-03-27T01:32:15Z
published_at: 2022-03-27T01:33:44Z
body:
  The first release worked with local development, but was not building new JS in CI. This release should fix that.

  **Full Changelog**: https://github.com/hydrosquall/datasette-nteract-data-explorer/compare/0.1.0...0.2.0


html_url: https://github.com/hydrosquall/datasette-nteract-data-explorer/releases/tag/0.2.3
node_id: RE_kwDOHEflbM4Dvx6r
tag_name: 0.2.3
target_commitish: main
name: 0.2.3 - Separate JS and Python build folders
created_at: 2022-03-27T02:20:42Z
published_at: 2022-03-27T02:21:47Z
body:
  The build systems from python and JS were interfering with each other. This should fix that.

  **Full Changelog**: https://github.com/hydrosquall/datasette-nteract-data-explorer/compare/0.2.2...0.2.3


html_url: https://github.com/simonw/datasette-auth0/releases/tag/0.1
node_id: RE_kwDOHEfRqM4Dv45e
tag_name: 0.1
target_commitish: main
name: 0.1
created_at: 2022-03-28T02:49:05Z
published_at: 2022-03-28T03:04:52Z
body:
  - Now includes a live demo, see [datasette-auth0-demo.datasette.io](https://datasette-auth0-demo.datasette.io/) [#1](https://github.com/simonw/datasette-auth0/issues/1)


html_url: https://github.com/simonw/shot-scraper/releases/tag/0.10
node_id: RE_kwDOG-A5W84DwQdf
tag_name: 0.10
target_commitish: main
name: 0.10
created_at: 2022-03-29T00:25:29Z
published_at: 2022-03-29T00:25:33Z
body:
  - New `-b/--browser` option for the `shot-scraper install`, `shot`, `multi` and `javascript` commands. This can be used to install and run alternative browsers `firefox`, `chrome` or `chrome-beta`. Thanks, [Ben Welsh](https://github.com/palewire). [#53](https://github.com/simonw/shot-scraper/issues/53)
  - New `--timeout` option for `shot-scraper shot` and `shot-scraper multi`. Thanks, [Ben Welsh](https://github.com/palewire). [#47](https://github.com/simonw/shot-scraper/pull/47)
  - `shot-scraper multi` now continues to create other shots despite a timeout error, unless `--fail-on-error` is passed. Thanks, [Ryan Cheley](https://github.com/ryancheley). [#50](https://github.com/simonw/shot-scraper/issues/50)
  - README now describes the `async () => { ... }` pattern for `shot-scraper javascript`, as discussed in [Extracting web page content using Readability.js and shot-scraper](https://til.simonwillison.net/shot-scraper/readability).
  - New demo: [shot-scraper-template](https://github.com/simonw/shot-scraper-template), which lets you setup shot-scraper running in a GitHub repository in a single click. [See also Instantly create a GitHub repository to take screenshots of a web page](https://simonwillison.net/2022/Mar/14/shot-scraper-template/).
  - New scraping demo: [scrape-hacker-news-by-domain](https://github.com/simonw/scrape-hacker-news-by-domain) uses `shot-scraper javascript` to scrape a web page. See [Scraping web pages from the command-line with shot-scraper](https://simonwillison.net/2022/Mar/14/scraping-web-pages-shot-scraper/).


html_url: https://github.com/simonw/pypi-to-sqlite/releases/tag/0.1
node_id: RE_kwDOHI-jK84Dz3jp
tag_name: 0.1
target_commitish: main
name: 0.1
created_at: 2022-04-07T23:40:43Z
published_at: 2022-04-07T23:43:55Z
body:
  - Initial release, with `pypi-to-sqlite pypi.db datasette sqlite-utils` command. [#1](https://github.com/simonw/pypi-to-sqlite/issues/1)


html_url: https://github.com/simonw/pypi-to-sqlite/releases/tag/0.2
node_id: RE_kwDOHI-jK84D0EUb
tag_name: 0.2
target_commitish: main
name: 0.2
created_at: 2022-04-08T14:47:07Z
published_at: 2022-04-08T14:48:21Z
body:
  - New `--prefix prefix` option for creating tables with a prefix - so `pypi-to-sqlite data.db datasette --prefix pypi_` will create tables called `pypi_packages`, `pypi_versions` and `pypi_releases`. [#6](https://github.com/simonw/pypi-to-sqlite/issues/6)


html_url: https://github.com/simonw/pypi-to-sqlite/releases/tag/0.2.2
node_id: RE_kwDOHI-jK84D0E2e
tag_name: 0.2.2
target_commitish: main
name: 0.2.2
created_at: 2022-04-08T15:11:36Z
published_at: 2022-04-08T15:12:05Z
body:
  - Fixed a bug where the command could not be safely run more than once against the same database file. [#7](https://github.com/simonw/pypi-to-sqlite/issues/7)


html_url: https://github.com/simonw/shot-scraper/releases/tag/0.11
node_id: RE_kwDOG-A5W84D0J5S
tag_name: 0.11
target_commitish: main
name: 0.11
created_at: 2022-04-08T20:02:26Z
published_at: 2022-04-08T20:06:57Z
body:
  - New `shot-scraper accessibility --timeout` option, thanks [Ben Welsh](https://github.com/palewire). [#59](https://github.com/simonw/shot-scraper/pull/59)
  - `shot-scraper auth --browser` option for authentication using a browser other than Chromium. [#61](https://github.com/simonw/shot-scraper/issues/61)
  - Using `--quality` now results in a JPEG file with the correct `.jpg` extension. Thanks, [Ian Wootten](https://github.com/iwootten). [#58](https://github.com/simonw/shot-scraper/pull/58)
  - New `--reduced-motion` flag for emulating the "prefers-reduced-motion" media feature. Thanks, [Ryan Murphy](https://github.com/rdmurphy). [#49](https://github.com/simonw/shot-scraper/issues/49)

reactions:
  {"url": "https://api.github.com/repos/simonw/shot-scraper/releases/64003666/reactions", "total_count": 5, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 2, "rocket": 1, "eyes": 2}


html_url: https://github.com/simonw/shot-scraper/releases/tag/0.12
node_id: RE_kwDOG-A5W84D0t_M
tag_name: 0.12
target_commitish: main
name: 0.12
created_at: 2022-04-11T19:32:44Z
published_at: 2022-04-11T19:32:50Z
body:
  - `--selector-all` option to take a screenshot that encompasses every element matching the specified CSS selector - complements `--selector` which takes a screenshot of just the first element matching that selector. See [Specifying elements using JavaScript filters](https://github.com/simonw/shot-scraper/blob/0.12/README.md#specifying-elements-using-javascript-filters). [#64](https://github.com/simonw/shot-scraper/issues/64)
  - `selector_all:` and `selectors_all:` keys in the `shot-scraper multi` YAML format.
  - `--js-selector` and `--js-selector-all` options for specifying elements to screenshot using a JavaScript expression, for cases which cannot be handled using CSS selectors [#43](https://github.com/simonw/shot-scraper/issues/43). The following example takes a screenshot of the first paragraph on the page that mentions `shot-scraper`:
      ```
      shot-scraper https://github.com/simonw/shot-scraper \
        --js-selector 'el.tagName == "P" && el.innerText.includes("shot-scraper")'
      ```
  - `js_selector:`, `js_selectors:`, `js_selector_all:` and `js_selectors_all:` equivalent keys in the `shot-scraper multi` YAML format.
  - `--user-agent` option for setting a custom user agent header. [#63](https://github.com/simonw/shot-scraper/issues/63)
  - `--browser webkit` option for running [WebKit](https://playwright.dev/docs/browsers#webkit) - thanks, [Ryan Murphy](https://github.com/rdmurphy). [#56](https://github.com/simonw/shot-scraper/pull/56)


html_url: https://github.com/simonw/geojson-to-sqlite/releases/tag/1.0.1
node_id: RE_kwDODiU8M84D01H4
tag_name: 1.0.1
target_commitish: main
name: 1.0.1
created_at: 2022-04-12T02:17:38Z
published_at: 2022-04-12T02:19:54Z
body:
  - Fixed bug where `--spatialite` option failed with an error if the GeoJSON features being loaded consisted of just a geometry with no properties. [#30](https://github.com/simonw/geojson-to-sqlite/issues/30)


html_url: https://github.com/simonw/geojson-to-sqlite/releases/tag/1.1
node_id: RE_kwDODiU8M84D1X4A
tag_name: 1.1
target_commitish: main
name: 1.1
created_at: 2022-04-13T04:55:59Z
published_at: 2022-04-13T05:00:09Z
body:
  - Command can now accept more than one GeoJSON file. The contents of all of the files will be inserted into the same table. [#21](https://github.com/simonw/geojson-to-sqlite/issues/21)
  - Using the `--spatial-index` option against the same database twice no longer triggers an error.
  - Now depends on [sqlite-utils 3.23](https://sqlite-utils.datasette.io/en/stable/changelog.html#v3-23) on higher, taking advantage of the SpatiaLite utilities in that library. Thanks, [Chris Amico](https://github.com/eyeseast). [#28](https://github.com/simonw/geojson-to-sqlite/issues/28)


html_url: https://github.com/simonw/sqlite-utils/releases/tag/3.26
node_id: RE_kwDOCGYnMM4D1uRS
tag_name: 3.26
target_commitish: main
name: 3.26
created_at: 2022-04-13T22:51:48Z
published_at: 2022-04-13T22:53:36Z
body:
  -   New `errors=r.IGNORE/r.SET_NULL` parameter for the `r.parsedatetime()` and `r.parsedate()` [convert recipes](https://sqlite-utils.datasette.io/en/stable/cli.html#cli-convert-recipes). ([#416](https://github.com/simonw/sqlite-utils/issues/416))
  -   Fixed a bug where `--multi` could not be used in combination with `--dry-run` for the [convert](https://sqlite-utils.datasette.io/en/stable/cli.html#cli-convert) command. ([#415](https://github.com/simonw/sqlite-utils/issues/415))
  -   New documentation: [Using a convert() function to execute initialization](https://sqlite-utils.datasette.io/en/stable/cli.html#cli-convert-complex). ([#420](https://github.com/simonw/sqlite-utils/issues/420))
  -   More robust detection for whether or not `deterministic=True` is supported. ([#425](https://github.com/simonw/sqlite-utils/issues/425))


html_url: https://github.com/simonw/geojson-to-sqlite/releases/tag/1.1.1
node_id: RE_kwDODiU8M84D1urd
tag_name: 1.1.1
target_commitish: main
name: 1.1.1
created_at: 2022-04-13T23:37:53Z
published_at: 2022-04-13T23:39:25Z
body:
  - Fixed a bug where a `_tmp` column remained after importing a geometry with no additional properties. [#30](https://github.com/simonw/geojson-to-sqlite/issues/30)
  - Added `--version` option to show the current installed version of the tool. [#32](https://github.com/simonw/geojson-to-sqlite/issues/32)


html_url: https://github.com/hydrosquall/datasette-nteract-data-explorer/releases/tag/0.3.1
node_id: RE_kwDOHEflbM4D2Zan
tag_name: 0.3.1
target_commitish: main
name: 0.3.1 - Custom SQL Queries, Published Demo
created_at: 2022-04-16T23:16:34Z
published_at: 2022-04-16T23:18:58Z
body:
  ## What's Changed

  * feature: enable data explorer to activate when writing custom SQL by @hydrosquall in https://github.com/hydrosquall/datasette-nteract-data-explorer/pull/5
  * docs: replace browser history example with happy planet index dataset by @hydrosquall in https://github.com/hydrosquall/datasette-nteract-data-explorer/pull/6
  * build: deploy demo site to vercel by @hydrosquall in https://github.com/hydrosquall/datasette-nteract-data-explorer/pull/8
  * release: 0.3.0 by @hydrosquall in https://github.com/hydrosquall/datasette-nteract-data-explorer/pull/11

  **Full Changelog**: https://github.com/hydrosquall/datasette-nteract-data-explorer/compare/0.2.1...0.3.1


html_url: https://github.com/simonw/shot-scraper/releases/tag/0.13
node_id: RE_kwDOG-A5W84D2rqd
tag_name: 0.13
target_commitish: main
name: 0.13
created_at: 2022-04-18T18:21:30Z
published_at: 2022-04-18T18:29:46Z
body:
  - `shot-scraper auth --devtools` option for opening DevTools when authenticating with a site. [#66](https://github.com/simonw/shot-scraper/issues/66)
  - `shot-scraper multi -n/--no-clobber` option for skipping screenshots where the output image already exists on disk. [#70](https://github.com/simonw/shot-scraper/issues/70)


html_url: https://github.com/simonw/django-sql-dashboard/releases/tag/1.1
node_id: RE_kwDOFLLS6s4D3Llv
tag_name: 1.1
target_commitish: main
name: 1.1
created_at: 2022-04-20T00:23:41Z
published_at: 2022-04-20T00:24:47Z
body:
  - `<br>` tag is now supported in HTML output. [#150](https://github.com/simonw/django-sql-dashboard/issues/150)


html_url: https://github.com/simonw/datasette-total-page-time/releases/tag/0.1
node_id: RE_kwDOHPc0N84D5hc6
tag_name: 0.1
target_commitish: main
name: 0.1
created_at: 2022-04-26T22:10:20Z
published_at: 2022-04-26T22:11:48Z
body:
  - Initial implementation. Installing this plugin adds a "Page took 38.310ms" note to the Datasette footer.


html_url: https://github.com/simonw/datasette-gzip/releases/tag/0.1
node_id: RE_kwDOHPj_Zs4D52JT
tag_name: 0.1
target_commitish: main
name: 0.1
created_at: 2022-04-27T17:25:10Z
published_at: 2022-04-27T17:27:49Z
body:
  - Initial release. Adds gzip support to Datasette, respecting the `Accept-Encoding` HTTP header. A thin wrapper around the [GZipMiddleware](https://www.starlette.io/middleware/#gzipmiddleware) from Starlette.


html_url: https://github.com/simonw/datasette-gzip/releases/tag/0.2
node_id: RE_kwDOHPj_Zs4D6Pgo
tag_name: 0.2
target_commitish: main
name: 0.2
created_at: 2022-04-28T16:05:18Z
published_at: 2022-04-28T16:06:51Z
body:
  - No longer includes [Starlette](https://www.starlette.io/) as a dependency. Now uses [asgi-gzip](https://github.com/simonw/asgi-gzip), a smaller package which extracts the `GZipMiddleware` class from Starlette. [#2](https://github.com/simonw/datasette-gzip/issues/2)


html_url: https://github.com/simonw/datasette-copy-to-memory/releases/tag/0.1a0
node_id: RE_kwDOHQLzfc4D6VCd
tag_name: 0.1a0
target_commitish: main
name: 0.1a0
created_at: 2022-04-28T20:03:43Z
published_at: 2022-04-28T20:05:30Z
body:
  - Initial prototype: copies each attached database file to an in-memory database on startup. [#1](https://github.com/simonw/datasette-copy-to-memory/issues/1)


html_url: https://github.com/simonw/datasette-copy-to-memory/releases/tag/0.1a1
node_id: RE_kwDOHQLzfc4D6VwF
tag_name: 0.1a1
target_commitish: main
name: 0.1a1
created_at: 2022-04-28T20:47:24Z
published_at: 2022-04-28T20:48:43Z
body:
  - [Plugin configuration](https://github.com/simonw/datasette-copy-to-memory/blob/0.1a1/README.md#configuration) can now be used to select specific databases to be copied into memory on startup. [#2](https://github.com/simonw/datasette-copy-to-memory/issues/2)


html_url: https://github.com/simonw/datasette-copy-to-memory/releases/tag/0.1a2
node_id: RE_kwDOHQLzfc4D6WCj
tag_name: 0.1a2
target_commitish: main
name: 0.1a2
created_at: 2022-04-28T21:06:25Z
published_at: 2022-04-28T21:07:09Z
body:
  - Fixed crash when attempting to load an immutable database. [#3](https://github.com/simonw/datasette-copy-to-memory/issues/3)


html_url: https://github.com/simonw/datasette-copy-to-memory/releases/tag/0.1a3
node_id: RE_kwDOHQLzfc4D6Wn-
tag_name: 0.1a3
target_commitish: main
name: 0.1a3
created_at: 2022-04-28T21:47:08Z
published_at: 2022-04-28T21:48:59Z
body:
  - Fixed a bug where the plugin would generate unnecessary `file:...` files every time it ran. [#4](https://github.com/simonw/datasette-copy-to-memory/issues/4)
  - Plugin now has a live demo running on [latest-with-plugins.datasette.io/fixtures_memory](https://latest-with-plugins.datasette.io/fixtures_memory). [#5](https://github.com/simonw/datasette-copy-to-memory/issues/5)


html_url: https://github.com/simonw/datasette-copy-to-memory/releases/tag/0.2
node_id: RE_kwDOHQLzfc4D6yVB
tag_name: 0.2
target_commitish: main
name: 0.2
created_at: 2022-04-30T19:47:07Z
published_at: 2022-04-30T19:49:29Z
body:
  - New `replace: true` plugin configuration option for replacing the file database with the in-memory one, rather than adding a new `/db_memory` route that duplicates that database. [#6](https://github.com/simonw/datasette-copy-to-memory/issues/6)


html_url: https://github.com/simonw/s3-credentials/releases/tag/0.11
node_id: RE_kwDOGUV9is4D65Ld
tag_name: 0.11
target_commitish: main
name: 0.11
created_at: 2022-05-01T22:33:02Z
published_at: 2022-05-01T22:36:13Z
body:
  - New `set-cors-policy` and `get-cors-policy` for altering the CORS policy for a bucket, [documented here](https://github.com/simonw/s3-credentials/blob/0.11/README.md#set-cors-policy-and-get-cors-policy). [#47](https://github.com/simonw/s3-credentials/issues/47)
  - Each command now shows example usage in `--help`. [#67](https://github.com/simonw/s3-credentials/issues/67)
  - New [documentation page](https://github.com/simonw/s3-credentials/blob/main/help.md) listing the `--help` for every command.

reactions:
  {"url": "https://api.github.com/repos/simonw/s3-credentials/releases/65770205/reactions", "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


html_url: https://github.com/simonw/sqlite-utils/releases/tag/3.26.1
node_id: RE_kwDOCGYnMM4D7IuS
tag_name: 3.26.1
target_commitish: main
name: 3.26.1
created_at: 2022-05-02T18:17:19Z
published_at: 2022-05-02T18:18:01Z
body:
  - Now depends on [click-default-group-wheel](https://github.com/simonw/click-default-group-wheel), a pure Python wheel package. This means you can install and use this package with [Pyodide](https://pyodide.org/), which can run Python entirely in your browser using WebAssembly. ([#429](https://github.com/simonw/sqlite-utils/pull/429))

    Try that out using the [Pyodide REPL](https://pyodide.org/en/stable/console.html):

    ```pycon
    >>> import micropip
    >>> await micropip.install("sqlite-utils")
    >>> import sqlite_utils
    >>> db = sqlite_utils.Database(memory=True)
    >>> list(db.query("select 3 * 5"))
    [{'3 * 5': 15}]
    ```

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/releases/65833874/reactions", "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 1, "eyes": 0}


html_url: https://github.com/simonw/datasette/releases/tag/0.62a0
node_id: RE_kwDOBm6k_c4D7ME1
tag_name: 0.62a0
target_commitish: main
name: 0.62a0
created_at: 2022-05-02T21:38:34Z
published_at: 2022-05-02T21:39:52Z
body:
  -   Datasette now runs some SQL queries in parallel. This has limited impact on performance, see [this research issue](https://github.com/simonw/datasette/issues/1727) for details.
  -   Datasette should now be compatible with Pyodide. ([#1733](https://github.com/simonw/datasette/issues/1733))
  -   `datasette publish cloudrun` has a new `--timeout` option which can be used to increase the time limit applied by the Google Cloud build environment. Thanks, Tim Sherratt. ([#1717](https://github.com/simonw/datasette/pull/1717))
  -   Spaces in database names are now encoded as `+` rather than `~20`. ([#1701](https://github.com/simonw/datasette/issues/1701))
  -   `<Binary: 2427344 bytes>` is now displayed as `<Binary: 2,427,344 bytes>` and is accompanied by tooltip showing "2.3MB". ([#1712](https://github.com/simonw/datasette/issues/1712))
  -   Don't show the facet option in the cog menu if faceting is not allowed. ([#1683](https://github.com/simonw/datasette/issues/1683))
  -   Code examples in the documentation are now all formatted using Black. ([#1718](https://github.com/simonw/datasette/issues/1718))
  -   `Request.fake()` method is now documented, see [Request object](https://docs.datasette.io/en/latest/internals.html#internals-request).


html_url: https://github.com/simonw/datasette-insert/releases/tag/0.7
node_id: RE_kwDOELgXO84D_mRl
tag_name: 0.7
target_commitish: main
name: 0.7
created_at: 2022-05-16T18:43:10Z
published_at: 2022-05-16T18:45:09Z
body:
  - New `/-/upsert/db/table?pk=id` API endpoint for performing upserts, where partial JSON documents can be provided and those keys will be updated on existing row or used to create new rows if a matching row does not exist. [#9](https://github.com/simonw/datasette-insert/issues/9)


html_url: https://github.com/simonw/datasette-upload-dbs/releases/tag/0.1
node_id: RE_kwDOHSfuMs4D_shq
tag_name: 0.1
target_commitish: main
name: 0.1
created_at: 2022-05-17T03:05:00Z
published_at: 2022-05-17T03:06:55Z
body:
  - Initial release. Authorized users can upload new SQLite database files to Datasette - they will be stored, served and will persist across server restarts. [#1](https://github.com/simonw/datasette-upload-dbs/issues/1)


html_url: https://github.com/simonw/datasette-upload-dbs/releases/tag/0.1.1
node_id: RE_kwDOHSfuMs4D_s5N
tag_name: 0.1.1
target_commitish: main
name: 0.1.1
created_at: 2022-05-17T03:51:19Z
published_at: 2022-05-17T03:52:31Z
body:
  - Fixed bug where file name field continued to be displayed during the upload. [#3](https://github.com/simonw/datasette-upload-dbs/issues/3)
  - Fixed bug where errors displayed using JavaScript did not provide an easy path to trying the upload again. [#4](https://github.com/simonw/datasette-upload-dbs/issues/4)


html_url: https://github.com/simonw/shot-scraper/releases/tag/0.14
node_id: RE_kwDOG-A5W84EA3Ib
tag_name: 0.14
target_commitish: main
name: 0.14
created_at: 2022-05-19T23:43:37Z
published_at: 2022-05-19T23:45:02Z
body:
  - Documentation is now hosted at **[shot-scraper.datasette.io](https://shot-scraper.datasette.io/)** and has been expanded to multiple pages. [#73](https://github.com/simonw/shot-scraper/issues/73)
  - `shot-scraper $URL --wait-for EXPRESSION` can be used to take the screenshot only once the provided JavaScript expression returns `true`. See [Waiting until a specific condition](https://shot-scraper.datasette.io/en/stable/screenshots.html#waiting-until-a-specific-condition). [#72](https://github.com/simonw/shot-scraper/issues/72)
  - The `wait_for:` key in the YAML format used by `shot-scraper multi` provides equivalent functionality for scripted multiple screenshots.
  - New documentation on [Using shot-scraper with GitHub Actions](https://shot-scraper.datasette.io/en/stable/github-actions.html). [#74](https://github.com/simonw/shot-scraper/issues/74)
  - New example: [Extracting page content with Readability.js](https://shot-scraper.datasette.io/en/stable/javascript.html#example-extracting-page-content-with-readability-js)


html_url: https://github.com/simonw/shot-scraper/releases/tag/0.14.1
node_id: RE_kwDOG-A5W84EBcrs
tag_name: 0.14.1
target_commitish: main
name: 0.14.1
created_at: 2022-05-22T14:22:50Z
published_at: 2022-05-22T14:24:06Z
body:
  - Switched [documentation site](https://shot-scraper.datasette.io/) over to the [Furo](https://github.com/pradyunsg/furo) theme. [#77](https://github.com/simonw/shot-scraper/issues/77)


html_url: https://github.com/simonw/datasette-auth-existing-cookies/releases/tag/1.0a0
node_id: RE_kwDODhLDp84EDxCY
tag_name: 1.0a0
target_commitish: main
name: 1.0a0
created_at: 2022-05-29T15:12:11Z
published_at: 2022-05-29T15:15:37Z
body:
  This alpha release breaks backwards compatibility, and completely redesigns how this plugin works. [#13](https://github.com/simonw/datasette-auth-existing-cookies/issues/13)

  - Plugin now uses the [actor_from_request()](https://docs.datasette.io/en/stable/plugin_hooks.html#plugin-hook-actor-from-request) plugin hook
  - Implementation is greatly simplified: the forbidden mechanism and redirect to login via a signed `next=` parameter components have been removed
  - Plugin configuration settings have been renamed, see [the README](https://github.com/simonw/datasette-auth-existing-cookies/blob/1.0a0/README.md#configuration)


html_url: https://github.com/simonw/datasette-auth-existing-cookies/releases/tag/1.0a1
node_id: RE_kwDODhLDp84EDxui
tag_name: 1.0a1
target_commitish: main
name: 1.0a1
created_at: 2022-05-29T17:37:30Z
published_at: 2022-05-29T17:38:24Z
body:
  - Fix for bug: `AttributeError: 'Datasette' object has no attribute '_auth_existing_cookies_cache'` [#19](https://github.com/simonw/datasette-auth-existing-cookies/issues/19)


html_url: https://github.com/simonw/datasette-graphql/releases/tag/2.0.2
node_id: RE_kwDOEPNYIc4EEFoU
tag_name: 2.0.2
target_commitish: main
name: 2.0.2
created_at: 2022-05-30T18:21:50Z
published_at: 2022-05-30T18:23:05Z
body:
  - Upgrade dependencies to `graphene>=3.1.0` and `graphql-core>=3.2.1`. [#88](https://github.com/simonw/datasette-graphql/pull/88)


html_url: https://github.com/simonw/datasette-upload-csvs/releases/tag/0.7
node_id: RE_kwDODf8oPs4EFjuQ
tag_name: 0.7
target_commitish: main
name: 0.7
created_at: 2022-06-03T15:28:57Z
published_at: 2022-06-03T15:30:55Z
body:
  - Now requires and is compatible with `datasette>=0.61`. [#21](https://github.com/simonw/datasette-upload-csvs/issues/21)
  - Dropped support for Python 3.6


html_url: https://github.com/simonw/datasette-upload-csvs/releases/tag/0.7.1
node_id: RE_kwDODf8oPs4EHlKF
tag_name: 0.7.1
target_commitish: main
name: 0.7.1
created_at: 2022-06-09T19:38:14Z
published_at: 2022-06-09T19:40:01Z
body:
  - Fixed bug where request would block while upload was being processed. [#22](https://github.com/simonw/datasette-upload-csvs/issues/22)
  - CSV files that use an encoding other than `utf-8` are now supported. [#23](https://github.com/simonw/datasette-upload-csvs/issues/23)
  - Errors during import are now displayed to the user. [#24](https://github.com/simonw/datasette-upload-csvs/issues/24)


html_url: https://github.com/simonw/datasette-socrata/releases/tag/0.1
node_id: RE_kwDOHeFHys4EHm8e
tag_name: 0.1
target_commitish: main
name: 0.1
created_at: 2022-06-09T21:24:07Z
published_at: 2022-06-09T21:24:37Z
body:
  - Initial release. Adds a `/-/import-socrata` page which can be used to import data by providing the URL to a dataset on Socrata. [#1](https://github.com/simonw/datasette-socrata/issues/1)


html_url: https://github.com/simonw/datasette-socrata/releases/tag/0.1.1
node_id: RE_kwDOHeFHys4EH617
tag_name: 0.1.1
target_commitish: main
name: 0.1.1
created_at: 2022-06-10T18:53:37Z
published_at: 2022-06-10T18:54:22Z
body:
  - Fixed bug where error messages relating to incorrect configuration were served as 403 forbidden. [#9](https://github.com/simonw/datasette-socrata/issues/9)


html_url: https://github.com/simonw/datasette-ics/releases/tag/0.5.1
node_id: RE_kwDODoa7Dc4EH-Cl
tag_name: 0.5.1
target_commitish: main
name: 0.5.1
created_at: 2022-06-10T22:52:34Z
published_at: 2022-06-10T22:54:54Z
body:
  - Added [live demo](https://github.com/simonw/datasette-ics/blob/0.5.1/README.md#demo) and screenshot to the documentation. [#5](https://github.com/simonw/datasette-ics/issues/5)


html_url: https://github.com/simonw/datasette-socrata/releases/tag/0.2
node_id: RE_kwDOHeFHys4EIDOD
tag_name: 0.2
target_commitish: main
name: 0.2
created_at: 2022-06-11T15:54:03Z
published_at: 2022-06-11T15:54:55Z
body:
  - Table metadata in Datasette now reflects metadata imported from Socrata. [#11](https://github.com/simonw/datasette-socrata/issues/11)


html_url: https://github.com/simonw/sqlite-utils/releases/tag/3.27
node_id: RE_kwDOCGYnMM4EJDTY
tag_name: 3.27
target_commitish: main
name: 3.27
created_at: 2022-06-15T04:30:47Z
published_at: 2022-06-15T04:34:45Z
body:
  See also [the annotated release notes](https://simonwillison.net/2022/Jun/19/weeknotes/#sqlite-utils-3-27) for this release.

  -   Documentation now uses the [Furo](https://github.com/pradyunsg/furo) Sphinx theme. ([#435](https://github.com/simonw/sqlite-utils/issues/435))
  -   Code examples in documentation now have a "copy to clipboard" button. ([#436](https://github.com/simonw/sqlite-utils/issues/436))
  -   `sqlite_utils.utils.utils.rows_from_file()` is now a documented API, see [Reading rows from a file](https://sqlite-utils.datasette.io/en/stable/python-api.html#python-api-rows-from-file). ([#443](https://github.com/simonw/sqlite-utils/issues/443))
  -   `rows_from_file()` has two new parameters to help handle CSV files with rows that contain more values than are listed in that CSV file's headings: `ignore_extras=True` and `extras_key="name-of-key"`. ([#440](https://github.com/simonw/sqlite-utils/issues/440))
  -   `sqlite_utils.utils.maximize_csv_field_size_limit()` helper function for increasing the field size limit for reading CSV files to its maximum, see [Setting the maximum CSV field size limit](https://sqlite-utils.datasette.io/en/stable/python-api.html#python-api-maximize-csv-field-size-limit). ([#442](https://github.com/simonw/sqlite-utils/issues/442))
  -   `table.search(where=, where_args=)` parameters for adding additional `WHERE` clauses to a search query. The `where=` parameter is available on `table.search_sql(...)` as well. See [Searching with table.search()](https://sqlite-utils.datasette.io/en/stable/python-api.html#python-api-fts-search). ([#441](https://github.com/simonw/sqlite-utils/issues/441))
  -   Fixed bug where `table.detect_fts()` and other search-related functions could fail if two FTS-enabled tables had names that were prefixes of each other. ([#434](https://github.com/simonw/sqlite-utils/issues/434))


html_url: https://github.com/simonw/datasette-socrata/releases/tag/0.3
node_id: RE_kwDOHeFHys4EKD4G
tag_name: 0.3
target_commitish: main
name: 0.3
created_at: 2022-06-17T20:25:42Z
published_at: 2022-06-17T20:29:53Z
body:
  - Any errors that occur during an import are now stored in the `errors` column of the `socrata_imports` table, and displayed by the JavaScript progress indicator. [#12](https://github.com/simonw/datasette-socrata/issues/12)
  - CSV imports with long field values no longer trigger an error. [#13](https://github.com/simonw/datasette-socrata/issues/13)
  - Progress bars now show the number of records imported and remaining, and hint to refresh the page when the import is complete. [#10](https://github.com/simonw/datasette-socrata/issues/10)
  - Checks for low disk space using the plugin hook provided by [datasette-low-disk-space-hook](https://github.com/simonw/datasette-low-disk-space-hook). [#4](https://github.com/simonw/datasette-socrata/issues/4)


html_url: https://github.com/simonw/datasette-scale-to-zero/releases/tag/0.1
node_id: RE_kwDOHilbp84EK_lJ
tag_name: 0.1
target_commitish: main
name: 0.1
created_at: 2022-06-21T22:50:33Z
published_at: 2022-06-21T22:51:59Z
body:
  - Initial release. Can be configured to cause Datasette to exit if it has not received traffic in a specified time period. [#1](https://github.com/simonw/datasette-scale-to-zero/issues/1)


html_url: https://github.com/simonw/datasette-scale-to-zero/releases/tag/0.1.1
node_id: RE_kwDOHilbp84ELAT4
tag_name: 0.1.1
target_commitish: main
name: 0.1.1
created_at: 2022-06-22T00:06:18Z
published_at: 2022-06-22T00:07:40Z
body:
  - Reduced log output when server exits. [#2](https://github.com/simonw/datasette-scale-to-zero/issues/2)


html_url: https://github.com/simonw/datasette-scale-to-zero/releases/tag/0.1.2
node_id: RE_kwDOHilbp84EMCgo
tag_name: 0.1.2
target_commitish: main
name: 0.1.2
created_at: 2022-06-23T00:05:34Z
published_at: 2022-06-23T00:06:21Z
body:
  - No longer logs a traceback on server exit. [#2](https://github.com/simonw/datasette-scale-to-zero/issues/2)


html_url: https://github.com/simonw/s3-ocr/releases/tag/0.1a0
node_id: RE_kwDOHk6Aq84EOJ27
tag_name: 0.1a0
target_commitish: main
name: 0.1a0
created_at: 2022-06-29T02:50:39Z
published_at: 2022-06-29T02:53:10Z
body:
  - `s3-ocr start <bucket>` command for triggering OCR runs using [Textract](https://aws.amazon.com/textract/) for every PDF file in a bucket. [#1](https://github.com/simonw/s3-ocr/issues/1)
  - `s3-ocr status <bucket>` command for checking on the status of the ongoing OCR tasks.


html_url: https://github.com/simonw/s3-ocr/releases/tag/0.2a0
node_id: RE_kwDOHk6Aq84EOd_O
tag_name: 0.2a0
target_commitish: main
name: 0.2a0
created_at: 2022-06-29T19:34:38Z
published_at: 2022-06-29T19:35:21Z
body:
   - New `s3-ocr index database.db name-of-bucket` command for creating a SQLite database containing the OCR results that have been written to the bucket. [#2](https://github.com/simonw/s3-ocr/issues/2)


html_url: https://github.com/simonw/s3-ocr/releases/tag/0.3
node_id: RE_kwDOHk6Aq84EOjkQ
tag_name: 0.3
target_commitish: main
name: 0.3
created_at: 2022-06-30T00:43:10Z
published_at: 2022-06-30T00:44:10Z
body:
  First non-alpha release.

  - Breaking change: the order of arguments for `s3-ocr index <bucket> <database_file>` has been swapped, for consistency with other commands. [#9](https://github.com/simonw/s3-ocr/issues/9)
  - Breaking change: the `start` command no longer defaults to processing every `.pdf` file in the bucket. It now accepts a list of keys, or use the `--all` option to process every PDF file. [#10](https://github.com/simonw/s3-ocr/issues/10)
  - New `s3-ocr fetch <bucket> <path>` command for fetching the raw OCR JSON data for that file. [#7](https://github.com/simonw/s3-ocr/issues/7)
  - New `s3-ocr text <bucket> <path>` command for outputting just the extracted OCR text for a specified file. [#8](https://github.com/simonw/s3-ocr/issues/8)


html_url: https://github.com/simonw/s3-credentials/releases/tag/0.12
node_id: RE_kwDOGUV9is4EO568
tag_name: 0.12
target_commitish: main
name: 0.12
created_at: 2022-06-30T20:00:19Z
published_at: 2022-06-30T20:02:53Z
body:
  - New `--statement JSON` option for both the `s3-credentials create` and `s3-credentials policy` commands, allowing one or more additional policy statements (provided as JSON strings) to be added to the generated IAM policy. [#72](https://github.com/simonw/s3-credentials/issues/72)


html_url: https://github.com/simonw/s3-ocr/releases/tag/0.4
node_id: RE_kwDOHk6Aq84EO7CV
tag_name: 0.4
target_commitish: main
name: 0.4
created_at: 2022-06-30T21:01:50Z
published_at: 2022-06-30T21:03:44Z
body:
  - New command: `s3-ocr inspect-job <job_id>` returns information about the status of a specific job. [#15](https://github.com/simonw/s3-ocr/issues/15)
  - Added a live demo at [s3-ocr-demo.datasette.io](https://s3-ocr-demo.datasette.io/). [#16](https://github.com/simonw/s3-ocr/issues/16)


html_url: https://github.com/simonw/datasette-edit-schema/releases/tag/0.5
node_id: RE_kwDODsD4Ds4EPQl-
tag_name: 0.5
target_commitish: main
name: 0.5
created_at: 2022-07-01T23:12:05Z
published_at: 2022-07-01T23:15:59Z
body:
  - More human-friendly labels for column types. [#29](https://github.com/simonw/datasette-edit-schema/issues/29)
  - `edit-schema` permission check now considers the database name. [#32](https://github.com/simonw/datasette-edit-schema/issues/32)
  - Now depends on `datasette>=0.59` [#30](https://github.com/simonw/datasette-edit-schema/issues/30) and `sqlite-utils>=3.10`. [#33](https://github.com/simonw/datasette-edit-schema/issues/33)


html_url: https://github.com/simonw/datasette-graphql/releases/tag/2.1
node_id: RE_kwDOEPNYIc4EPXJx
tag_name: 2.1
target_commitish: main
name: 2.1
created_at: 2022-07-03T01:17:11Z
published_at: 2022-07-03T01:19:01Z
body:
  - New [graphql_extra_fields() plugin hook](https://datasette.io/plugins/datasette-graphql#user-content-adding-custom-fields-with-plugins) allowing other plugins to add extra fields to the GraphQL schema. [#69](https://github.com/simonw/datasette-graphql/issues/69)


html_url: https://github.com/simonw/datasette-packages/releases/tag/0.2
node_id: RE_kwDOHENadM4EPXVQ
tag_name: 0.2
target_commitish: main
name: 0.2
created_at: 2022-07-03T02:27:31Z
published_at: 2022-07-03T02:29:02Z
body:
  - Now exposes a `packages` GraphQL field if you also have version 2.1 or higher of the [datasette-graphql plugin](https://datasette.io/plugins/datasette-graphql) installed. [#4](https://github.com/simonw/datasette-packages/issues/4)


html_url: https://github.com/simonw/datasette-upload-csvs/releases/tag/0.7.2
node_id: RE_kwDODf8oPs4EPb9Q
tag_name: 0.7.2
target_commitish: main
name: 0.7.2
created_at: 2022-07-03T20:58:03Z
published_at: 2022-07-03T20:59:48Z
body:
  - Fixed bug where encoding of file was not correctly detected if non-ASCII characters occurred after the first 2KB. The tool now inspects the first 2MB of content (as originally intended) and also upgrades ASCII to `latin-1` since ASCII is a complete subset of `latin-1` and using `latin-1` increases the chance of a successful import. [#25](https://github.com/simonw/datasette-upload-csvs/issues/25)


html_url: https://github.com/simonw/datasette-expose-env/releases/tag/0.1
node_id: RE_kwDOHmgSA84EPcGa
tag_name: 0.1
target_commitish: main
name: 0.1
created_at: 2022-07-03T21:26:23Z
published_at: 2022-07-03T21:28:55Z
body:
  - Initial release: configure this plugin to expose specific environment variables at `/-/env`. [#1](https://github.com/simonw/datasette-expose-env/issues/1)


html_url: https://github.com/simonw/datasette-redirect-to-https/releases/tag/0.2
node_id: RE_kwDOGaS1TM4EPdFu
tag_name: 0.2
target_commitish: main
name: 0.2
created_at: 2022-07-04T01:01:25Z
published_at: 2022-07-04T01:02:25Z
body:
  - Optional support for `x-forwarded-proto: http` headers, using a new configuration setting. [#2](https://github.com/simonw/datasette-redirect-to-https/issues/2)


html_url: https://github.com/simonw/sqlite-comprehend/releases/tag/0.1
node_id: RE_kwDOHoFAns4ERIOJ
tag_name: 0.1
target_commitish: main
name: 0.1
created_at: 2022-07-08T19:58:49Z
published_at: 2022-07-08T20:01:38Z
body:
  - Documentation, in the [README](https://github.com/simonw/sqlite-comprehend/blob/main/README.md). [#3](https://github.com/simonw/sqlite-comprehend/issues/3)
  - New schema design, with a separate `comprehend_entities` table. [#4](https://github.com/simonw/sqlite-comprehend/issues/4)
  - Running the command multiple times will not re-submit rows that have already been processed. [#7](https://github.com/simonw/sqlite-comprehend/issues/7)
  - Errors from the Comprehend API are now logged to standard error. [#8](https://github.com/simonw/sqlite-comprehend/issues/8)


html_url: https://github.com/simonw/sqlite-diffable/releases/tag/0.3
node_id: RE_kwDOC6Gvzs4ERKcf
tag_name: 0.3
target_commitish: main
name: 0.3
created_at: 2022-07-08T23:15:59Z
published_at: 2022-07-08T23:33:16Z
body:
  - New `sqlite-diffable load` command for restoring a previously dumped database. [#3](https://github.com/simonw/sqlite-diffable/issues/3)


html_url: https://github.com/simonw/sqlite-diffable/releases/tag/0.4
node_id: RE_kwDOC6Gvzs4ERmOY
tag_name: 0.4
target_commitish: main
name: 0.4
created_at: 2022-07-11T16:47:58Z
published_at: 2022-07-11T16:48:46Z
body:
  - `sqlite-diffable load ... --replace` option to replace tables that already exist in the destination database file. [#6](https://github.com/simonw/sqlite-diffable/issues/6)


html_url: https://github.com/simonw/sqlite-comprehend/releases/tag/0.2
node_id: RE_kwDOHoFAns4ERqDp
tag_name: 0.2
target_commitish: main
name: 0.2
created_at: 2022-07-11T19:11:57Z
published_at: 2022-07-11T19:13:02Z
body:
  - `sqlite-comprehend entities --strip-tags` option for stripping HTML tags before sending content to the entity extraction API. [#9](https://github.com/simonw/sqlite-comprehend/issues/9)


html_url: https://github.com/simonw/sqlite-comprehend/releases/tag/0.2.1
node_id: RE_kwDOHoFAns4ERqY9
tag_name: 0.2.1
target_commitish: main
name: 0.2.1
created_at: 2022-07-11T19:32:54Z
published_at: 2022-07-11T19:33:28Z
body:
  - Fixed bug where commands failed to be correctly registered on Linux systems. [#11](https://github.com/simonw/sqlite-comprehend/issues/11)


html_url: https://github.com/simonw/datasette-ics/releases/tag/0.5.2
node_id: RE_kwDODoa7Dc4ERvP9
tag_name: 0.5.2
target_commitish: main
name: 0.5.2
created_at: 2022-07-12T02:07:12Z
published_at: 2022-07-12T02:08:11Z
body:
  - Fix `ImportError` bug by upgrading to `ics` 0.7.2. [#6](https://github.com/simonw/datasette-ics/issues/6)


html_url: https://github.com/simonw/datasette-app/releases/tag/0.2.2
node_id: RE_kwDOF8z3PM4ESGTe
tag_name: 0.2.2
target_commitish: main
name: 0.2.2
created_at: 2022-07-13T00:18:05Z
published_at: 2022-07-13T00:20:06Z
body:
  - Fixed bug where application could crash on launch. Thanks, [Michael C](https://github.com/simonw/datasette-app-support/pull/20). [#139](https://github.com/simonw/datasette-app/issues/139)
  - Upgraded to Electron 19 and latest Datasette


html_url: https://github.com/simonw/datasette-app/releases/tag/0.2.2
node_id: RE_kwDOF8z3PM4ESGyA
tag_name: 0.2.2
target_commitish: main
name: 0.2.2
created_at: 2022-07-13T01:06:45Z
published_at: 2022-07-13T01:08:17Z
body:
  -   Fixed bug where application could crash on launch. Thanks, [Michael C](https://github.com/simonw/datasette-app-support/pull/20). [#139](https://github.com/simonw/datasette-app/issues/139)
  -   Upgraded to Electron 19 and latest Datasette


html_url: https://github.com/simonw/datasette-publish-vercel/releases/tag/0.14
node_id: RE_kwDODx4htM4ESV8s
tag_name: 0.14
target_commitish: main
name: 0.14
created_at: 2022-07-13T15:45:18Z
published_at: 2022-07-13T15:48:22Z
body:
  - Fixed bug caused by removal of `--no-clipboard` option in [Vercel 27.0.0](https://github.com/vercel/vercel/releases/tag/vercel%4027.0.0). [#57](https://github.com/simonw/datasette-publish-vercel/issues/57)
  - Errors that occur when running the `vercel` command are now reflected in the exit status of `datasette publish vercel`.


html_url: https://github.com/simonw/sqlite-utils/releases/tag/3.28
node_id: RE_kwDOCGYnMM4ETJ-y
tag_name: 3.28
target_commitish: main
name: 3.28
created_at: 2022-07-15T22:56:01Z
published_at: 2022-07-15T23:02:45Z
body:
  -   New [table.duplicate(new_name)](https://sqlite-utils.datasette.io/en/stable/python-api.html#python-api-duplicate) method for creating a copy of a table with a matching schema and row contents. Thanks, [David](https://github.com/davidleejy). ([#449](https://github.com/simonw/sqlite-utils/issues/449))
  -   New `sqlite-utils duplicate data.db table_name new_name` CLI command for [Duplicating tables](https://sqlite-utils.datasette.io/en/stable/cli.html#cli-duplicate-table). ([#454](https://github.com/simonw/sqlite-utils/issues/454))
  -   `sqlite_utils.utils.rows_from_file()` is now a [documented API](https://sqlite-utils.datasette.io/en/stable/reference.html#reference-utils-rows-from-file). It can be used to read a sequence of dictionaries from a file-like object containing CSV, TSV, JSON or newline-delimited JSON. It can be passed an explicit format or can attempt to detect the format automatically. ([#443](https://github.com/simonw/sqlite-utils/issues/443))
  -   `sqlite_utils.utils.TypeTracker` is now a documented API for detecting the likely column types for a sequence of string rows, see [Detecting column types using TypeTracker](https://sqlite-utils.datasette.io/en/stable/python-api.html#python-api-typetracker). ([#445](https://github.com/simonw/sqlite-utils/issues/445))
  -   `sqlite_utils.utils.chunks()` is now a documented API for [splitting an iterator into chunks](https://sqlite-utils.datasette.io/en/stable/reference.html#reference-utils-chunks). ([#451](https://github.com/simonw/sqlite-utils/issues/451))
  -   `sqlite-utils enable-fts` now has a `--replace` option for replacing the existing FTS configuration for a table. ([#450](https://github.com/simonw/sqlite-utils/issues/450))
  -   The `create-index`, `add-column` and `duplicate` commands all now take a `--ignore` option for ignoring errors should the database not be in the right state for them to operate. ([#450](https://github.com/simonw/sqlite-utils/issues/450))


html_url: https://github.com/simonw/datasette/releases/tag/0.62a1
node_id: RE_kwDOBm6k_c4ETXH1
tag_name: 0.62a1
target_commitish: main
name: 0.62a1
created_at: 2022-07-18T01:06:37Z
published_at: 2022-07-18T01:09:05Z
body:
  - New plugin hook: [handle_exception()](https://docs.datasette.io/en/stable/plugin_hooks.html#plugin-hook-handle-exception), for custom handling of exceptions caught by Datasette. ([#1770](https://github.com/simonw/datasette/issues/1770))
  - The [render_cell()](https://docs.datasette.io/en/stable/plugin_hooks.html#plugin-hook-render-cell) plugin hook is now also passed a `row` argument, representing the `sqlite3.Row` object that is being rendered. ([#1300](https://github.com/simonw/datasette/issues/1300))
  - New `--nolock` option for ignoring file locks when opening read-only databases. ([#1744](https://github.com/simonw/datasette/issues/1744))
  - Documentation now uses the [Furo](https://github.com/pradyunsg/furo) Sphinx theme. ([#1746](https://github.com/simonw/datasette/issues/1746))
  - Datasette now has a [Discord community](https://discord.gg/ktd74dm5mw).
  - Database file downloads now implement conditional GET using ETags. ([#1739](https://github.com/simonw/datasette/issues/1739))
  - Examples in the documentation now include a copy-to-clipboard button. ([#1748](https://github.com/simonw/datasette/issues/1748))
  - HTML for facet results and suggested results has been extracted out into new templates `_facet_results.html` and `_suggested_facets.html`. Thanks, M. Nasimul Haque. ([#1759](https://github.com/simonw/datasette/pull/1759))


html_url: https://github.com/simonw/datasette-sentry/releases/tag/0.2a0
node_id: RE_kwDODh5N084ETrWB
tag_name: 0.2a0
target_commitish: main
name: 0.2a0
created_at: 2022-07-18T20:29:03Z
published_at: 2022-07-18T20:30:04Z
body:
  - Use new `handle_exception()` hook added in Datasette 0.61a1. [#1](https://github.com/simonw/datasette-sentry/issues/1)


html_url: https://github.com/simonw/datasette-graphql/releases/tag/2.1.1
node_id: RE_kwDOEPNYIc4ETsA3
tag_name: 2.1.1
target_commitish: main
name: 2.1.1
created_at: 2022-07-18T21:11:20Z
published_at: 2022-07-18T21:13:34Z
body:
  - Simplified example code in the documentation for the `graphql_extra_fields()` hook.
  - Workaround for a bug where some tables could cause a 500 error on the regular table view page. [#90](https://github.com/simonw/datasette-graphql/issues/90)


html_url: https://github.com/simonw/s3-ocr/releases/tag/0.5
node_id: RE_kwDOHk6Aq84ETvH0
tag_name: 0.5
target_commitish: main
name: 0.5
created_at: 2022-07-19T02:31:21Z
published_at: 2022-07-19T02:35:36Z
body:
  - Ability to run OCR against just the PDF files contained within a specific folder in the S3 bucket, using `s3-ocr start my-bucket --prefix my-prefix/`. [#20](https://github.com/simonw/s3-ocr/issues/20)
  - New command: `s3-ocr dedupe my-bucket` - scans the bucket for any new files that are duplicates of files that have already been OCRd and writes out job results to reuse existing OCR results and avoid processing them a second time in the future. [#19](https://github.com/simonw/s3-ocr/issues/19)


html_url: https://github.com/simonw/datasette-insert/releases/tag/0.8
node_id: RE_kwDOELgXO84EVF5o
tag_name: 0.8
target_commitish: main
name: 0.8
created_at: 2022-07-22T17:50:41Z
published_at: 2022-07-22T17:52:23Z
body:
  - Now serves CORS headers if Datasette is started with the `--cors` option. [#17](https://github.com/simonw/datasette-insert/issues/17)


html_url: https://github.com/simonw/datasette-publish-vercel/releases/tag/0.14.1
node_id: RE_kwDODx4htM4EVMCO
tag_name: 0.14.1
target_commitish: main
name: 0.14.1
created_at: 2022-07-23T02:57:11Z
published_at: 2022-07-23T02:58:36Z
body:
  - Fix for 500 error when run with latest `@vercel/python` [#58](https://github.com/simonw/datasette-publish-vercel/issues/58)


html_url: https://github.com/simonw/datasette-sqlite-fts4/releases/tag/0.3.1
node_id: RE_kwDOCcM_5s4EWzKC
tag_name: 0.3.1
target_commitish: master
name: 0.3.1
created_at: 2022-07-28T16:41:34Z
published_at: 2022-07-28T16:47:54Z
body:
  - Now depends on [sqlite-fts4 1.0.2](https://github.com/simonw/sqlite-fts4/releases/tag/1.0.2) for compatibility with big-endian systems.


html_url: https://github.com/simonw/shot-scraper/releases/tag/0.14.2
node_id: RE_kwDOG-A5W84EW1QM
tag_name: 0.14.2
target_commitish: main
name: 0.14.2
created_at: 2022-07-28T18:31:24Z
published_at: 2022-07-28T18:32:10Z
body:
  - Documentation improvements. [0.14.1...0.14.2](https://github.com/simonw/shot-scraper/compare/0.14.1...0.14.2)


html_url: https://github.com/simonw/s3-credentials/releases/tag/0.12.1
node_id: RE_kwDOGUV9is4EX5xN
tag_name: 0.12.1
target_commitish: main
name: 0.12.1
created_at: 2022-08-01T19:45:53Z
published_at: 2022-08-01T19:50:53Z
body:
  - Using the `--policy` or `--statement` options now implies `--user-permissions-boundary=none`. Previously it was easy to use these options to accidentally create credentials that did not work as expected since they would have a default permissions boundary that locked them down to only being able to access S3. [#74](https://github.com/simonw/s3-credentials/issues/74)
  - The `s3-credentials.AmazonS3FullAccess` role created by this tool in order to issue temporary credentials previously used the default `MaxSessionDuration` value of 3600, preventing it from creating credentials that could last more than an hour. This has been increased to 12 hours. See [this issue comment](https://github.com/simonw/s3-credentials/issues/75#issuecomment-1201643771) for instructions on fixing your existing role if this bug is affecting your account. [#75](https://github.com/simonw/s3-credentials/issues/75)


html_url: https://github.com/simonw/shot-scraper/releases/tag/0.14.3
node_id: RE_kwDOG-A5W84EYVrr
tag_name: 0.14.3
target_commitish: main
name: 0.14.3
created_at: 2022-08-02T23:40:20Z
published_at: 2022-08-02T23:41:37Z
body:
  - Improved example workflow in [Optimizing PNGs using Oxipng](https://shot-scraper.datasette.io/en/stable/github-actions.html#optimizing-pngs-using-oxipng).
  - Fixed typos in README and documentation. [#83](https://github.com/simonw/shot-scraper/pull/83)


html_url: https://github.com/simonw/datasette-scale-to-zero/releases/tag/0.2
node_id: RE_kwDOHilbp84EZf3-
tag_name: 0.2
target_commitish: main
name: 0.2
created_at: 2022-08-05T22:28:12Z
published_at: 2022-08-05T22:29:17Z
body:
  - New `"max-age": "10h"` configuration setting, which causes the server to exit after the specified amount of time whether or not it has received any traffic. [#3](https://github.com/simonw/datasette-scale-to-zero/issues/3)


html_url: https://github.com/simonw/s3-ocr/releases/tag/0.6
node_id: RE_kwDOHk6Aq84EZq2d
tag_name: 0.6
target_commitish: main
name: 0.6
created_at: 2022-08-07T17:38:41Z
published_at: 2022-08-07T17:42:07Z
body:
  - `s3-ocr start` now automatically pauses and then retries if Textract complains that there are too many jobs running. This can be turned into an early exit with an error message using the new `--no-retry` option. [#21](https://github.com/simonw/s3-ocr/issues/21)
  - New `s3-ocr start --dry-run` option for displaying what would happen without starting the OCR process. [#22](https://github.com/simonw/s3-ocr/issues/22)
  - Textract now runs in the same region as the S3 bucket it is writing to, avoiding an error. [#24](https://github.com/simonw/s3-ocr/issues/24)


html_url: https://github.com/hydrosquall/datasette-nteract-data-explorer/releases/tag/0.4.1
node_id: RE_kwDOHEflbM4EaAHe
tag_name: 0.4.1
target_commitish: main
name: 0.4.1 - Enable permalinks to graphs
created_at: 2022-08-08T19:59:34Z
published_at: 2022-08-08T20:02:05Z
body:
  ## What's Changed
  * feat: add demo site landing page by @hydrosquall in https://github.com/hydrosquall/datasette-nteract-data-explorer/pull/14
  * feature: synchronize graph state to URL parameters, enabling permalinking + sharing graphs by @hydrosquall in https://github.com/hydrosquall/datasette-nteract-data-explorer/pull/19
  * Added PR templates + Issue contributing guide

  **Full Changelog**: https://github.com/hydrosquall/datasette-nteract-data-explorer/compare/0.3.1...0.4.1


html_url: https://github.com/simonw/s3-ocr/releases/tag/0.6.1
node_id: RE_kwDOHk6Aq84EaaWy
tag_name: 0.6.1
target_commitish: main
name: 0.6.1
created_at: 2022-08-09T19:39:05Z
published_at: 2022-08-09T19:40:19Z
body:
  - Now pins to `click>=8.0`, which should avoid a bug where installing this on a machine with an older version of Click present would lead to the commands failing to register. [#25](https://github.com/simonw/s3-ocr/issues/25)
  - `s3-ocr --help` now includes links to the documentation and changelog.


html_url: https://github.com/simonw/s3-ocr/releases/tag/0.6.2
node_id: RE_kwDOHk6Aq84Eabg7
tag_name: 0.6.2
target_commitish: main
name: 0.6.2
created_at: 2022-08-09T20:34:49Z
published_at: 2022-08-09T20:35:32Z
body:
  - Fixed bug where commands were sometimes not properly registered. [#26](https://github.com/simonw/s3-ocr/issues/26)


html_url: https://github.com/simonw/s3-ocr/releases/tag/0.6.3
node_id: RE_kwDOHk6Aq84EahSc
tag_name: 0.6.3
target_commitish: main
name: 0.6.3
created_at: 2022-08-10T04:42:11Z
published_at: 2022-08-10T04:43:17Z
body:
  - Pages with no OCR text on them are now recorded as rows with empty strings, instead of being skipped entirely. [#23](https://github.com/simonw/s3-ocr/issues/23)


html_url: https://github.com/simonw/s3-credentials/releases/tag/0.13
node_id: RE_kwDOGUV9is4EbhH5
tag_name: 0.13
target_commitish: main
name: 0.13
created_at: 2022-08-12T18:16:30Z
published_at: 2022-08-12T18:21:02Z
body:
  - Documentation now lives on a dedicated documentation website: https://s3-credentials.readthedocs.io/ [#71](https://github.com/simonw/s3-credentials/issues/71)
  - `s3-credentials create ... --website --create-bucket` now creates an S3 bucket that is configured to act as a website, with `index.html` an the index page and `error.html` as the page used for any errors. [#21](https://github.com/simonw/s3-credentials/issues/21)
  - `s3-credentials list-buckets --details` now returns the bucket region and the URL to the website, if it is configured to act as a website. [#77](https://github.com/simonw/s3-credentials/issues/77)
  - Fixed a bug where `list-bucket` would return an error if the bucket (or specified `--prefix`) was empty. [#76](https://github.com/simonw/s3-credentials/issues/76)


html_url: https://github.com/simonw/datasette/releases/tag/0.62
node_id: RE_kwDOBm6k_c4EbwbF
tag_name: 0.62
target_commitish: main
name: 0.62
created_at: 2022-08-14T17:32:42Z
published_at: 2022-08-14T17:43:05Z
body:
  Datasette can now run entirely in your browser using WebAssembly. Try out [Datasette Lite](https://lite.datasette.io/), take a look [at the code](https://github.com/simonw/datasette-lite) or read more about it in [Datasette Lite: a server-side Python web application running in a browser](https://simonwillison.net/2022/May/4/datasette-lite/).

  Datasette now has a [Discord community](https://discord.gg/ktd74dm5mw) for questions and discussions about Datasette and its ecosystem of projects.

  ## Features

  -   Datasette is now compatible with [Pyodide](https://pyodide.org/). This is the enabling technology behind [Datasette Lite](https://lite.datasette.io/). ([#1733](https://github.com/simonw/datasette/issues/1733))
  -   Database file downloads now implement conditional GET using ETags. ([#1739](https://github.com/simonw/datasette/issues/1739))
  -   HTML for facet results and suggested results has been extracted out into new templates `_facet_results.html` and `_suggested_facets.html`. Thanks, M. Nasimul Haque. ([#1759](https://github.com/simonw/datasette/pull/1759))
  -   Datasette now runs some SQL queries in parallel. This has limited impact on performance, see [this research issue](https://github.com/simonw/datasette/issues/1727) for details.
  -   New `--nolock` option for ignoring file locks when opening read-only databases. ([#1744](https://github.com/simonw/datasette/issues/1744))
  -   Spaces in the database names in URLs are now encoded as `+` rather than `~20`. ([#1701](https://github.com/simonw/datasette/issues/1701))
  -   `<Binary: 2427344 bytes>` is now displayed as `<Binary: 2,427,344 bytes>` and is accompanied by tooltip showing "2.3MB". ([#1712](https://github.com/simonw/datasette/issues/1712))
  -   The base Docker image used by `datasette publish cloudrun`, `datasette package` and the [official Datasette image](https://hub.docker.com/datasetteproject/datasette) has been upgraded to `3.10.6-slim-bullseye`. ([#1768](https://github.com/simonw/datasette/issues/1768))
  -   Canned writable queries against immutable databases now show a warning message. ([#1728](https://github.com/simonw/datasette/issues/1728))
  -   `datasette publish cloudrun` has a new `--timeout` option which can be used to increase the time limit applied by the Google Cloud build environment. Thanks, Tim Sherratt. ([#1717](https://github.com/simonw/datasette/pull/1717))
  -   `datasette publish cloudrun` has new `--min-instances` and `--max-instances` options. ([#1779](https://github.com/simonw/datasette/issues/1779))

  ## Plugin hooks

  -   New plugin hook: [handle_exception()](https://docs.datasette.io/en/stable/plugin_hooks.html#plugin-hook-handle-exception), for custom handling of exceptions caught by Datasette. ([#1770](https://github.com/simonw/datasette/issues/1770))
  -   The [render_cell()](https://docs.datasette.io/en/stable/plugin_hooks.html#plugin-hook-render-cell) plugin hook is now also passed a `row` argument, representing the `sqlite3.Row` object that is being rendered. ([#1300](https://github.com/simonw/datasette/issues/1300))
  -   The [configuration directory](https://docs.datasette.io/en/stable/settings.html#config-dir) is now stored in `datasette.config_dir`, making it available to plugins. Thanks, Chris Amico. ([#1766](https://github.com/simonw/datasette/pull/1766))

  ## Bug fixes

  -   Don't show the facet option in the cog menu if faceting is not allowed. ([#1683](https://github.com/simonw/datasette/issues/1683))
  -   `?_sort` and `?_sort_desc` now work if the column that is being sorted has been excluded from the query using `?_col=` or `?_nocol=`. ([#1773](https://github.com/simonw/datasette/issues/1773))
  -   Fixed bug where `?_sort_desc` was duplicated in the URL every time the Apply button was clicked. ([#1738](https://github.com/simonw/datasette/issues/1738))

  ## Documentation

  -   Examples in the documentation now include a copy-to-clipboard button. ([#1748](https://github.com/simonw/datasette/issues/1748))
  -   Documentation now uses the [Furo](https://github.com/pradyunsg/furo) Sphinx theme. ([#1746](https://github.com/simonw/datasette/issues/1746))
  -   Code examples in the documentation are now all formatted using Black. ([#1718](https://github.com/simonw/datasette/issues/1718))
  -   `Request.fake()` method is now documented, see [Request object](https://docs.datasette.io/en/stable/internals.html#internals-request).
  -   New documentation for plugin authors: [Registering a plugin for the duration of a test](https://docs.datasette.io/en/stable/testing_plugins.html#testing-plugins-register-in-test). ([#903](https://github.com/simonw/datasette/issues/903))

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/releases/74385093/reactions", "total_count": 2, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 2, "rocket": 0, "eyes": 0}


html_url: https://github.com/simonw/datasette-sentry/releases/tag/0.2
node_id: RE_kwDODh5N084Ebx0-
tag_name: 0.2
target_commitish: main
name: 0.2
created_at: 2022-08-14T22:58:44Z
published_at: 2022-08-14T23:00:36Z
body:
  - Now uses the new [handle_exception()](https://docs.datasette.io/en/stable/plugin_hooks.html#handle-exception-datasette-request-exception) plugin hook introduced in [Datasette 0.62](https://docs.datasette.io/en/stable/changelog.html#v0-62). [#3](https://github.com/simonw/datasette-sentry/issues/3)


html_url: https://github.com/simonw/sqlite-diffable/releases/tag/0.5
node_id: RE_kwDOC6Gvzs4EdRqX
tag_name: 0.5
target_commitish: main
name: 0.5
created_at: 2022-08-18T22:47:58Z
published_at: 2022-08-18T22:49:30Z
body:
  - `sqlite-diffable objects path-to/table.ndjson` command for converting a newline-delimited file of JSON arrays into a sequence of JSON objects. [#7](https://github.com/simonw/sqlite-diffable/issues/7)


html_url: https://github.com/simonw/datasette-mp3-audio/releases/tag/0.1
node_id: RE_kwDOH2kMZM4EdwwE
tag_name: 0.1
target_commitish: main
name: 0.1
created_at: 2022-08-20T16:38:46Z
published_at: 2022-08-20T16:45:42Z
body:
  - Initial release. Adds `<audio>` players to any table cell containing the URL to a `.mp3` file. [#1](https://github.com/simonw/datasette-mp3-audio/issues/1)


html_url: https://github.com/simonw/datasette-mp3-audio/releases/tag/0.2
node_id: RE_kwDOH2kMZM4EdyNt
tag_name: 0.2
target_commitish: main
name: 0.2
created_at: 2022-08-20T23:14:51Z
published_at: 2022-08-20T23:14:56Z
body:
  - Added a "Play all MP3s" button to any table page listing more than one MP3. [#2](https://github.com/simonw/datasette-mp3-audio/issues/2)


html_url: https://github.com/dogsheep/pocket-to-sqlite/releases/tag/0.2.2
node_id: RE_kwDODLZ_YM4EeJML
tag_name: 0.2.2
target_commitish: main
name: 0.2.2
created_at: 2022-08-22T16:18:16Z
published_at: 2022-08-22T16:21:34Z
body:
  - `-a` is now the shortcut for `--auth` - previously it was duplicated as a shortcut for `--all`. Thanks, [Thomas Lockney](https://github.com/tlockney).  [#7](https://github.com/dogsheep/pocket-to-sqlite/issues/7)
  - `pocket-to-sqlite auth` command now updates the existing `auth.json` file rather than over-writing it. [#10](https://github.com/dogsheep/pocket-to-sqlite/issues/10)


html_url: https://github.com/simonw/datasette-edit-schema/releases/tag/0.5.1
node_id: RE_kwDODsD4Ds4EeP4S
tag_name: 0.5.1
target_commitish: main
name: 0.5.1
created_at: 2022-08-22T22:45:03Z
published_at: 2022-08-22T22:45:58Z
body:
  - Fixed error trying to add new floating point columns to a table. [#34](https://github.com/simonw/datasette-edit-schema/issues/34)


html_url: https://github.com/simonw/datasette-publish-vercel/releases/tag/0.14.2
node_id: RE_kwDODx4htM4EfFfQ
tag_name: 0.14.2
target_commitish: main
name: 0.14.2
created_at: 2022-08-24T17:42:15Z
published_at: 2022-08-24T17:43:41Z
body:
  - Fixed bug where using boolean settings such as `--setting allow_downloads 0` would cause deployments to serve 500 errors.  [#61](https://github.com/simonw/datasette-publish-vercel/issues/61)


html_url: https://github.com/hydrosquall/datasette-nteract-data-explorer/releases/tag/0.4.2
node_id: RE_kwDOHEflbM4EgMBS
tag_name: 0.4.2
target_commitish: main
name: 0.4.2 - Datasette-Lite hook
created_at: 2022-08-27T16:17:37Z
published_at: 2022-08-27T16:31:11Z
body:
  ## What's Changed
  * fix(demo): build updated demo JS statics as part of build pipeline by @hydrosquall in https://github.com/hydrosquall/datasette-nteract-data-explorer/pull/24
  * feat: proof of concept initialization hook for instantiating JS plugins in datasette-lite by @hydrosquall in https://github.com/hydrosquall/datasette-nteract-data-explorer/pull/26
  * bump: version 0.4.2 by @hydrosquall in https://github.com/hydrosquall/datasette-nteract-data-explorer/pull/27


  **Full Changelog**: https://github.com/hydrosquall/datasette-nteract-data-explorer/compare/0.4.1...0.4.2


html_url: https://github.com/simonw/sqlite-utils/releases/tag/3.29
node_id: RE_kwDOCGYnMM4EgPTo
tag_name: 3.29
target_commitish: main
name: 3.29
created_at: 2022-08-28T03:48:36Z
published_at: 2022-08-28T03:50:29Z
body:
  - The `sqlite-utils query`, `memory` and `bulk` commands now all accept a new `--functions` option. This can be passed a string of Python code, and any callable objects defined in that code will be made available to SQL queries as custom SQL functions. See [Defining custom SQL functions](https://sqlite-utils.datasette.io/en/stable/cli.html#cli-query-functions) for details. ([#471](https://github.com/simonw/sqlite-utils/issues/471))
  - `db[table].create(...)` method now accepts a new `transform=True` parameter. If the table already exists it will be transform to match the schema configuration options passed to the function. This may result in columns being added or dropped, column types being changed, column order being updated or not null and default values for columns being set. ([#467](https://github.com/simonw/sqlite-utils/issues/467))
  - Related to the above, the `sqlite-utils create-table` command now accepts a `--transform` option.
  - New introspection property: `table.default_values` returns a dictionary mapping each column name with a default value to the configured default value. ([#475](https://github.com/simonw/sqlite-utils/issues/475))
  - The `--load-extension` option can now be provided a path to a compiled SQLite extension module accompanied by the name of an entrypoint, separated by a colon - for example `--load-extension ./lines0:sqlite3_lines0_noread_init`. This feature is modelled on code first [contributed to Datasette](https://github.com/simonw/datasette/pull/1789) by Alex Garcia. ([#470](https://github.com/simonw/sqlite-utils/issues/470))
  - Functions registered using the [db.register_function()](https://sqlite-utils.datasette.io/en/stable/python-api.html#python-api-register-function) method can now have a custom name specified using the new `db.register_function(fn, name=...)` parameter. ([#458](https://github.com/simonw/sqlite-utils/issues/458))
  - [sqlite-utils rows](https://sqlite-utils.datasette.io/en/stable/cli.html#cli-rows) has a new `--order` option for specifying the sort order for the returned rows. ([#469](https://github.com/simonw/sqlite-utils/issues/469))
  - All of the CLI options that accept Python code blocks can now all be used to define functions that can access modules imported in that same block of code without needing to use the `global` keyword. ([#472](https://github.com/simonw/sqlite-utils/issues/472))
  - Fixed bug where `table.extract()` would not behave correctly for columns containing null values. Thanks, Forest Gregg. ([#423](https://github.com/simonw/sqlite-utils/issues/423))
  - New tutorial: [Cleaning data with sqlite-utils and Datasette](https://datasette.io/tutorials/clean-data) shows how to use `sqlite-utils` to import and clean an example CSV file.
  - Datasette and `sqlite-utils` now have a Discord community. [Join the Discord here](https://discord.gg/Ass7bCAMDw).

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/releases/75560168/reactions", "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0}


html_url: https://github.com/simonw/datasette-sitemap/releases/tag/0.1
node_id: RE_kwDOH50_7s4Egw44
tag_name: 0.1
target_commitish: main
name: 0.1
created_at: 2022-08-29T22:18:35Z
published_at: 2022-08-29T22:19:49Z
body:
  - Initial release: configure a SQL query to return a `/sitemap.xml` for your site. [#1](https://github.com/simonw/datasette-sitemap/issues/1)


html_url: https://github.com/simonw/datasette-block-robots/releases/tag/1.1
node_id: RE_kwDOEFljXc4EhJ2H
tag_name: 1.1
target_commitish: main
name: 1.1
created_at: 2022-08-30T16:23:32Z
published_at: 2022-08-30T16:25:38Z
body:
  - Now registers a new [block_robots_extra_lines() plugin hook](https://github.com/simonw/datasette-block-robots/blob/1.1/README.md#extending-this-with-other-plugins) which other plugins can use to contribute to `/robots.txt`. [#6](https://github.com/simonw/datasette-block-robots/issues/6)


html_url: https://github.com/simonw/datasette-sitemap/releases/tag/1.0
node_id: RE_kwDOH50_7s4EhME_
tag_name: 1.0
target_commitish: main
name: 1.0
created_at: 2022-08-30T17:56:48Z
published_at: 2022-08-30T17:59:53Z
body:
  - This plugin now interacts correctly with the [datasette-block-robots](https://datasette.io/plugins/datasette-block-robots) plugin, adding the `Sitemap:` reference to the existing `robots.txt` file if that other plugin is installed. [#4](https://github.com/simonw/datasette-sitemap/issues/4)
  - Now registers a new [sitemap_extra_paths() plugin hook](https://github.com/simonw/datasette-sitemap/blob/1.0/README.md#adding-paths-to-the-sitemap-from-other-plugins) which other plugins can use to contribute to `/sitemap.xml`. [#5](https://github.com/simonw/datasette-sitemap/issues/5)


html_url: https://github.com/simonw/datasette-render-image-tags/releases/tag/0.1
node_id: RE_kwDOH7vZQ84Eivrv
tag_name: 0.1
target_commitish: main
name: 0.1
created_at: 2022-09-04T00:47:07Z
published_at: 2022-09-04T00:48:32Z
body:
  - Initial release. Cells containing a URL that end in an image extension will be converted into image tags with a width of 200 pixels.


html_url: https://github.com/simonw/datasette-upload-csvs/releases/tag/0.8
node_id: RE_kwDODf8oPs4EkSIA
tag_name: 0.8
target_commitish: main
name: 0.8
created_at: 2022-09-08T17:04:08Z
published_at: 2022-09-08T17:07:41Z
body:
  - The user now gets to confirm the name of the table to be created before uploading the file. [#17](https://github.com/simonw/datasette-upload-csvs/issues/17)
  - Column types are automatically detected based on the uploaded data. Previously all columns were TEXT columns. [#26](https://github.com/simonw/datasette-upload-csvs/issues/26)
  - If a table with the desired name already exists, a suffix of `_2` or higher is appended. [#29](https://github.com/simonw/datasette-upload-csvs/issues/29)
  - Fixed a bug where usage alongside a custom `base.html` template with `extra_head` content would be ignored. [#27](https://github.com/simonw/datasette-upload-csvs/issues/27)


html_url: https://github.com/simonw/datasette-upload-csvs/releases/tag/0.8.1
node_id: RE_kwDODf8oPs4EkWYG
tag_name: 0.8.1
target_commitish: main
name: 0.8.1
created_at: 2022-09-08T19:50:41Z
published_at: 2022-09-08T19:51:53Z
body:
  - Fixed compatibility bug where running this plugin at the same time as [datasette-upload-dbs](https://datasette.io/plugins/datasette-upload-dbs) could trigger errors. [#30](https://github.com/simonw/datasette-upload-csvs/issues/30)


html_url: https://github.com/simonw/datasette-upload-dbs/releases/tag/0.1.2
node_id: RE_kwDOHSfuMs4EktwB
tag_name: 0.1.2
target_commitish: main
name: 0.1.2
created_at: 2022-09-09T15:57:58Z
published_at: 2022-09-09T15:58:38Z
body:
  - Uploaded databases are now treated as mutable - previously they were incorrectly configured to be immutable. [#6](https://github.com/simonw/datasette-upload-dbs/issues/6)


html_url: https://github.com/simonw/shot-scraper/releases/tag/0.15
node_id: RE_kwDOG-A5W84Elfy_
tag_name: 0.15
target_commitish: main
name: 0.15
created_at: 2022-09-12T20:56:20Z
published_at: 2022-09-12T20:57:07Z
body:
  - New [--log-requests option](https://shot-scraper.datasette.io/en/stable/screenshots.html#logging-all-requests) for logging out newline-delimited JSON showing all requests made while rendering a page, including image and asset loads. [#88](https://github.com/simonw/shot-scraper/issues/88)
  - New [shot-scraper pdf](https://shot-scraper.datasette.io/en/stable/pdf.html) formatting options: `--format`, `--width`, `--height`, `--scale` and `--print-background`. Thanks, Eddie Chapman. [#87](https://github.com/simonw/shot-scraper/pull/87)
  - Removed `-h` shortcut for help - use `--help` instead. `-h` was clashing with the shorter version of `--height`. Thanks, Matthew Bafford. [#84](https://github.com/simonw/shot-scraper/pull/84)


html_url: https://github.com/simonw/shot-scraper/releases/tag/0.15.1
node_id: RE_kwDOG-A5W84El2ZJ
tag_name: 0.15.1
target_commitish: main
name: 0.15.1
created_at: 2022-09-13T17:14:41Z
published_at: 2022-09-13T17:15:53Z
body:
  - Fix error message shown when `--log-requests` tries to log a request with an unknown response size. [#89](https://github.com/simonw/shot-scraper/issues/89)


html_url: https://github.com/simonw/shot-scraper/releases/tag/0.16
node_id: RE_kwDOG-A5W84Emm6W
tag_name: 0.16
target_commitish: main
name: 0.16
created_at: 2022-09-15T03:16:51Z
published_at: 2022-09-15T03:18:44Z
body:
  - Fixed error message displayed when attempting to use `--log-requests` on a page that includes a redirect. [#90](https://github.com/simonw/shot-scraper/issues/90)
  - `--log-requests` output now includes a `"status": 200` field indicating the HTTP status code returned by that response.


html_url: https://github.com/simonw/s3-credentials/releases/tag/0.14
node_id: RE_kwDOGUV9is4EnDXL
tag_name: 0.14
target_commitish: main
name: 0.14
created_at: 2022-09-15T23:40:59Z
published_at: 2022-09-15T23:43:10Z
body:
  - `s3-credentials put-objects` command ([docs](https://s3-credentials.readthedocs.io/en/stable/other-commands.html#put-objects)) for uploading more than one file or directory to an S3 bucket at a time. [#68](https://github.com/simonw/s3-credentials/issues/68)
  - `s3-credentials get-objects` command ([docs](https://s3-credentials.readthedocs.io/en/stable/other-commands.html#get-objects)) for downloading multiple files from an S3 bucket. [#78](https://github.com/simonw/s3-credentials/issues/78)


html_url: https://github.com/simonw/datasette-sandstorm-support/releases/tag/0.2
node_id: RE_kwDOH99rF84EnUm-
tag_name: 0.2
target_commitish: main
name: 0.2
created_at: 2022-09-16T17:12:49Z
published_at: 2022-09-16T17:14:12Z
body:
  - Additional permissions. Thanks, [@ocdtrekkie](https://github.com/ocdtrekkie). [#3](https://github.com/simonw/datasette-sandstorm-support/pull/3)


html_url: https://github.com/simonw/datasette/releases/tag/0.63a0
node_id: RE_kwDOBm6k_c4EqTKK
tag_name: 0.63a0
target_commitish: main
name: 0.63a0
created_at: 2022-09-26T21:14:25Z
published_at: 2022-09-26T21:56:30Z
body:
  - The [prepare_jinja2_environment(env, datasette)](https://docs.datasette.io/en/latest/plugin_hooks.html#plugin-hook-prepare-jinja2-environment) plugin hook now accepts an optional `datasette` argument. Hook implementations can also now return an `async` function which will be awaited automatically. ([#1809](https://github.com/simonw/datasette/issues/1809))
  - `--load-extension` option now supports entrypoints. Thanks, Alex Garcia. ([#1789](https://github.com/simonw/datasette/pull/1789))
  - New tutorial: [Cleaning data with sqlite-utils and Datasette](https://datasette.io/tutorials/clean-data).
  - Facet size can now be set per-table with the new `facet_size` table metadata option. ([#1804](https://github.com/simonw/datasette/issues/1804))
  - `truncate_cells_html` setting now also affects long URLs in columns. ([#1805](https://github.com/simonw/datasette/issues/1805))
  - `Database(is_mutable=)` now defaults to `True`. ([#1808](https://github.com/simonw/datasette/issues/1808))
  - Non-JavaScript textarea now increases height to fit the SQL query. ([#1786](https://github.com/simonw/datasette/issues/1786))
  - More detailed command descriptions on the [CLI reference](https://docs.datasette.io/en/latest/cli-reference.html#cli-reference) page. ([#1787](https://github.com/simonw/datasette/issues/1787))
  - Datasette no longer enforces upper bounds on its depenedencies. ([#1800](https://github.com/simonw/datasette/issues/1800))
  - Facets are now displayed with better line-breaks in long values. Thanks, Daniel Rech. ([#1794](https://github.com/simonw/datasette/pull/1794))
  - The `settings.json` file used in [Configuration directory mode](https://docs.datasette.io/en/latest/settings.html#config-dir) is now validated on startup. ([#1816](https://github.com/simonw/datasette/issues/1816))


html_url: https://github.com/simonw/datasette-pretty-json/releases/tag/0.2.2
node_id: RE_kwDOCf9bl84Erd7d
tag_name: 0.2.2
target_commitish: master
name: 0.2.2
created_at: 2022-09-28T21:05:21Z
published_at: 2022-09-28T21:06:31Z
body:
  - Fix for compatibility with latest Jinja. [#2](https://github.com/simonw/datasette-pretty-json/issues/2)


html_url: https://github.com/simonw/datasette-publish-fly/releases/tag/1.2
node_id: RE_kwDODs4PE84Er5IS
tag_name: 1.2
target_commitish: main
name: 1.2
created_at: 2022-09-29T17:23:42Z
published_at: 2022-09-29T17:25:15Z
body:
  - New `-o/--org` option for specifying the Fly organization to deploy to - defaults to "personal". [#22](https://github.com/simonw/datasette-publish-fly/issues/22)
  - Fixed bug where port no needs to be specified as an integer. [#21](https://github.com/simonw/datasette-publish-fly/issues/21)


html_url: https://github.com/simonw/datasette-public/releases/tag/0.1
node_id: RE_kwDOIIe2Hs4EtZLc
tag_name: 0.1
target_commitish: main
name: 0.1
created_at: 2022-10-05T00:02:42Z
published_at: 2022-10-05T00:05:59Z
body:
  - Initial release. Provides a mechanism for toggling specific tables public even if the overall Datasette instance is private. [#1](https://github.com/simonw/datasette-public/issues/1)


html_url: https://github.com/simonw/datasette-search-all/releases/tag/1.1
node_id: RE_kwDODqd5284EtaMg
tag_name: 1.1
target_commitish: main
name: 1.1
created_at: 2022-10-05T01:52:37Z
published_at: 2022-10-05T01:53:33Z
body:
  - Plugin now checks table permissions, and only attempts to search tables that the user has permission to view. [#16](https://github.com/simonw/datasette-search-all/issues/16)
  - Now requires Datasette 0.61 or higher


html_url: https://github.com/simonw/datasette-sentry/releases/tag/0.3
node_id: RE_kwDODh5N084EuL5i
tag_name: 0.3
target_commitish: main
name: 0.3
created_at: 2022-10-06T22:29:42Z
published_at: 2022-10-06T22:31:29Z
body:
  - Sentry errors now include information about the HTTP request again. [#4](https://github.com/simonw/datasette-sentry/issues/4)
  - New `sample_rate` plugin configuration option for setting the [Sentry sample rate](https://docs.sentry.io/platforms/python/configuration/sampling/) - accepts a floating point number between 0.0 and 1.0. [#2](https://github.com/simonw/datasette-sentry/issues/2)


html_url: https://github.com/simonw/datasette-public/releases/tag/0.2
node_id: RE_kwDOIIe2Hs4EukjZ
tag_name: 0.2
target_commitish: main
name: 0.2
created_at: 2022-10-07T22:33:04Z
published_at: 2022-10-07T22:34:48Z
body:
  - Fix compatibility bug when installed alongside `datasette-leaflet-geojson`. [#4](https://github.com/simonw/datasette-public/issues/4)
  - The `execute-sql` permission check now checks to see if the user has the `view-database` permission. [#3](https://github.com/simonw/datasette-public/issues/3)


html_url: https://github.com/simonw/shot-scraper/releases/tag/0.17
node_id: RE_kwDOG-A5W84ExAab
tag_name: 0.17
target_commitish: main
name: 0.17
created_at: 2022-10-14T19:30:50Z
published_at: 2022-10-14T19:32:23Z
body:
  - New `shot-scraper multi -o` option for specifying a subset of one or more output files to execute that are defined in the YAML. This is useful for testing a larger `shots.yml` file without re-taking every screenshot every time the command is run. [#94](https://github.com/simonw/shot-scraper/issues/94)


html_url: https://github.com/simonw/shot-scraper/releases/tag/1.0
node_id: RE_kwDOG-A5W84ExKln
tag_name: 1.0
target_commitish: main
name: 1.0
created_at: 2022-10-15T19:45:04Z
published_at: 2022-10-15T19:46:37Z
body:
  - New `shot-scraper html URL` command ([documented here](https://shot-scraper.datasette.io/en/stable/html.html)) for outputting the final HTML of a page, after JavaScript has been executed. [#96](https://github.com/simonw/shot-scraper/issues/96)
  - `shot-scraper javascript` has a new `-r/--raw` option for outputting the result of the JavaScript expression as a raw string rather than JSON encoded ([shot-scraper javascript documentation](https://shot-scraper.datasette.io/en/stable/javascript.html)). [#95](https://github.com/simonw/shot-scraper/issues/95)
  - Tutorial: [Automating screenshots for the Datasette documentation using shot-scraper](https://simonwillison.net/2022/Oct/14/automating-screenshots/).


html_url: https://github.com/hydrosquall/datasette-nteract-data-explorer/releases/tag/0.5.0
node_id: RE_kwDOHEflbM4EyNdv
tag_name: 0.5.0
target_commitish: main
name: 0.5.0 - Datasette-Lite Compatibility Patch
created_at: 2022-10-18T23:51:40Z
published_at: 2022-10-18T23:54:42Z
body:
  ## What's Changed

  * feat: disable URL state sync feature when running in datasette-lite by @hydrosquall in https://github.com/hydrosquall/datasette-nteract-data-explorer/pull/30

  **Full Changelog**: https://github.com/hydrosquall/datasette-nteract-data-explorer/compare/0.4.2...0.5.0


html_url: https://github.com/hydrosquall/datasette-nteract-data-explorer/releases/tag/0.5.1
node_id: RE_kwDOHEflbM4EyNk5
tag_name: 0.5.1
target_commitish: main
name: 0.5.1
created_at: 2022-10-19T00:06:35Z
published_at: 2022-10-19T00:07:29Z
body:
  ## What's Changed
  * docs: Update the release action trigger guidelines in CONTRIBUTING.md by @hydrosquall in https://github.com/hydrosquall/datasette-nteract-data-explorer/pull/33
  * fix: reverse inverted boolean for datasette membership test by @hydrosquall in https://github.com/hydrosquall/datasette-nteract-data-explorer/pull/34


  **Full Changelog**: https://github.com/hydrosquall/datasette-nteract-data-explorer/compare/0.5.0...0.5.1


html_url: https://github.com/simonw/datasette-gunicorn/releases/tag/0.1
node_id: RE_kwDOISPElM4Ez40F
tag_name: 0.1
target_commitish: main
name: 0.1
created_at: 2022-10-22T20:55:47Z
published_at: 2022-10-22T20:57:44Z
body:
  - Initial release. `datasette gunicorn my.db -w 4` starts Datasette using [Gunicorn](https://gunicorn.org/) with four workers. [#1](https://github.com/simonw/datasette-gunicorn/issues/1)


html_url: https://github.com/simonw/datasette/releases/tag/0.63a1
node_id: RE_kwDOBm6k_c4E0CdG
tag_name: 0.63a1
target_commitish: main
name: 0.63a1
created_at: 2022-10-24T03:07:09Z
published_at: 2022-10-24T03:11:47Z
body:
  - SQL query is now re-displayed when terminated with a time limit error. ([#1819](https://github.com/simonw/datasette/issues/1819))
  - New documentation on [Running Datasette using OpenRC](https://docs.datasette.io/en/latest/deploying.html#deploying-openrc) - thanks, Adam Simpson. ([#1825](https://github.com/simonw/datasette/pull/1825))
  - The [inspect data](https://docs.datasette.io/en/latest/performance.html#performance-inspect) mechanism is now used to speed up server startup - thanks, Forest Gregg. ([#1834](https://github.com/simonw/datasette/issues/1834))
  - In [Configuration directory mode](https://docs.datasette.io/en/latest/settings.html#config-dir) databases with filenames ending in `.sqlite` or `.sqlite3` are now automatically added to the Datasette instance. ([#1646](https://github.com/simonw/datasette/issues/1646))
  - Breadcrumb navigation display now respects the current user's permissions. ([#1831](https://github.com/simonw/datasette/issues/1831))
  - Screenshots in the documentation are now maintained using [shot-scraper](https://shot-scraper.datasette.io/), as described in [Automating screenshots for the Datasette documentation using shot-scraper](https://simonwillison.net/2022/Oct/14/automating-screenshots/). ([#1844](https://github.com/simonw/datasette/issues/1844))
  - The [datasette.check_visibility()](https://docs.datasette.io/en/latest/internals.html#datasette-check-visibility) method now accepts an optional `permissions=` list, allowing it to take multiple permissions into account at once when deciding if something should be shown as public or private. This has been used to correctly display padlock icons in more places in the Datasette interface. ([#1829](https://github.com/simonw/datasette/issues/1829))

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/releases/80750406/reactions", "total_count": 2, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 2, "rocket": 0, "eyes": 0}


html_url: https://github.com/simonw/shot-scraper/releases/tag/1.0.1
node_id: RE_kwDOG-A5W84E0b4Z
tag_name: 1.0.1
target_commitish: main
name: 1.0.1
created_at: 2022-10-24T23:06:56Z
published_at: 2022-10-24T23:07:59Z
body:
  - [shot-scraper.datasette.io](https://shot-scraper.datasette.io/) homepage now includes an example screenshot. [#97](https://github.com/simonw/shot-scraper/issues/97)


html_url: https://github.com/simonw/datasette-indieauth/releases/tag/1.2.1
node_id: RE_kwDOEqb-cc4E0eHL
tag_name: 1.2.1
target_commitish: main
name: 1.2.1
created_at: 2022-10-25T01:33:36Z
published_at: 2022-10-25T01:34:47Z
body:
  - Upgraded for compatibility with latest Datasette. [#24](https://github.com/simonw/datasette-indieauth/issues/24), [#25](https://github.com/simonw/datasette-indieauth/issues/25)
  - Dropped support for Python 3.6.


html_url: https://github.com/simonw/sqlite-utils/releases/tag/3.30
node_id: RE_kwDOCGYnMM4E06wk
tag_name: 3.30
target_commitish: main
name: 3.30
created_at: 2022-10-25T22:34:30Z
published_at: 2022-10-25T22:34:48Z
body:
  - Now tested against Python 3.11. ([#502](https://github.com/simonw/sqlite-utils/issues/502))
  - New `table.search_sql(include_rank=True)` option, which adds a `rank` column to the generated SQL. Thanks, Jacob Chapman. ([#480](https://github.com/simonw/sqlite-utils/pull/480))
  - Progress bars now display for newline-delimited JSON files using the `--nl` option. Thanks, Mischa Untaga. ([#485](https://github.com/simonw/sqlite-utils/issues/485))
  - New `db.close()` method. ([#504](https://github.com/simonw/sqlite-utils/issues/504))
  - Conversion functions passed to [table.convert(...)](https://sqlite-utils.datasette.io/en/stable/python-api.html#python-api-convert) can now return lists or dictionaries, which will be inserted into the database as JSON strings. ([#495](https://github.com/simonw/sqlite-utils/issues/495))
  - `sqlite-utils install` and `sqlite-utils uninstall` commands for installing packages into the same virtual environment as `sqlite-utils`, [described here](https://sqlite-utils.datasette.io/en/stable/cli.html#cli-install). ([#483](https://github.com/simonw/sqlite-utils/issues/483))
  - New [sqlite_utils.utils.flatten()](https://sqlite-utils.datasette.io/en/stable/reference.html#reference-utils-flatten) utility function. ([#500](https://github.com/simonw/sqlite-utils/issues/500))
  - Documentation on [using Just](https://sqlite-utils.datasette.io/en/stable/contributing.html#contributing-just) to run tests, linters and build documentation.
  - Documentation now covers the [Release process](https://sqlite-utils.datasette.io/en/stable/contributing.html#release-process) for this package.


html_url: https://github.com/simonw/datasette/releases/tag/0.63
node_id: RE_kwDOBm6k_c4E1_dR
tag_name: 0.63
target_commitish: main
name: 0.63
created_at: 2022-10-27T22:11:26Z
published_at: 2022-10-27T22:13:32Z
body:
  See [Datasette 0.63: The annotated release notes](https://simonwillison.net/2022/Oct/27/datasette-0-63/) for more background on the changes in this release.

  ## Features

  - Now tested against Python 3.11. Docker containers used by `datasette publish` and `datasette package` both now use that version of Python. ([#1853](https://github.com/simonw/datasette/issues/1853))
  - `--load-extension` option now supports entrypoints. Thanks, Alex Garcia. ([#1789](https://github.com/simonw/datasette/pull/1789))
  - Facet size can now be set per-table with the new `facet_size` table metadata option. ([#1804](https://github.com/simonw/datasette/issues/1804))
  - The [truncate_cells_html](https://docs.datasette.io/en/stable/settings.html#setting-truncate-cells-html) setting now also affects long URLs in columns. ([#1805](https://github.com/simonw/datasette/issues/1805))
  - The non-JavaScript SQL editor textarea now increases height to fit the SQL query. ([#1786](https://github.com/simonw/datasette/issues/1786))
  - Facets are now displayed with better line-breaks in long values. Thanks, Daniel Rech. ([#1794](https://github.com/simonw/datasette/pull/1794))
  - The `settings.json` file used in [Configuration directory mode](https://docs.datasette.io/en/stable/settings.html#config-dir) is now validated on startup. ([#1816](https://github.com/simonw/datasette/issues/1816))
  - SQL queries can now include leading SQL comments, using `/* ... */` or `-- ...` syntax. Thanks, Charles Nepote. ([#1860](https://github.com/simonw/datasette/issues/1860))
  - SQL query is now re-displayed when terminated with a time limit error. ([#1819](https://github.com/simonw/datasette/issues/1819))
  - The [inspect data](https://docs.datasette.io/en/stable/performance.html#performance-inspect) mechanism is now used to speed up server startup - thanks, Forest Gregg. ([#1834](https://github.com/simonw/datasette/issues/1834))
  - In [Configuration directory mode](https://docs.datasette.io/en/stable/settings.html#config-dir) databases with filenames ending in `.sqlite` or `.sqlite3` are now automatically added to the Datasette instance. ([#1646](https://github.com/simonw/datasette/issues/1646))
  - Breadcrumb navigation display now respects the current user's permissions. ([#1831](https://github.com/simonw/datasette/issues/1831))

  ## Plugin hooks and internals

  - The [prepare_jinja2_environment(env, datasette)](https://docs.datasette.io/en/stable/plugin_hooks.html#plugin-hook-prepare-jinja2-environment) plugin hook now accepts an optional `datasette` argument. Hook implementations can also now return an `async` function which will be awaited automatically. ([#1809](https://github.com/simonw/datasette/issues/1809))
  - `Database(is_mutable=)` now defaults to `True`. ([#1808](https://github.com/simonw/datasette/issues/1808))
  - The [datasette.check_visibility()](https://docs.datasette.io/en/stable/internals.html#datasette-check-visibility) method now accepts an optional `permissions=` list, allowing it to take multiple permissions into account at once when deciding if something should be shown as public or private. This has been used to correctly display padlock icons in more places in the Datasette interface. ([#1829](https://github.com/simonw/datasette/issues/1829))
  - Datasette no longer enforces upper bounds on its dependencies. ([#1800](https://github.com/simonw/datasette/issues/1800))

  ## Documentation

  - New tutorial: [Cleaning data with sqlite-utils and Datasette](https://datasette.io/tutorials/clean-data).
  - Screenshots in the documentation are now maintained using [shot-scraper](https://shot-scraper.datasette.io/), as described in [Automating screenshots for the Datasette documentation using shot-scraper](https://simonwillison.net/2022/Oct/14/automating-screenshots/). ([#1844](https://github.com/simonw/datasette/issues/1844))
  - More detailed command descriptions on the [CLI reference](https://docs.datasette.io/en/stable/cli-reference.html#cli-reference) page. ([#1787](https://github.com/simonw/datasette/issues/1787))
  - New documentation on [Running Datasette using OpenRC](https://docs.datasette.io/en/stable/deploying.html#deploying-openrc) - thanks, Adam Simpson. ([#1825](https://github.com/simonw/datasette/pull/1825))

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/releases/81262417/reactions", "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0}


html_url: https://github.com/simonw/datasette-edit-templates/releases/tag/0.1
node_id: RE_kwDOEmnmRs4E1_9V
tag_name: 0.1
target_commitish: main
name: 0.1
created_at: 2022-10-27T22:53:40Z
published_at: 2022-10-27T22:54:35Z
body:
  - Now depends on [Datasette 0.63](https://docs.datasette.io/en/stable/changelog.html#v0-63), replacing an ugly plugin hook workaround.


html_url: https://github.com/eyeseast/geocode-sqlite/releases/tag/v0.7.0
node_id: RE_kwDOEXxXas4E3I8K
tag_name: v0.7.0
target_commitish: main
name: OpenCage
created_at: 2022-10-31T15:03:29Z
published_at: 2022-10-31T15:07:47Z
body:
  The [OpenCage](https://opencagedata.com/) geocoding service is now supported. Thanks @sbscully. This library is now tested on Python 3.11.

  ## What's Changed
  * Add OpenCage option to CLI by @sbscully in https://github.com/eyeseast/geocode-sqlite/pull/39

  ## New Contributors
  * @sbscully made their first contribution in https://github.com/eyeseast/geocode-sqlite/pull/39

  **Full Changelog**: https://github.com/eyeseast/geocode-sqlite/compare/v0.6.1...v0.7.0


html_url: https://github.com/eyeseast/geocode-sqlite/releases/tag/v0.8.0
node_id: RE_kwDOEXxXas4E5k20
tag_name: v0.8.0
target_commitish: main
name: Capture raw geocoding response
created_at: 2022-11-06T16:58:06Z
published_at: 2022-11-06T16:59:21Z
body:
  ## What's Changed
  * Just a typo! by @shaver in https://github.com/eyeseast/geocode-sqlite/pull/40
  * Capture raw geocoding results by @eyeseast in https://github.com/eyeseast/geocode-sqlite/pull/41

  ## New Contributors
  * @shaver made their first contribution in https://github.com/eyeseast/geocode-sqlite/pull/40

  **Full Changelog**: https://github.com/eyeseast/geocode-sqlite/compare/v0.7.0...v0.8.0


html_url: https://github.com/eyeseast/geocode-sqlite/releases/tag/v0.8.1
node_id: RE_kwDOEXxXas4E5k7K
tag_name: v0.8.1
target_commitish: main
name: v0.8.1
created_at: 2022-11-06T17:10:17Z
published_at: 2022-11-06T17:11:24Z
body:
  ## What's Changed
  * Raw results in geocode_table, too by @eyeseast in https://github.com/eyeseast/geocode-sqlite/pull/42

  **Full Changelog**: https://github.com/eyeseast/geocode-sqlite/compare/v0.8.0...v0.8.1


html_url: https://github.com/eyeseast/geocode-sqlite/releases/tag/v0.8.2
node_id: RE_kwDOEXxXas4E58Xf
tag_name: v0.8.2
target_commitish: main
name: v0.8.2
created_at: 2022-11-07T17:28:31Z
published_at: 2022-11-07T17:31:06Z
body:
  ## What's Changed
  * Require requests. More forgiving timeout. by @eyeseast in https://github.com/eyeseast/geocode-sqlite/pull/43


  **Full Changelog**: https://github.com/eyeseast/geocode-sqlite/compare/v0.8.1...v0.8.2


html_url: https://github.com/simonw/datasette/releases/tag/0.63.1
node_id: RE_kwDOBm6k_c4E7ubF
tag_name: 0.63.1
target_commitish: main
name: 0.63.1
created_at: 2022-11-11T07:01:20Z
published_at: 2022-11-11T07:02:36Z
body:
  - Fixed a bug where Datasette's table filter form would not redirect correctly when run behind a proxy using the [base_url](https://docs.datasette.io/en/stable/settings.html#setting-base-url) setting. ([#1883](https://github.com/simonw/datasette/issues/1883))
  - SQL query is now shown wrapped in a `<textarea>` if a query exceeds a time limit. ([#1876](https://github.com/simonw/datasette/issues/1876))
  - Fixed an intermittent "Too many open files" error while running the test suite. ([#1843](https://github.com/simonw/datasette/issues/1843))
  - New [db.close()](https://docs.datasette.io/en/stable/internals.html#database-close) internal method.


# tutorials

path: /tutorials/explore
title: Exploring a database with Datasette
body:
  Tutorials > Exploring a database with Datasette

  Exploring a database with Datasette
  This tutorial will teach you how to use Datasette to explore a new database.
  Datasette offers flexible tools for exploring data tables. It's always worth spending time familiarizing yourself with data in its raw, tabula form before thinking about ways to apply more sophisticated analysis or visualization.
  The congress-legislators data
  We'll be using an example database of Members of the United States Congress, 1789 to present. I built this example using data from the unitedstates/congress-legislators project on GitHub, maintained by Joshua Tauberer, Eric Mill and over 100 other contributors.
  Access the example in Datasette here: https://congress-legislators.datasettes.com/legislators

  Understanding tables
  The key to understanding relational databases such as SQLite (which we are using here), MySQL, PostgreSQL, Oracle and so on is to learn to think in tables.
  Database tables are like spreadsheets - they have columns and rows, and each row provides a value for every column. Unlike spreadsheets you can't merge cells together or apply formatting - a database table is a straight-forward two-dimensional grid. It's just the data.
  Each column in a table has a type. In SQLite these types can be one of text, integer, real (for floating point values) and blob (for binary data). The type of the column matters because it affects what happens when you sort the table by that column, or which mathematical operations you can use against those values.  
  Browsing tables
  The tables in our example database are:

  legislators - 12,590 rows - individuals who have served as senators or representatives
  legislator_terms - 44,542 rows - terms served by legislators, back to 1789
  executives - 80 rows - individuals who have served as President or Vice President
  executive_terms - 129 rows - terms served by executives
  offices - 1,192 rows - district officers maintained by current legislators
  social_media - 525 rows - social media accounts for current legislators

  Follow these links now and explore the tables to get a feel for the way the data is organized in this database.
  Tables can relate to each other
  Sometimes you'll see a row in a table that links to another table, for example the legislator_id column in the legislator_terms table:

  Clicking one of the linked names in that column will take you to a page for that individual legislator row that looks like this:

  The "Links from other tables" section shows how many rows in other tables refer back to this legislator.
  In a database these are called "foreign keys" - they work by storing the ID of a row from another table in a dedicated foreign key column.
  Foreign keys are key to understanding why relational databases are so much more powerful than separate, stand-alone data files stored in something like CSV.
  Using facets
  Facets are one of the most powerful Datasette features. They can help you take a table with thousands of rows and start quickly identifying interesting trends and patterns within that data.
  I'll show some examples using the executive_terms table.
  Facets can be applied in two ways: you can select an option from the "suggested facets" list, or you can select the "Facet by this" option from the cog menu next to each column.

  Once selected, the facet interface is shown above the table, like in this example:

  Each facet shows a list of the most common values for that column, with a total count number for each of those values.
  These numbers will update as you further filter the data: if you select "Democratic" you will see just the rows matching that party, and the "type" facet will update to show you that there have been 21 presidentals terms and 19 vice presidental terms for that party.

  Exercises

  Apply the type, state and party facets to the legislator_terms table
  Use them to filter just to Republican Senators from North Carolina (NC)
  Figure out which state has had the highest number of senatorial terms held by a member of the Republican party. (solution if you get stuck)

  Using filters
  When you select a facet, you're applying a filter to the data. These are reflected in the boxes at the top of the page:

  You don't have to use faceting for these - you can edit them directly.
  To see Democratic Vice Presidents just for the 1800s, we can add a filter for rows where the 'start' column begins with '18':

  Here's the result of that query.

  Exercises
  Using the legislator_terms table, find:

  Every senatorial term served in Delaware (DE)
  ... that started in a year starting with 18
  ... and then facet by party to see which parties had the most senatorial terms

  Sharing links
  Every page within Datasette is designed to be shared. Copy and paste the page URL to share it with someone else.
  This includes applied filters and facets - so if you find an interesting way to explore the data you can share that directly by sharing the URL.
  Some examples:

  Every senatorial term in the state of Texas
  Every executive term held by a member of the Democratic-Republican party

  Exercise: share some links!
  Use the above exercises to find something interesting in the data. Copy and paste that URL to your notes, or share it with a friend, or post about it on Twitter.
  Exporting data
  If you can see it in Datasette, you can export the raw data. This is a fundamental principle of the project.
  Look for the CSV and .json links on any page to export the data in those formats.

  The "Advanced export" box provides additional options. Use "download file" to download the CSV file to your computer. 
  The "expand labels" option will add an extra column with the label associated with any foreign key columns - in this example, it will add legislator_id_label to accompany the ID in legislator_id.
  Additional formats can be enabled using plugins. The example instance is running the datasette-copyable plugin, which provides a `"copyable" link which can be used to export the data in different formats using copy-and-paste!
  Datasette can also be configured to allow users to download the entire database as a single file. You can find that legislators.db download option at the bottom of https://congress-legislators.datasettes.com/legislators - the entire database is only 5.2MB.
  Exercises

  Download a CSV file of Democratic Vice Presidents in the 1800s, using the "download file" checkbox in the "Advanced export" box.
  Use the "copyable" option to copy and paste that data directly into Google Sheets, Excel or Numbers.

  Data visualization via plugins
  You may have noticed this already on the offices table, which shows an interactive map of all of the office locations:

  You can click on clusters of markers to zoom in, and click on individual markers to see the details of the row.
  It's fun to use these maps to spot obvious outliers - in this case, the office of representative Aumua Amata Radewagen in American Samoa instantly stands out.
  This is another example of a Datasette plugin - in this case datasette-cluster-map. Plugins can add additional features to Datasette, such as visualizations or alternative export methods (see "copyable" described above).
  For a larger example of a map visualization, take a look at this map of 33,000+ global power plants visualized using datasette-cluster-map.
  Next steps
  Now that you know how to explore tables using Datasette, the next covers how to use Datasette to learn SQL.


path: /tutorials/learn-sql
title: Learn SQL with Datasette
body:
  Tutorials > Learn SQL with Datasette

  Learn SQL with Datasette
  This tutorial follows the tutorial on exploring data with Datasette and shows how you can use Datasette to start learning to write custom SQL queries of your own.
  We will be using the same example database as that tutorial:

  https://congress-legislators.datasettes.com/legislators

  This database contains information about US Presidents, Vice Presidents and Members of Congress from 1789 to today. You can follow the previous tutorial to become familiar with the data.
  SQL and SQLite
  Datasette is software that runs on top of a SQL database. There are many different SQL database systems, such as MySQL, PostgreSQL, Microsoft SQL Server and Oracle. The database used by Datasette is called SQLite.
  You may not realize it, but you already use SQLite every day: it is built into many popular applications, including Google Chrome and Firefox, and runs on laptops, iPhones, Android phones and all sorts of other smaller devices.
  SQL stands for Structured Query Language - it is a text language for running queries against a database. Every database implements a slightly different dialect of SQL - for this tutorial I will try to keep to the subset of SQLite's dialect that is most likely to work across other databases as well.
  View and edit SQL
  Every table page in Datasette - such as this one - includes a "View and edit SQL" link that looks like this:

  Click that link to see the SQL query that was used for the current page - including any filters that have been applied, in this case the filter for bio_gender = "F".
  You can then edit that query to make it do something else!

  Exercises
  Start with that example, then:

  Click "Format SQL" to tidy this up and make it easier to read and edit
  Modify that query to return only the id and name columns
  Get it to order by name instead of sorting by id
  Change the p0 value to M to return male instead of female legislators
  Try replacing the list of columns with select * from - a shortcut for all columns

  The structure of a basic select query
  Here's the formatted version of the above query:
  select
    id,
    name,
    id_bioguide,
    id_govtrack,
    id_icpsr,
    id_wikipedia,
    id_wikidata,
    id_google_entity_id,
    name_first,
    name_last,
    bio_birthday,
    bio_gender,
    id_house_history,
    name_middle,
    name_nickname,
    id_ballotpedia,
    name_suffix,
    id_bioguide_previous,
    id_house_history_alternate,
    other_names,
    id_thomas,
    id_cspan,
    id_votesmart,
    id_lis,
    name_official_full,
    id_opensecrets,
    id_fec,
    id_maplight,
    leadership_roles,
    family
  from
    legislators
  where
    "bio_gender" = :p0
  order by
    id
  limit
    101

  select
  The select section specifies which columns you would like to return. Each column name is separated by a comma - but if you have a comma following the last column you you will get an error message.

  near "from": syntax error

  from
  The from section specifies which table the records should be selected from - here we want the legislators table.
  where
  The where section adds filter conditions. These can be combined using and, for example this query will select just legislators who are male and have a Jr. name suffix:
  where
    "bio_gender" = 'M'
    and "name_suffix" = 'Jr.'

  This section is optional - if you do not include a where clause you will get back every row in the table.
  order by
  The optional order by clause specifies the order you woud like the rows to be returned in. This would order them alphabetically by name:
  order by
    name

  Or add desc to reverse the order. This query returns the youngest legislators:
  select
    *
  from
    legislators
  order by
    bio_birthday desc

  limit
  The limit 101 clause limits the query to returning just the first 101 results. In most SQL databases omitting this will cause all results will be returned - but Datasette applies an additional limit of 1,000 (example here) to prevent large queries from causing performance issues.
  Named parameters
  In the previous tutorial we used filters to list presidental terms that occurred in the 1800s, by filtering on rows where the start column began with the string 18 and the type column equals prez.
  Here's that example as a filtered table.

  Clicking that "View and edit SQL" link (and then clicking "Format SQL") shows this query:

  select
    rowid,
    type,
    start,
    [end],
    party,
    how,
    executive_id
  from
    executive_terms
  where
    "start" like :p0
    and "type" = :p1

  The :p0 and :p1 fields here correspond to "named parameters" - they provide a safe way to pass values to a SQL query, without having to worry about SQL injection.
  These field names are extracted from the query. If you change the where clause to look like this:
  where
    "start" like :century
    and "type" = :type

  Then fields called century and type will be displayed instead of p0 and p1.
  Since they are form fields, you can change those values too. Try changing prez to viceprez to see the vice presidential terms for the 1800s instead.
  If you weren't using named parameters, the where clause of the query could look like this instead:
  where
    "start" like '18%'
    and "type" = 'prez'

  String values like '18%' and 'prez' must be enclosed in single or double quotes (single quotes are preferred), if you are not using the :p0 syntax.
  The double quotes around the column names here are optional - they are only required if the column has a name that might clash with an existing SQL keyword, such as select or where - or if the column name contains a space. So the following where clause would work the same way:
  where
    start like '18%'
    and type = 'prez'

  SQL LIKE queries
  To find rows where the start column begins with the string 18 we use this where filter:
  where
    "start" like '18%'

  The SQL like operator applies wildcards to strings - % means "match anything" and _ (underscore) means "match a single character".
  Exercise

  Use a LIKE filter to find presidential terms that occurred in the 90s decade of any century - you'll need to use both _ and % for this. (solution)

  SQL Joins
  The executive_terms table has a column called executive_id which displays both a numeric ID and a link containing the name of the executive:

  But... when you click the "View and edit SQL" link the resulting query returns just the ID, not the name.

  A SQL join can be used to combine the data from multiple tables. Here's a query that uses a join to show the name pulled from the executives table:
  select
    executive_terms.type,
    executive_terms.start,
    executive_terms.[end],
    executive_terms.party,
    executive_terms.how,
    executives.name
  from
    executive_terms
    join executives
      on executive_terms.executive_id = executives.id

  Try that here.

  Since there are now two tables involved in the query, it's a good idea to include explicit table names as part of the select clause:
  select
    executive_terms.type,
    executive_terms.start,
    executive_terms.[end],
    executive_terms.party,
    executive_terms.how,
    executives.name

  If a column exists in only one of the two tables you can use its name without specifying the table, but this can quickly get confusing so it's better to always use the table names when you are executing a join.
  The from clause is where the join is defined. This describes how the two tables should be combined together:
  from
    executive_terms
    join executives
      on executive_terms.executive_id = executives.id

  We are joining executive_terms to executives, using the fact that the executive_id column in executive_terms contains values from the id column in executives.
  This is why IDs and foreign keys are important concepts to understand in SQL databases!
  This kind of join is also known as an "inner join" - it is the most commonly used join. Other join types include outer joins and full joins, but those are beyond the scope of this tutorial.
  GROUP BY / COUNT
  A common operation in SQL is to ask for a count of the most popular values in a column.
  Datasette exposes this capability in its faceting feature. Under the hood, that feature works by executing a group by / count query.
  The following query answers the question: which party has had the most presidental terms?
  select
    party,
    count(*)
  from
    executive_terms
  where
    type = 'prez'
  group by
    party

  Try that query here.

  The select clause asks for the party column and the result of running a count(*). count() is a SQL aggregate function, which operates on a group of results. So we need to define a group.
  The group by party clause at the end creates those groups. The combination of the two produces the desired result.
  You can add this order by clause at the end to see the groups sorted from highest to lowest:
  order by
    count(*) desc

  Here's the result.
  WHERE ... IN
  SQL queries can be nested together in a number of interesting ways.
  One of the most useful for ad-hoc data analysis is the where column in (select ...) pattern.
  Let's build a query to see everyone who has been president and has also been vice president.
  We'll start with a query to return the names of every vice president. This requires a join against executives, because the executive_terms table includes the presidential and vice presidential terms, but doesn't include the names of those individuals.
  select
    executives.name
  from
    executive_terms
    join executives
      on executive_terms.executive_id = executives.id
  where
    type = 'viceprez'

  This query returns 61 rows, including some duplicate rows for individuals who served more than one term as VP.
  We could use select distinct to remove those duplicates if we wanted to.
  If we want to see everyone who has been a president and ALSO been a vice-president, we can combine two queries together like this:
  select
    distinct executives.name
  from
    executive_terms
    join executives
      on executive_terms.executive_id = executives.id
  where
    type = 'prez'
    and name in (
      select
        executives.name
      from
        executive_terms
        join executives
          on executive_terms.executive_id = executives.id
      where
        type = 'viceprez'
    )

  This query returns 15 names, starting with John Adams and finishing with Joseph Biden.

  ⚠️ An earlier version of this example revealed a subtle bug: there were two separate individuals in the executives table with the same name, George Bush and George Bush! So even though the query appeared to return the correct results it actually contained a bug. This is a useful demonstration of how important it is to work with unique, de-duplicated identifiers where-ever possible, rather than assuming that things like names are unique.

  Common Table Expressions
  This is a more advanced SQL technique that I find myself using every day: it can make complex SQL queries much easier to write and understand.
  Common Table Expressions, or CTEs, allow you to define a temporary alias for a select that lasts for the duration of the current query. You can then treat it as if it was a regular table.
  Here's an example that creates two CTEs, one called presidents and one called vice_presidents, and then uses them to answer the earlier question about presidents who had also served as vice presidents:
  with presidents as (
    select
      executives.name
    from
      executive_terms
      join executives
        on executive_terms.executive_id = executives.id
    where
      executive_terms.type = 'prez'
  ),
  vice_presidents as (
    select
      executives.name
    from
      executive_terms
      join executives
        on executive_terms.executive_id = executives.id
    where
      executive_terms.type = 'viceprez'
  )
  select
    distinct name
  from
    presidents
  where name in vice_presidents

  Try this CTE query here.
  The two CTEs are defined using with alias_name as (select ...), second_alias as (select ...) - then the query that returns the final results is added on at the end.
  Creating bookmarkable apps
  Since every page on Datasette can be linked to (see Sharing links) and named parameters automatically add form input to the query page, you can combine these two features to create bookmarkable apps.
  This is particularly useful for collaborating with others who don't yet know SQL. You can write SQL queries that solve problems for them, add parameters that they can customize and then send them the entire custom application as a link that they can bookmark.
  Let's create an application that returns the president and vice president for any given date.
  Here's the SQL query we will be using:
  select
    executives.name,
    executive_terms.type,
    executive_terms.start,
    executive_terms.end,
    executive_terms.party
  from
    executive_terms
    join executives
      on executive_terms.executive_id = executives.id
  where
    start <= :date
    and end > :date

  This query takes a :date parameter, which should be of the format yyyy-mm-dd - for example 2016-01-01.
  It joins against the executives table to get their name, and then filters to rows where the term start is less than or equal to the specified date, and the end is greater than that date. We use <= for one of these to ensure there are no gaps or overlaps.
  If we execute the query without a date it will return 0 results, but give us a form field to enter the date:

  Entering a date - for example 2016-01-01 - returns the president and vice president for that date:

  We've built an application! We can send this link to anyone right now and they'll be able to run that same query.
  The SQL at the top of the page could feel a little intimidating though. That's what the "hide" link is for - clicking on that hides the SQL query, providing a new link that you can share that won't cause the user to scroll down past the SQL in order to interact with the query.
  One last note: the application we have built also doubles up as an API. Add .json to the path portion of the URL, or click the .json link, to get back the data as JSON. Add &_shape=array to the URL to get back a more compact form of JSON, ideal for integrating with other applications.
  Next steps
  Fully mastering SQL can take years. This tutorial has attempted to cover SQL basics and introduce some more advanced techniques to help manage larger queries, but there's plenty more to learn. The SQLite SELECT documentation offers a comprehensive reference to SQL as understood by SQLite, but the best way to learn is to keep on trying new things against different data, looking things up as you go along.


path: /tutorials/clean-data
title: Cleaning data with sqlite-utils and Datasette
body:
  Tutorials > Cleaning data with sqlite-utils and Datasette

  Cleaning data with sqlite-utils and Datasette

  This tutorial will show you how to use the sqlite-utils command line tool import data into SQLite, clean it and then explore it with Datasette.

  The concepts in this tutorial are also demonstrated in this video of a talk I gave at the HYTRADBOI conference in April 2022.



  Start with some CSV data
  The Florida Fish and Wildlife Conservation Commission maintain a database of manatee carcass recovery locations, with details dating back to April 1974.
  You can request the latest copy as a CSV file from their website - or download this copy of the CSV: Manatee_Carcass_Recovery_Locations_in_Florida.csv generated on July 31st 2022.
  Install sqlite-utils
  Here are the installation instructions. If you are on a Mac with Homebrew installed you can run brew install sqlite-utils. Python users can run pip install sqlite-utils.
  Import the CSV into a SQLite database
  sqlite-utils is a command line tool, so you'll need to open up a terminal on your computer and change directory to the place where you saved the CSV file.
  You can create a new manatees.db SQLite database like this:
  sqlite-utils insert manatees.db locations \
    Manatee_Carcass_Recovery_Locations_in_Florida.csv --csv -d

  This sqlite-utils insert command inserts data into a database table, creating the table if necessary.

  manatees.db is the SQLite database file we are using - this will be created if it doesn't exist yet.
  locations is the name of the table we are creating
  Manatee_Carcass_Recovery_Locations_in_Florida.csv is the CSV file we are importing
  --csv tells the tool to treat the incoming data as CSV data
  The -d option tells the tool to automatically detect the types of data. Without this, every column will be treated as text. With this option columns that have numeric data in will be used to populate numeric columns.

  Having created the database, we can use the schema command to view its schema:
  sqlite-utils schema manatees.db

  The output of that command looks like this:
  CREATE TABLE "locations" (
     [X] FLOAT,
     [Y] FLOAT,
     [OBJECTID] INTEGER,
     [FIELDID] TEXT,
     [REPDATE] TEXT,
     [REPYEAR] INTEGER,
     [REPMONTH] INTEGER,
     [REPDAY] INTEGER,
     [SEX] TEXT,
     [TLENGTH] FLOAT,
     [STATE] TEXT,
     [COUNTY] TEXT,
     [LAT] FLOAT,
     [LONG_] FLOAT,
     [DCODE] INTEGER,
     [MORTALITY] TEXT,
     [created_user] TEXT,
     [created_date] TEXT,
     [last_edited_user] TEXT,
     [last_edited_date] TEXT
  );

  The locations table here has been created with columns that match the columns that were present in the CSV file. The columns have the appropriate types thanks to the -d option we passed to sqlite-utils insert.
  We can use the sqlite-utils tables command to see those tables:
  sqlite-utils tables manatees.db --counts

  Output is:
  [{"table": "locations", "count": 13568}]

  The --counts option adds a count of the rows in each table. That's over 13,000 deceased manatees!
  By default, the tables command outputs JSON. You can change this to be a table format using the --table or -t options:
  sqlite-utils tables manatees.db --counts -t
  table        count
  ---------  -------
  locations    13568

  Querying the data
  Let's run a SQL query against your data. The sqlite-utils command to do that looks like this:
  sqlite-utils manatees.db "select * from locations limit 2"

  This outputs JSON, showing the first two rows and every column (thanks to that select *):
  [{"X": -80.1176098026034, "Y": 26.1004059191504, "OBJECTID": 149433, "FIELDID": "M7401", "REPDATE": "1974/04/03 00:00:00+00", "REPYEAR": 1974, "REPMONTH": 4, "REPDAY": 3, "SEX": "F", "TLENGTH": 260.0, "STATE": "FL", "COUNTY": "Broward", "LAT": 26.100401, "LONG_": -80.117607, "DCODE": 9, "MORTALITY": "Undetermined: Other", "created_user": "FWC", "created_date": "2021/10/15 11:30:55+00", "last_edited_user": "FWC", "last_edited_date": "2021/10/15 11:30:55+00"},
   {"X": -80.1664354041335, "Y": 25.8337161801256, "OBJECTID": 149434, "FIELDID": "M7402", "REPDATE": "1974/06/27 00:00:00+00", "REPYEAR": 1974, "REPMONTH": 6, "REPDAY": 27, "SEX": "M", "TLENGTH": 290.0, "STATE": "FL", "COUNTY": "Miami-Dade", "LAT": 25.833711, "LONG_": -80.166433, "DCODE": 1, "MORTALITY": "Human Related: Watercraft Collision", "created_user": "FWC", "created_date": "2021/10/15 11:30:55+00", "last_edited_user": "FWC", "last_edited_date": "2021/10/15 11:30:55+00"}]

  Let's specify some columns, limit to 10 and add -t to output the results as a table.
  sqlite-utils manatees.db -t \
    "select REPDATE, MORTALITY, COUNTY from locations limit 10"

  This outputs:
  REPDATE                 MORTALITY                            COUNTY
  ----------------------  -----------------------------------  ------------
  1974/04/03 00:00:00+00  Undetermined: Other                  Broward
  1974/06/27 00:00:00+00  Human Related: Watercraft Collision  Miami-Dade
  1974/08/20 00:00:00+00  Human Related: Watercraft Collision  Miami-Dade
  1974/10/23 00:00:00+00  Human Related: Other                 Volusia
  1974/11/11 00:00:00+00  Human Related: Other                 Brevard
  1974/12/25 00:00:00+00  Human Related: Watercraft Collision  Citrus
  1974/12/27 00:00:00+00  Undetermined: Too Decomposed         Indian River
  1975/01/01 00:00:00+00  Verified: Not Necropsied             Brevard
  1975/01/19 00:00:00+00  Human Related: Watercraft Collision  Miami-Dade
  1975/01/31 00:00:00+00  Verified: Not Necropsied             Brevard

  Opening it in Datasette
  Datasette provides a browser-based interface for exploring a SQLite database. Install that, (brew install datasette works for Homebrew users) and run it against your database:
  datasette manatees.db

  Then navigate to http://localhost:8001/ to start exploring your data.
  Transforming the columns
  Exploring the data helps identify improvements we can make to the way it is structured.
  The LAT and LONG_ column look like they might be latitudes and longitudes. Renaming those columns to latitude and longitude will make that more obvious, and will also let us visualize the data using a Datasette plugin in a moment.
  The created_user, last_edited_user and STATE columns aren't interesting: they always store the same value.
  The X and Y columns are duplicates of the latitude and longitude.
  Finally, the FIELDID column looks to be a unique identifier for every row in the database. This would make a good primary key for our table.
  The sqlite-utils transform command can be used to apply transformations to a table - rename columns, dropping columns, assigning primary keys and more.
  The following command will make a bunch of changes to the table structure in one go:
  sqlite-utils transform manatees.db locations \
    --rename LAT latitude \
    --rename LONG_ longitude \
    --drop created_user \
    --drop last_edited_user \
    --drop X \
    --drop Y \
    --drop STATE \
    --drop OBJECTID \
    --pk FIELDID

  Now run sqlite-utils schema manatees.db to see the result of those changes:
  CREATE TABLE "locations" (
     [FIELDID] TEXT PRIMARY KEY,
     [REPDATE] TEXT,
     [REPYEAR] INTEGER,
     [REPMONTH] INTEGER,
     [REPDAY] INTEGER,
     [SEX] TEXT,
     [TLENGTH] FLOAT,
     [COUNTY] TEXT,
     [latitude] FLOAT,
     [longitude] FLOAT,
     [DCODE] INTEGER,
     [MORTALITY] TEXT,
     [created_date] TEXT,
     [last_edited_date] TEXT
  );

  Visualizing the locations on a map
  Datasette supports plugins, which enable extra features such as data visualizations.
  The datasette-cluster-map plugin runs against any table with a latitude and longitude column, displaying them as points on a map. This is why we renamed those LAT and LONG_ columns earlier!
  To install the plugin, quit the Datasette server (using Ctrl+C in the terminal window) and run the following:
  datasette install datasette-cluster-map

  Then start Datasette running again:
  datasette manatees.db

  If you visit http://localhost:8001/manatees/locations you will now see the first 1,000 rows as points on the map:

  Click the "load all" button to load and display all 13,000+ locations.
  Fixing the date columns
  The date columns currently look like this:
  2021/10/15 11:30:55+00

  This is using YYYY/MM/DD format (at least it's not that confusing - for Europeans - American MM/DD/YYYY thing). I prefer the ISO standard format YYYY-MM-DD - so let's convert it.
  The sqlite-utils transform command can be used to apply a transformation function to the values in one or more columns.
  Here's how to use that to parse miscellaneous date and time formats and turn them into that standard ISO format:
  sqlite-utils convert manatees.db locations \
    REPDATE created_date last_edited_date \
    'r.parsedatetime(value)'

  This runs against the REPDATE, created_date and last_edited_date columns. It will use the built-in r.parsedatetime() recipe (documented here) to convert those values and save them back to the same column.
  Run that command, then view the data in Datasette again. Those columns have now been converted!
  Extracting columns into a separate table
  There's one more improvement we will make to our data.
  Open the table in Datasette, then use either the suggested facets list or the cog menu at the top of the columns to facet by both DCODE and MORTALITY.

  The result counts shown for the two facets are the same. This suggests that these two columns represent the same data - a DCODE of 1 corresponds to a MORTALITY of "Human Related: Watercraft Collision", 2 means "Human Related: Flood Gate/Canal Lock" and so on.
  We can use the sqlite-utils extract command to extract these columns into a separate table.
  sqlite-utils extract manatees.db locations DCODE MORTALITY \
    --rename MORTALITY name \
    --table mortality

  Running this will create a new table called mortality and populate it with one row for each of the DCODE and MORTALITY pairs in the database. It removes those columns from the locations table and replaces them with a mortality_id foreign key column pointing to the new table.
  Here's the output of sqlite-utils schema manatees.db showing the impact of those changes on the schema:
  CREATE TABLE [mortality] (
    [id] INTEGER PRIMARY KEY,
    [DCODE] INTEGER,
    [name] TEXT
  );
  CREATE TABLE "locations" (
    [FIELDID] TEXT PRIMARY KEY,
    [REPDATE] TEXT,
    [REPYEAR] INTEGER,
    [REPMONTH] INTEGER,
    [REPDAY] INTEGER,
    [SEX] TEXT,
    [TLENGTH] FLOAT,
    [COUNTY] TEXT,
    [latitude] FLOAT,
    [longitude] FLOAT,
    [mortality_id] INTEGER,
    [created_date] TEXT,
    [last_edited_date] TEXT,
    FOREIGN KEY([mortality_id]) REFERENCES [mortality]([id])
  );
  CREATE UNIQUE INDEX [idx_mortality_DCODE_name]
     ON [mortality] ([DCODE], [name]);

  Further reading
  sqlite-utils offers a lot more tools for cleaning up data. Check out this documentation and these blog posts for more details.

  sqlite-utils command line tool documentation
  Executing advanced ALTER TABLE operations in SQLite
  Apply conversion functions to data in SQLite columns with the sqlite-utils CLI tool
  Refactoring databases with sqlite-utils extract


# pypi_packages

name: airtable-export
summary: Export Airtable data to files on disk
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/airtable-export
license: Apache License, Version 2.0
package_url: https://pypi.org/project/airtable-export/
project_url: https://pypi.org/project/airtable-export/
release_url: https://pypi.org/project/airtable-export/0.7.1/
version: 0.7.1
description:
  # airtable-export

  [![PyPI](https://img.shields.io/pypi/v/airtable-export.svg)](https://pypi.org/project/airtable-export/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/airtable-export?include_prereleases&label=changelog)](https://github.com/simonw/airtable-export/releases)
  [![Tests](https://github.com/simonw/airtable-export/workflows/Test/badge.svg)](https://github.com/simonw/airtable-export/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/airtable-export/blob/master/LICENSE)

  Export Airtable data to files on disk

  ## Installation

  Install this tool using `pip`:

      $ pip install airtable-export

  ## Usage

  You will need to know the following information:

  - Your Airtable base ID - this is a string starting with `app...`
  - Your Airtable API key - this is a string starting with `key...`
  - The names of each of the tables that you wish to export

  You can export all of your data to a folder called `export/` by running the following:

      airtable-export export base_id table1 table2 --key=key

  This example would create two files: `export/table1.yml` and `export/table2.yml`.

  Rather than passing the API key using the `--key` option you can set it as an environment variable called `AIRTABLE_KEY`.

  ## Export options

  By default the tool exports your data as YAML.

  You can also export as JSON or as [newline delimited JSON](http://ndjson.org/) using the `--json` or `--ndjson` options:

      airtable-export export base_id table1 table2 --key=key --ndjson

  You can pass multiple format options at once. This command will create a `.json`, `.yml` and `.ndjson` file for each exported table:

      airtable-export export base_id table1 table2 \
          --key=key --ndjson --yaml --json

  ### SQLite database export

  You can export tables to a SQLite database file using the `--sqlite database.db` option:

      airtable-export export base_id table1 table2 \
          --key=key --sqlite database.db

  This can be combined with other format options. If you only specify `--sqlite` the export directory argument will be ignored.

  The SQLite database will have a table created for each table you export. Those tables will have a primary key column called `airtable_id`.

  If you run this command against an existing SQLite database records with matching primary keys will be over-written by new records from the export.

  ## Request options

  By default the tool uses [python-httpx](https://www.python-httpx.org)'s default configurations.

  You can override the `user-agent` using the `--user-agent` option:

      airtable-export export base_id table1 table2 --key=key --user-agent "Airtable Export Robot"

  You can override the [timeout during a network read operation](https://www.python-httpx.org/advanced/#fine-tuning-the-configuration) using the `--http-read-timeout` option. If not set, this defaults to 5s.

      airtable-export export base_id table1 table2 --key=key --http-read-timeout 60

  ## Running this using GitHub Actions

  [GitHub Actions](https://github.com/features/actions) is GitHub's workflow automation product. You can use it to run `airtable-export` in order to back up your Airtable data to a GitHub repository. Doing this gives you a visible commit history of changes you make to your Airtable data - like [this one](https://github.com/natbat/rockybeaches/commits/main/airtable).

  To run this for your own Airtable database you'll first need to add the following secrets to your GitHub repository:

  <dl>
    <dt>AIRTABLE_BASE_ID</dt>
    <dd>The base ID, a string beginning `app...`</dd>
    <dt>AIRTABLE_KEY</dt>
    <dd>Your Airtable API key</dd>
    <dt>AIRTABLE_TABLES</dt>
    <dd>A space separated list of the Airtable tables that you want to backup. If any of these contain spaces you will need to enclose them in single quotes, e.g. <samp>'My table with spaces in the name' OtherTableWithNoSpaces</samp></dd>
  </dl>

  Once you have set those secrets, add the following as a file called `.github/workflows/backup-airtable.yml`:
  ```yaml
  name: Backup Airtable

  on:
    workflow_dispatch:
    schedule:
    - cron: '32 0 * * *'

  jobs:
    build:
      runs-on: ubuntu-latest
      steps:
      - name: Check out repo
        uses: actions/checkout@v2
      - name: Set up Python
        uses: actions/setup-python@v2
        with:
          python-version: 3.8
      - uses: actions/cache@v2
        name: Configure pip caching
        with:
          path: ~/.cache/pip
          key: ${{ runner.os }}-pip-
          restore-keys: |
            ${{ runner.os }}-pip-
      - name: Install airtable-export
        run: |
          pip install airtable-export
      - name: Backup Airtable to backups/
        env:
          AIRTABLE_BASE_ID: ${{ secrets.AIRTABLE_BASE_ID }}
          AIRTABLE_KEY: ${{ secrets.AIRTABLE_KEY }}
          AIRTABLE_TABLES: ${{ secrets.AIRTABLE_TABLES }}
        run: |-
          airtable-export backups $AIRTABLE_BASE_ID $AIRTABLE_TABLES -v
      - name: Commit and push if it changed
        run: |-
          git config user.name "Automated"
          git config user.email "actions@users.noreply.github.com"
          git add -A
          timestamp=$(date -u)
          git commit -m "Latest data: ${timestamp}" || exit 0
          git push
  ```
  This will run once a day (at 32 minutes past midnight UTC) and will also run if you manually click the "Run workflow" button, see [GitHub Actions: Manual triggers with workflow_dispatch](https://github.blog/changelog/2020-07-06-github-actions-manual-triggers-with-workflow_dispatch/).

  ## Development

  To contribute to this tool, first checkout the code. Then create a new virtual environment:

      cd airtable-export
      python -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/airtable-export/actions", "Changelog": "https://github.com/simonw/airtable-export/releases", "Homepage": "https://github.com/simonw/airtable-export", "Issues": "https://github.com/simonw/airtable-export/issues"}

requires_dist:
  ["click", "PyYAML", "httpx", "sqlite-utils", "pytest ; extra == 'test'", "pytest-mock ; extra == 'test'"]


name: csv-diff
summary: Python CLI tool and library for diffing CSV and JSON files
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/csv-diff
license: Apache License, Version 2.0
package_url: https://pypi.org/project/csv-diff/
project_url: https://pypi.org/project/csv-diff/
project_urls: {"Homepage": "https://github.com/simonw/csv-diff"}
release_url: https://pypi.org/project/csv-diff/1.1/
requires_dist: ["click", "dictdiffer", "pytest ; extra == 'test'"]
version: 1.1
classifiers:
  ["Development Status :: 4 - Beta", "Intended Audience :: Developers", "Intended Audience :: End Users/Desktop", "Intended Audience :: Science/Research", "License :: OSI Approved :: Apache Software License", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7"]

description:
  # csv-diff

  [![PyPI](https://img.shields.io/pypi/v/csv-diff.svg)](https://pypi.org/project/csv-diff/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/csv-diff?include_prereleases&label=changelog)](https://github.com/simonw/csv-diff/releases)
  [![Tests](https://github.com/simonw/csv-diff/workflows/Test/badge.svg)](https://github.com/simonw/csv-diff/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/csv-diff/blob/main/LICENSE)

  Tool for viewing the difference between two CSV, TSV or JSON files. See [Generating a commit log for San Francisco’s official list of trees](https://simonwillison.net/2019/Mar/13/tree-history/) (and the [sf-tree-history repo commit log](https://github.com/simonw/sf-tree-history/commits)) for background information on this project.

  ## Installation

      pip install csv-diff

  ## Usage

  Consider two CSV files:

  `one.csv`

      id,name,age
      1,Cleo,4
      2,Pancakes,2

  `two.csv`

      id,name,age
      1,Cleo,5
      3,Bailey,1

  `csv-diff` can show a human-readable summary of differences between the files:

      $ csv-diff one.csv two.csv --key=id
      1 row changed, 1 row added, 1 row removed

      1 row changed

        Row 1
          age: "4" => "5"

      1 row added

        id: 3
        name: Bailey
        age: 1

      1 row removed

        id: 2
        name: Pancakes
        age: 2

  The `--key=id` option means that the `id` column should be treated as the unique key, to identify which records have changed.

  The tool will automatically detect if your files are comma- or tab-separated. You can over-ride this automatic detection and force the tool to use a specific format using `--format=tsv` or `--format=csv`.

  You can also feed it JSON files, provided they are a JSON array of objects where each object has the same keys. Use `--format=json` if your input files are JSON.

  Use `--show-unchanged` to include full details of the unchanged values for rows with at least one change in the diff output:

      % csv-diff one.csv two.csv --key=id --show-unchanged
      1 row changed

        id: 1
          age: "4" => "5"

          Unchanged:
            name: "Cleo"

  You can use the `--json` option to get a machine-readable difference:

      $ csv-diff one.csv two.csv --key=id --json
      {
          "added": [
              {
                  "id": "3",
                  "name": "Bailey",
                  "age": "1"
              }
          ],
          "removed": [
              {
                  "id": "2",
                  "name": "Pancakes",
                  "age": "2"
              }
          ],
          "changed": [
              {
                  "key": "1",
                  "changes": {
                      "age": [
                          "4",
                          "5"
                      ]
                  }
              }
          ],
          "columns_added": [],
          "columns_removed": []
      }

  ## As a Python library

  You can also import the Python library into your own code like so:

      from csv_diff import load_csv, compare
      diff = compare(
          load_csv(open("one.csv"), key="id"),
          load_csv(open("two.csv"), key="id")
      )

  `diff` will now contain the same data structure as the output in the `--json` example above.

  If the columns in the CSV have changed, those added or removed columns will be ignored when calculating changes made to specific rows.


name: csvs-to-sqlite
summary: Convert CSV files into a SQLite database
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/csvs-to-sqlite
license: Apache License, Version 2.0
package_url: https://pypi.org/project/csvs-to-sqlite/
project_url: https://pypi.org/project/csvs-to-sqlite/
project_urls: {"Homepage": "https://github.com/simonw/csvs-to-sqlite"}
release_url: https://pypi.org/project/csvs-to-sqlite/1.3/
version: 1.3
classifiers:
  ["Intended Audience :: Developers", "Intended Audience :: End Users/Desktop", "Intended Audience :: Science/Research", "License :: OSI Approved :: Apache Software License", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Programming Language :: Python :: 3.8", "Programming Language :: Python :: 3.9", "Topic :: Database"]

description:
  # csvs-to-sqlite

  [![PyPI](https://img.shields.io/pypi/v/csvs-to-sqlite.svg)](https://pypi.org/project/csvs-to-sqlite/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/csvs-to-sqlite?include_prereleases&label=changelog)](https://github.com/simonw/csvs-to-sqlite/releases)
  [![Tests](https://github.com/simonw/csvs-to-sqlite/workflows/Test/badge.svg)](https://github.com/simonw/csvs-to-sqlite/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/csvs-to-sqlite/blob/main/LICENSE)

  Convert CSV files into a SQLite database. Browse and publish that SQLite database with [Datasette](https://github.com/simonw/datasette).

  Basic usage:

      csvs-to-sqlite myfile.csv mydatabase.db

  This will create a new SQLite database called `mydatabase.db` containing a
  single table, `myfile`, containing the CSV content.

  You can provide multiple CSV files:

      csvs-to-sqlite one.csv two.csv bundle.db

  The `bundle.db` database will contain two tables, `one` and `two`.

  This means you can use wildcards:

      csvs-to-sqlite ~/Downloads/*.csv my-downloads.db

  If you pass a path to one or more directories, the script will recursively
  search those directories for CSV files and create tables for each one.

      csvs-to-sqlite ~/path/to/directory all-my-csvs.db

  ## Handling TSV (tab-separated values)

  You can use the `-s` option to specify a different delimiter. If you want
  to use a tab character you'll need to apply shell escaping like so:

      csvs-to-sqlite my-file.tsv my-file.db -s $'\t'

  ## Refactoring columns into separate lookup tables

  Let's say you have a CSV file that looks like this:

      county,precinct,office,district,party,candidate,votes
      Clark,1,President,,REP,John R. Kasich,5
      Clark,2,President,,REP,John R. Kasich,0
      Clark,3,President,,REP,John R. Kasich,7

  ([Real example taken from the Open Elections project](https://github.com/openelections/openelections-data-sd/blob/master/2016/20160607__sd__primary__clark__precinct.csv))

  You can now convert selected columns into separate lookup tables using the new
  `--extract-column` option (shortname: `-c`) - for example:

      csvs-to-sqlite openelections-data-*/*.csv \
          -c county:County:name \
          -c precinct:Precinct:name \
          -c office -c district -c party -c candidate \
          openelections.db

  The format is as follows:

      column_name:optional_table_name:optional_table_value_column_name

  If you just specify the column name e.g. `-c office`, the following table will
  be created:

      CREATE TABLE "office" (
          "id" INTEGER PRIMARY KEY,
          "value" TEXT
      );

  If you specify all three options, e.g. `-c precinct:Precinct:name` the table
  will look like this:

      CREATE TABLE "Precinct" (
          "id" INTEGER PRIMARY KEY,
          "name" TEXT
      );

  The original tables will be created like this:

      CREATE TABLE "ca__primary__san_francisco__precinct" (
          "county" INTEGER,
          "precinct" INTEGER,
          "office" INTEGER,
          "district" INTEGER,
          "party" INTEGER,
          "candidate" INTEGER,
          "votes" INTEGER,
          FOREIGN KEY (county) REFERENCES County(id),
          FOREIGN KEY (party) REFERENCES party(id),
          FOREIGN KEY (precinct) REFERENCES Precinct(id),
          FOREIGN KEY (office) REFERENCES office(id),
          FOREIGN KEY (candidate) REFERENCES candidate(id)
      );

  They will be populated with IDs that reference the new derived tables.

  ## Installation

      $ pip install csvs-to-sqlite

  `csvs-to-sqlite` now requires Python 3. If you are running Python 2 you can install the last version to support Python 2:

      $ pip install csvs-to-sqlite==0.9.2

  ## csvs-to-sqlite --help

  <!-- [[[cog
  import cog
  from csvs_to_sqlite import cli
  from click.testing import CliRunner
  runner = CliRunner()
  result = runner.invoke(cli.cli, ["--help"])
  help = result.output.replace("Usage: cli", "Usage: csvs-to-sqlite")
  cog.out(
      "```\n{}\n```".format(help)
  )
  ]]] -->
  ```
  Usage: csvs-to-sqlite [OPTIONS] PATHS... DBNAME

    PATHS: paths to individual .csv files or to directories containing .csvs

    DBNAME: name of the SQLite database file to create

  Options:
    -s, --separator TEXT            Field separator in input .csv
    -q, --quoting INTEGER           Control field quoting behavior per csv.QUOTE_*
                                    constants. Use one of QUOTE_MINIMAL (0),
                                    QUOTE_ALL (1), QUOTE_NONNUMERIC (2) or
                                    QUOTE_NONE (3).

    --skip-errors                   Skip lines with too many fields instead of
                                    stopping the import

    --replace-tables                Replace tables if they already exist
    -t, --table TEXT                Table to use (instead of using CSV filename)
    -c, --extract-column TEXT       One or more columns to 'extract' into a
                                    separate lookup table. If you pass a simple
                                    column name that column will be replaced with
                                    integer foreign key references to a new table
                                    of that name. You can customize the name of
                                    the table like so:     state:States:state_name
                                    
                                    This will pull unique values from the 'state'
                                    column and use them to populate a new 'States'
                                    table, with an id column primary key and a
                                    state_name column containing the strings from
                                    the original column.

    -d, --date TEXT                 One or more columns to parse into ISO
                                    formatted dates

    -dt, --datetime TEXT            One or more columns to parse into ISO
                                    formatted datetimes

    -df, --datetime-format TEXT     One or more custom date format strings to try
                                    when parsing dates/datetimes

    -pk, --primary-key TEXT         One or more columns to use as the primary key
    -f, --fts TEXT                  One or more columns to use to populate a full-
                                    text index

    -i, --index TEXT                Add index on this column (or a compound index
                                    with -i col1,col2)

    --shape TEXT                    Custom shape for the DB table - format is
                                    csvcol:dbcol(TYPE),...

    --filename-column TEXT          Add a column with this name and populate with
                                    CSV file name

    --fixed-column <TEXT TEXT>...   Populate column with a fixed string
    --fixed-column-int <TEXT INTEGER>...
                                    Populate column with a fixed integer
    --fixed-column-float <TEXT FLOAT>...
                                    Populate column with a fixed float
    --no-index-fks                  Skip adding index to foreign key columns
                                    created using --extract-column (default is to
                                    add them)

    --no-fulltext-fks               Skip adding full-text index on values
                                    extracted using --extract-column (default is
                                    to add them)

    --just-strings                  Import all columns as text strings by default
                                    (and, if specified, still obey --shape,
                                    --date/datetime, and --datetime-format)

    --version                       Show the version and exit.
    --help                          Show this message and exit.

  ```
  <!-- [[[end]]] -->

requires_dist:
  ["click (~=7.0)", "dateparser (>=1.0)", "pandas (>=1.0)", "py-lru-cache (~=0.1.4)", "six", "pytest ; extra == 'test'", "cogapp ; extra == 'test'"]


name: datasette-atom
summary: Datasette plugin that adds a .atom output format
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-atom
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-atom/
project_url: https://pypi.org/project/datasette-atom/
release_url: https://pypi.org/project/datasette-atom/0.8.1/
version: 0.8.1
description:
  # datasette-atom

  [![PyPI](https://img.shields.io/pypi/v/datasette-atom.svg)](https://pypi.org/project/datasette-atom/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-atom?include_prereleases&label=changelog)](https://github.com/simonw/datasette-atom/releases)
  [![Tests](https://github.com/simonw/datasette-atom/workflows/Test/badge.svg)](https://github.com/simonw/datasette-atom/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-atom/blob/main/LICENSE)

  Datasette plugin that adds support for generating [Atom feeds](https://validator.w3.org/feed/docs/atom.html) with the results of a SQL query.

  ## Installation

  Install this plugin in the same environment as Datasette to enable the `.atom` output extension.

      $ pip install datasette-atom

  ## Usage

  To create an Atom feed you need to define a custom SQL query that returns a required set of columns:

  * `atom_id` - a unique ID for each row. [This article](https://web.archive.org/web/20080211143232/http://diveintomark.org/archives/2004/05/28/howto-atom-id) has suggestions about ways to create these IDs.
  * `atom_title` - a title for that row.
  * `atom_updated` - an [RFC 3339](http://www.faqs.org/rfcs/rfc3339.html) timestamp representing the last time the entry was modified in a significant way. This can usually be the time that the row was created.

  The following columns are optional:

  * `atom_content` - content that should be shown in the feed. This will be treated as a regular string, so any embedded HTML tags will be escaped when they are displayed.
  * `atom_content_html` - content that should be shown in the feed. This will be treated as an HTML string, and will be sanitized using [Bleach](https://github.com/mozilla/bleach) to ensure it does not have any malicious code in it before being returned as part of a `<content type="html">` Atom element. If both are provided, this will be used in place of `atom_content`.
  * `atom_link` - a URL that should be used as the link that the feed entry points to.
  * `atom_author_name` - the name of the author of the entry. If you provide this you can also provide `atom_author_uri` and `atom_author_email` with a URL and e-mail address for that author.

  A query that returns these columns can then be returned as an Atom feed by adding the `.atom` extension.

  ## Example

  Here is an example SQL query which generates an Atom feed for new entries on [www.niche-museums.com](https://www.niche-museums.com/):

  ```sql
  select
    'tag:niche-museums.com,' || substr(created, 0, 11) || ':' || id as atom_id,
    name as atom_title,
    created as atom_updated,
    'https://www.niche-museums.com/browse/museums/' || id as atom_link,
    coalesce(
      '<img src="' || photo_url || '?w=800&amp;h=400&amp;fit=crop&amp;auto=compress">',
      ''
    ) || '<p>' || description || '</p>' as atom_content_html
  from
    museums
  order by
    created desc
  limit
    15
  ```

  You can try this query by [pasting it in here](https://www.niche-museums.com/browse) - then click the `.atom` link to see it as an Atom feed.

  ## Using a canned query

  Datasette's [canned query mechanism](https://datasette.readthedocs.io/en/stable/sql_queries.html#canned-queries) is a useful way to configure feeds. If a canned query definition has a `title` that will be used as the title of the Atom feed.

  Here's an example, defined using a `metadata.yaml` file:

  ```yaml
  databases:
    browse:
      queries:
        feed:
          title: Niche Museums
          sql: |-
            select
              'tag:niche-museums.com,' || substr(created, 0, 11) || ':' || id as atom_id,
              name as atom_title,
              created as atom_updated,
              'https://www.niche-museums.com/browse/museums/' || id as atom_link,
              coalesce(
                '<img src="' || photo_url || '?w=800&amp;h=400&amp;fit=crop&amp;auto=compress">',
                ''
              ) || '<p>' || description || '</p>' as atom_content_html
            from
              museums
            order by
              created desc
            limit
              15
  ```
  ## Disabling HTML filtering

  The HTML allow-list used by Bleach for the `atom_content_html` column can be found in the `clean(html)` function at the bottom of [datasette_atom/__init__.py](https://github.com/simonw/datasette-atom/blob/main/datasette_atom/__init__.py).

  You can disable Bleach entirely for Atom feeds generated using a canned query. You should only do this if you are certain that no user-provided HTML could be included in that value.

  Here's how to do that in `metadata.json`:

  ```json
  {
    "plugins": {
      "datasette-atom": {
        "allow_unsafe_html_in_canned_queries": true
      }
    }
  }
  ```
  Setting this to `true` will disable Bleach filtering for all canned queries across all databases.

  You can disable Bleach filtering just for a specific list of canned queries like so:

  ```json
  {
    "plugins": {
      "datasette-atom": {
        "allow_unsafe_html_in_canned_queries": {
          "museums": ["latest", "moderation"]
        }
      }
    }
  }
  ```
  This will disable Bleach just for the canned queries called `latest` and `moderation` in the `museums.db` database.

project_urls:
  {"CI": "https://github.com/simonw/datasette-atom/actions", "Changelog": "https://github.com/simonw/datasette-atom/releases", "Homepage": "https://github.com/simonw/datasette-atom", "Issues": "https://github.com/simonw/datasette-atom/issues"}

requires_dist:
  ["datasette (>=0.49)", "feedgen", "bleach", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "httpx ; extra == 'test'"]


name: datasette-auth-existing-cookies
summary: Datasette plugin that authenticates users based on existing domain cookies
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-auth-existing-cookies
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-auth-existing-cookies/
project_url: https://pypi.org/project/datasette-auth-existing-cookies/
project_urls: {"Homepage": "https://github.com/simonw/datasette-auth-existing-cookies"}
release_url: https://pypi.org/project/datasette-auth-existing-cookies/0.7/
requires_python: >=3.6
version: 0.7
description:
  # datasette-auth-existing-cookies

  [![PyPI](https://img.shields.io/pypi/v/datasette-auth-existing-cookies.svg)](https://pypi.org/project/datasette-auth-existing-cookies/)
  [![CircleCI](https://circleci.com/gh/simonw/datasette-auth-existing-cookies.svg?style=svg)](https://circleci.com/gh/simonw/datasette-auth-existing-cookies)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://google.com/simonw/datasette-auth-existing-cookies/blob/master/LICENSE)

  Datasette plugin that authenticates users based on existing domain cookies.

  ## When to use this

  This plugin allows you to build custom authentication for Datasette when you are hosting a Datasette instance on the same domain as another, authenticated website.

  Consider a website on `www.example.com` which supports user authentication.

  You could run Datasette on `data.example.com` in a way that lets it see cookies that were set for the `.example.com` domain.

  Using this plugin, you could build an API endpoint at `www.example.com/user-for-cookies` which returns a JSON object representing the currently signed-in user, based on their cookies.

  The plugin can protect any hits to any `data.example.com` pages by passing their cookies through to that API and seeing if the user should be logged in or not.

  You can also use subclassing to decode existing cookies using some other mechanism.

  ## Configuration

  This plugin requires some configuration in the Datasette [metadata.json file](https://datasette.readthedocs.io/en/stable/plugins.html#plugin-configuration).

  It needs to know the following:

  * Which domain cookies should it be paying attention to? If you are authenticating against Dango this is probably `["sessionid"]`.
  * What's an API it can send the incoming cookies to that will decipher them into some user information?
  * Where should it redirect the user if they need to sign in?

  Example configuration setting all three of these values looks like this:

  ```json
  {
      "plugins": {
          "datasette-auth-existing-cookies": {
              "api_url": "http://www.example.com/user-from-cookies",
              "auth_redirect_url": "http://www.example.com/login",
              "original_cookies": ["sessionid"]
          }
      }
  }
  ```

  With this configuration the user's current `sessionid` cookie will be passed to the API URL, as a regular cookie header.

  You can use the `"headers_to_forward"` configuration option to specify a list of additional headers from the request that should be forwarded on to the `api_url` as querystring parameters. For example, if you add this to the above configuration:

  ```json
              "headers_to_forward": ["host", "x-forwarded-for"]
  ```

  Then a hit to `https://data.example.com/` would make the following API call:

      http://www.example.com/user-from-cookies?host=data.example.com&x-forwarded-for=64.18.15.255

  The API URL should then return either an empty JSON object if the user is not currently signed in:

  ```json
  {}
  ```

  Or a JSON object representing the user if they ARE signed in:

  ```json
  {
      "id": 123,
      "username": "simonw"
  }
  ```

  This object can contain any keys that you like - the information will be stored in a new signed cookie and made available to Datasette code as the `"auth"` dictionary on the ASGI `scope`.

  I suggest including at least an `id` and a `username`.

  ## Templates

  You probably want your user's to be able to see that they are signed in. The plugin makes the `auth` data from above directly available within Datasette's templates. You could use a custom `base.html` template (see [template documentation](https://datasette.readthedocs.io/en/stable/custom_templates.html#custom-templates)) that looks like this:

  ```html+django
  {% extends "default:base.html" %}

  {% block extra_head %}
  <style type="text/css">
  .hd .logout {
      float: right;
      text-align: right;
      padding-left: 1em;
  }
  </style>
  {% endblock %}

  {% block nav %}
      {{ super() }}
      {% if auth and auth.username %}
          <p class="logout">
              <strong>{{ auth.username }}</strong> &middot; <a href="https://www.example.com/logout">Log out</a>
          </p>
      {% endif %}
  {% endblock %}
  ```

  ## Other options

  - `require_auth`. This defaults to `True`. You can set it to `False` if you want unauthenticated users to be able to view the Datasette instance.
  - `cookie_secret`. You can use this to set the signing secret that will be used for the cookie set by this plugin (you should use [secret configuration values](https://datasette.readthedocs.io/en/stable/plugins.html#secret-configuration-values) for this). If you do not set a secret the plugin will create one on first run and store it in an appropriate state directory based on your operating system (the `user_state_dir` according to [appdirs](https://pypi.org/project/appdirs/)).
  - `cookie_ttl`. The plugin sets its own cookie to avoid hitting the backend API for every incoming request. By default it still hits the API at most every 10 seconds, in case the user has signed out on the main site. You can raise or lower the timeout using this setting.
  - `trust_x_forwarded_proto`. If you are running behind a proxy that adds HTTPS support for you, you may find that the plugin incorrectly constructs `?next=` URLs with the incorrect scheme. If you know your proxy sends the `x-forwarded-proto` header (you can investigate this with the [datasette-debug-asgi](https://github.com/simonw/datasette-debug-asgi) plugin) setting the `trust_x_forwarded_proto` option to True will cause the plugin to trust that header.
  - `next_secret`. See below.

  ## Login redirect mechanism

  If the user does not have a valid authentication cookie they will be redirected to an existing login page.

  This page is specified using the `auth_redirect_url` setting.

  Given the above example configuration, the URL that the user should be sent to after they log in will be specified as the `?next=` parameter to that page, for example:

      http://www.example.com/login?next=http://foo.example.com/

  It is up to you to program the login endpoint such that it is not vulnerable to an [Unvalidated redirect vulnerability](https://cheatsheetseries.owasp.org/cheatsheets/Unvalidated_Redirects_and_Forwards_Cheat_Sheet.html).

  One way to do this is by verifying that the URL passed to `?next=` is a URL that belongs to a trusted website. Django's own login view [does this](https://github.com/django/django/blob/50cf183d219face91822c75fa0a15fe2fe3cb32d/django/contrib/auth/views.py#L69-L80) by verifying that the URL hostname is on an approved list.

  Another way to do this is to use the `next_secret` configuration parameter to set a signing secret for that URL. This signing secret will be used to construct a `?next_sig=` signed token using the Python [itsdangerous](https://pythonhosted.org/itsdangerous/) module, like this:

      ?next_sig=Imh0dHBzOi8vZGVtby5leGFtcGxlLmNvbS9mb28vYmFyIg.7JdhRCoP7Ow1cRF1ZVengC-qk6c

  You should use Datasette's [secret configuration values](https://datasette.readthedocs.io/en/stable/plugins.html#secret-configuration-values) mechanism to set this secret from an environment variable, like so:

      {
          "plugins": {
              "datasette-auth-existing-cookies": {
                  "api_url": "http://www.example.com/user-from-cookies",
                  "auth_redirect_url": "http://www.example.com/login",
                  "original_cookies": ["sessionid"],
                  "next_secret":  {
                      "$env": "NEXT_SECRET"
                  }
              }
          }
      }

  You can verify this secret in Python code for your own login form like so:

  ```python
  from itsdangerous import URLSafeSerializer, BadSignature

  def verify_next_sig(next_sig):
      signer = URLSafeSerializer(next_secret)
      try:
          decoded = signer.loads(next_sig)
          return True
      except BadSignature:
          return False
  ```

  If you want to roll your own signing mechanism here you can do so by subclassing `ExistingCookiesAuth` and over-riding the `build_auth_redirect(next_url)` method.

  ## Permissions

  If the current user is signed in but should not have permission to access the Datasette instance, you can indicate so by having the API return the following:

  ```json
  {
      "forbidden": "You do not have permission to access this page."
  }
  ```

  The key must be `"forbidden"`. The value can be any string - it will be displayed to the user.

  This is particularly useful when handling multiple different subdomains. You may get an API call to the following:

      http://www.example.com/user-from-cookies?host=a-team.example.com

  You can check if the authenticated user (based on their cookies) has permission to access to the `a-team` Datasette instance, and return a `"forbidden"` JSON object if they should not be able to view it.

  If a user is allowed to access Datasette (because the API returned their user identity as JSON), the plugin will set a cookie on that subdomain granting them access.

  This cookie defaults to expiring after ten seconds. This means that if a user has permission removed for any reason they will still have up to ten seconds in which they will be able to continue accessing Datasette. If this is not acceptable to you the `cookie_ttl` setting can be used to reduce this timeout, at the expense of incurring more frequent API calls to check user permissions.

requires_dist:
  ["appdirs", "httpx", "itsdangerous", "datasette ; extra == 'test'", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "asgiref (~=3.1.2) ; extra == 'test'"]


name: datasette-auth-github
summary: Datasette plugin and ASGI middleware that authenticates users against GitHub
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-auth-github
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-auth-github/
project_url: https://pypi.org/project/datasette-auth-github/
project_urls: {"Homepage": "https://github.com/simonw/datasette-auth-github"}
release_url: https://pypi.org/project/datasette-auth-github/0.13.1/
version: 0.13.1
description:
  # datasette-auth-github

  [![PyPI](https://img.shields.io/pypi/v/datasette-auth-github.svg)](https://pypi.org/project/datasette-auth-github/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-auth-github?include_prereleases&label=changelog)](https://github.com/simonw/datasette-auth-github/releases)
  [![Tests](https://github.com/simonw/datasette-auth-github/workflows/Test/badge.svg)](https://github.com/simonw/datasette-auth-github/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-auth-github/blob/main/LICENSE)

  Datasette plugin that authenticates users against GitHub.

  <!-- toc -->

  - [Setup instructions](#setup-instructions)
  - [The authenticated actor](#the-authenticated-actor)
  - [Restricting access to specific users](#restricting-access-to-specific-users)
  - [Restricting access to specific GitHub organizations or teams](#restricting-access-to-specific-github-organizations-or-teams)
  - [What to do if a user is removed from an organization or team](#what-to-do-if-a-user-is-removed-from-an-organization-or-team)

  <!-- tocstop -->

  ## Setup instructions

  * Install the plugin: `datasette install datasette-auth-github`
  * Create a GitHub OAuth app: https://github.com/settings/applications/new
  * Set the Authorization callback URL to `http://127.0.0.1:8001/-/github-auth-callback`
  * Create a `metadata.json` file with the following structure:

  ```json
  {
      "title": "datasette-auth-github demo",
      "plugins": {
          "datasette-auth-github": {
              "client_id": {"$env": "GITHUB_CLIENT_ID"},
              "client_secret": {"$env": "GITHUB_CLIENT_SECRET"}
          }
      }
  }
  ```

  Now you can start Datasette like this, passing in the secrets as environment variables:

      $ GITHUB_CLIENT_ID=XXX GITHUB_CLIENT_SECRET=YYY datasette \
          fixtures.db -m metadata.json

  Note that hard-coding secrets in `metadata.json` is a bad idea as they will be visible to anyone who can navigate to `/-/metadata`. Instead, we use Datasette's mechanism for [adding secret plugin configuration options](https://docs.datasette.io/en/stable/plugins.html#secret-configuration-values).

  By default anonymous users will still be able to interact with Datasette. If you wish all users to have to sign in with a GitHub account first, add this to your ``metadata.json``:

  ```json
  {
      "allow": {
          "id": "*"
      },
      "plugins": {
          "datasette-auth-github": {
              "...": "..."
          }
      }
  }
  ```
  ## The authenticated actor

  Visit `/-/actor` when signed in to see the shape of the authenticated actor. It should look something like this:

  ```json
  {
      "actor": {
          "display": "simonw",
          "gh_id": "9599",
          "gh_name": "Simon Willison",
          "gh_login": "simonw",
          "gh_email": "...",
          "gh_orgs": [
              "dogsheep",
              "datasette-project"
          ],
          "gh_teams": [
              "dogsheep/test"
          ]
      }
  }
  ```

  The `gh_orgs` and `gh_teams` properties will only be present if you used `load_teams` or `load_orgs`, documented below.

  ## Restricting access to specific users

  You can use Datasette's [permissions mechanism](https://docs.datasette.io/en/stable/authentication.html) to specify which user or users are allowed to access your instance. Here's how to restrict access to just GitHub user `simonw`:

  ```json
  {
      "allow": {
          "gh_login": "simonw"
      },
      "plugins": {
          "datasette-auth-github": {
              "...": "..."
          }
      }
  }
  ```

  This `"allow"` block can be positioned at the database, table or query level instead: see [Configuring permissions in metadata.json](https://docs.datasette.io/en/stable/authentication.html#configuring-permissions-in-metadata-json) for details.

  Note that GitHub allows users to change their username, and it is possible for other people to claim old usernames. If you are concerned that your users may change their usernames you can key the allow blocks against GitHub user IDs instead, which do not change:

  ```json
  {
      "allow": {
          "gh_id": "9599"
      }
  }
  ```

  ## Restricting access to specific GitHub organizations or teams

  You can also restrict access to users who are members of a specific GitHub organization.

  You'll need to configure the plugin to check if the user is a member of that organization when they first sign in. You can do that using the `"load_orgs"` plugin configuration option.

  Then you can use `"allow": {"gh_orgs": [...]}` to specify which organizations are allowed access.

  ```json
  {
      "plugins": {
          "datasette-auth-github": {
              "...": "...",
              "load_orgs": ["your-organization"]
          }
      },
      "allow": {
          "gh_orgs": "your-organization"
      }
  }
  ```

  If your organization is [arranged into teams](https://help.github.com/en/articles/organizing-members-into-teams) you can restrict access to a specific team like this:

  ```json
  {
      "plugins": {
          "datasette-auth-github": {
              "...": "...",
              "load_teams": [
                  "your-organization/staff",
                  "your-organization/engineering",
              ]
          }
      },
      "allows": {
          "gh_team": "your-organization/engineering"
      }
  }
  ```

  ## What to do if a user is removed from an organization or team

  A user's organization and team memberships are checked once, when they first sign in. Those teams and organizations are then persisted in the user's signed `ds_actor` cookie.

  This means that if a user is removed from an organization or team but still has a Datasette cookie, they will still be able to access that Datasette instance.

  You can remedy this by rotating the `DATASETTE_SECRET` environment variable any time you make changes to your GitHub organization members.

  Changing this value will cause all of your existing users to be signed out, by invalidating their cookies. When they sign back in again their new memberships will be recorded in a new cookie.

  See [Configuring the secret](https://docs.datasette.io/en/stable/settings.html?highlight=secret#configuring-the-secret) in the Datasette documentation for more details.

requires_dist:
  ["datasette (>=0.51)", "datasette ; extra == 'test'", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "sqlite-utils ; extra == 'test'", "pytest-httpx ; extra == 'test'"]


name: datasette-auth-passwords
summary: Datasette plugin for authenticating access using passwords
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-auth-passwords
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-auth-passwords/
project_url: https://pypi.org/project/datasette-auth-passwords/
release_url: https://pypi.org/project/datasette-auth-passwords/1.0/
version: 1.0
description:
  # datasette-auth-passwords

  [![PyPI](https://img.shields.io/pypi/v/datasette-auth-passwords.svg)](https://pypi.org/project/datasette-auth-passwords/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-auth-passwords?label=changelog)](https://github.com/simonw/datasette-auth-passwords/releases)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-auth-passwords/blob/master/LICENSE)

  Datasette plugin for authenticating access using passwords

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-auth-passwords

  ## Demo

  A demo of this plugin is running at https://datasette-auth-passwords-demo.datasette.io/

  The demo is configured to show the `public.db` database to everyone, but the `private.db` database only to logged in users.

  You can log in at https://datasette-auth-passwords-demo.datasette.io/-/login with username `root` and password `password!`.

  ## Usage

  This plugin works based on a list of username/password accounts that are hard-coded into the plugin configuration.

  First, you'll need to create a password hash. There are three ways to do that:

  - Install the plugin, then use the interactive tool located at `/-/password-tool`
  - Use the hosted version of that tool at https://datasette-auth-passwords-demo.datasette.io/-/password-tool
  - Use the `datasette hash-password` command, described below

  Now add the following to your `metadata.json`:

  ```json
  {
      "plugins": {
          "datasette-auth-passwords": {
              "someusername_password_hash": {
                  "$env": "PASSWORD_HASH_1"
              }
          }
      }
  }
  ```

  The password hash can now be specified in an environment variable when you run Datasette. You can do that like so:

      PASSWORD_HASH_1='pbkdf2_sha256$...' \
          datasette -m metadata.json

  Be sure to use single quotes here otherwise the `$` symbols in the password hash may be incorrectly interpreted by your shell.

  You will now be able to log in to your instance using the form at `/-/login` with `someusername` as the username and the password that you used to create your hash as the password.

  You can include as many accounts as you like in the configuration, each with different usernames.

  ### datasette hash-password

  The plugin exposes a new CLI command, `datasette hash-password`. You can run this without arguments to interactively create a new password hash:
  ```
  % datasette hash-password
  Password: 
  Repeat for confirmation: 
  pbkdf2_sha256$260000$1513...
  ```
  Or if you want to use it as part of a script, you can add the `--no-confirm` option to generate a hash directly from a value passed to standard input:
  ```
  % echo 'my password' | datasette hash-password --no-confirm
  pbkdf2_sha256$260000$daa...
  ```
  ### Specifying actors

  By default, a logged in user will result in an [actor block](https://datasette.readthedocs.io/en/stable/authentication.html#actors) that just contains their username:

  ```json
  {
      "id": "someusername"
  }
  ```

  You can customize the actor that will be used for a username by including an `"actors"` configuration block, like this:

  ```json
  {
      "plugins": {
          "datasette-auth-passwords": {
              "someusername_password_hash": {
                  "$env": "PASSWORD_HASH_1"
              },
              "actors": {
                  "someusername": {
                      "id": "someusername",
                      "name": "Some user"
                  }
              }
          }
      }
  }
  ```
  ### HTTP Basic authentication option

  This plugin defaults to implementing login using an HTML form that sets a signed authentication cookie.

  You can alternatively configure it to use [HTTP Basic authentication](https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication#basic_authentication_scheme) instead.

  Do this by adding `"http_basic_auth": true` to the `datasette-auth-passwords` block in your plugin configuration.

  This option introduces the following behaviour:

  - Account usernames and passwords are configured in the same way as form-based authentication
  - Every page within Datasette - even pages that normally do not use authentication, such as static assets - will display a browser login prompt
  - Users will be unable to log out without closing their browser entirely

  There is a demo of this mode at https://datasette-auth-passwords-http-basic-demo.datasette.io/ - sign in with username `root` and password `password!`

  ### Using with datasette publish

  If you are publishing data using a [datasette publish](https://datasette.readthedocs.io/en/stable/publish.html#datasette-publish) command you can use the `--plugin-secret` option to securely configure your password hashes (see [secret configuration values](https://datasette.readthedocs.io/en/stable/plugins.html#secret-configuration-values)).

  You would run the command something like this:

      datasette publish cloudrun mydatabase.db \
          --install datasette-auth-passwords \
          --plugin-secret datasette-auth-passwords root_password_hash 'pbkdf2_sha256$...' \
          --service datasette-auth-passwords-demo

  This will allow you to log in as username `root` using the password that you used to create the hash.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-auth-passwords
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-auth-passwords/actions", "Changelog": "https://github.com/simonw/datasette-auth-passwords/releases", "Homepage": "https://github.com/simonw/datasette-auth-passwords", "Issues": "https://github.com/simonw/datasette-auth-passwords/issues"}

requires_dist:
  ["datasette (>=0.59)", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "httpx ; extra == 'test'"]


name: datasette-auth-tokens
summary: Datasette plugin for authenticating access using API tokens
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-auth-tokens
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-auth-tokens/
project_url: https://pypi.org/project/datasette-auth-tokens/
release_url: https://pypi.org/project/datasette-auth-tokens/0.3/
version: 0.3
description:
  # datasette-auth-tokens

  [![PyPI](https://img.shields.io/pypi/v/datasette-auth-tokens.svg)](https://pypi.org/project/datasette-auth-tokens/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-auth-tokens?include_prereleases&label=changelog)](https://github.com/simonw/datasette-auth-tokens/releases)
  [![Tests](https://github.com/simonw/datasette-auth-tokens/workflows/Test/badge.svg)](https://github.com/simonw/datasette-auth-tokens/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-auth-tokens/blob/main/LICENSE)

  Datasette plugin for authenticating access using API tokens

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ pip install datasette-auth-tokens

  ## Hard-coded tokens

  Read about Datasette's [authentication and permissions system](https://datasette.readthedocs.io/en/latest/authentication.html).

  This plugin lets you configure secret API tokens which can be used to make authenticated requests to Datasette.

  First, create a random API token. A useful recipe for doing that is the following:

      $ python -c 'import secrets; print(secrets.token_hex(32))'
      5f9a486dd807de632200b17508c75002bb66ca6fde1993db1de6cbd446362589

  Decide on the actor that this token should represent, for example:

  ```json
  {
      "bot_id": "my-bot"
  }
  ```

  You can then use `"allow"` blocks to provide that token with permission to access specific actions. To enable access to a configured writable SQL query you could use this in your `metadata.json`:

  ```json
  {
      "plugins": {
          "datasette-auth-tokens": {
              "tokens": [
                  {
                      "token": {
                          "$env": "BOT_TOKEN"
                      },
                      "actor": {
                          "bot_id": "my-bot"
                      }
                  }
              ]
          }
      },
      "databases": {
          ":memory:": {
              "queries": {
                  "show_version": {
                      "sql": "select sqlite_version()",
                      "allow": {
                          "bot_id": "my-bot"
                      }
                  }
              }
          }
      }
  }
  ```
  This uses Datasette's [secret configuration values mechanism](https://datasette.readthedocs.io/en/stable/plugins.html#secret-configuration-values) to allow the secret token to be passed as an environment variable.

  Run Datasette like this:

      BOT_TOKEN="this-is-the-secret-token" \
          datasette -m metadata.json

  You can now run authenticated API queries like this:

      $ curl -H 'Authorization: Bearer this-is-the-secret-token' \
        'http://127.0.0.1:8001/:memory:/show_version.json?_shape=array'
      [{"sqlite_version()": "3.31.1"}]

  Additionally you can allow passing the token as a query string parameter, although that's disabled by default given the security implications of URLs with secret tokens included. This may be useful to easily allow embedding data between different services.

  Simply enable it using the `param` config value:

  ```json
  {
      "plugins": {
          "datasette-auth-tokens": {
              "tokens": [
                  {
                      "token": {
                          "$env": "BOT_TOKEN"
                      },
                      "actor": {
                          "bot_id": "my-bot"
                      },
                  }
              ],
              "param": "_auth_token"
          }
      },
      "databases": {
          ":memory:": {
              "queries": {
                  "show_version": {
                      "sql": "select sqlite_version()",
                      "allow": {
                          "bot_id": "my-bot"
                      }
                  }
              }
          }
      }
  }
  ```

  You can now run authenticated API queries like this:

      $ curl http://127.0.0.1:8001/:memory:/show_version.json?_shape=array&_auth_token=this-is-the-secret-token
      [{"sqlite_version()": "3.31.1"}]

  ## Tokens from your database

  As an alternative (or in addition) to the hard-coded list of tokens you can store tokens in a database table and configure the plugin to access them using a SQL query.

  Your query needs to take a `:token_id` parameter and return at least two columns: one called `token_secret` and one called `actor_*` - usually `actor_id`. Further `actor_` prefixed columns can be returned to provide more details for the authenticated actor.

  Here's a simple example of a configuration query:

  ```sql
  select actor_id, actor_name, token_secret from tokens where token_id = :token_id
  ```

  This can run against a table like this one:

  | token_id | token_secret | actor_id | actor_name |
  | -------- | ------------ | -------- | ---------- |
  | 1        | bd3c94f51fcd | 78       | Cleopaws   |
  | 2        | 86681b4d6f66 | 32       | Pancakes   |

  The tokens are formed as the token ID, then a hyphen, then the token secret. For example:

  - `1-bd3c94f51fcd`
  - `2-86681b4d6f66`

  The SQL query will be executed with the portion before the hyphen as the `:token_id` parameter.

  The `token_secret` value returned by the query will be compared to the portion of the token after the hyphen to check if the token is valid.

  Columns with a prefix of `actor_` will be used to populate the actor dictionary. In the above example, a token of `2-86681b4d6f66` will become an actor dictionary of `{"id": 32, "name": "Pancakes"}`.

  To configure this, use a `"query"` block in your plugin configuration like this:

  ```json
  {
      "plugins": {
          "datasette-auth-tokens": {
              "query": {
                  "sql": "select actor_id, actor_name, token_secret from tokens where token_id = :token_id",
                  "database": "tokens"
              }
          }
      },
      "databases": {
          "tokens": {
              "allow": {}
          }
      }
  }
  ```
  The `"sql"` key here contains the SQL query. The `"database"` key has the name of the attached database file that the query should be executed against - in this case it would execute against `tokens.db`.

  ### Securing your tokens

  Anyone with access to your Datasette instance can use it to read the `token_secret` column in your tokens table. This probably isn't what you want!

  To avoid this, you should lock down access to that table. The configuration example above shows how to do this using an `"allow": {}` block. Consult Datasette's [Permissions documentation](https://datasette.readthedocs.io/en/stable/authentication.html#permissions) for more information about how to lock down this kind of access.

project_urls:
  {"CI": "https://github.com/simonw/datasette-auth-tokens/actions", "Changelog": "https://github.com/simonw/datasette-auth-tokens/releases", "Homepage": "https://github.com/simonw/datasette-auth-tokens", "Issues": "https://github.com/simonw/datasette-auth-tokens/issues"}

requires_dist:
  ["datasette (>=0.44)", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "httpx ; extra == 'test'", "sqlite-utils ; extra == 'test'"]


name: datasette-auth0
summary: Datasette plugin that authenticates users using Auth0
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-auth0
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-auth0/
project_url: https://pypi.org/project/datasette-auth0/
release_url: https://pypi.org/project/datasette-auth0/0.1/
requires_python: >=3.7
version: 0.1
description:
  # datasette-auth0

  [![PyPI](https://img.shields.io/pypi/v/datasette-auth0.svg)](https://pypi.org/project/datasette-auth0/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-auth0?include_prereleases&label=changelog)](https://github.com/simonw/datasette-auth0/releases)
  [![Tests](https://github.com/simonw/datasette-auth0/workflows/Test/badge.svg)](https://github.com/simonw/datasette-auth0/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-auth0/blob/main/LICENSE)

  Datasette plugin that authenticates users using [Auth0](https://auth0.com/)

  See [Simplest possible OAuth authentication with Auth0](https://til.simonwillison.net/auth0/oauth-with-auth0) for more about how this plugin works.

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-auth0

  ## Demo

  You can try this out at [datasette-auth0-demo.datasette.io](https://datasette-auth0-demo.datasette.io/) - click on the top right menu icon and select "Sign in with Auth0".

  ## Initial configuration

  First, create a new application in Auth0. You will need the domain, client ID and client secret for that application.

  The domain should be something like `mysite.us.auth0.com`.

  Add `http://127.0.0.1:8001/-/auth0-callback` to the list of Allowed Callback URLs.

  Then configure these plugin secrets using `metadata.yml`:

  ```yaml
  plugins:
    datasette-auth0:
      domain:
        "$env": AUTH0_DOMAIN
      client_id:
        "$env": AUTH0_CLIENT_ID
      client_secret:
        "$env": AUTH0_CLIENT_SECRET
  ```
  Only the `client_secret` needs to be kept secret, but for consistency I recommend using the `$env` mechanism for all three.

  In development, you can run Datasette and pass in environment variables like this:
  ```
  AUTH0_DOMAIN="your-domain.us.auth0.com" \
  AUTH0_CLIENT_ID="...client-id-goes-here..." \
  AUTH0_CLIENT_SECRET="...secret-goes-here..." \
  datasette -m metadata.yml
  ```

  If you are deploying using `datasette publish` you can pass these using `--plugin-secret`. For example, to deploy using Cloud Run you might run the following:
  ```
  datasette publish cloudrun mydatabase.db \
  --install datasette-auth0 \
  --plugin-secret datasette-auth0 domain "your-domain.us.auth0.com" \
  --plugin-secret datasette-auth0 client_id "your-client-id" \
  --plugin-secret datasette-auth0 client_secret "your-client-secret" \
  --service datasette-auth0-demo
  ```
  Once your Datasette instance is deployed, you will need to add its callback URL to the "Allowed Callback URLs" list in Auth0.

  The callback URL should be something like:

      https://url-to-your-datasette/-/auth0-callback

  ## Usage

  Once installed, a "Sign in with Auth0" menu item will appear in the Datasette main menu.

  You can sign in and then visit the `/-/actor` page to see full details of the `auth0` profile that has been authenticated.

  You can then use [Datasette permissions](https://docs.datasette.io/en/stable/authentication.html#configuring-permissions-in-metadata-json) to grant or deny access to different parts of Datasette based on the authenticated user.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-auth0
      python3 -mvenv venv
      source venv/bin/activate

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-auth0/actions", "Changelog": "https://github.com/simonw/datasette-auth0/releases", "Homepage": "https://github.com/simonw/datasette-auth0", "Issues": "https://github.com/simonw/datasette-auth0/issues"}

requires_dist:
  ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "pytest-httpx ; extra == 'test'"]


name: datasette-backup
summary: Plugin adding backup options to Datasette
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-backup
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-backup/
project_url: https://pypi.org/project/datasette-backup/
release_url: https://pypi.org/project/datasette-backup/0.1/
version: 0.1
description:
  # datasette-backup

  [![PyPI](https://img.shields.io/pypi/v/datasette-backup.svg)](https://pypi.org/project/datasette-backup/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-backup?include_prereleases&label=changelog)](https://github.com/simonw/datasette-backup/releases)
  [![Tests](https://github.com/simonw/datasette-backup/workflows/Test/badge.svg)](https://github.com/simonw/datasette-backup/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-backup/blob/main/LICENSE)

  Plugin adding backup options to Datasette

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-backup

  ## Usage

  Once installed, you can download a SQL backup of any of your databases from:

      /-/backup/dbname.sql

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-backup
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-backup/actions", "Changelog": "https://github.com/simonw/datasette-backup/releases", "Homepage": "https://github.com/simonw/datasette-backup", "Issues": "https://github.com/simonw/datasette-backup/issues"}

requires_dist:
  ["datasette", "sqlite-dump (>=0.1.1)", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "httpx ; extra == 'test'", "sqlite-utils ; extra == 'test'"]


name: datasette-basemap
summary: A basemap for Datasette and datasette-leaflet
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-basemap
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-basemap/
project_url: https://pypi.org/project/datasette-basemap/
release_url: https://pypi.org/project/datasette-basemap/0.2/
requires_dist: ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
requires_python: >=3.6
version: 0.2
description:
  # datasette-basemap

  [![PyPI](https://img.shields.io/pypi/v/datasette-basemap.svg)](https://pypi.org/project/datasette-basemap/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-basemap?include_prereleases&label=changelog)](https://github.com/simonw/datasette-basemap/releases)
  [![Tests](https://github.com/simonw/datasette-basemap/workflows/Test/badge.svg)](https://github.com/simonw/datasette-basemap/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-basemap/blob/main/LICENSE)

  A basemap for Datasette and datasette-leaflet

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-basemap

  ## Usage

  This plugin will make a `basemap` database available containing OpenStreetMap tiles in the [mbtiles](https://github.com/mapbox/mbtiles-spec) format. It is designed for use with the [datasette-tiles](https://datasette.io/plugins/datasette-tiles) tile server plugin.

  ## Demo

  You can preview this map at https://datasette-tiles-demo.datasette.io/-/tiles/basemap and browse the database directly at https://datasette-tiles-demo.datasette.io/basemap

  ## License

  The data bundled with this package is © OpenStreetMap contributors, licensed under the [Open Data Commons Open Database License](https://opendatacommons.org/licenses/odbl/). See [this page](https://www.openstreetmap.org/copyright) for more details.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-basemap
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-basemap/actions", "Changelog": "https://github.com/simonw/datasette-basemap/releases", "Homepage": "https://github.com/simonw/datasette-basemap", "Issues": "https://github.com/simonw/datasette-basemap/issues"}


name: datasette-block-robots
summary: Datasette plugin that blocks all robots using robots.txt
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-block-robots
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-block-robots/
project_url: https://pypi.org/project/datasette-block-robots/
release_url: https://pypi.org/project/datasette-block-robots/1.1/
version: 1.1
description:
  # datasette-block-robots

  [![PyPI](https://img.shields.io/pypi/v/datasette-block-robots.svg)](https://pypi.org/project/datasette-block-robots/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-block-robots?label=changelog)](https://github.com/simonw/datasette-block-robots/releases)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-block-robots/blob/master/LICENSE)

  Datasette plugin that blocks robots and crawlers using robots.txt

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ pip install datasette-block-robots

  ## Usage

  Having installed the plugin, `/robots.txt` on your Datasette instance will return the following:

      User-agent: *
      Disallow: /

  This will request all robots and crawlers not to visit any of the pages on your site.

  Here's a demo of the plugin in action: https://sqlite-generate-demo.datasette.io/robots.txt

  ## Configuration

  By default the plugin will block all access to the site, using `Disallow: /`.

  If you want the index page to be indexed by search engines without crawling the database, table or row pages themselves, you can use the following:

  ```json
  {
      "plugins": {
          "datasette-block-robots": {
              "allow_only_index": true
          }
      }
  }
  ```
  This will return a `/robots.txt` like so:

      User-agent: *
      Disallow: /db1
      Disallow: /db2

  With a `Disallow` line for every attached database.

  To block access to specific areas of the site using custom paths, add this to your `metadata.json` configuration file:

  ```json
  {
      "plugins": {
          "datasette-block-robots": {
              "disallow": ["/mydatabase/mytable"]
          }
      }
  }
  ```
  This will result in a `/robots.txt` that looks like this:

      User-agent: *
      Disallow: /mydatabase/mytable

  Alternatively you can set the full contents of the `robots.txt` file using the `literal` configuration option. Here's how to do that if you are using YAML rather than JSON and have a `metadata.yml` file:

  ```yaml
  plugins:
      datasette-block-robots:
          literal: |-
              User-agent: *
              Disallow: /
              User-agent: Bingbot
              User-agent: Googlebot
              Disallow:
  ```
  This example would block all crawlers with the exception of Googlebot and Bingbot, which are allowed to crawl the entire site.

  ## Extending this with other plugins

  This plugin adds a new [plugin hook](https://docs.datasette.io/en/stable/plugin_hooks.html) to Datasete called `block_robots_extra_lines()` which can be used by other plugins to add their own additional lines to the `robots.txt` file.

  The hook can optionally accept these parameters:

  - `datasette`: The current [Datasette instance](https://docs.datasette.io/en/stable/internals.html#datasette-class). You can use this to execute SQL queries or read plugin configuration settings.
  - `request`: The [Request object](https://docs.datasette.io/en/stable/internals.html#request-object) representing the incoming request to `/robots.txt`.

  The hook should return a list of strings, each representing a line to be added to the `robots.txt` file.

  It can also return an `async def` function, which will be awaited and used to generate a list of lines. Use this option if you need to make `await` calls inside you hook implementation.

  This example uses the hook to add a `Sitemap: http://example.com/sitemap.xml` line to the `robots.txt` file:

  ```python
  from datasette import hookimpl

  @hookimpl
  def block_robots_extra_lines(datasette, request):
      return [
          "Sitemap: {}".format(datasette.absolute_url(request, "/sitemap.xml")),
      ]
  ```
  This example blocks access to paths based on a database query:

  ```python
  @hookimpl
  def block_robots_extra_lines(datasette):
      async def inner():
          db = datasette.get_database()
          result = await db.execute("select path from mytable")
          return [
              "Disallow: /{}".format(row["path"]) for row in result
          ]
      return inner
  ```
  [datasette-sitemap](https://datasette.io/plugins/datasette-sitemap) is an example of a plugin that uses this hook.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-block-robots
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-block-robots/actions", "Changelog": "https://github.com/simonw/datasette-block-robots/releases", "Homepage": "https://github.com/simonw/datasette-block-robots", "Issues": "https://github.com/simonw/datasette-block-robots/issues"}

requires_dist:
  ["datasette (>=0.50)", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "httpx ; extra == 'test'"]


name: datasette-block
summary: Block all access to specific path prefixes
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-block
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-block/
project_url: https://pypi.org/project/datasette-block/
release_url: https://pypi.org/project/datasette-block/0.1.1/
requires_python: >=3.6
version: 0.1.1
description:
  # datasette-block

  [![PyPI](https://img.shields.io/pypi/v/datasette-block.svg)](https://pypi.org/project/datasette-block/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-block?include_prereleases&label=changelog)](https://github.com/simonw/datasette-block/releases)
  [![Tests](https://github.com/simonw/datasette-block/workflows/Test/badge.svg)](https://github.com/simonw/datasette-block/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-block/blob/main/LICENSE)

  Block all access to specific path prefixes

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-block

  ## Configuration

  Add the following to `metadata.json` to block specific path prefixes:

  ```json
  {
      "plugins": {
          "datasette-block": {
              "prefixes": ["/all/"]
          }
      }
  }
  ```

  This will cause a 403 error to be returned for any path beginning with `/all/`.

  This blocking happens as an ASGI wrapper around Datasette.

  ## Why would you need this?

  You almost always would not. I use it with `datasette-ripgrep` to block access to static assets for unauthenticated users.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-block
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-block/actions", "Changelog": "https://github.com/simonw/datasette-block/releases", "Homepage": "https://github.com/simonw/datasette-block", "Issues": "https://github.com/simonw/datasette-block/issues"}

requires_dist:
  ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "asgi-lifespan ; extra == 'test'"]


name: datasette-bplist
summary: Datasette plugin for working with Apple's binary plist format
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-bplist
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-bplist/
project_url: https://pypi.org/project/datasette-bplist/
project_urls: {"Homepage": "https://github.com/simonw/datasette-bplist"}
release_url: https://pypi.org/project/datasette-bplist/0.1/
requires_dist: ["datasette", "bpylist", "pytest ; extra == 'test'"]
version: 0.1
description:
  # datasette-bplist

  [![PyPI](https://img.shields.io/pypi/v/datasette-bplist.svg)](https://pypi.org/project/datasette-bplist/)
  [![CircleCI](https://circleci.com/gh/simonw/datasette-bplist.svg?style=svg)](https://circleci.com/gh/simonw/datasette-bplist)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-bplist/blob/master/LICENSE)

  Datasette plugin for working with Apple's [binary plist](https://en.wikipedia.org/wiki/Property_list) format.

  This plugin adds two features: a display hook and a SQL function.

  The display hook will detect any database values that are encoded using the binary plist format. It will decode them, convert them into JSON and display them pretty-printed in the Datasette UI.

  The SQL function `bplist_to_json(value)` can be used inside a SQL query to convert a binary plist value into a JSON string. This can then be used with SQLite's `json_extract()` function or with the [datasette-jq](https://github.com/simonw/datasette-jq) plugin to further analyze that data as part of a SQL query.

  Install this plugin in the same environment as Datasette to enable this new functionality:

      pip install datasette-bplist

  ## Trying it out

  If you use a Mac you already have plenty of SQLite databases that contain binary plist data.

  One example is the database that powers the Apple Photos app.

  This database tends to be locked, so you will need to create a copy of the database in order to run queries against it:

      cp ~/Pictures/Photos\ Library.photoslibrary/database/photos.db /tmp/photos.db

  The database also makes use of custom SQLite extensions which prevent it from opening in Datasette.

  You can work around this by exporting the data that you want to experiment with into a new SQLite file.

  I recommend trying this plugin against the `RKMaster_dataNote` table, which contains plist-encoded EXIF metadata about the photos you have taken.

  You can export that table into a fresh database like so:

      sqlite3 /tmp/photos.db ".dump RKMaster_dataNote" | sqlite3 /tmp/exif.db

  Now run `datasette /tmp/exif.db` and you can start trying out the plugin.

  ## Using the bplist_to_json() SQL function

  Once you have the `exif.db` demo working, you can try the `bplist_to_json()` SQL function.

  Here's a query that shows the camera lenses you have used the most often to take photos:

      select
          json_extract(
              bplist_to_json(value),
              "$.{Exif}.LensModel"
          ) as lens,
          count(*) as n
      from RKMaster_dataNote
      group by lens
      order by n desc;

  If you have a large number of photos this query can take a long time to execute, so you may need to increase the SQL time limit enforced by Datasette like so:

      $ datasette /tmp/exif.db \
          --config sql_time_limit_ms:10000

  Here's another query, showing the time at which you took every photo in your library which is classified as as screenshot:

      select
          attachedToId,
          json_extract(
              bplist_to_json(value),
              "$.{Exif}.DateTimeOriginal"
          )
      from RKMaster_dataNote
      where
          json_extract(
              bplist_to_json(value),
              "$.{Exif}.UserComment"
          ) = "Screenshot"

  And if you install the [datasette-cluster-map](https://github.com/simonw/datasette-cluster-map) plugin, this query will show you a map of your most recent 1000 photos:

      select
          *, 
          json_extract(
              bplist_to_json(value),
              "$.{GPS}.Latitude"
          ) as latitude,
          -json_extract(
              bplist_to_json(value),
              "$.{GPS}.Longitude"
          ) as longitude,
          json_extract(
              bplist_to_json(value),
              "$.{Exif}.DateTimeOriginal"
          ) as datetime
      from
          RKMaster_dataNote
      where
          latitude is not null
      order by
          attachedToId desc


name: datasette-clone
summary: Create a local copy of database files from a Datasette instance
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-clone
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-clone/
project_url: https://pypi.org/project/datasette-clone/
project_urls: {"Homepage": "https://github.com/simonw/datasette-clone"}
release_url: https://pypi.org/project/datasette-clone/0.5/
requires_dist: ["requests", "click", "pytest ; extra == 'test'", "requests-mock ; extra == 'test'"]
version: 0.5
description:
  # datasette-clone

  [![PyPI](https://img.shields.io/pypi/v/datasette-clone.svg)](https://pypi.org/project/datasette-clone/)
  [![CircleCI](https://circleci.com/gh/simonw/datasette-clone.svg?style=svg)](https://circleci.com/gh/simonw/datasette-clone)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-clone/blob/master/LICENSE)

  Create a local copy of database files from a Datasette instance.

  See [datasette-clone](https://simonwillison.net/2020/Apr/14/datasette-clone/) on my blog for background on this project.

  ## How to install

      $ pip install datasette-clone

  ## Usage

  This only works against Datasette instances running immutable databases (with the `-i` option). Databases published using the `datasette publish` command should be compatible with this tool.

  To download copies of all `.db` files from an instance, run:

      datasette-clone https://latest.datasette.io

  You can provide an optional second argument to specify a directory:

      datasette-clone https://latest.datasette.io /tmp/here-please

  The command stores its own copy of a `databases.json` manifest and uses it to only download databases that have changed the next time you run the command.

  It also stores a copy of the instance's `metadata.json` to ensure you have a copy of any source and licensing information for the downloaded databases.

  If your instance is protected by an API token, you can use `--token` to provide it:

      datasette-clone https://latest.datasette.io --token=xyz

  For verbose output showing what the tool is doing, use `-v`.


name: datasette-cluster-map
summary: Datasette plugin that shows a map for any data with latitude/longitude columns
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-cluster-map
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-cluster-map/
project_url: https://pypi.org/project/datasette-cluster-map/
release_url: https://pypi.org/project/datasette-cluster-map/0.17.2/
version: 0.17.2
description:
  # datasette-cluster-map

  [![PyPI](https://img.shields.io/pypi/v/datasette-cluster-map.svg)](https://pypi.org/project/datasette-cluster-map/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-cluster-map?include_prereleases&label=changelog)](https://github.com/simonw/datasette-cluster-map/releases)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-cluster-map/blob/main/LICENSE)

  A [Datasette plugin](https://docs.datasette.io/en/stable/plugins.html) that detects tables with `latitude` and `longitude` columns and then plots them on a map using [Leaflet.markercluster](https://github.com/Leaflet/Leaflet.markercluster).

  More about this project: [Datasette plugins, and building a clustered map visualization](https://simonwillison.net/2018/Apr/20/datasette-plugins/).

  ## Demo

  [global-power-plants.datasettes.com](https://global-power-plants.datasettes.com/global-power-plants/global-power-plants) hosts a demo of this plugin running against a database of 33,000 power plants around the world.

  ![Cluster map demo](https://static.simonwillison.net/static/2020/global-power-plants.png)

  ## Installation

  Run `datasette install datasette-cluster-map` to add this plugin to your Datasette virtual environment. Datasette will automatically load the plugin if it is installed in this way.

  If you are deploying using the `datasette publish` command you can use the `--install` option:

      datasette publish cloudrun mydb.db --install=datasette-cluster-map

  If any of your tables have a `latitude` and `longitude` column, a map will be automatically displayed.

  ## Configuration

  If your columns are called something else you can configure the column names using [plugin configuration](https://docs.datasette.io/en/stable/plugins.html#plugin-configuration) in a `metadata.json` file. For example, if all of your columns are called `xlat` and `xlng` you can create a `metadata.json` file like this:

  ```json
  {
      "title": "Regular metadata keys can go here too",
      "plugins": {
          "datasette-cluster-map": {
              "latitude_column": "xlat",
              "longitude_column": "xlng"
          }
      }
  }
  ```

  Then run Datasette like this:

      datasette mydata.db -m metadata.json

  This will configure the required column names for every database loaded by that Datasette instance.

  If you want to customize the column names for just one table in one database, you can do something like this:

  ```json
  {
      "databases": {
          "polar-bears": {
              "tables": {
                  "USGS_WC_eartag_deployments_2009-2011": {
                      "plugins": {
                          "datasette-cluster-map": {
                              "latitude_column": "Capture Latitude",
                              "longitude_column": "Capture Longitude"
                          }
                      }
                  }
              }
          }
      }
  }
  ```

  You can also use a custom SQL query to rename those columns to `latitude` and `longitude`, [for example](https://polar-bears.now.sh/polar-bears?sql=select+*%2C%0D%0A++++%22Capture+Latitude%22+as+latitude%2C%0D%0A++++%22Capture+Longitude%22+as+longitude%0D%0Afrom+%5BUSGS_WC_eartag_deployments_2009-2011%5D):

  ```sql
  select *,
      "Capture Latitude" as latitude,
      "Capture Longitude" as longitude
  from [USGS_WC_eartag_deployments_2009-2011]
  ```

  The map defaults to being displayed above the main results table on the page. You can use the `"container"` plugin setting to provide a CSS selector indicating an element that the map should be appended to instead.

  ## Custom tile layers

  You can customize the tile layer used  by the maps using the `tile_layer` and `tile_layer_options` configuration settings. For example, to use the [Stamen Watercolor tiles](http://maps.stamen.com/watercolor/#12/37.7706/-122.3782) you can use these settings:

  ```json
  {
      "plugins": {
          "datasette-cluster-map": {
              "tile_layer": "https://stamen-tiles-{s}.a.ssl.fastly.net/watercolor/{z}/{x}/{y}.{ext}",
              "tile_layer_options": {
                  "attribution": "Map tiles by <a href=\"http://stamen.com\">Stamen Design</a>, <a href=\"http://creativecommons.org/licenses/by/3.0\">CC BY 3.0</a> &mdash; Map data &copy; <a href=\"https://www.openstreetmap.org/copyright\">OpenStreetMap</a> contributors",
                  "subdomains": "abcd",
                  "minZoom": 1,
                  "maxZoom": 16,
                  "ext": "jpg"
              }
          }
      }
  }
  ```
  The [Leaflet Providers preview list](https://leaflet-extras.github.io/leaflet-providers/preview/index.html) has details of many other tile layers you can use.

  ## Custom marker popups

  The marker popup defaults to displaying the data for the underlying database row.

  You can customize this by including a `popup` column in your results containing JSON that defines a more useful popup.

  The JSON in the popup column should look something like this:

  ```json
  {
      "image": "https://niche-museums.imgix.net/dodgems.heic?w=800&h=400&fit=crop",
      "alt": "Dingles Fairground Heritage Centre",
      "title": "Dingles Fairground Heritage Centre",
      "description": "Home of the National Fairground Collection, Dingles has over 45,000 indoor square feet of vintage fairground rides... and you can go on them! Highlights include the last complete surviving and opera",
      "link": "/browse/museums/26"
  }
  ```

  Each of these columns is optional.

  - `title` is the title to show at the top of the popup
  - `image` is the URL to an image to display in the popup
  - `alt` is the alt attribute to use for that image
  - `description` is a longer string of text to use as a description
  - `link` is a URL that the marker content should link to

  You can use the SQLite `json_object()` function to construct this data dynamically as part of your SQL query. Here's an example:

  ```sql
  select json_object(
    'image', photo_url || '?w=800&h=400&fit=crop',
    'title', name,
    'description', substr(description, 0, 200),
    'link', '/browse/museums/' || id
    ) as popup,
    latitude, longitude from museums
  where id in (26, 27) order by id
  ```

  [Try that example here](https://www.niche-museums.com/browse?sql=select+json_object%28%0D%0A++%27image%27%2C+photo_url+%7C%7C+%27%3Fw%3D800%26h%3D400%26fit%3Dcrop%27%2C%0D%0A++%27title%27%2C+name%2C%0D%0A++%27description%27%2C+substr%28description%2C+0%2C+200%29%2C%0D%0A++%27link%27%2C+%27%2Fbrowse%2Fmuseums%2F%27+%7C%7C+id%0D%0A++%29+as+popup%2C%0D%0A++latitude%2C+longitude+from+museums) or take a look at [this demo built using a SQL view](https://dogsheep-photos.dogsheep.net/public/photos_on_a_map).

  ## How I deployed the demo

      datasette publish cloudrun global-power-plants.db \
          --service global-power-plants \
          --metadata metadata.json \
          --install=datasette-cluster-map \
          --extra-options="--config facet_time_limit_ms:1000"

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-cluster-map
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-cluster-map/actions", "Changelog": "https://github.com/simonw/datasette-cluster-map/releases", "Homepage": "https://github.com/simonw/datasette-cluster-map", "Issues": "https://github.com/simonw/datasette-cluster-map/issues"}

requires_dist:
  ["datasette (>=0.54)", "datasette-leaflet (>=0.2.2)", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "httpx ; extra == 'test'", "sqlite-utils ; extra == 'test'"]


name: datasette-column-inspect
summary: Experimental Datasette plugin for inspecting columns
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-column-inspect
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-column-inspect/
project_url: https://pypi.org/project/datasette-column-inspect/
project_urls: {"Homepage": "https://github.com/simonw/datasette-column-inspect"}
release_url: https://pypi.org/project/datasette-column-inspect/0.2a0/
version: 0.2a0
description:
  # datasette-column-inspect

  [![PyPI](https://img.shields.io/pypi/v/datasette-column-inspect.svg)](https://pypi.org/project/datasette-column-inspect/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-column-inspect?include_prereleases&label=changelog)](https://github.com/simonw/datasette-column-inspect/releases)
  [![Tests](https://github.com/simonw/datasette-column-inspect/workflows/Test/badge.svg)](https://github.com/simonw/datasette-column-inspect/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-column-inspect/blob/main/LICENSE)

  Highly experimental Datasette plugin for inspecting columns.

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ pip install datasette-column-inspect

  ## Usage

  This plugin adds an icon to each column on the table page which opens an inspection side panel.

requires_dist:
  ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "httpx ; extra == 'test'", "sqlite-utils ; extra == 'test'"]


name: datasette-configure-asgi
summary: Datasette plugin for configuring arbitrary ASGI middleware
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-configure-asgi
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-configure-asgi/
project_url: https://pypi.org/project/datasette-configure-asgi/
project_urls: {"Homepage": "https://github.com/simonw/datasette-configure-asgi"}
release_url: https://pypi.org/project/datasette-configure-asgi/0.1/
version: 0.1
description:
  # datasette-configure-asgi

  [![PyPI](https://img.shields.io/pypi/v/datasette-configure-asgi.svg)](https://pypi.org/project/datasette-configure-asgi/)
  [![CircleCI](https://circleci.com/gh/simonw/datasette-configure-asgi.svg?style=svg)](https://circleci.com/gh/simonw/datasette-configure-asgi)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-configure-asgi/blob/master/LICENSE)

  Datasette plugin for configuring arbitrary ASGI middleware

  ## Installation

      pip install datasette-configure-asgi

  ## Usage

  This plugin only takes effect if your `metadata.json` file contains relevant top-level plugin configuration in a `"datasette-configure-asgi"` configuration key.

  For example, to wrap your Datasette instance in the `asgi-log-to-sqlite` middleware configured to write logs to `/tmp/log.db` you would use the following:

  ```json
  {
      "plugins": {
          "datasette-configure-asgi": [
              {
                  "class": "asgi_log_to_sqlite.AsgiLogToSqlite",
                  "args": {
                      "file": "/tmp/log.db"
                  }
              }
          ]
      }
  }
  ```

  The `"datasette-configure-asgi"` key should be a list of JSON objects. Each object should have a `"class"` key indicating the class to be used, and an optional `"args"` key providing any necessary arguments to be passed to that class constructor.

  ## Plugin structure

  This plugin can be used to wrap your Datasette instance in any ASGI middleware that conforms to the following structure:

  ```python
  class SomeAsgiMiddleware:
      def __init__(self, app, arg1, arg2):
          self.app = app
          self.arg1 = arg1
          self.arg2 = arg2

      async def __call__(self, scope, receive, send):
          start = time.time()
          await self.app(scope, receive, send)
          end = time.time()
          print("Time taken: {}".format(end - start))
  ```

  So the middleware is a class with a constructor which takes the wrapped application as a first argument, `app`, followed by further named arguments to configure the middleware. It provides an `async def __call__(self, scope, receive, send)` method to implement the middleware's behavior.

requires_dist:
  ["pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "asgiref (==3.1.2) ; extra == 'test'", "datasette ; extra == 'test'"]


name: datasette-configure-fts
summary: Datasette plugin for enabling full-text search against selected table columns
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-configure-fts
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-configure-fts/
project_url: https://pypi.org/project/datasette-configure-fts/
project_urls: {"Homepage": "https://github.com/simonw/datasette-configure-fts"}
release_url: https://pypi.org/project/datasette-configure-fts/1.1/
version: 1.1
description:
  # datasette-configure-fts

  [![PyPI](https://img.shields.io/pypi/v/datasette-configure-fts.svg)](https://pypi.org/project/datasette-configure-fts/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-configure-fts?include_prereleases&label=changelog)](https://github.com/simonw/datasette-configure-fts/releases)
  [![Tests](https://github.com/simonw/datasette-configure-fts/workflows/Test/badge.svg)](https://github.com/simonw/datasette-configure-fts/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-configure-fts/blob/main/LICENSE)

  Datasette plugin for enabling full-text search against selected table columns

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-configure-fts

  ## Usage

  Having installed the plugin, visit `/-/configure-fts` on your Datasette instance to configure FTS for tables on attached writable databases.

  By default only [the root actor](https://datasette.readthedocs.io/en/stable/authentication.html#using-the-root-actor) can access the page - so you'll need to run Datasette with the `--root` option and click on the link shown in the terminal to sign in and access the page.

  The `configure-fts` permission governs access. You can use permission plugins such as [datasette-permissions-sql](https://github.com/simonw/datasette-permissions-sql) to grant additional access to the write interface.

requires_dist:
  ["datasette (>=0.51)", "sqlite-utils (>=2.10)", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "httpx ; extra == 'test'"]


name: datasette-copy-to-memory
summary: Copy database files into an in-memory database on startup
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-copy-to-memory
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-copy-to-memory/
project_url: https://pypi.org/project/datasette-copy-to-memory/
release_url: https://pypi.org/project/datasette-copy-to-memory/0.2/
requires_python: >=3.7
version: 0.2
description:
  # datasette-copy-to-memory

  [![PyPI](https://img.shields.io/pypi/v/datasette-copy-to-memory.svg)](https://pypi.org/project/datasette-copy-to-memory/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-copy-to-memory?include_prereleases&label=changelog)](https://github.com/simonw/datasette-copy-to-memory/releases)
  [![Tests](https://github.com/simonw/datasette-copy-to-memory/workflows/Test/badge.svg)](https://github.com/simonw/datasette-copy-to-memory/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-copy-to-memory/blob/main/LICENSE)

  Copy database files into an in-memory database on startup

  This plugin is **highly experimental**. It currently exists to support Datasette performance research, and is not designed for actual production usage.

  ## Installation

  Install this plugin in the same environment as Datasette.

      datasette install datasette-copy-to-memory

  ## Usage

  On startup, Datasette will create an in-memory named database for each attached database. This database will have the same name but with `_memory` at the end.

  So running this:

      datasette fixtures.db

  Will serve two databases: the original at `/fixtures` and the in-memory copy at `/fixtures_memory`.

  ## Demo

  A demo is running on [latest-with-plugins.datasette.io](https://latest-with-plugins.datasette.io/) - the [/fixtures_memory](https://latest-with-plugins.datasette.io/fixtures_memory) table there is provided by this plugin.

  ## Configuration

  By default every attached database file will be loaded into a `_memory` copy.

  You can use plugin configuration to specify just a subset of the database. For example, to create `github_memory` but not `fixtures_memory` you would use the following `metadata.yml` file:

  ```yaml
  plugins:
    datasette-copy-to-memory:
      databases:
      - github
  ```
  Then start Datasette like this:

      datasette github.db fixtures.db -m metadata.yml

  If you don't want to have a `fixtures` and `fixtures_memory` database, you can use `replace: true` to have the plugin replace the file-backed database with the new in-memory one, reusing the same database name:

  ```yaml
  plugins:
    datasette-copy-to-memory:
      replace: true
  ```
  Then:

      datasette github.db fixtures.db -m metadata.yml

  This will result in both `/github` and `/fixtures` but no `/github_memory` or `/fixtures_memory`.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-copy-to-memory
      python3 -m venv venv
      source venv/bin/activate

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-copy-to-memory/actions", "Changelog": "https://github.com/simonw/datasette-copy-to-memory/releases", "Homepage": "https://github.com/simonw/datasette-copy-to-memory", "Issues": "https://github.com/simonw/datasette-copy-to-memory/issues"}

requires_dist:
  ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "sqlite-utils ; extra == 'test'"]


name: datasette-copyable
summary: Datasette plugin for outputting tables in formats suitable for copy and paste
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-copyable
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-copyable/
project_url: https://pypi.org/project/datasette-copyable/
release_url: https://pypi.org/project/datasette-copyable/0.3.1/
version: 0.3.1
description:
  # datasette-copyable

  [![PyPI](https://img.shields.io/pypi/v/datasette-copyable.svg)](https://pypi.org/project/datasette-copyable/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-copyable?include_prereleases&label=changelog)](https://github.com/simonw/datasette-copyable/releases)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-copyable/blob/master/LICENSE)

  Datasette plugin for outputting tables in formats suitable for copy and paste

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ pip install datasette-copyable

  ## Demo

  You can try this plugin on [fivethirtyeight.datasettes.com](https://fivethirtyeight.datasettes.com/) - browse for tables or queries there and look for the "copyable" link. Here's an example for a table of [airline safety data](https://fivethirtyeight.datasettes.com/fivethirtyeight/airline-safety%2Fairline-safety.copyable).

  ## Usage

  This plugin adds a `.copyable` output extension to every table, view and query.

  Navigating to this page will show an interface allowing you to select a format for copying and pasting the demo. The default is TSV, which is suitable for copying into Google Sheets or Excel.

  You can add `?_raw=1` to get back just the raw data.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-copyable
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-copyable/actions", "Changelog": "https://github.com/simonw/datasette-copyable/releases", "Homepage": "https://github.com/simonw/datasette-copyable", "Issues": "https://github.com/simonw/datasette-copyable/issues"}

requires_dist:
  ["datasette (>=0.49)", "tabulate", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "httpx ; extra == 'test'", "sqlite-utils ; extra == 'test'"]


name: datasette-cors
summary: Datasette plugin for configuring CORS headers
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-cors
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-cors/
project_url: https://pypi.org/project/datasette-cors/
project_urls: {"Homepage": "https://github.com/simonw/datasette-cors"}
release_url: https://pypi.org/project/datasette-cors/0.3/
version: 0.3
description:
  # datasette-cors

  [![PyPI](https://img.shields.io/pypi/v/datasette-cors.svg)](https://pypi.org/project/datasette-cors/)
  [![CircleCI](https://circleci.com/gh/simonw/datasette-cors.svg?style=svg)](https://circleci.com/gh/simonw/datasette-cors)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-cors/blob/master/LICENSE)

  Datasette plugin for configuring CORS headers, based on https://github.com/simonw/asgi-cors

  You can use this plugin to allow JavaScript running on a whitelisted set of domains to make `fetch()` calls to the JSON API provided by your Datasette instance.

  ## Installation

      pip install datasette-cors

  ## Configuration

  You need to add some configuration to your Datasette `metadata.json` file for this plugin to take effect.

  To whitelist specific domains, use this:

  ```json
  {
      "plugins": {
          "datasette-cors": {
              "hosts": ["https://www.example.com"]
          }
      }
  }
  ```

  You can also whitelist patterns like this:

  ```json
  {
      "plugins": {
          "datasette-cors": {
              "host_wildcards": ["https://*.example.com"]
          }
      }
  }
  ```

  ## Testing it

  To test this plugin out, run it locally by saving one of the above examples as `metadata.json` and running this:

      $ datasette --memory -m metadata.json

  Now visit https://www.example.com/ in your browser, open the browser developer console and paste in the following:

  ```javascript
  fetch("http://127.0.0.1:8001/:memory:.json?sql=select+sqlite_version%28%29").then(r => r.json()).then(console.log)
  ```

  If the plugin is running correctly, you will see the JSON response output to the console.

requires_dist:
  ["asgi-cors (~=0.3)", "datasette (~=0.29) ; extra == 'test'", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "asgiref (~=3.1.2) ; extra == 'test'"]


name: datasette-css-properties
summary: Experimental Datasette output plugin using CSS properties
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-css-properties
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-css-properties/
project_url: https://pypi.org/project/datasette-css-properties/
release_url: https://pypi.org/project/datasette-css-properties/0.2/
requires_dist: ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
requires_python: >=3.6
version: 0.2
description:
  # datasette-css-properties

  [![PyPI](https://img.shields.io/pypi/v/datasette-css-properties.svg)](https://pypi.org/project/datasette-css-properties/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-css-properties?include_prereleases&label=changelog)](https://github.com/simonw/datasette-css-properties/releases)
  [![Tests](https://github.com/simonw/datasette-css-properties/workflows/Test/badge.svg)](https://github.com/simonw/datasette-css-properties/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-css-properties/blob/main/LICENSE)

  Extremely experimental Datasette output plugin using CSS properties, inspired by [Custom Properties as State](https://css-tricks.com/custom-properties-as-state/) by Chris Coyier.

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-css-properties

  ## Usage

  Once installed, this plugin adds a `.css` output format to every query result. This will return the first row in the query as a valid CSS file, defining each column as a custom property:

  Example: https://latest-with-plugins.datasette.io/fixtures/roadside_attractions.css produces:

  ```css
  :root {
    --pk: '1';
    --name: 'The Mystery Spot';
    --address: '465 Mystery Spot Road, Santa Cruz, CA 95065';
    --latitude: '37.0167';
    --longitude: '-122.0024';
  }
  ```

  If you link this stylesheet to your page you can then do things like this;

  ```html
  <link rel="stylesheet" href="https://latest-with-plugins.datasette.io/fixtures/roadside_attractions.css">
  <style>
  .attraction-name:after { content: var(--name); }
  </style>
  <p class="attraction-name">Attraction name: </p>
  ```

  Values will be quoted as CSS strings by default. If you want to return a "raw" value without the quotes - for example to set a CSS property that is numeric or a color, you can specify that column name using the `?_raw=column-name` parameter. This can be passed multiple times.

  Consider [this example query](https://latest-with-plugins.datasette.io/github?sql=select%0D%0A++%27%23%27+||+substr(sha%2C+0%2C+6)+as+[custom-bg]%0D%0Afrom%0D%0A++commits%0D%0Aorder+by%0D%0A++author_date+desc%0D%0Alimit%0D%0A++1%3B):

  ```sql
  select
    '#' || substr(sha, 0, 6) as [custom-bg]
  from
    commits
  order by
    author_date desc
  limit
    1;
  ```

  This returns the first 6 characters of the most recently authored commit with a `#` prefix. The `.css` [output rendered version](https://latest-with-plugins.datasette.io/github.css?sql=select%0D%0A++%27%23%27+||+substr(sha%2C+0%2C+6)+as+[custom-bg]%0D%0Afrom%0D%0A++commits%0D%0Aorder+by%0D%0A++author_date+desc%0D%0Alimit%0D%0A++1%3B) looks like this:

  ```css
  :root {
    --custom-bg: '#97fb1';
  }
  ```

  Adding `?_raw=custom-bg` to the URL produces [this instead](https://latest-with-plugins.datasette.io/github.css?sql=select%0D%0A++%27%23%27+||+substr(sha%2C+0%2C+6)+as+[custom-bg]%0D%0Afrom%0D%0A++commits%0D%0Aorder+by%0D%0A++author_date+desc%0D%0Alimit%0D%0A++1%3B&_raw=custom-bg):

  ```css
  :root {
    --custom-bg: #97fb1;
  }
  ```

  This can then be used as a color value like so:

  ```css
  h1 {
      background-color: var(--custom-bg);
  }
  ```

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-css-properties
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-css-properties/actions", "Changelog": "https://github.com/simonw/datasette-css-properties/releases", "Homepage": "https://github.com/simonw/datasette-css-properties", "Issues": "https://github.com/simonw/datasette-css-properties/issues"}


name: datasette-dashboards
summary: Datasette plugin providing data dashboards from metadata
classifiers: []
author: Romain Clement
description_content_type: text/markdown
home_page: https://github.com/rclement/datasette-dashboards
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-dashboards/
project_url: https://pypi.org/project/datasette-dashboards/
release_url: https://pypi.org/project/datasette-dashboards/0.2.0/
requires_python: >=3.7
version: 0.2.0
description:
  # datasette-dashboards

  > Datasette plugin providing data dashboards from metadata

  [![PyPI](https://img.shields.io/pypi/v/datasette-dashboards.svg)](https://pypi.org/project/datasette-dashboards/)
  [![CI/CD](https://github.com/rclement/datasette-dashboards/actions/workflows/ci-cd.yml/badge.svg)](https://github.com/rclement/datasette-dashboards/actions/workflows/ci-cd.yml)
  [![Coverage Status](https://img.shields.io/codecov/c/github/rclement/datasette-dashboards)](https://codecov.io/gh/rclement/datasette-dashboards)
  [![License](https://img.shields.io/github/license/rclement/datasette-dashboards)](https://github.com/rclement/datasette-dashboards/blob/master/LICENSE)

  Try out a live demo at [https://datasette-dashboards-demo.vercel.app](https://datasette-dashboards-demo.vercel.app/-/dashboards)

  **WARNING**: this plugin is still experimental and not ready for production.
  Some breaking changes might happen between releases before reaching a stable version.
  Use it at your own risks!

  ![Datasette Dashboards Demo](https://raw.githubusercontent.com/rclement/datasette-dashboards/master/demo/datasette-dashboards-demo.png)

  ## Installation

  Install this plugin in the same environment as Datasette:

  ```bash
  $ datasette install datasette-dashboards
  ```

  ## Usage

  Define dashboards within `metadata.yml` / `metadata.json`:

  ```yaml
  plugins:
    datasette-dashboards:
      my-dashboard:
        title: My Dashboard
        description: Showing some nice metrics
        layout:
          - [analysis-note, events-count]
          - [analysis-note, events-source]
        filters:
          date_start:
            name: Date Start
            type: date
            default: "2021-01-01"
          date_end:
            name: Date End
            type: date
        charts:
          analysis-note:
            library: markdown
            display: |-
              # Analysis notes
              > A quick rundown of events statistics and KPIs

          events-count:
            title: Total number of events
            db: jobs
            query: SELECT count(*) as count FROM events
            library: metric
            display:
              field: count
              prefix:
              suffix:

          events-source:
            title: Number of events by source
            db: jobs
            query: SELECT source, count(*) as count FROM events WHERE TRUE [[ AND date >= date(:date_start) ]] [[ AND date <= date(:date_end) ]] GROUP BY source ORDER BY count DESC
            library: vega
            display:
              mark: { type: bar, tooltip: true }
              encoding:
                color: { field: source, type: nominal }
                theta: { field: count, type: quantitative }
  ```

  A new menu entry is now available, pointing at `/-/dashboards` to access all defined dashboards.

  ### Properties

  Dashboard properties:

  | Property      | Type     | Description           |
  | ------------- | -------- | --------------------- |
  | `title`       | `string` | Dashboard title       |
  | `description` | `string` | Dashboard description |
  | `layout`      | `array`  | Dashboard layout      |
  | `filters`     | `object` | Dashboard filters     |

  Dashboard filters:

  | Property  | Type               | Description                            |
  | --------- | ------------------ | -------------------------------------- |
  | `name`    | `string`           | Filter display name                    |
  | `type`    | `string`           | Filter type (`text`, `date`, `number`) |
  | `default` | `string`, `number` | (optional) Filter default value        |
  | `min`     | `number`           | (optional) Filter minimum value        |
  | `max`     | `number`           | (optional) Filter maximum value        |
  | `step`    | `number`           | (optional) Filter stepping value       |

  Common chart properties for all chart types:

  | Property  | Type     | Description                                              |
  | --------- | -------- | -------------------------------------------------------- |
  | `title`   | `string` | Chart title                                              |
  | `db`      | `string` | Database name against which to run the query             |
  | `query`   | `string` | SQL query to run and extract data from                   |
  | `library` | `string` | One of supported libraries: `vega`, `markdown`, `metric` |
  | `display` | `object` | Chart display specification (depend on the used library) |

  To define SQL queries using dashboard filters:

  ```sql
  SELECT * FROM mytable [[ WHERE col >= :my_filter ]]
  ```

  ```sql
  SELECT * FROM mytable WHERE TRUE [[ AND col1 = :my_filter_1 ]] [[ AND col2 = :my_filter_2 ]]
  ```

  #### Vega properties

  Available configuration for `vega` charts:

  | Property  | Type     | Description               |
  | --------- | -------- | ------------------------- |
  | `library` | `string` | Must be set to `vega`     |
  | `display` | `object` | Vega specification object |

  Notes about the `display` property:

  - Requires a valid [Vega specification object](https://vega.github.io/vega-lite/docs/)
  - Some fields are pre-defined: `$schema`, `title`, `width`, `view`, `config`, `data`
  - All fields are passed along as-is (overriding pre-defined fields if any)
  - Only `mark` and `encoding` fields are required as the bare-minimum

  #### Markdown properties

  Available configuration for `markdown` chart:

  | Property  | Type     | Description                                       |
  | --------- | -------- | ------------------------------------------------- |
  | `library` | `string` | Must be set to `markdown`                         |
  | `display` | `string` | Multi-line string containing the Markdown content |

  Note :

  - Some common properties do not apply and can be omitted: `title`, `db`, `query`
  - Markdown rendering is done by [`datasette-render-markdown`](https://datasette.io/plugins/datasette-render-markdown)
  - To configure Markdown rendering, extensions can be enabled in [metadata](https://datasette.io/plugins/datasette-render-markdown#user-content-markdown-extensions)

  #### Metric properties

  Available configuration for `metric` chart:

  | Property         | Type     | Description                               |
  | ---------------- | -------- | ----------------------------------------- |
  | `library`        | `string` | Must be set to `metric`                   |
  | `display.field`  | `string` | Numerical field to be displayed as metric |
  | `display.prefix` | `string` | Prefix to be displayed before metric      |
  | `display.suffix` | `string` | Prefix to be displayed after metric       |

  Note:

  - The `display.field` must reference a single-numerical value from the SQL query
    (e.g. numerical `number` field in `SELECT count(*) as number FROM events`)

  ### Dashboard layout

  The default dashboard layout will present two charts per row (one per row on mobile).
  To make use of custom dashboard layout using [CSS Grid Layout](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Grid_Layout),
  define the `layout` array property as a grid / matrix:

  - Each entry represents a row of charts
  - Each column is referring a chart by its property name

  ## Development

  To set up this plugin locally, first checkout the code.
  Then create a new virtual environment and the required dependencies:

  ```bash
  pipenv install -d
  pipenv shell
  ```

  To run the tests:

  ```bash
  pytest
  ```

  ## Demo

  With the developmnent environment setup, you can run the demo locally:

  ```bash
  datasette --metadata demo/metadata.yml demo/jobs.db
  ```

  ## License

  Licensed under Apache License, Version 2.0

  Copyright (c) 2021 - present Romain Clement

project_urls:
  {"Changelog": "https://github.com/rclement/datasette-dashboards/blob/master/CHANGELOG.md", "Homepage": "https://github.com/rclement/datasette-dashboards"}

requires_dist:
  ["datasette", "datasette-render-markdown", "faker ; extra == 'test'", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "sqlite-utils ; extra == 'test'"]


name: datasette-dateutil
summary: dateutil functions for Datasette
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-dateutil
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-dateutil/
project_url: https://pypi.org/project/datasette-dateutil/
release_url: https://pypi.org/project/datasette-dateutil/0.3/
version: 0.3
description:
  # datasette-dateutil

  [![PyPI](https://img.shields.io/pypi/v/datasette-dateutil.svg)](https://pypi.org/project/datasette-dateutil/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-dateutil?include_prereleases&label=changelog)](https://github.com/simonw/datasette-dateutil/releases)
  [![Tests](https://github.com/simonw/datasette-dateutil/workflows/Test/badge.svg)](https://github.com/simonw/datasette-dateutil/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-dateutil/blob/main/LICENSE)

  dateutil functions for Datasette

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-dateutil

  ## Usage

  This function adds custom SQL functions that expose functionality from the [dateutil](https://dateutil.readthedocs.io/) Python library.

  Once installed, the following SQL functions become available:

  ### Parsing date strings

  - `dateutil_parse(text)` - returns an ISO8601 date string parsed from the text, or `null` if the input could not be parsed. `dateutil_parse("10 october 2020 3pm")` returns `2020-10-10T15:00:00`.
  - `dateutil_parse_fuzzy(text)` - same as `dateutil_parse()` but this also works against strings that contain a date somewhere within them - that date will be returned, or `null` if no dates could be found. `dateutil_parse_fuzzy("This is due 10 september")` returns `2020-09-10T00:00:00` (but will start returning the 2021 version of that if the year is 2021).

  The `dateutil_parse()` and `dateutil_parse_fuzzy()` functions both follow the American convention of assuming that `1/2/2020` lists the month first, evaluating this example to the 2nd of January.

  If you want to assume that the day comes first, use these two functions instead:

  - `dateutil_parse_dayfirst(text)`
  - `dateutil_parse_fuzzy_dayfirst(text)`

  Here's a query demonstrating these functions:

  ```sql
  select
    dateutil_parse("10 october 2020 3pm"),
    dateutil_parse_fuzzy("This is due 10 september"),
    dateutil_parse("1/2/2020"),
    dateutil_parse("2020-03-04"),
    dateutil_parse_dayfirst("2020-03-04");
  ```

  [Try that query](https://latest-with-plugins.datasette.io/fixtures?sql=select%0D%0A++dateutil_parse%28%2210+october+2020+3pm%22%29%2C%0D%0A++dateutil_parse_fuzzy%28%22This+is+due+10+september%22%29%2C%0D%0A++dateutil_parse%28%221%2F2%2F2020%22%29%2C%0D%0A++dateutil_parse%28%222020-03-04%22%29%2C%0D%0A++dateutil_parse_dayfirst%28%222020-03-04%22%29%3B)

  ### Optional default dates

  The `dateutil_parse()`, `dateutil_parse_fuzzy()`, `dateutil_parse_dayfirst()` and `dateutil_parse_fuzzy_dayfirst()` functions all accept an optional second argument specifying a "default" datetime to consider if some of the details are missing. For example, the following:
  ```sql
  select dateutil_parse('1st october', '1985-01-01')
  ```
  Will return `1985-10-01T00:00:00` - the missing year is replaced with the year from the default date.

  [Example query demonstrating the default date argument](https://latest-with-plugins.datasette.io/fixtures?sql=with+times+as+%28%0D%0A++select%0D%0A++++datetime%28%27now%27%29+as+t%0D%0A++union%0D%0A++select%0D%0A++++datetime%28%27now%27%2C+%27-1+year%27%29%0D%0A++union%0D%0A++select%0D%0A++++datetime%28%27now%27%2C+%27-3+years%27%29%0D%0A%29%0D%0Aselect+t%2C+dateutil_parse_fuzzy%28%22This+is+due+10+september%22%2C+t%29+from+times)

  ### Calculating Easter

  - `dateutil_easter(year)` - returns the date for Easter in that year, for example `dateutil_easter("2020")` returns `2020-04-12`.

  [Example Easter query](https://latest-with-plugins.datasette.io/fixtures?sql=select%0D%0A++dateutil_easter%282019%29%2C%0D%0A++dateutil_easter%282020%29%2C%0D%0A++dateutil_easter%282021%29)

  ### JSON arrays of dates

  Several functions return JSON arrays of date strings. These can be used with SQLite's `json_each()` function to perform joins against dates from a specific date range or recurrence rule.

  These functions can return up to 10,000 results. They will return an error if more than 10,000 dates would be returned - this is to protect against denial of service attacks.

  - `dateutil_dates_between('1 january 2020', '5 jan 2020')` - given two dates (in any format that can be handled by `dateutil_parse()`) this function returns a JSON string containing the dates between those two days, inclusive. This example returns `["2020-01-01", "2020-01-02", "2020-01-03", "2020-01-04", "2020-01-05"]`.
  - `dateutil_dates_between('1 january 2020', '5 jan 2020', 0)` - set the optional third argument to `0` to specify that you would like this to be exclusive of the last day. This example returns `["2020-01-01", "2020-01-02", "2020-01-03", "2020-01-04"]`.

  [Try these queries](https://latest-with-plugins.datasette.io/fixtures?sql=select%0D%0A++dateutil_dates_between%28%271+january+2020%27%2C+%275+jan+2020%27%29%2C%0D%0A++dateutil_dates_between%28%271+january+2020%27%2C+%275+jan+2020%27%2C+0%29)

  The `dateutil_rrule()` and `dateutil_rrule_date()` functions accept the iCalendar standard ``rrule` format - see [the dateutil documentation](https://dateutil.readthedocs.io/en/stable/rrule.html#rrulestr-examples) for more examples.

  This format lets you specify recurrence rules such as "the next four last mondays of the month".

  - `dateutil_rrule(rrule, optional_dtsart)` - given an rrule returns a JSON array of ISO datetimes. The second argument is optional and will be treated as the start date for the rule.
  - `dateutil_rrule_date(rrule, optional_dtsart)` - same as `dateutil_rrule()` but returns ISO dates.

  Example query:

  ```sql
  select
    dateutil_rrule('FREQ=HOURLY;COUNT=5'),
    dateutil_rrule_date(
      'FREQ=DAILY;COUNT=3',
      '1st jan 2020'
    );
  ```
  [Try the rrule example query](https://latest-with-plugins.datasette.io/fixtures?sql=select%0D%0A++dateutil_rrule('FREQ%3DHOURLY%3BCOUNT%3D5')%2C%0D%0A++dateutil_rrule_date(%0D%0A++++'FREQ%3DDAILY%3BCOUNT%3D3'%2C%0D%0A++++'1st+jan+2020'%0D%0A++)%3B)

  ### Joining data using json_each()

  SQLite's [json_each() function](https://www.sqlite.org/json1.html#jeach) can be used to turn a JSON array of dates into a table that can be joined against other data. Here's a query that returns a table showing every day in January 2019:

  ```sql
  select
    value as date
  from
    json_each(
      dateutil_dates_between('1 Jan 2019', '31 Jan 2019')
    )
  ```
  [Try that query](https://latest-with-plugins.datasette.io/fixtures?sql=select%0D%0A++value+as+date%0D%0Afrom%0D%0A++json_each%28%0D%0A++++dateutil_dates_between%28%271+Jan+2019%27%2C+%2731+Jan+2019%27%29%0D%0A++%29)

  You can run joins against this table by assigning it a name using SQLite's [support for Common Table Expressions (CTEs)](https://sqlite.org/lang_with.html).

  This example query uses `substr(created, 0, 11)` to retrieve the date portion of the `created` column in the [facetable demo table](https://latest-with-plugins.datasette.io/fixtures/facetable), then joins that against the table of days in January to calculate the count of rows created on each day. The `LEFT JOIN` against `days_in_january` ensures that days which had no created records are still returned in the results, with a count of 0.

  ```sql
  with created_dates as (
    select
      substr(created, 0, 11) as date
    from
      facetable
  ),
  days_in_january as (
    select
      value as date
    from
      json_each(
        dateutil_dates_between('1 Jan 2019', '31 Jan 2019')
      )
  )
  select
    days_in_january.date,
    count(created_dates.date) as total
  from
    days_in_january
    left join created_dates on days_in_january.date = created_dates.date
  group by
    days_in_january.date;
  ```
  [Try that query](https://latest-with-plugins.datasette.io/fixtures?sql=with+created_dates+as+%28%0D%0A++select%0D%0A++++substr%28created%2C+0%2C+11%29+as+date%0D%0A++from%0D%0A++++facetable%0D%0A%29%2C%0D%0Adays_in_january+as+%28%0D%0A++select%0D%0A++++value+as+date%0D%0A++from%0D%0A++++json_each%28%0D%0A++++++dateutil_dates_between%28%271+Jan+2019%27%2C+%2731+Jan+2019%27%29%0D%0A++++%29%0D%0A%29%0D%0Aselect%0D%0A++days_in_january.date%2C%0D%0A++count%28created_dates.date%29+as+total%0D%0Afrom%0D%0A++days_in_january%0D%0A++left+join+created_dates+on+days_in_january.date+%3D+created_dates.date%0D%0Agroup+by%0D%0A++days_in_january.date%3B#g.mark=bar&g.x_column=date&g.x_type=ordinal&g.y_column=total&g.y_type=quantitative) with a bar chart rendered using the [datasette-vega](https://github.com/simonw/datasette-vega) plugin.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-dateutil
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-dateutil/actions", "Changelog": "https://github.com/simonw/datasette-dateutil/releases", "Homepage": "https://github.com/simonw/datasette-dateutil", "Issues": "https://github.com/simonw/datasette-dateutil/issues"}

requires_dist:
  ["datasette", "python-dateutil", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "httpx ; extra == 'test'"]


name: datasette-debug-asgi
summary: Datasette plugin for dumping out the ASGI scope
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-debug-asgi
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-debug-asgi/
project_url: https://pypi.org/project/datasette-debug-asgi/
release_url: https://pypi.org/project/datasette-debug-asgi/1.1/
requires_dist: ["datasette (>=0.50)", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
requires_python: >=3.6
version: 1.1
description:
  # datasette-debug-asgi

  [![PyPI](https://img.shields.io/pypi/v/datasette-debug-asgi.svg)](https://pypi.org/project/datasette-debug-asgi/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-debug-asgi?include_prereleases&label=changelog)](https://github.com/simonw/datasette-debug-asgi/releases)
  [![Tests](https://github.com/simonw/datasette-debug-asgi/workflows/Test/badge.svg)](https://github.com/simonw/datasette-debug-asgi/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-debug-asgi/blob/main/LICENSE)

  Datasette plugin for dumping out the ASGI scope.

  Adds a new URL at `/-/asgi-scope` which shows the current ASGI scope. Demo here: https://datasette.io/-/asgi-scope

  ## Installation

      pip install datasette-debug-asgi

  ## Usage

  Visit `/-/asgi-scope` to see debug output showing the ASGI scope.

  You can add query string parameters such as `/-/asgi-scope?q=hello`.

  You can also add extra path components such as `/-/asgi-scope/more/path/here`.

project_urls:
  {"CI": "https://github.com/simonw/datasette-debug-asgi/actions", "Changelog": "https://github.com/simonw/datasette-debug-asgi/releases", "Homepage": "https://github.com/simonw/datasette-debug-asgi", "Issues": "https://github.com/simonw/datasette-debug-asgi/issues"}


name: datasette-edit-schema
summary: Datasette plugin for modifying table schemas
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-edit-schema
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-edit-schema/
project_url: https://pypi.org/project/datasette-edit-schema/
project_urls: {"Homepage": "https://github.com/simonw/datasette-edit-schema"}
release_url: https://pypi.org/project/datasette-edit-schema/0.5.1/
requires_python: >=3.7
version: 0.5.1
description:
  # datasette-edit-schema

  [![PyPI](https://img.shields.io/pypi/v/datasette-edit-schema.svg)](https://pypi.org/project/datasette-edit-schema/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-edit-schema?include_prereleases&label=changelog)](https://github.com/simonw/datasette-edit-schema/releases)
  [![Tests](https://github.com/simonw/datasette-edit-schema/workflows/Test/badge.svg)](https://github.com/simonw/datasette-edit-schema/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-edit-schema/blob/master/LICENSE)

  Datasette plugin for modifying table schemas

  ## Features

  * Add new columns to a table
  * Rename columns in a table
  * Modify the type of columns in a table
  * Re-order the columns in a table
  * Rename a table
  * Delete a table

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ pip install datasette-edit-schema

  ## Usage

  Navigate to `/-/edit-schema/dbname/tablename` on your Datasette instance to edit a specific table.

  Use `/-/edit-schema/dbname` to create a new table in a specific database.

  By default only [the root actor](https://datasette.readthedocs.io/en/stable/authentication.html#using-the-root-actor) can access the page - so you'll need to run Datasette with the `--root` option and click on the link shown in the terminal to sign in and access the page.

  ## Permissions

  The `edit-schema` permission governs access. You can use permission plugins such as [datasette-permissions-sql](https://github.com/simonw/datasette-permissions-sql) to grant additional access to the write interface.

  These permission checks will call the `permission_allowed()` plugin hook with three arguments:

  - `action` will be the string `"edit-schema"`
  - `actor` will be the currently authenticated actor - usually a dictionary
  - `resource` will be the string name of the database

  ## Screenshot

  ![datasette-edit-schema interface](https://raw.githubusercontent.com/simonw/datasette-edit-schema/main/datasette-edit-schema.png)

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-edit-schema
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

requires_dist:
  ["datasette (>=0.59)", "sqlite-utils (>=3.10)", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]


name: datasette-edit-templates
summary: Plugin allowing Datasette templates to be edited within Datasette
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-edit-templates
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-edit-templates/
project_url: https://pypi.org/project/datasette-edit-templates/
release_url: https://pypi.org/project/datasette-edit-templates/0.1/
requires_python: >=3.7
version: 0.1
description:
  # datasette-edit-templates

  [![PyPI](https://img.shields.io/pypi/v/datasette-edit-templates.svg)](https://pypi.org/project/datasette-edit-templates/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-edit-templates?include_prereleases&label=changelog)](https://github.com/simonw/datasette-edit-templates/releases)
  [![Tests](https://github.com/simonw/datasette-edit-templates/workflows/Test/badge.svg)](https://github.com/simonw/datasette-edit-templates/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-edit-templates/blob/main/LICENSE)

  Plugin allowing Datasette templates to be edited within Datasette.

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-edit-templates

  ## Usage

  Once installed, sign in as the root user using `datasette mydb.db --root`.

  On startup. a `_templates_` table will be created in the database you are running Datasette against.

  Use the app menu to navigate to the `/-/edit-templates` page, and edit templates there.

  Changes should become visible instantly, and will be persisted to your database.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-edit-templates
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-edit-templates/actions", "Changelog": "https://github.com/simonw/datasette-edit-templates/releases", "Homepage": "https://github.com/simonw/datasette-edit-templates", "Issues": "https://github.com/simonw/datasette-edit-templates/issues"}

requires_dist:
  ["datasette (>=0.63)", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "sqlite-utils ; extra == 'test'"]


name: datasette-export-notebook
summary: Datasette plugin providing instructions for exporting data to Jupyter or Observable
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-export-notebook
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-export-notebook/
project_url: https://pypi.org/project/datasette-export-notebook/
release_url: https://pypi.org/project/datasette-export-notebook/1.0/
requires_python: >=3.6
version: 1.0
description:
  # datasette-export-notebook

  [![PyPI](https://img.shields.io/pypi/v/datasette-export-notebook.svg)](https://pypi.org/project/datasette-export-notebook/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-export-notebook?include_prereleases&label=changelog)](https://github.com/simonw/datasette-export-notebook/releases)
  [![Tests](https://github.com/simonw/datasette-export-notebook/workflows/Test/badge.svg)](https://github.com/simonw/datasette-export-notebook/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-export-notebook/blob/main/LICENSE)

  Datasette plugin providing instructions for exporting data to a [Jupyter](https://jupyter.org/) or [Observable](https://observablehq.com/) notebook.

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-export-notebook

  ## Usage

  Once installed, the plugin will add a `.Notebook` export option to every table and query. Clicking on this link will show instructions for exporting the data to Jupyter or Observable.

  ## Demo

  You can see this plugin in action on the [latest-with-plugins.datasette.io](https://latest-with-plugins.datasette.io/) Datasette instance - for example on [/github/commits.Notebook](https://latest-with-plugins.datasette.io/github/commits.Notebook).

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-export-notebook
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-export-notebook/actions", "Changelog": "https://github.com/simonw/datasette-export-notebook/releases", "Homepage": "https://github.com/simonw/datasette-export-notebook", "Issues": "https://github.com/simonw/datasette-export-notebook/issues"}

requires_dist:
  ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "sqlite-utils ; extra == 'test'"]


name: datasette-expose-env
summary: Datasette plugin to expose selected environment variables at /-/env for debugging
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-expose-env
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-expose-env/
project_url: https://pypi.org/project/datasette-expose-env/
release_url: https://pypi.org/project/datasette-expose-env/0.1/
requires_dist: ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
requires_python: >=3.7
version: 0.1
description:
  # datasette-expose-env

  [![PyPI](https://img.shields.io/pypi/v/datasette-expose-env.svg)](https://pypi.org/project/datasette-expose-env/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-expose-env?include_prereleases&label=changelog)](https://github.com/simonw/datasette-expose-env/releases)
  [![Tests](https://github.com/simonw/datasette-expose-env/workflows/Test/badge.svg)](https://github.com/simonw/datasette-expose-env/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-expose-env/blob/main/LICENSE)

  Datasette plugin to expose selected environment variables at `/-/env` for debugging

  ## Installation

  Install this plugin in the same environment as Datasette.

      datasette install datasette-expose-env

  ## Configuration

  Decide on a list of environment variables you would like to expose, then add the following to your `metadata.yml` configuration:

  ```yaml
  plugins:
      datasette-expose-env:
      - ENV_VAR_1
      - ENV_VAR_2
      - ENV_VAR_3
  ```

  If you are using JSON in a `metadata.json` file use the following:

  ```json
  {
      "plugins": {
          "datasette-expose-env": [
              "ENV_VAR_1",
              "ENV_VAR_2",
              "ENV_VAR_3"
          ]
      }
  }
  ```

  Visit `/-/env` on your Datasette instance to see the values of the environment variables.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-expose-env
      python3 -m venv venv
      source venv/bin/activate

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-expose-env/actions", "Changelog": "https://github.com/simonw/datasette-expose-env/releases", "Homepage": "https://github.com/simonw/datasette-expose-env", "Issues": "https://github.com/simonw/datasette-expose-env/issues"}


name: datasette-external-links-new-tabs
summary: Datasette plugin to open external links in new tabs
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Jacob Weisz
description_content_type: text/markdown
home_page: https://github.com/ocdtrekkie/datasette-external-links-new-tabs
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-external-links-new-tabs/
project_url: https://pypi.org/project/datasette-external-links-new-tabs/
release_url: https://pypi.org/project/datasette-external-links-new-tabs/0.1/
requires_dist: ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
requires_python: >=3.7
version: 0.1
description:
  # datasette-external-links-new-tabs

  [![PyPI](https://img.shields.io/pypi/v/datasette-external-links-new-tabs.svg)](https://pypi.org/project/datasette-external-links-new-tabs/)
  [![Changelog](https://img.shields.io/github/v/release/ocdtrekkie/datasette-external-links-new-tabs?include_prereleases&label=changelog)](https://github.com/ocdtrekkie/datasette-external-links-new-tabs/releases)
  [![Tests](https://github.com/ocdtrekkie/datasette-external-links-new-tabs/workflows/Test/badge.svg)](https://github.com/ocdtrekkie/datasette-external-links-new-tabs/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/ocdtrekkie/datasette-external-links-new-tabs/blob/main/LICENSE)

  Datasette plugin to open external links in new tabs

  ## Installation

  Install this plugin in the same environment as Datasette.

      datasette install datasette-external-links-new-tabs

  ## Usage

  There are no usage instructions, it simply opens external links in a new tab.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-external-links-new-tabs
      python3 -m venv venv
      source venv/bin/activate

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/ocdtrekkie/datasette-external-links-new-tabs/actions", "Changelog": "https://github.com/ocdtrekkie/datasette-external-links-new-tabs/releases", "Homepage": "https://github.com/ocdtrekkie/datasette-external-links-new-tabs", "Issues": "https://github.com/ocdtrekkie/datasette-external-links-new-tabs/issues"}


name: datasette-geojson-map
summary: Render a map for any query with a geometry column
classifiers: []
author: Chris Amico
description_content_type: text/markdown
home_page: https://github.com/eyeseast/datasette-geojson-map
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-geojson-map/
project_url: https://pypi.org/project/datasette-geojson-map/
release_url: https://pypi.org/project/datasette-geojson-map/0.4.0/
requires_python: >=3.6
version: 0.4.0
description:
  # datasette-geojson-map

  [![PyPI](https://img.shields.io/pypi/v/datasette-geojson-map.svg)](https://pypi.org/project/datasette-geojson-map/)
  [![Changelog](https://img.shields.io/github/v/release/eyeseast/datasette-geojson-map?include_prereleases&label=changelog)](https://github.com/eyeseast/datasette-geojson-map/releases)
  [![Tests](https://github.com/eyeseast/datasette-geojson-map/workflows/Test/badge.svg)](https://github.com/eyeseast/datasette-geojson-map/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/eyeseast/datasette-geojson-map/blob/main/LICENSE)

  Render a map for any query with a geometry column

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-geojson-map

  ## Usage

  Start by loading a GIS file.

  For example, you might use [geojson-to-sqlite](https://pypi.org/project/geojson-to-sqlite/) or [shapefile-to-sqlite](https://pypi.org/project/shapefile-to-sqlite/) to load [neighborhood boundaries](https://bostonopendata-boston.opendata.arcgis.com/datasets/3525b0ee6e6b427f9aab5d0a1d0a1a28_0/explore) into a SQLite database.

  ```sh
  wget -O neighborhoods.geojson https://opendata.arcgis.com/datasets/3525b0ee6e6b427f9aab5d0a1d0a1a28_0.geojson
  geojson-to-sqlite boston.db neighborhoods neighborhoods.geojson
  ```

  (The command above uses Spatialite, but that's not required.)

  Start up `datasette` and navigate to the `neighborhoods` table.

  ```sh
  datasette serve boston.db

  # in another terminal tab
  open http://localhost:8001/boston/neighborhoods
  ```

  You should see a map centered on Boston with each neighborhood outlined. Clicking a boundary will bring up a popup with details on that feature.

  ![Boston neighbhorhoods map](img/boston-neighborhoods-map.png)

  This plugin relies on (and will install) [datasette-geojson](https://github.com/eyeseast/datasette-geojson). Any query that includes a `geometry` column will produce a map of the results. This also includes single row views.

  Run the included `demo` project to see it live.

  ## Configuration

  This project uses the same map configuration as [datasette-cluster-map](https://github.com/simonw/datasette-cluster-map). Here's how you would use [Stamen's terrain tiles](http://maps.stamen.com/terrain/#12/37.7706/-122.3782):

  ```yaml
  plugins:
    datasette-geojson-map:
      tile_layer: https://stamen-tiles-{s}.a.ssl.fastly.net/terrain/{z}/{x}/{y}.{ext}
      tile_layer_options:
        attribution: >-
          Map tiles by <a href="http://stamen.com">Stamen Design</a>, 
          under <a href="http://creativecommons.org/licenses/by/3.0">CC BY 3.0</a>. 
          Data by <a href="http://openstreetmap.org">OpenStreetMap</a>, 
          under <a href="http://www.openstreetmap.org/copyright">ODbL</a>.
        subdomains: abcd
        minZoom: 1
        maxZoom: 16
        ext: jpg
  ```

  Options:

  - `tile_layer`: Use a URL template that can be passed to a [Leaflet Tilelayer](https://leafletjs.com/reference-1.7.1.html#tilelayer)
  - `tile_layer_options`: All options will be passed to the tile layer. See [Leaflet documentation](https://leafletjs.com/reference-1.7.1.html#tilelayer) for more on possible values here.

  ## Styling map features

  Map features can be styled using the [simplestyle-spec](https://github.com/mapbox/simplestyle-spec). This requires setting specific fields on returned rows. Here's an example:

  ```sql
  SELECT Name, geometry, "#ff0000" as fill, "#0000ff" as stroke, 0.2 as stroke-width,  from neighborhoods
  ```

  That will render a neighborhood map where each polygon is filled in red, outlined in blue and lines are 0.2 pixels wide.

  A more useful approach would use the `CASE` statement to color features based on data:

  ```sql
  SELECT
    Name,
    geometry,
    CASE
      Name
      WHEN "Roslindale" THEN "#ff0000"
      WHEN "Dorchester" THEN "#0000ff"
      ELSE "#dddddd"
    END fill
  FROM
    neighborhoods
  ```

  This will fill Roslindale in red, Dorchester in blue and all other neighborhoods in gray.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-geojson-map
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/eyeseast/datasette-geojson-map/actions", "Changelog": "https://github.com/eyeseast/datasette-geojson-map/releases", "Homepage": "https://github.com/eyeseast/datasette-geojson-map", "Issues": "https://github.com/eyeseast/datasette-geojson-map/issues"}

requires_dist:
  ["datasette", "datasette-geojson", "datasette-leaflet", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "geojson-to-sqlite ; extra == 'test'"]


name: datasette-geojson
summary: Add GeoJSON as an output option
classifiers: []
author: Chris Amico
description_content_type: text/markdown
home_page: https://github.com/eyeseast/datasette-geojson
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-geojson/
project_url: https://pypi.org/project/datasette-geojson/
release_url: https://pypi.org/project/datasette-geojson/0.3.1/
requires_python: >=3.6
version: 0.3.1
description:
  # datasette-geojson

  [![PyPI](https://img.shields.io/pypi/v/datasette-geojson.svg)](https://pypi.org/project/datasette-geojson/)
  [![Changelog](https://img.shields.io/github/v/release/eyeseast/datasette-geojson?include_prereleases&label=changelog)](https://github.com/eyeseast/datasette-geojson/releases)
  [![Tests](https://github.com/eyeseast/datasette-geojson/workflows/Test/badge.svg)](https://github.com/eyeseast/datasette-geojson/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/eyeseast/datasette-geojson/blob/main/LICENSE)

  Add GeoJSON as an output option for datasette queries.

  ## Installation

  Install this plugin in the same environment as Datasette.

      datasette install datasette-geojson

  ## Usage

  To render GeoJSON, add a `.geojson` extension to any query URL that includes a `geometry` column. That column should be a valid [GeoJSON geometry](https://datatracker.ietf.org/doc/html/rfc7946#section-3.1).

  For example, you might use [geojson-to-sqlite](https://pypi.org/project/geojson-to-sqlite/) or [shapefile-to-sqlite](https://pypi.org/project/shapefile-to-sqlite/) to load [neighborhood boundaries](https://bostonopendata-boston.opendata.arcgis.com/datasets/3525b0ee6e6b427f9aab5d0a1d0a1a28_0/explore) into a SQLite database.

  ```sh
  wget -O neighborhoods.geojson https://opendata.arcgis.com/datasets/3525b0ee6e6b427f9aab5d0a1d0a1a28_0.geojson
  geojson-to-sqlite boston.db neighborhoods neighborhoods.geojson --spatial-index # create a spatial index
  datasette serve boston.db --load-extension spatialite
  ```

  If you're using Spatialite, the geometry column will be in a binary format. If not, make sure the `geometry` column is a well-formed [GeoJSON geometry](https://datatracker.ietf.org/doc/html/rfc7946#section-3.1). If you used `geojson-to-sqlite` or `shapefile-to-sqlite`, you should be all set.

  Run this query in Datasette and you'll see a link to download GeoJSON:

  ```sql
  select
    rowid,
    OBJECTID,
    Name,
    Acres,
    Neighborhood_ID,
    SqMiles,
    ShapeSTArea,
    ShapeSTLength,
    geometry
  from
    neighborhoods
  order by
    rowid
  limit
    101
  ```

  Note that the geometry column needs to be explicitly _named_ `geometry` or you won't get the option to export GeoJSON. If you want to use a different column, rename it with `AS`: `SELECT other AS geometry FROM my_table`.

  ![export geojson](img/export-options.png)

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-geojson
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/eyeseast/datasette-geojson/actions", "Changelog": "https://github.com/eyeseast/datasette-geojson/releases", "Homepage": "https://github.com/eyeseast/datasette-geojson", "Issues": "https://github.com/eyeseast/datasette-geojson/issues"}

requires_dist:
  ["datasette", "geojson", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "geojson-to-sqlite ; extra == 'test'"]


name: datasette-glitch
summary: Utilities to help run Datasette on Glitch
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-glitch
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-glitch/
project_url: https://pypi.org/project/datasette-glitch/
release_url: https://pypi.org/project/datasette-glitch/0.1/
version: 0.1
description:
  # datasette-glitch

  [![PyPI](https://img.shields.io/pypi/v/datasette-glitch.svg)](https://pypi.org/project/datasette-glitch/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-glitch?label=changelog)](https://github.com/simonw/datasette-glitch/releases)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-glitch/blob/master/LICENSE)

  Utilities to help run Datasette on Glitch

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ pip install datasette-glitch

  ## Usage

  This plugin outputs a special link which will sign you into Datasette as the root user.

  Click Tools -> Logs in the Glitch editor interface after your app starts to see the link.

project_urls:
  {"CI": "https://github.com/simonw/datasette-glitch/actions", "Changelog": "https://github.com/simonw/datasette-glitch/releases", "Homepage": "https://github.com/simonw/datasette-glitch", "Issues": "https://github.com/simonw/datasette-glitch/issues"}

requires_dist:
  ["datasette (>=0.45)", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "httpx ; extra == 'test'"]


name: datasette-graphql
summary: Datasette plugin providing an automatic GraphQL API for your SQLite databases
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-graphql
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-graphql/
project_url: https://pypi.org/project/datasette-graphql/
release_url: https://pypi.org/project/datasette-graphql/2.1.1/
version: 2.1.1
description:
  # datasette-graphql

  [![PyPI](https://img.shields.io/pypi/v/datasette-graphql.svg)](https://pypi.org/project/datasette-graphql/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-graphql?include_prereleases&label=changelog)](https://github.com/simonw/datasette-graphql/releases)
  [![Tests](https://github.com/simonw/datasette-graphql/workflows/Test/badge.svg)](https://github.com/simonw/datasette-graphql/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-graphql/blob/main/LICENSE)

  **Datasette plugin providing an automatic GraphQL API for your SQLite databases**

  Read more about this project: [GraphQL in Datasette with the new datasette-graphql plugin](https://simonwillison.net/2020/Aug/7/datasette-graphql/)

  Try out a live demo at [datasette-graphql-demo.datasette.io/graphql](https://datasette-graphql-demo.datasette.io/graphql?query=%7B%0A%20%20repos(first%3A10%2C%20search%3A%20%22sql%22%2C%20sort_desc%3A%20created_at)%20%7B%0A%20%20%20%20totalCount%0A%20%20%20%20pageInfo%20%7B%0A%20%20%20%20%20%20endCursor%0A%20%20%20%20%20%20hasNextPage%0A%20%20%20%20%7D%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20description_%0A%20%20%20%20%09stargazers_count%0A%20%20%20%20%20%20created_at%0A%20%20%20%20%20%20owner%20%7B%0A%20%20%20%20%20%20%20%20name%0A%20%20%20%20%20%20%20%20html_url%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A)

  <!-- toc -->

  - [Installation](#installation)
  - [Usage](#usage)
    * [Querying for tables and columns](#querying-for-tables-and-columns)
    * [Fetching a single record](#fetching-a-single-record)
    * [Accessing nested objects](#accessing-nested-objects)
    * [Accessing related objects](#accessing-related-objects)
    * [Filtering tables](#filtering-tables)
    * [Sorting](#sorting)
    * [Pagination](#pagination)
    * [Search](#search)
    * [Columns containing JSON strings](#columns-containing-json-strings)
    * [Auto camelCase](#auto-camelcase)
    * [CORS](#cors)
    * [Execution limits](#execution-limits)
  - [The graphql() template function](#the-graphql-template-function)
  - [Adding custom fields with plugins](#adding-custom-fields-with-plugins)
  - [Development](#development)

  <!-- tocstop -->

  ![GraphiQL animated demo](https://static.simonwillison.net/static/2020/graphiql.gif)

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-graphql

  ## Usage

  This plugin sets up `/graphql` as a GraphQL endpoint for the first attached database.

  If you have multiple attached databases each will get its own endpoint at `/graphql/name_of_database`.

  The automatically generated GraphQL schema is available at `/graphql/name_of_database.graphql` - here's [an example](https://datasette-graphql-demo.datasette.io/graphql/github.graphql).

  ### Querying for tables and columns

  Individual tables (and SQL views) can be queried like this:

  ```graphql
  {
    repos {
      nodes {
        id
        full_name
        description_
      }
    }
  }
  ```
  [Try this query](https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20repos%20%7B%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20id%0A%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20description_%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A)

  In this example query the underlying database table is called `repos` and its columns include `id`, `full_name` and `description`. Since `description` is a reserved word the query needs to ask for `description_` instead.

  ### Fetching a single record

  If you only want to fetch a single record - for example if you want to fetch a row by its primary key - you can use the `tablename_row` field:

  ```graphql
  {
    repos_row(id: 107914493) {
      id
      full_name
      description_
    }
  }
  ```
  [Try this query](https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20repos_row%28id%3A%20107914493%29%20%7B%0A%20%20%20%20id%0A%20%20%20%20full_name%0A%20%20%20%20description_%0A%20%20%7D%0A%7D%0A)

  The `tablename_row` field accepts the primary key column (or columns) as arguments. It also supports the same `filter:`, `search:`, `sort:` and `sort_desc:` arguments as the `tablename` field, described below.

  ### Accessing nested objects

  If a column is a foreign key to another table, you can request columns from the table pointed to by that foreign key using a nested query like this:

  ```graphql
  {
    repos {
      nodes {
        id
        full_name
        owner {
          id
          login
        }
      }
    }
  }
  ```
  [Try this query](https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20repos%20%7B%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20id%0A%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20owner%20%7B%0A%20%20%20%20%20%20%20%20id%0A%20%20%20%20%20%20%20%20login%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A)

  ### Accessing related objects

  If another table has a foreign key back to the table you are accessing, you can fetch rows from that related table.

  Consider a `users` table which is related to `repos` - a repo has a foreign key back to the user that owns the repository. The `users` object type will have a `repos_by_owner_list` field which can be used to access those related repos:

  ```graphql
  {
    users(first: 1, search: "simonw") {
      nodes {
        name
        repos_by_owner_list(first: 5) {
          totalCount
          nodes {
            full_name
          }
        }
      }
    }
  }
  ```
  [Try this query](https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20users%28first%3A%201%2C%20search%3A%20%22simonw%22%29%20%7B%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20name%0A%20%20%20%20%20%20repos_by_owner_list%28first%3A%205%29%20%7B%0A%20%20%20%20%20%20%20%20totalCount%0A%20%20%20%20%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A)


  ### Filtering tables

  You can filter the rows returned for a specific table using the `filter:` argument. This accepts a filter object mapping columns to operations. For example, to return just repositories with the Apache 2 license and more than 10 stars:

  ```graphql
  {
    repos(filter: {license: {eq: "apache-2.0"}, stargazers_count: {gt: 10}}) {
      nodes {
        full_name
        stargazers_count
        license {
          key
        }
      }
    }
  }
  ```
  [Try this query](https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20repos%28filter%3A%20%7Blicense%3A%20%7Beq%3A%20%22apache-2.0%22%7D%2C%20stargazers_count%3A%20%7Bgt%3A%2010%7D%7D%29%20%7B%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20stargazers_count%0A%20%20%20%20%20%20license%20%7B%0A%20%20%20%20%20%20%20%20key%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A)

  See [table filters examples](https://github.com/simonw/datasette-graphql/blob/main/examples/filters.md) for more operations, and [column filter arguments](https://docs.datasette.io/en/stable/json_api.html#column-filter-arguments) in the Datasette documentation for details of how those operations work.

  These same filters can be used on nested relationships, like so:

  ```graphql
  {
    users_row(id: 9599) {
      name
      repos_by_owner_list(filter: {name: {startswith: "datasette-"}}) {
        totalCount
        nodes {
          full_name
        }
      }
    }
  }
  ```
  [Try this query](https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20users_row%28id%3A%209599%29%20%7B%0A%20%20%20%20name%0A%20%20%20%20repos_by_owner_list%28filter%3A%20%7Bname%3A%20%7Bstartswith%3A%20%22datasette-%22%7D%7D%29%20%7B%0A%20%20%20%20%20%20totalCount%0A%20%20%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A)


  The `where:` argument can be used as an alternative to `filter:` when the thing you are expressing is too complex to be modeled using a filter expression. It accepts a string fragment of SQL that will be included in the `WHERE` clause of the SQL query.

  ```graphql
  {
    repos(where: "name='sqlite-utils' or name like 'datasette-%'") {
      totalCount
      nodes {
        full_name
      }
    }
  }
  ```
  [Try this query](https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20repos%28where%3A%20%22name%3D%27sqlite-utils%27%20or%20name%20like%20%27datasette-%25%27%22%29%20%7B%0A%20%20%20%20totalCount%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20full_name%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A)

  ### Sorting

  You can set a sort order for results from a table using the `sort:` or `sort_desc:` arguments. The value for this argument should be the name of the column you wish to sort (or sort-descending) by.

  ```graphql
  {
    repos(sort_desc: stargazers_count) {
      nodes {
        full_name
        stargazers_count
      }
    }
  }
  ```
  [Try this query](https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20repos%28sort_desc%3A%20stargazers_count%29%20%7B%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20stargazers_count%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A)

  ### Pagination

  By default the first 10 rows will be returned. You can control this using the `first:` argument.

  ```graphql
  {
    repos(first: 20) {
      totalCount
      pageInfo {
        hasNextPage
        endCursor
      }
      nodes {
        full_name
        stargazers_count
        license {
          key
        }
      }
    }
  }
  ```
  [Try this query](https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20repos%28first%3A%2020%29%20%7B%0A%20%20%20%20totalCount%0A%20%20%20%20pageInfo%20%7B%0A%20%20%20%20%20%20hasNextPage%0A%20%20%20%20%20%20endCursor%0A%20%20%20%20%7D%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20stargazers_count%0A%20%20%20%20%20%20license%20%7B%0A%20%20%20%20%20%20%20%20key%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A)

  The `totalCount` field returns the total number of records that match the query.

  Requesting the `pageInfo.endCursor` field provides you with the value you need to request the next page. You can pass this to the `after:` argument to request the next page.

  ```graphql
  {
    repos(first: 20, after: "134874019") {
      totalCount
      pageInfo {
        hasNextPage
        endCursor
      }
      nodes {
        full_name
        stargazers_count
        license {
          key
        }
      }
    }
  }
  ```
  [Try this query](https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20repos%28first%3A%2020%2C%20after%3A%20%22134874019%22%29%20%7B%0A%20%20%20%20totalCount%0A%20%20%20%20pageInfo%20%7B%0A%20%20%20%20%20%20hasNextPage%0A%20%20%20%20%20%20endCursor%0A%20%20%20%20%7D%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20stargazers_count%0A%20%20%20%20%20%20license%20%7B%0A%20%20%20%20%20%20%20%20key%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A)

  The `hasNextPage` field tells you if there are any more records.

  ### Search

  If a table has been configured to use SQLite full-text search you can execute searches against it using the `search:` argument:

  ```graphql
  {
    repos(search: "datasette") {
      totalCount
      pageInfo {
        hasNextPage
        endCursor
      }
      nodes {
        full_name
        description_
      }
    }
  }
  ```
  [Try this query](https://datasette-graphql-demo.datasette.io/graphql?query=%0A%7B%0A%20%20repos%28search%3A%20%22datasette%22%29%20%7B%0A%20%20%20%20totalCount%0A%20%20%20%20pageInfo%20%7B%0A%20%20%20%20%20%20hasNextPage%0A%20%20%20%20%20%20endCursor%0A%20%20%20%20%7D%0A%20%20%20%20nodes%20%7B%0A%20%20%20%20%20%20full_name%0A%20%20%20%20%20%20description_%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A)

  The [sqlite-utils](https://sqlite-utils.datasette.io/) Python library and CLI tool can be used to add full-text search to an existing database table.

  ### Columns containing JSON strings

  If your table has a column that contains data encoded as JSON, `datasette-graphql` will make that column available as an encoded JSON string. Clients calling your API will need to parse the string as JSON in order to access the data.

  You can return the data as a nested structure by configuring that column to be treated as a JSON column. The [plugin configuration](https://docs.datasette.io/en/stable/plugins.html#plugin-configuration) for that in `metadata.json` looks like this:

  ```json
  {
      "databases": {
          "test": {
              "tables": {
                  "repos": {
                      "plugins": {
                          "datasette-graphql": {
                              "json_columns": [
                                  "tags"
                              ]
                          }
                      }
                  }
              }
          }
      }
  }
  ```

  ### Auto camelCase

  The names of your columns and tables default to being matched by their representations in GraphQL.

  If you have tables with `names_like_this` you may want to work with them in GraphQL using `namesLikeThis`, for consistency with GraphQL and JavaScript conventions.

  You can turn on automatic camelCase using the `"auto_camelcase"` plugin configuration setting in `metadata.json`, like this:

  ```json
  {
      "plugins": {
          "datasette-graphql": {
              "auto_camelcase": true
          }
      }
  }
  ```

  ### CORS

  This plugin obeys the `--cors` option passed to the `datasette` command-line tool. If you pass `--cors` it adds the following CORS HTTP headers to allow JavaScript running on other domains to access the GraphQL API:

      access-control-allow-headers: content-type
      access-control-allow-method: POST
      access-control-allow-origin: *

  ### Execution limits

  The plugin implements two limits by default:

  - The total time spent executing all of the underlying SQL queries that make up the GraphQL execution must not exceed 1000ms (one second)
  - The total number of SQL table queries executed as a result of nested GraphQL fields must not exceed 100

  These limits can be customized using the `num_queries_limit` and `time_limit_ms` plugin configuration settings, for example in `metadata.json`:

  ```json
  {
      "plugins": {
          "datasette-graphql": {
              "num_queries_limit": 200,
              "time_limit_ms": 5000
          }
      }
  }
  ```
  Setting these to `0` will disable the limit checks entirely.

  ## The graphql() template function

  The plugin also makes a Jinja template function available called `graphql()`. You can use that function in your Datasette [custom templates](https://docs.datasette.io/en/stable/custom_templates.html#custom-templates) like so:

  ```html+jinja
  {% set users = graphql("""
  {
      users {
          nodes {
              name
              points
              score
          }
      }
  }
  """)["users"] %}
  {% for user in users.nodes %}
      <p>{{ user.name }} - points: {{ user.points }}, score = {{ user.score }}</p>
  {% endfor %}
  ```

  The function executes a GraphQL query against the generated schema and returns the results. You can assign those results to a variable in your template and then loop through and display them.

  By default the query will be run against the first attached database. You can use the optional second argument to the function to specify a different database - for example, to run against an attached `github.db` database you would do this:

  ```html+jinja
  {% set user = graphql("""
  {
      users_row(id:9599) {
          name
          login
          avatar_url
      }
  }
  """, "github")["users_row"] %}

  <h1>Hello, {{ user.name }}</h1>
  ```

  You can use [GraphQL variables](https://graphql.org/learn/queries/#variables) in these template calls by passing them to the `variables=` argument:

  ```html+jinja
  {% set user = graphql("""
  query ($id: Int) {
      users_row(id: $id) {
          name
          login
          avatar_url
      }
  }
  """, database="github", variables={"id": 9599})["users_row"] %}

  <h1>Hello, {{ user.name }}</h1>
  ```
  ## Adding custom fields with plugins

  `datasette-graphql` adds a new [plugin hook](https://docs.datasette.io/en/stable/writing_plugins.html) to Datasette which can be used to add custom fields to your GraphQL schema.

  The plugin hook looks like this:

  ```python
  @hookimpl
  def graphql_extra_fields(datasette, database):
      "A list of (name, field_type) tuples to include in the GraphQL schema"
  ```

  You can use this hook to return a list of tuples describing additional fields that should be exposed in your schema. Each tuple should consist of a string naming the new field, plus a [Graphene Field object](https://docs.graphene-python.org/en/latest/types/objecttypes/) that specifies the schema and provides a `resolver` function.

  This example implementation uses `pkg_resources` to return a list of currently installed Python packages:

  ```python
  import graphene
  from datasette import hookimpl
  import pkg_resources


  @hookimpl
  def graphql_extra_fields():
      class Package(graphene.ObjectType):
          "An installed package"
          name = graphene.String()
          version = graphene.String()

      def resolve_packages(root, info):
          return [
              {"name": d.project_name, "version": d.version}
              for d in pkg_resources.working_set
          ]

      return [
          (
              "packages",
              graphene.Field(
                  graphene.List(Package),
                  description="List of installed packages",
                  resolver=resolve_packages,
              ),
          ),
      ]
  ```

  With this plugin installed, the following GraphQL query can be used to retrieve a list of installed packages:

  ```graphql
  {
    packages {
      name
      version
    }
  }
  ```

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-graphql
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-graphql/actions", "Changelog": "https://github.com/simonw/datasette-graphql/releases", "Homepage": "https://github.com/simonw/datasette-graphql", "Issues": "https://github.com/simonw/datasette-graphql/issues"}

requires_dist:
  ["datasette (>=0.58.1)", "graphene (<4.0,>=3.1.0)", "graphql-core (>=3.2.1)", "sqlite-utils", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]


name: datasette-gunicorn
summary: Run a Datasette server using Gunicorn
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-gunicorn
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-gunicorn/
project_url: https://pypi.org/project/datasette-gunicorn/
release_url: https://pypi.org/project/datasette-gunicorn/0.1/
requires_python: >=3.7
version: 0.1
description:
  # datasette-gunicorn

  [![PyPI](https://img.shields.io/pypi/v/datasette-gunicorn.svg)](https://pypi.org/project/datasette-gunicorn/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-gunicorn?include_prereleases&label=changelog)](https://github.com/simonw/datasette-gunicorn/releases)
  [![Tests](https://github.com/simonw/datasette-gunicorn/workflows/Test/badge.svg)](https://github.com/simonw/datasette-gunicorn/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-gunicorn/blob/main/LICENSE)

  Run a [Datasette](https://datasette.io/) server using [Gunicorn](https://gunicorn.org/)

  ## Installation

  Install this plugin in the same environment as Datasette.

      datasette install datasette-gunicorn

  ## Usage

  The plugin adds a new `datasette gunicorn` command. This takes most of the same options as `datasette serve`, plus one more option for setting the number of Gunicorn workers to start:

  `-w/--workers X` - set the number of workers. Defaults to 1.

  To start serving a database using 4 workers, run the following:

      datasette gunicorn fixtures.db -w 4

  It is advisable to switch your datasette [into WAL mode](https://til.simonwillison.net/sqlite/enabling-wal-mode) to get the best performance out of this configuration:

      sqlite3 fixtures.db 'PRAGMA journal_mode=WAL;'

  Run `datasette gunicorn --help` for a full list of options (which are the same as `datasette serve --help`, with the addition of the new `-w` option).

  ## datasette gunicorn --help

  Not all of the options to `datasette serve` are supported. Here's the full list of available options:

  <!-- [[[cog
  import cog
  from datasette import cli
  from click.testing import CliRunner
  runner = CliRunner()
  result = runner.invoke(cli.cli, ["gunicorn", "--help"])
  help = result.output.replace("Usage: cli", "Usage: datasette")
  cog.out(
      "```\n{}\n```".format(help)
  )
  ]]] -->
  ```
  Usage: datasette gunicorn [OPTIONS] [FILES]...

    Start a Gunicorn server running to serve Datasette

  Options:
    -i, --immutable PATH      Database files to open in immutable mode
    -h, --host TEXT           Host for server. Defaults to 127.0.0.1 which means
                              only connections from the local machine will be
                              allowed. Use 0.0.0.0 to listen to all IPs and allow
                              access from other machines.
    -p, --port INTEGER RANGE  Port for server, defaults to 8001. Use -p 0 to
                              automatically assign an available port.
                              [0<=x<=65535]
    --cors                    Enable CORS by serving Access-Control-Allow-Origin:
                              *
    --load-extension TEXT     Path to a SQLite extension to load
    --inspect-file TEXT       Path to JSON file created using "datasette inspect"
    -m, --metadata FILENAME   Path to JSON/YAML file containing license/source
                              metadata
    --template-dir DIRECTORY  Path to directory containing custom templates
    --plugins-dir DIRECTORY   Path to directory containing custom plugins
    --static MOUNT:DIRECTORY  Serve static files from this directory at /MOUNT/...
    --memory                  Make /_memory database available
    --config CONFIG           Deprecated: set config option using
                              configname:value. Use --setting instead.
    --setting SETTING...      Setting, see
                              docs.datasette.io/en/stable/settings.html
    --secret TEXT             Secret used for signing secure values, such as
                              signed cookies
    --version-note TEXT       Additional note to show on /-/versions
    --help-settings           Show available settings
    --create                  Create database files if they do not exist
    --crossdb                 Enable cross-database joins using the /_memory
                              database
    --nolock                  Ignore locking, open locked files in read-only mode
    -w, --workers INTEGER     Number of Gunicorn workers  [default: 1]
    --help                    Show this message and exit.

  ```
  <!-- [[[end]]] -->

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-gunicorn
      python3 -m venv venv
      source venv/bin/activate

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-gunicorn/actions", "Changelog": "https://github.com/simonw/datasette-gunicorn/releases", "Homepage": "https://github.com/simonw/datasette-gunicorn", "Issues": "https://github.com/simonw/datasette-gunicorn/issues"}

requires_dist:
  ["datasette", "gunicorn", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "cogapp ; extra == 'test'"]


name: datasette-gzip
summary: Add gzip compression to Datasette
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-gzip
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-gzip/
project_url: https://pypi.org/project/datasette-gzip/
release_url: https://pypi.org/project/datasette-gzip/0.2/
requires_dist: ["datasette", "asgi-gzip", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
requires_python: >=3.7
version: 0.2
description:
  # datasette-gzip

  [![PyPI](https://img.shields.io/pypi/v/datasette-gzip.svg)](https://pypi.org/project/datasette-gzip/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-gzip?include_prereleases&label=changelog)](https://github.com/simonw/datasette-gzip/releases)
  [![Tests](https://github.com/simonw/datasette-gzip/workflows/Test/badge.svg)](https://github.com/simonw/datasette-gzip/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-gzip/blob/main/LICENSE)

  Add gzip compression to Datasette

  ## Installation

  Install this plugin in the same environment as Datasette.

      datasette install datasette-gzip

  ## Usage

  Once installed, Datasette will obey the `Accept-Encoding:` header sent by browsers or other user agents and return content compressed in the most appropriate way.

  This plugin is a thin wrapper for the [asgi-gzip library](https://github.com/simonw/asgi-gzip), which extracts the [GzipMiddleware](https://www.starlette.io/middleware/#gzipmiddleware) from Starlette.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-gzip
      python3 -mvenv venv
      source venv/bin/activate

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-gzip/actions", "Changelog": "https://github.com/simonw/datasette-gzip/releases", "Homepage": "https://github.com/simonw/datasette-gzip", "Issues": "https://github.com/simonw/datasette-gzip/issues"}


name: datasette-hashed-urls
summary: Optimize Datasette performance behind a caching proxy
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-hashed-urls
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-hashed-urls/
project_url: https://pypi.org/project/datasette-hashed-urls/
release_url: https://pypi.org/project/datasette-hashed-urls/0.4/
requires_python: >=3.7
version: 0.4
description:
  # datasette-hashed-urls

  [![PyPI](https://img.shields.io/pypi/v/datasette-hashed-urls.svg)](https://pypi.org/project/datasette-hashed-urls/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-hashed-urls?include_prereleases&label=changelog)](https://github.com/simonw/datasette-hashed-urls/releases)
  [![Tests](https://github.com/simonw/datasette-hashed-urls/workflows/Test/badge.svg)](https://github.com/simonw/datasette-hashed-urls/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-hashed-urls/blob/main/LICENSE)

  Optimize Datasette performance behind a caching proxy

  When you open a database file in immutable mode using the `-i` option, Datasette calculates a SHA-256 hash of the contents of that file on startup.

  This content hash can then optionally be used to create URLs that are guaranteed to change if the contents of the file changes in the future.

  The result is pages  that can be cached indefinitely by both browsers and caching proxies - providing a significant performance boost.

  ## Demo

  A demo of this plugin is running at https://datasette-hashed-urls.vercel.app/

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-hashed-urls

  ## Usage

  Once installed, this plugin will act on any immutable database files that are loaded into Datasette:

      datasette -i fixtures.db

  The database will automatically be renamed to incorporate a hash of the contents of the SQLite file - so the above database would be served as:

      http://127.0.0.1:8001/fixtures-aa7318b

  Every page that accesss that database, including JSON endpoints, will be served with the following far-future cache expiry header:

      cache-control: max-age=31536000, public

  Here `max-age=31536000` is the number of seconds in a year.

  A caching proxy such as Cloudflare can then be used to cache and accelerate content served by Datasette.

  When the database file is updated and the server is restarted, the hash will change and content will be served from a new URL. Any hits to the previous hashed URLs will be automatically redirected.

  If you run Datasette using the `--crossdb` option to enable [cross-database queries](https://docs.datasette.io/en/stable/sql_queries.html#cross-database-queries) the `_memory` database will also have a hash added to its URL - in this case, the hash will be a combination of the hashes of the other attached databases.

  ## Configuration

  You can use the `max_age` plugin configuration setting to change the cache duration specified in the `cache-control` HTTP header.

  To set the cache expiry time to one hour you would add this to your Datasette `metadata.json` configuration file:

  ```json
  {
      "plugins": {
          "datasette-hashed-urls": {
              "max_age": 3600
          }
      }
  }
  ```

  ## History

  This functionality used to ship as part of Datasette itself, as a feature called [Hashed URL mode](https://docs.datasette.io/en/0.60.2/performance.html#hashed-url-mode).

  That feature has been deprecated and will be removed in Datasette 1.0. This plugin should be used as an alternative.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-hashed-urls
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-hashed-urls/actions", "Changelog": "https://github.com/simonw/datasette-hashed-urls/releases", "Homepage": "https://github.com/simonw/datasette-hashed-urls", "Issues": "https://github.com/simonw/datasette-hashed-urls/issues"}

requires_dist:
  ["datasette (>=0.61.1)", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "sqlite-utils ; extra == 'test'"]


name: datasette-haversine
summary: Datasette plugin that adds a custom SQL function for haversine distances
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-haversine
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-haversine/
project_url: https://pypi.org/project/datasette-haversine/
project_urls: {"Homepage": "https://github.com/simonw/datasette-haversine"}
release_url: https://pypi.org/project/datasette-haversine/0.2/
requires_dist: ["datasette", "haversine", "pytest ; extra == 'test'"]
version: 0.2
description:
  # datasette-haversine

  [![PyPI](https://img.shields.io/pypi/v/datasette-haversine.svg)](https://pypi.org/project/datasette-haversine/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-haversine?include_prereleases&label=changelog)](https://github.com/simonw/datasette-haversine/releases)
  [![Tests](https://github.com/simonw/datasette-haversine/workflows/Test/badge.svg)](https://github.com/simonw/datasette-haversine/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-haversine/blob/main/LICENSE)

  Datasette plugin that adds a custom SQL function for haversine distances

  Install this plugin in the same environment as Datasette to enable the `haversine()` SQL function.

      $ pip install datasette-haversine

  The plugin is built on top of the [haversine](https://github.com/mapado/haversine) library.

  ## haversine() to calculate distances

  ```sql
  select haversine(lat1, lon1, lat2, lon2);
  ```

  This will return the distance in kilometers between the point defined by `(lat1, lon1)` and the point defined by `(lat2, lon2)`.

  ## Custom units

  By default `haversine()` returns results in km. You can pass an optional third argument to get results in a different unit:

  - `ft` for feet
  - `m` for meters
  - `in` for inches
  - `mi` for miles
  - `nmi` for nautical miles
  - `km` for kilometers (the default)

  ```sql
  select haversine(lat1, lon1, lat2, lon2, 'mi');
  ```


name: datasette-hovercards
summary: Add preview hovercards to links in Datasette
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-hovercards
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-hovercards/
project_url: https://pypi.org/project/datasette-hovercards/
release_url: https://pypi.org/project/datasette-hovercards/0.1a0/
requires_dist: ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
requires_python: >=3.6
version: 0.1a0
description:
  # datasette-hovercards

  [![PyPI](https://img.shields.io/pypi/v/datasette-hovercards.svg)](https://pypi.org/project/datasette-hovercards/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-hovercards?include_prereleases&label=changelog)](https://github.com/simonw/datasette-hovercards/releases)
  [![Tests](https://github.com/simonw/datasette-hovercards/workflows/Test/badge.svg)](https://github.com/simonw/datasette-hovercards/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-hovercards/blob/main/LICENSE)

  Add preview hovercards to links in Datasette

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-hovercards

  ## Usage

  Once installed, hovering over a link to a row within the Datasette interface - for example a foreign key reference on the table page - should show a hovercard with a preview of that row.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-hovercards
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-hovercards/actions", "Changelog": "https://github.com/simonw/datasette-hovercards/releases", "Homepage": "https://github.com/simonw/datasette-hovercards", "Issues": "https://github.com/simonw/datasette-hovercards/issues"}


name: datasette-ics
summary: Datasette plugin for outputting iCalendar files
classifiers: ["Framework :: Datasette", "License :: OSI Approved :: Apache Software License"]
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-ics
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-ics/
project_url: https://pypi.org/project/datasette-ics/
release_url: https://pypi.org/project/datasette-ics/0.5.2/
version: 0.5.2
description:
  # datasette-ics

  [![PyPI](https://img.shields.io/pypi/v/datasette-ics.svg)](https://pypi.org/project/datasette-ics/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-ics?include_prereleases&label=changelog)](https://github.com/simonw/datasette-ics/releases)
  [![Tests](https://github.com/simonw/datasette-ics/workflows/Test/badge.svg)](https://github.com/simonw/datasette-ics/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-ics/blob/main/LICENSE)

  Datasette plugin that adds support for generating [iCalendar .ics files](https://tools.ietf.org/html/rfc5545) with the results of a SQL query.

  ## Installation

  Install this plugin in the same environment as Datasette to enable the `.ics` output extension.

      $ pip install datasette-ics

  ## Usage

  To create an iCalendar file you need to define a custom SQL query that returns a required set of columns:

  * `event_name` - the short name for the event
  * `event_dtstart` - when the event starts

  The following columns are optional:

  * `event_dtend` - when the event ends
  * `event_duration` - the duration of the event (use instead of `dtend`)
  * `event_description` - a longer description of the event
  * `event_uid` - a globally unique identifier for this event
  * `event_tzid` - the timezone for the event, e.g. `America/Chicago`

  A query that returns these columns can then be returned as an ics feed by adding the `.ics` extension.

  ## Demo

  [This SQL query]([https://www.rockybeaches.com/data?sql=with+inner+as+(%0D%0A++select%0D%0A++++datetime%2C%0D%0A++++substr(datetime%2C+0%2C+11)+as+date%2C%0D%0A++++mllw_feet%2C%0D%0A++++lag(mllw_feet)+over+win+as+previous_mllw_feet%2C%0D%0A++++lead(mllw_feet)+over+win+as+next_mllw_feet%0D%0A++from%0D%0A++++tide_predictions%0D%0A++where%0D%0A++++station_id+%3D+%3Astation_id%0D%0A++++and+datetime+%3E%3D+date()%0D%0A++++window+win+as+(%0D%0A++++++order+by%0D%0A++++++++datetime%0D%0A++++)%0D%0A++order+by%0D%0A++++datetime%0D%0A)%2C%0D%0Alowest_tide_per_day+as+(%0D%0A++select%0D%0A++++date%2C%0D%0A++++datetime%2C%0D%0A++++mllw_feet%0D%0A++from%0D%0A++++inner%0D%0A++where%0D%0A++++mllw_feet+%3C%3D+previous_mllw_feet%0D%0A++++and+mllw_feet+%3C%3D+next_mllw_feet%0D%0A)%0D%0Aselect%0D%0A++min(datetime)+as+event_dtstart%2C%0D%0A++%27Low+tide%3A+%27+||+mllw_feet+||+%27+feet%27+as+event_name%2C%0D%0A++%27America%2FLos_Angeles%27+as+event_tzid%0D%0Afrom%0D%0A++lowest_tide_per_day%0D%0Agroup+by%0D%0A++date%0D%0Aorder+by%0D%0A++date&station_id=9414131) calculates the lowest tide per day at Pillar Point in Half Moon Bay, California.

  Since the query returns `event_name`, `event_dtstart` and `event_tzid` columns it produces [this ICS feed](https://www.rockybeaches.com/data.ics?sql=with+inner+as+(%0D%0A++select%0D%0A++++datetime%2C%0D%0A++++substr(datetime%2C+0%2C+11)+as+date%2C%0D%0A++++mllw_feet%2C%0D%0A++++lag(mllw_feet)+over+win+as+previous_mllw_feet%2C%0D%0A++++lead(mllw_feet)+over+win+as+next_mllw_feet%0D%0A++from%0D%0A++++tide_predictions%0D%0A++where%0D%0A++++station_id+%3D+%3Astation_id%0D%0A++++and+datetime+%3E%3D+date()%0D%0A++++window+win+as+(%0D%0A++++++order+by%0D%0A++++++++datetime%0D%0A++++)%0D%0A++order+by%0D%0A++++datetime%0D%0A)%2C%0D%0Alowest_tide_per_day+as+(%0D%0A++select%0D%0A++++date%2C%0D%0A++++datetime%2C%0D%0A++++mllw_feet%0D%0A++from%0D%0A++++inner%0D%0A++where%0D%0A++++mllw_feet+%3C%3D+previous_mllw_feet%0D%0A++++and+mllw_feet+%3C%3D+next_mllw_feet%0D%0A)%0D%0Aselect%0D%0A++min(datetime)+as+event_dtstart%2C%0D%0A++%27Low+tide%3A+%27+||+mllw_feet+||+%27+feet%27+as+event_name%2C%0D%0A++%27America%2FLos_Angeles%27+as+event_tzid%0D%0Afrom%0D%0A++lowest_tide_per_day%0D%0Agroup+by%0D%0A++date%0D%0Aorder+by%0D%0A++date&station_id=9414131). If you subscribe to that in a calendar application such as Apple Calendar you get something that looks like this:

  ![Apple Calendar showing low tides at Pillar Point during a week](https://user-images.githubusercontent.com/9599/173158984-e5ec6bd0-33fc-4fc0-ba9d-17ae674f310a.jpg)

  ## Using a canned query

  Datasette's [canned query mechanism](https://datasette.readthedocs.io/en/stable/sql_queries.html#canned-queries) can be used to configure calendars. If a canned query definition has a `title` that will be used as the title of the calendar.

  Here's an example, defined using a `metadata.yaml` file:

  ```yaml
  databases:
    mydatabase:
      queries:
        calendar:
          title: My Calendar
          sql: |-
            select
              title as event_name,
              start as event_dtstart,
              description as event_description
            from
              events
            order by
              start
            limit
              100
  ```
  This will result in a calendar feed at `http://localhost:8001/mydatabase/calendar.ics`

project_urls:
  {"CI": "https://github.com/simonw/datasette-ics/actions", "Changelog": "https://github.com/simonw/datasette-ics/releases", "Homepage": "https://github.com/simonw/datasette-ics", "Issues": "https://github.com/simonw/datasette-ics/issues"}

requires_dist:
  ["datasette (>=0.49)", "ics (==0.7.2)", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]


name: datasette-import-table
summary: Datasette plugin for importing tables from other Datasette instances
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-import-table
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-import-table/
project_url: https://pypi.org/project/datasette-import-table/
release_url: https://pypi.org/project/datasette-import-table/0.3/
version: 0.3
description:
  # datasette-import-table

  [![PyPI](https://img.shields.io/pypi/v/datasette-import-table.svg)](https://pypi.org/project/datasette-import-table/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-import-table?include_prereleases&label=changelog)](https://github.com/simonw/datasette-import-table/releases)
  [![Tests](https://github.com/simonw/datasette-import-table/workflows/Test/badge.svg)](https://github.com/simonw/datasette-import-table/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-import-table/blob/main/LICENSE)

  Datasette plugin for importing tables from other Datasette instances

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-import-table

  ## Usage

  Visit `/-/import-table` for the interface. Paste in the URL to a table page on another Datasette instance and click the button to import that table.

  By default only [the root actor](https://datasette.readthedocs.io/en/stable/authentication.html#using-the-root-actor) can access the page - so you'll need to run Datasette with the `--root` option and click on the link shown in the terminal to sign in and access the page.

  The `import-table` permission governs access. You can use permission plugins such as [datasette-permissions-sql](https://github.com/simonw/datasette-permissions-sql) to grant additional access to the write interface.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-import-table
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-import-table/actions", "Changelog": "https://github.com/simonw/datasette-import-table/releases", "Homepage": "https://github.com/simonw/datasette-import-table", "Issues": "https://github.com/simonw/datasette-import-table/issues"}

requires_dist:
  ["datasette", "httpx", "sqlite-utils", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "httpx ; extra == 'test'", "pytest-httpx ; extra == 'test'"]


name: datasette-indieauth
summary: Datasette authentication using IndieAuth and RelMeAuth
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-indieauth
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-indieauth/
project_url: https://pypi.org/project/datasette-indieauth/
release_url: https://pypi.org/project/datasette-indieauth/1.2.1/
requires_python: >=3.7
version: 1.2.1
description:
  # datasette-indieauth

  [![PyPI](https://img.shields.io/pypi/v/datasette-indieauth.svg)](https://pypi.org/project/datasette-indieauth/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-indieauth?include_prereleases&label=changelog)](https://github.com/simonw/datasette-indieauth/releases)
  [![codecov](https://codecov.io/gh/simonw/datasette-indieauth/branch/main/graph/badge.svg)](https://codecov.io/gh/simonw/datasette-indieauth)
  [![Tests](https://github.com/simonw/datasette-indieauth/workflows/Test/badge.svg)](https://github.com/simonw/datasette-indieauth/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-indieauth/blob/main/LICENSE)

  Datasette authentication using [IndieAuth](https://indieauth.net/).

  ## Demo

  You can try out the latest version of this plugin at [datasette-indieauth-demo.datasette.io](https://datasette-indieauth-demo.datasette.io/-/indieauth)

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-indieauth

  ## Usage

  Ensure you have a website with a domain that supports IndieAuth or RelMeAuth. The easiest way to do that is to add the following HTML to your homepage, linking to your personal GitHub profile:

  ```html
  <link href="https://github.com/simonw" rel="me">
  <link rel="authorization_endpoint" href="https://indieauth.com/auth">
  ```
  Your GitHub profile needs to link back to your website, to prove that your GitHub account should be a valid identifier for that page.

  Now visit `/-/indieauth` on your Datasette instance to begin the sign-in progress.

  ## Actor

  When a user signs in using IndieAuth they will be recieve a signed `ds_actor` cookie identifying them as an [actor](https://docs.datasette.io/en/stable/authentication.html#actors) that looks like this:

  ```json
  {
      "me": "https://simonwillison.net/",
      "display": "simonwillison.net"
  }
  ```

  If the IndieAuth server returned additional `"profile"` fields those will be merged into the actor. You can visit `/-/actor` on your Datasette instance to see the full actor you are currently signed in as.

  ## Restricting access with the restrict_access plugin configuration

  You can use [Datasette's permissions system](https://docs.datasette.io/en/stable/authentication.html#permissions) to control permissions of authenticated users - by default, an authenticated user will be able to perform the same actions as an unauthenticated user.

  As a shortcut if you want to lock down access to your instance entirely to just specific users, you can use the `restrict_access` plugin configuration option like this:

  ```json
  {
      "plugins": {
          "datasette-indieauth": {
              "restrict_access": "https://simonwillison.net/"
          }
      }
  }
  ```

  This can be a string or a list of user identifiers. It can also be a space separated list, which means you can use it with the [datasette publish](https://docs.datasette.io/en/stable/publish.html#datasette-publish) `--plugin-secret` configuration option to set permissions as part of a deployment, like this:
  ```
  datasette publish vercel mydb.db --project my-secret-db \
      --install datasette-indieauth \
      --plugin-secret datasette-indieauth restrict_access https://simonwillison.net/
  ```
  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-indieauth
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-indieauth/actions", "Changelog": "https://github.com/simonw/datasette-indieauth/releases", "Homepage": "https://github.com/simonw/datasette-indieauth", "Issues": "https://github.com/simonw/datasette-indieauth/issues"}

requires_dist:
  ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "httpx ; extra == 'test'", "pytest-httpx ; extra == 'test'", "mf2py ; extra == 'test'"]


name: datasette-init
summary: Ensure specific tables and views exist on startup
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-init
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-init/
project_url: https://pypi.org/project/datasette-init/
release_url: https://pypi.org/project/datasette-init/0.2/
version: 0.2
description:
  # datasette-init

  [![PyPI](https://img.shields.io/pypi/v/datasette-init.svg)](https://pypi.org/project/datasette-init/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-init?label=changelog)](https://github.com/simonw/datasette-init/releases)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-init/blob/master/LICENSE)

  Ensure specific tables and views exist on startup

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ pip install datasette-init

  ## Usage

  This plugin is configured using `metadata.json` (or `metadata.yaml`). 

  ### Creating tables

  Add a block like this that specifies the tables you would like to ensure exist:

  ```json
  {
    "plugins": {
      "datasette-init": {
        "my_database": {
          "tables": {
            "dogs": {
              "columns": {
                "id": "integer",
                "name": "text",
                "age": "integer",
                "weight": "float"
              },
              "pk": "id"
            }
          }
        }
      }
    }
  }
  ```

  Any tables that do not yet exist will be created when Datasette first starts.

  Valid column types are `"integer"`, `"text"`, `"float"` and `"blob"`.

  The `"pk"` is optional, and is used to define the primary key. To define a compound primary key (across more than one column) use a list of column names here:
  ```json
      "pk": ["id1", "id2"]
  ```

  ### Creating views

  The plugin can also be used to create views:

  ```json
  {
    "plugins": {
      "datasette-init": {
        "my_database": {
          "views": {
            "my_view": "select 1 + 1"
          }
        }
      }
    }
  }
  ```

  Each view in the ``"views"`` block will be created when the Database first starts. If a view with the same name already exists it will be replaced with the new definition.


  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-init
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-init/actions", "Changelog": "https://github.com/simonw/datasette-init/releases", "Homepage": "https://github.com/simonw/datasette-init", "Issues": "https://github.com/simonw/datasette-init/issues"}

requires_dist:
  ["datasette (>=0.45)", "sqlite-utils", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "httpx ; extra == 'test'"]


name: datasette-insert
summary: Datasette plugin for inserting and updating data
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://datasette.io/plugins/datasette-insert
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-insert/
project_url: https://pypi.org/project/datasette-insert/
release_url: https://pypi.org/project/datasette-insert/0.8/
requires_python: >=3.7
version: 0.8
description:
  # datasette-insert

  [![PyPI](https://img.shields.io/pypi/v/datasette-insert.svg)](https://pypi.org/project/datasette-insert/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-insert?include_prereleases&label=changelog)](https://github.com/simonw/datasette-insert/releases)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-insert/blob/master/LICENSE)

  Datasette plugin for inserting and updating data

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ pip install datasette-insert

  This plugin should always be deployed with additional configuration to prevent unauthenticated access, see notes below.

  If you are trying it out on your own local machine, you can `pip install` the [datasette-insert-unsafe](https://github.com/simonw/datasette-insert-unsafe) plugin to allow access without needing to set up authentication or permissions separately.

  ## Inserting data and creating tables

  Start datasette and make sure it has a writable SQLite database attached to it. If you have not yet created a database file you can use this:

      datasette data.db --create

  The `--create` option will create a new empty `data.db` database file if it does not already exist.

  The plugin adds an endpoint that allows data to be inserted or updated and tables to be created by POSTing JSON data to the following URL:

      /-/insert/name-of-database/name-of-table

  The JSON should look like this:

  ```json
  [
      {
          "id": 1,
          "name": "Cleopaws",
          "age": 5
      },
      {
          "id": 2,
          "name": "Pancakes",
          "age": 5
      }
  ]
  ```

  The first time data is posted to the URL a table of that name will be created if it does not aready exist, with the desired columns.

  You can specify which column should be used as the primary key using the `?pk=` URL argument.

  Here's how to POST to a database and create a new table using the Python `requests` library:

  ```python
  import requests

  requests.post("http://localhost:8001/-/insert/data/dogs?pk=id", json=[
      {
          "id": 1,
          "name": "Cleopaws",
          "age": 5
      },
      {
          "id": 2,
          "name": "Pancakes",
          "age": 4
      }
  ])
  ```
  And here's how to do the same thing using `curl`:

  ```
  curl --request POST \
    --data '[
        {
          "id": 1,
          "name": "Cleopaws",
          "age": 5
        },
        {
          "id": 2,
          "name": "Pancakes",
          "age": 4
        }
      ]' \
      'http://localhost:8001/-/insert/data/dogs?pk=id'
  ```
  Or by piping in JSON like so:

      cat dogs.json | curl --request POST -d @- \
          'http://localhost:8001/-/insert/data/dogs?pk=id'

  ### Inserting a single row

  If you are inserting a single row you can optionally send it as a dictionary rather than a list with a single item:

  ```
  curl --request POST \
    --data '{
        "id": 1,
        "name": "Cleopaws",
        "age": 5
      }' \
      'http://localhost:8001/-/insert/data/dogs?pk=id'
  ```

  ### Automatically adding new columns

  If you send data to an existing table with keys that are not reflected by the existing columns, you will get an HTTP 400 error with a JSON response like this:

  ```json
  {
      "status": 400,
      "error": "Unknown keys: 'foo'",
      "error_code": "unknown_keys"
  }
  ```

  If you add `?alter=1` to the URL you are posting to any missing columns will be automatically added:

  ```
  curl --request POST \
    --data '[
        {
          "id": 3,
          "name": "Boris",
          "age": 1,
          "breed": "Husky"
        }
      ]' \
      'http://localhost:8001/-/insert/data/dogs?alter=1'
  ```

  ## Upserting data

  An "upsert" operation can be used to partially update a record. With upserts you can send a subset of the keys and, if the ID matches the specified primary key, they will be used to update an existing record.

  Upserts can be sent to the `/-/upsert` API endpoint.

  This example will update the dog with ID=1's age from 5 to 7:
  ```
  curl --request POST \
    --data '{
        "id": 1,
        "age": 7
      }' \
      'http://localhost:3322/-/upsert/data/dogs?pk=id'
  ```
  Like the `/-/insert` endpoint, the `/-/upsert` endpoint can accept an array of objects too. It also supports the `?alter=1` option.

  ## Permissions and authentication

  This plugin defaults to denying all access, to help ensure people don't accidentally deploy it on the open internet in an unsafe configuration.

  You can read about [Datasette's approach to authentication](https://datasette.readthedocs.io/en/stable/authentication.html) in the Datasette manual.

  You can install the `datasette-insert-unsafe` plugin to run in unsafe mode, where all access is allowed by default.

  I recommend using this plugin in conjunction with [datasette-auth-tokens](https://github.com/simonw/datasette-auth-tokens), which provides a mechanism for making authenticated calls using API tokens.

  You can then use ["allow" blocks](https://datasette.readthedocs.io/en/stable/authentication.html#defining-permissions-with-allow-blocks) in the `datasette-insert` plugin configuration to specify which authenticated tokens are allowed to make use of the API.

  Here's an example `metadata.json` file which restricts access to the `/-/insert` API to an API token defined in an `INSERT_TOKEN` environment variable:

  ```json
  {
      "plugins": {
          "datasette-insert": {
              "allow": {
                  "bot": "insert-bot"
              }
          },
          "datasette-auth-tokens": {
              "tokens": [
                  {
                      "token": {
                          "$env": "INSERT_TOKEN"
                      },
                      "actor": {
                          "bot": "insert-bot"
                      }
                  }
              ]
          }
      }
  }
  ```
  With this configuration in place you can start Datasette like this:

      INSERT_TOKEN=abc123 datasette data.db -m metadata.json

  You can now send data to the API using `curl` like this:

  ```
  curl --request POST \
    -H "Authorization: Bearer abc123" \
    --data '[
        {
          "id": 3,
          "name": "Boris",
          "age": 1,
          "breed": "Husky"
        }
      ]' \
      'http://localhost:8001/-/insert/data/dogs'
  ```

  Or using the Python `requests` library like so:

  ```python
  requests.post(
      "http://localhost:8001/-/insert/data/dogs",
      json={"id": 1, "name": "Cleopaws", "age": 5},
      headers={"Authorization": "bearer abc123"},
  )
  ```

  ### Finely grained permissions

  Using an `"allow"` block as described above grants full permission to the features enabled by the API.

  The API implements several new Datasett permissions, which other plugins can use to make more finely grained decisions.

  The full set of permissions are as follows:

  - `insert:all` - all permissions - this is used by the `"allow"` block described above. Argument: `database_name`
  - `insert:insert-update` - the ability to insert data into an existing table, or to update data by its primary key. Arguments: `(database_name, table_name)`
  - `insert:create-table` - the ability to create a new table. Argument: `database_name`
  - `insert:alter-table` - the ability to add columns to an existing table (using `?alter=1`). Arguments: `(database_name, table_name)`

  You can use plugins like [datasette-permissions-sql](https://github.com/simonw/datasette-permissions-sql) to hook into these more detailed permissions for finely grained control over what actions each authenticated actor can take.

  Plugins that implement the [permission_allowed()](https://datasette.readthedocs.io/en/stable/plugin_hooks.html#plugin-hook-permission-allowed) plugin hook can take full control over these permission decisions.

  ## CORS

  If you start Datasette with the `datasette --cors` option the following HTTP headers will be added to resources served by this plugin:

      Access-Control-Allow-Origin: *
      Access-Control-Allow-Headers: content-type,authorization
      Access-Control-Allow-Methods: POST

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-insert
      python3 -m venv venv
      source venv/bin/activate

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-insert/actions", "Changelog": "https://github.com/simonw/datasette-insert/releases", "Homepage": "https://datasette.io/plugins/datasette-insert", "Issues": "https://github.com/simonw/datasette-insert/issues"}

requires_dist:
  ["datasette (>=0.46)", "sqlite-utils", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "httpx ; extra == 'test'", "datasette-auth-tokens ; extra == 'test'"]


name: datasette-jellyfish
summary: Datasette plugin adding SQL functions for fuzzy text matching powered by Jellyfish
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://datasette.io/plugins/datasette-jellyfish
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-jellyfish/
project_url: https://pypi.org/project/datasette-jellyfish/
release_url: https://pypi.org/project/datasette-jellyfish/1.0.1/
requires_dist: ["datasette", "jellyfish (>=0.8.2)", "pytest ; extra == 'test'"]
version: 1.0.1
description:
  # datasette-jellyfish

  [![PyPI](https://img.shields.io/pypi/v/datasette-jellyfish.svg)](https://pypi.org/project/datasette-jellyfish/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-jellyfish?include_prereleases&label=changelog)](https://github.com/simonw/datasette-jellyfish/releases)
  [![Tests](https://github.com/simonw/datasette-jellyfish/workflows/Test/badge.svg)](https://github.com/simonw/datasette-jellyfish/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-jellyfish/blob/main/LICENSE)

  Datasette plugin that adds custom SQL functions for fuzzy string matching, built on top of the [Jellyfish](https://github.com/jamesturk/jellyfish) Python library by James Turk and Michael Stephens.

  Interactive demos:

  * [soundex, metaphone, nysiis, match_rating_codex comparison](https://latest-with-plugins.datasette.io/fixtures?sql=SELECT%0D%0A++++soundex%28%3As%29%2C+%0D%0A++++metaphone%28%3As%29%2C+%0D%0A++++nysiis%28%3As%29%2C+%0D%0A++++match_rating_codex%28%3As%29&s=demo).
  * [distance functions comparison](https://latest-with-plugins.datasette.io/fixtures?sql=SELECT%0D%0A++++levenshtein_distance%28%3As1%2C+%3As2%29%2C%0D%0A++++damerau_levenshtein_distance%28%3As1%2C+%3As2%29%2C%0D%0A++++hamming_distance%28%3As1%2C+%3As2%29%2C%0D%0A++++jaro_similarity%28%3As1%2C+%3As2%29%2C%0D%0A++++jaro_winkler_similarity%28%3As1%2C+%3As2%29%2C%0D%0A++++match_rating_comparison%28%3As1%2C+%3As2%29%3B&s1=barrack+obama&s2=barrack+h+obama)

  Examples:

      SELECT soundex("hello");
          -- Outputs H400
      SELECT metaphone("hello");
          -- Outputs HL
      SELECT nysiis("hello");
          -- Outputs HAL
      SELECT match_rating_codex("hello");
          -- Outputs HLL
      SELECT porter_stem("running");
          -- Outputs run
      SELECT levenshtein_distance("hello", "hello world");
          -- Outputs 6
      SELECT damerau_levenshtein_distance("hello", "hello world");
          -- Outputs 6
      SELECT hamming_distance("hello", "hello world");
          -- Outputs 6
      SELECT jaro_similarity("hello", "hello world");
          -- Outputs 0.8181818181818182
      SELECT jaro_winkler_similarity("hello", "hello world");
          -- Outputs 0.890909090909091
      SELECT match_rating_comparison("hello", "helloo");
          -- Outputs 1

  See [the Jellyfish documentation](https://jellyfish.readthedocs.io/en/latest/) for an explanation of each of these functions.

project_urls:
  {"CI": "https://github.com/simonw/datasette-jellyfish/actions", "Changelog": "https://github.com/simonw/datasette-jellyfish/releases", "Homepage": "https://datasette.io/plugins/datasette-jellyfish", "Issues": "https://github.com/simonw/datasette-jellyfish/issues"}


name: datasette-jq
summary: Datasette plugin that adds custom SQL functions for executing jq expressions against JSON values
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-jq
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-jq/
project_url: https://pypi.org/project/datasette-jq/
project_urls: {"Homepage": "https://github.com/simonw/datasette-jq"}
release_url: https://pypi.org/project/datasette-jq/0.2.1/
requires_dist: ["datasette", "pyjq", "six", "pytest ; extra == 'test'"]
version: 0.2.1
description:
  # datasette-jq

  [![PyPI](https://img.shields.io/pypi/v/datasette-jq.svg)](https://pypi.org/project/datasette-jq/)
  [![CircleCI](https://circleci.com/gh/simonw/datasette-jq.svg?style=svg)](https://circleci.com/gh/simonw/datasette-jq)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-jq/blob/master/LICENSE)

  Datasette plugin that adds custom SQL functions for executing [jq](https://stedolan.github.io/jq/) expressions against JSON values.

  Install this plugin in the same environment as Datasette to enable the `jq()` SQL function.

  Usage:

      select jq(
          column_with_json,
          "{top_3: .classifiers[:3], v: .version}"
      )

  See [the jq manual](https://stedolan.github.io/jq/manual/#Basicfilters) for full details of supported expression syntax.

  ## Interactive demo

  You can try this plugin out at [datasette-jq-demo.datasette.io](https://datasette-jq-demo.datasette.io/)

  Sample query:

      select package, "https://pypi.org/project/" || package || "/" as url,
      jq(info, "{summary: .info.summary, author: .info.author, versions: .releases|keys|reverse}")
      from packages

  [Try this query out](https://datasette-jq-demo.datasette.io/demo?sql=select+package%2C+%22https%3A%2F%2Fpypi.org%2Fproject%2F%22+%7C%7C+package+%7C%7C+%22%2F%22+as+url%2C%0D%0Ajq%28info%2C+%22%7Bsummary%3A+.info.summary%2C+author%3A+.info.author%2C+versions%3A+.releases%7Ckeys%7Creverse%7D%22%29%0D%0Afrom+packages) in the interactive demo.


name: datasette-json-html
summary: Datasette plugin for rendering HTML based on JSON values
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://datasette.io/plugins/datasette-json-html
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-json-html/
project_url: https://pypi.org/project/datasette-json-html/
release_url: https://pypi.org/project/datasette-json-html/1.0.1/
version: 1.0.1
description:
  # datasette-json-html

  [![PyPI](https://img.shields.io/pypi/v/datasette-json-html.svg)](https://pypi.org/project/datasette-json-html/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-json-html?include_prereleases&label=changelog)](https://github.com/simonw/datasette-json-html/releases)
  [![Tests](https://github.com/simonw/datasette-json-html/workflows/Test/badge.svg)](https://github.com/simonw/datasette-remote-metadata/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-json-html/blob/main/LICENSE)

  Datasette plugin for rendering HTML based on JSON values, using the [render_cell plugin hook](https://docs.datasette.io/en/stable/plugin_hooks.html#render-cell-value-column-table-database-datasette).

  This plugin looks for cell values that match a very specific JSON format and converts them into HTML when they are rendered by the Datasette interface.

  ## Links

      {
          "href": "https://simonwillison.net/",
          "label": "Simon Willison"
      }

  Will be rendered as an `<a href="">` link:

      <a href="https://simonwillison.net/">Simon Willison</a>

  You can set a tooltip on the link using a `"title"` key:


      {
          "href": "https://simonwillison.net/",
          "label": "Simon Willison",
          "title": "My blog"
      }

  Produces:

      <a href="https://simonwillison.net/" title="My blog">Simon Willison</a>

  You can also include a description, which will be displayed below the link. If descriptions include newlines they will be converted to `<br>` elements:

      select json_object(
          "href", "https://simonwillison.net/",
          "label", "Simon Willison",
          "description", "This can contain" || x'0a' || "newlines"
      )

  Produces:

      <strong><a href="https://simonwillison.net/">Simon Willison</a></strong><br>This can contain<br>newlines

  * [Literal JSON link demo](https://datasette-json-html.datasette.io/demo?sql=select+%27%7B%0D%0A++++%22href%22%3A+%22https%3A%2F%2Fsimonwillison.net%2F%22%2C%0D%0A++++%22label%22%3A+%22Simon+Willison%22%2C%0D%0A++++%22title%22%3A+%22My+blog%22%0D%0A%7D%27)

  ## List of links

      [
          {
              "href": "https://simonwillison.net/",
              "label": "Simon Willison"
          },
          {
              "href": "https://github.com/simonw/datasette",
              "label": "Datasette"
          }
      ]

  Will be rendered as a comma-separated list of `<a href="">` links:

      <a href="https://simonwillison.net/">Simon Willison</a>,
      <a href="https://github.com/simonw/datasette">Datasette</a>

  The `href` property must begin with `https://` or `http://` or `/`, to avoid potential XSS injection attacks (for example URLs that begin with `javascript:`).

  Lists of links cannot include `"description"` keys.

  * [Literal list of links demo](https://datasette-json-html.datasette.io/demo?sql=select+%27%5B%0D%0A++++%7B%0D%0A++++++++%22href%22%3A+%22https%3A%2F%2Fsimonwillison.net%2F%22%2C%0D%0A++++++++%22label%22%3A+%22Simon+Willison%22%0D%0A++++%7D%2C%0D%0A++++%7B%0D%0A++++++++%22href%22%3A+%22https%3A%2F%2Fgithub.com%2Fsimonw%2Fdatasette%22%2C%0D%0A++++++++%22label%22%3A+%22Datasette%22%0D%0A++++%7D%0D%0A%5D%27)

  ## Images

  The image tag is more complex. The most basic version looks like this:

      {
          "img_src": "https://placekitten.com/200/300"
      }

  This will render as:

      <img src="https://placekitten.com/200/300">

  But you can also include one or more of `alt`, `caption`, `width` and `href`.

  If you include width or alt, they will be added as attributes:

      {
          "img_src": "https://placekitten.com/200/300",
          "alt": "Kitten",
          "width": 200
      }

  Produces:

      <img src="https://placekitten.com/200/300"
          alt="Kitten" width="200">

  * [Literal image demo](https://datasette-json-html.datasette.io/demo?sql=select+%27%7B%0D%0A++++%22img_src%22%3A+%22https%3A%2F%2Fplacekitten.com%2F200%2F300%22%2C%0D%0A++++%22alt%22%3A+%22Kitten%22%2C%0D%0A++++%22width%22%3A+200%0D%0A%7D%27)

  The `href` key will cause the image to be wrapped in a link:

      {
          "img_src": "https://placekitten.com/200/300",
          "href": "http://www.example.com"
      }

  Produces:

      <a href="http://www.example.com">
          <img src="https://placekitten.com/200/300">
      </a>

  The `caption` key wraps everything in a fancy figure/figcaption block:

      {
          "img_src": "https://placekitten.com/200/300",
          "caption": "Kitten caption"
      }

  Produces:

      <figure>
          <img src="https://placekitten.com/200/300"></a>
          <figcaption>Kitten caption</figcaption>
      </figure>

  ## Preformatted text

  You can use `{"pre": "text"}` to render text in a `<pre>` HTML tag:

      {
          "pre": "This\nhas\nnewlines"
      }

  Produces:

      <pre>This
      has
      newlines</pre>

  If the value attached to the `"pre"` key is itself a JSON object, that JSON will be pretty-printed:

      {
          "pre": {
              "this": {
                  "object": ["is", "nested"]
              }
          }
      }

  Produces:

      <pre>{
        &#34;this&#34;: {
          &#34;object&#34;: [
            &#34;is&#34;,
            &#34;nested&#34;
          ]
        }
      }</pre>

  * [Preformatted text with JSON demo](https://datasette-json-html.datasette.io/demo?sql=select+%27%7B%0D%0A++++%22pre%22%3A+%7B%0D%0A++++++++%22this%22%3A+%7B%0D%0A++++++++++++%22object%22%3A+%5B%22is%22%2C+%22nested%22%5D%0D%0A++++++++%7D%0D%0A++++%7D%0D%0A%7D%27)
  * [Preformatted text demo showing the Mandelbrot Set](https://datasette-json-html.datasette.io/demo?sql=WITH+RECURSIVE%0D%0A++xaxis%28x%29+AS+%28VALUES%28-2.0%29+UNION+ALL+SELECT+x%2B0.05+FROM+xaxis+WHERE+x%3C1.2%29%2C%0D%0A++yaxis%28y%29+AS+%28VALUES%28-1.0%29+UNION+ALL+SELECT+y%2B0.1+FROM+yaxis+WHERE+y%3C1.0%29%2C%0D%0A++m%28iter%2C+cx%2C+cy%2C+x%2C+y%29+AS+%28%0D%0A++++SELECT+0%2C+x%2C+y%2C+0.0%2C+0.0+FROM+xaxis%2C+yaxis%0D%0A++++UNION+ALL%0D%0A++++SELECT+iter%2B1%2C+cx%2C+cy%2C+x*x-y*y+%2B+cx%2C+2.0*x*y+%2B+cy+FROM+m+%0D%0A+++++WHERE+%28x*x+%2B+y*y%29+%3C+4.0+AND+iter%3C28%0D%0A++%29%2C%0D%0A++m2%28iter%2C+cx%2C+cy%29+AS+%28%0D%0A++++SELECT+max%28iter%29%2C+cx%2C+cy+FROM+m+GROUP+BY+cx%2C+cy%0D%0A++%29%2C%0D%0A++a%28t%29+AS+%28%0D%0A++++SELECT+group_concat%28+substr%28%27+.%2B*%23%27%2C+1%2Bmin%28iter%2F7%2C4%29%2C+1%29%2C+%27%27%29+%0D%0A++++FROM+m2+GROUP+BY+cy%0D%0A++%29%0D%0ASELECT+json_object%28%27pre%27%2C+group_concat%28rtrim%28t%29%2Cx%270a%27%29%29+FROM+a%3B) using [this example](https://www.sqlite.org/lang_with.html#outlandish_recursive_query_examples) from the SQLite documentation

  ## Using these with SQLite JSON functions

  The most powerful way to make use of this plugin is in conjunction with SQLite's [JSON functions](https://www.sqlite.org/json1.html). For example:

      select json_object(
          "href", "https://simonwillison.net/",
          "label", "Simon Willison"
      );

  * [json_object() link demo](https://datasette-json-html.datasette.io/demo?sql=select+json_object%28%0D%0A++++%22href%22%2C+%22https%3A%2F%2Fsimonwillison.net%2F%22%2C%0D%0A++++%22label%22%2C+%22Simon+Willison%22%0D%0A%29%3B)

  You can use these functions to construct JSON objects that work with the plugin from data in a table:

      select id, json_object(
          "href", url, "label", text
      ) from mytable;

  * [Demo that builds links against a table](https://datasette-json-html.datasette.io/demo?sql=select+json_object%28%22href%22%2C+url%2C+%22label%22%2C+package%2C+%22title%22%2C+package+%7C%7C+%22+%22+%7C%7C+url%29+as+package+from+packages)

  The `json_group_array()` function is an aggregate function similar to `group_concat()` - it allows you to construct lists of JSON objects in conjunction with a `GROUP BY` clause.

  This means you can use it to construct dynamic lists of links, for example:

      select
          substr(package, 0, 12) as prefix,
          json_group_array(
              json_object(
                  "href", url,
                  "label", package
              )
          ) as package_links
      from packages
      group by prefix

  * [Demo of json_group_array()](https://datasette-json-html.datasette.io/demo?sql=select%0D%0A++++substr%28package%2C+0%2C+12%29+as+prefix%2C%0D%0A++++json_group_array%28%0D%0A++++++++json_object%28%0D%0A++++++++++++%22href%22%2C+url%2C%0D%0A++++++++++++%22label%22%2C+package%0D%0A++++++++%29%0D%0A++++%29+as+package_links%0D%0Afrom+packages%0D%0Agroup+by+prefix)

  ## The `urllib_quote_plus()` SQL function

  Since this plugin is designed to be used with SQL that constructs the underlying JSON structure, it is likely you will need to construct dynamic URLs from results returned by a SQL query.

  This plugin registers a custom SQLite function called `urllib_quote_plus()` to help you do that. It lets you use Python's [urllib.parse.quote\_plus() function](https://docs.python.org/3/library/urllib.parse.html#urllib.parse.quote_plus) from within a SQL query.

  Here's an example of how you might use it:

      select id, json_object(
          "href",
          "/mydatabase/other_table?_search=" || urllib_quote_plus(text),
          "label", text
      ) from mytable;

project_urls:
  {"CI": "https://github.com/simonw/datasette-json-html/actions", "Changelog": "https://github.com/simonw/datasette-json-html/releases", "Homepage": "https://datasette.io/plugins/datasette-json-html", "Issues": "https://github.com/simonw/datasette-json-html/issues"}

requires_dist:
  ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'", "httpx ; extra == 'test'"]


name: datasette-jupyterlite
summary: JupyterLite as a Datasette plugin
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-jupyterlite
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-jupyterlite/
project_url: https://pypi.org/project/datasette-jupyterlite/
release_url: https://pypi.org/project/datasette-jupyterlite/0.1a1/
requires_python: >=3.7
version: 0.1a1
description:
  # datasette-jupyterlite

  [![PyPI](https://img.shields.io/pypi/v/datasette-jupyterlite.svg)](https://pypi.org/project/datasette-jupyterlite/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-jupyterlite?include_prereleases&label=changelog)](https://github.com/simonw/datasette-jupyterlite/releases)
  [![Tests](https://github.com/simonw/datasette-jupyterlite/workflows/Test/badge.svg)](https://github.com/simonw/datasette-jupyterlite/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-jupyterlite/blob/main/LICENSE)

  [JupyterLite](https://jupyterlite.readthedocs.io/en/latest/) as a Datasette plugin

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-jupyterlite

  ## Demo

  You can try out a demo of the plugin here: https://latest-with-plugins.datasette.io/jupyterlite/

  Run this example code in a Pyolite notebook to pull all of the data from the [github/stars](https://latest-with-plugins.datasette.io/github/stars) table into a Pandas DataFrame:

  ```python
  import pandas, pyodide

  df = pandas.read_csv(pyodide.open_url(
    "https://latest-with-plugins.datasette.io/github/stars.csv?_labels=on&_stream=on&_size=max")
  )
  ```

  ## Usage

  Once installed, visit `/jupyterlite/` to access JupyterLite served from your Datasette instance.

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-jupyterlite
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and test dependencies:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-jupyterlite/actions", "Changelog": "https://github.com/simonw/datasette-jupyterlite/releases", "Homepage": "https://github.com/simonw/datasette-jupyterlite", "Issues": "https://github.com/simonw/datasette-jupyterlite/issues"}

requires_dist:
  ["datasette", "jupyterlite", "importlib-resources", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]


name: datasette-leaflet-freedraw
summary: Draw polygons on maps in Datasette
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-leaflet-freedraw
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-leaflet-freedraw/
project_url: https://pypi.org/project/datasette-leaflet-freedraw/
release_url: https://pypi.org/project/datasette-leaflet-freedraw/0.3.1/
requires_python: >=3.6
version: 0.3.1
description:
  # datasette-leaflet-freedraw

  [![PyPI](https://img.shields.io/pypi/v/datasette-leaflet-freedraw.svg)](https://pypi.org/project/datasette-leaflet-freedraw/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-leaflet-freedraw?include_prereleases&label=changelog)](https://github.com/simonw/datasette-leaflet-freedraw/releases)
  [![Tests](https://github.com/simonw/datasette-leaflet-freedraw/workflows/Test/badge.svg)](https://github.com/simonw/datasette-leaflet-freedraw/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-leaflet-freedraw/blob/main/LICENSE)

  Draw polygons on maps in Datasette

  Project background: [Drawing shapes on a map to query a SpatiaLite database](https://simonwillison.net/2021/Jan/24/drawing-shapes-spatialite/).

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-leaflet-freedraw

  ## Usage

  If a table has a SpatiaLite `geometry` column, the plugin will add a map interface to the table page allowing users to draw a shape on the map to find rows with a geometry that intersects that shape.

  The plugin can also work with arbitrary SQL queries. There it looks for input fields with a name of `freedraw` or that ends in `_freedraw` and replaces them with a map interface.

  The map interface uses the [FreeDraw](https://freedraw.herokuapp.com/) Leaflet plugin.

  ## Demo

  You can try out this plugin to run searches against the GreenInfo Network California Protected Areas Database. Here's [an example query](https://calands.datasettes.com/calands?sql=select%0D%0A++AsGeoJSON%28geometry%29%2C+*%0D%0Afrom%0D%0A++CPAD_2020a_SuperUnits%0D%0Awhere%0D%0A++PARK_NAME+like+%27%25mini%25%27+and%0D%0A++Intersects%28GeomFromGeoJSON%28%3Afreedraw%29%2C+geometry%29+%3D+1%0D%0A++and+CPAD_2020a_SuperUnits.rowid+in+%28%0D%0A++++select%0D%0A++++++rowid%0D%0A++++from%0D%0A++++++SpatialIndex%0D%0A++++where%0D%0A++++++f_table_name+%3D+%27CPAD_2020a_SuperUnits%27%0D%0A++++++and+search_frame+%3D+GeomFromGeoJSON%28%3Afreedraw%29%0D%0A++%29&freedraw=%7B%22type%22%3A%22MultiPolygon%22%2C%22coordinates%22%3A%5B%5B%5B%5B-122.42202758789064%2C37.82280243352759%5D%2C%5B-122.39868164062501%2C37.823887203271454%5D%2C%5B-122.38220214843751%2C37.81846319511331%5D%2C%5B-122.35061645507814%2C37.77071473849611%5D%2C%5B-122.34924316406251%2C37.74465712069939%5D%2C%5B-122.37258911132814%2C37.703380457832374%5D%2C%5B-122.39044189453125%2C37.690340943717715%5D%2C%5B-122.41241455078126%2C37.680559803205135%5D%2C%5B-122.44262695312501%2C37.67295135774715%5D%2C%5B-122.47283935546876%2C37.67295135774715%5D%2C%5B-122.52502441406251%2C37.68382032669382%5D%2C%5B-122.53463745117189%2C37.6892542140253%5D%2C%5B-122.54699707031251%2C37.690340943717715%5D%2C%5B-122.55798339843751%2C37.72945260537781%5D%2C%5B-122.54287719726564%2C37.77831314799672%5D%2C%5B-122.49893188476564%2C37.81303878836991%5D%2C%5B-122.46185302734376%2C37.82822612280363%5D%2C%5B-122.42889404296876%2C37.82822612280363%5D%2C%5B-122.42202758789064%2C37.82280243352759%5D%5D%5D%5D%7D) showing mini parks in San Francisco:

  ```sql
  select
    AsGeoJSON(geometry), *
  from
    CPAD_2020a_SuperUnits
  where
    PARK_NAME like '%mini%' and
    Intersects(GeomFromGeoJSON(:freedraw), geometry) = 1
    and CPAD_2020a_SuperUnits.rowid in (
      select
        rowid
      from
        SpatialIndex
      where
        f_table_name = 'CPAD_2020a_SuperUnits'
        and search_frame = GeomFromGeoJSON(:freedraw)
    )
  ```

  ![Screenshot of the plugin in action](https://static.simonwillison.net/static/2021/datasette-leaflet-freedraw.png)

  ## Development

  To set up this plugin locally, first checkout the code. Then create a new virtual environment:

      cd datasette-leaflet-freedraw
      python3 -mvenv venv
      source venv/bin/activate

  Or if you are using `pipenv`:

      pipenv shell

  Now install the dependencies and tests:

      pip install -e '.[test]'

  To run the tests:

      pytest

project_urls:
  {"CI": "https://github.com/simonw/datasette-leaflet-freedraw/actions", "Changelog": "https://github.com/simonw/datasette-leaflet-freedraw/releases", "Homepage": "https://github.com/simonw/datasette-leaflet-freedraw", "Issues": "https://github.com/simonw/datasette-leaflet-freedraw/issues"}

requires_dist:
  ["datasette (>=0.60)", "datasette-leaflet (>=0.2)", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]


name: datasette-leaflet-geojson
summary: A Datasette plugin that renders GeoJSON columns using Leaflet
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-leaflet-geojson
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-leaflet-geojson/
project_url: https://pypi.org/project/datasette-leaflet-geojson/
project_urls: {"Homepage": "https://github.com/simonw/datasette-leaflet-geojson"}
release_url: https://pypi.org/project/datasette-leaflet-geojson/0.8/
version: 0.8
description:
  # datasette-leaflet-geojson

  [![PyPI](https://img.shields.io/pypi/v/datasette-leaflet-geojson.svg)](https://pypi.org/project/datasette-leaflet-geojson/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-leaflet-geojson?include_prereleases&label=changelog)](https://github.com/simonw/datasette-leaflet-geojson/releases)
  [![Tests](https://github.com/simonw/datasette-leaflet-geojson/workflows/Test/badge.svg)](https://github.com/simonw/datasette-leaflet-geojson/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-leaflet-geojson/blob/main/LICENSE)

  Datasette plugin that replaces any GeoJSON column values with a Leaflet map

  ## Installation

  Install this plugin in the same environment as Datasette.

      $ datasette install datasette-leaflet-geojson

  ## Usage

  Any columns containing valid GeoJSON strings will have their contents replaced with a Leaflet map when they are displayed in the Datasette interface.

  ## Demo

  You can try this plugin out at https://calands.datasettes.com/calands/superunits_with_maps?MNG_AGENCY=Palo+Alto%2C+City+of

  ![datasette-leaflet-geojson in action](https://raw.github.com/simonw/datasette-leaflet-geojson/main/datasette-leaflet-geojson.png)

  ## Configuration

  By default this plugin displays maps for the first ten rows, and shows a "Click to load map" prompt for rows past the first ten.

  You can change this limit using the `default_maps_to_load` plugin configuration setting. Add this to your `metadata.json`:

  ```json
  {
      "plugins": {
          "datasette-leaflet-geojson": {
              "default_maps_to_load": 20
          }
      }
  }
  ```
  Then run Datasette with `datasette mydb.db -m metadata.json`.

requires_dist:
  ["datasette (>=0.54)", "datasette-leaflet (>=0.2)", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]


name: datasette-leaflet
summary: A plugin that bundles Leaflet.js for Datasette
classifiers: []
author: Simon Willison
description_content_type: text/markdown
home_page: https://github.com/simonw/datasette-leaflet
license: Apache License, Version 2.0
package_url: https://pypi.org/project/datasette-leaflet/
project_url: https://pypi.org/project/datasette-leaflet/
release_url: https://pypi.org/project/datasette-leaflet/0.2.2/
requires_dist: ["datasette", "pytest ; extra == 'test'", "pytest-asyncio ; extra == 'test'"]
requires_python: >=3.6
version: 0.2.2
description:
  # datasette-leaflet

  [![PyPI](https://img.shields.io/pypi/v/datasette-leaflet.svg)](https://pypi.org/project/datasette-leaflet/)
  [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-leaflet?include_prereleases&label=changelog)](https://github.com/simonw/datasette-leaflet/releases)
  [![Tests](https://github.com/simonw/datasette-leaflet/workflows/Test/badge.svg)](https://github.com/simonw/datasette-leaflet/actions?query=workflow%3ATest)
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-leaflet/blob/main/LICENSE)

  Datasette plugin adding the [Leaflet](https://leafletjs.com/) JavaScript library.

  A growing number of Datasette plugins depend on the Leaflet JavaScript mapping library. They each have their own way of loading Leaflet, which could result in loading it multiple times (with multiple versions) if more than one plugin is installed.

  This library is intended to solve this problem, by providing a single plugin they can all depend on that loads Leaflet in a reusable way.

  Plugins that use this:

  - [datasette-leaflet-freedraw](https://datasette.io/plugins/datasette-leaflet-freedraw)
  - [datasette-leaflet-geojson](https://datasette.io/plugins/datasette-leaflet-geojson)
  - [datasette-cluster-map](https://datasette.io/plugins/datasette-cluster-map)

  ## Installation

  You can install this plugin like so:

      datasette install datasette-leaflet

  Usually this plugin will be a dependency of other plugins, so it should be installed automatically when you install them.

  ## Usage

  The plugin makes `leaflet.js` and `leaflet.css` available as static files. It provides two custom template variables with the URLs of those two files.

  - `{{ datasette_leaflet_url }}` is the URL to the JavaScript
  - `{{ datasette_leaflet_css_url }}` is the URL to the CSS

  These URLs are also made available as global JavaScript constants:

  - `datasette.leaflet.JAVASCRIPT_URL`
  - `datasette.leaflet.CSS_URL`

  The JavaScript is packaed as a [JavaScript module](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules). You can dynamically import the JavaScript from a custom template like this:

  ```html+jinja
  <script type="module">
  import('{{ datasette_leaflet_url }}')
    .then((leaflet) => {
      /* Use leaflet here */
    });
  </script>
  ```

  You can load the CSS like this:

  ```html+jinja
  <link rel="stylesheet" href="{{ datasette_leaflet_css_url }}">
  ```

  Or dynamically like this:

  ```html+jinja
  <script>
  let link = document.createElement('link');
  link.rel = 'stylesheet';
  link.href = '{{ datasette_leaflet_css_url }}';
  document.head.appendChild(link);
  </script>
  ```

  Here's a full example that loads the JavaScript and CSS and renders a map:

  ```html+jinja
  <script type="module">
  window.DATASETTE_CLUSTER_MAP_TILE_LAYER = "https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png";
  window.DATASETTE_CLUSTER_MAP_TILE_LAYER_OPTIONS = {"maxZoom": 19, "detectRetina": true, "attribution": "&copy; <a href=\"https://www.openstreetmap.org/copyright\">OpenStreetMap</a> contributors"};
  let link = document.createElement('link');
  link.rel = 'stylesheet';
  link.href = '{{ datasette_leaflet_css_url }}';
  document.head.appendChild(link);
  import('{{ datasette_leaflet_url }}')
    .then((leaflet) => {
      let div = document.createElement('div');
      div.style.height = '400px';
      document.querySelector('.content').appendChild(div);
      let tiles = leaflet.tileLayer(
          window.DATASETTE_CLUSTER_MAP_TILE_LAYER,
          window.DATASETTE_CLUSTER_MAP_TILE_LAYER_OPTIONS
      );
      let map = leaflet.map(div, {
          center: leaflet.latLng(0, 0),
          zoom: 2,
          layers: [tiles]
      });
    });
  </script>
  ```

project_urls:
  {"CI": "https://github.com/simonw/datasette-leaflet/actions", "Changelog": "https://github.com/simonw/datasette-leaflet/releases", "Homepage": "https://github.com/simonw/datasette-leaflet", "Issues": "https://github.com/simonw/datasette-leaflet/issues"}


