uses
8 rows
This data as json, CSV (advanced)
| _id ▼ | _path | text | html | title | summary | order |
|---|---|---|---|---|---|---|
| 0574db8535765cbe166676a6b73fd0205331313b | for/rapid-prototyping.md | Want to build a prototype? Datasette lets you turn any source of data into a powerful JSON or [GraphQL](https://github.com/simonw/datasette-graphql) API, ideal for use as the backend for an interactive prototype. Use Datasette with **[Observable notebooks](https://observablehq.com/)** to quickly try out new visualizations and explore the Datasette API using JavaScript. Datasette works as a great data for **[Jupyter notebooks](https://jupyter.org/)** too. - **[Analyzing ScotRail audio announcements with Datasette—from prototype to production](https://simonwillison.net/2022/Aug/21/scotrail/)** talks through the full process of creating an initial rapid prototype using Datasette Lite and growing that into a full production Datasette instance. - **[Fast Autocomplete Search for Your Website](https://simonwillison.net/2018/Dec/19/fast-autocomplete-search/)** shows how to build a full autocomplete search engine for your website, powered by Datasette and a few dozen lines of custom JavaScript. | <p>Want to build a prototype? Datasette lets you turn any source of data into a powerful JSON or <a href="https://github.com/simonw/datasette-graphql">GraphQL</a> API, ideal for use as the backend for an interactive prototype.</p> <p>Use Datasette with <strong><a href="https://observablehq.com/">Observable notebooks</a></strong> to quickly try out new visualizations and explore the Datasette API using JavaScript.</p> <p>Datasette works as a great data for <strong><a href="https://jupyter.org/">Jupyter notebooks</a></strong> too.</p> <ul> <li><strong><a href="https://simonwillison.net/2022/Aug/21/scotrail/">Analyzing ScotRail audio announcements with Datasette—from prototype to production</a></strong> talks through the full process of creating an initial rapid prototype using Datasette Lite and growing that into a full production Datasette instance.</li> <li><strong><a href="https://simonwillison.net/2018/Dec/19/fast-autocomplete-search/">Fast Autocomplete Search for Your Website</a></strong> shows how to build a full autocomplete search engine for your website, powered by Datasette and a few dozen lines of custom JavaScript.</li> </ul> | Datasette for rapid prototyping | Spin up a JSON (or GraphQL) API for any data in minutes. Use it to prototype and prove your ideas without building a custom backend. | 7 |
| 1940a8c4df2a2e0e7c95a457ce8dc4e7033c24d9 | for/publishing-data.md | Datasette was [originally inspired](https://simonwillison.net/2018/Aug/19/instantly-publish-datasette/) by work on [the Guardian Data Blog](https://www.theguardian.com/news/datablog/2011/jan/27/data-store-office-for-national-statistics), an initiative to publish the raw data behind the stories in the news. Datasette encourages sharing data as tables that can be explored by users and exported out as JSON or CSV. It's like publishing a CSV file but with much deeper flexibility for visitors to remix, visualize and export data. Data published by Datasette can be deployed to both traditional and serverless hosting platforms. The `datasette publish` command, [documented here](https://docs.datasette.io/en/stable/publish.html), can deploy to [Heroku](https://heroku.com/) and [Google Cloud Run](https://cloud.google.com/run). Plugins can be used to deploy Datasette on [Vercel](https://vercel.com/) or [fly.io](https://fly.io/). Datasette also works great on [Glitch](https://glitch.com/). When combined with GitHub Actions, Datasette can be used to automatically deploy new databases on a scheduled basis. See [Deploying a data API using GitHub Actions and Cloud Run](https://simonwillison.net/2020/Jan/21/github-actions-cloud-run/) for more details on implementing this pattern. | <p>Datasette was <a href="https://simonwillison.net/2018/Aug/19/instantly-publish-datasette/">originally inspired</a> by work on <a href="https://www.theguardian.com/news/datablog/2011/jan/27/data-store-office-for-national-statistics">the Guardian Data Blog</a>, an initiative to publish the raw data behind the stories in the news.</p> <p>Datasette encourages sharing data as tables that can be explored by users and exported out as JSON or CSV. It's like publishing a CSV file but with much deeper flexibility for visitors to remix, visualize and export data.</p> <p>Data published by Datasette can be deployed to both traditional and serverless hosting platforms.</p> <p>The <code>datasette publish</code> command, <a href="https://docs.datasette.io/en/stable/publish.html">documented here</a>, can deploy to <a href="https://heroku.com/">Heroku</a> and <a href="https://cloud.google.com/run">Google Cloud Run</a>. Plugins can be used to deploy Datasette on <a href="https://vercel.com/">Vercel</a> or <a href="https://fly.io/">fly.io</a>.</p> <p>Datasette also works great on <a href="https://glitch.com/">Glitch</a>.</p> <p>When combined with GitHub Actions, Datasette can be used to automatically deploy new databases on a scheduled basis. See <a href="https://simonwillison.net/2020/Jan/21/github-actions-cloud-run/">Deploying a data API using GitHub Actions and Cloud Run</a> for more details on implementing this pattern.</p> | Datasette for publishing data | The built-in "datasette publish" command lets you instantly publish your data to hosting providers like Google Cloud Run, Heroku or Vercel. | 2 |
| 2a478757dc7f379d19b6c77d1710024b8f02a1e1 | for/serverless.md | Serverless providers such as [Google Cloud Run](https://cloud.google.com/run) and [Vercel](https://vercel.com/) provide robust, scalable, inexpensive hosting for stateless web applications. Databases are usually provided as separate add-ons to serverless hosting. If your data is static or changes infrequently, Datasette can be used to unlock the benefits of serverless hosting without needing a separate hosted database. Datasette can be used to implement the [<strong>Baked Data</strong>](https://simonwillison.net/2021/Jul/28/baked-data/) pattern, where any structured data needed by your application is bundled and deployed with the application code itself, as part of the same container. This site is an example of Baked Data in action: datasette.io is deployed to Google Cloud Run by build scripts running [on GitHub Actions](https://github.com/simonw/datasette.io/blob/main/.github/workflows/deploy.yml). Many of the [Datasette examples](/examples) listed on this site are also deployed in this manner. | <p>Serverless providers such as <a href="https://cloud.google.com/run">Google Cloud Run</a> and <a href="https://vercel.com/">Vercel</a> provide robust, scalable, inexpensive hosting for stateless web applications.</p> <p>Databases are usually provided as separate add-ons to serverless hosting. If your data is static or changes infrequently, Datasette can be used to unlock the benefits of serverless hosting without needing a separate hosted database.</p> <p>Datasette can be used to implement the <a href="https://simonwillison.net/2021/Jul/28/baked-data/"><strong>Baked Data</strong></a> pattern, where any structured data needed by your application is bundled and deployed with the application code itself, as part of the same container.</p> <p>This site is an example of Baked Data in action: datasette.io is deployed to Google Cloud Run by build scripts running <a href="https://github.com/simonw/datasette.io/blob/main/.github/workflows/deploy.yml">on GitHub Actions</a>.</p> <p>Many of the <a href="/examples">Datasette examples</a> listed on this site are also deployed in this manner.</p> | Serverless read-only databases with Datasette | Serverless hosting usually depends on external hosted databases, but Datasette lets you implement the Baked Data pattern to bundle your data with your application code and deploy it to serverless providers. | 8 |
| 41624e65228293a1622b9eee3a435699801dead1 | for/websites.md | Datasette templates can be used to build custom websites with Datasette under the hood. Examples include the site you are browsing right now. ## [datasette.io](https://datasette.io/) The official Datasette website runs on top of a customized Datasette instance. - [datasette.io, an official project website for Datasette](https://simonwillison.net/2020/Dec/13/datasette-io/) talks through the basics of how the site works. - [Building a search engine for datasette.io](https://simonwillison.net/2020/Dec/19/dogsheep-beta/) explains how the [site search](https://datasette.io/-/beta) works, using the [dogsheep-beta](https://datasette.io/plugins/dogsheep-beta) search plugin. - [The Baked Data architectural pattern](https://simonwillison.net/2021/Jul/28/baked-data/) describes the Baked Data architecture pattern used by the site, and provides a detailed description of how the build scripts for the site work. - [github.com/simonw/datasette.io](https://github.com/simonw/datasette.io) has the full source code for the site, including the GitHub Actions workflows that build the databases and deploy them to Google Cloud Run. - [datasette.io/content](https://datasette.io/content) is the Datasette interface for the main database that powers the site. ## [til.simonwillison.net](https://til.simonwillison.net/) A blog sharing TILs - Today I Learned snippets. - [Using a self-rewriting README powered by GitHub Actions to track TILs](https://simonwillison.net/2020/Apr/20/self-rewriting-readme/) introduces the site. - The site uses the [datasette-atom](https://datasette.io/plugins/datasette-atom) plugin to provide an Atom feed of new entries. - [til.simonwillison.net/tils](https://til.simonwillison.net/tils) exposes the underlying database. - The site is deployed to [Vercel](https://vercel.com/) using the [datasette-publish-vercel](https://datasette.io/plugins/datasette-publish-vercel) plugin. ## [Niche Museums](https://www.niche-museums.com/) [www.niche-museums.com](https://www.niche-museums.com/) is a directory of small and nich… | <p>Datasette templates can be used to build custom websites with Datasette under the hood. Examples include the site you are browsing right now.</p> <h2><a href="https://datasette.io/">datasette.io</a></h2> <p>The official Datasette website runs on top of a customized Datasette instance.</p> <ul> <li><a href="https://simonwillison.net/2020/Dec/13/datasette-io/">datasette.io, an official project website for Datasette</a> talks through the basics of how the site works.</li> <li><a href="https://simonwillison.net/2020/Dec/19/dogsheep-beta/">Building a search engine for datasette.io</a> explains how the <a href="https://datasette.io/-/beta">site search</a> works, using the <a href="https://datasette.io/plugins/dogsheep-beta">dogsheep-beta</a> search plugin.</li> <li><a href="https://simonwillison.net/2021/Jul/28/baked-data/">The Baked Data architectural pattern</a> describes the Baked Data architecture pattern used by the site, and provides a detailed description of how the build scripts for the site work.</li> <li><a href="https://github.com/simonw/datasette.io">github.com/simonw/datasette.io</a> has the full source code for the site, including the GitHub Actions workflows that build the databases and deploy them to Google Cloud Run.</li> <li><a href="https://datasette.io/content">datasette.io/content</a> is the Datasette interface for the main database that powers the site.</li> </ul> <h2><a href="https://til.simonwillison.net/">til.simonwillison.net</a></h2> <p>A blog sharing TILs - Today I Learned snippets.</p> <ul> <li><a href="https://simonwillison.net/2020/Apr/20/self-rewriting-readme/">Using a self-rewriting README powered by GitHub Actions to track TILs</a> introduces the site.</li> <li>The site uses the <a href="https://datasette.io/plugins/datasette-atom">datasette-atom</a> plugin to provide an Atom feed of new entries.</li> <li><a href="https://til.simonwillison.net/tils">til.simonwillison.net/tils</a> exposes the underlying database.</li> <li>The site is deployed to <a href="https://vercel.com/">Vercel</… | Datasette for websites | Datasette can be used to power dynamic-static websites: sites that run on serverless hosting while providing dynamic data-backed functionality. | 3 |
| 61a01ca9a0e3de0a0f0af9691661af44ea6e149d | for/data-journalism.md | Datasette was originally designed as a tool for data journalists, to help report on data-driven stories, crunch through large datasets and [publish the results](/for/publishing-data). [This demo](https://vimeo.com/436903714) from [SRCCON 2020](https://2020.srccon.org/) shows how Datasette can be used to analyze PPP loan data released by the Small Business Administration. <iframe style="max-width: 100%" src="https://player.vimeo.com/video/436903714" width="640" height="400" frameborder="0" allow="autoplay; fullscreen" allowfullscreen></iframe> You can try the demo [here](https://sba-loans-covid-19.datasettes.com/), or read more about it in [this article](https://simonwillison.net/2020/Jul/9/sba-covid-19-ppp-loans/). | <p>Datasette was originally designed as a tool for data journalists, to help report on data-driven stories, crunch through large datasets and <a href="/for/publishing-data">publish the results</a>.</p> <p><a href="https://vimeo.com/436903714">This demo</a> from <a href="https://2020.srccon.org/">SRCCON 2020</a> shows how Datasette can be used to analyze PPP loan data released by the Small Business Administration. </p> <iframe style="max-width: 100%" src="https://player.vimeo.com/video/436903714" width="640" height="400" frameborder="0" allow="autoplay; fullscreen" allowfullscreen></iframe> <p>You can try the demo <a href="https://sba-loans-covid-19.datasettes.com/">here</a>, or read more about it in <a href="https://simonwillison.net/2020/Jul/9/sba-covid-19-ppp-loans/">this article</a>.</p> | Datasette for data journalism | Datasette was originally designed as a tool for data journalists, to help report on data-driven stories, crunch through large datasets and publish the results. | 4 |
| 863b689b3625af13bbba9bb9077c997fe3a04d13 | for/search.md | Datasette is built on top of [SQLite](https://sqlite.org/), which includes a robust, full-featured [full-text search](https://www.sqlite.org/fts5.html) implementation. Datasette automatically detects tables which have been configured for full-text search and adds a search box which can be used by human site users or accessed from the JSON or CSV APIs. You can try this out now by [searching FARA records](https://fara.datasettes.com/) or [global power plants](https://global-power-plants.datasettes.com/global-power-plants/global-power-plants). ## Enabling search for a table The [sqlite-utils command-line tool](https://sqlite-utils.datasette.io/en/stable/cli.html#configuring-full-text-search) can be used to enable full-text search on a specific set of columns for a table: % sqlite-utils enable-fts mydb.db documents title summary Alternatively, you can configure search directly within the Datasette interface by installing the [datasette-configure-fts](https://github.com/simonw/datasette-configure-fts) plugin. [Fast Autocomplete Search for Your Website](https://simonwillison.net/2018/Dec/19/fast-autocomplete-search/) is a tutorial that shows how to write a scraper that populates a full-text search index in Datasette, then build a JavaScript autocomplete search interface using the Datasette JSON API. ## Implementing search with a canned query Datasette [canned queries](https://docs.datasette.io/en/stable/sql_queries.html#canned-queries) can be used to configure a SQL query which returns search results for a table, ordered by relevance. The search on [www.niche-museums.com](https://www.niche-museums.com/) uses this approach. Here's an example search, for `bones`: - [www.niche-museums.com/browse/search?q=bones](https://www.niche-museums.com/browse/search?q=bones) This uses the `search` canned query, which is [defined here]([https://github.com/simonw/til/blob/8f961be162868c53b5c484272091bdab703a747a/metadata.yaml#L16-L32](https://github.com/simonw/museums/blob/74e999c0e82781302bf0346a761ee5d88e168863/metada… | <p>Datasette is built on top of <a href="https://sqlite.org/">SQLite</a>, which includes a robust, full-featured <a href="https://www.sqlite.org/fts5.html">full-text search</a> implementation.</p> <p>Datasette automatically detects tables which have been configured for full-text search and adds a search box which can be used by human site users or accessed from the JSON or CSV APIs.</p> <p>You can try this out now by <a href="https://fara.datasettes.com/">searching FARA records</a> or <a href="https://global-power-plants.datasettes.com/global-power-plants/global-power-plants">global power plants</a>.</p> <h2>Enabling search for a table</h2> <p>The <a href="https://sqlite-utils.datasette.io/en/stable/cli.html#configuring-full-text-search">sqlite-utils command-line tool</a> can be used to enable full-text search on a specific set of columns for a table:</p> <pre><code>% sqlite-utils enable-fts mydb.db documents title summary </code></pre> <p>Alternatively, you can configure search directly within the Datasette interface by installing the <a href="https://github.com/simonw/datasette-configure-fts">datasette-configure-fts</a> plugin.</p> <p><a href="https://simonwillison.net/2018/Dec/19/fast-autocomplete-search/">Fast Autocomplete Search for Your Website</a> is a tutorial that shows how to write a scraper that populates a full-text search index in Datasette, then build a JavaScript autocomplete search interface using the Datasette JSON API.</p> <h2>Implementing search with a canned query</h2> <p>Datasette <a href="https://docs.datasette.io/en/stable/sql_queries.html#canned-queries">canned queries</a> can be used to configure a SQL query which returns search results for a table, ordered by relevance.</p> <p>The search on <a href="https://www.niche-museums.com/">www.niche-museums.com</a> uses this approach. Here's an example search, for <code>bones</code>:</p> <ul> <li><a href="https://www.niche-museums.com/browse/search?q=bones">www.niche-museums.com/browse/search?q=bones</a></li> </ul> <p>This uses the <code>search</… | Datasette for search | Datasette uses SQLite's built-in full-text search feature to provide faceted search over any correctly configured collection of records. | 6 |
| ce0b74d7bd16f128e2bd4664b3aa910162b5373f | for/geospatial.md | The [SpatiaLite](https://www.gaia-gis.it/fossil/libspatialite/index) extension for SQLite can be used with Datasette to enable a full suite of geospatial SQL functions.  Datasette has a number of plugins and tools that can be used to work with geospatial data. Geospatial plugins include: - [datasette-cluster-map](https://datasette.io/plugins/datasette-cluster-map) renders a map of markers for any table or query with `latitude` and `longitude` columns - [datasette-geojson](https://datasette.io/plugins/datasette-geojson) adds a `.geojson` extension which can export SpatiaLite geometries as [GeoJSON](https://geojson.org/) - [datasette-geojson-map](https://datasette.io/plugins/datasette-geojson-map) renders that GeoJSON output on a map - [datasette-leaflet-freedraw](https://datasette.io/plugins/datasette-leaflet-freedraw) adds an interface to filter geometries by drawing a shape on a map - [datasette-leaflet-geojson](https://datasette.io/plugins/datasette-leaflet-geojson) shows maps inline for any column that includes GeoJSON - [datasette-tiles](https://datasette.io/plugins/datasette-tiles) allows Datasette to serve map tile images stored using the [MBTiles](https://github.com/mapbox/mbtiles-spec) format Tools for working with geospatial data include: - [geojson-to-sqlite](https://datasette.io/tools/geojson-to-sqlite), a command line utility for loading GeoJSON data into a SQLite or SpatiaLite database - [shapefile-to-sqlite](https://datasette.io/tools/shapefile-to-sqlite), a command line utility for loading shapefiles into SQLite or SpatiaLite - [download-tiles](https://datasette.io/tools/download-tiles) can be used to download map tiles and store them in MBTiles, suitable for use with [datasette-tiles](… | <p>The <a href="https://www.gaia-gis.it/fossil/libspatialite/index">SpatiaLite</a> extension for SQLite can be used with Datasette to enable a full suite of geospatial SQL functions.</p> <p><img alt="A screenshot of Datasette running some geospatial plugins" src="https://camo.githubusercontent.com/56a2ed7e65e28e726d4c3abfbe56bd7960524e6af6512fc2cee083bdd7fcd863/68747470733a2f2f7374617469632e73696d6f6e77696c6c69736f6e2e6e65742f7374617469632f323032312f6461746173657474652d6c6561666c65742d66726565647261772e706e67" /></p> <p>Datasette has a number of plugins and tools that can be used to work with geospatial data.</p> <p>Geospatial plugins include:</p> <ul> <li><a href="https://datasette.io/plugins/datasette-cluster-map">datasette-cluster-map</a> renders a map of markers for any table or query with <code>latitude</code> and <code>longitude</code> columns</li> <li><a href="https://datasette.io/plugins/datasette-geojson">datasette-geojson</a> adds a <code>.geojson</code> extension which can export SpatiaLite geometries as <a href="https://geojson.org/">GeoJSON</a></li> <li><a href="https://datasette.io/plugins/datasette-geojson-map">datasette-geojson-map</a> renders that GeoJSON output on a map</li> <li><a href="https://datasette.io/plugins/datasette-leaflet-freedraw">datasette-leaflet-freedraw</a> adds an interface to filter geometries by drawing a shape on a map</li> <li><a href="https://datasette.io/plugins/datasette-leaflet-geojson">datasette-leaflet-geojson</a> shows maps inline for any column that includes GeoJSON</li> <li><a href="https://datasette.io/plugins/datasette-tiles">datasette-tiles</a> allows Datasette to serve map tile images stored using the <a href="https://github.com/mapbox/mbtiles-spec">MBTiles</a> format</li> </ul> <p>Tools for working with geospatial data include:</p> <ul> <li><a href="https://datasette.io/tools/geojson-to-sqlite">geojson-to-sqlite</a>, a command line utility for loading GeoJSON data into a SQLite or SpatiaLite database</li> <li><a href="https://datasette.io/tools/shapefile-to-sq… | Datasette for geospatial analysis | Combining Datasette with SpatiaLite provides a powerful set of tools for analyzing and visualizing geospatial data. | 5 |
| ddae37b01edec73668ab75f7e9799f91b580a554 | for/exploratory-analysis.md | Your first step with any new data set should be to start exploring it. Datasette provides multiple tools for doing this. ## Start browsing the data Datasette's table interface lets you start scrolling through data straight away.  ## Facets Datasette automatically identifies columns with less than twenty unique values and gives you the option to facet by them - seeing the most common values, and selecting those to filter the set. This provides a powerful way to start spotting patterns in the data.  ## Visualization Plugins such as [datasette-vega](https://github.com/simonw/datasette-vega) and [datasette-cluster-map](https://github.com/simonw/datasette-cluster-map) provide tools for interactively visualizing data directly within the Datasette interface.  | <p>Your first step with any new data set should be to start exploring it. Datasette provides multiple tools for doing this.</p> <h2>Start browsing the data</h2> <p>Datasette's table interface lets you start scrolling through data straight away.</p> <p><img alt="Screenshot of the table interface" src="/static/datasette-screenshot.png" /></p> <h2>Facets</h2> <p>Datasette automatically identifies columns with less than twenty unique values and gives you the option to facet by them - seeing the most common values, and selecting those to filter the set. This provides a powerful way to start spotting patterns in the data.</p> <p><img alt="Screenshot of the facet interface" src="/static/screenshots/facets.png" /></p> <h2>Visualization</h2> <p>Plugins such as <a href="https://github.com/simonw/datasette-vega">datasette-vega</a> and <a href="https://github.com/simonw/datasette-cluster-map">datasette-cluster-map</a> provide tools for interactively visualizing data directly within the Datasette interface.</p> <p><img alt="Screenshot of datasette-cluster-map" src="/static/screenshots/datasette-cluster-map.png" /></p> | Datasette for exploratory analysis | Import data from CSVs, JSON, database connections and more. Datasette will automatically show you patterns in your data and help you share your findings with your colleagues. | 1 |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [uses] ( [_id] TEXT PRIMARY KEY, [_path] TEXT, [text] TEXT, [html] TEXT, [title] TEXT, [summary] TEXT, [order] INTEGER );