repos
31 rows where topics contains "datasette-tool"
This data as json, CSV (advanced)
Suggested facets: owner, homepage, size, stargazers_count, watchers_count, has_projects, has_wiki, forks_count, open_issues_count, topics, forks, open_issues, watchers, default_branch, permissions, network_count, subscribers_count, created_at (date), updated_at (date), pushed_at (date), topics (array)
| id ▼ | node_id | name | full_name | private | owner | html_url | description | fork | created_at | updated_at | pushed_at | homepage | size | stargazers_count | watchers_count | language | has_issues | has_projects | has_downloads | has_wiki | has_pages | forks_count | archived | disabled | open_issues_count | license | topics | forks | open_issues | watchers | default_branch | permissions | temp_clone_token | organization | network_count | subscribers_count | readme | readme_html | allow_forking | visibility | is_template | template_repository | web_commit_signoff_required | has_discussions |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 110509816 | MDEwOlJlcG9zaXRvcnkxMTA1MDk4MTY= | csvs-to-sqlite | simonw/csvs-to-sqlite | 0 | simonw 9599 | https://github.com/simonw/csvs-to-sqlite | Convert CSV files into a SQLite database | 0 | 2017-11-13T06:38:21Z | 2021-11-18T16:33:39Z | 2021-11-18T16:35:33Z | 138 | 655 | 655 | Python | 1 | 1 | 1 | 1 | 0 | 50 | 0 | 0 | 34 | Apache License 2.0 apache-2.0 | ["click", "csv", "datasette", "datasette-io", "datasette-tool", "pandas", "python", "sqlite"] | 50 | 34 | 655 | main | {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false} | 50 | 17 | # csvs-to-sqlite [](https://pypi.org/project/csvs-to-sqlite/) [](https://github.com/simonw/csvs-to-sqlite/releases) [](https://github.com/simonw/csvs-to-sqlite/actions?query=workflow%3ATest) [](https://github.com/simonw/csvs-to-sqlite/blob/main/LICENSE) Convert CSV files into a SQLite database. Browse and publish that SQLite database with [Datasette](https://github.com/simonw/datasette). Basic usage: csvs-to-sqlite myfile.csv mydatabase.db This will create a new SQLite database called `mydatabase.db` containing a single table, `myfile`, containing the CSV content. You can provide multiple CSV files: csvs-to-sqlite one.csv two.csv bundle.db The `bundle.db` database will contain two tables, `one` and `two`. This means you can use wildcards: csvs-to-sqlite ~/Downloads/*.csv my-downloads.db If you pass a path to one or more directories, the script will recursively search those directories for CSV files and create tables for each one. csvs-to-sqlite ~/path/to/directory all-my-csvs.db ## Handling TSV (tab-separated values) You can use the `-s` option to specify a different delimiter. If you want to use a tab character you'll need to apply shell escaping like so: csvs-to-sqlite my-file.tsv my-file.db -s $'\t' ## Refactoring columns into separate lookup tables Let's say you have a CSV file that looks like this: county,precinct,office,district,party,candidate,votes Clark,1,President,,REP,John R. Kasich,5 Clark,2,President,,REP,John R. Kasich,0 Clark,3,President,,REP,John R. Kasich,7 ([Real example taken from the Open Elections project](https://github.com/openelections/openelections-data-sd/blob/master/2016/20160607__sd__primary__clark__precinct.csv)) You can n… | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-csvs-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-csvs-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>csvs-to-sqlite</h1> <p dir="auto"><a href="https://pypi.org/project/csvs-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/4869c8a624ce3dbf120decc95900d98af23c54cf5c87d65f7164e640a5fcda6b/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f637376732d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/csvs-to-sqlite.svg" style="max-width: 100%;"></a> <a href="https://github.com/simonw/csvs-to-sqlite/releases"><img src="https://camo.githubusercontent.com/c0cc260f11a725723219d723be3829d23edfc0962d0e8d191fa2bcd6ceaa749f/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f637376732d746f2d73716c6974653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/csvs-to-sqlite?include_prereleases&label=changelog" style="max-width: 100%;"></a> <a href="https://github.com/simonw/csvs-to-sqlite/actions?query=workflow%3ATest"><img src="https://github.com/simonw/csvs-to-sqlite/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a> <a href="https://github.com/simonw/csvs-to-sqlite/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9… | 1 | public | 0 | ||||||
| 140912432 | MDEwOlJlcG9zaXRvcnkxNDA5MTI0MzI= | sqlite-utils | simonw/sqlite-utils | 0 | simonw 9599 | https://github.com/simonw/sqlite-utils | Python CLI utility and library for manipulating SQLite databases | 0 | 2018-07-14T03:21:46Z | 2022-11-15T18:12:16Z | 2022-11-15T15:53:38Z | https://sqlite-utils.datasette.io | 1437 | 1029 | 1029 | Python | 1 | 1 | 1 | 1 | 0 | 79 | 0 | 0 | 72 | Apache License 2.0 apache-2.0 | ["cli", "click", "datasette", "datasette-io", "datasette-tool", "python", "sqlite", "sqlite-database"] | 79 | 72 | 1029 | main | {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false} | 79 | 16 | # sqlite-utils [](https://pypi.org/project/sqlite-utils/) [](https://sqlite-utils.datasette.io/en/stable/changelog.html) [](https://pypi.org/project/sqlite-utils/) [](https://github.com/simonw/sqlite-utils/actions?query=workflow%3ATest) [](http://sqlite-utils.datasette.io/en/stable/?badge=stable) [](https://codecov.io/gh/simonw/sqlite-utils) [](https://github.com/simonw/sqlite-utils/blob/main/LICENSE) [](https://discord.gg/Ass7bCAMDw) Python CLI utility and library for manipulating SQLite databases. ## Some feature highlights - [Pipe JSON](https://sqlite-utils.datasette.io/en/stable/cli.html#inserting-json-data) (or [CSV or TSV](https://sqlite-utils.datasette.io/en/stable/cli.html#inserting-csv-or-tsv-data)) directly into a new SQLite database file, automatically creating a table with the appropriate schema - [Run in-memory SQL queries](https://sqlite-utils.datasette.io/en/stable/cli.html#querying-data-directly-using-an-in-memory-database), including joins, directly against data in CSV, TSV or JSON files and view the results - [Configure SQLite full-text search](https://sqlite-utils.datasette.io/en/stable/cli.html#configuring-full-text-search) against your database tables and run search queries against them, ordered by relevance - Run [transformations against your tables](https://sqlite-utils.datasette.io/en/stable/cli.html#transforming-tables) to make schema changes t… | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-sqlite-utils" class="anchor" aria-hidden="true" href="#user-content-sqlite-utils"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>sqlite-utils</h1> <p dir="auto"><a href="https://pypi.org/project/sqlite-utils/" rel="nofollow"><img src="https://camo.githubusercontent.com/f691f8124616f99c7978ecb2a58841aa6a7ace31234b0c87d392c4ef8db853e4/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f73716c6974652d7574696c732e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/sqlite-utils.svg" style="max-width: 100%;"></a> <a href="https://sqlite-utils.datasette.io/en/stable/changelog.html" rel="nofollow"><img src="https://camo.githubusercontent.com/b114e00dffe1e5b24d21c5cd11dc97f8a6ab83da51acbb971e5daebeeb47d690/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f73716c6974652d7574696c733f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/sqlite-utils?include_prereleases&label=changelog" style="max-width: 100%;"></a> <a href="https://pypi.org/project/sqlite-utils/" rel="nofollow"><img src="https://camo.githubusercontent.com/2413fbc1359bcdcf500aa31b497c8c25acf7db3f3fc3a33c540e809c984a189e/68747470733a2f2f696d672e736869656c64732e696f2f707970692f707976657273696f6e732f73716c6974652d7574696c732e7376673f6c6f676f3d707974686f6e266c6f676f436f6c6f723d776869… | 1 | public | 0 | 0 | 0 | |||
| 166159072 | MDEwOlJlcG9zaXRvcnkxNjYxNTkwNzI= | db-to-sqlite | simonw/db-to-sqlite | 0 | simonw 9599 | https://github.com/simonw/db-to-sqlite | CLI tool for exporting tables or queries from any SQL database to a SQLite file | 0 | 2019-01-17T04:16:48Z | 2021-06-11T22:52:12Z | 2021-06-11T22:55:56Z | 77 | 226 | 226 | Python | 1 | 1 | 1 | 1 | 0 | 12 | 0 | 0 | 2 | Apache License 2.0 apache-2.0 | ["sqlalchemy", "sqlite", "datasette", "datasette-io", "datasette-tool"] | 12 | 2 | 226 | main | {"admin": false, "push": false, "pull": false} | 12 | 4 | # db-to-sqlite [](https://pypi.python.org/pypi/db-to-sqlite) [](https://github.com/simonw/db-to-sqlite/releases) [](https://github.com/simonw/db-to-sqlite/actions?query=workflow%3ATest) [](https://github.com/simonw/db-to-sqlite/blob/main/LICENSE) CLI tool for exporting tables or queries from any SQL database to a SQLite file. ## Installation Install from PyPI like so: pip install db-to-sqlite If you want to use it with MySQL, you can install the extra dependency like this: pip install 'db-to-sqlite[mysql]' Installing the `mysqlclient` library on OS X can be tricky - I've found [this recipe](https://gist.github.com/simonw/90ac0afd204cd0d6d9c3135c3888d116) to work (run that before installing `db-to-sqlite`). For PostgreSQL, use this: pip install 'db-to-sqlite[postgresql]' ## Usage Usage: db-to-sqlite [OPTIONS] CONNECTION PATH Load data from any database into SQLite. PATH is a path to the SQLite file to create, e.c. /tmp/my_database.db CONNECTION is a SQLAlchemy connection string, for example: postgresql://localhost/my_database postgresql://username:passwd@localhost/my_database mysql://root@localhost/my_database mysql://username:passwd@localhost/my_database More: https://docs.sqlalchemy.org/en/13/core/engines.html#database-urls Options: --version Show the version and exit. --all Detect and copy all tables --table TEXT Specific tables to copy --skip TEXT When using --all skip these tables --redact TEXT... (table, column) pairs to redact with *** --sql TEXT Optional SQL q… | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-db-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-db-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>db-to-sqlite</h1> <p><a href="https://pypi.python.org/pypi/db-to-sqlite" rel="nofollow"><img src="https://camo.githubusercontent.com/b34869a6692a0e2ab6754463aad8578fe9f594788d99d4fd7ae2d815735d1660/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f64622d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/db-to-sqlite.svg" style="max-width:100%;"></a> <a href="https://github.com/simonw/db-to-sqlite/releases"><img src="https://camo.githubusercontent.com/0f168b599361c3b2242a18a2b84f49d1c4e5520c7c425a89f43fd1b6f337f299/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f64622d746f2d73716c6974653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/db-to-sqlite?include_prereleases&label=changelog" style="max-width:100%;"></a> <a href="https://github.com/simonw/db-to-sqlite/actions?query=workflow%3ATest"><img src="https://github.com/simonw/db-to-sqlite/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a> <a href="https://github.com/simonw/db-to-sqlite/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733… | |||||||||
| 167759846 | MDEwOlJlcG9zaXRvcnkxNjc3NTk4NDY= | markdown-to-sqlite | simonw/markdown-to-sqlite | 0 | simonw 9599 | https://github.com/simonw/markdown-to-sqlite | CLI tool for loading markdown files into a SQLite database | 0 | 2019-01-27T02:04:54Z | 2022-05-13T18:09:26Z | 2022-05-13T18:09:22Z | 13 | 49 | 49 | Python | 1 | 1 | 1 | 1 | 0 | 2 | 0 | 0 | 2 | Apache License 2.0 apache-2.0 | ["datasette-io", "datasette-tool", "markdown", "sqlite", "yaml"] | 2 | 2 | 49 | main | {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false} | 2 | 3 | # markdown-to-sqlite [](https://pypi.python.org/pypi/markdown-to-sqlite) [](https://github.com/simonw/markdown-to-sqlite/releases) [](https://github.com/simonw/markdown-to-sqlite/actions?query=workflow%3ATest) [](https://github.com/simonw/markdown-to-sqlite/blob/main/LICENSE) CLI tool for loading markdown files into a SQLite database. YAML embedded in the markdown files will be used to populate additional columns. Usage: markdown-to-sqlite [OPTIONS] DBNAME TABLE PATHS... For example: $ markdown-to-sqlite docs.db documents file1.md file2.md ## Breaking change Prior to version 1.0 this argument order was different - markdown files were listed before the database and table. | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-markdown-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-markdown-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>markdown-to-sqlite</h1> <p dir="auto"><a href="https://pypi.python.org/pypi/markdown-to-sqlite" rel="nofollow"><img src="https://camo.githubusercontent.com/e5aa6980c413976937ea698f111c31b580e6186e9f018673df37b3e9fe1047fb/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6d61726b646f776e2d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/markdown-to-sqlite.svg" style="max-width: 100%;"></a> <a href="https://github.com/simonw/markdown-to-sqlite/releases"><img src="https://camo.githubusercontent.com/29f48f8deb508081ca40341be55a1cdbe25dcd7a637e5277a24b82f167643ab6/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6d61726b646f776e2d746f2d73716c6974653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/markdown-to-sqlite?include_prereleases&label=changelog" style="max-width: 100%;"></a> <a href="https://github.com/simonw/markdown-to-sqlite/actions?query=workflow%3ATest"><img src="https://github.com/simonw/markdown-to-sqlite/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a> <a href="https://github.com/simonw/markdown-to-sqlite/blob/main/LICENSE"><img src="ht… | 1 | public | 0 | ||||||
| 168474970 | MDEwOlJlcG9zaXRvcnkxNjg0NzQ5NzA= | dbf-to-sqlite | simonw/dbf-to-sqlite | 0 | simonw 9599 | https://github.com/simonw/dbf-to-sqlite | CLI tool for converting DBF files (dBase, FoxPro etc) to SQLite | 0 | 2019-01-31T06:30:46Z | 2021-03-23T01:29:41Z | 2020-02-16T00:41:20Z | 8 | 25 | 25 | Python | 1 | 1 | 1 | 1 | 0 | 8 | 0 | 0 | 3 | Apache License 2.0 apache-2.0 | ["sqlite", "foxpro", "dbf", "dbase", "datasette-io", "datasette-tool"] | 8 | 3 | 25 | master | {"admin": false, "push": false, "pull": false} | 8 | 2 | # dbf-to-sqlite [](https://pypi.python.org/pypi/dbf-to-sqlite) [](https://travis-ci.com/simonw/dbf-to-sqlite) [](https://github.com/simonw/dbf-to-sqlite/blob/master/LICENSE) CLI tool for converting DBF files (dBase, FoxPro etc) to SQLite. ## Installation pip install dbf-to-sqlite ## Usage $ dbf-to-sqlite --help Usage: dbf-to-sqlite [OPTIONS] DBF_PATHS... SQLITE_DB Convert DBF files (dBase, FoxPro etc) to SQLite https://github.com/simonw/dbf-to-sqlite Options: --version Show the version and exit. --table TEXT Table name to use (only valid for single files) -v, --verbose Show what's going on --help Show this message and exit. Example usage: $ dbf-to-sqlite *.DBF database.db This will create a new SQLite database called `database.db` containing one table for each of the `DBF` files in the current directory. Looking for DBF files to try this out on? Try downloading the [Himalayan Database](http://himalayandatabase.com/) of all expeditions that have climbed in the Nepal Himalaya. | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-dbf-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-dbf-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>dbf-to-sqlite</h1> <p><a href="https://pypi.python.org/pypi/dbf-to-sqlite" rel="nofollow"><img src="https://camo.githubusercontent.com/59d39789edaeb918bb1febd34597470b4d6cab449725f3db3c0bb59d3c02551a/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6462662d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/dbf-to-sqlite.svg" style="max-width:100%;"></a> <a href="https://travis-ci.com/simonw/dbf-to-sqlite" rel="nofollow"><img src="https://camo.githubusercontent.com/d7e086b366bfa3de03a5c75c2112c011ac4eaf71863fed90253e3b5c28e13f89/68747470733a2f2f7472617669732d63692e636f6d2f73696d6f6e772f6462662d746f2d73716c6974652e7376673f6272616e63683d6d6173746572" alt="Travis CI" data-canonical-src="https://travis-ci.com/simonw/dbf-to-sqlite.svg?branch=master" style="max-width:100%;"></a> <a href="https://github.com/simonw/dbf-to-sqlite/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p> <p>CLI tool for converting DBF files (dBase, FoxPr… | |||||||||
| 175321497 | MDEwOlJlcG9zaXRvcnkxNzUzMjE0OTc= | csv-diff | simonw/csv-diff | 0 | simonw 9599 | https://github.com/simonw/csv-diff | Python CLI tool and library for diffing CSV and JSON files | 0 | 2019-03-13T01:11:26Z | 2022-07-29T20:01:02Z | 2022-07-29T20:00:59Z | 34 | 198 | 198 | Python | 1 | 1 | 1 | 1 | 0 | 29 | 0 | 0 | 18 | Apache License 2.0 apache-2.0 | ["click", "csv", "datasette-io", "datasette-tool", "diff", "git-scraping"] | 29 | 18 | 198 | main | {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false} | 29 | 7 | # csv-diff [](https://pypi.org/project/csv-diff/) [](https://github.com/simonw/csv-diff/releases) [](https://github.com/simonw/csv-diff/actions?query=workflow%3ATest) [](https://github.com/simonw/csv-diff/blob/main/LICENSE) Tool for viewing the difference between two CSV, TSV or JSON files. See [Generating a commit log for San Francisco’s official list of trees](https://simonwillison.net/2019/Mar/13/tree-history/) (and the [sf-tree-history repo commit log](https://github.com/simonw/sf-tree-history/commits)) for background information on this project. ## Installation pip install csv-diff ## Usage Consider two CSV files: `one.csv` id,name,age 1,Cleo,4 2,Pancakes,2 `two.csv` id,name,age 1,Cleo,5 3,Bailey,1 `csv-diff` can show a human-readable summary of differences between the files: $ csv-diff one.csv two.csv --key=id 1 row changed, 1 row added, 1 row removed 1 row changed Row 1 age: "4" => "5" 1 row added id: 3 name: Bailey age: 1 1 row removed id: 2 name: Pancakes age: 2 The `--key=id` option means that the `id` column should be treated as the unique key, to identify which records have changed. The tool will automatically detect if your files are comma- or tab-separated. You can over-ride this automatic detection and force the tool to use a specific format using `--format=tsv` or `--format=csv`. You can also feed it JSON files, provided they are a JSON array of objects where each object has the same keys. Use `--format=json` if your input files are JSON. Use `--show-unchanged` to include full details of the unchanged values for rows with at least one change in the diff output: % csv-diff one.csv two.c… | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-csv-diff" class="anchor" aria-hidden="true" href="#user-content-csv-diff"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>csv-diff</h1> <p dir="auto"><a href="https://pypi.org/project/csv-diff/" rel="nofollow"><img src="https://camo.githubusercontent.com/75784b8c5ee65df6e894c25c0efe54d360e3b6a33714da9aa0c6bb86ede1f153/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6373762d646966662e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/csv-diff.svg" style="max-width: 100%;"></a> <a href="https://github.com/simonw/csv-diff/releases"><img src="https://camo.githubusercontent.com/a8ece0f4436cb61b1524e3722c02363faf7aa45fe7e62f6634604f2c30421517/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6373762d646966663f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/csv-diff?include_prereleases&label=changelog" style="max-width: 100%;"></a> <a href="https://github.com/simonw/csv-diff/actions?query=workflow%3ATest"><img src="https://github.com/simonw/csv-diff/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a> <a href="https://github.com/simonw/csv-diff/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2… | 1 | public | 0 | 0 | |||||
| 175550127 | MDEwOlJlcG9zaXRvcnkxNzU1NTAxMjc= | yaml-to-sqlite | simonw/yaml-to-sqlite | 0 | simonw 9599 | https://github.com/simonw/yaml-to-sqlite | Utility for converting YAML files to SQLite | 0 | 2019-03-14T04:49:08Z | 2021-06-13T09:04:40Z | 2021-06-13T04:45:52Z | 19 | 36 | 36 | Python | 1 | 1 | 1 | 1 | 0 | 2 | 0 | 0 | 0 | Apache License 2.0 apache-2.0 | ["yaml", "sqlite", "datasette-io", "datasette-tool"] | 2 | 0 | 36 | main | {"admin": false, "push": false, "pull": false} | 2 | 1 | # yaml-to-sqlite [](https://pypi.org/project/yaml-to-sqlite/) [](https://github.com/simonw/yaml-to-sqlite/releases) [](https://github.com/simonw/yaml-to-sqlite/actions?query=workflow%3ATest) [](https://github.com/simonw/yaml-to-sqlite/blob/main/LICENSE) Load the contents of a YAML file into a SQLite database table. ``` $ yaml-to-sqlite --help Usage: yaml-to-sqlite [OPTIONS] DB_PATH TABLE YAML_FILE Convert YAML files to SQLite Options: --version Show the version and exit. --pk TEXT Column to use as a primary key --single-column TEXT If YAML file is a list of values, populate this column --help Show this message and exit. ``` ## Usage Given a `news.yml` file containing the following: ```yaml - date: 2021-06-05 body: |- [Datasette 0.57](https://docs.datasette.io/en/stable/changelog.html#v0-57) is out with an important security patch. - date: 2021-05-10 body: |- [Django SQL Dashboard](https://simonwillison.net/2021/May/10/django-sql-dashboard/) is a new tool that brings a useful authenticated subset of Datasette to Django projects that are built on top of PostgreSQL. ``` Running this command: ```bash $ yaml-to-sqlite news.db stories news.yml ``` Will create a database file with this schema: ```bash $ sqlite-utils schema news.db CREATE TABLE [stories] ( [date] TEXT, [body] TEXT ); ``` The `--pk` option can be used to set a column as the primary key for the table: ```bash $ yaml-to-sqlite news.db stories news.yml --pk date $ sqlite-utils schema news.db CREATE TABLE [stories] ( [date] TEXT PRIMARY KEY, [body] TEXT ); ``` ## Single column YAML lists The `--single-column` option can be used when the YAML file is a list of values, for … | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-yaml-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-yaml-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>yaml-to-sqlite</h1> <p><a href="https://pypi.org/project/yaml-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/32afda5e7bc913df42ad343b589f5d20c4fb51d9755037f4af1df86149cd0d94/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f79616d6c2d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/yaml-to-sqlite.svg" style="max-width:100%;"></a> <a href="https://github.com/simonw/yaml-to-sqlite/releases"><img src="https://camo.githubusercontent.com/83cd14b53497376686dfabbadad49ff0e485a8a84f8266713a3c89fe7181ccbc/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f79616d6c2d746f2d73716c6974653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/yaml-to-sqlite?include_prereleases&label=changelog" style="max-width:100%;"></a> <a href="https://github.com/simonw/yaml-to-sqlite/actions?query=workflow%3ATest"><img src="https://github.com/simonw/yaml-to-sqlite/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a> <a href="https://github.com/simonw/yaml-to-sqlite/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4… | |||||||||
| 195145678 | MDEwOlJlcG9zaXRvcnkxOTUxNDU2Nzg= | sqlite-diffable | simonw/sqlite-diffable | 0 | simonw 9599 | https://github.com/simonw/sqlite-diffable | Tools for dumping/loading a SQLite database to diffable directory structure | 0 | 2019-07-04T00:58:46Z | 2022-07-12T17:00:19Z | 2022-08-18T22:49:29Z | 30 | 42 | 42 | Python | 1 | 1 | 1 | 1 | 0 | 3 | 0 | 0 | 3 | Apache License 2.0 apache-2.0 | ["datasette-io", "datasette-tool", "sqlite"] | 3 | 3 | 42 | main | {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false} | 3 | 1 | # sqlite-diffable [](https://pypi.org/project/sqlite-diffable/) [](https://github.com/simonw/sqlite-diffable/releases) [](https://github.com/simonw/sqlite-diffable/blob/main/LICENSE) Tools for dumping/loading a SQLite database to diffable directory structure ## Installation pip install sqlite-diffable ## Demo The repository at [simonw/simonwillisonblog-backup](https://github.com/simonw/simonwillisonblog-backup) contains a backup of the database on my blog, https://simonwillison.net/ - created using this tool. ## Dumping a database Given a SQLite database called `fixtures.db` containing a table `facetable`, the following will dump out that table to the `dump/` directory: sqlite-diffable dump fixtures.db dump/ facetable To dump out every table in that database, use `--all`: sqlite-diffable dump fixtures.db dump/ --all ## Loading a database To load a previously dumped database, run the following: sqlite-diffable load restored.db dump/ This will show an error if any of the tables that are being restored already exist in the database file. You can replace those tables (dropping them before restoring them) using the `--replace` option: sqlite-diffable load restored.db dump/ --replace ## Converting to JSON objects Table rows are stored in the `.ndjson` files as newline-delimited JSON arrays, like this: ``` ["a", "a", "a-a", 63, null, 0.7364712141640124, "$null"] ["a", "b", "a-b", 51, null, 0.6020187290499803, "$null"] ``` Sometimes it can be more convenient to work with a list of JSON objects. The `sqlite-diffable objects` command can read a `.ndjson` file and its accompanying `.metadata.json` file and output JSON objects to standard output: sqlite-diffable objects fixtures.db dump/sortable.ndjson The output of that command looks somethi… | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-sqlite-diffable" class="anchor" aria-hidden="true" href="#user-content-sqlite-diffable"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>sqlite-diffable</h1> <p dir="auto"><a href="https://pypi.org/project/sqlite-diffable/" rel="nofollow"><img src="https://camo.githubusercontent.com/f397aed437dd402178061cc28cd230ddaa228d7a19d4e6e33dd7d9d6b9aa7016/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f73716c6974652d6469666661626c652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/sqlite-diffable.svg" style="max-width: 100%;"></a> <a href="https://github.com/simonw/sqlite-diffable/releases"><img src="https://camo.githubusercontent.com/ad89df075a7b5a43a53000aaf22188af53405e2b5177102cb1b550954996dac4/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f73716c6974652d6469666661626c653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/sqlite-diffable?include_prereleases&label=changelog" style="max-width: 100%;"></a> <a href="https://github.com/simonw/sqlite-diffable/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-sr… | 1 | public | 0 | 0 | |||||
| 197431109 | MDEwOlJlcG9zaXRvcnkxOTc0MzExMDk= | dogsheep-beta | dogsheep/dogsheep-beta | 0 | dogsheep 53015001 | https://github.com/dogsheep/dogsheep-beta | Build a search index across content from multiple SQLite database tables and run faceted searches against it using Datasette | 0 | 2019-07-17T17:07:26Z | 2021-06-13T14:39:01Z | 2021-06-13T14:38:59Z | https://dogsheep.github.io/ | 61 | 78 | 78 | Python | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 11 | ["search", "datasette", "datasette-plugin", "dogsheep", "datasette-io", "datasette-tool"] | 0 | 11 | 78 | main | {"admin": false, "push": false, "pull": false} | dogsheep 53015001 | 0 | 4 | # dogsheep-beta [](https://pypi.org/project/dogsheep-beta/) [](https://github.com/dogsheep/beta/releases) [](https://github.com/dogsheep/beta/actions?query=workflow%3ATest) [](https://github.com/dogsheep/beta/blob/main/LICENSE) Build a search index across content from multiple SQLite database tables and run faceted searches against it using Datasette ## Example A live example of this plugin is running at https://datasette.io/-/beta - configured using [this YAML file](https://github.com/simonw/datasette.io/blob/main/templates/dogsheep-beta.yml). Read more about how this example works in [Building a search engine for datasette.io](https://simonwillison.net/2020/Dec/19/dogsheep-beta/). ## Installation Install this tool like so: $ pip install dogsheep-beta ## Usage Run the indexer using the `dogsheep-beta` command-line tool: $ dogsheep-beta index dogsheep.db config.yml The `config.yml` file contains details of the databases and document types that should be indexed: ```yaml twitter.db: tweets: sql: |- select tweets.id as key, 'Tweet by @' || users.screen_name as title, tweets.created_at as timestamp, tweets.full_text as search_1 from tweets join users on tweets.user = users.id users: sql: |- select id as key, name || ' @' || screen_name as title, created_at as timestamp, description as search_1 from users ``` This will create a `search_index` table in the `dogsheep.db` database populated by data from those SQL queries. By default the search index that this tool creates will be configured for Porter stemming.… | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-dogsheep-beta" class="anchor" aria-hidden="true" href="#user-content-dogsheep-beta"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>dogsheep-beta</h1> <p><a href="https://pypi.org/project/dogsheep-beta/" rel="nofollow"><img src="https://camo.githubusercontent.com/f8d17c422fbd622ab366c57086226b7f90b6ab9057aa2f3b9f6844dd29bac733/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f646f6773686565702d626574612e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/dogsheep-beta.svg" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/beta/releases"><img src="https://camo.githubusercontent.com/341e696d2cba7197bea24e27cc6db041753c20587a00ced21e77516d26a54730/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f646f6773686565702f626574613f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/dogsheep/beta?include_prereleases&label=changelog" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/beta/actions?query=workflow%3ATest"><img src="https://github.com/dogsheep/beta/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/beta/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f626… | ||||||||
| 197882382 | MDEwOlJlcG9zaXRvcnkxOTc4ODIzODI= | healthkit-to-sqlite | dogsheep/healthkit-to-sqlite | 0 | dogsheep 53015001 | https://github.com/dogsheep/healthkit-to-sqlite | Convert an Apple Healthkit export zip to a SQLite database | 0 | 2019-07-20T05:03:12Z | 2021-08-20T00:55:34Z | 2021-08-20T00:56:17Z | https://datasette.io/tools/healthkit-to-sqlite | 29 | 91 | 91 | Python | 1 | 1 | 1 | 1 | 0 | 4 | 0 | 0 | 8 | Apache License 2.0 apache-2.0 | ["datasette", "datasette-io", "datasette-tool", "dogsheep", "healthkit", "sqlite"] | 4 | 8 | 91 | main | {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false} | dogsheep 53015001 | 4 | 3 | # healthkit-to-sqlite [](https://pypi.org/project/healthkit-to-sqlite/) [](https://github.com/dogsheep/healthkit-to-sqlite/releases) [](https://github.com/dogsheep/healthkit-to-sqlite/actions?query=workflow%3ATest) [](https://github.com/dogsheep/healthkit-to-sqlite/blob/main/LICENSE) Convert an Apple Healthkit export zip to a SQLite database ## How to install $ pip install healthkit-to-sqlite ## How to use First you need to export your Apple HealthKit data. 1. On your iPhone, open the "Health" app 2. Click the profile icon in the top right 3. Click "Export Health Data" at the bottom of that page 4. Save the resulting file somewhere you can access it, or AirDrop it directly to your laptop. Now you can convert the resulting `export.zip` file to SQLite like so: $ healthkit-to-sqlite export.zip healthkit.db A progress bar will be displayed. You can disable this using `--silent`. ``` Importing from HealthKit [#-------------] 5% 00:01:33 ``` You can explore the resulting data using [Datasette](https://datasette.readthedocs.io/) like this: $ datasette healthkit.db | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-healthkit-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-healthkit-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>healthkit-to-sqlite</h1> <p><a href="https://pypi.org/project/healthkit-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/d8fd72edb8183afd279306fabff220e4c1670906c58c256e9e3bd7fbdea8c76f/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6865616c74686b69742d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/healthkit-to-sqlite.svg" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/healthkit-to-sqlite/releases"><img src="https://camo.githubusercontent.com/9092a0b3a68a53e13d8470fdf4909dda3cabe09e9e74b53960ad2063c79ae8e4/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f646f6773686565702f6865616c74686b69742d746f2d73716c6974653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/dogsheep/healthkit-to-sqlite?include_prereleases&label=changelog" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/healthkit-to-sqlite/actions?query=workflow%3ATest"><img src="https://github.com/dogsheep/healthkit-to-sqlite/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/healthkit-to-sqlite/blob/main/LICENSE"><img src="ht… | |||||||
| 205429375 | MDEwOlJlcG9zaXRvcnkyMDU0MjkzNzU= | swarm-to-sqlite | dogsheep/swarm-to-sqlite | 0 | dogsheep 53015001 | https://github.com/dogsheep/swarm-to-sqlite | Create a SQLite database containing your checkin history from Foursquare Swarm | 0 | 2019-08-30T17:37:29Z | 2021-02-22T07:58:39Z | 2021-01-18T04:36:03Z | 49 | 37 | 37 | Python | 1 | 1 | 1 | 1 | 0 | 1 | 0 | 0 | 1 | Apache License 2.0 apache-2.0 | ["sqlite", "foursquare", "swarm", "foursquare-api", "datasette", "dogsheep", "datasette-io", "datasette-tool"] | 1 | 1 | 37 | main | {"admin": false, "push": false, "pull": false} | dogsheep 53015001 | 1 | 3 | # swarm-to-sqlite [](https://pypi.org/project/swarm-to-sqlite/) [](https://github.com/dogsheep/swarm-to-sqlite/releases) [](https://github.com/dogsheep/swarm-to-sqlite/actions?query=workflow%3ATest) [](https://github.com/dogsheep/swarm-to-sqlite/blob/main/LICENSE) Create a SQLite database containing your checkin history from Foursquare Swarm. ## How to install $ pip install swarm-to-sqlite ## Usage You will need to first obtain a valid OAuth token for your Foursquare account. You can do so using this tool: https://your-foursquare-oauth-token.glitch.me/ Simplest usage is to simply provide the name of the database file you wish to write to. The tool will prompt you to paste in your token, and will then download your checkins and store them in the specified database file. $ swarm-to-sqlite checkins.db Please provide your Foursquare OAuth token: Importing 3699 checkins [#########-----------------------] 27% 00:02:31 You can also pass the token as a command-line option: $ swarm-to-sqlite checkins.db --token=XXX Or as an environment variable: $ export FOURSQUARE_TOKEN=XXX $ swarm-to-sqlite checkins.db To retrieve just checkins within the past X hours, days or weeks, use the `--since=` option. For example, to pull only checkins that happened within the last 10 days use: $ swarm-to-sqlite checkins.db --token=XXX --since=10d Use `2w` for two weeks, `10h` for ten hours, `3d` for three days. In addition to saving the checkins to a database, you can also write them to a JSON file using the `--save` option: $ swarm-to-sqlite checkins.db --save=checkins.json Having done this, you can re-import checkins directly from that file (rather than making API calls to … | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-swarm-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-swarm-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>swarm-to-sqlite</h1> <p><a href="https://pypi.org/project/swarm-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/336537cfcc544f29699c00a2aa4b5a199c9a21f53e43aff833e268e664735ed9/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f737761726d2d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/swarm-to-sqlite.svg" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/swarm-to-sqlite/releases"><img src="https://camo.githubusercontent.com/3a244282c89501bdea48af0387ebed5c0a03d41ef8f87331ed3fe63421f42c4f/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f646f6773686565702f737761726d2d746f2d73716c6974653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/dogsheep/swarm-to-sqlite?include_prereleases&label=changelog" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/swarm-to-sqlite/actions?query=workflow%3ATest"><img src="https://github.com/dogsheep/swarm-to-sqlite/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/swarm-to-sqlite/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb084… | ||||||||
| 206156866 | MDEwOlJlcG9zaXRvcnkyMDYxNTY4NjY= | twitter-to-sqlite | dogsheep/twitter-to-sqlite | 0 | dogsheep 53015001 | https://github.com/dogsheep/twitter-to-sqlite | Save data from Twitter to a SQLite database | 0 | 2019-09-03T19:30:08Z | 2021-12-26T18:08:43Z | 2021-12-26T18:08:40Z | 298 | 269 | 269 | Python | 1 | 1 | 1 | 1 | 0 | 13 | 0 | 0 | 10 | Apache License 2.0 apache-2.0 | ["datasette", "datasette-io", "datasette-tool", "dogsheep", "sqlite", "twitter", "twitter-api"] | 13 | 10 | 269 | main | {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false} | dogsheep 53015001 | 13 | 5 | # twitter-to-sqlite [](https://pypi.org/project/twitter-to-sqlite/) [](https://github.com/dogsheep/twitter-to-sqlite/releases) [](https://github.com/dogsheep/twitter-to-sqlite/actions?query=workflow%3ATest) [](https://github.com/dogsheep/twitter-to-sqlite/blob/main/LICENSE) Save data from Twitter to a SQLite database. **This tool currently uses Twitter API v1**. You may be unable to use it if you do not have an API key for that version of the API. <!-- toc --> - [How to install](#how-to-install) - [Authentication](#authentication) - [Retrieving tweets by specific accounts](#retrieving-tweets-by-specific-accounts) - [Retrieve user profiles in bulk](#retrieve-user-profiles-in-bulk) - [Retrieve tweets in bulk](#retrieve-tweets-in-bulk) - [Retrieving Twitter followers](#retrieving-twitter-followers) - [Retrieving friends](#retrieving-friends) - [Retrieving favorited tweets](#retrieving-favorited-tweets) - [Retrieving Twitter lists](#retrieving-twitter-lists) - [Retrieving Twitter list memberships](#retrieving-twitter-list-memberships) - [Retrieving just follower and friend IDs](#retrieving-just-follower-and-friend-ids) - [Retrieving tweets from your home timeline](#retrieving-tweets-from-your-home-timeline) - [Retrieving your mentions](#retrieving-your-mentions) - [Providing input from a SQL query with --sql and --attach](#providing-input-from-a-sql-query-with---sql-and---attach) - [Running searches](#running-searches) - [Capturing tweets in real-time with track and follow](#capturing-tweets-in-real-time-with-track-and-follow) * [track](#track) * [follow](#follow) - [Importing data from your Twitter archive](#importing-data-from-your-twitter-archive) - [Design notes](#design-notes) <!-- … | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-twitter-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-twitter-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>twitter-to-sqlite</h1> <p dir="auto"><a href="https://pypi.org/project/twitter-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/fce3d286daf4a0ac037476f32a7f1885dd7f90329ec1df80f7fe6b9322c72f5c/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f747769747465722d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/twitter-to-sqlite.svg" style="max-width: 100%;"></a> <a href="https://github.com/dogsheep/twitter-to-sqlite/releases"><img src="https://camo.githubusercontent.com/e1228185d86e3eb446efcb27ff81748cdad3b90ac14ae58c61dab61a90bb992d/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f646f6773686565702f747769747465722d746f2d73716c6974653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/dogsheep/twitter-to-sqlite?include_prereleases&label=changelog" style="max-width: 100%;"></a> <a href="https://github.com/dogsheep/twitter-to-sqlite/actions?query=workflow%3ATest"><img src="https://github.com/dogsheep/twitter-to-sqlite/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a> <a href="https://github.com/dogsheep/twitter-to-sqlite/blob/main/LICENSE"><img src="https… | 1 | public | 0 | |||||
| 206202864 | MDEwOlJlcG9zaXRvcnkyMDYyMDI4NjQ= | inaturalist-to-sqlite | dogsheep/inaturalist-to-sqlite | 0 | dogsheep 53015001 | https://github.com/dogsheep/inaturalist-to-sqlite | Create a SQLite database containing your observation history from iNaturalist | 0 | 2019-09-04T01:21:21Z | 2020-12-19T05:18:38Z | 2020-10-22T00:08:58Z | 17 | 2 | 2 | Python | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | Apache License 2.0 apache-2.0 | ["sqlite", "inaturalist", "datasette", "dogsheep", "datasette-io", "datasette-tool"] | 0 | 0 | 2 | master | {"admin": false, "push": false, "pull": false} | dogsheep 53015001 | 0 | 1 | # inaturalist-to-sqlite [](https://pypi.org/project/inaturalist-to-sqlite/) [](https://circleci.com/gh/dogsheep/inaturalist-to-sqlite) [](https://github.com/dogsheep/inaturalist-to-sqlite/blob/master/LICENSE) Create a SQLite database containing your observation history from [iNaturalist](https://www.inaturalist.org/). ## How to install $ pip install inaturalist-to-sqlite ## Usage $ inaturalist-to-sqlite inaturalist.db yourusername (Or try `simonw` if you don't yet have an iNaturalist account) This will import all of your iNaturalist observations into a SQLite database called `inaturalist.db`. | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-inaturalist-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-inaturalist-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>inaturalist-to-sqlite</h1> <p><a href="https://pypi.org/project/inaturalist-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/0b4aee6bb6f3aeb706c5195fce3537b66445f052f38021aefeb48672cf06cf74/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f696e61747572616c6973742d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/inaturalist-to-sqlite.svg" style="max-width:100%;"></a> <a href="https://circleci.com/gh/dogsheep/inaturalist-to-sqlite" rel="nofollow"><img src="https://camo.githubusercontent.com/96dcb5c0cfa03bf010686f57c71dfef278ab6aaa53eb39cb48d28bde427c55b7/68747470733a2f2f636972636c6563692e636f6d2f67682f646f6773686565702f696e61747572616c6973742d746f2d73716c6974652e7376673f7374796c653d737667" alt="CircleCI" data-canonical-src="https://circleci.com/gh/dogsheep/inaturalist-to-sqlite.svg?style=svg" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/inaturalist-to-sqlite/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%2… | ||||||||
| 206649770 | MDEwOlJlcG9zaXRvcnkyMDY2NDk3NzA= | google-takeout-to-sqlite | dogsheep/google-takeout-to-sqlite | 0 | dogsheep 53015001 | https://github.com/dogsheep/google-takeout-to-sqlite | Save data from Google Takeout to a SQLite database | 0 | 2019-09-05T20:15:15Z | 2021-06-08T15:31:47Z | 2021-02-24T00:34:55Z | 14 | 51 | 51 | Python | 1 | 1 | 1 | 1 | 0 | 4 | 0 | 0 | 6 | Apache License 2.0 apache-2.0 | ["google", "sqlite", "datasette", "dogsheep", "datasette-io", "datasette-tool"] | 4 | 6 | 51 | master | {"admin": false, "push": false, "pull": false} | dogsheep 53015001 | 4 | 3 | # google-takeout-to-sqlite [](https://pypi.org/project/google-takeout-to-sqlite/) [](https://circleci.com/gh/dogsheep/google-takeout-to-sqlite) [](https://github.com/dogsheep/google-takeout-to-sqlite/blob/master/LICENSE) Save data from google-takeout to a SQLite database. ## How to install $ pip install google-takeout-to-sqlite Request your Google data from https://takeout.google.com/ - wait for the email and download the zip file. This tool only supports a subset of the available options. More will be added over time. ## My Activity You can request the "My Activity" export and then import it with the following command: $ google-takeout-to-sqlite my-activity takeout.db ~/Downloads/takeout-20190530.zip This will create a database file called `takeout.db` if one does not already exist. ## Location History Your location history records latitude, longitude and timestame for where Google has tracked your location. You can import it using this command: $ google-takeout-to-sqlite location-history takeout.db ~/Downloads/takeout-20190530.zip ## Browsing your data with Datasette Once you have imported Google data into a SQLite database file you can browse your data using [Datasette](https://github.com/simonw/datasette). Install Datasette like so: $ pip install datasette Now browse your data by running this and then visiting `http://localhost:8001/` $ datasette takeout.db Install the [datasette-cluster-map](https://github.com/simonw/datasette-cluster-map) plugin to see your location history on a map: $ pip install datasette-cluster-map | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-google-takeout-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-google-takeout-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>google-takeout-to-sqlite</h1> <p><a href="https://pypi.org/project/google-takeout-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/fb9e0c4c7734b2904f2b1aaaa4ce05fa195167989240be80a7c8e56e2b2a000e/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f676f6f676c652d74616b656f75742d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/google-takeout-to-sqlite.svg" style="max-width:100%;"></a> <a href="https://circleci.com/gh/dogsheep/google-takeout-to-sqlite" rel="nofollow"><img src="https://camo.githubusercontent.com/41bc34953e618d9a884ec9a8bad7d759bf580fc6b7c27982d3c4af4531121246/68747470733a2f2f636972636c6563692e636f6d2f67682f646f6773686565702f676f6f676c652d74616b656f75742d746f2d73716c6974652e7376673f7374796c653d737667" alt="CircleCI" data-canonical-src="https://circleci.com/gh/dogsheep/google-takeout-to-sqlite.svg?style=svg" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/google-takeout-to-sqlite/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://i… | ||||||||
| 207052882 | MDEwOlJlcG9zaXRvcnkyMDcwNTI4ODI= | github-to-sqlite | dogsheep/github-to-sqlite | 0 | dogsheep 53015001 | https://github.com/dogsheep/github-to-sqlite | Save data from GitHub to a SQLite database | 0 | 2019-09-08T02:50:28Z | 2022-09-20T04:36:37Z | 2022-09-28T21:07:54Z | https://github-to-sqlite.dogsheep.net/ | 143 | 235 | 235 | Python | 1 | 1 | 1 | 1 | 0 | 32 | 0 | 0 | 20 | Apache License 2.0 apache-2.0 | ["datasette", "datasette-io", "datasette-tool", "dogsheep", "github-api", "sqlite"] | 32 | 20 | 235 | main | {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false} | dogsheep 53015001 | 32 | 6 | # github-to-sqlite [](https://pypi.org/project/github-to-sqlite/) [](https://github.com/dogsheep/github-to-sqlite/releases) [](https://github.com/dogsheep/github-to-sqlite/actions?query=workflow%3ATest) [](https://github.com/dogsheep/github-to-sqlite/blob/main/LICENSE) Save data from GitHub to a SQLite database. <!-- toc --> - [Demo](#demo) - [How to install](#how-to-install) - [Authentication](#authentication) - [Fetching issues for a repository](#fetching-issues-for-a-repository) - [Fetching pull requests for a repository](#fetching-pull-requests-for-a-repository) - [Fetching issue comments for a repository](#fetching-issue-comments-for-a-repository) - [Fetching commits for a repository](#fetching-commits-for-a-repository) - [Fetching releases for a repository](#fetching-releases-for-a-repository) - [Fetching tags for a repository](#fetching-tags-for-a-repository) - [Fetching contributors to a repository](#fetching-contributors-to-a-repository) - [Fetching repos belonging to a user or organization](#fetching-repos-belonging-to-a-user-or-organization) - [Fetching specific repositories](#fetching-specific-repositories) - [Fetching repos that have been starred by a user](#fetching-repos-that-have-been-starred-by-a-user) - [Fetching users that have starred specific repos](#fetching-users-that-have-starred-specific-repos) - [Fetching GitHub Actions workflows](#fetching-github-actions-workflows) - [Scraping dependents for a repository](#scraping-dependents-for-a-repository) - [Fetching emojis](#fetching-emojis) - [Making authenticated API calls](#making-authenticated-api-calls) <!-- tocstop --> ## Demo https://github-to-sqlite.dogsheep.net/ hosts a [Datasette](https://datasette.io/) demo of a … | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-github-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-github-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>github-to-sqlite</h1> <p dir="auto"><a href="https://pypi.org/project/github-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/515e6efa4aef15e83b08072e5490c9040420223b2355d1d715cb628f66d60dff/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6769746875622d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/github-to-sqlite.svg" style="max-width: 100%;"></a> <a href="https://github.com/dogsheep/github-to-sqlite/releases"><img src="https://camo.githubusercontent.com/ae3fbf680cae0fca1e9126549e1fd0e14e756f6f77046e878db7b1c9cbd78911/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f646f6773686565702f6769746875622d746f2d73716c6974653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/dogsheep/github-to-sqlite?include_prereleases&label=changelog" style="max-width: 100%;"></a> <a href="https://github.com/dogsheep/github-to-sqlite/actions?query=workflow%3ATest"><img src="https://github.com/dogsheep/github-to-sqlite/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a> <a href="https://github.com/dogsheep/github-to-sqlite/blob/main/LICENSE"><img src="https://camo.github… | 1 | public | 0 | 0 | |||
| 209590345 | MDEwOlJlcG9zaXRvcnkyMDk1OTAzNDU= | genome-to-sqlite | dogsheep/genome-to-sqlite | 0 | dogsheep 53015001 | https://github.com/dogsheep/genome-to-sqlite | Import your genome into a SQLite database | 0 | 2019-09-19T15:38:39Z | 2021-01-18T19:39:48Z | 2019-09-19T15:41:17Z | 9 | 13 | 13 | Python | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 2 | Apache License 2.0 apache-2.0 | ["genetics", "sqlite", "23andme", "personal-analytics", "datasette", "dogsheep", "datasette-io", "datasette-tool"] | 0 | 2 | 13 | master | {"admin": false, "push": false, "pull": false} | dogsheep 53015001 | 0 | 2 | # genome-to-sqlite [](https://pypi.org/project/genome-to-sqlite/) [](https://circleci.com/gh/dogsheep/genome-to-sqlite) [](https://github.com/dogsheep/genome-to-sqlite/blob/master/LICENSE) Import your genome into a SQLite database. ## How to install $ pip install genome-to-sqlite ## How to use First, export your genome. This tool has only been tested against 23andMe so far. You can request an export of your genome from https://you.23andme.com/tools/data/download/ Now you can convert the resulting `export.zip` file to SQLite like so: $ genome-to-sqlite export.zip genome.db A progress bar will be displayed. You can disable this using `--silent`. ``` Importing genome [#----------------] 5% 00:01:33 ``` You can explore the resulting data using [Datasette](https://datasette.readthedocs.io/) like this: $ datasette genome.db --config facet_time_limit_ms:1000 Bumping up the facet time limit is useful in order to enable faceting by chromosome: http://127.0.0.1:8001/genome/genome?_facet=chromosome&_sort=position | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-genome-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-genome-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>genome-to-sqlite</h1> <p><a href="https://pypi.org/project/genome-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/a91d14ecfe9aa2d73da4818a7a898033e301380d4c8477d06a17dc20bd871c84/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f67656e6f6d652d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/genome-to-sqlite.svg" style="max-width:100%;"></a> <a href="https://circleci.com/gh/dogsheep/genome-to-sqlite" rel="nofollow"><img src="https://camo.githubusercontent.com/5f7f5be2791ff007c49d3d153ef38baa81770a11c0326e5f6b8ffd7ee939d00f/68747470733a2f2f636972636c6563692e636f6d2f67682f646f6773686565702f67656e6f6d652d746f2d73716c6974652e7376673f7374796c653d737667" alt="CircleCI" data-canonical-src="https://circleci.com/gh/dogsheep/genome-to-sqlite.svg?style=svg" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/genome-to-sqlite/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p> <p>Import yo… | ||||||||
| 213286752 | MDEwOlJlcG9zaXRvcnkyMTMyODY3NTI= | pocket-to-sqlite | dogsheep/pocket-to-sqlite | 0 | dogsheep 53015001 | https://github.com/dogsheep/pocket-to-sqlite | Create a SQLite database containing data from your Pocket account | 0 | 2019-10-07T03:24:14Z | 2022-08-21T21:11:59Z | 2022-08-22T16:21:34Z | 20 | 63 | 63 | Python | 1 | 1 | 1 | 1 | 0 | 3 | 0 | 0 | 5 | Apache License 2.0 apache-2.0 | ["datasette", "datasette-io", "datasette-tool", "dogsheep", "pocket", "pocket-api", "sqlite"] | 3 | 5 | 63 | main | {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false} | dogsheep 53015001 | 3 | 4 | # pocket-to-sqlite [](https://pypi.org/project/pocket-to-sqlite/) [](https://github.com/dogsheep/pocket-to-sqlite/releases) [](https://github.com/dogsheep/pocket-to-sqlite/actions?query=workflow%3ATest) [](https://github.com/dogsheep/pocket-to-sqlite/blob/main/LICENSE) Create a SQLite database containing data from your [Pocket](https://getpocket.com/) account. ## How to install $ pip install pocket-to-sqlite ## Usage You will need to first obtain a valid OAuth token for your Pocket account. You can do this by running the `auth` command and following the prompts: $ pocket-to-sqlite auth Visit this page and sign in with your Pocket account: https://getpocket.com/auth/author... Once you have signed in there, hit <enter> to continue Authentication tokens written to auth.json Now you can fetch all of your items from Pocket like this: $ pocket-to-sqlite fetch pocket.db The first time you run this command it will fetch all of your items, and display a progress bar while it does it. On subsequent runs it will only fetch new items. You can force it to fetch everything from the beginning again using `--all`. Use `--silent` to disable the progress bar. ## Using with Datasette The SQLite database produced by this tool is designed to be browsed using [Datasette](https://datasette.readthedocs.io/). Use the [datasette-render-timestamps](https://github.com/simonw/datasette-render-timestamps) plugin to improve the display of the timestamp values. | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-pocket-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-pocket-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>pocket-to-sqlite</h1> <p dir="auto"><a href="https://pypi.org/project/pocket-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/98050bc158f5e93f2c3576b03fa6a25b4a8091bb6d44774e07d95ae2842e4a3e/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f706f636b65742d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/pocket-to-sqlite.svg" style="max-width: 100%;"></a> <a href="https://github.com/dogsheep/pocket-to-sqlite/releases"><img src="https://camo.githubusercontent.com/b37c773b526e69ffcc72b9c02cbcd977d1b6e6a9bcf317c3accfc04b975e960c/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f646f6773686565702f706f636b65742d746f2d73716c6974653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/dogsheep/pocket-to-sqlite?include_prereleases&label=changelog" style="max-width: 100%;"></a> <a href="https://github.com/dogsheep/pocket-to-sqlite/actions?query=workflow%3ATest"><img src="https://github.com/dogsheep/pocket-to-sqlite/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a> <a href="https://github.com/dogsheep/pocket-to-sqlite/blob/main/LICENSE"><img src="https://camo.github… | 1 | public | 0 | 0 | ||||
| 219372133 | MDEwOlJlcG9zaXRvcnkyMTkzNzIxMzM= | sqlite-transform | simonw/sqlite-transform | 0 | simonw 9599 | https://github.com/simonw/sqlite-transform | Tool for running transformations on columns in a SQLite database | 0 | 2019-11-03T22:07:53Z | 2021-08-02T22:06:23Z | 2021-08-02T22:07:57Z | 64 | 29 | 29 | Python | 1 | 1 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | Apache License 2.0 apache-2.0 | ["sqlite", "datasette-io", "datasette-tool"] | 1 | 0 | 29 | main | {"admin": false, "push": false, "pull": false} | 1 | 1 | # sqlite-transform  [](https://pypi.org/project/sqlite-transform/) [](https://github.com/simonw/sqlite-transform/releases) [](https://github.com/simonw/sqlite-transform/actions?query=workflow%3ATest) [](https://github.com/dogsheep/sqlite-transform/blob/main/LICENSE) Tool for running transformations on columns in a SQLite database. > **:warning: This tool is no longer maintained** > > I added a new tool to [sqlite-utils](https://sqlite-utils.datasette.io/) called [sqlite-utils convert](https://sqlite-utils.datasette.io/en/stable/cli.html#converting-data-in-columns) which provides a super-set of the functionality originally provided here. `sqlite-transform` is no longer maintained, and I recommend switching to using `sqlite-utils convert` instead. ## How to install pip install sqlite-transform ## parsedate and parsedatetime These subcommands will run all values in the specified column through `dateutils.parser.parse()` and replace them with the result, formatted as an ISO timestamp or ISO date. For example, if a row in the database has an `opened` column which contains `10/10/2019 08:10:00 PM`, running the following command: sqlite-transform parsedatetime my.db mytable opened Will result in that value being replaced by `2019-10-10T20:10:00`. Using the `parsedate` subcommand here would result in `2019-10-10` instead. In the case of ambiguous dates such as `03/04/05` these commands both default to assuming American-style `mm/dd/yy` format. You can pass `--dayfirst` to specify that the day should be assumed to be first, or `--yearfirst` for the year. ## jsonsplit The `jsonsplit` subcommand takes columns t… | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-sqlite-transform" class="anchor" aria-hidden="true" href="#user-content-sqlite-transform"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>sqlite-transform</h1> <p><a target="_blank" rel="noopener noreferrer" href="https://camo.githubusercontent.com/818df98789ea0f246ed427c6efefc9450fdab68f50d69b83ecbbda8dda1d82b8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6e6f2532306c6f6e6765722d6d61696e7461696e65642d726564"><img src="https://camo.githubusercontent.com/818df98789ea0f246ed427c6efefc9450fdab68f50d69b83ecbbda8dda1d82b8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6e6f2532306c6f6e6765722d6d61696e7461696e65642d726564" alt="No longer maintained" data-canonical-src="https://img.shields.io/badge/no%20longer-maintained-red" style="max-width:100%;"></a> <a href="https://pypi.org/project/sqlite-transform/" rel="nofollow"><img src="https://camo.githubusercontent.com/607faf62b18abed6c31fc21c85d2e93f7800da5d89495d6df6355b2bb0b11a38/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f73716c6974652d7472616e73666f726d2e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/sqlite-transform.svg" style="max-width:100%;"></a> <a href="https://github.com/simonw/sqlite-transform/releases"><img src="https://camo.githubusercontent.com/351675bc73115e1f71304704ec82e7402dd85f44a866cd2580a9e1ce4194384f/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f… | |||||||||
| 237321267 | MDEwOlJlcG9zaXRvcnkyMzczMjEyNjc= | geojson-to-sqlite | simonw/geojson-to-sqlite | 0 | simonw 9599 | https://github.com/simonw/geojson-to-sqlite | CLI tool for converting GeoJSON files to SQLite (with SpatiaLite) | 0 | 2020-01-30T22:51:05Z | 2022-03-05T00:40:56Z | 2022-04-13T23:39:25Z | 117 | 34 | 34 | Python | 1 | 1 | 1 | 1 | 0 | 3 | 0 | 0 | 4 | Apache License 2.0 apache-2.0 | ["datasette-io", "datasette-tool", "geojson", "gis", "sqlite"] | 3 | 4 | 34 | main | {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false} | 3 | 3 | # geojson-to-sqlite [](https://pypi.org/project/geojson-to-sqlite/) [](https://github.com/simonw/geojson-to-sqlite/releases) [](https://github.com/simonw/geojson-to-sqlite/actions?query=workflow%3ATest) [](https://github.com/simonw/geojson-to-sqlite/blob/main/LICENSE) CLI tool for converting GeoJSON to SQLite (optionally with SpatiaLite) [RFC 7946: The GeoJSON Format](https://tools.ietf.org/html/rfc7946) ## How to install $ pip install geojson-to-sqlite ## How to use You can run this tool against a GeoJSON file like so: $ geojson-to-sqlite my.db features features.geojson This will load all of the features from the `features.geojson` file into a table called `features`. Each row will have a `geometry` column containing the feature geometry, and columns for each of the keys found in any `properties` attached to those features. (To bundle all properties into a single JSON object, use the `--properties` flag.) The table will be created the first time you run the command. On subsequent runs you can use the `--alter` option to add any new columns that are missing from the table. You can pass more than one GeoJSON file, in which case the contents of all of the files will be inserted into the same table. If your features have an `"id"` property it will be used as the primary key for the table. You can also use `--pk=PROPERTY` with the name of a different property to use that as the primary key instead. If you don't want to use the `"id"` as the primary key (maybe it contains duplicate values) you can use `--pk ''` to specify no primary key. Specifying a primary key also will allow you to upsert data into the rows instead of insert data into new rows. If no primary key is specified,… | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-geojson-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-geojson-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>geojson-to-sqlite</h1> <p dir="auto"><a href="https://pypi.org/project/geojson-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/b0c77834f0d6adf37e62573e121ccadd92711433ad2c5b4dd506610919762dab/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f67656f6a736f6e2d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/geojson-to-sqlite.svg" style="max-width: 100%;"></a> <a href="https://github.com/simonw/geojson-to-sqlite/releases"><img src="https://camo.githubusercontent.com/e88a12c13acc77ac4d0338f065dfeb664076b3414a7ee2560e8d465e828c1320/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f67656f6a736f6e2d746f2d73716c6974653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/geojson-to-sqlite?include_prereleases&label=changelog" style="max-width: 100%;"></a> <a href="https://github.com/simonw/geojson-to-sqlite/actions?query=workflow%3ATest"><img src="https://github.com/simonw/geojson-to-sqlite/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a> <a href="https://github.com/simonw/geojson-to-sqlite/blob/main/LICENSE"><img src="https://camo.github… | 1 | public | 0 | ||||||
| 240815938 | MDEwOlJlcG9zaXRvcnkyNDA4MTU5Mzg= | shapefile-to-sqlite | simonw/shapefile-to-sqlite | 0 | simonw 9599 | https://github.com/simonw/shapefile-to-sqlite | Load shapefiles into a SQLite (optionally SpatiaLite) database | 0 | 2020-02-16T01:55:29Z | 2021-03-26T08:39:43Z | 2020-08-23T06:00:41Z | 54 | 15 | 15 | Python | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 3 | Apache License 2.0 apache-2.0 | ["sqlite", "gis", "spatialite", "shapefiles", "datasette", "datasette-io", "datasette-tool"] | 0 | 3 | 15 | main | {"admin": false, "push": false, "pull": false} | 0 | 1 | # shapefile-to-sqlite [](https://pypi.org/project/shapefile-to-sqlite/) [](https://circleci.com/gh/simonw/shapefile-to-sqlite) [](https://github.com/simonw/shapefile-to-sqlite/blob/main/LICENSE) Load shapefiles into a SQLite (optionally SpatiaLite) database. Project background: [Things I learned about shapefiles building shapefile-to-sqlite](https://simonwillison.net/2020/Feb/19/shapefile-to-sqlite/) ## How to install $ pip install shapefile-to-sqlite ## How to use You can run this tool against a shapefile file like so: $ shapefile-to-sqlite my.db features.shp This will load the geometries as GeoJSON in a text column. ## Using with SpatiaLite If you have [SpatiaLite](https://www.gaia-gis.it/fossil/libspatialite/index) available you can load them as SpatiaLite geometries like this: $ shapefile-to-sqlite my.db features.shp --spatialite The data will be loaded into a table called `features` - based on the name of the shapefile. You can specify an alternative table name using `--table`: $ shapefile-to-sqlite my.db features.shp --table=places --spatialite The tool will search for the SpatiaLite module in the following locations: - `/usr/lib/x86_64-linux-gnu/mod_spatialite.so` - `/usr/local/lib/mod_spatialite.dylib` If you have installed the module in another location, you can use the `--spatialite_mod=xxx` option to specify where: $ shapefile-to-sqlite my.db features.shp \ --spatialite_mod=/usr/lib/mod_spatialite.dylib You can use the `--spatial-index` option to create a spatial index on the `geometry` column: $ shapefile-to-sqlite my.db features.shp --spatial-index You can omit `--spatialite` if you use either `--spatialite-mod` or `--spatial-index`. ## Projections By default, this tool will attempt to convert geometries in the shapefile to the WGS 84 projec… | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-shapefile-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-shapefile-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>shapefile-to-sqlite</h1> <p><a href="https://pypi.org/project/shapefile-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/491f86f2d61c3cf6ce08c6c019a47686ccbe35187c465aeb784876d3adcc236b/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f736861706566696c652d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/shapefile-to-sqlite.svg" style="max-width:100%;"></a> <a href="https://circleci.com/gh/simonw/shapefile-to-sqlite" rel="nofollow"><img src="https://camo.githubusercontent.com/665b44f51d2c08eeb25388a24485a1591ebe793b914807ae371fe96fb242c0aa/68747470733a2f2f636972636c6563692e636f6d2f67682f73696d6f6e772f736861706566696c652d746f2d73716c6974652e7376673f7374796c653d737667" alt="CircleCI" data-canonical-src="https://circleci.com/gh/simonw/shapefile-to-sqlite.svg?style=svg" style="max-width:100%;"></a> <a href="https://github.com/simonw/shapefile-to-sqlite/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%… | |||||||||
| 245670670 | MDEwOlJlcG9zaXRvcnkyNDU2NzA2NzA= | fec-to-sqlite | simonw/fec-to-sqlite | 0 | simonw 9599 | https://github.com/simonw/fec-to-sqlite | Save FEC campaign finance data to a SQLite database | 0 | 2020-03-07T16:52:49Z | 2020-12-19T05:09:05Z | 2020-03-07T18:21:48Z | 16 | 8 | 8 | Python | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 1 | Apache License 2.0 apache-2.0 | ["sqlite", "fec", "datasette", "datasette-io", "datasette-tool"] | 0 | 1 | 8 | master | {"admin": false, "push": false, "pull": false} | 0 | 2 | # fec-to-sqlite [](https://pypi.org/project/fec-to-sqlite/) [](https://circleci.com/gh/simonw/fec-to-sqlite) [](https://github.com/simonw/fec-to-sqlite/blob/master/LICENSE) Create a SQLite database using FEC campaign contributions data. This tool builds on [fecfile](https://github.com/esonderegger/) by Evan Sonderegger. ## How to install $ pip install fec-to-sqlite ## Usage $ fec-to-sqlite filings filings.db 1146148 This fetches the filing with ID `1146148` and stores it in tables in a SQLite database called `filings.db`. It will create any tables it needs. You can pass more than one filing ID, separated by spaces. | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-fec-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-fec-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>fec-to-sqlite</h1> <p><a href="https://pypi.org/project/fec-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/93308ffe0f5302a01fe685a905be46fc42abddee239c695af80456aabcd72e94/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6665632d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/fec-to-sqlite.svg" style="max-width:100%;"></a> <a href="https://circleci.com/gh/simonw/fec-to-sqlite" rel="nofollow"><img src="https://camo.githubusercontent.com/46d17ba2dacfcf6081f8db0a40f9dfd848e9bb2af54c6652e8358a01a88ba2cd/68747470733a2f2f636972636c6563692e636f6d2f67682f73696d6f6e772f6665632d746f2d73716c6974652e7376673f7374796c653d737667" alt="CircleCI" data-canonical-src="https://circleci.com/gh/simonw/fec-to-sqlite.svg?style=svg" style="max-width:100%;"></a> <a href="https://github.com/simonw/fec-to-sqlite/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p> <p>Create a SQLite database using FEC campaign contributio… | |||||||||
| 248903544 | MDEwOlJlcG9zaXRvcnkyNDg5MDM1NDQ= | hacker-news-to-sqlite | dogsheep/hacker-news-to-sqlite | 0 | dogsheep 53015001 | https://github.com/dogsheep/hacker-news-to-sqlite | Create a SQLite database containing data pulled from Hacker News | 0 | 2020-03-21T04:02:05Z | 2021-06-06T22:42:00Z | 2021-03-13T19:15:06Z | 19 | 25 | 25 | Python | 1 | 1 | 1 | 1 | 0 | 2 | 0 | 0 | 0 | Apache License 2.0 apache-2.0 | ["hacker-news", "datasette", "dogsheep", "datasette-io", "datasette-tool"] | 2 | 0 | 25 | main | {"admin": false, "push": false, "pull": false} | dogsheep 53015001 | 2 | 1 | # hacker-news-to-sqlite [](https://pypi.org/project/hacker-news-to-sqlite/) [](https://github.com/dogsheep/hacker-news-to-sqlite/releases) [](https://github.com/dogsheep/hacker-news-to-sqlite/actions?query=workflow%3ATest) [](https://github.com/simonw/hacker-news-to-sqlite/blob/main/LICENSE) Create a SQLite database containing data fetched from [Hacker News](https://news.ycombinator.com/). ## How to install $ pip install hacker-news-to-sqlite ## Usage $ hacker-news-to-sqlite user hacker-news.db your-username Importing items: 37%|███████████ | 845/2297 [05:09<11:02, 2.19it/s] Imports all of your Hacker News submissions and comments into a SQLite database called `hacker-news.db`. $ hacker-news-to-sqlite trees hacker-news.db 22640038 22643218 Fetches the entire comments tree in which any of those content IDs appears. ## Browsing your data with Datasette You can use [Datasette](https://datasette.readthedocs.org/) to browse your data. Install Datasette like this: $ pip install datasette Now run it against your `hacker-news.db` file like so: $ datasette hacker-news.db Visit `https://localhost:8001/` to search and explore your data. You can improve the display of your data usinng the [datasette-render-timestamps](https://github.com/simonw/datasette-render-timestamps) and [datasette-render-html](https://github.com/simonw/datasette-render-html) plugins. Install them like this: $ pip install datasette-render-timestamps datasette-render-html Now save the following configuration in a file called `metadata.json`: ```json { "databases": { "hacker-news": { "tables": { "items": { … | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-hacker-news-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-hacker-news-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>hacker-news-to-sqlite</h1> <p><a href="https://pypi.org/project/hacker-news-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/bf6d88d26ea4d8f1f396c1b1fc88ae74380de6a30f5a792c9f29664f21a219dd/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6861636b65722d6e6577732d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/hacker-news-to-sqlite.svg" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/hacker-news-to-sqlite/releases"><img src="https://camo.githubusercontent.com/090ef507ee9402a786e40313d173f516f1dbdedf3df703402296594a4ebcad73/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f646f6773686565702f6861636b65722d6e6577732d746f2d73716c6974653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/dogsheep/hacker-news-to-sqlite?include_prereleases&label=changelog" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/hacker-news-to-sqlite/actions?query=workflow%3ATest"><img src="https://github.com/dogsheep/hacker-news-to-sqlite/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a> <a href="https://github.com/simonw/hacker-news-to-sqlite/blob/… | ||||||||
| 255460347 | MDEwOlJlcG9zaXRvcnkyNTU0NjAzNDc= | datasette-clone | simonw/datasette-clone | 0 | simonw 9599 | https://github.com/simonw/datasette-clone | Create a local copy of database files from a Datasette instance | 0 | 2020-04-13T23:05:41Z | 2021-06-08T15:33:21Z | 2021-02-22T19:32:36Z | 20 | 2 | 2 | Python | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | Apache License 2.0 apache-2.0 | ["datasette", "datasette-io", "datasette-tool"] | 0 | 0 | 2 | main | {"admin": false, "push": false, "pull": false} | 0 | 1 | # datasette-clone [](https://pypi.org/project/datasette-clone/) [](https://github.com/simonw/datasette-clone/releases) [](https://github.com/simonw/datasette-clone/actions?query=workflow%3ATest) [](https://github.com/simonw/datasette-clone/blob/main/LICENSE) Create a local copy of database files from a Datasette instance. See [datasette-clone](https://simonwillison.net/2020/Apr/14/datasette-clone/) on my blog for background on this project. ## How to install $ pip install datasette-clone ## Usage This only works against Datasette instances running immutable databases (with the `-i` option). Databases published using the `datasette publish` command should be compatible with this tool. To download copies of all `.db` files from an instance, run: datasette-clone https://latest.datasette.io You can provide an optional second argument to specify a directory: datasette-clone https://latest.datasette.io /tmp/here-please The command stores its own copy of a `databases.json` manifest and uses it to only download databases that have changed the next time you run the command. It also stores a copy of the instance's `metadata.json` to ensure you have a copy of any source and licensing information for the downloaded databases. If your instance is protected by an API token, you can use `--token` to provide it: datasette-clone https://latest.datasette.io --token=xyz For verbose output showing what the tool is doing, use `-v`. | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-datasette-clone" class="anchor" aria-hidden="true" href="#user-content-datasette-clone"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>datasette-clone</h1> <p><a href="https://pypi.org/project/datasette-clone/" rel="nofollow"><img src="https://camo.githubusercontent.com/17cc348c1197fd918ac911e525efd416a5d4ca5d3f00729cbf85c930ac5dbde6/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6461746173657474652d636c6f6e652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/datasette-clone.svg" style="max-width:100%;"></a> <a href="https://github.com/simonw/datasette-clone/releases"><img src="https://camo.githubusercontent.com/9f8615567ed0f9b4d38d5f69dafcc8ae2fa97aa13895779172645064f16fcc9f/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6461746173657474652d636c6f6e653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/datasette-clone?include_prereleases&label=changelog" style="max-width:100%;"></a> <a href="https://github.com/simonw/datasette-clone/actions?query=workflow%3ATest"><img src="https://github.com/simonw/datasette-clone/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a> <a href="https://github.com/simonw/datasette-clone/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb… | |||||||||
| 256834907 | MDEwOlJlcG9zaXRvcnkyNTY4MzQ5MDc= | dogsheep-photos | dogsheep/dogsheep-photos | 0 | dogsheep 53015001 | https://github.com/dogsheep/dogsheep-photos | Upload your photos to S3 and import metadata about them into a SQLite database | 0 | 2020-04-18T19:22:13Z | 2021-11-04T20:45:03Z | 2021-11-04T20:45:00Z | 68 | 124 | 124 | Python | 1 | 1 | 1 | 1 | 0 | 7 | 0 | 0 | 19 | Apache License 2.0 apache-2.0 | ["datasette", "datasette-io", "datasette-tool", "dogsheep", "sqlite"] | 7 | 19 | 124 | master | {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false} | dogsheep 53015001 | 7 | 10 | # dogsheep-photos [](https://pypi.org/project/dogsheep-photos/) [](https://github.com/dogsheep/dogsheep-photos/releases) [](https://circleci.com/gh/dogsheep/dogsheep-photos) [](https://github.com/dogsheep/dogsheep-photos/blob/master/LICENSE) Save details of your photos to a SQLite database and upload them to S3. See [Using SQL to find my best photo of a pelican according to Apple Photos](https://simonwillison.net/2020/May/21/apple-photos-sqlite/) for background information on this project. ## What these tools do These tools are a work-in-progress mechanism for taking full ownership of your photos. The core idea is to help implement the following: * Every photo you have taken lives in a single, private Amazon S3 bucket * You have a single SQLite database file which stores metadata about those photos - potentially pulled from multiple different places. This may include EXIF data, Apple Photos, the results of running machine learning APIs against photos and much more besides. * You can then use [Datasette](https://github.com/simonw/datasette) to explore your own photos. I'm a heavy user of Apple Photos so the initial releases of this tool will have a bias towards that, but ideally I would like a subset of these tools to be useful to people no matter which core photo solution they are using. ## Installation $ pip install dogsheep-photos ## Authentication (if using S3) If you want to use S3 to store your photos, you will need to first create S3 credentials for a new, dedicated bucket. You may find the [s3-credentials tool](https://github.com/simonw/s3-credentials) useful for this. Run this command and paste in your credentials. You will need three values: the name of your S3 bucket, your Ac… | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-dogsheep-photos" class="anchor" aria-hidden="true" href="#user-content-dogsheep-photos"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>dogsheep-photos</h1> <p><a href="https://pypi.org/project/dogsheep-photos/" rel="nofollow"><img src="https://camo.githubusercontent.com/c21d10e4250454707420755a0d4b90c97709771d795ffa55b495c98956a3938d/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f646f6773686565702d70686f746f732e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/dogsheep-photos.svg" style="max-width: 100%;"></a> <a href="https://github.com/dogsheep/dogsheep-photos/releases"><img src="https://camo.githubusercontent.com/c7bead2f1c989034f62914278a736eecdfbab79ce4d9b3ab804d1692e230d79d/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f646f6773686565702f646f6773686565702d70686f746f733f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/dogsheep/dogsheep-photos?include_prereleases&label=changelog" style="max-width: 100%;"></a> <a href="https://circleci.com/gh/dogsheep/dogsheep-photos" rel="nofollow"><img src="https://camo.githubusercontent.com/3aefe86d4a3560c1ca080cf5b7e4dcc5788b239df8b7a3fed40e622b834c22e8/68747470733a2f2f636972636c6563692e636f6d2f67682f646f6773686565702f646f6773686565702d70686f746f732e7376673f7374796c653d737667" alt="CircleCI" data-canonical-src… | 1 | public | 0 | |||||
| 274264484 | MDEwOlJlcG9zaXRvcnkyNzQyNjQ0ODQ= | sqlite-generate | simonw/sqlite-generate | 0 | simonw 9599 | https://github.com/simonw/sqlite-generate | Tool for generating demo SQLite databases | 0 | 2020-06-22T23:36:44Z | 2021-02-27T15:25:26Z | 2021-02-27T15:25:24Z | https://sqlite-generate-demo.datasette.io/ | 56 | 17 | 17 | Python | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | Apache License 2.0 apache-2.0 | ["sqlite", "datasette-io", "datasette-tool"] | 0 | 0 | 17 | main | {"admin": false, "push": false, "pull": false} | 0 | 2 | # sqlite-generate [](https://pypi.org/project/sqlite-generate/) [](https://github.com/simonw/sqlite-generate/releases) [](https://github.com/simonw/sqlite-generate/blob/master/LICENSE) Tool for generating demo SQLite databases ## Installation Install this plugin using `pip`: $ pip install sqlite-generate ## Demo You can see a demo of the database generated using this command running in [Datasette](https://github.com/simonw/datasette) at https://sqlite-generate-demo.datasette.io/ The demo is generated using the following command: sqlite-generate demo.db --seed seed --fts --columns=10 --fks=0,3 --pks=0,2 ## Usage To generate a SQLite database file called `data.db` with 10 randomly named tables in it, run the following: sqlite-generate data.db You can use the `--tables` option to generate a different number of tables: sqlite-generate data.db --tables 20 You can run the command against the same database file multiple times to keep adding new tables, using different settings for each batch of generated tables. By default each table will contain a random number of rows between 0 and 200. You can customize this with the `--rows` option: sqlite-generate data.db --rows 20 This will insert 20 rows into each table. sqlite-generate data.db --rows 500,2000 This inserts a random number of rows between 500 and 2000 into each table. Each table will have 5 columns. You can change this using `--columns`: sqlite-generate data.db --columns 10 `--columns` can also accept a range: sqlite-generate data.db --columns 5,15 You can control the random number seed used with the `--seed` option. This will result in the exact same database file being created by multiple runs of the tool: sqlite-generate data.db --seed=myseed By default each table will contain betwee… | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-sqlite-generate" class="anchor" aria-hidden="true" href="#user-content-sqlite-generate"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>sqlite-generate</h1> <p><a href="https://pypi.org/project/sqlite-generate/" rel="nofollow"><img src="https://camo.githubusercontent.com/f0bc12d9a036f8faadbe40bbc37caa416eadf33d6694322ca385c16f2302b575/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f73716c6974652d67656e65726174652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/sqlite-generate.svg" style="max-width:100%;"></a> <a href="https://github.com/simonw/sqlite-generate/releases"><img src="https://camo.githubusercontent.com/8acbae82ad62477a2630aee86e3c2c6c498a06acc6fd6d6c08e51779ea3905de/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f73716c6974652d67656e65726174653f6c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/sqlite-generate?label=changelog" style="max-width:100%;"></a> <a href="https://github.com/simonw/sqlite-generate/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"… | ||||||||
| 291339086 | MDEwOlJlcG9zaXRvcnkyOTEzMzkwODY= | airtable-export | simonw/airtable-export | 0 | simonw 9599 | https://github.com/simonw/airtable-export | Export Airtable data to YAML, JSON or SQLite files on disk | 0 | 2020-08-29T19:51:37Z | 2021-06-08T17:30:30Z | 2021-04-09T23:41:52Z | https://datasette.io/tools/airtable-export | 41 | 33 | 33 | Python | 1 | 1 | 1 | 1 | 0 | 5 | 0 | 0 | 6 | Apache License 2.0 apache-2.0 | ["yaml", "airtable", "airtable-api", "datasette-io", "datasette-tool"] | 5 | 6 | 33 | main | {"admin": false, "push": false, "pull": false} | 5 | 3 | # airtable-export [](https://pypi.org/project/airtable-export/) [](https://github.com/simonw/airtable-export/releases) [](https://github.com/simonw/airtable-export/actions?query=workflow%3ATest) [](https://github.com/simonw/airtable-export/blob/master/LICENSE) Export Airtable data to files on disk ## Installation Install this tool using `pip`: $ pip install airtable-export ## Usage You will need to know the following information: - Your Airtable base ID - this is a string starting with `app...` - Your Airtable API key - this is a string starting with `key...` - The names of each of the tables that you wish to export You can export all of your data to a folder called `export/` by running the following: airtable-export export base_id table1 table2 --key=key This example would create two files: `export/table1.yml` and `export/table2.yml`. Rather than passing the API key using the `--key` option you can set it as an environment variable called `AIRTABLE_KEY`. ## Export options By default the tool exports your data as YAML. You can also export as JSON or as [newline delimited JSON](http://ndjson.org/) using the `--json` or `--ndjson` options: airtable-export export base_id table1 table2 --key=key --ndjson You can pass multiple format options at once. This command will create a `.json`, `.yml` and `.ndjson` file for each exported table: airtable-export export base_id table1 table2 \ --key=key --ndjson --yaml --json ### SQLite database export You can export tables to a SQLite database file using the `--sqlite database.db` option: airtable-export export base_id table1 table2 \ --key=key --sqlite database.db This can be combined with other format options.… | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-airtable-export" class="anchor" aria-hidden="true" href="#user-content-airtable-export"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>airtable-export</h1> <p><a href="https://pypi.org/project/airtable-export/" rel="nofollow"><img src="https://camo.githubusercontent.com/2635595699cd87784148506d703996615d75d6e8d6d0eba3928a886a5ebac963/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6169727461626c652d6578706f72742e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/airtable-export.svg" style="max-width:100%;"></a> <a href="https://github.com/simonw/airtable-export/releases"><img src="https://camo.githubusercontent.com/361aa58985cc1f5841b899de85bedd6e7ced2a253334b4dc390bd1762b2f8be5/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f6169727461626c652d6578706f72743f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/airtable-export?include_prereleases&label=changelog" style="max-width:100%;"></a> <a href="https://github.com/simonw/airtable-export/actions?query=workflow%3ATest"><img src="https://github.com/simonw/airtable-export/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a> <a href="https://github.com/simonw/airtable-export/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802… | ||||||||
| 303218369 | MDEwOlJlcG9zaXRvcnkzMDMyMTgzNjk= | evernote-to-sqlite | dogsheep/evernote-to-sqlite | 0 | dogsheep 53015001 | https://github.com/dogsheep/evernote-to-sqlite | Tools for converting Evernote content to SQLite | 0 | 2020-10-11T21:45:49Z | 2021-08-26T19:01:54Z | 2021-08-26T19:02:47Z | 51 | 24 | 24 | Python | 1 | 1 | 1 | 1 | 0 | 4 | 0 | 0 | 3 | Apache License 2.0 apache-2.0 | ["datasette-io", "datasette-tool", "dogsheep", "evernote", "sqlite"] | 4 | 3 | 24 | main | {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false} | dogsheep 53015001 | 4 | 4 | # evernote-to-sqlite [](https://pypi.org/project/evernote-to-sqlite/) [](https://github.com/dogsheep/evernote-to-sqlite/releases) [](https://github.com/dogsheep/evernote-to-sqlite/actions?query=workflow%3ATest) [](https://github.com/dogsheep/evernote-to-sqlite/blob/master/LICENSE) Tools for converting Evernote content to SQLite. See [Building an Evernote to SQLite exporter](https://simonwillison.net/2020/Oct/16/building-evernote-sqlite-exporter/) for background on this project. ## Installation Install this tool using `pip`: $ pip install evernote-to-sqlite ## Usage Currently the only available command is `evernote-to-sqlite enex`, which converts Evernote's ENEX export files into a SQLite database. You can create [an ENEX export](https://help.evernote.com/hc/en-us/articles/209005557-Export-notes-and-notebooks-as-ENEX-or-HTML) in the Evernote desktop application by selecting some notes (or all of your notes) and using the `File -> Export Notes...` menu option. This used to be able to export everything in one go, but it looks like more recent Evernote versions only allow exporting up to fifty notes at a time, or let you export an entire notebook by right-clicking on the notebook and selecting "Export notebook...". You can convert that file to SQLite like so: $ evernote-to-sqlite enex evernote.db MyNotes.enex This will display a progress bar and create a SQLite database file called `evernote.db`. ### Limitations Unfortunately the ENEX export format does not include a unique identifier for each note. This means you cannot use this tool to re-import notes after they have been updated - you should consider this tool to be a one-time transformation of an ENEX file into an equi… | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-evernote-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-evernote-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>evernote-to-sqlite</h1> <p><a href="https://pypi.org/project/evernote-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/0035842afd045b61cc5c6ee9c8394c181ac4daf3530bf212349f37a05bebcdbb/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f657665726e6f74652d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/evernote-to-sqlite.svg" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/evernote-to-sqlite/releases"><img src="https://camo.githubusercontent.com/b51eee78bbc51296102af124839ed665bf3f17755f4dbff95c4bc8ed0713d763/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f646f6773686565702f657665726e6f74652d746f2d73716c6974653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/dogsheep/evernote-to-sqlite?include_prereleases&label=changelog" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/evernote-to-sqlite/actions?query=workflow%3ATest"><img src="https://github.com/dogsheep/evernote-to-sqlite/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/evernote-to-sqlite/blob/master/LICENSE"><img src="https://camo.g… | ||||||||
| 305199661 | MDEwOlJlcG9zaXRvcnkzMDUxOTk2NjE= | sphinx-to-sqlite | simonw/sphinx-to-sqlite | 0 | simonw 9599 | https://github.com/simonw/sphinx-to-sqlite | Create a SQLite database from Sphinx documentation | 0 | 2020-10-18T21:26:55Z | 2020-12-19T05:08:12Z | 2020-10-22T04:55:45Z | 9 | 2 | 2 | Python | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 2 | Apache License 2.0 apache-2.0 | ["sqlite", "sphinx", "datasette-io", "datasette-tool"] | 0 | 2 | 2 | main | {"admin": false, "push": false, "pull": false} | 0 | 2 | # sphinx-to-sqlite [](https://pypi.org/project/sphinx-to-sqlite/) [](https://github.com/simonw/sphinx-to-sqlite/releases) [](https://github.com/simonw/sphinx-to-sqlite/actions?query=workflow%3ATest) [](https://github.com/simonw/sphinx-to-sqlite/blob/master/LICENSE) Create a SQLite database from Sphinx documentation. ## Demo You can see the results of running this tool against the [Datasette documentation](https://docs.datasette.io/) at https://latest-docs.datasette.io/docs/sections ## Installation Install this tool using `pip`: $ pip install sphinx-to-sqlite ## Usage First run `sphinx-build` with the `-b xml` option to create XML files in your `_build/` directory. Then run: $ sphinx-to-sqlite docs.db path/to/_build To build the SQLite database. ## Development To contribute to this tool, first checkout the code. Then create a new virtual environment: cd sphinx-to-sqlite python -mvenv venv source venv/bin/activate Or if you are using `pipenv`: pipenv shell Now install the dependencies and tests: pip install -e '.[test]' To run the tests: pytest | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-sphinx-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-sphinx-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>sphinx-to-sqlite</h1> <p><a href="https://pypi.org/project/sphinx-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/877d22c6402df75a56257c7d5a426d35ee787ab139307e4a7060c7d706b4c6cd/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f737068696e782d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/sphinx-to-sqlite.svg" style="max-width:100%;"></a> <a href="https://github.com/simonw/sphinx-to-sqlite/releases"><img src="https://camo.githubusercontent.com/69509856ca4dd51d3c9e67530adbd0f1f662719a608252a8d9cb5bdc3b90590b/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f737068696e782d746f2d73716c6974653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/sphinx-to-sqlite?include_prereleases&label=changelog" style="max-width:100%;"></a> <a href="https://github.com/simonw/sphinx-to-sqlite/actions?query=workflow%3ATest"><img src="https://github.com/simonw/sphinx-to-sqlite/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a> <a href="https://github.com/simonw/sphinx-to-sqlite/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0… | |||||||||
| 335372050 | MDEwOlJlcG9zaXRvcnkzMzUzNzIwNTA= | download-tiles | simonw/download-tiles | 0 | simonw 9599 | https://github.com/simonw/download-tiles | Download map tiles and store them in an MBTiles database | 0 | 2021-02-02T17:37:49Z | 2021-05-29T07:22:58Z | 2021-02-16T04:19:59Z | https://datasette.io/tools/download-tiles | 26 | 9 | 9 | Python | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | Apache License 2.0 apache-2.0 | ["openstreetmap", "mbtiles", "datasette-io", "datasette-tool"] | 0 | 0 | 9 | main | {"admin": false, "push": false, "pull": false} | 0 | 1 | # download-tiles [](https://pypi.org/project/download-tiles/) [](https://github.com/simonw/download-tiles/releases) [](https://github.com/simonw/download-tiles/actions?query=workflow%3ATest) [](https://github.com/simonw/download-tiles/blob/master/LICENSE) Download map tiles and store them in an MBTiles database ## Installation Install this tool using `pip`: $ pip install download-tiles ## Usage This tool downloads tiles from a specified [TMS (Tile Map Server)](https://wiki.openstreetmap.org/wiki/TMS) server for a specified bounding box and range of zoom levels and stores those tiles in a MBTiles SQLite database. It is a command-line wrapper around the [Landez](https://github.com/makinacorpus/landez) Python libary. **Please use this tool responsibly**. Consult the usage policies of the tile servers you are interacting with, for example the [OpenStreetMap Tile Usage Policy](https://operations.osmfoundation.org/policies/tiles/). Running the following will download zoom levels 0-3 of OpenStreetMap, 85 tiles total, and store them in a SQLite database called `world.mbtiles`: download-tiles world.mbtiles You can customize which tile and zoom levels are downloaded using command options: `--zoom-levels=0-3` or `-z=0-3` The different zoom levels to download. Specify a single number, e.g. `15`, or a range of numbers e.g. `0-4`. Be careful with this setting as you can easily go over the limits requested by the underlying tile server. `--bbox=3.9,-6.3,14.5,10.2` or `-b=3.9,-6.3,14.5,10.2` The bounding box to fetch. Should be specified as `min-lon,min-lat,max-lon,max-lat`. You can use [bboxfinder.com](http://bboxfinder.com/) to find these for different areas. `--city=london` or `--country=madagas… | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-download-tiles" class="anchor" aria-hidden="true" href="#user-content-download-tiles"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>download-tiles</h1> <p><a href="https://pypi.org/project/download-tiles/" rel="nofollow"><img src="https://camo.githubusercontent.com/82c46313526394b774727137f12ac6fbf6606364edbee19ebd99c951953b04b5/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f646f776e6c6f61642d74696c65732e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/download-tiles.svg" style="max-width:100%;"></a> <a href="https://github.com/simonw/download-tiles/releases"><img src="https://camo.githubusercontent.com/32d363e282d2f95ba1b135630d328b8b61459c3e97b2b4e2fe1ec629be13d80e/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f646f776e6c6f61642d74696c65733f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/download-tiles?include_prereleases&label=changelog" style="max-width:100%;"></a> <a href="https://github.com/simonw/download-tiles/actions?query=workflow%3ATest"><img src="https://github.com/simonw/download-tiles/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a> <a href="https://github.com/simonw/download-tiles/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7… | ||||||||
| 346597557 | MDEwOlJlcG9zaXRvcnkzNDY1OTc1NTc= | tableau-to-sqlite | simonw/tableau-to-sqlite | 0 | simonw 9599 | https://github.com/simonw/tableau-to-sqlite | Fetch data from Tableau into a SQLite database | 0 | 2021-03-11T06:12:02Z | 2021-06-10T04:40:44Z | 2021-04-29T16:11:03Z | 212 | 8 | 8 | Python | 1 | 1 | 1 | 1 | 0 | 2 | 0 | 0 | 2 | Apache License 2.0 apache-2.0 | ["datasette-io", "datasette-tool"] | 2 | 2 | 8 | main | {"admin": false, "push": false, "pull": false} | 2 | 1 | # tableau-to-sqlite [](https://pypi.org/project/tableau-to-sqlite/) [](https://github.com/simonw/tableau-to-sqlite/releases) [](https://github.com/simonw/tableau-to-sqlite/actions?query=workflow%3ATest) [](https://github.com/simonw/tableau-to-sqlite/blob/master/LICENSE) Fetch data from Tableau into a SQLite database. A wrapper around [TableauScraper](https://github.com/bertrandmartel/tableau-scraping/). ## Installation Install this tool using `pip`: $ pip install tableau-to-sqlite ## Usage If you have the URL to a Tableau dashboard like this: https://results.mo.gov/t/COVID19/views/VaccinationsDashboard/Vaccinations You can pass that directly to the tool: tableau-to-sqlite tableau.db \ https://results.mo.gov/t/COVID19/views/VaccinationsDashboard/Vaccinations This will create a SQLite database called `tableau.db` containing one table for each of the worksheets in that dashboard. If the dashboard is hosted on https://public.tableau.com/ you can instead provide the view name. This will be two strings separated by a `/` symbol - something like this: OregonCOVID-19VaccineProviderEnrollment/COVID-19VaccineProviderEnrollment Now run the tool like this: tableau-to-sqlite tableau.db \ OregonCOVID-19VaccineProviderEnrollment/COVID-19VaccineProviderEnrollment ## Get the data as JSON or CSV If you're building a [git scraper](https://simonwillison.net/2020/Oct/9/git-scraping/) you may want to convert the data gathered by this tool to CSV or JSON to check into your repository. You can do that using [sqlite-utils](https://sqlite-utils.datasette.io/). Install it using `pip`: pip install sqlite-utils You can dump out a table as JSON like so: sqlite-utils ro… | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-tableau-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-tableau-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>tableau-to-sqlite</h1> <p><a href="https://pypi.org/project/tableau-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/bdda6efc6980f655c99f0475b322f27bbc413e7e13e847e55e697dd135d10601/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f7461626c6561752d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/tableau-to-sqlite.svg" style="max-width:100%;"></a> <a href="https://github.com/simonw/tableau-to-sqlite/releases"><img src="https://camo.githubusercontent.com/fd18da3a8541734c627a9be68134477abe876a8695165916ff7c8c5bbf7269eb/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f7461626c6561752d746f2d73716c6974653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/tableau-to-sqlite?include_prereleases&label=changelog" style="max-width:100%;"></a> <a href="https://github.com/simonw/tableau-to-sqlite/actions?query=workflow%3ATest"><img src="https://github.com/simonw/tableau-to-sqlite/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a> <a href="https://github.com/simonw/tableau-to-sqlite/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104… | |||||||||
| 347263722 | MDEwOlJlcG9zaXRvcnkzNDcyNjM3MjI= | django-sql-dashboard | simonw/django-sql-dashboard | 0 | simonw 9599 | https://github.com/simonw/django-sql-dashboard | Django app for building dashboards using raw SQL queries | 0 | 2021-03-13T03:38:23Z | 2022-04-19T01:13:12Z | 2022-04-20T00:27:39Z | https://django-sql-dashboard.datasette.io/ | 513 | 335 | 335 | Python | 1 | 1 | 1 | 1 | 0 | 28 | 0 | 0 | 25 | Apache License 2.0 apache-2.0 | ["dashboards", "datasette-io", "datasette-tool", "django", "sql"] | 28 | 25 | 335 | main | {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false} | 28 | 9 | # django-sql-dashboard [](https://pypi.org/project/django-sql-dashboard/) [](https://github.com/simonw/django-sql-dashboard/releases) [](https://github.com/simonw/django-sql-dashboard/actions?query=workflow%3ATest) [](http://django-sql-dashboard.datasette.io/en/latest/?badge=latest) [](https://github.com/simonw/django-sql-dashboard/blob/main/LICENSE) Django SQL Dashboard provides an authenticated interface for executing read-only SQL queries directly against your PostgreSQL database, bringing a useful subset of [Datasette](https://datasette.io/) to Django. Applications include ad-hoc analysis and debugging, plus the creation of reporting dashboards that can be shared with team members or published online. See my blog for [more about this project](https://simonwillison.net/2021/May/10/django-sql-dashboard/), including [a video demo](https://www.youtube.com/watch?v=ausrmMZkPEY). Features include: - Safely run read-only one or more SQL queries against your database and view the results in your browser - Bookmark queries and share those links with other members of your team - Create [saved dashboards](https://django-sql-dashboard.datasette.io/en/latest/saved-dashboards.html) from your queries, with full control over who can view and edit them - [Named parameters](https://django-sql-dashboard.datasette.io/en/latest/sql.html#sql-parameters) such as `select * from entries where id = %(id)s` will be turned into form fields, allowing quick creation of interactive dashboards - Produce [bar charts](https://django-sql-dashboard.datasette.io/en/latest/widgets.html#bar-label-bar-quantity), [progres… | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-django-sql-dashboard" class="anchor" aria-hidden="true" href="#user-content-django-sql-dashboard"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>django-sql-dashboard</h1> <p dir="auto"><a href="https://pypi.org/project/django-sql-dashboard/" rel="nofollow"><img src="https://camo.githubusercontent.com/87a7771e261b30e7b0da09a83f2d6120ce484b068a0d844a41d2f945141194d1/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f646a616e676f2d73716c2d64617368626f6172642e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/django-sql-dashboard.svg" style="max-width: 100%;"></a> <a href="https://github.com/simonw/django-sql-dashboard/releases"><img src="https://camo.githubusercontent.com/f3c931ccf5487f5df160f339f767aba9ac45e15a5f4e8beb37fd6e33cbc239c0/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f73696d6f6e772f646a616e676f2d73716c2d64617368626f6172643f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/simonw/django-sql-dashboard?include_prereleases&label=changelog" style="max-width: 100%;"></a> <a href="https://github.com/simonw/django-sql-dashboard/actions?query=workflow%3ATest"><img src="https://github.com/simonw/django-sql-dashboard/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a> <a href="http://django-sql-dashboard.datasette.io/en/latest/?b… | 1 | public | 0 |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [repos] (
[id] INTEGER PRIMARY KEY,
[node_id] TEXT,
[name] TEXT,
[full_name] TEXT,
[private] INTEGER,
[owner] INTEGER REFERENCES [users]([id]),
[html_url] TEXT,
[description] TEXT,
[fork] INTEGER,
[created_at] TEXT,
[updated_at] TEXT,
[pushed_at] TEXT,
[homepage] TEXT,
[size] INTEGER,
[stargazers_count] INTEGER,
[watchers_count] INTEGER,
[language] TEXT,
[has_issues] INTEGER,
[has_projects] INTEGER,
[has_downloads] INTEGER,
[has_wiki] INTEGER,
[has_pages] INTEGER,
[forks_count] INTEGER,
[archived] INTEGER,
[disabled] INTEGER,
[open_issues_count] INTEGER,
[license] TEXT,
[topics] TEXT,
[forks] INTEGER,
[open_issues] INTEGER,
[watchers] INTEGER,
[default_branch] TEXT,
[permissions] TEXT,
[temp_clone_token] TEXT,
[organization] INTEGER REFERENCES [users]([id]),
[network_count] INTEGER,
[subscribers_count] INTEGER, [readme] TEXT, [readme_html] TEXT, [allow_forking] INTEGER, [visibility] TEXT, [is_template] INTEGER, [template_repository] TEXT, [web_commit_signoff_required] INTEGER, [has_discussions] INTEGER,
FOREIGN KEY([license]) REFERENCES [licenses]([key])
);
CREATE INDEX [idx_repos_license]
ON [repos] ([license]);
CREATE INDEX [idx_repos_organization]
ON [repos] ([organization]);
CREATE INDEX [idx_repos_owner]
ON [repos] ([owner]);