repos
12 rows where organization = 53015001
This data as json, CSV (advanced)
Suggested facets: homepage, has_projects, has_wiki, forks_count, open_issues_count, forks, open_issues, default_branch, permissions, network_count, subscribers_count, created_at (date), updated_at (date), pushed_at (date), topics (array)
| id ▼ | node_id | name | full_name | private | owner | html_url | description | fork | created_at | updated_at | pushed_at | homepage | size | stargazers_count | watchers_count | language | has_issues | has_projects | has_downloads | has_wiki | has_pages | forks_count | archived | disabled | open_issues_count | license | topics | forks | open_issues | watchers | default_branch | permissions | temp_clone_token | organization | network_count | subscribers_count | readme | readme_html | allow_forking | visibility | is_template | template_repository | web_commit_signoff_required | has_discussions |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 197431109 | MDEwOlJlcG9zaXRvcnkxOTc0MzExMDk= | dogsheep-beta | dogsheep/dogsheep-beta | 0 | dogsheep 53015001 | https://github.com/dogsheep/dogsheep-beta | Build a search index across content from multiple SQLite database tables and run faceted searches against it using Datasette | 0 | 2019-07-17T17:07:26Z | 2021-06-13T14:39:01Z | 2021-06-13T14:38:59Z | https://dogsheep.github.io/ | 61 | 78 | 78 | Python | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 11 | ["search", "datasette", "datasette-plugin", "dogsheep", "datasette-io", "datasette-tool"] | 0 | 11 | 78 | main | {"admin": false, "push": false, "pull": false} | dogsheep 53015001 | 0 | 4 | # dogsheep-beta [](https://pypi.org/project/dogsheep-beta/) [](https://github.com/dogsheep/beta/releases) [](https://github.com/dogsheep/beta/actions?query=workflow%3ATest) [](https://github.com/dogsheep/beta/blob/main/LICENSE) Build a search index across content from multiple SQLite database tables and run faceted searches against it using Datasette ## Example A live example of this plugin is running at https://datasette.io/-/beta - configured using [this YAML file](https://github.com/simonw/datasette.io/blob/main/templates/dogsheep-beta.yml). Read more about how this example works in [Building a search engine for datasette.io](https://simonwillison.net/2020/Dec/19/dogsheep-beta/). ## Installation Install this tool like so: $ pip install dogsheep-beta ## Usage Run the indexer using the `dogsheep-beta` command-line tool: $ dogsheep-beta index dogsheep.db config.yml The `config.yml` file contains details of the databases and document types that should be indexed: ```yaml twitter.db: tweets: sql: |- select tweets.id as key, 'Tweet by @' || users.screen_name as title, tweets.created_at as timestamp, tweets.full_text as search_1 from tweets join users on tweets.user = users.id users: sql: |- select id as key, name || ' @' || screen_name as title, created_at as timestamp, description as search_1 from users ``` This will create a `search_index` table in the `dogsheep.db` database populated by data from those SQL queries. By default the search index that this tool creates will be configured for Porter stemming.… | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-dogsheep-beta" class="anchor" aria-hidden="true" href="#user-content-dogsheep-beta"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>dogsheep-beta</h1> <p><a href="https://pypi.org/project/dogsheep-beta/" rel="nofollow"><img src="https://camo.githubusercontent.com/f8d17c422fbd622ab366c57086226b7f90b6ab9057aa2f3b9f6844dd29bac733/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f646f6773686565702d626574612e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/dogsheep-beta.svg" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/beta/releases"><img src="https://camo.githubusercontent.com/341e696d2cba7197bea24e27cc6db041753c20587a00ced21e77516d26a54730/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f646f6773686565702f626574613f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/dogsheep/beta?include_prereleases&label=changelog" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/beta/actions?query=workflow%3ATest"><img src="https://github.com/dogsheep/beta/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/beta/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f626… | ||||||||
| 197882382 | MDEwOlJlcG9zaXRvcnkxOTc4ODIzODI= | healthkit-to-sqlite | dogsheep/healthkit-to-sqlite | 0 | dogsheep 53015001 | https://github.com/dogsheep/healthkit-to-sqlite | Convert an Apple Healthkit export zip to a SQLite database | 0 | 2019-07-20T05:03:12Z | 2021-08-20T00:55:34Z | 2021-08-20T00:56:17Z | https://datasette.io/tools/healthkit-to-sqlite | 29 | 91 | 91 | Python | 1 | 1 | 1 | 1 | 0 | 4 | 0 | 0 | 8 | Apache License 2.0 apache-2.0 | ["datasette", "datasette-io", "datasette-tool", "dogsheep", "healthkit", "sqlite"] | 4 | 8 | 91 | main | {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false} | dogsheep 53015001 | 4 | 3 | # healthkit-to-sqlite [](https://pypi.org/project/healthkit-to-sqlite/) [](https://github.com/dogsheep/healthkit-to-sqlite/releases) [](https://github.com/dogsheep/healthkit-to-sqlite/actions?query=workflow%3ATest) [](https://github.com/dogsheep/healthkit-to-sqlite/blob/main/LICENSE) Convert an Apple Healthkit export zip to a SQLite database ## How to install $ pip install healthkit-to-sqlite ## How to use First you need to export your Apple HealthKit data. 1. On your iPhone, open the "Health" app 2. Click the profile icon in the top right 3. Click "Export Health Data" at the bottom of that page 4. Save the resulting file somewhere you can access it, or AirDrop it directly to your laptop. Now you can convert the resulting `export.zip` file to SQLite like so: $ healthkit-to-sqlite export.zip healthkit.db A progress bar will be displayed. You can disable this using `--silent`. ``` Importing from HealthKit [#-------------] 5% 00:01:33 ``` You can explore the resulting data using [Datasette](https://datasette.readthedocs.io/) like this: $ datasette healthkit.db | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-healthkit-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-healthkit-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>healthkit-to-sqlite</h1> <p><a href="https://pypi.org/project/healthkit-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/d8fd72edb8183afd279306fabff220e4c1670906c58c256e9e3bd7fbdea8c76f/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6865616c74686b69742d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/healthkit-to-sqlite.svg" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/healthkit-to-sqlite/releases"><img src="https://camo.githubusercontent.com/9092a0b3a68a53e13d8470fdf4909dda3cabe09e9e74b53960ad2063c79ae8e4/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f646f6773686565702f6865616c74686b69742d746f2d73716c6974653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/dogsheep/healthkit-to-sqlite?include_prereleases&label=changelog" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/healthkit-to-sqlite/actions?query=workflow%3ATest"><img src="https://github.com/dogsheep/healthkit-to-sqlite/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/healthkit-to-sqlite/blob/main/LICENSE"><img src="ht… | |||||||
| 205429375 | MDEwOlJlcG9zaXRvcnkyMDU0MjkzNzU= | swarm-to-sqlite | dogsheep/swarm-to-sqlite | 0 | dogsheep 53015001 | https://github.com/dogsheep/swarm-to-sqlite | Create a SQLite database containing your checkin history from Foursquare Swarm | 0 | 2019-08-30T17:37:29Z | 2021-02-22T07:58:39Z | 2021-01-18T04:36:03Z | 49 | 37 | 37 | Python | 1 | 1 | 1 | 1 | 0 | 1 | 0 | 0 | 1 | Apache License 2.0 apache-2.0 | ["sqlite", "foursquare", "swarm", "foursquare-api", "datasette", "dogsheep", "datasette-io", "datasette-tool"] | 1 | 1 | 37 | main | {"admin": false, "push": false, "pull": false} | dogsheep 53015001 | 1 | 3 | # swarm-to-sqlite [](https://pypi.org/project/swarm-to-sqlite/) [](https://github.com/dogsheep/swarm-to-sqlite/releases) [](https://github.com/dogsheep/swarm-to-sqlite/actions?query=workflow%3ATest) [](https://github.com/dogsheep/swarm-to-sqlite/blob/main/LICENSE) Create a SQLite database containing your checkin history from Foursquare Swarm. ## How to install $ pip install swarm-to-sqlite ## Usage You will need to first obtain a valid OAuth token for your Foursquare account. You can do so using this tool: https://your-foursquare-oauth-token.glitch.me/ Simplest usage is to simply provide the name of the database file you wish to write to. The tool will prompt you to paste in your token, and will then download your checkins and store them in the specified database file. $ swarm-to-sqlite checkins.db Please provide your Foursquare OAuth token: Importing 3699 checkins [#########-----------------------] 27% 00:02:31 You can also pass the token as a command-line option: $ swarm-to-sqlite checkins.db --token=XXX Or as an environment variable: $ export FOURSQUARE_TOKEN=XXX $ swarm-to-sqlite checkins.db To retrieve just checkins within the past X hours, days or weeks, use the `--since=` option. For example, to pull only checkins that happened within the last 10 days use: $ swarm-to-sqlite checkins.db --token=XXX --since=10d Use `2w` for two weeks, `10h` for ten hours, `3d` for three days. In addition to saving the checkins to a database, you can also write them to a JSON file using the `--save` option: $ swarm-to-sqlite checkins.db --save=checkins.json Having done this, you can re-import checkins directly from that file (rather than making API calls to … | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-swarm-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-swarm-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>swarm-to-sqlite</h1> <p><a href="https://pypi.org/project/swarm-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/336537cfcc544f29699c00a2aa4b5a199c9a21f53e43aff833e268e664735ed9/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f737761726d2d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/swarm-to-sqlite.svg" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/swarm-to-sqlite/releases"><img src="https://camo.githubusercontent.com/3a244282c89501bdea48af0387ebed5c0a03d41ef8f87331ed3fe63421f42c4f/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f646f6773686565702f737761726d2d746f2d73716c6974653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/dogsheep/swarm-to-sqlite?include_prereleases&label=changelog" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/swarm-to-sqlite/actions?query=workflow%3ATest"><img src="https://github.com/dogsheep/swarm-to-sqlite/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/swarm-to-sqlite/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb084… | ||||||||
| 206156866 | MDEwOlJlcG9zaXRvcnkyMDYxNTY4NjY= | twitter-to-sqlite | dogsheep/twitter-to-sqlite | 0 | dogsheep 53015001 | https://github.com/dogsheep/twitter-to-sqlite | Save data from Twitter to a SQLite database | 0 | 2019-09-03T19:30:08Z | 2021-12-26T18:08:43Z | 2021-12-26T18:08:40Z | 298 | 269 | 269 | Python | 1 | 1 | 1 | 1 | 0 | 13 | 0 | 0 | 10 | Apache License 2.0 apache-2.0 | ["datasette", "datasette-io", "datasette-tool", "dogsheep", "sqlite", "twitter", "twitter-api"] | 13 | 10 | 269 | main | {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false} | dogsheep 53015001 | 13 | 5 | # twitter-to-sqlite [](https://pypi.org/project/twitter-to-sqlite/) [](https://github.com/dogsheep/twitter-to-sqlite/releases) [](https://github.com/dogsheep/twitter-to-sqlite/actions?query=workflow%3ATest) [](https://github.com/dogsheep/twitter-to-sqlite/blob/main/LICENSE) Save data from Twitter to a SQLite database. **This tool currently uses Twitter API v1**. You may be unable to use it if you do not have an API key for that version of the API. <!-- toc --> - [How to install](#how-to-install) - [Authentication](#authentication) - [Retrieving tweets by specific accounts](#retrieving-tweets-by-specific-accounts) - [Retrieve user profiles in bulk](#retrieve-user-profiles-in-bulk) - [Retrieve tweets in bulk](#retrieve-tweets-in-bulk) - [Retrieving Twitter followers](#retrieving-twitter-followers) - [Retrieving friends](#retrieving-friends) - [Retrieving favorited tweets](#retrieving-favorited-tweets) - [Retrieving Twitter lists](#retrieving-twitter-lists) - [Retrieving Twitter list memberships](#retrieving-twitter-list-memberships) - [Retrieving just follower and friend IDs](#retrieving-just-follower-and-friend-ids) - [Retrieving tweets from your home timeline](#retrieving-tweets-from-your-home-timeline) - [Retrieving your mentions](#retrieving-your-mentions) - [Providing input from a SQL query with --sql and --attach](#providing-input-from-a-sql-query-with---sql-and---attach) - [Running searches](#running-searches) - [Capturing tweets in real-time with track and follow](#capturing-tweets-in-real-time-with-track-and-follow) * [track](#track) * [follow](#follow) - [Importing data from your Twitter archive](#importing-data-from-your-twitter-archive) - [Design notes](#design-notes) <!-- … | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-twitter-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-twitter-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>twitter-to-sqlite</h1> <p dir="auto"><a href="https://pypi.org/project/twitter-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/fce3d286daf4a0ac037476f32a7f1885dd7f90329ec1df80f7fe6b9322c72f5c/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f747769747465722d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/twitter-to-sqlite.svg" style="max-width: 100%;"></a> <a href="https://github.com/dogsheep/twitter-to-sqlite/releases"><img src="https://camo.githubusercontent.com/e1228185d86e3eb446efcb27ff81748cdad3b90ac14ae58c61dab61a90bb992d/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f646f6773686565702f747769747465722d746f2d73716c6974653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/dogsheep/twitter-to-sqlite?include_prereleases&label=changelog" style="max-width: 100%;"></a> <a href="https://github.com/dogsheep/twitter-to-sqlite/actions?query=workflow%3ATest"><img src="https://github.com/dogsheep/twitter-to-sqlite/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a> <a href="https://github.com/dogsheep/twitter-to-sqlite/blob/main/LICENSE"><img src="https… | 1 | public | 0 | |||||
| 206202864 | MDEwOlJlcG9zaXRvcnkyMDYyMDI4NjQ= | inaturalist-to-sqlite | dogsheep/inaturalist-to-sqlite | 0 | dogsheep 53015001 | https://github.com/dogsheep/inaturalist-to-sqlite | Create a SQLite database containing your observation history from iNaturalist | 0 | 2019-09-04T01:21:21Z | 2020-12-19T05:18:38Z | 2020-10-22T00:08:58Z | 17 | 2 | 2 | Python | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | Apache License 2.0 apache-2.0 | ["sqlite", "inaturalist", "datasette", "dogsheep", "datasette-io", "datasette-tool"] | 0 | 0 | 2 | master | {"admin": false, "push": false, "pull": false} | dogsheep 53015001 | 0 | 1 | # inaturalist-to-sqlite [](https://pypi.org/project/inaturalist-to-sqlite/) [](https://circleci.com/gh/dogsheep/inaturalist-to-sqlite) [](https://github.com/dogsheep/inaturalist-to-sqlite/blob/master/LICENSE) Create a SQLite database containing your observation history from [iNaturalist](https://www.inaturalist.org/). ## How to install $ pip install inaturalist-to-sqlite ## Usage $ inaturalist-to-sqlite inaturalist.db yourusername (Or try `simonw` if you don't yet have an iNaturalist account) This will import all of your iNaturalist observations into a SQLite database called `inaturalist.db`. | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-inaturalist-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-inaturalist-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>inaturalist-to-sqlite</h1> <p><a href="https://pypi.org/project/inaturalist-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/0b4aee6bb6f3aeb706c5195fce3537b66445f052f38021aefeb48672cf06cf74/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f696e61747572616c6973742d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/inaturalist-to-sqlite.svg" style="max-width:100%;"></a> <a href="https://circleci.com/gh/dogsheep/inaturalist-to-sqlite" rel="nofollow"><img src="https://camo.githubusercontent.com/96dcb5c0cfa03bf010686f57c71dfef278ab6aaa53eb39cb48d28bde427c55b7/68747470733a2f2f636972636c6563692e636f6d2f67682f646f6773686565702f696e61747572616c6973742d746f2d73716c6974652e7376673f7374796c653d737667" alt="CircleCI" data-canonical-src="https://circleci.com/gh/dogsheep/inaturalist-to-sqlite.svg?style=svg" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/inaturalist-to-sqlite/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%2… | ||||||||
| 206649770 | MDEwOlJlcG9zaXRvcnkyMDY2NDk3NzA= | google-takeout-to-sqlite | dogsheep/google-takeout-to-sqlite | 0 | dogsheep 53015001 | https://github.com/dogsheep/google-takeout-to-sqlite | Save data from Google Takeout to a SQLite database | 0 | 2019-09-05T20:15:15Z | 2021-06-08T15:31:47Z | 2021-02-24T00:34:55Z | 14 | 51 | 51 | Python | 1 | 1 | 1 | 1 | 0 | 4 | 0 | 0 | 6 | Apache License 2.0 apache-2.0 | ["google", "sqlite", "datasette", "dogsheep", "datasette-io", "datasette-tool"] | 4 | 6 | 51 | master | {"admin": false, "push": false, "pull": false} | dogsheep 53015001 | 4 | 3 | # google-takeout-to-sqlite [](https://pypi.org/project/google-takeout-to-sqlite/) [](https://circleci.com/gh/dogsheep/google-takeout-to-sqlite) [](https://github.com/dogsheep/google-takeout-to-sqlite/blob/master/LICENSE) Save data from google-takeout to a SQLite database. ## How to install $ pip install google-takeout-to-sqlite Request your Google data from https://takeout.google.com/ - wait for the email and download the zip file. This tool only supports a subset of the available options. More will be added over time. ## My Activity You can request the "My Activity" export and then import it with the following command: $ google-takeout-to-sqlite my-activity takeout.db ~/Downloads/takeout-20190530.zip This will create a database file called `takeout.db` if one does not already exist. ## Location History Your location history records latitude, longitude and timestame for where Google has tracked your location. You can import it using this command: $ google-takeout-to-sqlite location-history takeout.db ~/Downloads/takeout-20190530.zip ## Browsing your data with Datasette Once you have imported Google data into a SQLite database file you can browse your data using [Datasette](https://github.com/simonw/datasette). Install Datasette like so: $ pip install datasette Now browse your data by running this and then visiting `http://localhost:8001/` $ datasette takeout.db Install the [datasette-cluster-map](https://github.com/simonw/datasette-cluster-map) plugin to see your location history on a map: $ pip install datasette-cluster-map | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-google-takeout-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-google-takeout-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>google-takeout-to-sqlite</h1> <p><a href="https://pypi.org/project/google-takeout-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/fb9e0c4c7734b2904f2b1aaaa4ce05fa195167989240be80a7c8e56e2b2a000e/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f676f6f676c652d74616b656f75742d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/google-takeout-to-sqlite.svg" style="max-width:100%;"></a> <a href="https://circleci.com/gh/dogsheep/google-takeout-to-sqlite" rel="nofollow"><img src="https://camo.githubusercontent.com/41bc34953e618d9a884ec9a8bad7d759bf580fc6b7c27982d3c4af4531121246/68747470733a2f2f636972636c6563692e636f6d2f67682f646f6773686565702f676f6f676c652d74616b656f75742d746f2d73716c6974652e7376673f7374796c653d737667" alt="CircleCI" data-canonical-src="https://circleci.com/gh/dogsheep/google-takeout-to-sqlite.svg?style=svg" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/google-takeout-to-sqlite/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://i… | ||||||||
| 207052882 | MDEwOlJlcG9zaXRvcnkyMDcwNTI4ODI= | github-to-sqlite | dogsheep/github-to-sqlite | 0 | dogsheep 53015001 | https://github.com/dogsheep/github-to-sqlite | Save data from GitHub to a SQLite database | 0 | 2019-09-08T02:50:28Z | 2022-09-20T04:36:37Z | 2022-09-28T21:07:54Z | https://github-to-sqlite.dogsheep.net/ | 143 | 235 | 235 | Python | 1 | 1 | 1 | 1 | 0 | 32 | 0 | 0 | 20 | Apache License 2.0 apache-2.0 | ["datasette", "datasette-io", "datasette-tool", "dogsheep", "github-api", "sqlite"] | 32 | 20 | 235 | main | {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false} | dogsheep 53015001 | 32 | 6 | # github-to-sqlite [](https://pypi.org/project/github-to-sqlite/) [](https://github.com/dogsheep/github-to-sqlite/releases) [](https://github.com/dogsheep/github-to-sqlite/actions?query=workflow%3ATest) [](https://github.com/dogsheep/github-to-sqlite/blob/main/LICENSE) Save data from GitHub to a SQLite database. <!-- toc --> - [Demo](#demo) - [How to install](#how-to-install) - [Authentication](#authentication) - [Fetching issues for a repository](#fetching-issues-for-a-repository) - [Fetching pull requests for a repository](#fetching-pull-requests-for-a-repository) - [Fetching issue comments for a repository](#fetching-issue-comments-for-a-repository) - [Fetching commits for a repository](#fetching-commits-for-a-repository) - [Fetching releases for a repository](#fetching-releases-for-a-repository) - [Fetching tags for a repository](#fetching-tags-for-a-repository) - [Fetching contributors to a repository](#fetching-contributors-to-a-repository) - [Fetching repos belonging to a user or organization](#fetching-repos-belonging-to-a-user-or-organization) - [Fetching specific repositories](#fetching-specific-repositories) - [Fetching repos that have been starred by a user](#fetching-repos-that-have-been-starred-by-a-user) - [Fetching users that have starred specific repos](#fetching-users-that-have-starred-specific-repos) - [Fetching GitHub Actions workflows](#fetching-github-actions-workflows) - [Scraping dependents for a repository](#scraping-dependents-for-a-repository) - [Fetching emojis](#fetching-emojis) - [Making authenticated API calls](#making-authenticated-api-calls) <!-- tocstop --> ## Demo https://github-to-sqlite.dogsheep.net/ hosts a [Datasette](https://datasette.io/) demo of a … | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-github-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-github-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>github-to-sqlite</h1> <p dir="auto"><a href="https://pypi.org/project/github-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/515e6efa4aef15e83b08072e5490c9040420223b2355d1d715cb628f66d60dff/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6769746875622d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/github-to-sqlite.svg" style="max-width: 100%;"></a> <a href="https://github.com/dogsheep/github-to-sqlite/releases"><img src="https://camo.githubusercontent.com/ae3fbf680cae0fca1e9126549e1fd0e14e756f6f77046e878db7b1c9cbd78911/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f646f6773686565702f6769746875622d746f2d73716c6974653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/dogsheep/github-to-sqlite?include_prereleases&label=changelog" style="max-width: 100%;"></a> <a href="https://github.com/dogsheep/github-to-sqlite/actions?query=workflow%3ATest"><img src="https://github.com/dogsheep/github-to-sqlite/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a> <a href="https://github.com/dogsheep/github-to-sqlite/blob/main/LICENSE"><img src="https://camo.github… | 1 | public | 0 | 0 | |||
| 209590345 | MDEwOlJlcG9zaXRvcnkyMDk1OTAzNDU= | genome-to-sqlite | dogsheep/genome-to-sqlite | 0 | dogsheep 53015001 | https://github.com/dogsheep/genome-to-sqlite | Import your genome into a SQLite database | 0 | 2019-09-19T15:38:39Z | 2021-01-18T19:39:48Z | 2019-09-19T15:41:17Z | 9 | 13 | 13 | Python | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 2 | Apache License 2.0 apache-2.0 | ["genetics", "sqlite", "23andme", "personal-analytics", "datasette", "dogsheep", "datasette-io", "datasette-tool"] | 0 | 2 | 13 | master | {"admin": false, "push": false, "pull": false} | dogsheep 53015001 | 0 | 2 | # genome-to-sqlite [](https://pypi.org/project/genome-to-sqlite/) [](https://circleci.com/gh/dogsheep/genome-to-sqlite) [](https://github.com/dogsheep/genome-to-sqlite/blob/master/LICENSE) Import your genome into a SQLite database. ## How to install $ pip install genome-to-sqlite ## How to use First, export your genome. This tool has only been tested against 23andMe so far. You can request an export of your genome from https://you.23andme.com/tools/data/download/ Now you can convert the resulting `export.zip` file to SQLite like so: $ genome-to-sqlite export.zip genome.db A progress bar will be displayed. You can disable this using `--silent`. ``` Importing genome [#----------------] 5% 00:01:33 ``` You can explore the resulting data using [Datasette](https://datasette.readthedocs.io/) like this: $ datasette genome.db --config facet_time_limit_ms:1000 Bumping up the facet time limit is useful in order to enable faceting by chromosome: http://127.0.0.1:8001/genome/genome?_facet=chromosome&_sort=position | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-genome-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-genome-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>genome-to-sqlite</h1> <p><a href="https://pypi.org/project/genome-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/a91d14ecfe9aa2d73da4818a7a898033e301380d4c8477d06a17dc20bd871c84/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f67656e6f6d652d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/genome-to-sqlite.svg" style="max-width:100%;"></a> <a href="https://circleci.com/gh/dogsheep/genome-to-sqlite" rel="nofollow"><img src="https://camo.githubusercontent.com/5f7f5be2791ff007c49d3d153ef38baa81770a11c0326e5f6b8ffd7ee939d00f/68747470733a2f2f636972636c6563692e636f6d2f67682f646f6773686565702f67656e6f6d652d746f2d73716c6974652e7376673f7374796c653d737667" alt="CircleCI" data-canonical-src="https://circleci.com/gh/dogsheep/genome-to-sqlite.svg?style=svg" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/genome-to-sqlite/blob/master/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p> <p>Import yo… | ||||||||
| 213286752 | MDEwOlJlcG9zaXRvcnkyMTMyODY3NTI= | pocket-to-sqlite | dogsheep/pocket-to-sqlite | 0 | dogsheep 53015001 | https://github.com/dogsheep/pocket-to-sqlite | Create a SQLite database containing data from your Pocket account | 0 | 2019-10-07T03:24:14Z | 2022-08-21T21:11:59Z | 2022-08-22T16:21:34Z | 20 | 63 | 63 | Python | 1 | 1 | 1 | 1 | 0 | 3 | 0 | 0 | 5 | Apache License 2.0 apache-2.0 | ["datasette", "datasette-io", "datasette-tool", "dogsheep", "pocket", "pocket-api", "sqlite"] | 3 | 5 | 63 | main | {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false} | dogsheep 53015001 | 3 | 4 | # pocket-to-sqlite [](https://pypi.org/project/pocket-to-sqlite/) [](https://github.com/dogsheep/pocket-to-sqlite/releases) [](https://github.com/dogsheep/pocket-to-sqlite/actions?query=workflow%3ATest) [](https://github.com/dogsheep/pocket-to-sqlite/blob/main/LICENSE) Create a SQLite database containing data from your [Pocket](https://getpocket.com/) account. ## How to install $ pip install pocket-to-sqlite ## Usage You will need to first obtain a valid OAuth token for your Pocket account. You can do this by running the `auth` command and following the prompts: $ pocket-to-sqlite auth Visit this page and sign in with your Pocket account: https://getpocket.com/auth/author... Once you have signed in there, hit <enter> to continue Authentication tokens written to auth.json Now you can fetch all of your items from Pocket like this: $ pocket-to-sqlite fetch pocket.db The first time you run this command it will fetch all of your items, and display a progress bar while it does it. On subsequent runs it will only fetch new items. You can force it to fetch everything from the beginning again using `--all`. Use `--silent` to disable the progress bar. ## Using with Datasette The SQLite database produced by this tool is designed to be browsed using [Datasette](https://datasette.readthedocs.io/). Use the [datasette-render-timestamps](https://github.com/simonw/datasette-render-timestamps) plugin to improve the display of the timestamp values. | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1 dir="auto"><a id="user-content-pocket-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-pocket-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>pocket-to-sqlite</h1> <p dir="auto"><a href="https://pypi.org/project/pocket-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/98050bc158f5e93f2c3576b03fa6a25b4a8091bb6d44774e07d95ae2842e4a3e/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f706f636b65742d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/pocket-to-sqlite.svg" style="max-width: 100%;"></a> <a href="https://github.com/dogsheep/pocket-to-sqlite/releases"><img src="https://camo.githubusercontent.com/b37c773b526e69ffcc72b9c02cbcd977d1b6e6a9bcf317c3accfc04b975e960c/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f646f6773686565702f706f636b65742d746f2d73716c6974653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/dogsheep/pocket-to-sqlite?include_prereleases&label=changelog" style="max-width: 100%;"></a> <a href="https://github.com/dogsheep/pocket-to-sqlite/actions?query=workflow%3ATest"><img src="https://github.com/dogsheep/pocket-to-sqlite/workflows/Test/badge.svg" alt="Tests" style="max-width: 100%;"></a> <a href="https://github.com/dogsheep/pocket-to-sqlite/blob/main/LICENSE"><img src="https://camo.github… | 1 | public | 0 | 0 | ||||
| 248903544 | MDEwOlJlcG9zaXRvcnkyNDg5MDM1NDQ= | hacker-news-to-sqlite | dogsheep/hacker-news-to-sqlite | 0 | dogsheep 53015001 | https://github.com/dogsheep/hacker-news-to-sqlite | Create a SQLite database containing data pulled from Hacker News | 0 | 2020-03-21T04:02:05Z | 2021-06-06T22:42:00Z | 2021-03-13T19:15:06Z | 19 | 25 | 25 | Python | 1 | 1 | 1 | 1 | 0 | 2 | 0 | 0 | 0 | Apache License 2.0 apache-2.0 | ["hacker-news", "datasette", "dogsheep", "datasette-io", "datasette-tool"] | 2 | 0 | 25 | main | {"admin": false, "push": false, "pull": false} | dogsheep 53015001 | 2 | 1 | # hacker-news-to-sqlite [](https://pypi.org/project/hacker-news-to-sqlite/) [](https://github.com/dogsheep/hacker-news-to-sqlite/releases) [](https://github.com/dogsheep/hacker-news-to-sqlite/actions?query=workflow%3ATest) [](https://github.com/simonw/hacker-news-to-sqlite/blob/main/LICENSE) Create a SQLite database containing data fetched from [Hacker News](https://news.ycombinator.com/). ## How to install $ pip install hacker-news-to-sqlite ## Usage $ hacker-news-to-sqlite user hacker-news.db your-username Importing items: 37%|███████████ | 845/2297 [05:09<11:02, 2.19it/s] Imports all of your Hacker News submissions and comments into a SQLite database called `hacker-news.db`. $ hacker-news-to-sqlite trees hacker-news.db 22640038 22643218 Fetches the entire comments tree in which any of those content IDs appears. ## Browsing your data with Datasette You can use [Datasette](https://datasette.readthedocs.org/) to browse your data. Install Datasette like this: $ pip install datasette Now run it against your `hacker-news.db` file like so: $ datasette hacker-news.db Visit `https://localhost:8001/` to search and explore your data. You can improve the display of your data usinng the [datasette-render-timestamps](https://github.com/simonw/datasette-render-timestamps) and [datasette-render-html](https://github.com/simonw/datasette-render-html) plugins. Install them like this: $ pip install datasette-render-timestamps datasette-render-html Now save the following configuration in a file called `metadata.json`: ```json { "databases": { "hacker-news": { "tables": { "items": { … | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-hacker-news-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-hacker-news-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>hacker-news-to-sqlite</h1> <p><a href="https://pypi.org/project/hacker-news-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/bf6d88d26ea4d8f1f396c1b1fc88ae74380de6a30f5a792c9f29664f21a219dd/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f6861636b65722d6e6577732d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/hacker-news-to-sqlite.svg" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/hacker-news-to-sqlite/releases"><img src="https://camo.githubusercontent.com/090ef507ee9402a786e40313d173f516f1dbdedf3df703402296594a4ebcad73/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f646f6773686565702f6861636b65722d6e6577732d746f2d73716c6974653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/dogsheep/hacker-news-to-sqlite?include_prereleases&label=changelog" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/hacker-news-to-sqlite/actions?query=workflow%3ATest"><img src="https://github.com/dogsheep/hacker-news-to-sqlite/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a> <a href="https://github.com/simonw/hacker-news-to-sqlite/blob/… | ||||||||
| 256834907 | MDEwOlJlcG9zaXRvcnkyNTY4MzQ5MDc= | dogsheep-photos | dogsheep/dogsheep-photos | 0 | dogsheep 53015001 | https://github.com/dogsheep/dogsheep-photos | Upload your photos to S3 and import metadata about them into a SQLite database | 0 | 2020-04-18T19:22:13Z | 2021-11-04T20:45:03Z | 2021-11-04T20:45:00Z | 68 | 124 | 124 | Python | 1 | 1 | 1 | 1 | 0 | 7 | 0 | 0 | 19 | Apache License 2.0 apache-2.0 | ["datasette", "datasette-io", "datasette-tool", "dogsheep", "sqlite"] | 7 | 19 | 124 | master | {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false} | dogsheep 53015001 | 7 | 10 | # dogsheep-photos [](https://pypi.org/project/dogsheep-photos/) [](https://github.com/dogsheep/dogsheep-photos/releases) [](https://circleci.com/gh/dogsheep/dogsheep-photos) [](https://github.com/dogsheep/dogsheep-photos/blob/master/LICENSE) Save details of your photos to a SQLite database and upload them to S3. See [Using SQL to find my best photo of a pelican according to Apple Photos](https://simonwillison.net/2020/May/21/apple-photos-sqlite/) for background information on this project. ## What these tools do These tools are a work-in-progress mechanism for taking full ownership of your photos. The core idea is to help implement the following: * Every photo you have taken lives in a single, private Amazon S3 bucket * You have a single SQLite database file which stores metadata about those photos - potentially pulled from multiple different places. This may include EXIF data, Apple Photos, the results of running machine learning APIs against photos and much more besides. * You can then use [Datasette](https://github.com/simonw/datasette) to explore your own photos. I'm a heavy user of Apple Photos so the initial releases of this tool will have a bias towards that, but ideally I would like a subset of these tools to be useful to people no matter which core photo solution they are using. ## Installation $ pip install dogsheep-photos ## Authentication (if using S3) If you want to use S3 to store your photos, you will need to first create S3 credentials for a new, dedicated bucket. You may find the [s3-credentials tool](https://github.com/simonw/s3-credentials) useful for this. Run this command and paste in your credentials. You will need three values: the name of your S3 bucket, your Ac… | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-dogsheep-photos" class="anchor" aria-hidden="true" href="#user-content-dogsheep-photos"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>dogsheep-photos</h1> <p><a href="https://pypi.org/project/dogsheep-photos/" rel="nofollow"><img src="https://camo.githubusercontent.com/c21d10e4250454707420755a0d4b90c97709771d795ffa55b495c98956a3938d/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f646f6773686565702d70686f746f732e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/dogsheep-photos.svg" style="max-width: 100%;"></a> <a href="https://github.com/dogsheep/dogsheep-photos/releases"><img src="https://camo.githubusercontent.com/c7bead2f1c989034f62914278a736eecdfbab79ce4d9b3ab804d1692e230d79d/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f646f6773686565702f646f6773686565702d70686f746f733f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/dogsheep/dogsheep-photos?include_prereleases&label=changelog" style="max-width: 100%;"></a> <a href="https://circleci.com/gh/dogsheep/dogsheep-photos" rel="nofollow"><img src="https://camo.githubusercontent.com/3aefe86d4a3560c1ca080cf5b7e4dcc5788b239df8b7a3fed40e622b834c22e8/68747470733a2f2f636972636c6563692e636f6d2f67682f646f6773686565702f646f6773686565702d70686f746f732e7376673f7374796c653d737667" alt="CircleCI" data-canonical-src… | 1 | public | 0 | |||||
| 303218369 | MDEwOlJlcG9zaXRvcnkzMDMyMTgzNjk= | evernote-to-sqlite | dogsheep/evernote-to-sqlite | 0 | dogsheep 53015001 | https://github.com/dogsheep/evernote-to-sqlite | Tools for converting Evernote content to SQLite | 0 | 2020-10-11T21:45:49Z | 2021-08-26T19:01:54Z | 2021-08-26T19:02:47Z | 51 | 24 | 24 | Python | 1 | 1 | 1 | 1 | 0 | 4 | 0 | 0 | 3 | Apache License 2.0 apache-2.0 | ["datasette-io", "datasette-tool", "dogsheep", "evernote", "sqlite"] | 4 | 3 | 24 | main | {"admin": false, "maintain": false, "push": false, "triage": false, "pull": false} | dogsheep 53015001 | 4 | 4 | # evernote-to-sqlite [](https://pypi.org/project/evernote-to-sqlite/) [](https://github.com/dogsheep/evernote-to-sqlite/releases) [](https://github.com/dogsheep/evernote-to-sqlite/actions?query=workflow%3ATest) [](https://github.com/dogsheep/evernote-to-sqlite/blob/master/LICENSE) Tools for converting Evernote content to SQLite. See [Building an Evernote to SQLite exporter](https://simonwillison.net/2020/Oct/16/building-evernote-sqlite-exporter/) for background on this project. ## Installation Install this tool using `pip`: $ pip install evernote-to-sqlite ## Usage Currently the only available command is `evernote-to-sqlite enex`, which converts Evernote's ENEX export files into a SQLite database. You can create [an ENEX export](https://help.evernote.com/hc/en-us/articles/209005557-Export-notes-and-notebooks-as-ENEX-or-HTML) in the Evernote desktop application by selecting some notes (or all of your notes) and using the `File -> Export Notes...` menu option. This used to be able to export everything in one go, but it looks like more recent Evernote versions only allow exporting up to fifty notes at a time, or let you export an entire notebook by right-clicking on the notebook and selecting "Export notebook...". You can convert that file to SQLite like so: $ evernote-to-sqlite enex evernote.db MyNotes.enex This will display a progress bar and create a SQLite database file called `evernote.db`. ### Limitations Unfortunately the ENEX export format does not include a unique identifier for each note. This means you cannot use this tool to re-import notes after they have been updated - you should consider this tool to be a one-time transformation of an ENEX file into an equi… | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-evernote-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-evernote-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>evernote-to-sqlite</h1> <p><a href="https://pypi.org/project/evernote-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/0035842afd045b61cc5c6ee9c8394c181ac4daf3530bf212349f37a05bebcdbb/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f657665726e6f74652d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/evernote-to-sqlite.svg" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/evernote-to-sqlite/releases"><img src="https://camo.githubusercontent.com/b51eee78bbc51296102af124839ed665bf3f17755f4dbff95c4bc8ed0713d763/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f646f6773686565702f657665726e6f74652d746f2d73716c6974653f696e636c7564655f70726572656c6561736573266c6162656c3d6368616e67656c6f67" alt="Changelog" data-canonical-src="https://img.shields.io/github/v/release/dogsheep/evernote-to-sqlite?include_prereleases&label=changelog" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/evernote-to-sqlite/actions?query=workflow%3ATest"><img src="https://github.com/dogsheep/evernote-to-sqlite/workflows/Test/badge.svg" alt="Tests" style="max-width:100%;"></a> <a href="https://github.com/dogsheep/evernote-to-sqlite/blob/master/LICENSE"><img src="https://camo.g… |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [repos] (
[id] INTEGER PRIMARY KEY,
[node_id] TEXT,
[name] TEXT,
[full_name] TEXT,
[private] INTEGER,
[owner] INTEGER REFERENCES [users]([id]),
[html_url] TEXT,
[description] TEXT,
[fork] INTEGER,
[created_at] TEXT,
[updated_at] TEXT,
[pushed_at] TEXT,
[homepage] TEXT,
[size] INTEGER,
[stargazers_count] INTEGER,
[watchers_count] INTEGER,
[language] TEXT,
[has_issues] INTEGER,
[has_projects] INTEGER,
[has_downloads] INTEGER,
[has_wiki] INTEGER,
[has_pages] INTEGER,
[forks_count] INTEGER,
[archived] INTEGER,
[disabled] INTEGER,
[open_issues_count] INTEGER,
[license] TEXT,
[topics] TEXT,
[forks] INTEGER,
[open_issues] INTEGER,
[watchers] INTEGER,
[default_branch] TEXT,
[permissions] TEXT,
[temp_clone_token] TEXT,
[organization] INTEGER REFERENCES [users]([id]),
[network_count] INTEGER,
[subscribers_count] INTEGER, [readme] TEXT, [readme_html] TEXT, [allow_forking] INTEGER, [visibility] TEXT, [is_template] INTEGER, [template_repository] TEXT, [web_commit_signoff_required] INTEGER, [has_discussions] INTEGER,
FOREIGN KEY([license]) REFERENCES [licenses]([key])
);
CREATE INDEX [idx_repos_license]
ON [repos] ([license]);
CREATE INDEX [idx_repos_organization]
ON [repos] ([organization]);
CREATE INDEX [idx_repos_owner]
ON [repos] ([owner]);