repos: 240815938
This data as json
| id | node_id | name | full_name | private | owner | html_url | description | fork | created_at | updated_at | pushed_at | homepage | size | stargazers_count | watchers_count | language | has_issues | has_projects | has_downloads | has_wiki | has_pages | forks_count | archived | disabled | open_issues_count | license | topics | forks | open_issues | watchers | default_branch | permissions | temp_clone_token | organization | network_count | subscribers_count | readme | readme_html | allow_forking | visibility | is_template | template_repository | web_commit_signoff_required | has_discussions |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 240815938 | MDEwOlJlcG9zaXRvcnkyNDA4MTU5Mzg= | shapefile-to-sqlite | simonw/shapefile-to-sqlite | 0 | 9599 | https://github.com/simonw/shapefile-to-sqlite | Load shapefiles into a SQLite (optionally SpatiaLite) database | 0 | 2020-02-16T01:55:29Z | 2021-03-26T08:39:43Z | 2020-08-23T06:00:41Z | 54 | 15 | 15 | Python | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 3 | apache-2.0 | ["sqlite", "gis", "spatialite", "shapefiles", "datasette", "datasette-io", "datasette-tool"] | 0 | 3 | 15 | main | {"admin": false, "push": false, "pull": false} | 0 | 1 | # shapefile-to-sqlite [](https://pypi.org/project/shapefile-to-sqlite/) [](https://circleci.com/gh/simonw/shapefile-to-sqlite) [](https://github.com/simonw/shapefile-to-sqlite/blob/main/LICENSE) Load shapefiles into a SQLite (optionally SpatiaLite) database. Project background: [Things I learned about shapefiles building shapefile-to-sqlite](https://simonwillison.net/2020/Feb/19/shapefile-to-sqlite/) ## How to install $ pip install shapefile-to-sqlite ## How to use You can run this tool against a shapefile file like so: $ shapefile-to-sqlite my.db features.shp This will load the geometries as GeoJSON in a text column. ## Using with SpatiaLite If you have [SpatiaLite](https://www.gaia-gis.it/fossil/libspatialite/index) available you can load them as SpatiaLite geometries like this: $ shapefile-to-sqlite my.db features.shp --spatialite The data will be loaded into a table called `features` - based on the name of the shapefile. You can specify an alternative table name using `--table`: $ shapefile-to-sqlite my.db features.shp --table=places --spatialite The tool will search for the SpatiaLite module in the following locations: - `/usr/lib/x86_64-linux-gnu/mod_spatialite.so` - `/usr/local/lib/mod_spatialite.dylib` If you have installed the module in another location, you can use the `--spatialite_mod=xxx` option to specify where: $ shapefile-to-sqlite my.db features.shp \ --spatialite_mod=/usr/lib/mod_spatialite.dylib You can use the `--spatial-index` option to create a spatial index on the `geometry` column: $ shapefile-to-sqlite my.db features.shp --spatial-index You can omit `--spatialite` if you use either `--spatialite-mod` or `--spatial-index`. ## Projections By default, this tool will attempt to convert geometries in the shapefile to the WGS 84 projection, for best conformance with the [GeoJSON specification](https://tools.ietf.org/html/rfc7946). If you want it to leave the data in whatever projection was used by the shapefile, use the `--crs=keep` option. You can convert the data to another output projection by passing it to the `--crs` option. For example, to convert to [EPSG:2227](https://epsg.io/2227) (California zone 3) use `--crs=espg:2227`. The full list of formats accepted by the `--crs` option is [documented here](https://pyproj4.github.io/pyproj/stable/api/crs.html#pyproj.crs.CRS.__init__). ## Extracting columns If your data contains columns with a small number of heavily duplicated values - the names of specific agencies responsible for parcels of land for example - you can extract those columns into separate lookup tables referenced by foreign keys using the `-c` option: $ shapefile-to-sqlite my.db features.shp -c agency This will create a `agency` table with `id` and `name` columns, and will create the `agency` column in your main table as an integer foreign key reference to that table. The `-c` option can be used multiple times. [CPAD_2020a_Units](https://calands.datasettes.com/calands/CPAD_2020a_Units) is an example of a table created using the `-c` option. | <div id="readme" class="md" data-path="README.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-shapefile-to-sqlite" class="anchor" aria-hidden="true" href="#user-content-shapefile-to-sqlite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>shapefile-to-sqlite</h1> <p><a href="https://pypi.org/project/shapefile-to-sqlite/" rel="nofollow"><img src="https://camo.githubusercontent.com/491f86f2d61c3cf6ce08c6c019a47686ccbe35187c465aeb784876d3adcc236b/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f736861706566696c652d746f2d73716c6974652e737667" alt="PyPI" data-canonical-src="https://img.shields.io/pypi/v/shapefile-to-sqlite.svg" style="max-width:100%;"></a> <a href="https://circleci.com/gh/simonw/shapefile-to-sqlite" rel="nofollow"><img src="https://camo.githubusercontent.com/665b44f51d2c08eeb25388a24485a1591ebe793b914807ae371fe96fb242c0aa/68747470733a2f2f636972636c6563692e636f6d2f67682f73696d6f6e772f736861706566696c652d746f2d73716c6974652e7376673f7374796c653d737667" alt="CircleCI" data-canonical-src="https://circleci.com/gh/simonw/shapefile-to-sqlite.svg?style=svg" style="max-width:100%;"></a> <a href="https://github.com/simonw/shapefile-to-sqlite/blob/main/LICENSE"><img src="https://camo.githubusercontent.com/1698104e976c681143eb0841f9675c6f802bb7aa832afc0c7a4e719b1f3cf955/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322e302d626c75652e737667" alt="License" data-canonical-src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" style="max-width:100%;"></a></p> <p>Load shapefiles into a SQLite (optionally SpatiaLite) database.</p> <p>Project background: <a href="https://simonwillison.net/2020/Feb/19/shapefile-to-sqlite/" rel="nofollow">Things I learned about shapefiles building shapefile-to-sqlite</a></p> <h2><a id="user-content-how-to-install" class="anchor" aria-hidden="true" href="#user-content-how-to-install"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>How to install</h2> <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ pip install shapefile-to-sqlite "><pre><code>$ pip install shapefile-to-sqlite </code></pre></div> <h2><a id="user-content-how-to-use" class="anchor" aria-hidden="true" href="#user-content-how-to-use"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>How to use</h2> <p>You can run this tool against a shapefile file like so:</p> <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ shapefile-to-sqlite my.db features.shp "><pre><code>$ shapefile-to-sqlite my.db features.shp </code></pre></div> <p>This will load the geometries as GeoJSON in a text column.</p> <h2><a id="user-content-using-with-spatialite" class="anchor" aria-hidden="true" href="#user-content-using-with-spatialite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Using with SpatiaLite</h2> <p>If you have <a href="https://www.gaia-gis.it/fossil/libspatialite/index" rel="nofollow">SpatiaLite</a> available you can load them as SpatiaLite geometries like this:</p> <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ shapefile-to-sqlite my.db features.shp --spatialite "><pre><code>$ shapefile-to-sqlite my.db features.shp --spatialite </code></pre></div> <p>The data will be loaded into a table called <code>features</code> - based on the name of the shapefile. You can specify an alternative table name using <code>--table</code>:</p> <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ shapefile-to-sqlite my.db features.shp --table=places --spatialite "><pre><code>$ shapefile-to-sqlite my.db features.shp --table=places --spatialite </code></pre></div> <p>The tool will search for the SpatiaLite module in the following locations:</p> <ul> <li><code>/usr/lib/x86_64-linux-gnu/mod_spatialite.so</code></li> <li><code>/usr/local/lib/mod_spatialite.dylib</code></li> </ul> <p>If you have installed the module in another location, you can use the <code>--spatialite_mod=xxx</code> option to specify where:</p> <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ shapefile-to-sqlite my.db features.shp \ --spatialite_mod=/usr/lib/mod_spatialite.dylib "><pre><code>$ shapefile-to-sqlite my.db features.shp \ --spatialite_mod=/usr/lib/mod_spatialite.dylib </code></pre></div> <p>You can use the <code>--spatial-index</code> option to create a spatial index on the <code>geometry</code> column:</p> <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ shapefile-to-sqlite my.db features.shp --spatial-index "><pre><code>$ shapefile-to-sqlite my.db features.shp --spatial-index </code></pre></div> <p>You can omit <code>--spatialite</code> if you use either <code>--spatialite-mod</code> or <code>--spatial-index</code>.</p> <h2><a id="user-content-projections" class="anchor" aria-hidden="true" href="#user-content-projections"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Projections</h2> <p>By default, this tool will attempt to convert geometries in the shapefile to the WGS 84 projection, for best conformance with the <a href="https://tools.ietf.org/html/rfc7946" rel="nofollow">GeoJSON specification</a>.</p> <p>If you want it to leave the data in whatever projection was used by the shapefile, use the <code>--crs=keep</code> option.</p> <p>You can convert the data to another output projection by passing it to the <code>--crs</code> option. For example, to convert to <a href="https://epsg.io/2227" rel="nofollow">EPSG:2227</a> (California zone 3) use <code>--crs=espg:2227</code>.</p> <p>The full list of formats accepted by the <code>--crs</code> option is <a href="https://pyproj4.github.io/pyproj/stable/api/crs.html#pyproj.crs.CRS.__init__" rel="nofollow">documented here</a>.</p> <h2><a id="user-content-extracting-columns" class="anchor" aria-hidden="true" href="#user-content-extracting-columns"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Extracting columns</h2> <p>If your data contains columns with a small number of heavily duplicated values - the names of specific agencies responsible for parcels of land for example - you can extract those columns into separate lookup tables referenced by foreign keys using the <code>-c</code> option:</p> <div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ shapefile-to-sqlite my.db features.shp -c agency "><pre><code>$ shapefile-to-sqlite my.db features.shp -c agency </code></pre></div> <p>This will create a <code>agency</code> table with <code>id</code> and <code>name</code> columns, and will create the <code>agency</code> column in your main table as an integer foreign key reference to that table.</p> <p>The <code>-c</code> option can be used multiple times.</p> <p><a href="https://calands.datasettes.com/calands/CPAD_2020a_Units" rel="nofollow">CPAD_2020a_Units</a> is an example of a table created using the <code>-c</code> option.</p> </article></div> |
Links from other tables
- 5 rows from repo in releases