retoor/dataset
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
615 Commits 2 Branches 0 Tags
c0177a850f488098c2d9a1d4f16fc12c048b2b6b
Commit Graph
615 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Friedrich Lindenberg c0177a850f Add MariaDB 2020-03-15 13:40:06 +01:00
Friedrich Lindenberg dd286b33da Add postgresql 2020-03-15 13:29:39 +01:00
Friedrich Lindenberg 7749574264 Dev dependencies 2020-03-15 13:24:19 +01:00
Friedrich Lindenberg 666c8185c8 Begin migrating to GitHub Actions 2020-03-15 13:15:31 +01:00
Friedrich Lindenberg 0600f3eb43 Merge pull request #315 from mynameisfiber/feat/chunked-update 
Added ChunkedUpdate to leverage `update_many`
 2020-03-15 12:48:02 +01:00
Friedrich Lindenberg c84ea316db Merge pull request #314 from mynameisfiber/feat/chunked-callback 
Add optional callback to ChunkedInsert
 2020-03-15 12:46:38 +01:00
Micha Gorelick ff98cfc5d4 Added ChunkedUpdate to leverage update_many 
The ChunkedInsert object is great for speeding up inserts. This commit
does the same for updates. It chunks up updates then sends them to the
table's `update_many` in bundles that contain updates to the same
fields.

In doing this the ChunkedInsert and new ChunkedUpdate were refactored to
inherit from a common `_Chunker` object that does most of the chunking
logic. This should also make it simple to add a ChunkedUpsert object.
 2020-03-13 19:00:54 +01:00
Micha Gorelick 03b7532997 fix flake8 errors 2020-03-13 15:05:07 +01:00
Micha Gorelick a9a15966e2 Add optional callback to ChunkedInsert 
This commit adds an optional `callback` argument to the `ChunkedInsert`
object. This callback is a callable object which gets called before the
chunked insert happens. This is useful for clearing any local caches
that may be in place to deal with the eventual consistency resulting
from the delayed nature of the chunked inserts.

For example,

```
cache = set()
chunked_table = ChunkedInsert(table, callback=lambda queue: cache.clear())
while True:
    data = get_data_id()
    key = data['key']
    if key in cache or table.find_one(key=key)
        continue
    cache.add(key)
    chunked_table.insert(data)
```
 2020-03-13 14:29:03 +01:00
Friedrich Lindenberg 73ff374513 Bump version: 1.2.2 → 1.2.3 2020-03-10 16:28:52 -05:00
Friedrich Lindenberg a513888824 Lock column updates 2020-03-10 16:28:35 -05:00
Friedrich Lindenberg 894819fe99 Bump version: 1.2.1 → 1.2.2 2020-02-24 18:27:42 +01:00
Friedrich Lindenberg 7291cf5ca0 Merge pull request #313 from Frankkkkk/master 
Fix race condition when getting column keys of new table
 2020-02-24 15:46:04 +01:00
Frank Villaro-Dixon b6ed0587ed Fix race condition when getting column keys of new table 
_columns_keys() sets _columns to {}, but _sync_table() empties _columns
when table is empty.
 2020-02-24 13:02:31 +01:00
Friedrich Lindenberg c01e717436 Bump version: 1.2.0 → 1.2.1 2020-02-23 19:17:20 +01:00
Friedrich Lindenberg 72ecf561fe Make column lookups properly case-insensitive, refs #310. 2020-02-23 19:13:34 +01:00
Friedrich Lindenberg 5ddff753b7 Bump version: 1.1.2 → 1.2.0 2020-01-11 16:50:39 +01:00
Friedrich Lindenberg e7b212b589 Fix up changelog a bit 2020-01-11 16:50:30 +01:00
Friedrich Lindenberg b362f25816 Use default pool for Sqlite, refs #283, #163. 2020-01-11 16:47:01 +01:00
Friedrich Lindenberg 354a51bf98 Add support for database views, fixes #235, fixes #267. 2020-01-11 16:46:04 +01:00
Friedrich Lindenberg fc3af9e1f1 Fix memory leak, fixes #226. 2020-01-11 15:01:22 +01:00
Friedrich Lindenberg 71b9421d60 Document ensure_schema behaviour, fixes #225. 2020-01-11 14:38:21 +01:00
Friedrich Lindenberg 39385b7e0a Make has_column case insensitive, refs #270. 2020-01-11 14:31:51 +01:00
Friedrich Lindenberg 9b1f8ea649 Simplify 2020-01-11 14:31:07 +01:00
Friedrich Lindenberg 45ae60349c Remove use of six because its 2020 2020-01-11 14:06:57 +01:00
Friedrich Lindenberg ce28a0d4e8 Pass through column arguments, fixes #233, #266. 2020-01-11 13:55:57 +01:00
Friedrich Lindenberg e766fa25f0 Fix incorrect query generation, fixes #290. 2020-01-11 13:47:34 +01:00
Friedrich Lindenberg d7f23933f1 Fix doc error, fixes #297. 2020-01-11 13:36:28 +01:00
Friedrich Lindenberg d0c4e9fcc7 Implement method to close database connections, fixes #246. 2020-01-11 13:16:30 +01:00
Friedrich Lindenberg 00d2279abf Split _args_to_clause into two functions 2020-01-11 13:02:32 +01:00
Friedrich Lindenberg 5a93c61bf6 Merge pull request #302 from jsalsman/patch-2 
allow for ilike clauses
 2020-01-11 12:48:13 +01:00
Friedrich Lindenberg bd70c2aa9e Merge pull request #301 from abmyii/more_find_operators 
Implement additional find operators (and relevant tests)
 2020-01-11 12:43:29 +01:00
Friedrich Lindenberg 2b6352b65c Don't test Python 2.7 any more, use 3.8 throughout 2020-01-11 12:41:43 +01:00
Friedrich Lindenberg 5a09e72fd1 Fix error introduced in #305. 2020-01-11 12:37:50 +01:00
Friedrich Lindenberg db3ef16566 pep8 2020-01-11 12:11:30 +01:00
Friedrich Lindenberg 901d6a99d9 Merge pull request #305 from abmyii/retain_columns_order 
Use lists instead of sets to maintain order of column names
 2020-01-11 11:51:45 +01:00
Friedrich Lindenberg 01834d65db Merge pull request #307 from timgates42/bugfix/typo_accessed 
Fix simple typo: acessed -> accessed
 2020-01-11 11:51:15 +01:00
Tim Gates ec4723e691 Fix simple typo: acessed -> accessed 
Closes #306
 2020-01-05 11:02:03 +11:00
Abdurrahmaan Iqbal e2431bc344 Use lists instead of sets to maintain order of column names 2019-12-24 18:03:35 +00:00
James Salsman 800a0fa518 allow for ilike clauses 
case-insensitive (sub)string matching
 2019-10-14 00:03:01 -07:00
Abdurrahmaan Iqbal d8cf515ffe Implement additional find operators (and relevant tests) 2019-08-08 12:55:29 +01:00
Friedrich Lindenberg 43e9431865 Merge pull request #296 from conorreid/bigint_check 
Add support for BigInteger
 2019-07-14 17:58:43 +02:00
conorreid 8409c40645 switch all ints to bigints 2019-07-13 10:42:20 -04:00
Friedrich Lindenberg 081cb5ec7a Merge pull request #298 from abmyii/master 
Implement update_many, upsert_many and refactor for a 2x speed-up of insert_many
 2019-07-13 14:47:04 +02:00
Friedrich Lindenberg 6baff6fa34 Merge pull request #284 from al42and/upsert_id 
Fix bug when UPSERTing a column named 'id'
 2019-07-13 14:28:32 +02:00
Friedrich Lindenberg 4c46cd2594 Merge pull request #277 from zachvalenta/patch-2 
fix typos
 2019-07-13 14:14:14 +02:00
Friedrich Lindenberg 2f264deed2 Merge pull request #285 from saimn/table-doc 
Add missing methods in Table api doc
 2019-07-13 14:13:44 +02:00
Abdurrahmaan Iqbal 19a73759ca Remove f-string for wider compatiblity 2019-07-09 09:50:36 +01:00
Abdurrahmaan Iqbal 6874889591 Fix logic error in insert_many 2019-07-09 09:45:18 +01:00
Abdurrahmaan Iqbal 82c6cdc990 Add whitespace around arithmetic operator so flake8 test passes 2019-07-09 09:41:21 +01:00