73c4c0f4bc
refactor: simplify database and table interactions
...
feat: update sqlalchemy version to 2.0.0
refactor: improve table insert and update methods
refactor: streamline table select and distinct queries
refactor: simplify table delete and schema creation
feat: test: update tests to reflect api changes
2025-11-09 17:02:19 +01:00
Simon Wörpel
d61d309637
Respect _limit and _offset kwargs for select distinct
2023-08-08 15:42:38 +02:00
Friedrich Lindenberg
b1638a41d6
Switch from nose to pytest
2021-10-30 12:07:06 +02:00
Friedrich Lindenberg
02d3d0333f
Support disabling auto-increment on primary keys, fixes #351 .
2020-12-29 20:03:45 +01:00
Friedrich Lindenberg
3fe0ae10aa
Seems it was that lock.
2020-08-23 12:23:00 +02:00
Friedrich Lindenberg
28109d6826
Minor fixes
2020-08-23 12:10:10 +02:00
Friedrich Lindenberg
a17f2c8d5c
Apply black
2020-08-02 12:52:11 +02:00
Friedrich Lindenberg
6469ad5c42
Use default DB more
2020-06-29 08:33:41 +02:00
Ben Fasoli
aeaab50043
Adds support to serialize dict as JSON
...
There are likely some vendor-specific enhancements (such as `JSONB` in PostgreSQL) but the current method of type guessing doesn't know anything about the underlying database.
2020-03-25 19:35:36 -07:00
Micha Gorelick
ff98cfc5d4
Added ChunkedUpdate to leverage update_many
...
The ChunkedInsert object is great for speeding up inserts. This commit
does the same for updates. It chunks up updates then sends them to the
table's `update_many` in bundles that contain updates to the same
fields.
In doing this the ChunkedInsert and new ChunkedUpdate were refactored to
inherit from a common `_Chunker` object that does most of the chunking
logic. This should also make it simple to add a ChunkedUpsert object.
2020-03-13 19:00:54 +01:00
Micha Gorelick
03b7532997
fix flake8 errors
2020-03-13 15:05:07 +01:00
Micha Gorelick
a9a15966e2
Add optional callback to ChunkedInsert
...
This commit adds an optional `callback` argument to the `ChunkedInsert`
object. This callback is a callable object which gets called before the
chunked insert happens. This is useful for clearing any local caches
that may be in place to deal with the eventual consistency resulting
from the delayed nature of the chunked inserts.
For example,
```
cache = set()
chunked_table = ChunkedInsert(table, callback=lambda queue: cache.clear())
while True:
data = get_data_id()
key = data['key']
if key in cache or table.find_one(key=key)
continue
cache.add(key)
chunked_table.insert(data)
```
2020-03-13 14:29:03 +01:00
Friedrich Lindenberg
72ecf561fe
Make column lookups properly case-insensitive, refs #310 .
2020-02-23 19:13:34 +01:00
Abdurrahmaan Iqbal
d8cf515ffe
Implement additional find operators (and relevant tests)
2019-08-08 12:55:29 +01:00
Friedrich Lindenberg
43e9431865
Merge pull request #296 from conorreid/bigint_check
...
Add support for BigInteger
2019-07-14 17:58:43 +02:00
conorreid
8409c40645
switch all ints to bigints
2019-07-13 10:42:20 -04:00
Friedrich Lindenberg
081cb5ec7a
Merge pull request #298 from abmyii/master
...
Implement update_many, upsert_many and refactor for a 2x speed-up of insert_many
2019-07-13 14:47:04 +02:00
Friedrich Lindenberg
6baff6fa34
Merge pull request #284 from al42and/upsert_id
...
Fix bug when UPSERTing a column named 'id'
2019-07-13 14:28:32 +02:00
Abdurrahmaan Iqbal
82c6cdc990
Add whitespace around arithmetic operator so flake8 test passes
2019-07-09 09:41:21 +01:00
Abdurrahmaan Iqbal
a9f3eb86b2
Add tests for new functions
2019-07-08 18:10:00 +01:00
conorreid
96f57f161a
added tests for big int vs integer check
2019-06-21 11:49:01 -04:00
Stefan Wehrmeyer
a008d120c7
Fix WS
2019-03-09 11:39:25 +01:00
Stefan Wehrmeyer
046f3434ff
Remove polyfill for Python < 2.7
2019-03-09 11:39:00 +01:00
Andrey Alekseenko
25477717bc
Fix bug when UPSERTing a column named 'id'
2019-02-14 22:35:47 -05:00
Friedrich Lindenberg
8324350f2e
Try to fix up tests, Python 3.8 warnings
2019-01-31 14:56:19 +01:00
Simon Conseil
c69827de7c
Add support for advanced queries in distinct
...
Fix #263
2018-09-26 17:36:09 +02:00
Unknown
f9bb652034
add find operators
2018-06-13 02:29:32 +02:00
Friedrich Lindenberg
2cf9da068a
fix flake8 complaints
2017-12-05 17:21:08 +01:00
Friedrich Lindenberg
8226e51408
option for streamed responses.
2017-12-04 23:22:08 +01:00
Stefan Wehrmeyer
e9aa9298f0
Drop unicode prefix
2017-09-29 14:10:17 +02:00
Friedrich Lindenberg
b8598b230a
Rename test, refs #217 .
2017-09-09 18:37:37 +02:00
Friedrich Lindenberg
fc59bc59e6
Fix up imports, refs #217 .
2017-09-09 18:34:32 +02:00
Friedrich Lindenberg
a049691749
Remove datafreeze component, fixes #217
2017-09-09 18:24:34 +02:00
Friedrich Lindenberg
4232606d27
Move to a model where the table is created lazily, with an initial set of columns
2017-09-03 23:23:57 +02:00
Friedrich Lindenberg
e30cf24195
Rewrite data change functions on table.
2017-09-03 10:05:17 +02:00
Friedrich Lindenberg
37d7f47d39
Make table instances singleton-ish.
2017-09-02 22:35:29 +02:00
Friedrich Lindenberg
bf17deeb7f
forgot a print statement
2017-09-02 20:40:52 +02:00
Friedrich Lindenberg
13cbff37fe
Fix postgres errors.
2017-09-02 20:33:11 +02:00
Friedrich Lindenberg
a2748b7fde
Reduce dependence on internal metadata caching, refs #208 .
2017-09-02 19:35:01 +02:00
Friedrich Lindenberg
672b0bc8c4
test fixes
2017-09-02 17:31:05 +02:00
Friedrich Lindenberg
47cdf52323
fix mysql issue, test no-column table.
2017-09-02 17:25:52 +02:00
Friedrich Lindenberg
cc7787036b
still more cases of using the engine and not the executable in transaction
2017-09-02 17:17:24 +02:00
Friedrich Lindenberg
a4c73a8fb8
Begin implementing a types handler instead of using plain text types.
...
this is potentially BREAKING scripts which use the string syntax.
2017-09-02 16:47:04 +02:00
Friedrich Lindenberg
edc41e4d82
Merge branch 'master' into create_column_by_example
2017-09-02 08:22:22 +02:00
Stefan Wehrmeyer
ffe7beae12
Make test read nicer
2017-05-16 16:59:18 +02:00
Stefan Wehrmeyer
3f330df20f
Check for clauses and filters in delete
...
Guard against wrong API use before dropping all data
2017-05-16 16:57:07 +02:00
Paul Fitzpatrick
94bc6e09c0
add Table.create_column_by_example method
...
This adds a method to create a column by giving an example of
the data it will contain rather than by specifying its
SQLAlchemy type. This extends the range of operations that
can be performed without reading the SQLAlchemy docs.
2016-10-26 09:29:55 -04:00
Stefan Wehrmeyer
6fc8bfed51
Skip invalid values test on mysql
2016-10-17 15:16:13 +02:00
Stefan Wehrmeyer
a92a315b60
Test behaviour of _step, assert it still retrieves all data
2016-10-15 22:55:50 +02:00
Saun Shewanown
c1e82bb9ae
Added insert_ignore and changed upsert.
2016-04-22 11:35:14 -04:00
