Compare commits

..
21 Commits
Author SHA1 Message Date
retoor b6ac734183 fix: suppress unused parameter warnings and remove dead code in cli and main
SORM tests / Compile (push) Successful in 6s
SORM build / build (push) Successful in 19s
2024-12-08 19:27:05 +00:00
retoor 985b469af4 chore: remove debug console.log statements from user authentication flow 2024-12-08 18:59:58 +00:00
retoor 60f98fdec7 chore: strip debug print statements and dead csv cleanup code from sormc and sormq 2024-12-08 18:58:50 +00:00
retoor 5565dd7e81 feat: add rlib.h header with 8.4k lines of core utility macros, type definitions, and MIT license boilerplate 2024-12-02 13:21:44 +00:00
retoor e28a87af74 fix: replace broken curl download with local cp for rlib.h in build workflow 2024-12-02 13:21:11 +00:00
retoor dbe70775d5 chore: add rlib.h download step before build in build.yaml workflow 2024-12-02 13:18:12 +00:00
retoor fc73c4221b feat: add rlib.h dependency and update includes to local path for runtime library support 2024-11-28 22:28:33 +00:00
retoor 0dfeef5d9d chore: fix trailing quote typo in build workflow echo command 2024-11-22 15:31:46 +00:00
retoor 228163c92d chore: fix apt-get command syntax in build workflow by removing extraneous 'get' 2024-11-22 15:30:38 +00:00
retoor 9f2941859d chore: replace named steps with echo placeholders in CI build workflow 2024-11-22 15:30:01 +00:00
retoor faa3749681 chore: add gitea ci workflow with build steps for sorm project 2024-11-22 15:27:37 +00:00
retoor d473276fd8 chore: remove trailing whitespace and reorder includes in cli.h, main.c, sorm.h, sorm.py, str.h 2024-11-22 14:51:47 +00:00
retoor 625e78ba79 docs: clarify C standard compatibility and fix punctuation in README dependencies section 2024-11-22 13:57:47 +00:00
retoor f54e36bdd8 docs: add building requirements and thread safety sections to readme
The README now includes a new "Building" section with build command and dependency list, and moves the "Thread safety" section after "Design choices" for better logical flow. The caution about missing rlib.h has been removed.
2024-11-22 13:56:40 +00:00
retoor f3a3dfef9e docs: add caution note about missing rlib.h dependency and future plans 2024-11-22 13:51:55 +00:00
retoor 01415a8fe9 feat: add Gitea workflow for automated SORM tests on push events
- Created .gitea/workflows/test.yaml with a Compile job running on ubuntu-latest
- Configured workflow to trigger on push events with echo steps for event, branch, and repository info
- Added checkout action and file listing step using gitea workspace variable
2024-11-22 13:45:03 +00:00
retoor b9e687ebdb chore: fix grammar and phrasing errors in README documentation files 2024-11-22 13:45:03 +00:00
retoor 1da30958d1 docs: fix formatting of code literals and casing in README examples 2024-11-22 13:45:03 +00:00
retoor f81ad80fa6 docs: add language identifier to C code blocks in README examples 2024-11-22 13:45:03 +00:00
retoor e58479ccd1 docs: add comprehensive project overview, setup instructions, and C/Python API examples to README
The README now includes a full project description explaining the SORM (SQL ORM) philosophy, thread safety considerations, design choices favoring native types for cross-language compatibility, and detailed C API examples covering connection, table creation, and query execution. Also adds Python support documentation with variadic function calling patterns through ctypes.
2024-11-22 13:45:03 +00:00
retoor 0f0657496b feat: add initial project scaffold with clang-format, gitignore, Makefile, core C headers, main entry, Python bindings, and dynamic library build 2024-11-22 13:45:03 +00:00
11 changed files with 8908 additions and 491 deletions
+21
View File
@@ -0,0 +1,21 @@
name: SORM build
run-name: Build 🚀
on: [push]
jobs:
build:
runs-on: ubuntu-latest
steps:
- run: echo "Update apt repository"
- run: apt update
- run: echo "Installing dependencies"
- run: apt install build-essential sqlite3 libsqlite3-dev libreadline-dev libpython3-dev make -y
- name: Check out repository code
uses: actions/checkout@v4
- name: List files in the repository
run: |
ls ${{ gitea.workspace }}
- run: cp rlib.h /usr/include/
- run: echo "Build project (misses rlib.h)"
- run: make build
- run: echo "This job's status is ${{ job.status }}."
+5 -4
View File
@@ -1,9 +1,10 @@
all: build run
build:
gcc main.c -lsqlite3 -lreadline -o sorm
gcc -shared -o sorm.so -fPIC main.c -lsqlite3 -lreadline
gcc main.c -Wextra -Wall -lsqlite3 -lreadline -o sorm
gcc -shared -o sorm.so -fPIC main.c -lsqlite3 -lreadline
run:
./sorm
./sorm
+27 -3
View File
@@ -1,5 +1,9 @@
# SORM
## Todo
- auto complete table / view names.
## Description
SORM stands for SQL ORM. I made this because I have a love/hate relationship with both. I combined them together to have the best of both worlds!
@@ -10,14 +14,34 @@ Examples of common used functions are:
- `sormq(int conn, char *sql, ...)` execute query. Variadic arguments. Works like `printf`. Returns result in CSV format in case of `SELECT`.
- `sorm_csvd(char *csv_data)` dumps your CSV result data to a nice fixed content width table in the terminal.
## Thread safety
I wonder if I have configured sqlite3 the right way for thread safety. It maybe requires a manual compilation of the shared object file. Will look into that. SORM is written with thread safety in mind.
## Design choices
I use mainly native types and not custom structs. For example, the db parameter is an int. This is so it can easily communicate with other languages using a shared object file.
Same argument is for the result set of `sormq` (the query function) resulting in a `char *` containing CSV data.
While the performance is nice, it's not written with performance in mind at all.
## Thread safety
I wonder if I have configured sqlite3 the right way for thread safety. It maybe requires a manual compilation of the shared object file. Will look into that. SORM is written with thread safety in mind.
## Building
It has only one build command for the shared object file and the REPL/CLI.
It has not been compiled with optimizations / warnings enabled. Don't recommend for now.
```bash
make build
```
### Requirements
This project has the following dependencies:
- sqlite3-dev (`apt install sqlite3-dev`)
- readline(-dev?) (no idea how to install)
- gcc
- make
The project probably works from C99 to C2X.
### Caution
This project won't compile on your local PC because `rlib.h` is missing. This is my bundle of libraries that I regularely use. I will add this later as 3rd party lib. Maybe I will cherry pick only the functions that SORM uses out of it to keep the source clean, else I will have thousands of lines nothing to do with this project in the source.
## Python support
The Python library is low quality. I made it just for fun and test. This is not a defitive version. But it show very well how to communicate with a shared object file. I'm sure someone will be happy with examples how to use variadic functions trough Python to C. See `sorm.py`
+86 -102
View File
@@ -1,128 +1,116 @@
#ifndef SORM_CLI_H
#define SORM_CLI_H
#include <rlib.h>
#include "sorm.h"
#include <fcntl.h>
#include <readline/history.h>
#include <readline/readline.h>
#include <rlib.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/stat.h>
#include <readline/readline.h>
#include <readline/history.h>
#include "sorm.h"
#include <unistd.h>
const char * history_filename = ".sorm_history";
const char *history_filename = ".sorm_history";
int _sorm_readline_accept_line(int count, int key) {
int _sorm_readline_accept_line(int count, int key) {
(void)count;
(void)key;
if (strchr(rl_line_buffer, ';')) {
rl_done = 1;
rl_done = 1;
return 0;
}
rl_insert_text("\n");
return 0;
}
char* _sorm_autocompletion_generator(const char* text, int state) {
const char* completions[] = {
"exit",
sorm_last_query,
sorm_last_query_expanded,
"python",
"history",
"memory",
"truncate",
NULL
};
int list_index;
char *_sorm_autocompletion_generator(const char *text, int state) {
const char *completions[] = {"exit", sorm_last_query, sorm_last_query_expanded, "python", "history", "memory", "truncate", NULL};
int list_index;
if (!state) {
list_index = 0;
}
while (completions[list_index] != NULL) {
if (strncmp(completions[list_index], text, strlen(text)) == 0) {
return strdup(completions[list_index++]);
}
list_index++;
}
return NULL;
}
char** _sorm_autocomplete(const char* text, int start, int end) {
char **_sorm_autocomplete(const char *text, int start, int end) {
(void)start;
(void)end;
rl_attempted_completion_over = 1;
return rl_completion_matches(text, _sorm_autocompletion_generator);
}
int
_hs_read_file (const char *filename, char *buffer, size_t size)
{
int _hs_read_file(const char *filename, char *buffer, size_t size) {
int fd;
ssize_t bytes_read;
fd = open (filename, O_RDONLY);
fd = open(filename, O_RDONLY);
if (fd < 0)
return -1;
bytes_read = read (fd, buffer, size);
close (fd);
bytes_read = read(fd, buffer, size);
close(fd);
if (bytes_read < 0 || (size_t)bytes_read != size)
return -1;
return 0;
}
int
sorm_cli_history_dump (const char *filename)
{
int sorm_cli_history_dump(const char *filename) {
register int line_start, line_end;
char *input;
struct stat finfo;
size_t file_size;
if (stat (filename, &finfo) < 0)
if (stat(filename, &finfo) < 0)
return -1;
file_size = (size_t)finfo.st_size;
input = (char *)malloc (file_size + 1);
input = (char *)malloc(file_size + 1);
if (!input)
return -1;
if (_hs_read_file (filename, input, file_size) < 0) {
free (input);
if (_hs_read_file(filename, input, file_size) < 0) {
free(input);
return -1;
}
input[file_size] = '\0';
for (line_start = line_end = 0; line_end < file_size; line_end++) {
for (line_start = line_end = 0; (size_t)line_end < file_size; line_end++) {
if (input[line_end] == '\n') {
input[line_end] = '\0';
printf ("%s\n",input + line_start);
printf("%s\n", input + line_start);
line_start = line_end + 1;
}
}
if (line_start < file_size)
printf ("%s\n",input + line_start);
free (input);
return where_history ();
if ((size_t)line_start < file_size)
printf("%s\n", input + line_start);
free(input);
return where_history();
}
char sorm_history_filename[4096];
char sorm_history_filename[4096];
void sorm_cli_init(const char * history_filename){
void sorm_cli_init(const char *history_filename) {
strcpy(sorm_history_filename, history_filename);
rl_bind_key('\n',NULL);
rl_bind_key('\n', NULL);
rl_bind_key('\r', NULL);
rl_add_defun("custom-accept-line", _sorm_readline_accept_line, '\n');
rl_add_defun("custom-accept-line-cr", _sorm_readline_accept_line, '\r');
@@ -130,55 +118,51 @@ void sorm_cli_init(const char * history_filename){
rl_attempted_completion_function = _sorm_autocomplete;
rl_variable_bind("show-all-if-ambiguous", "on");
rl_variable_bind("menu-complete-display-prefix", "on");
using_history();
read_history(sorm_history_filename);
read_history(sorm_history_filename);
}
char sorm_cli_previous_command[sizeof(rl_line_buffer)];
char * sorm_cli_readline(char * prompt){
char * result = readline(prompt);
if(strcmp(rl_line_buffer,sorm_cli_previous_command)){
add_history((const char *)result);
write_history(sorm_history_filename);
}
strcpy(sorm_cli_previous_command,rl_line_buffer);
return result;
char *sorm_cli_readline(char *prompt) {
char *result = readline(prompt);
if (strcmp(rl_line_buffer, sorm_cli_previous_command)) {
add_history((const char *)result);
write_history(sorm_history_filename);
}
strcpy(sorm_cli_previous_command, rl_line_buffer);
return result;
}
bool sormrepl_handle_command(char * command){
if(!strncmp(command, "history",7)){
sorm_cli_history_dump(history_filename);
return true;
}
bool sormrepl_handle_command(char *command) {
if (!strncmp(command, "history", 7)) {
sorm_cli_history_dump(history_filename);
return true;
}
return false;
}
void sormrepl(int sorm){
sorm_t * db = sormg(sorm);
char sql[4097];
void sormrepl(int sorm) {
sorm_cli_init(history_filename);
char * query;
while((query = sorm_cli_readline("sql> "))){
if(sormrepl_handle_command(query))
char *query;
while ((query = sorm_cli_readline("sql> "))) {
if (sormrepl_handle_command(query))
continue;
sorm_ptr res = sormq(sorm,query);
if(res){
if(sormqt(query) == SORM_SELECT){
int length = sormlv(res);
sormfmtd(res);
free(res);
}else if(sormqt(query) == SORM_DELETE){
printf("%d records affected.\n",res);
}else if(sormqt(query) == SORM_INSERT){
printf("Last insert id: %d.\n",res);
}
sorm_ptr res = sormq(sorm, query);
if (res) {
if (sormqt(query) == SORM_SELECT) {
sormfmtd(res);
free(res);
} else if (sormqt(query) == SORM_DELETE) {
printf("%d records affected.\n", res);
} else if (sormqt(query) == SORM_INSERT) {
printf("Last insert id: %d.\n", res);
}
printf("Rows: %lld, Execute %s, Format: %s\n",sorm_row_count, format_time(_sorm_query_duration),format_time(_sorm_result_format_duration));
printf("%s\n",rmalloc_stats());
}
printf("Rows: %lld, Execute %s, Format: %s\n", sorm_row_count, format_time(_sorm_query_duration),
format_time(_sorm_result_format_duration));
printf("%s\n", rmalloc_stats());
}
}
#endif
+15 -30
View File
@@ -1,39 +1,24 @@
#include "sorm.h"
#include "cli.h"
#include "sorm.h"
int main() {
int db = sormc("db.sqlite3");
//sormq(db,"DROP TABLE IF EXISTS pony;");
printf("%d\n",db);
sormq(db, "CREATE TABLE IF NOT EXISTS pony (id INTEGER PRIMARY KEY AUTOINCREMENT,name,age);",NULL);
sorm_pk iid = sormq(db, "INSERT INTO pony (id,name,age) VALUES (NULL,%s,%d);",
"Teenii",
19
);
iid = sormq(db, "INSERT INTO pony (id,name,age) VALUES (NULL,%s,%d);",
"Amber",
20
);
iid = sormq(db, "INSERT INTO pony (id,name,age) VALUES (NULL,%s,%d);",
"Feuerherz",
20
);
iid = sormq(db, "INSERT INTO pony (id,name,age) VALUES (NULL,%s,%d);",
"Retoor",
34
);
sorm_str csv = sormq(db, "SELECT * FROM pony WHERE id in (?i,?i,?i)",1,2,3);
sorm_str csv2 = sormq(db, "SELECT * FROM pony WHERE id = %d and age = %d ", 1,33);
sorm_str csv3 = sormq(db, "SELECT * FROM pony LIMIT 2");
//free(csv3);
//free(csv2);
if(csv2)
printf("%s\n",csv2);
printf("%s\n",csv3);
// sormq(db,"DROP TABLE IF EXISTS pony;");
sormq(db, "CREATE TABLE IF NOT EXISTS pony (id INTEGER PRIMARY KEY AUTOINCREMENT,name,age);", NULL);
sormq(db, "INSERT INTO pony (id,name,age) VALUES (NULL,%s,%d);", "Teenii", 19);
sormq(db, "INSERT INTO pony (id,name,age) VALUES (NULL,%s,%d);", "Amber", 20);
sormq(db, "INSERT INTO pony (id,name,age) VALUES (NULL,%s,%d);", "Feuerherz", 20);
sormq(db, "INSERT INTO pony (id,name,age) VALUES (NULL,%s,%d);", "Retoor", 34);
sorm_str csv2 = (sorm_str)sormq(db, "SELECT * FROM pony WHERE id = %d and age = %d ", 1, 33);
sorm_str csv3 = (sorm_str)sormq(db, "SELECT * FROM pony LIMIT 2");
// free(csv3);
// free(csv2);
if (csv2)
printf("%s\n", csv2);
printf("%s\n", csv3);
free(csv3);
sormd(db);
printf("%s\n",rmalloc_stats());
printf("%s\n", rmalloc_stats());
db = sormc("db.sqlite3");
sormrepl(db);
}
+8448
View File
File diff suppressed because it is too large Load Diff
BIN
View File
Binary file not shown.
+233 -278
View File
@@ -1,21 +1,20 @@
#ifndef SORM_H
#define SORM_H
#include <rlib.h>
#include "str.h"
#include <ctype.h>
#include "rlib.h"
#include <sqlite3.h>
#include <stdarg.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdbool.h>
#include <stdarg.h>
#include <ctype.h>
#include "str.h"
sqlite3 *db;
sqlite3_stmt *stmt;
char * sorm_last_query = NULL;
char * sorm_last_query_expanded = NULL;
char *sorm_last_query = NULL;
char *sorm_last_query_expanded = NULL;
nsecs_t _sorm_query_start = 0;
nsecs_t _sorm_query_end = 0;
@@ -28,10 +27,10 @@ nsecs_t _sorm_result_format_duration = 0;
int64_t sorm_row_count = 0;
typedef struct sorm_t {
sqlite3 * conn;
sqlite3 *conn;
int current_row;
int current_column;
char * csv;
char *csv;
nsecs_t time_query_start;
nsecs_t time_query_end;
nsecs_t time_query_duration;
@@ -39,26 +38,23 @@ typedef struct sorm_t {
nsecs_t time_result_format_end;
nsecs_t time_result_format_duration;
} sorm_t;
typedef char * sorm_pk;
typedef char * sorm_int;
typedef char * sorm_ptr;
typedef unsigned char * sorm_str;
typedef char *sorm_pk;
typedef char *sorm_int;
typedef char *sorm_ptr;
typedef unsigned char *sorm_str;
typedef double sorm_double;
typedef double sorm_float;
typedef bool sorm_bool;
int sormc(char * path);
int sormc(char *path);
void sormd(int db);
char * sormpt(char * sql, int number);
unsigned int sormcq(char * sql, char * out);
unsigned int sormpc(char * sql);
sqlite3_stmt * sormb(sorm_t* db, char * sql, ...);
sorm_ptr sormq(int db, char * sql, ...);
char * sorm_csvc(int db, sqlite3_stmt * stmt);
char * sorm_csvd(int db, sqlite3_stmt * stmt);
char * sorm_csv(int db,sqlite3_stmt * stmt);
char *sormpt(char *sql, int number);
unsigned int sormcq(char *sql, char *out);
unsigned int sormpc(char *sql);
sorm_ptr sormq(int db, char *sql, ...);
char *sorm_csvc(int db, sqlite3_stmt *stmt);
char *sorm_csvd(int db, sqlite3_stmt *stmt);
char *sorm_csv(int db, sqlite3_stmt *stmt);
typedef enum sorm_query_t {
SORM_UNKNOWN = 0,
@@ -71,26 +67,19 @@ typedef enum sorm_query_t {
const int sorm_null = -1337;
sorm_t ** sorm_instances = NULL;
sorm_t **sorm_instances = NULL;
int sorm_instance_count = 0;
int sormc(char * path){
int sormc(char *path) {
// sorm connect
printf("HIERR\n");
sorm_instance_count++;
sorm_instance_count++;
sorm_instances = realloc(sorm_instances,sorm_instance_count * sizeof(sorm_t *) + sorm_instance_count * sizeof(sorm_t));
printf("HIERR\n");
sorm_t * db = &sorm_instances[sorm_instance_count - 1];
sorm_instances = realloc(sorm_instances, sorm_instance_count * sizeof(sorm_t *) + sorm_instance_count * sizeof(sorm_t));
sorm_t *db = (sorm_t *)&sorm_instances[sorm_instance_count - 1];
printf("HIERR\n");
db->conn = NULL;
printf("HIERR\n");
db->csv = NULL;
db->current_column = 0;
db->current_row = 0;
@@ -100,94 +89,86 @@ int sormc(char * path){
db->time_result_format_duration = 0;
db->time_result_format_end = 0;
db->time_result_format_start = 0;
if(sqlite3_open(path,&db->conn))
{
fprintf(stderr, "Can't open database: %s\n", sqlite3_errmsg(db->conn));
return 0;
if (sqlite3_open(path, &db->conn)) {
fprintf(stderr, "Can't open database: %s\n", sqlite3_errmsg(db->conn));
return 0;
}
printf("DONE!\n");
return sorm_instance_count;
}
sorm_t * sormg(int ptr){
return &sorm_instances[ptr -1];
}
sorm_t *sormg(int ptr) { return (sorm_t *)&sorm_instances[ptr - 1]; }
char * sormgcsv(int ptr){
char *sormgcsv(int ptr) {
/* sorm get csv*/
sorm_t * db = sormg(ptr);
sorm_t *db = sormg(ptr);
return db->csv;
}
void sormd(int sorm){
sorm_t * db = sormg(sorm);
if(sqlite3_close(db->conn))
{
void sormd(int sorm) {
sorm_t *db = sormg(sorm);
if (sqlite3_close(db->conn)) {
fprintf(stderr, "Error closing database: %s\n", sqlite3_errmsg(db->conn));
}
if(sorm_last_query){
if (sorm_last_query) {
free(sorm_last_query);
sorm_last_query = NULL;
}
if(sorm_last_query_expanded){
if (sorm_last_query_expanded) {
free(sorm_last_query_expanded);
sorm_last_query_expanded = NULL;
}
}
char * sormpt(char * sql, int number){
char *sormpt(char *sql, int number) {
// param type
char * sqlp = sql;
char * result = NULL;
char *sqlp = sql;
char *result = NULL;
int index = 0;
while(*sqlp){
if(*sqlp == '%' || *sqlp == '?'){
while (*sqlp) {
if (*sqlp == '%' || *sqlp == '?') {
sqlp++;
switch(*sqlp){
case 'f':
result = "double";
break;
case 's':
result = "text";
break;
case 'd':
result = "int";
break;
case 'b':
result = "blob";
break;
default:
result = "?";
break;
switch (*sqlp) {
case 'f':
result = "double";
break;
case 's':
result = "text";
break;
case 'd':
result = "int";
break;
case 'b':
result = "blob";
break;
default:
result = "?";
break;
}
sqlp++;
index++;
}
if(index == number){
if (index == number) {
return result;
}
if(*sqlp)
if (*sqlp)
sqlp++;
}
if(number > index){
if (number > index) {
printf("RETURNED\n");
return NULL;
}
return NULL;
}
unsigned int sormcq(char * sql, char * out){
unsigned int sormcq(char *sql, char *out) {
// clean query
// converts %s %i parameters to ?
unsigned int count = 0;
while(*sql){
if(*sql != '%' && *sql != '?')
*out = *sql;
else{
while (*sql) {
if (*sql != '%' && *sql != '?')
*out = *sql;
else {
count++;
sql++;
*out = '?';
@@ -199,25 +180,25 @@ unsigned int sormcq(char * sql, char * out){
return count;
}
unsigned int sormpc(char * sql){
unsigned int sormpc(char *sql) {
int number = 0;
while(sormpt(sql,number) != NULL)
while (sormpt(sql, number) != NULL)
number++;
printf("FOUND: %d\n",number);
printf("FOUND: %d\n", number);
return number;
}
char * sormcts(int column_type){
if(column_type == SQLITE_INTEGER)
char *sormcts(int column_type) {
if (column_type == SQLITE_INTEGER)
return "integer";
else if(column_type == SQLITE_TEXT)
else if (column_type == SQLITE_TEXT)
return "text";
else if(column_type == SQLITE_FLOAT)
else if (column_type == SQLITE_FLOAT)
return "float";
else if(column_type == SQLITE_NULL)
else if (column_type == SQLITE_NULL)
return "null";
else if(column_type == SQLITE_BLOB)
else if (column_type == SQLITE_BLOB)
return "blob";
return "?";
}
@@ -225,25 +206,26 @@ char * sormcts(int column_type){
Execute 3.35s, Format: 36.77s
Memory usage: 537 GB, 96.026 allocated, 96.024 freed, 2 in use.
*/
char * sorm_csvc(int db, sqlite3_stmt * stmt){
sormstr_t * str = sormstrn(512);
unsigned int column_count = sqlite3_column_count(stmt);
for(int i = 0; i < column_count; i++){
const char * column_name = sqlite3_column_name(stmt,i);
sormstra(str,column_name);
sormstra(str,"(");
char column_type[1000] = "";
sprintf(column_type,"%s",sormcts(sqlite3_column_type(stmt,i)));
sormstra(str,column_type);
sormstra(str,")");
// if(i != column_count - 1)
sormstra(str,";");
}
char *sorm_csvc(int db, sqlite3_stmt *stmt) {
(void)db;
sormstr_t *str = sormstrn(512);
unsigned int column_count = sqlite3_column_count(stmt);
for (uint i = 0; i < column_count; i++) {
const char *column_name = sqlite3_column_name(stmt, i);
sormstra(str, column_name);
sormstra(str, "(");
char column_type[1000] = "";
sprintf(column_type, "%s", sormcts(sqlite3_column_type(stmt, i)));
sormstra(str, column_type);
sormstra(str, ")");
// if(i != column_count - 1)
sormstra(str, ";");
}
return sormstrc(str);
}
char * sorm_csvd(int sorm, sqlite3_stmt * stmt) {
sorm_t * db = sormg(sorm);
char *sorm_csvd(int sorm, sqlite3_stmt *stmt) {
(void)sorm;
int rc = SQLITE_ROW;
int column_count = sqlite3_column_count(stmt);
/*
@@ -257,136 +239,119 @@ char * sorm_csvd(int sorm, sqlite3_stmt * stmt) {
xecute 3.42s, Format: 37.33s
Memory usage: 6 MB, 96.052 (re)allocated, 96.024 unqiue freed, 2 in use.
*/
sormstr_t * str = sormstrn(512);
while(rc == SQLITE_ROW){
sorm_row_count++;
for(int field_index = 0; field_index < column_count; field_index++){
if(sqlite3_column_type(stmt,field_index) == SQLITE_INTEGER){
char temp[1000] = "";
sprintf(temp, "%lld",sqlite3_column_int64(stmt,field_index));
sormstra(str,temp);
}else if(sqlite3_column_type(stmt,field_index) == SQLITE_FLOAT){
char temp[1000] = "";
sprintf(temp, "%f",sqlite3_column_double(stmt,field_index));
sormstra(str,temp);
} else if(sqlite3_column_type(stmt,field_index) == SQLITE3_TEXT){
const char * temp = sqlite3_column_text(stmt,field_index);
sormstra(str,temp);
} else {
// exit(1);
}
// if(field_index != column_count - 1)
sormstra(str,";");
sormstr_t *str = sormstrn(512);
while (rc == SQLITE_ROW) {
sorm_row_count++;
for (int field_index = 0; field_index < column_count; field_index++) {
if (sqlite3_column_type(stmt, field_index) == SQLITE_INTEGER) {
char temp[1000] = "";
sprintf(temp, "%lld", sqlite3_column_int64(stmt, field_index));
sormstra(str, temp);
} else if (sqlite3_column_type(stmt, field_index) == SQLITE_FLOAT) {
char temp[1000] = "";
sprintf(temp, "%f", sqlite3_column_double(stmt, field_index));
sormstra(str, temp);
} else if (sqlite3_column_type(stmt, field_index) == SQLITE3_TEXT) {
const char *temp = ( char *)sqlite3_column_text(stmt, field_index);
sormstra(str, temp);
} else {
// exit(1);
}
sormstra(str,"\n");
rc = sqlite3_step(stmt);
// if(field_index != column_count - 1)
sormstra(str, ";");
}
char * text = sormstrc(str);
if(*text)
if(text[strlen(text)-1] == '\n')
text[strlen(text)-1] = 0;
return strdup(text);
sormstra(str, "\n");
rc = sqlite3_step(stmt);
}
char *text = sormstrc(str);
if (*text)
if (text[strlen(text) - 1] == '\n')
text[strlen(text) - 1] = 0;
return strdup(text);
}
char * sorm_csv(int sorm,sqlite3_stmt * stmt){
sorm_t * db = sormg(sorm);
char *sorm_csv(int sorm, sqlite3_stmt *stmt) {
sorm_row_count = 0;
char * column_names = sorm_csvc(sorm,stmt);
char * data = sorm_csvd(sorm,stmt);
char * result = (char *)malloc(strlen(column_names) + strlen(data) + 2);
char *column_names = sorm_csvc(sorm, stmt);
char *data = sorm_csvd(sorm, stmt);
char *result = (char *)malloc(strlen(column_names) + strlen(data) + 2);
result[0] = 0;
strcat(result,column_names);
if(*column_names)
strcat(result,"\n");
strcat(result, column_names);
if (*column_names)
strcat(result, "\n");
free(column_names);
strcat(result,data);
strcat(result, data);
free(data);
return result;
}
sqlite3_stmt * sormb(sorm_t* db, char * sql, ...){
// Bind parameters to statement and return amount of parameters
int rc = 0;
sqlite3_stmt * stmt;
va_list args;
va_start(args,sql);
unsigned int number = 0;
char * clean_query = (char *)malloc(strlen(sql) + 1);
unsigned int parameter_count = sormcq(sql,clean_query);
free(clean_query);
return stmt;
}
char * sormm(sorm_t * db){
char *sormm(sorm_t *db) {
(void)db;
/* sormmemory */
return rmalloc_stats();
}
sorm_ptr sormq(int sorm, char * sql, ...){
sorm_t * db = sormg(sorm);
if(db->csv){
//free(db->csv);
//db->csv = NULL;
}
sorm_ptr sormq(int sorm, char *sql, ...) {
sorm_t *db = sormg(sorm);
_sorm_query_start = nsecs();
db->time_query_start = nsecs();
va_list args;
va_start (args,sql);
sqlite3_stmt * stmt;
va_start(args, sql);
sqlite3_stmt *stmt;
sorm_ptr result = NULL;
char * clean_query = malloc(strlen(sql) + 1);
unsigned int parameter_count = sormcq(sql,clean_query);
char *clean_query = malloc(strlen(sql) + 1);
uint parameter_count = sormcq(sql, clean_query);
int rc = sqlite3_prepare_v2(db->conn, clean_query, -1, &stmt, 0);
if (rc != SQLITE_OK) {
fprintf(stderr, "%s\n", sqlite3_errmsg(db->conn));
}
free(clean_query);
int number = 0;
for(int i = 0; i < parameter_count; i++){
for (uint i = 0; i < parameter_count; i++) {
number++;
char * column_type = sormpt(sql,number);
int arg_index = number - 1;
if(!strcmp(column_type, "int") || !strcmp(column_type, "integer") || !strcmp(column_type, "number")) {
rc = sqlite3_bind_int(stmt, number, va_arg(args,int));
}else if(!strcmp(column_type, "int64")){
rc = sqlite3_bind_int64(stmt, number,va_arg(args,__uint64_t));
}else if(!strcmp(column_type, "double") || !strcmp(column_type, "dec") || !strcmp(column_type, "decimal") || !strcmp(column_type, "float")){
rc = sqlite3_bind_double(stmt, number,va_arg(args,double));
}else if(!strcmp(column_type, "blob")){
size_t size = (size_t)va_arg(args,size_t);
unsigned char * data = va_arg(args,unsigned char *);
char *column_type = sormpt(sql, number);
if (!strcmp(column_type, "int") || !strcmp(column_type, "integer") || !strcmp(column_type, "number")) {
rc = sqlite3_bind_int(stmt, number, va_arg(args, int));
} else if (!strcmp(column_type, "int64")) {
rc = sqlite3_bind_int64(stmt, number, va_arg(args, __uint64_t));
} else if (!strcmp(column_type, "double") || !strcmp(column_type, "dec") || !strcmp(column_type, "decimal") ||
!strcmp(column_type, "float")) {
rc = sqlite3_bind_double(stmt, number, va_arg(args, double));
} else if (!strcmp(column_type, "blob")) {
size_t size = (size_t)va_arg(args, size_t);
unsigned char *data = va_arg(args, unsigned char *);
rc = sqlite3_bind_blob(stmt, number, data, size, SQLITE_STATIC);
}else if(!strcmp(column_type,"text") || !strcmp(column_type,"str") || !strcmp(column_type,"string")) {
unsigned char * data = va_arg(args, unsigned char *);
rc = sqlite3_bind_text(stmt, number, data, -1, SQLITE_STATIC);
} else if (!strcmp(column_type, "text") || !strcmp(column_type, "str") || !strcmp(column_type, "string")) {
unsigned char *data = va_arg(args, unsigned char *);
rc = sqlite3_bind_text(stmt, number, (char *)data, -1, SQLITE_STATIC);
}
if (rc != SQLITE_OK) {
fprintf(stderr, "Failed to bind parameters: %s\n", sqlite3_errmsg(db->conn));
result = NULL;
}
}
rc = sqlite3_step(stmt);
rc = sqlite3_step(stmt);
if (rc != SQLITE_DONE && rc != SQLITE_ROW) {
fprintf(stderr, "Execution failed: %s\n", sqlite3_errmsg(db->conn));
}else if(rc == SQLITE_DONE){
if(!sqlite3_strnicmp(sql,"SELECT",6)){
} else if (rc == SQLITE_DONE) {
if (!sqlite3_strnicmp(sql, "SELECT", 6)) {
result = 0;
}else {
result = (sorm_ptr)sqlite3_last_insert_rowid(db->conn);
} else {
result = (sorm_ptr)sqlite3_last_insert_rowid(db->conn);
}
} else if (rc == SQLITE_ROW) {
result = sorm_csv(sorm,stmt);
result = sorm_csv(sorm, stmt);
}
else{
else {
fprintf(stderr, "Execution failed: %s\n", sqlite3_errmsg(db->conn));
}
if(sorm_last_query){
if (sorm_last_query) {
free(sorm_last_query);
}
if(sorm_last_query){
if (sorm_last_query) {
free(sorm_last_query_expanded);
}
sorm_last_query = strdup(sqlite3_sql(stmt));
@@ -400,103 +365,99 @@ sorm_ptr sormq(int sorm, char * sql, ...){
return result;
}
char sormlc(char * sql){
char sormlc(char *sql) {
// returns last char
char last_char = 0;
while(*sql){
if(*sql == ' ' || *sql == '\t' || *sql == '\n')
continue ;
while (*sql) {
if (*sql == ' ' || *sql == '\t' || *sql == '\n')
continue;
//printf("%c\n",*sql);
// printf("%c\n",*sql);
last_char = *sql;
sql++;
}
return last_char;
}
int sormlv(char * csv){
int sormlv(char *csv) {
size_t longest = 0;
while(*csv){
char * found = strstr(csv,";");
if(found){
if(found - csv > longest)
longest = found-csv;
while (*csv) {
char *found = strstr(csv, ";");
if (found) {
if (found - csv > (long int)longest)
longest = found - csv;
csv = csv + (found - csv) + 1;
}else{
} else {
break;
}
}
return longest;
}
sorm_query_t sormqt(char * sql){
while(*sql && isspace(*sql))
sorm_query_t sormqt(char *sql) {
while (*sql && isspace(*sql))
sql++;
if(!sqlite3_strnicmp(sql,"select",6))
if (!sqlite3_strnicmp(sql, "select", 6))
return SORM_SELECT;
else if(!sqlite3_strnicmp(sql, "update",6))
else if (!sqlite3_strnicmp(sql, "update", 6))
return SORM_UPDATE;
else if(!sqlite3_strnicmp(sql, "delete",6))
else if (!sqlite3_strnicmp(sql, "delete", 6))
return SORM_DELETE;
else if(!sqlite3_strnicmp(sql, "create",6)){
else if (!sqlite3_strnicmp(sql, "create", 6)) {
return SORM_CREATE;
}else {
} else {
return SORM_UNKNOWN;
}
}
char * sormrq(FILE * f){
char *sormrq(FILE *f) {
static char buffer[4097];
buffer[0] = 0;
char * bufferp = buffer;
char *bufferp = buffer;
char c;
bool in_string = false;
while((c = fgetc(f)) != EOF && strlen(buffer) != sizeof(buffer) -2){
while ((c = fgetc(f)) != EOF && strlen(buffer) != sizeof(buffer) - 2) {
*bufferp = c;
if(c == '"')
{
if (c == '"') {
in_string = !in_string;
}
if(!in_string && c == ';'){
if (!in_string && c == ';') {
break;
}
bufferp++;
*bufferp = 0;
}
return strdup(buffer);
}
char * sormcsvn(char * csv){
if(!csv || !*csv)
char *sormcsvn(char *csv) {
if (!csv || !*csv)
return NULL;
char * pos = strstr(csv,";");
char * pos2 = strstr(csv,"\n");
if(pos2){
if(pos > pos2){
char *pos = strstr(csv, ";");
char *pos2 = strstr(csv, "\n");
if (pos2) {
if (pos > pos2) {
pos = pos2;
}
//pos = pos > pos2 ? pos2 : pos;
// pos = pos > pos2 ? pos2 : pos;
}
if(!pos || !*pos)
if (!pos || !*pos)
return strdup(csv);
int length = pos - csv;
char * result = malloc(length + 2);
strncpy(result,csv,length);
char *result = malloc(length + 2);
strncpy(result, csv, length);
result[length] = 0;
//csv += strlen(result);
// csv += strlen(result);
return result;
}
char * sormfmt(char * csv){
char *sormfmt(char *csv) {
_sorm_result_format_start = nsecs();
size_t longest = sormlv(csv);
char * field;
char *field;
/*
sormstrn(1)
Execute 3.77s, Format: 36.40s
@@ -511,15 +472,15 @@ char * sormfmt(char * csv){
Execute 3.11s, Format: 36.45s
Memory usage: 6 MB, 96.048 (re)allocated, 96.024 unqiue freed, 2 in use.
*/
sormstr_t * str = sormstrn(longest + 2);
while(*csv && (field = sormcsvn(csv))){
sormstra(str,field);
for(int i = 0; i < longest - strlen(field); i++)
sormstra(str," ");
sormstr_t *str = sormstrn(longest + 2);
while (*csv && (field = sormcsvn(csv))) {
sormstra(str, field);
for (size_t i = 0; i < longest - strlen(field); i++)
sormstra(str, " ");
csv += strlen(field);
while( *csv == ';' || *csv == '\n'){
if(*csv == '\n')
while (*csv == ';' || *csv == '\n') {
if (*csv == '\n')
sormstra(str, "\n");
csv++;
}
@@ -530,47 +491,41 @@ char * sormfmt(char * csv){
return sormstrc(str);
}
void apply_colors(char * csv){
char * end;
void apply_colors(char *csv) {
char *end;
bool even = false;
while(*csv){
printf("%s\n",csv);
end = strstr(csv,"\n");
char * line;
if(end)
{
line = (char *)malloc(end -csv + 1024);
strncpy(line,csv,end-csv);
}else{
while (*csv) {
printf("%s\n", csv);
end = strstr(csv, "\n");
char *line;
if (end) {
line = (char *)malloc(end - csv + 1024);
strncpy(line, csv, end - csv);
} else {
line = (char *)malloc(strlen(csv) + 1024);
strcpy(line, csv);
}
if(even){
printf("%s","\033[37m");
if (even) {
printf("%s", "\033[37m");
}
printf("%s\n",line);
printf("%s\n", line);
free(line);
if(even){
printf("%s","\033[0m");
if (even) {
printf("%s", "\033[0m");
}
even = !even;
csv += end-csv;
if(*csv && *(csv + 1))
csv += end - csv;
if (*csv && *(csv + 1))
csv++;
}
}
void sormfmtd(char * csv){
char * formatted = sormfmt(csv);
printf("%s\n",formatted);
void sormfmtd(char *csv) {
char *formatted = sormfmt(csv);
printf("%s\n", formatted);
free(formatted);
}
#endif
#endif
+57 -58
View File
@@ -1,38 +1,40 @@
import ctypes
import csv
import csv
from csv import DictReader
import io
import tempfile
import tempfile
import time
class DictReader:
def get_column_types(self):
types = []
for column in self.columns:
name = column.split("(")[0]
type = column.split("(")[1]
type = type.split(")")[0]
if type == "integer":
types.append(int)
if type == "text":
types.append(str)
return types
types = []
for column in self.columns:
name = column.split("(")[0]
type = column.split("(")[1]
type = type.split(")")[0]
if type == "integer":
types.append(int)
if type == "text":
types.append(str)
return types
def get_column_names(self):
names = []
for column in self.columns:
name = column.split("(")[0]
names.append(name)
return names
def __init__(self, data):
self.result = 0
if type(data) == int:
self.data = ''
self.data = ""
else:
self.data = data.decode();
self.data = data.decode()
self.rows = [row.split(";")[:-1] for row in self.data.split("\n")]
self.columns = self.rows[0]
self.rows.pop(0)
if type(data) == str:
@@ -41,47 +43,42 @@ class DictReader:
self.result = data
self.column_types = self.get_column_types()
for row in self.rows:
for index,field in enumerate(row):
for index, field in enumerate(row):
row[index] = self.column_types[index](field)
self.column_names = self.get_column_names()
def __iter__(self):
return self.rows.__iter__()
libc = ctypes.CDLL('libc.so.6');
libc = ctypes.CDLL("libc.so.6")
class Sorm:
def __init__(self):
self.lib = ctypes.CDLL('./sorm.so')
self.lib = ctypes.CDLL("./sorm.so")
self.sormq = self.lib.sormq
self.sormq.argtypes = [ctypes.c_int, ctypes.c_char_p];
self.sormq.argtypes = [ctypes.c_int, ctypes.c_char_p]
self.sormq.restype = ctypes.c_char_p
self.sormc = self.lib.sormc
self.sormc.argtypes = [ctypes.c_char_p];
self.sormc.restype = ctypes.c_int;
self.sormc.argtypes = [ctypes.c_char_p]
self.sormc.restype = ctypes.c_int
self.sormd = self.lib.sormd
self.sormd.argtypes = [ctypes.c_int];
self.sormd.argtypes = [ctypes.c_int]
self.sormd.restype = None
self.sormm = self.lib.sormm
self.sormm.argtypes = [ctypes.c_int];
self.sormm.argtypes = [ctypes.c_int]
self.sormm.restype = ctypes.c_char_p
class SormDb(Sorm):
def __init__(self,path):
super().__init__(
)
def __init__(self, path):
super().__init__()
self.path = path
self.conn = None
@@ -93,13 +90,13 @@ class SormDb(Sorm):
def __enter__(self):
self.c()
return self
def __exit__(self,*args, **kwargs):
def __exit__(self, *args, **kwargs):
self.d()
def q(self, sql,*args) -> DictReader:
def q(self, sql, *args) -> DictReader:
ctypes_list = []
for arg in args:
for arg in args:
if type(arg) == int:
ctypes_list.append(ctypes.c_int)
if type(arg) == str:
@@ -109,54 +106,56 @@ class SormDb(Sorm):
self.sormq.restype = ctypes.c_int
else:
self.sormq.restype = ctypes.c_char_p
params = tuple([self.conn, sql.encode()] + list(arg.encode() if type(arg) == str else arg for arg in args))
params = tuple(
[self.conn, sql.encode()]
+ list(arg.encode() if type(arg) == str else arg for arg in args)
)
result = DictReader(self.sormq(*params))
self.m = self.sormm(self.conn).decode()
return result
def d(self):
if not self.conn:
return
self.sormd(self.conn)
self.conn = None
self.conn = None
# Load the shared library
lib = ctypes.CDLL('./sorm.so')
lib = ctypes.CDLL("./sorm.so")
free = libc.free
free.argtypes = [ctypes.c_void_p]
free.restype = None
rsomm = lib.sormm
rsomm.argtypes = [ctypes.c_int];
rsomm.restype = ctypes.c_char_p;
rsomm.argtypes = [ctypes.c_int]
rsomm.restype = ctypes.c_char_p
disconnect = lib.sormd
disconnect.argtypes = [ctypes.c_int];
disconnect.argtypes = [ctypes.c_int]
start = time.time()
for x in range(1):
with SormDb("db.sqlite3") as db:
for x in range(1):
#db.q("BEGIN TRANSACTION")
#for x in range(100000):
# db.q("BEGIN TRANSACTION")
# for x in range(100000):
# db.q("INSERT INTO pony (name,age) VALUES (?s,?d);","Python Pony",1337)
#db.q("COMMIT")
result = db.q("SELECT * FROM pony WHERE id > ?d AND name like ?s ORDER BY id",1337, "%hon Pon%");
#for row in result:
# db.q("COMMIT")
result = db.q(
"SELECT * FROM pony WHERE id > ?d AND name like ?s ORDER BY id",
1337,
"%hon Pon%",
)
# for row in result:
# print(row)
print(result.column_names)
print(len(result.rows),"records")
print(db.m);
print(len(result.rows), "records")
print(db.m)
end = time.time()
duration = end - start
print("Duration: {}s".format(duration))
BIN
View File
Binary file not shown.
+16 -16
View File
@@ -1,19 +1,19 @@
#ifndef SORM_STR_H
#define SORM_STR_H
#include <rlib.h>
#include <string.h>
#include "rlib.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
typedef struct sormstr_t {
char * content;
char *content;
size_t length;
size_t buffer_size;
size_t size;
} sormstr_t;
sormstr_t * sormstrn(size_t buffer_size){
sormstr_t * result = (sormstr_t *)malloc(sizeof(sormstr_t));
sormstr_t *sormstrn(size_t buffer_size) {
sormstr_t *result = (sormstr_t *)malloc(sizeof(sormstr_t));
result->length = 0;
result->size = buffer_size;
result->buffer_size = buffer_size;
@@ -21,30 +21,30 @@ sormstr_t * sormstrn(size_t buffer_size){
result->content[0] = 0;
return result;
}
void sormstra(sormstr_t * str, const char * to_append){
void sormstra(sormstr_t *str, const char *to_append) {
size_t required_new_length = str->length + strlen(to_append);
str->length += strlen(to_append);
if(required_new_length > str->size){
if (required_new_length > str->size) {
str->size += required_new_length + str->buffer_size;
str->content = realloc(str->content,str->size + 1);
}else{
// printf("NO NDEED\n");
str->content = realloc(str->content, str->size + 1);
} else {
// printf("NO NDEED\n");
}
strcat(str->content,to_append);
strcat(str->content, to_append);
str->content[str->length] = 0;
}
void sormstrd(sormstr_t * str){
if(str->content){
void sormstrd(sormstr_t *str) {
if (str->content) {
free(str->content);
}
free(str);
}
char * sormstrc(sormstr_t * str){
char *sormstrc(sormstr_t *str) {
// sorm str convert
char * content = str->content;
char *content = str->content;
str->content = NULL;
sormstrd(str);
return content;
}
#endif
#endif