docs: add dataset ORM tutorial page and update navigation with GFM documentation

Add new Dataset ORM tutorial page covering CRUD operations, query operators, schema evolution, and JSON field handling. Update navigation.yaml to include the new tutorial in the tutorials section. Enhance markdown module documentation with GitHub Flavored Markdown support details including tables, task lists, and language identifier attributes for syntax highlighting. Update tutorial index page and pagination links to reflect the new tutorial order. Refactor dataset.wren module source to introduce Sql and QueryBuilder classes with soft-delete support and identifier validation.
This commit is contained in:
2026-01-26 09:21:03 +00:00
parent 04e467e09b
commit d953925605
40 changed files with 3478 additions and 547 deletions
+628
View File
@@ -0,0 +1,628 @@
{# retoor <retoor@molodetz.nl> #}
{% extends 'page.html' %}
{% set page_title = "Dataset ORM Tutorial" %}
{% set breadcrumb = [{"url": "tutorials/index.html", "title": "Tutorials"}, {"title": "Dataset ORM"}] %}
{% set prev_page = {"url": "tutorials/database-app.html", "title": "Database App"} %}
{% set next_page = {"url": "tutorials/template-rendering.html", "title": "Template Rendering"} %}
{% block article %}
<h1>Dataset ORM Tutorial</h1>
<p>The <code>dataset</code> module provides an ORM-like interface for SQLite databases with automatic schema management. This tutorial covers everything from basic CRUD operations to advanced querying patterns.</p>
<h2>Table of Contents</h2>
<ul>
<li><a href="#introduction">Introduction</a></li>
<li><a href="#getting-started">Getting Started</a></li>
<li><a href="#insert-operations">Insert Operations</a></li>
<li><a href="#querying-data">Querying Data</a></li>
<li><a href="#query-operators">Query Operators</a></li>
<li><a href="#update-operations">Update Operations</a></li>
<li><a href="#delete-operations">Delete Operations</a></li>
<li><a href="#schema-evolution">Schema Evolution</a></li>
<li><a href="#json-fields">Working with JSON Fields</a></li>
<li><a href="#multiple-tables">Multiple Tables</a></li>
<li><a href="#building-an-application">Building a Complete Application</a></li>
<li><a href="#best-practices">Best Practices</a></li>
</ul>
<h2 id="introduction">Introduction</h2>
<p>Traditional database programming requires you to define schemas upfront, write SQL queries manually, and handle type conversions. The dataset module eliminates this boilerplate:</p>
<ul>
<li><strong>Auto-schema</strong> - Tables and columns are created automatically when you insert data</li>
<li><strong>No SQL</strong> - Query using maps with intuitive operators</li>
<li><strong>Type conversion</strong> - Maps and Lists are automatically serialized to JSON</li>
<li><strong>Soft delete</strong> - Records are never permanently lost by default</li>
<li><strong>UUID primary keys</strong> - Every record gets a unique identifier</li>
</ul>
<h2 id="getting-started">Getting Started</h2>
<h3>Opening a Database</h3>
<p>Create a file-based database for persistent storage:</p>
<pre><code>import "dataset" for Dataset
var db = Dataset.open("myapp.db")
var users = db["users"]
db.close()</code></pre>
<p>Or use an in-memory database for testing:</p>
<pre><code>import "dataset" for Dataset
var db = Dataset.memory()
var users = db["users"]
db.close()</code></pre>
<h3>Basic Structure</h3>
<p>Every table automatically has three fields:</p>
<table>
<tr>
<th>Field</th>
<th>Type</th>
<th>Description</th>
</tr>
<tr>
<td><code>uid</code></td>
<td>TEXT (UUID)</td>
<td>Primary key, auto-generated</td>
</tr>
<tr>
<td><code>created_at</code></td>
<td>TEXT (ISO datetime)</td>
<td>Timestamp when record was created</td>
</tr>
<tr>
<td><code>deleted_at</code></td>
<td>TEXT (ISO datetime)</td>
<td>Timestamp when soft-deleted (null if active)</td>
</tr>
</table>
<h2 id="insert-operations">Insert Operations</h2>
<h3>Basic Insert</h3>
<pre><code>import "dataset" for Dataset
var db = Dataset.memory()
var users = db["users"]
var user = users.insert({
"name": "Alice",
"email": "alice@example.com",
"age": 28
})
System.print(user["uid"]) // e.g., "550e8400-e29b-41d4-a716-446655440000"
System.print(user["created_at"]) // e.g., "2024-01-15T10:30:00"
System.print(user["name"]) // "Alice"
db.close()</code></pre>
<h3>Custom UID</h3>
<p>Provide your own UID if needed:</p>
<pre><code>var user = users.insert({
"uid": "custom-id-12345",
"name": "Bob"
})</code></pre>
<h3>Type Inference</h3>
<p>Columns are created with appropriate SQLite types:</p>
<pre><code>users.insert({
"name": "Charlie", // TEXT
"age": 30, // INTEGER
"score": 95.5, // REAL
"active": true, // INTEGER (1 or 0)
"tags": ["a", "b"], // TEXT (JSON array)
"settings": { // TEXT (JSON object)
"theme": "dark"
}
})</code></pre>
<h2 id="querying-data">Querying Data</h2>
<h3>Find All Records</h3>
<pre><code>var allUsers = users.all()
for (user in allUsers) {
System.print("%(user["name"]) - %(user["email"])")
}</code></pre>
<h3>Find by Conditions</h3>
<pre><code>var admins = users.find({"role": "admin"})
var youngAdmins = users.find({
"role": "admin",
"age": 25
})</code></pre>
<h3>Find One Record</h3>
<pre><code>var user = users.findOne({"email": "alice@example.com"})
if (user != null) {
System.print("Found: %(user["name"])")
} else {
System.print("User not found")
}</code></pre>
<h3>Count Records</h3>
<pre><code>var totalUsers = users.count()
System.print("Total users: %(totalUsers)")</code></pre>
<h3>Graceful Handling of Missing Columns</h3>
<p>Querying a column that does not exist returns an empty list instead of crashing:</p>
<pre><code>var result = users.find({"nonexistent_column": "value"})
System.print(result.count) // 0</code></pre>
<h2 id="query-operators">Query Operators</h2>
<p>Use double-underscore suffixes for comparison operators:</p>
<h3>Comparison Operators</h3>
<pre><code>var adults = users.find({"age__gte": 18})
var teenagers = users.find({
"age__gte": 13,
"age__lt": 20
})
var notAdmin = users.find({"role__ne": "admin"})</code></pre>
<h3>Complete Operator Reference</h3>
<table>
<tr>
<th>Suffix</th>
<th>SQL</th>
<th>Example</th>
</tr>
<tr>
<td><code>__gt</code></td>
<td><code>&gt;</code></td>
<td><code>{"age__gt": 18}</code> - age greater than 18</td>
</tr>
<tr>
<td><code>__lt</code></td>
<td><code>&lt;</code></td>
<td><code>{"price__lt": 100}</code> - price less than 100</td>
</tr>
<tr>
<td><code>__gte</code></td>
<td><code>&gt;=</code></td>
<td><code>{"score__gte": 90}</code> - score 90 or higher</td>
</tr>
<tr>
<td><code>__lte</code></td>
<td><code>&lt;=</code></td>
<td><code>{"quantity__lte": 10}</code> - quantity 10 or less</td>
</tr>
<tr>
<td><code>__ne</code></td>
<td><code>!=</code></td>
<td><code>{"status__ne": "deleted"}</code> - status not deleted</td>
</tr>
<tr>
<td><code>__like</code></td>
<td><code>LIKE</code></td>
<td><code>{"name__like": "A\%"}</code> - name starts with A</td>
</tr>
<tr>
<td><code>__in</code></td>
<td><code>IN</code></td>
<td><code>{"status__in": ["active", "pending"]}</code></td>
</tr>
<tr>
<td><code>__null</code></td>
<td><code>IS NULL</code></td>
<td><code>{"deleted_at__null": true}</code> - is null</td>
</tr>
</table>
<h3>LIKE Patterns</h3>
<pre><code>var startsWithA = users.find({"name__like": "A\%"})
var containsSmith = users.find({"name__like": "\%Smith\%"})
var threeLetterNames = users.find({"name__like": "___"})</code></pre>
<div class="admonition note">
<div class="admonition-title">Note</div>
<p>In Wren strings, use <code>\%</code> for the SQL <code>%</code> wildcard and <code>_</code> for single character wildcard.</p>
</div>
<h3>IN Operator</h3>
<pre><code>var priorityUsers = users.find({
"role__in": ["admin", "moderator", "editor"]
})
var specificIds = users.find({
"uid__in": [
"550e8400-e29b-41d4-a716-446655440000",
"6ba7b810-9dad-11d1-80b4-00c04fd430c8"
]
})</code></pre>
<h3>NULL Checking</h3>
<pre><code>var usersWithEmail = users.find({"email__null": false})
var usersWithoutPhone = users.find({"phone__null": true})</code></pre>
<h3>Combining Conditions</h3>
<p>Multiple conditions are combined with AND:</p>
<pre><code>var results = products.find({
"category": "electronics",
"price__gte": 100,
"price__lte": 500,
"stock__gt": 0
})</code></pre>
<h2 id="update-operations">Update Operations</h2>
<h3>Basic Update</h3>
<pre><code>var user = users.findOne({"email": "alice@example.com"})
var changes = users.update({
"uid": user["uid"],
"name": "Alice Smith",
"age": 29
})
System.print("Rows affected: %(changes)")</code></pre>
<h3>Adding New Fields</h3>
<p>Updates can add new columns dynamically:</p>
<pre><code>users.update({
"uid": user["uid"],
"phone": "+1-555-0123",
"verified": true
})</code></pre>
<h2 id="delete-operations">Delete Operations</h2>
<h3>Soft Delete (Default)</h3>
<p>Soft delete sets <code>deleted_at</code> but keeps the record:</p>
<pre><code>var deleted = users.delete(user["uid"])
System.print(deleted) // true if found and deleted
var all = users.all()
// Soft-deleted records are excluded</code></pre>
<h3>Hard Delete</h3>
<p>Permanently remove a record:</p>
<pre><code>users.hardDelete(user["uid"])</code></pre>
<div class="admonition warning">
<div class="admonition-title">Warning</div>
<p>Hard delete is permanent. The record cannot be recovered.</p>
</div>
<h2 id="schema-evolution">Schema Evolution</h2>
<p>The dataset module handles schema changes automatically:</p>
<pre><code>import "dataset" for Dataset
var db = Dataset.memory()
var users = db["users"]
users.insert({"name": "Alice"})
System.print(users.columns) // {"uid": "TEXT", "created_at": "TEXT", "deleted_at": "TEXT", "name": "TEXT"}
users.insert({"name": "Bob", "email": "bob@example.com", "age": 30})
System.print(users.columns) // Now includes "email" and "age"
var alice = users.findOne({"name": "Alice"})
System.print(alice["email"]) // null (column exists but Alice has no value)
db.close()</code></pre>
<h2 id="json-fields">Working with JSON Fields</h2>
<h3>Storing Complex Data</h3>
<pre><code>import "dataset" for Dataset
var db = Dataset.memory()
var posts = db["posts"]
posts.insert({
"title": "Getting Started with Wren",
"tags": ["wren", "programming", "tutorial"],
"metadata": {
"author": "Alice",
"views": 1250,
"featured": true,
"related": [101, 102, 103]
}
})
var post = posts.findOne({"title": "Getting Started with Wren"})
System.print(post["tags"]) // ["wren", "programming", "tutorial"]
System.print(post["tags"][0]) // "wren"
System.print(post["metadata"]["author"]) // "Alice"
System.print(post["metadata"]["views"]) // 1250
System.print(post["metadata"]["related"][0]) // 101
db.close()</code></pre>
<h3>Nested Updates</h3>
<p>To update nested JSON, retrieve the full object, modify it, and save:</p>
<pre><code>var post = posts.findOne({"title": "Getting Started with Wren"})
var metadata = post["metadata"]
metadata["views"] = metadata["views"] + 1
posts.update({
"uid": post["uid"],
"metadata": metadata
})</code></pre>
<h2 id="multiple-tables">Multiple Tables</h2>
<pre><code>import "dataset" for Dataset
var db = Dataset.open("blog.db")
var authors = db["authors"]
var posts = db["posts"]
var comments = db["comments"]
var author = authors.insert({
"name": "Alice",
"bio": "Tech writer and developer"
})
var post = posts.insert({
"author_uid": author["uid"],
"title": "Introduction to Dataset",
"content": "The dataset module provides..."
})
comments.insert({
"post_uid": post["uid"],
"author_name": "Bob",
"content": "Great article!"
})
System.print(db.tables) // ["authors", "posts", "comments"]
var postComments = comments.find({"post_uid": post["uid"]})
System.print("Comments on post: %(postComments.count)")
db.close()</code></pre>
<h2 id="building-an-application">Building a Complete Application</h2>
<p>Let's build a task management application:</p>
<pre><code>import "dataset" for Dataset
import "io" for Stdin
class TaskManager {
construct new(dbPath) {
_db = Dataset.open(dbPath)
_tasks = _db["tasks"]
_categories = _db["categories"]
initDefaults()
}
initDefaults() {
if (_categories.count() == 0) {
_categories.insert({"name": "Work", "color": "blue"})
_categories.insert({"name": "Personal", "color": "green"})
_categories.insert({"name": "Shopping", "color": "orange"})
}
}
addTask(title, categoryName, priority) {
var category = _categories.findOne({"name": categoryName})
return _tasks.insert({
"title": title,
"category_uid": category != null ? category["uid"] : null,
"priority": priority,
"completed": false
})
}
listTasks() {
var tasks = _tasks.find({"completed": false})
for (task in tasks) {
var cat = task["category_uid"] != null ?
_categories.findOne({"uid": task["category_uid"]}) : null
var catName = cat != null ? cat["name"] : "None"
System.print("[%(task["priority"])] %(task["title"]) (%(catName))")
}
}
completeTask(title) {
var task = _tasks.findOne({"title": title})
if (task != null) {
_tasks.update({"uid": task["uid"], "completed": true})
return true
}
return false
}
listCategories() {
var categories = _categories.all()
for (cat in categories) {
var count = _tasks.find({
"category_uid": cat["uid"],
"completed": false
}).count
System.print("%(cat["name"]): %(count) tasks")
}
}
stats() {
var total = _tasks.count()
var completed = _tasks.find({"completed": true}).count
var pending = _tasks.find({"completed": false}).count
var highPriority = _tasks.find({
"completed": false,
"priority__gte": 3
}).count
System.print("Total: %(total)")
System.print("Completed: %(completed)")
System.print("Pending: %(pending)")
System.print("High priority: %(highPriority)")
}
close() {
_db.close()
}
}
var app = TaskManager.new("tasks.db")
app.addTask("Write documentation", "Work", 3)
app.addTask("Buy groceries", "Shopping", 2)
app.addTask("Learn Wren", "Personal", 3)
System.print("--- Tasks ---")
app.listTasks()
System.print("\n--- Categories ---")
app.listCategories()
System.print("\n--- Statistics ---")
app.stats()
app.completeTask("Write documentation")
System.print("\n--- After completion ---")
app.listTasks()
app.close()</code></pre>
<p>Output:</p>
<pre><code>--- Tasks ---
[3] Write documentation (Work)
[2] Buy groceries (Shopping)
[3] Learn Wren (Personal)
--- Categories ---
Work: 1 tasks
Personal: 1 tasks
Shopping: 1 tasks
--- Statistics ---
Total: 3
Completed: 0
Pending: 3
High priority: 2
--- After completion ---
[2] Buy groceries (Shopping)
[3] Learn Wren (Personal)</code></pre>
<h2 id="best-practices">Best Practices</h2>
<h3>1. Use Meaningful Table Names</h3>
<pre><code>// Good
var users = db["users"]
var orderItems = db["order_items"]
// Avoid
var t1 = db["t1"]
var data = db["data"]</code></pre>
<h3>2. Close the Database</h3>
<pre><code>var db = Dataset.open("app.db")
// ... operations ...
db.close() // Always close when done</code></pre>
<h3>3. Use Soft Delete for Recoverable Data</h3>
<pre><code>users.delete(uid) // Preferred: data can be recovered
users.hardDelete(uid) // Only when permanent deletion is required</code></pre>
<h3>4. Check for null Results</h3>
<pre><code>var user = users.findOne({"email": email})
if (user == null) {
System.print("User not found")
return
}
// Use user safely</code></pre>
<h3>5. Use In-Memory Databases for Tests</h3>
<pre><code>var db = Dataset.memory() // Fast, isolated, no cleanup needed</code></pre>
<h3>6. Use uid Instead of id</h3>
<pre><code>// The primary key is "uid", not "id"
var user = users.insert({"name": "Alice"})
System.print(user["uid"]) // Correct
// user["id"] does not exist</code></pre>
<h3>7. Store Related Data with UIDs</h3>
<pre><code>var author = authors.insert({"name": "Alice"})
var post = posts.insert({
"author_uid": author["uid"], // Reference by uid
"title": "My Post"
})</code></pre>
<div class="admonition tip">
<div class="admonition-title">Tip</div>
<p>For complex queries that cannot be expressed with the query operators (JOINs, GROUP BY, subqueries), access the underlying SQLite database via <code>db.db</code> and write raw SQL.</p>
</div>
<pre><code>var results = db.db.query("
SELECT p.title, a.name as author_name
FROM posts p
JOIN authors a ON p.author_uid = a.uid
WHERE p.deleted_at IS NULL
")</code></pre>
<h2>Next Steps</h2>
<ul>
<li>See the <a href="../api/dataset.html">Dataset API Reference</a> for complete method documentation</li>
<li>Learn about <a href="../api/sqlite.html">raw SQLite access</a> for complex queries</li>
<li>Build a <a href="web-server.html">web application</a> with dataset storage</li>
<li>Use <a href="template-rendering.html">Jinja templates</a> to render database content</li>
</ul>
{% endblock %}