chore: update css, js files

feat: implement channel creation, settings, and deletion UI
feat: add RPC methods for channel operations

.gitignore

@@ -1,3 +1,5 @@
+
+snek-container-compose.yml
 .r_history
 .vscode
 .history

CHANGELOG.md

@@ -0,0 +1,194 @@
+# Changelog
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+## Version 1.24.0 - 2026-01-03
+
+update css, js files
+
+**Changes:** 3 files, 60 lines
+**Languages:** CSS (44 lines), JavaScript (16 lines)
+
+## Version 1.23.0 - 2026-01-03
+
+update py files
+
+**Changes:** 1 files, 30 lines
+**Languages:** Python (30 lines)
+
+## Version 1.22.0 - 2026-01-03
+
+update py files
+
+**Changes:** 1 files, 8 lines
+**Languages:** Python (8 lines)
+
+## Version 1.21.0 - 2026-01-03
+
+update py files
+
+**Changes:** 1 files, 2 lines
+**Languages:** Python (2 lines)
+
+## Version 1.20.0 - 2026-01-03
+
+update css files
+
+**Changes:** 2 files, 25 lines
+**Languages:** CSS (25 lines)
+
+## Version 1.19.0 - 2026-01-03
+
+update css, html, js files
+
+**Changes:** 4 files, 311 lines
+**Languages:** CSS (137 lines), HTML (3 lines), JavaScript (171 lines)
+
+## Version 1.18.0 - 2026-01-03
+
+update html, js files
+
+**Changes:** 2 files, 271 lines
+**Languages:** HTML (39 lines), JavaScript (232 lines)
+
+## Version 1.17.0 - 2026-01-03
+
+update css, js, py files
+
+**Changes:** 4 files, 17 lines
+**Languages:** CSS (11 lines), JavaScript (2 lines), Python (4 lines)
+
+## Version 1.16.0 - 2026-01-03
+
+update css, js files
+
+**Changes:** 3 files, 15 lines
+**Languages:** CSS (11 lines), JavaScript (4 lines)
+
+## Version 1.15.0 - 2026-01-03
+
+update css files
+
+**Changes:** 1 files, 19 lines
+**Languages:** CSS (19 lines)
+
+## Version 1.14.0 - 2025-12-26
+
+Users can now create, configure settings for, and delete channels through dedicated dialog interfaces. Developers access new RPC methods to support these channel management operations.
+
+**Changes:** 9 files, 801 lines
+**Languages:** CSS (127 lines), HTML (517 lines), Python (157 lines)
+
+## Version 1.13.0 - 2025-12-24
+
+Improves performance in the balancer, socket service, and cache by removing async locks. Adds database connection injection for testing in the app and updates pytest configuration.
+
+**Changes:** 5 files, 291 lines
+**Languages:** Other (6 lines), Python (285 lines)
+
+## Version 1.12.0 - 2025-12-21
+
+The socket service enhances concurrency and performance through improved locking and asynchronous handling. The RPC view renames the scheduled list to tasks for clearer API terminology.
+
+**Changes:** 2 files, 243 lines
+**Languages:** Python (243 lines)
+
+## Version 1.11.0 - 2025-12-19
+
+Adds the ability to configure whether user registration is open or closed via system settings. Administrators can now toggle registration openness to control access to the registration form.
+
+**Changes:** 5 files, 262 lines
+**Languages:** HTML (185 lines), Python (77 lines)
+
+## Version 1.10.0 - 2025-12-19
+
+Users must now receive an invitation to register for an account, as open registration is disabled.
+
+**Changes:** 1 files, 6 lines
+**Languages:** HTML (6 lines)
+
+## Version 1.9.0 - 2025-12-18
+
+Adds a debug logging option to the serve command for enhanced troubleshooting. Improves error handling across the application and corrects a typo in the ip2location middleware.
+
+**Changes:** 148 files, 1047 lines
+**Languages:** JavaScript (299 lines), Python (748 lines)
+
+## Version 1.8.0 - 2025-12-18
+
+The socket service now handles errors more robustly and prevents crashes through improved safety checks. Socket methods support better concurrency and provide enhanced logging for developers.
+
+**Changes:** 4 files, 2279 lines
+**Languages:** JavaScript (592 lines), Python (1687 lines)
+
+## Version 1.7.0 - 2025-12-17
+
+Fixes socket cleanup in the websocket handler to prevent resource leaks and improve connection stability.
+
+**Changes:** 1 files, 29 lines
+**Languages:** Python (29 lines)
+
+## Version 1.6.0 - 2025-12-17
+
+Removes presence debounce to make user departures instant. Simplifies websocket connection and error handling in RPC views for improved reliability.
+
+**Changes:** 2 files, 98 lines
+**Languages:** Python (98 lines)
+
+## Version 1.5.0 - 2025-12-17
+
+remove umami analytics script
+
+**Changes:** 1 files, 1 lines
+**Languages:** HTML (1 lines)
+
+## Version 1.4.0 - 2025-12-17
+
+Updates the socket service to improve connection stability and error handling.
+
+**Changes:** 1 files, 4 lines
+**Languages:** Python (4 lines)
+
+## Version 1.3.0 - 2025-12-17
+
+Users now receive notifications when other users join or depart the application. Departure notifications are debounced to reduce the frequency of rapid successive alerts.
+
+**Changes:** 5 files, 418 lines
+**Languages:** HTML (1 lines), JavaScript (259 lines), Python (158 lines)
+
+## Version 1.2.0 - 2025-12-17
+
+Removes Umami analytics integration, eliminating user tracking functionality. Developers must handle analytics separately if needed.
+
+**Changes:** 2 files, 2 lines
+**Languages:** HTML (1 lines), Python (1 lines)
+
+## Version 1.1.0 - 2025-12-17
+
+Fixes potential errors in forum message handling by adding a null check for the star field, preventing crashes when the field is missing. Updates the message list display to handle starred messages more reliably.
+
+**Changes:** 3 files, 10 lines
+**Languages:** JavaScript (2 lines), Python (8 lines)

DockerfileUbuntu

@@ -10,3 +10,9 @@ RUN chmod +x r
 
 RUN mv r /usr/local/bin/r 
 
+RUN echo 'root:root' | chpasswd
+
+COPY ./terminal /opt/bootstrap
+COPY ./terminal /opt/snek
+RUN cp -r /root /opt/bootstrap/root
+COPY ./terminal/entry /usr/local/bin/entry

Makefile

@@ -20,7 +20,10 @@ serve: run
 
 run:
 	.venv/bin/snek serve
-	
+
+maintenance:
+	.venv/bin/snek maintenance
+
 install: ubuntu
 	python3.12 -m venv .venv 
 	$(PIP) install -e .

cert.pem

@@ -0,0 +1,21 @@
+-----BEGIN CERTIFICATE-----
+MIIDazCCAlOgAwIBAgIUB7PQvHZD6v8hfxeaDbU3hC0nGQQwDQYJKoZIhvcNAQEL
+BQAwRTELMAkGA1UEBhMCTkwxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoM
+GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0yNTA0MDYxOTUzMDhaFw0yNjA0
+MDYxOTUzMDhaMEUxCzAJBgNVBAYTAk5MMRMwEQYDVQQIDApTb21lLVN0YXRlMSEw
+HwYDVQQKDBhJbnRlcm5ldCBXaWRnaXRzIFB0eSBMdGQwggEiMA0GCSqGSIb3DQEB
+AQUAA4IBDwAwggEKAoIBAQCtYf8PP7QjRJOfK6zmfAZhSKwMowCSYijKeChxsgyn
+hDDE8A/OuOuluJh6M/X+ZH0Q4HWTAaTwrXesBBPhie+4KmtsykiI7QEHXVVrWHba
+6t5ymKiFiu+rWMwJVznS7T8K+DPGLRO2bF71Fme4ofJ2Plb7PnF53R4Tc3aTMdIW
+HrUsU1JMNmbCibSVlkfPXSg/HY3XLysCrtrldPHYbTGvBcDUil7qZ8hZ8ZxLMzu3
+GPo6awPc0RBqw3tZu6SCECwQJEM0gX2n5nSyVz+fVgvLozNL9kV89hbZo7H/M37O
+zmxVNwsAwoHpAGmnYs3ZYt4Q8duYjF1AtgZyXgXgdMdLAgMBAAGjUzBRMB0GA1Ud
+DgQWBBQtGeiVTYjzWb2hTqJwipRVXU1LnzAfBgNVHSMEGDAWgBQtGeiVTYjzWb2h
+TqJwipRVXU1LnzAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQAc
+1BacrGMlCd5nfYuvQfv0DdTVGc2FSqxPMRGrZKfjvjemgPMs0+DqUwCJiR6oEOGb
+atOYoIBX9KGXSUKRYYc/N75bslwfV1CclNqd2mPxULfks/D8cAzf2mgw4kYSaDHs
+tJkywBe9L6eIK4cQ5YJvutVNVKMYPi+9w+wKog/FafkamFfX/3SLCkGmV0Vv4g0q
+Ro9KmTTQpJUvd63X8bONLs1t8p+HQfWmKlhuVn5+mncNdGREe8dbciXE5FKu8luN
+dr/twoTZTPhmIHPmVEeNxS8hFSiu0iUPTO0HcCAODILGbtXbClA+1Z0ukiRfUya6
+tgVuEk0c64L86qGP7Ply
+-----END CERTIFICATE-----

key.pem

@@ -0,0 +1,28 @@
+-----BEGIN PRIVATE KEY-----
+MIIEvwIBADANBgkqhkiG9w0BAQEFAASCBKkwggSlAgEAAoIBAQCtYf8PP7QjRJOf
+K6zmfAZhSKwMowCSYijKeChxsgynhDDE8A/OuOuluJh6M/X+ZH0Q4HWTAaTwrXes
+BBPhie+4KmtsykiI7QEHXVVrWHba6t5ymKiFiu+rWMwJVznS7T8K+DPGLRO2bF71
+Fme4ofJ2Plb7PnF53R4Tc3aTMdIWHrUsU1JMNmbCibSVlkfPXSg/HY3XLysCrtrl
+dPHYbTGvBcDUil7qZ8hZ8ZxLMzu3GPo6awPc0RBqw3tZu6SCECwQJEM0gX2n5nSy
+Vz+fVgvLozNL9kV89hbZo7H/M37OzmxVNwsAwoHpAGmnYs3ZYt4Q8duYjF1AtgZy
+XgXgdMdLAgMBAAECggEAFnbkqz8fweoNY8mEOiDGWth695rZuh20bKIA63+cRXV1
+NC8T0pRXGT5qUyW5sQpSwgWzINGiY09hJWJ/M5vBpDpVd4pbYj0DAxyZXV01mSER
+TVvGNKH5x65WUWeB0Hh40J0JaEXy5edIrmIGx6oEAO9hfxAUzStUeES05QFxgk1Q
+RI4rKgvVt4W4wEGqSqX7OMwSU1EHJkX+IKYUdXvFA4Gi192mHHhX9MMDK/RSaDOC
+1ZzHzHeKoTlf4jaUcwATlibo8ExGu4wsY+y3+NKE15o6D36AZD7ObqDOF1RsyfGG
+eyljXzcglZAJN9Ctrz0xj5Xt22HqwsPO0o0mJ7URYQKBgQDcUWiu2acJJyjEJ89F
+aiw3z5RvyO9LksHXwkf6gAV+dro/JeUf7u9Qgz3bwnoqwL16u+vjZxrtcpzkjc2C
++DIr6spCf8XkneJ2FovrFDe6oJSFxbgeexkQEBgw0TskRKILN8PGS6FAOfe8Zkwz
+OHAJOYjxoVVoSeDPnxdu6uwJSQKBgQDJdpwZrtjGKSxkcMJzUlmp3XAPdlI1hZkl
+v56Sdj6+Wz9bNTFlgiPHS+4Z7M+LyotShOEqwMfe+MDqVxTIB9TWfnmvnFDxI1VB
+orHogWVWMHOqPJAzGrrWgbG2CSIiwQ3WFxU1nXqAeNk9aIFidGco87l3lVb4XEZs
+eoUOUic/8wKBgQCK6r3x+gULjWhz/pH/t8l3y2hR78WKxld5XuQZvB06t0wKQy+s
+qfC1uHsJlR+I04zl1ZYQBdQBwlHQ/uSFX0/rRxkPQxeZZkADq4W/zTiycUwU6S2F
+8qJD8ZH/Pf5niOsP3bKQ1uEu6R4e6fXEGiLyfheuG8cJggPBhhO1eWUpGQKBgQDC
+L+OzFce46gLyJYYopl3qz5iuLrx6/nVp31O3lOZRkZ52CcW9ND3MYjH1Jz++XNMC
+DTcEgKGnGFrLBnjvfiz3Ox2L2b5jUE1jYLDfjanh8/3pP0s3FzK0hHqJHjCbEz6E
+9+bnsQ1dPB8Zg9wCzHSLErHYxEf6SOdQtJ//98wBZQKBgQDLON5QPUAJ21uZRvwv
+9LsjKMpd5f/L6/q5j6YYXNpys5MREUgryDpR/uqcmyBuxCU3vBeK8tpYJzfXqO45
+5jFoiKhtEFXjb1+d18ACKg1gXQF0Ljry59HGiZOw7IubRPHh9CDdT5tzynylipr3
+xhhX7RsDOYMFKmn59DS1CQCZAA==
+-----END PRIVATE KEY-----

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "Snek"
-version = "1.0.0"
+version = "1.24.0"
 readme = "README.md"
 #license = { file = "LICENSE", content-type="text/markdown" }
 description = "Snek Chat Application by Molodetz"
@@ -38,6 +38,10 @@ dependencies = [
     "humanize",
     "Pillow",
     "pillow-heif",
+    "IP2Location",
+    "bleach",
+    "sentry-sdk",
+    "bcrypt"
 ]
 
 [tool.setuptools.packages.find]
@@ -48,3 +52,10 @@ where = ["src"] # <-- this changed
 
 [project.scripts]
 snek = "snek.__main__:main"
+
+[project.optional-dependencies]
+test = [
+    "pytest",
+    "pytest-asyncio",
+    "pytest-aiohttp"
+]

pytest.ini

@@ -0,0 +1,15 @@
+[pytest]
+testpaths = tests
+python_files = test_*.py
+python_classes = Test*
+python_functions = test_*
+addopts =
+    --strict-markers
+    --disable-warnings
+    --tb=short
+    -v
+markers =
+    unit: Unit tests
+    integration: Integration tests
+    slow: Slow running tests
+asyncio_mode = auto

review.md

@@ -0,0 +1,55 @@
+# Project Quality Review: Snek
+
+## Overview
+The Snek project is a comprehensive web-based application, functioning as a collaborative platform or chat system with extensive features including user management, channels, repositories, containers, real-time communication via WebSockets, Docker integration, Git support, and more. It leverages Python (primarily with aiohttp for the backend) and JavaScript for the frontend. The codebase is organized into modules such as app, mapper, model, service, view, system, and static assets. This review is based on a literal examination of every Python file in `src/snek/*` recursively, as their full contents have been provided.
+
+## Strengths
+- **Modular Architecture**: The code exhibits strong separation of concerns with dedicated modules for mappers (data access), models (data structures), services (business logic), views (HTTP/WebSocket handlers), and system utilities. This promotes maintainability and scalability.
+- **Asynchronous Support**: Extensive use of async/await throughout (e.g., in `src/snek/app.py`, `src/snek/service/socket.py`, `src/snek/view/rpc.py`), effectively handling I/O-bound operations like WebSockets, database queries, and external API calls.
+- **Feature-Rich**: Supports a wide array of features, including user authentication, file uploads, terminal emulation, Docker container management, Git repositories, WebDAV, SSH, forums, push notifications, and avatar generation. Integration with external tools like Docker and Git is notable.
+- **Frontend Components**: Custom JavaScript components (e.g., in `src/snek/static/njet.js`) provide a modern, component-based UI with event handling and REST clients.
+- **Caching and Utilities**: Robust caching implementation (e.g., in `src/snek/system/cache.py`) with LRU eviction, and utility services (e.g., `src/snek/service/util.py`) enhance performance and reusability.
+- **Model and Field System**: A sophisticated model system (e.g., in `src/snek/system/model.py`) with typed fields (e.g., UUIDField, CreatedField) ensures data integrity and validation.
+- **Extensibility**: The codebase includes hooks for extensions (e.g., Jinja2 extensions in `src/snek/system/markdown.py`, `src/snek/system/template.py`) and sub-applications (e.g., forum in `src/snek/forum.py`).
+- **Error Handling in Places**: Some areas show good error handling (e.g., try-except in `src/snek/service/socket.py`, `src/snek/view/rpc.py`), though not universal.
+
+## Weaknesses
+- **Syntax Errors**:
+  - In `src/snek/research/serpentarium.py`, line 25: `self.setattr(self, "db", self.get)` – `setattr` is misspelled; it should be `setattr`. Additionally, the line ends with `)`, which is misplaced and causes a syntax error. Line 26: `self.setattr(self, "db", self.set)` – similar misspelling and incorrect assignment (self.set is not defined).
+  - In `src/snek/sync.py`, line 25: `self.setattr(self, "db", self.get)` – `setattr` misspelled and misplaced `)`. Line 26: `self.setattr(self, "db", self.set)` – same issues. Line 27: `super()` – called without arguments, which may fail in classes with multiple inheritance or if arguments are expected.
+- **Security Concerns**:
+  - **Injection Vulnerabilities**: Raw SQL queries without parameterization (e.g., in `src/snek/service/channel.py`, `src/snek/service/user.py`, `src/snek/view/rpc.py`) risk SQL injection. For example, in `src/snek/service/channel.py`, queries like `f"SELECT ... WHERE channel_uid=:channel_uid {history_start_filter}"` use string formatting, but not all are parameterized.
+  - **Authentication Gaps**: Some WebSocket and RPC endpoints lack visible authentication checks (e.g., in `src/snek/view/rpc.py`, methods like `echo` and `query` don't enforce login). Basic auth in `src/snek/webdav.py` and `src/snek/sssh.py` is present but hardcoded.
+  - **Input Validation**: Minimal input sanitization (e.g., in `src/snek/system/form.py`, forms have placeholders but no regex or length checks enforced). User inputs in RPC calls (e.g., `src/snek/view/rpc.py`) are not validated.
+  - **Hardcoded Secrets**: Database paths (e.g., 'sqlite:///snek.db' in multiple files), keys (e.g., SESSION_KEY in `src/snek/app.py`), and credentials are hardcoded, posing risks if exposed.
+  - **Privilege Escalation**: Admin checks are inconsistent (e.g., in `src/snek/service/channel.py`, `src/snek/view/rpc.py`), and some operations (e.g., clearing channels) only check `is_admin` without further validation.
+  - **WebDAV and SSH**: `src/snek/webdav.py` and `src/snek/sssh.py` handle file access but lack rate limiting or detailed permission checks.
+- **Code Quality Issues**:
+  - **Inconsistent Naming and Style**: Mixed camelCase and snake_case (e.g., `set_typing` vs. `generateUniqueId` in JS). Some classes have similar names (e.g., `DatasetWebSocketView` in research files).
+  - **Lack of Documentation**: Few docstrings or comments (e.g., no docstrings in `src/snek/service/user.py`, `src/snek/view/channel.py`). Methods are often self-explanatory but lack context.
+  - **Hardcoded Values**: URLs, paths, and constants are hardcoded (e.g., in `src/snek/static/njet.js`, `src/snek/app.py`).
+  - **Global State and Side Effects**: Use of global variables (e.g., in `src/snek/system/markdown.py`, `src/snek/view/rpc.py`) and mutable defaults can lead to bugs.
+  - **Performance Issues**: No visible optimization for large datasets (e.g., in `src/snek/service/channel_message.py`, queries could be inefficient). Process pools in `src/snek/service/channel_message.py` are per-message, potentially wasteful.
+  - **Error Handling Gaps**: Many methods lack try-except (e.g., in `src/snek/system/docker.py`, `src/snek/system/terminal.py`). Exceptions are sometimes caught but not logged properly.
+  - **Dependencies**: Imports like `dataset`, `git`, `pymongo` (implied) are not version-pinned, risking compatibility issues.
+  - **Testing Absence**: No visible unit or integration tests in the codebase.
+- **Potential Bugs**:
+  - In `src/snek/research/serpentarium.py`, `self.setattr(self, "db", self.set)` assigns undefined `self.set`.
+  - In `src/snek/sync.py`, similar assignment issues.
+  - In `src/snek/view/rpc.py`, `self.user_uid` property assumes `self.view.session.get("uid")`, but no null checks.
+  - In `src/snek/system/docker.py`, `ComposeFileManager` uses subprocess without full error handling.
+  - In `src/snek/service/channel_message.py`, executor pools per UID could lead to resource leaks if not cleaned up.
+  - In `src/snek/forum.py`, event listeners are added but no removal logic is visible.
+- **Maintainability**: Large files (e.g., `src/snek/view/rpc.py` is over 1000 lines) and complex methods (e.g., in `src/snek/app.py`) make refactoring hard. Some code duplication (e.g., WebSocket handling in multiple views).
+
+## Recommendations
+- **Fix Syntax Errors Immediately**: Correct `setattr` spellings, remove misplaced `)`, and fix `super()` calls in `src/snek/research/serpentarium.py` and `src/snek/sync.py` to prevent runtime failures.
+- **Enhance Security**: Implement parameterized queries, add input validation (e.g., using regex in `src/snek/system/form.py`), enforce authentication in all endpoints, and use environment variables for secrets.
+- **Improve Code Quality**: Add docstrings, comments, and consistent naming. Refactor large methods and remove hardcoded values.
+- **Add Error Handling and Testing**: Wrap risky operations in try-except, log errors, and introduce unit tests (e.g., using pytest).
+- **Optimize Performance**: Review query efficiency, add indexing, and manage resources (e.g., executor pools).
+- **Address Bugs**: Fix undefined assignments and add null checks.
+- **General**: Pin dependencies, review for race conditions in async code, and consider code reviews or linters (e.g., flake8, mypy).
+
+## Grade
+B- (Solid foundation with good architecture and features, but critical syntax errors, security vulnerabilities, and quality issues require immediate attention to avoid production risks.)

src/snek/__init__.py

@@ -1 +1,70 @@
+# retoor <retoor@molodetz.nl>
 
+"""
+MIT License
+
+Copyright (c) 2025 retoor
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+Author: retoor <retoor@molodetz.nl>
+Description: Utility to load environment variables from a .env file.
+"""
+
+import os
+from typing import Optional
+
+def load_env(file_path: str = '.env') -> None:
+    """
+    Loads environment variables from a specified file into the current process environment.
+
+    Args:
+        file_path (str): Path to the environment file. Defaults to '.env'.
+
+    Returns:
+        None
+
+    Raises:
+        FileNotFoundError: If the specified file does not exist.
+        IOError: If an I/O error occurs during file reading.
+    """
+    try:
+        with open(file_path, 'r') as env_file:
+            for line in env_file:
+                line = line.strip()
+                # Skip empty lines and comments
+                if not line or line.startswith('#'):
+                    continue
+                # Skip lines without '='
+                if '=' not in line:
+                    continue
+                # Split into key and value at the first '='
+                key, value = line.split('=', 1)
+                # Set environment variable
+                os.environ[key.strip()] = value.strip()
+    except FileNotFoundError:
+        raise FileNotFoundError(f"Environment file '{file_path}' not found.")
+    except IOError as e:
+        raise IOError(f"Error reading environment file '{file_path}': {e}")
+
+
+try:
+    load_env()
+except Exception as e:
+    pass

src/snek/__main__.py

@@ -1,27 +1,88 @@
-import click
-import uvloop
-from aiohttp import web
+# retoor <retoor@molodetz.nl>
+
 import asyncio
-from snek.app import Application
-from IPython import start_ipython
-import sqlite3
+import logging
 import pathlib
-import shutil 
+import shutil
+import sqlite3
+
+import click
+from aiohttp import web
+
+from snek.app import Application
+from snek.shell import Shell
+
+logging.basicConfig(level=logging.INFO)
+
 
 @click.group()
 def cli():
     pass
 
 @cli.command()
-@click.option('--db_path',default="snek.db", help='Database to initialize if not exists.')
-@click.option('--source',default=None, help='Database to initialize if not exists.')
-def init(db_path,source):
+def export():
+
+    app = Application(db_path="sqlite:///snek.db")
+
+    async def fix_message(message):
+        message = {
+            "uid": message["uid"],
+            "user_uid": message["user_uid"],
+            "text": message["message"],
+            "sent": message["created_at"],
+        }
+        user = await app.services.user.get(uid=message["user_uid"])
+        message["user"] = user and user["username"] or None
+        return (message["user"] or "") + ": " + (message["text"] or "")
+    async def run():
+
+        result = []
+        for channel in app.db["channel"].find(
+            is_private=False, is_listed=True, tag="public"
+        ):
+            print(f"Dumping channel: {channel['label']}.")
+            result += [
+                await fix_message(record)
+                for record in app.db["channel_message"].find(
+                    channel_uid=channel["uid"], order_by="created_at"
+                )
+            ]
+            print("Dump succesfull!")
+        print("Converting to json.")
+        print("Converting succesful, now writing to dump.txt")
+        with open("dump.txt", "w") as f:
+            f.write("\n\n".join(result))
+        print("Dump written to dump.json")
+    asyncio.run(run())
+
+
+@cli.command()
+def statistics():
+    async def run():
+        app = Application(db_path="sqlite:///snek.db")
+        app.services.statistics.database()
+    asyncio.run(run())
+
+@cli.command()
+def maintenance():
+    async def run():
+        app = Application(db_path="sqlite:///snek.db")
+        await app.services.container.maintenance()
+        await app.services.channel_message.maintenance()
+    asyncio.run(run())
+
+@cli.command()
+@click.option(
+    "--db_path", default="snek.db", help="Database to initialize if not exists."
+)
+@click.option("--source", default=None, help="Database to initialize if not exists.")
+def init(db_path, source):
     if source and pathlib.Path(source).exists():
         print(f"Copying {source} to {db_path}")
-        shutil.copy2(source,db_path)
+        shutil.copy2(source, db_path)
         print("Database initialized.")
         return
- 
+
     if pathlib.Path(db_path).exists():
         return
     print(f"Initializing database at {db_path}")
@@ -33,25 +94,55 @@ def init(db_path,source):
     db.close()
     print("Database initialized.")
 
-@cli.command()
-@click.option('--port', default=8081, show_default=True, help='Port to run the application on')
-@click.option('--host', default='0.0.0.0', show_default=True, help='Host to run the application on')
-@click.option('--db_path', default='snek.db', show_default=True, help='Database path for the application')
-def serve(port, host, db_path):
-    #init(db_path)
-    #asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
-    web.run_app(
-        Application(db_path=f"sqlite:///{db_path}"), port=port, host=host
-    )
 
 @cli.command()
-@click.option('--db_path', default='snek.db', show_default=True, help='Database path for the application')
+@click.option(
+    "--port", default=8081, show_default=True, help="Port to run the application on"
+)
+@click.option(
+    "--host",
+    default="0.0.0.0",
+    show_default=True,
+    help="Host to run the application on",
+)
+@click.option(
+    "--db_path",
+    default="snek.db",
+    show_default=True,
+    help="Database path for the application",
+)
+@click.option(
+    "--debug",
+    is_flag=True,
+    default=False,
+    help="Enable debug logging",
+)
+def serve(port, host, db_path, debug):
+    if debug:
+        logging.getLogger().setLevel(logging.DEBUG)
+        logging.getLogger("snek").setLevel(logging.DEBUG)
+    web.run_app(Application(db_path=f"sqlite:///{db_path}"), port=port, host=host)
+
+
+@cli.command()
+@click.option(
+    "--db_path",
+    default="snek.db",
+    show_default=True,
+    help="Database path for the application",
+)
 def shell(db_path):
-    app = Application(db_path=f"sqlite:///{db_path}")
-    start_ipython(argv=[], user_ns={'app': app})
+    Shell(db_path).run()
 
 def main():
+    try:
+        import sentry_sdk
+        sentry_sdk.init("https://ab6147c2f3354c819768c7e89455557b@gt.molodetz.nl/1")
+    except ImportError:
+        print("Could not import sentry_sdk")
+
     cli()
 
+
 if __name__ == "__main__":
     main()

src/snek/app.py

@@ -1,16 +1,17 @@
+# retoor <retoor@molodetz.nl>
+
 import asyncio
 import logging
 import pathlib
+import ssl
 import time
 import uuid
-from datetime import datetime
-from snek import snode 
-from snek.view.threads import ThreadsView
-import json 
-logging.basicConfig(level=logging.DEBUG)
-
 from concurrent.futures import ThreadPoolExecutor
+from contextlib import asynccontextmanager
+from datetime import datetime
+from ipaddress import ip_address
 
+import IP2Location
 from aiohttp import web
 from aiohttp_session import (
     get_session as session_get,
@@ -21,45 +22,74 @@ from aiohttp_session.cookie_storage import EncryptedCookieStorage
 from app.app import Application as BaseApplication
 from jinja2 import FileSystemLoader
 
-from snek.sssh import start_ssh_server
-from snek.docs.app import Application as DocsApplication
+from snek import snode
 from snek.mapper import get_mappers
 from snek.service import get_services
+from snek.sgit import GitApplication
+from snek.sssh import start_ssh_server
 from snek.system import http
 from snek.system.cache import Cache
 from snek.system.markdown import MarkdownExtension
-from snek.system.middleware import auth_middleware, cors_middleware
+from snek.system.middleware import auth_middleware, cors_middleware, csp_middleware
+from snek.system.config import config
 from snek.system.profiler import profiler_handler
-from snek.system.template import EmojiExtension, LinkifyExtension, PythonExtension
+from snek.system.template import (
+    EmojiExtension,
+    LinkifyExtension,
+    PythonExtension,
+    sanitize_html,
+)
+from snek.view.new import NewView
 from snek.view.about import AboutHTMLView, AboutMDView
 from snek.view.avatar import AvatarView
+from snek.view.channel import ChannelAttachmentView,ChannelAttachmentUploadView, ChannelView
 from snek.view.docs import DocsHTMLView, DocsMDView
-from snek.view.drive import DriveView
-from snek.view.drive import DriveApiView
+from snek.view.drive import DriveApiView, DriveView
+from snek.view.channel import ChannelDriveApiView
+from snek.view.container import ContainerView
 from snek.view.index import IndexView
 from snek.view.login import LoginView
 from snek.view.logout import LogoutView
+from snek.view.push import PushView
 from snek.view.register import RegisterView
-from snek.view.rpc import RPCView
 from snek.view.repository import RepositoryView
+from snek.view.rpc import RPCView
 from snek.view.search_user import SearchUserView
-from snek.view.settings.repositories import RepositoriesIndexView
-from snek.view.settings.repositories import RepositoriesCreateView
-from snek.view.settings.repositories import RepositoriesUpdateView
-from snek.view.settings.repositories import RepositoriesDeleteView
+from snek.view.threads import ThreadsView
+from snek.view.settings.containers import (
+    ContainersCreateView,
+    ContainersDeleteView,
+    ContainersIndexView,
+    ContainersUpdateView,
+)
 from snek.view.settings.index import SettingsIndexView
 from snek.view.settings.profile import SettingsProfileView
+from snek.view.settings.repositories import (
+    RepositoriesCreateView,
+    RepositoriesDeleteView,
+    RepositoriesIndexView,
+    RepositoriesUpdateView,
+)
+from snek.view.settings.profile_pages import (
+    ProfilePagesView,
+    ProfilePageCreateView,
+    ProfilePageEditView,
+    ProfilePageDeleteView,
+)
+from snek.view.profile_page import ProfilePageView
 from snek.view.stats import StatsView
 from snek.view.status import StatusView
 from snek.view.terminal import TerminalSocketView, TerminalView
 from snek.view.upload import UploadView
 from snek.view.user import UserView
 from snek.view.web import WebView
-from snek.view.channel import ChannelAttachmentView
-from snek.view.channel import ChannelView
-from snek.view.settings.containers import ContainersIndexView, ContainersCreateView, ContainersUpdateView, ContainersDeleteView
 from snek.webdav import WebdavApplication
-from snek.sgit import GitApplication
+from snek.forum import setup_forum
+from snek.system.template import whitelist_attributes
+
+logging.basicConfig(level=logging.DEBUG)
+logger = logging.getLogger(__name__)
+
 SESSION_KEY = b"c79a0c5fda4b424189c427d28c9f7c34"
 
 
@@ -70,25 +100,67 @@ async def session_middleware(request, handler):
     return response
 
 
+@web.middleware
+async def ip2location_middleware(request, handler):
+    response = await handler(request)
+    ip = request.headers.get("X-Forwarded-For", request.remote)
+    try:
+        ipaddr = ip_address(ip)
+        if ipaddr.is_private:
+            return response
+    except ValueError:
+        return response
+    if not request.session.get("uid"):
+        return response
+    user = await request.app.services.user.get(uid=request.session.get("uid"))
+    if not user:
+        return response
+    location = request.app.ip2location.get_all(ip)
+    if user["city"] != location.city:
+        user["country_long"] = location.country_long
+        user["country_short"] = location.country_short
+        user["city"] = location.city
+        user["region"] = location.region
+        user["latitude"] = location.latitude
+        user["longitude"] = location.longitude
+        user["ip"] = ip
+        await request.app.services.user.update(user)
+    return response
+
+
 @web.middleware
 async def trailing_slash_middleware(request, handler):
     if request.path and not request.path.endswith("/"):
-        # Redirect to the same path with a trailing slash
         raise web.HTTPFound(request.path + "/")
     return await handler(request)
 
 
 class Application(BaseApplication):
+    async def create_default_forum(self, app):
+        forums = [f async for f in self.services.forum.find(is_active=True)]
+        if not forums:
+            # Find admin user to be the creator
+            admin_user = await self.services.user.get(is_admin=True)
+            if admin_user:
+                await self.services.forum.create_forum(
+                    name="General Discussion",
+                    description="A place for general discussion.",
+                    created_by_uid=admin_user["uid"],
+                )
 
-    def __init__(self, *args, **kwargs):
+    def __init__(self, *args, db_connection=None, **kwargs):
         middlewares = [
             cors_middleware,
-            web.normalize_path_middleware(merge_slashes=True),
+            csp_middleware,
         ]
+        self._test_db = db_connection
         self.template_path = pathlib.Path(__file__).parent.joinpath("templates")
         self.static_path = pathlib.Path(__file__).parent.joinpath("static")
         super().__init__(
-            middlewares=middlewares, template_path=self.template_path, client_max_size=1024*1024*1024*5 *args, **kwargs
+            middlewares=middlewares,
+            template_path=self.template_path,
+            client_max_size=1024 * 1024 * 1024 * 5,
+            **kwargs,
         )
         session_setup(self, EncryptedCookieStorage(SESSION_KEY))
         self.tasks = asyncio.Queue()
@@ -98,33 +170,52 @@ class Application(BaseApplication):
         self.jinja2_env.add_extension(LinkifyExtension)
         self.jinja2_env.add_extension(PythonExtension)
         self.jinja2_env.add_extension(EmojiExtension)
+        self.jinja2_env.filters["sanitize"] = sanitize_html
         self.time_start = datetime.now()
         self.ssh_host = "0.0.0.0"
         self.ssh_port = 2242
-        self.setup_router()
+
+        self.forum = None
         self.ssh_server = None
         self.sync_service = None
         self.executor = None
         self.cache = Cache(self)
+        self.config = config
         self.services = get_services(app=self)
         self.mappers = get_mappers(app=self)
         self.broadcast_service = None
         self.user_availability_service_task = None
-        
+
+        self.setup_router()
+        base_path = pathlib.Path(__file__).parent
+        self.ip2location = IP2Location.IP2Location(
+            base_path.joinpath("IP2LOCATION-LITE-DB11.BIN")
+        )
         self.on_startup.append(self.prepare_asyncio)
         self.on_startup.append(self.start_user_availability_service)
         self.on_startup.append(self.start_ssh_server)
         self.on_startup.append(self.prepare_database)
-       
+        self.on_startup.append(self.create_default_forum)
+
+    @property
+    def db(self):
+        if self._test_db is not None:
+            return self._test_db
+        return self._db
+
+    @db.setter
+    def db(self, value):
+        self._db = value
+
     @property
     def uptime_seconds(self):
         return (datetime.now() - self.time_start).total_seconds()
 
-    @property 
+    @property
     def uptime(self):
         return self._format_uptime(self.uptime_seconds)
-    
-    def _format_uptime(self,seconds):
+
+    def _format_uptime(self, seconds):
         seconds = int(seconds)
         days, seconds = divmod(seconds, 86400)
         hours, seconds = divmod(seconds, 3600)
@@ -142,19 +233,20 @@ class Application(BaseApplication):
 
         return ", ".join(parts)
 
-
     async def start_user_availability_service(self, app):
-        app.user_availability_service_task = asyncio.create_task(app.services.socket.user_availability_service())
+        app.user_availability_service_task = asyncio.create_task(
+            app.services.socket.user_availability_service()
+        )
+
     async def snode_sync(self, app):
         self.sync_service = asyncio.create_task(snode.sync_service(app))
-    
+
     async def start_ssh_server(self, app):
-        app.ssh_server = await start_ssh_server(app,app.ssh_host,app.ssh_port)
+        app.ssh_server = await start_ssh_server(app, app.ssh_host, app.ssh_port)
         if app.ssh_server:
             asyncio.create_task(app.ssh_server.wait_closed())
 
     async def prepare_asyncio(self, app):
-        # app.loop = asyncio.get_running_loop()
         app.executor = ThreadPoolExecutor(max_workers=200)
         app.loop.set_default_executor(self.executor)
 
@@ -169,7 +261,7 @@ class Application(BaseApplication):
                 await task
                 self.tasks.task_done()
             except Exception as ex:
-                print(ex)
+                logger.error(f"Task runner error: {ex}")
             self.db.commit()
 
     async def prepare_database(self, app):
@@ -183,10 +275,10 @@ class Application(BaseApplication):
                 self.db["channel_member"].create_index(["channel_uid", "user_uid"])
             if not self.db["channel_message"].has_index(["channel_uid", "user_uid"]):
                 self.db["channel_message"].create_index(["channel_uid", "user_uid"])
-        except:
-            pass
+        except Exception as ex:
+            logger.warning(f"Index creation error: {ex}")
 
-        await app.services.drive.prepare_all()
+        await self.services.drive.prepare_all()
         self.loop.create_task(self.task_runner())
 
     def setup_router(self):
@@ -197,7 +289,10 @@ class Application(BaseApplication):
             name="static",
             show_index=True,
         )
+
+        self.router.add_view("/new.html", NewView)
         self.router.add_view("/profiler.html", profiler_handler)
+        self.router.add_view("/container/sock/{channel_uid}.json", ContainerView)
         self.router.add_view("/about.html", AboutHTMLView)
         self.router.add_view("/about.md", AboutMDView)
         self.router.add_view("/logout.json", LogoutView)
@@ -208,14 +303,12 @@ class Application(BaseApplication):
         self.router.add_view("/settings/index.html", SettingsIndexView)
         self.router.add_view("/settings/profile.html", SettingsProfileView)
         self.router.add_view("/settings/profile.json", SettingsProfileView)
+        self.router.add_view("/push.json", PushView)
         self.router.add_view("/web.html", WebView)
         self.router.add_view("/login.html", LoginView)
         self.router.add_view("/login.json", LoginView)
         self.router.add_view("/register.html", RegisterView)
         self.router.add_view("/register.json", RegisterView)
-        self.router.add_view("/drive/{rel_path:.*}", DriveView)
-        self.router.add_view("/drive.bin", UploadView)
-        self.router.add_view("/drive.bin/{uid}.{ext}", UploadView)
         self.router.add_view("/search-user.html", SearchUserView)
         self.router.add_view("/search-user.json", SearchUserView)
         self.router.add_view("/avatar/{uid}.svg", AvatarView)
@@ -223,38 +316,64 @@ class Application(BaseApplication):
         self.router.add_get("/http-photo", self.handle_http_photo)
         self.router.add_get("/rpc.ws", RPCView)
         self.router.add_get("/c/{channel:.*}", ChannelView)
-        self.router.add_view("/channel/{channel_uid}/attachment.bin",ChannelAttachmentView)
-        self.router.add_view("/channel/attachment/{relative_url:.*}",ChannelAttachmentView)
+        self.router.add_view(
+            "/channel/{channel_uid}/attachment.sock", ChannelAttachmentUploadView
+        )
+        self.router.add_view(
+            "/channel/attachment/{relative_url:.*}", ChannelAttachmentView
+        )
         self.router.add_view("/channel/{channel}.html", WebView)
         self.router.add_view("/threads.html", ThreadsView)
         self.router.add_view("/terminal.ws", TerminalSocketView)
         self.router.add_view("/terminal.html", TerminalView)
         self.router.add_view("/drive.json", DriveApiView)
         self.router.add_view("/drive.html", DriveView)
-        self.router.add_view("/drive/{drive}.json", DriveView)
+        self.router.add_view("/drive/{rel_path:.*}", DriveView)
         self.router.add_view("/stats.json", StatsView)
         self.router.add_view("/user/{user}.html", UserView)
+        self.router.add_view("/user/{user_uid}/{slug}.html", ProfilePageView)
         self.router.add_view("/repository/{username}/{repository}", RepositoryView)
-        self.router.add_view("/repository/{username}/{repository}/{path:.*}", RepositoryView)
+        self.router.add_view(
+            "/repository/{username}/{repository}/{path:.*}", RepositoryView
+        )
         self.router.add_view("/settings/repositories/index.html", RepositoriesIndexView)
-        self.router.add_view("/settings/repositories/create.html", RepositoriesCreateView)
-        self.router.add_view("/settings/repositories/repository/{name}/update.html", RepositoriesUpdateView)
-        self.router.add_view("/settings/repositories/repository/{name}/delete.html", RepositoriesDeleteView)
+        self.router.add_view(
+            "/settings/repositories/create.html", RepositoriesCreateView
+        )
+        self.router.add_view(
+            "/settings/repositories/repository/{name}/update.html",
+            RepositoriesUpdateView,
+        )
+        self.router.add_view(
+            "/settings/repositories/repository/{name}/delete.html",
+            RepositoriesDeleteView,
+        )
+        self.router.add_view("/settings/profile_pages/index.html", ProfilePagesView)
+        self.router.add_view("/settings/profile_pages/create.html", ProfilePageCreateView)
+        self.router.add_view(
+            "/settings/profile_pages/{page_uid}/edit.html", ProfilePageEditView
+        )
+        self.router.add_view(
+            "/settings/profile_pages/{page_uid}/delete.html", ProfilePageDeleteView
+        )
         self.router.add_view("/settings/containers/index.html", ContainersIndexView)
         self.router.add_view("/settings/containers/create.html", ContainersCreateView)
-        self.router.add_view("/settings/containers/container/{uid}/update.html", ContainersUpdateView)
-        self.router.add_view("/settings/containers/container/{uid}/delete.html", ContainersDeleteView)
+        self.router.add_view(
+            "/settings/containers/container/{uid}/update.html", ContainersUpdateView
+        )
+        self.router.add_view(
+            "/settings/containers/container/{uid}/delete.html", ContainersDeleteView
+        )
         self.webdav = WebdavApplication(self)
         self.git = GitApplication(self)
-        self.add_subapp("/webdav", self.webdav)
-        self.add_subapp("/git",self.git)
-        
-        #self.router.add_get("/{file_path:.*}", self.static_handler)
-    
-    async def handle_test(self, request):
 
-        return await self.render_template(
-            "test.html", request, context={"name": "retoor"}
+        self.add_subapp("/webdav", self.webdav)
+        self.add_subapp("/git", self.git)
+        setup_forum(self)
+
+    async def handle_test(self, request):
+        return await whitelist_attributes(
+            self.render_template("test.html", request, context={"name": "retoor"})
         )
 
     async def handle_http_get(self, request: web.Request):
@@ -269,19 +388,19 @@ class Application(BaseApplication):
             body=path.read_bytes(), headers={"Content-Type": "image/png"}
         )
 
-    # @time_cache_async(60)
     async def render_template(self, template, request, context=None):
+        start_time = time.perf_counter()
         channels = []
         if not context:
             context = {}
         context["rid"] = str(uuid.uuid4())
+        context["config"] = self.config
         if request.session.get("uid"):
             async for subscribed_channel in self.services.channel_member.find(
                 user_uid=request.session.get("uid"), deleted_at=None, is_banned=False
             ):
-                
                 parent_object = await subscribed_channel.get_channel()
-                
+
                 item = {}
                 other_user = await self.services.channel_member.get_other_dm_user(
                     subscribed_channel["channel_uid"], request.session.get("uid")
@@ -313,25 +432,27 @@ class Application(BaseApplication):
                     request.session.get("uid")
                 )
 
-        self.template_path.joinpath(template)
-
-        await self.services.user.get_template_path(request.session.get("uid"))
-
         self.original_loader = self.jinja2_env.loader
 
         self.jinja2_env.loader = await self.get_user_template_loader(
             request.session.get("uid")
         )
 
+        try:
+            context["nonce"] = request["csp_nonce"]
+        except KeyError:
+            context["nonce"] = "?"
+
         rendered = await super().render_template(template, request, context)
 
         self.jinja2_env.loader = self.original_loader
 
+        end_time = time.perf_counter()
+        logger.debug(f"render_template took {end_time - start_time:.4f} seconds")
         return rendered
 
-
     async def static_handler(self, request):
-        file_name = request.match_info.get('filename', '')
+        file_name = request.match_info.get("filename", "")
 
         paths = []
 
@@ -340,12 +461,12 @@ class Application(BaseApplication):
             user_static_path = await self.services.user.get_static_path(uid)
             if user_static_path:
                 paths.append(user_static_path)
-      
+
         for admin_uid in self.services.user.get_admin_uids():
             user_static_path = await self.services.user.get_static_path(admin_uid)
             if user_static_path:
                 paths.append(user_static_path)
-        
+
         paths.append(self.static_path)
 
         for path in paths:
@@ -365,16 +486,37 @@ class Application(BaseApplication):
             if user_template_path:
                 template_paths.append(user_template_path)
 
-
         template_paths.append(self.template_path)
         return FileSystemLoader(template_paths)
 
+    @asynccontextmanager
+    async def no_save(self):
+        stats = {"count": 0}
+
+        async def patched_save(*args, **kwargs):
+            await self.cache.set(args[0]["uid"], args[0])
+            stats["count"] = stats["count"] + 1
+            logger.debug(f"save is ignored {stats['count']} times")
+            return args[0]
+        save_original = self.services.channel_message.mapper.save
+        self.services.channel_message.mapper.save = patched_save
+        raised_exception = None
+        try:
+            yield
+        except Exception as ex:
+            raised_exception = ex
+        finally:
+            self.services.channel_message.mapper.save = save_original
+        if raised_exception:
+            raise raised_exception
 
 app = Application(db_path="sqlite:///snek.db")
 
 
 async def main():
-    await web._run_app(app, port=8081, host="0.0.0.0")
+    ssl_context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
+    ssl_context.load_cert_chain("cert.pem", "key.pem")
+    await web._run_app(app, port=8081, host="0.0.0.0", ssl_context=ssl_context)
 
 
 if __name__ == "__main__":

src/snek/balancer.py

@@ -1,34 +1,37 @@
+# retoor <retoor@molodetz.nl>
+
 import asyncio
 import sys
 
+
 class LoadBalancer:
     def __init__(self, backend_ports):
         self.backend_ports = backend_ports
         self.backend_processes = []
         self.client_counts = [0] * len(backend_ports)
-        self.lock = asyncio.Lock()
 
-    async def start_backend_servers(self,port,workers):
+    async def start_backend_servers(self, port, workers):
         for x in range(workers):
             port += 1
             process = await asyncio.create_subprocess_exec(
                 sys.executable,
                 sys.argv[0],
-                'backend',
+                "backend",
                 str(port),
                 stdout=asyncio.subprocess.PIPE,
-                stderr=asyncio.subprocess.PIPE
+                stderr=asyncio.subprocess.PIPE,
             )
             port += 1
             self.backend_processes.append(process)
-            print(f"Started backend server on port {(port-1)/port} with PID {process.pid}")
+            print(
+                f"Started backend server on port {(port-1)/port} with PID {process.pid}"
+            )
 
     async def handle_client(self, reader, writer):
-        async with self.lock:
-            min_clients = min(self.client_counts)
-            server_index = self.client_counts.index(min_clients)
-            self.client_counts[server_index] += 1
-        backend = ('127.0.0.1', self.backend_ports[server_index])
+        min_clients = min(self.client_counts)
+        server_index = self.client_counts.index(min_clients)
+        self.client_counts[server_index] += 1
+        backend = ("127.0.0.1", self.backend_ports[server_index])
         try:
             backend_reader, backend_writer = await asyncio.open_connection(*backend)
 
@@ -52,8 +55,7 @@ class LoadBalancer:
             print(f"Error: {e}")
         finally:
             writer.close()
-            async with self.lock:
-                self.client_counts[server_index] -= 1
+            self.client_counts[server_index] -= 1
 
     async def monitor(self):
         while True:
@@ -62,10 +64,10 @@ class LoadBalancer:
             for i, count in enumerate(self.client_counts):
                 print(f"Server {self.backend_ports[i]}: {count} clients")
 
-    async def start(self, host='0.0.0.0', port=8081,workers=5):
-        await self.start_backend_servers(port,workers)
+    async def start(self, host="0.0.0.0", port=8081, workers=5):
+        await self.start_backend_servers(port, workers)
         server = await asyncio.start_server(self.handle_client, host, port)
-        monitor_task = asyncio.create_task(self.monitor())
+        asyncio.create_task(self.monitor())
 
         # Handle shutdown gracefully
         try:
@@ -80,6 +82,7 @@ class LoadBalancer:
             await asyncio.gather(*(p.wait() for p in self.backend_processes))
             print("Backend processes terminated.")
 
+
 async def backend_echo_server(port):
     async def handle_echo(reader, writer):
         try:
@@ -94,10 +97,11 @@ async def backend_echo_server(port):
         finally:
             writer.close()
 
-    server = await asyncio.start_server(handle_echo, '127.0.0.1', port)
+    server = await asyncio.start_server(handle_echo, "127.0.0.1", port)
     print(f"Backend echo server running on port {port}")
     await server.serve_forever()
 
+
 async def main():
     backend_ports = [8001, 8003, 8005, 8006]
     # Launch backend echo servers
@@ -105,19 +109,20 @@ async def main():
     lb = LoadBalancer(backend_ports)
     await lb.start()
 
+
 if __name__ == "__main__":
     if len(sys.argv) > 1:
-        if sys.argv[1] == 'backend':
+        if sys.argv[1] == "backend":
             port = int(sys.argv[2])
             from snek.app import Application
+
             snek = Application(port=port)
-            web.run_app(snek, port=port, host='127.0.0.1')
-        elif sys.argv[1] == 'sync':
-            from snek.sync import app
-            web.run_app(snek, port=port, host='127.0.0.1')
+            web.run_app(snek, port=port, host="127.0.0.1")
+        elif sys.argv[1] == "sync":
+
+            web.run_app(snek, port=port, host="127.0.0.1")
     else:
         try:
             asyncio.run(main())
         except KeyboardInterrupt:
             print("Shutting down...")
-

src/snek/docs/app.py

@@ -1,3 +1,5 @@
+# retoor <retoor@molodetz.nl>
+
 import pathlib
 
 from aiohttp import web

src/snek/dump.py

@@ -1,3 +1,5 @@
+# retoor <retoor@molodetz.nl>
+
 import asyncio
 
 from snek.app import app

src/snek/form/__init__.py

@@ -0,0 +1,3 @@
+# retoor <retoor@molodetz.nl>
+
+

src/snek/form/login.py

@@ -1,3 +1,5 @@
+# retoor <retoor@molodetz.nl>
+
 from snek.system.form import Form, FormButtonElement, FormInputElement, HTMLElement
 
 

src/snek/form/register.py

@@ -1,3 +1,5 @@
+# retoor <retoor@molodetz.nl>
+
 from snek.system.form import Form, FormButtonElement, FormInputElement, HTMLElement
 
 

src/snek/form/search_user.py

@@ -1,3 +1,5 @@
+# retoor <retoor@molodetz.nl>
+
 from snek.system.form import Form, FormButtonElement, FormInputElement, HTMLElement
 
 

src/snek/form/settings/profile.py

@@ -1,3 +1,5 @@
+# retoor <retoor@molodetz.nl>
+
 from snek.system.form import Form, FormButtonElement, FormInputElement, HTMLElement
 
 

src/snek/forum.py

@@ -0,0 +1,95 @@
+# retoor <retoor@molodetz.nl>
+
+# forum_app.py
+import aiohttp.web
+from snek.view.forum import ForumIndexView, ForumView, ForumWebSocketView
+
+
+class ForumApplication(aiohttp.web.Application):
+    def __init__(self, parent, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        self.parent = parent
+        self.render_template = self.parent.render_template 
+        # Set up routes
+        self.setup_routes()
+        
+        # Set up notification listeners
+        self.setup_notifications()
+    
+    @property
+    def db(self):
+        return self.parent.db
+
+    @property
+    def services(self):
+        return self.parent.services
+    
+    def setup_routes(self):
+        """Set up all forum routes"""
+        # API routes
+        self.router.add_view("/index.html", ForumIndexView)
+        self.router.add_route("GET", "/api/forums", ForumView.get_forums)
+        self.router.add_route("GET", "/api/forums/{slug}", ForumView.get_forum)
+        self.router.add_route("POST", "/api/forums/{slug}/threads", ForumView.create_thread)
+        self.router.add_route("GET", "/api/threads/{thread_slug}", ForumView.get_thread)
+        self.router.add_route("POST", "/api/threads/{thread_uid}/posts", ForumView.create_post)
+        self.router.add_route("PUT", "/api/posts/{post_uid}", ForumView.edit_post)
+        self.router.add_route("DELETE", "/api/posts/{post_uid}", ForumView.delete_post)
+        self.router.add_route("POST", "/api/posts/{post_uid}/like", ForumView.toggle_like)
+        self.router.add_route("POST", "/api/threads/{thread_uid}/pin", ForumView.toggle_pin)
+        self.router.add_route("POST", "/api/threads/{thread_uid}/lock", ForumView.toggle_lock)
+        
+        # WebSocket route
+        self.router.add_view("/ws", ForumWebSocketView)
+        
+        # Static HTML route
+        self.router.add_route("GET", "/{path:.*}", self.serve_forum_html)
+    
+    def setup_notifications(self):
+        """Set up notification listeners for WebSocket broadcasting"""
+        # Forum notifications
+        self.services.forum.add_notification_listener("forum_created", self.on_forum_event)
+        
+        # Thread notifications
+        self.services.thread.add_notification_listener("thread_created", self.on_thread_event)
+        
+        # Post notifications
+        self.services.post.add_notification_listener("post_created", self.on_post_event)
+        self.services.post.add_notification_listener("post_edited", self.on_post_event)
+        self.services.post.add_notification_listener("post_deleted", self.on_post_event)
+        
+        # Like notifications
+        self.services.post_like.add_notification_listener("post_liked", self.on_like_event)
+        self.services.post_like.add_notification_listener("post_unliked", self.on_like_event)
+    
+    async def on_forum_event(self, event_type, data):
+        """Handle forum events"""
+        await ForumWebSocketView.broadcast_update(self, event_type, data)
+    
+    async def on_thread_event(self, event_type, data):
+        """Handle thread events"""
+        await ForumWebSocketView.broadcast_update(self, event_type, data)
+    
+    async def on_post_event(self, event_type, data):
+        """Handle post events"""
+        await ForumWebSocketView.broadcast_update(self, event_type, data)
+    
+    async def on_like_event(self, event_type, data):
+        """Handle like events"""
+        await ForumWebSocketView.broadcast_update(self, event_type, data)
+    
+    async def serve_forum_html(self, request):
+        """Serve the forum HTML with the web component"""
+        return await self.parent.render_template("forum.html", request)
+
+
+# Integration with main app
+def setup_forum(app):
+    """Set up forum sub-application"""
+    forum_app = ForumApplication(app)
+    app.add_subapp("/forum", forum_app)
+    app.forum_app = forum_app
+    # Register models and services if needed
+    # This would typically be done in your main app initialization
+
+    return forum_app

src/snek/gunicorn.py

@@ -1,3 +1,5 @@
+# retoor <retoor@molodetz.nl>
+
 from snek.app import app
 
 application = app

src/snek/mapper/__init__.py

@@ -1,21 +1,25 @@
+# retoor <retoor@molodetz.nl>
+
 import functools
 
 from snek.mapper.channel import ChannelMapper
+from snek.mapper.channel_attachment import ChannelAttachmentMapper
 from snek.mapper.channel_member import ChannelMemberMapper
 from snek.mapper.channel_message import ChannelMessageMapper
+from snek.mapper.container import ContainerMapper
 from snek.mapper.drive import DriveMapper
 from snek.mapper.drive_item import DriveItemMapper
 from snek.mapper.notification import NotificationMapper
+from snek.mapper.push import PushMapper
+from snek.mapper.repository import RepositoryMapper
 from snek.mapper.user import UserMapper
 from snek.mapper.user_property import UserPropertyMapper
-from snek.mapper.repository import RepositoryMapper
-from snek.mapper.channel_attachment import ChannelAttachmentMapper
-from snek.mapper.container import ContainerMapper
+from snek.mapper.profile_page import ProfilePageMapper
+from snek.mapper.forum import ForumMapper, ThreadMapper, PostMapper, PostLikeMapper
 from snek.system.object import Object
 
 
 @functools.cache
-
 def get_mappers(app=None):
     return Object(
         **{
@@ -30,9 +34,16 @@ def get_mappers(app=None):
             "repository": RepositoryMapper(app=app),
             "channel_attachment": ChannelAttachmentMapper(app=app),
             "container": ContainerMapper(app=app),
+            "push": PushMapper(app=app),
+            "forum": ForumMapper(app=app),
+            "thread": ThreadMapper(app=app),
+            "post": PostMapper(app=app),
+            "post_like": PostLikeMapper(app=app),
+            "profile_page": ProfilePageMapper(app=app),
         }
     )
 
 
 def get_mapper(name, app=None):
     return get_mappers(app=app)[name]
+

src/snek/mapper/channel.py

@@ -1,3 +1,5 @@
+# retoor <retoor@molodetz.nl>
+
 from snek.model.channel import ChannelModel
 from snek.system.mapper import BaseMapper
 

src/snek/mapper/channel_attachment.py

@@ -1,3 +1,5 @@
+# retoor <retoor@molodetz.nl>
+
 from snek.model.channel_attachment import ChannelAttachmentModel
 from snek.system.mapper import BaseMapper
 

src/snek/mapper/channel_member.py

@@ -1,3 +1,5 @@
+# retoor <retoor@molodetz.nl>
+
 from snek.model.channel_member import ChannelMemberModel
 from snek.system.mapper import BaseMapper
 

src/snek/mapper/channel_message.py

@@ -1,3 +1,5 @@
+# retoor <retoor@molodetz.nl>
+
 from snek.model.channel_message import ChannelMessageModel
 from snek.system.mapper import BaseMapper
 

src/snek/mapper/container.py

@@ -1,6 +1,9 @@
+# retoor <retoor@molodetz.nl>
+
 from snek.model.container import Container
 from snek.system.mapper import BaseMapper
 
+
 class ContainerMapper(BaseMapper):
     model_class = Container
-    table_name = "container"
+    table_name = "container"

src/snek/mapper/drive.py

@@ -1,3 +1,5 @@
+# retoor <retoor@molodetz.nl>
+
 from snek.model.drive import DriveModel
 from snek.system.mapper import BaseMapper
 

src/snek/mapper/drive_item.py

@@ -1,3 +1,5 @@
+# retoor <retoor@molodetz.nl>
+
 from snek.model.drive_item import DriveItemModel
 from snek.system.mapper import BaseMapper
 

src/snek/mapper/forum.py

@@ -0,0 +1,25 @@
+# retoor <retoor@molodetz.nl>
+
+# mapper/forum.py
+from snek.model.forum import ForumModel, ThreadModel, PostModel, PostLikeModel
+from snek.system.mapper import BaseMapper
+
+
+class ForumMapper(BaseMapper):
+    table_name = "forum"
+    model_class = ForumModel
+
+
+class ThreadMapper(BaseMapper):
+    table_name = "thread"
+    model_class = ThreadModel
+
+
+class PostMapper(BaseMapper):
+    table_name = "post"
+    model_class = PostModel
+
+
+class PostLikeMapper(BaseMapper):
+    table_name = "post_like"
+    model_class = PostLikeModel

src/snek/mapper/notification.py

@@ -1,3 +1,5 @@
+# retoor <retoor@molodetz.nl>
+
 from snek.model.notification import NotificationModel
 from snek.system.mapper import BaseMapper
 

src/snek/mapper/profile_page.py

@@ -0,0 +1,11 @@
+# retoor <retoor@molodetz.nl>
+
+import logging
+from snek.model.profile_page import ProfilePageModel
+from snek.system.mapper import BaseMapper
+
+logger = logging.getLogger(__name__)
+
+class ProfilePageMapper(BaseMapper):
+    table_name = "profile_page"
+    model_class = ProfilePageModel

src/snek/mapper/push.py

@@ -0,0 +1,9 @@
+# retoor <retoor@molodetz.nl>
+
+from snek.model.push_registration import PushRegistrationModel
+from snek.system.mapper import BaseMapper
+
+
+class PushMapper(BaseMapper):
+    model_class = PushRegistrationModel
+    table_name = "push_registration"

src/snek/mapper/repository.py

@@ -1,3 +1,5 @@
+# retoor <retoor@molodetz.nl>
+
 from snek.model.repository import RepositoryModel
 from snek.system.mapper import BaseMapper
 

src/snek/mapper/user.py

@@ -1,6 +1,12 @@
+# retoor <retoor@molodetz.nl>
+
+import logging
+
 from snek.model.user import UserModel
 from snek.system.mapper import BaseMapper
 
+logger = logging.getLogger(__name__)
+
 
 class UserMapper(BaseMapper):
     table_name = "user"
@@ -16,5 +22,5 @@ class UserMapper(BaseMapper):
                 )
             ]
         except Exception as ex:
-            print(ex)
+            logger.warning(f"Failed to get admin uids: {ex}")
             return []

src/snek/mapper/user_property.py

@@ -1,3 +1,5 @@
+# retoor <retoor@molodetz.nl>
+
 from snek.model.user_property import UserPropertyModel
 from snek.system.mapper import BaseMapper
 

src/snek/model/__init__.py

@@ -1,18 +1,22 @@
+# retoor <retoor@molodetz.nl>
+
 import functools
 
 from snek.model.channel import ChannelModel
+from snek.model.channel_attachment import ChannelAttachmentModel
 from snek.model.channel_member import ChannelMemberModel
 
 # from snek.model.channel_message import ChannelMessageModel
 from snek.model.channel_message import ChannelMessageModel
+from snek.model.container import Container
 from snek.model.drive import DriveModel
 from snek.model.drive_item import DriveItemModel
 from snek.model.notification import NotificationModel
+from snek.model.push_registration import PushRegistrationModel
+from snek.model.repository import RepositoryModel
 from snek.model.user import UserModel
 from snek.model.user_property import UserPropertyModel
-from snek.model.repository import RepositoryModel
-from snek.model.channel_attachment import ChannelAttachmentModel
-from snek.model.container import Container
+from snek.model.forum import ForumModel, ThreadModel, PostModel, PostLikeModel
 from snek.system.object import Object
 
 
@@ -31,6 +35,11 @@ def get_models():
             "repository": RepositoryModel,
             "channel_attachment": ChannelAttachmentModel,
             "container": Container,
+            "push_registration": PushRegistrationModel,
+            "forum": ForumModel,
+            "thread": ThreadModel,
+            "post": PostModel,
+            "post_like": PostLikeModel,
         }
     )
 

src/snek/model/channel.py

@@ -1,3 +1,5 @@
+# retoor <retoor@molodetz.nl>
+
 from snek.model.channel_message import ChannelMessageModel
 from snek.system.model import BaseModel, ModelField
 
@@ -11,16 +13,23 @@ class ChannelModel(BaseModel):
     is_listed = ModelField(name="is_listed", required=True, kind=bool, value=True)
     index = ModelField(name="index", required=True, kind=int, value=1000)
     last_message_on = ModelField(name="last_message_on", required=False, kind=str)
+    history_start = ModelField(name="history_start", required=False, kind=str)
+    
+    @property
+    def is_dm(self):
+        return 'dm' in self['tag'].lower()
 
     async def get_last_message(self) -> ChannelMessageModel:
+        history_start_filter = ""
+        if self["history_start"]:
+            history_start_filter = f" AND created_at > '{self['history_start']}' "
         try:
             async for model in self.app.services.channel_message.query(
-                "SELECT uid FROM channel_message WHERE channel_uid=:channel_uid ORDER BY created_at DESC LIMIT 1",
+                "SELECT uid FROM channel_message WHERE channel_uid=:channel_uid" + history_start_filter + " ORDER BY id DESC LIMIT 1",
                 {"channel_uid": self["uid"]},
             ):
-
                 return await self.app.services.channel_message.get(uid=model["uid"])
-        except:
+        except Exception:
             pass
         return None
 

src/snek/model/channel_attachment.py

@@ -1,4 +1,5 @@
-from snek.system.model import BaseModel
+# retoor <retoor@molodetz.nl>
+
 from snek.system.model import BaseModel, ModelField
 
 
@@ -11,6 +12,6 @@ class ChannelAttachmentModel(BaseModel):
     user_uid = ModelField(name="user_uid", required=True, kind=str)
     mime_type = ModelField(name="type", required=True, kind=str)
     relative_url = ModelField(name="relative_url", required=True, kind=str)
-    resource_type = ModelField(name="resource_type", required=True, kind=str,value="file")
-
-
+    resource_type = ModelField(
+        name="resource_type", required=True, kind=str, value="file"
+    )

src/snek/model/channel_member.py

@@ -1,3 +1,5 @@
+# retoor <retoor@molodetz.nl>
+
 from snek.system.model import BaseModel, ModelField
 
 

src/snek/model/channel_message.py

@@ -1,6 +1,10 @@
+# retoor <retoor@molodetz.nl>
+
+from datetime import datetime, timezone
+
 from snek.model.user import UserModel
 from snek.system.model import BaseModel, ModelField
-from datetime import datetime,timezone 
+
 
 class ChannelMessageModel(BaseModel):
     channel_uid = ModelField(name="channel_uid", required=True, kind=str)
@@ -10,7 +14,11 @@ class ChannelMessageModel(BaseModel):
     is_final = ModelField(name="is_final", required=True, kind=bool, value=True)
 
     def get_seconds_since_last_update(self):
-        return int((datetime.now(timezone.utc) - datetime.fromisoformat(self["updated_at"])).total_seconds())
+        return int(
+            (
+                datetime.now(timezone.utc) - datetime.fromisoformat(self["updated_at"])
+            ).total_seconds()
+        )
 
     async def get_user(self) -> UserModel:
         return await self.app.services.user.get(uid=self["user_uid"])

src/snek/model/container.py

@@ -1,5 +1,8 @@
+# retoor <retoor@molodetz.nl>
+
 from snek.system.model import BaseModel, ModelField
 
+
 class Container(BaseModel):
     id = ModelField(name="id", required=True, kind=str)
     name = ModelField(name="name", required=True, kind=str)

src/snek/model/drive.py

@@ -1,3 +1,5 @@
+# retoor <retoor@molodetz.nl>
+
 from snek.system.model import BaseModel, ModelField
 
 

src/snek/model/drive_item.py

@@ -1,3 +1,5 @@
+# retoor <retoor@molodetz.nl>
+
 import mimetypes
 
 from snek.system.model import BaseModel, ModelField
@@ -9,7 +11,9 @@ class DriveItemModel(BaseModel):
     path = ModelField(name="path", required=True, kind=str)
     file_type = ModelField(name="file_type", required=True, kind=str)
     file_size = ModelField(name="file_size", required=True, kind=int)
-    is_available = ModelField(name="is_available", required=True, kind=bool, initial_value=True)
+    is_available = ModelField(
+        name="is_available", required=True, kind=bool, initial_value=True
+    )
 
     @property
     def extension(self):

src/snek/model/forum.py

@@ -0,0 +1,101 @@
+# retoor <retoor@molodetz.nl>
+
+# models/forum.py
+from snek.system.model import BaseModel, ModelField
+
+
+class ForumModel(BaseModel):
+    """Forum categories"""
+    name = ModelField(name="name", required=True, kind=str, min_length=3, max_length=100)
+    description = ModelField(name="description", required=False, kind=str, max_length=500)
+    slug = ModelField(name="slug", required=True, kind=str, regex=r"^[a-z0-9-]+$", unique=True)
+    icon = ModelField(name="icon", required=False, kind=str)
+    position = ModelField(name="position", required=True, kind=int, value=0)
+    is_active = ModelField(name="is_active", required=True, kind=bool, value=True)
+    created_by_uid = ModelField(name="created_by_uid", required=True, kind=str)
+    thread_count = ModelField(name="thread_count", required=True, kind=int, value=0)
+    post_count = ModelField(name="post_count", required=True, kind=int, value=0)
+    last_post_at = ModelField(name="last_post_at", required=False, kind=str)
+    last_thread_uid = ModelField(name="last_thread_uid", required=False, kind=str)
+
+    async def get_threads(self, limit=50, offset=0):
+        async for thread in self.app.services.thread.find(
+            forum_uid=self["uid"], 
+            deleted_at=None,
+            _limit=limit,
+            _offset=offset,
+            order_by="-last_post_at"
+            #order_by="is_pinned DESC, last_post_at DESC"
+        ):
+            yield thread
+
+    async def increment_thread_count(self):
+        self["thread_count"] += 1
+        await self.save()
+
+    async def increment_post_count(self):
+        self["post_count"] += 1
+        await self.save()
+
+
+# models/thread.py
+class ThreadModel(BaseModel):
+    """Forum threads"""
+    forum_uid = ModelField(name="forum_uid", required=True, kind=str)
+    title = ModelField(name="title", required=True, kind=str, min_length=5, max_length=200)
+    slug = ModelField(name="slug", required=True, kind=str, regex=r"^[a-z0-9-]+$")
+    created_by_uid = ModelField(name="created_by_uid", required=True, kind=str)
+    is_pinned = ModelField(name="is_pinned", required=True, kind=bool, value=False)
+    is_locked = ModelField(name="is_locked", required=True, kind=bool, value=False)
+    view_count = ModelField(name="view_count", required=True, kind=int, value=0)
+    post_count = ModelField(name="post_count", required=True, kind=int, value=0)
+    last_post_at = ModelField(name="last_post_at", required=False, kind=str)
+    last_post_by_uid = ModelField(name="last_post_by_uid", required=False, kind=str)
+
+    async def get_posts(self, limit=50, offset=0):
+        async for post in self.app.services.post.find(
+            thread_uid=self["uid"],
+            deleted_at=None,
+            _limit=limit,
+            _offset=offset,
+            order_by="created_at"
+        ):
+            yield post
+
+    async def increment_view_count(self):
+        self["view_count"] += 1
+        await self.save()
+
+    async def increment_post_count(self):
+        self["post_count"] += 1
+        self["last_post_at"] = self.app.services.get_timestamp()
+        await self.save()
+
+
+# models/post.py  
+class PostModel(BaseModel):
+    """Forum posts"""
+    thread_uid = ModelField(name="thread_uid", required=True, kind=str)
+    forum_uid = ModelField(name="forum_uid", required=True, kind=str)
+    content = ModelField(name="content", required=True, kind=str, min_length=1, max_length=10000)
+    created_by_uid = ModelField(name="created_by_uid", required=True, kind=str)
+    edited_at = ModelField(name="edited_at", required=False, kind=str)
+    edited_by_uid = ModelField(name="edited_by_uid", required=False, kind=str)
+    is_first_post = ModelField(name="is_first_post", required=True, kind=bool, value=False)
+    like_count = ModelField(name="like_count", required=True, kind=int, value=0)
+    
+    async def get_author(self):
+        return await self.app.services.user.get(uid=self["created_by_uid"])
+
+    async def is_liked_by(self, user_uid):
+        return await self.app.services.post_like.exists(
+            post_uid=self["uid"],
+            user_uid=user_uid
+        )
+
+
+# models/post_like.py
+class PostLikeModel(BaseModel):
+    """Post likes"""
+    post_uid = ModelField(name="post_uid", required=True, kind=str)
+    user_uid = ModelField(name="user_uid", required=True, kind=str)

src/snek/model/notification.py

@@ -1,3 +1,5 @@
+# retoor <retoor@molodetz.nl>
+
 from snek.system.model import BaseModel, ModelField
 
 

src/snek/model/profile_page.py

@@ -0,0 +1,14 @@
+# retoor <retoor@molodetz.nl>
+
+import logging
+from snek.system.model import BaseModel, ModelField
+
+logger = logging.getLogger(__name__)
+
+class ProfilePageModel(BaseModel):
+    user_uid = ModelField(name="user_uid", required=True, kind=str)
+    title = ModelField(name="title", required=True, kind=str)
+    slug = ModelField(name="slug", required=True, kind=str)
+    content = ModelField(name="content", required=False, kind=str, value="")
+    order_index = ModelField(name="order_index", required=True, kind=int, value=0)
+    is_published = ModelField(name="is_published", required=True, kind=bool, value=True)

src/snek/model/push_registration.py

@@ -0,0 +1,10 @@
+# retoor <retoor@molodetz.nl>
+
+from snek.system.model import BaseModel, ModelField
+
+
+class PushRegistrationModel(BaseModel):
+    user_uid = ModelField(name="user_uid", required=True)
+    endpoint = ModelField(name="endpoint", required=True)
+    key_auth = ModelField(name="key_auth", required=True)
+    key_p256dh = ModelField(name="key_p256dh", required=True)

src/snek/model/repository.py

@@ -1,14 +1,14 @@
-from snek.model.user import UserModel
+# retoor <retoor@molodetz.nl>
+
 from snek.system.model import BaseModel, ModelField
 
 
 class RepositoryModel(BaseModel):
 
     user_uid = ModelField(name="user_uid", required=True, kind=str)
-    
+
     name = ModelField(name="name", required=True, kind=str)
 
+    description = ModelField(name="description", required=False, kind=str)
+
     is_private = ModelField(name="is_private", required=False, kind=bool)
-
-
-    

src/snek/model/user.py

@@ -1,3 +1,5 @@
+# retoor <retoor@molodetz.nl>
+
 from snek.system.model import BaseModel, ModelField
 
 
@@ -31,6 +33,14 @@ class UserModel(BaseModel):
 
     is_admin = ModelField(name="is_admin", required=False, kind=bool)
 
+    country_short = ModelField(name="country_short", required=False, kind=str)
+    country_long = ModelField(name="country_long", required=False, kind=str)
+    city = ModelField(name="city", required=False, kind=str)
+    latitude = ModelField(name="latitude", required=False, kind=float)
+    longitude = ModelField(name="longitude", required=False, kind=float)
+    region = ModelField(name="region", required=False, kind=str)
+    ip = ModelField(name="ip", required=False, kind=str)
+
     async def get_property(self, name):
         prop = await self.app.services.user_property.find_one(
             user_uid=self["uid"], name=name

src/snek/model/user_property.py

@@ -1,7 +1,9 @@
+# retoor <retoor@molodetz.nl>
+
 from snek.system.model import BaseModel, ModelField
 
 
 class UserPropertyModel(BaseModel):
     user_uid = ModelField(name="user_uid", required=True, kind=str)
     name = ModelField(name="name", required=True, kind=str)
-    value = ModelField(name="path", required=True, kind=str)
+    value = ModelField(name="value", required=True, kind=str)

src/snek/research/serpentarium.py

@@ -1,39 +1,15 @@
+# retoor <retoor@molodetz.nl>
 
-import json
 import asyncio
-import aiohttp
-from aiohttp import web
-import dataset
-import dataset.util
-import traceback
-import socket
-import base64
-import uuid 
+import json
+
 
 class DatasetMethod:
-    def __init__(self, dt, name):
-        self.dt = dt
-        self.name = name 
-
-    def __call__(self, *args, **kwargs):
-        return self.dt.ds.call(
-            self.dt.name,
-            self.name,
-            *args,
-            **kwargs
-        )
+    pass
 
 
 class DatasetTable:
-
-    def __init__(self, ds, name):
-        self.ds = ds 
-        self.name = name 
-
-    def __getattr__(self, name):
-        return DatasetMethod(self, name)
-
-
+    pass
 
 
 class WebSocketClient2:
@@ -43,200 +19,40 @@ class WebSocketClient2:
         self.websocket = None
         self.receive_queue = asyncio.Queue()
 
-        # Schedule connection setup
-        if self.loop.is_running():
-            # Schedule connect in the existing loop
-            self._connect_future = asyncio.run_coroutine_threadsafe(self._connect(), self.loop)
-        else:
-            # If loop isn't running, connect synchronously
-            self.loop.run_until_complete(self._connect())
-
-    async def _connect(self):
-        self.websocket = await websockets.connect(self.uri)
-        # Start listening for messages
-        asyncio.create_task(self._receive_loop())
-
-    async def _receive_loop(self):
-        try:
-            async for message in self.websocket:
-                await self.receive_queue.put(message)
-        except Exception:
-            pass  # Handle exceptions as needed
-
     def send(self, message: str):
-        if self.loop.is_running():
-            # Schedule send in the existing loop
-            asyncio.run_coroutine_threadsafe(self.websocket.send(message), self.loop)
-        else:
-            # If loop isn't running, run directly
-            self.loop.run_until_complete(self.websocket.send(message))
-
-    def receive(self):
-        # Wait for a message synchronously
-        future = asyncio.run_coroutine_threadsafe(self.receive_queue.get(), self.loop)
-        return future.result()
+        pass
 
     def close(self):
-        if self.websocket:
-            if self.loop.is_running():
-                asyncio.run_coroutine_threadsafe(self.websocket.close(), self.loop)
-            else:
-                self.loop.run_until_complete(self.websocket.close())
+        pass
 
 
-import websockets 
-
-class DatasetWrapper(object):
-
+class DatasetWrapper:
     def __init__(self):
-        self.ws = WebSocketClient() 
-
-    def begin(self):
-        self.call(None, 'begin')
+        pass
 
     def commit(self):
-        self.call(None, 'commit')
-
-    def __getitem__(self, name):
-        return DatasetTable(self, name)
+        pass
 
     def query(self, *args, **kwargs):
-        return self.call(None, 'query', *args, **kwargs)
-
-    def call(self, table, method, *args, **kwargs):
-        payload = {"table": table, "method": method, "args": args, "kwargs": kwargs,"call_uid":None}
-        #if method in ['find','find_one']:
-        payload["call_uid"] = str(uuid.uuid4())    
-        self.ws.write(json.dumps(payload))
-        if payload["call_uid"]:
-            response = self.ws.read()
-            return json.loads(response)['result']
-        return True
-
+        pass
 
 
 class DatasetWebSocketView:
     def __init__(self):
         self.ws = None
-        self.db = dataset.connect('sqlite:///snek.db')
-        self.setattr(self, "db", self.get)
-        self.setattr(self, "db", self.set)
-        )
-        super()
-    
+
     def format_result(self, result):
-        
-        try:
-            return dict(result)
-        except:
-            pass
-        try:
-            return [dict(row) for row in result]
-        except:
-            pass
-        return result
+        pass
 
     async def send_str(self, msg):
-        return await self.ws.send_str(msg)
+        pass
 
     def get(self, key):
-        returnl loads(dict(self.db['_kv'].get(key=key)['value']))
+        pass
 
     def set(self, key, value):
-        return self.db['_kv'].upsert({'key': key, 'value': json.dumps(value)}, ['key'])
+        pass
 
 
-
-    async def handle(self, request):
-        ws = web.WebSocketResponse()
-        await ws.prepare(request)
-        self.ws = ws
-
-        async for msg in ws:
-            if msg.type == aiohttp.WSMsgType.TEXT:
-                try:
-                    data = json.loads(msg.data)
-                    call_uid = data.get("call_uid")
-                    method = data.get("method")
-                    table_name = data.get("table")
-                    args = data.get("args", {})
-                    kwargs = data.get("kwargs", {})
-                    
-
-                    function = getattr(self.db, method, None)
-                    if table_name:
-                        function = getattr(self.db[table_name], method, None)
-                    
-                    print(method, table_name, args, kwargs,flush=True)
-                    
-                    if function:
-                        response = {}
-                        try:
-                            result = function(*args, **kwargs)
-                            print(result) 
-                            response['result'] = self.format_result(result)
-                            response["call_uid"] = call_uid
-                            response["success"] = True
-                        except Exception as e:
-                            response["call_uid"] = call_uid
-                            response["success"] = False
-                            response["error"] = str(e)
-                            response["traceback"] = traceback.format_exc()
-                        
-                        if call_uid:
-                            await self.send_str(json.dumps(response,default=str))
-                    else:
-                        await self.send_str(json.dumps({"status": "error", "error":"Method not found.","call_uid": call_uid}))
-                except Exception as e:
-                    await self.send_str(json.dumps({"success": False,"call_uid": call_uid, "error": str(e), "error": str(e), "traceback": traceback.format_exc()},default=str))
-            elif msg.type == aiohttp.WSMsgType.ERROR:
-                print('ws connection closed with exception %s' % ws.exception())
-
-        return ws
-
-    
-
-    
-
-app = web.Application()
-view = DatasetWebSocketView()
-app.router.add_get('/db', view.handle)
-
 async def run_server():
-
-
-    runner = web.AppRunner(app)
-    await runner.setup()
-    site = web.TCPSite(runner, 'localhost', 3131)
-    await site.start()
-
-    print("Server started at http://localhost:8080")
-    await asyncio.Event().wait()
-
-async def client():
-    print("x")
-    d = DatasetWrapper()
-    print("y")
-    
-    for x in range(100):
-        for x in range(100):
-            if d['test'].insert({"name": "test", "number":x}):
-                print(".",end="",flush=True)
-        print("")    
-        print(d['test'].find_one(name="test", order_by="-number")) 
-
-    print("DONE")
-
-
-
-import time 
-async def main():
-    await run_server()
-
-import sys 
-
-if __name__ == '__main__':
-    if sys.argv[1] == 'server':
-        asyncio.run(main())
-    if sys.argv[1] == 'client':
-        asyncio.run(client())
+    pass

src/snek/research/serptest.py

@@ -1,51 +1,56 @@
-import snek.serpentarium
-
-import time 
+# retoor <retoor@molodetz.nl>
 
+import time
 from concurrent.futures import ProcessPoolExecutor
 
+import snek.serpentarium
+
 durations = []
 
+
 def task1():
-    global durations 
+    global durations
     client = snek.serpentarium.DatasetWrapper()
 
-    start=time.time()
+    start = time.time()
     for x in range(1500):
-    
-        client['a'].delete()
-        client['a'].insert({"foo": x})
-        client['a'].find(foo=x) 
-        client['a'].find_one(foo=x)
-        client['a'].count()
-        #print(client['a'].find(foo=x) )
-        #print(client['a'].find_one(foo=x) )
-        #print(client['a'].count())
+
+        client["a"].delete()
+        client["a"].insert({"foo": x})
+        client["a"].find(foo=x)
+        client["a"].find_one(foo=x)
+        client["a"].count()
+        # print(client['a'].find(foo=x) )
+        # print(client['a'].find_one(foo=x) )
+        # print(client['a'].count())
     client.close()
     duration1 = f"{time.time()-start}"
     durations.append(duration1)
     print(durations)
 
+
 with ProcessPoolExecutor(max_workers=4) as executor:
-    tasks = [executor.submit(task1),
+    tasks = [
+        executor.submit(task1),
+        executor.submit(task1),
         executor.submit(task1),
         executor.submit(task1),
-        executor.submit(task1)
     ]
     for task in tasks:
         task.result()
 
 
-import dataset 
+import dataset
+
 client = dataset.connect("sqlite:///snek.db")
-start=time.time()
+start = time.time()
 for x in range(1500):
 
-    client['a'].delete()
-    client['a'].insert({"foo": x})
-    print([dict(row) for row in client['a'].find(foo=x)])
-    print(dict(client['a'].find_one(foo=x) ))
-    print(client['a'].count())
+    client["a"].delete()
+    client["a"].insert({"foo": x})
+    print([dict(row) for row in client["a"].find(foo=x)])
+    print(dict(client["a"].find_one(foo=x)))
+    print(client["a"].count())
 duration2 = f"{time.time()-start}"
 
-print(duration1,duration2)
+print(duration1, duration2)

src/snek/schema.sql

@@ -1,32 +1,33 @@
-CREATE TABLE IF NOT EXISTS http_access (
-	id INTEGER NOT NULL, 
-	created TEXT, 
-	path TEXT, 
-	duration FLOAT, 
-	PRIMARY KEY (id)
-);
-CREATE TABLE IF NOT EXISTS user (
+CREATE TABLE user (
 	id INTEGER NOT NULL, 
+	city TEXT, 
 	color TEXT, 
+	country_long TEXT, 
+	country_short TEXT, 
 	created_at TEXT, 
 	deleted_at TEXT, 
 	email TEXT, 
+	ip TEXT, 
 	is_admin TEXT, 
 	last_ping TEXT, 
+	latitude TEXT, 
+	longitude TEXT, 
 	nick TEXT, 
 	password TEXT, 
+	region TEXT, 
 	uid TEXT, 
 	updated_at TEXT, 
 	username TEXT, 
 	PRIMARY KEY (id)
 );
-CREATE INDEX IF NOT EXISTS ix_user_e2577dd78b54fe28 ON user (uid);
-CREATE TABLE IF NOT EXISTS channel (
+CREATE INDEX ix_user_e2577dd78b54fe28 ON user (uid);
+CREATE TABLE channel (
 	id INTEGER NOT NULL, 
 	created_at TEXT, 
 	created_by_uid TEXT, 
 	deleted_at TEXT, 
 	description TEXT, 
+	history_start TEXT, 
 	"index" BIGINT, 
 	is_listed BOOLEAN, 
 	is_private BOOLEAN, 
@@ -37,8 +38,8 @@ CREATE TABLE IF NOT EXISTS channel (
 	updated_at TEXT, 
 	PRIMARY KEY (id)
 );
-CREATE INDEX IF NOT EXISTS ix_channel_e2577dd78b54fe28 ON channel (uid);
-CREATE TABLE IF NOT EXISTS channel_member (
+CREATE INDEX ix_channel_e2577dd78b54fe28 ON channel (uid);
+CREATE TABLE channel_member (
 	id INTEGER NOT NULL, 
 	channel_uid TEXT, 
 	created_at TEXT, 
@@ -54,28 +55,31 @@ CREATE TABLE IF NOT EXISTS channel_member (
 	user_uid TEXT, 
 	PRIMARY KEY (id)
 );
-CREATE INDEX IF NOT EXISTS ix_channel_member_e2577dd78b54fe28 ON channel_member (uid);
-CREATE TABLE IF NOT EXISTS broadcast (
-	id INTEGER NOT NULL, 
-	channel_uid TEXT, 
-	message TEXT, 
-	created_at TEXT, 
-	PRIMARY KEY (id)
-);
-CREATE TABLE IF NOT EXISTS channel_message (
+CREATE INDEX ix_channel_member_e2577dd78b54fe28 ON channel_member (uid);
+CREATE TABLE channel_message (
 	id INTEGER NOT NULL, 
 	channel_uid TEXT, 
 	created_at TEXT, 
 	deleted_at TEXT, 
 	html TEXT, 
+	is_final BOOLEAN, 
 	message TEXT, 
 	uid TEXT, 
 	updated_at TEXT, 
 	user_uid TEXT, 
 	PRIMARY KEY (id)
 );
-CREATE INDEX IF NOT EXISTS ix_channel_message_e2577dd78b54fe28 ON channel_message (uid);
-CREATE TABLE IF NOT EXISTS notification (
+CREATE INDEX ix_channel_message_e2577dd78b54fe28 ON channel_message (uid);
+CREATE INDEX ix_channel_message_acb69f257bb37684 ON channel_message (is_final, user_uid, channel_uid);
+CREATE INDEX ix_channel_message_c6c4cddf281df93d ON channel_message (deleted_at);
+CREATE TABLE kv (
+	id INTEGER NOT NULL, 
+	"key" TEXT, 
+	value TEXT, 
+	PRIMARY KEY (id)
+);
+CREATE INDEX ix_kv_a62f2225bf70bfac ON kv ("key");
+CREATE TABLE notification (
 	id INTEGER NOT NULL, 
 	created_at TEXT, 
 	deleted_at TEXT, 
@@ -88,11 +92,38 @@ CREATE TABLE IF NOT EXISTS notification (
 	user_uid TEXT, 
 	PRIMARY KEY (id)
 );
-CREATE INDEX IF NOT EXISTS ix_notification_e2577dd78b54fe28 ON notification (uid);
-CREATE TABLE IF NOT EXISTS repository (
+CREATE INDEX ix_notification_e2577dd78b54fe28 ON notification (uid);
+CREATE UNIQUE INDEX ix_user_249ba36000029bbe ON user (username);
+CREATE INDEX ix_channel_member_9a1d4fb1836d9613 ON channel_member (channel_uid, user_uid);
+CREATE TABLE drive (
 	id INTEGER NOT NULL, 
 	created_at TEXT, 
 	deleted_at TEXT, 
+	name TEXT, 
+	uid TEXT, 
+	updated_at TEXT, 
+	user_uid TEXT, 
+	PRIMARY KEY (id)
+);
+CREATE INDEX ix_drive_e2577dd78b54fe28 ON drive (uid);
+CREATE TABLE push_registration (
+	id INTEGER NOT NULL, 
+	created_at TEXT, 
+	deleted_at TEXT, 
+	endpoint TEXT, 
+	key_auth TEXT, 
+	key_p256dh TEXT, 
+	uid TEXT, 
+	updated_at TEXT, 
+	user_uid TEXT, 
+	PRIMARY KEY (id)
+);
+CREATE INDEX ix_push_registration_e2577dd78b54fe28 ON push_registration (uid);
+CREATE TABLE repository (
+	id INTEGER NOT NULL, 
+	created_at TEXT, 
+	deleted_at TEXT, 
+	description TEXT, 
 	is_private BIGINT, 
 	name TEXT, 
 	uid TEXT, 
@@ -100,4 +131,19 @@ CREATE TABLE IF NOT EXISTS repository (
 	user_uid TEXT, 
 	PRIMARY KEY (id)
 );
-CREATE INDEX IF NOT EXISTS ix_repository_e2577dd78b54fe28 ON repository (uid);
+CREATE INDEX ix_repository_e2577dd78b54fe28 ON repository (uid);
+CREATE TABLE profile_page (
+	id INTEGER NOT NULL, 
+	content TEXT, 
+	created_at TEXT, 
+	deleted_at TEXT, 
+	is_published BOOLEAN, 
+	order_index BIGINT, 
+	slug TEXT, 
+	title TEXT, 
+	uid TEXT, 
+	updated_at TEXT, 
+	user_uid TEXT, 
+	PRIMARY KEY (id)
+);
+CREATE INDEX ix_profile_page_e2577dd78b54fe28 ON profile_page (uid);

src/snek/service/__init__.py

@@ -1,44 +1,69 @@
+# retoor <retoor@molodetz.nl>
+
 import functools
 
 from snek.service.channel import ChannelService
+from snek.service.channel_attachment import ChannelAttachmentService
 from snek.service.channel_member import ChannelMemberService
 from snek.service.channel_message import ChannelMessageService
 from snek.service.chat import ChatService
+from snek.service.container import ContainerService
+from snek.service.db import DBService
 from snek.service.drive import DriveService
 from snek.service.drive_item import DriveItemService
 from snek.service.notification import NotificationService
+from snek.service.push import PushService
+from snek.service.repository import RepositoryService
 from snek.service.socket import SocketService
 from snek.service.user import UserService
 from snek.service.user_property import UserPropertyService
 from snek.service.util import UtilService
-from snek.service.repository import RepositoryService
-from snek.service.channel_attachment import ChannelAttachmentService
-from snek.service.container import ContainerService
 from snek.system.object import Object
-from snek.service.db import DBService
+from snek.service.statistics import StatisticsService
+from snek.service.forum import ForumService, ThreadService, PostService, PostLikeService
+from snek.service.profile_page import ProfilePageService
+_service_registry = {}
+
+def register_service(name, service_cls):
+    _service_registry[name] = service_cls
+
+register = register_service
 
 @functools.cache
 def get_services(app):
-    return Object(
+    result =  Object(
         **{
-            "user": UserService(app=app),
-            "channel_member": ChannelMemberService(app=app),
-            "channel": ChannelService(app=app),
-            "channel_message": ChannelMessageService(app=app),
-            "chat": ChatService(app=app),
-            "socket": SocketService(app=app),
-            "notification": NotificationService(app=app),
-            "util": UtilService(app=app),
-            "drive": DriveService(app=app),
-            "drive_item": DriveItemService(app=app),
-            "user_property": UserPropertyService(app=app),
-            "repository": RepositoryService(app=app),
-            "db": DBService(app=app),
-            "channel_attachment": ChannelAttachmentService(app=app),
-            "container": ContainerService(app=app),
+            name: service_cls(app=app)
+            for name, service_cls in _service_registry.items()
         }
     )
-
+    result.register = register_service
+    return result 
 
 def get_service(name, app=None):
     return get_services(app=app)[name]
+
+
+register_service("user", UserService)
+register_service("channel_member", ChannelMemberService)
+register_service("channel", ChannelService)
+register_service("channel_message", ChannelMessageService)
+register_service("chat", ChatService)
+register_service("socket", SocketService)
+register_service("notification", NotificationService)
+register_service("util", UtilService)
+register_service("drive", DriveService)
+register_service("drive_item", DriveItemService)
+register_service("user_property", UserPropertyService)
+register_service("repository", RepositoryService)
+register_service("db", DBService)
+register_service("channel_attachment", ChannelAttachmentService)
+register_service("container", ContainerService)
+register_service("push", PushService)
+register_service("statistics", StatisticsService)
+register_service("forum", ForumService)
+register_service("thread", ThreadService)
+register_service("post", PostService)
+register_service("post_like", PostLikeService)
+register_service("profile_page", ProfilePageService)
+

src/snek/service/channel.py

@@ -1,19 +1,28 @@
+# retoor <retoor@molodetz.nl>
+
+import pathlib
 from datetime import datetime
 
 from snek.system.model import now
 from snek.system.service import BaseService
 
-import pathlib 
 
 class ChannelService(BaseService):
     mapper_name = "channel"
 
-    async def get_attachment_folder(self, channel_uid,ensure=False):
+    async def get_home_folder(self, channel_uid):
+        folder = pathlib.Path(f"./drive/{channel_uid}/container/home")
+        if not folder.exists():
+            try:
+                folder.mkdir(parents=True, exist_ok=True)
+            except OSError:
+                pass
+        return folder
+
+    async def get_attachment_folder(self, channel_uid, ensure=False):
         path = pathlib.Path(f"./drive/{channel_uid}/attachments")
         if ensure:
-            path.mkdir(
-                parents=True, exist_ok=True
-            )
+            path.mkdir(parents=True, exist_ok=True)
         return path
 
     async def get(self, uid=None, **kwargs):
@@ -56,6 +65,7 @@ class ChannelService(BaseService):
         model["is_private"] = is_private
         model["is_listed"] = is_listed
         if await self.save(model):
+            await self.services.container.create(model["uid"])
             return model
         raise Exception(f"Failed to create channel: {model.errors}.")
 
@@ -68,7 +78,10 @@ class ChannelService(BaseService):
         return channel
 
     async def get_recent_users(self, channel_uid):
-        async for user in self.query("SELECT user.uid, user.username,user.color,user.last_ping,user.nick FROM channel_member INNER JOIN user ON user.uid = channel_member.user_uid WHERE channel_uid=:channel_uid AND user.last_ping >= datetime('now', '-3 minutes') ORDER BY last_ping DESC LIMIT 30", {"channel_uid": channel_uid}):
+        async for user in self.query(
+            "SELECT user.uid, user.username,user.color,user.last_ping,user.nick FROM channel_member INNER JOIN user ON user.uid = channel_member.user_uid WHERE channel_uid=:channel_uid AND user.last_ping >= datetime('now', '-3 minutes') ORDER BY last_ping DESC LIMIT 30",
+            {"channel_uid": channel_uid},
+        ):
             yield user
 
     async def get_users(self, channel_uid):
@@ -102,6 +115,12 @@ class ChannelService(BaseService):
             channel = await self.get(uid=channel_member["channel_uid"])
             yield channel
 
+    async def clear(self, channel_uid):
+        model = await self.get(uid=channel_uid)
+        model['history_from'] = datetime.now()
+        await self.save(model)
+
+
     async def ensure_public_channel(self, created_by_uid):
         model = await self.get(is_listed=True, tag="public")
         is_moderator = False

src/snek/service/channel_attachment.py

@@ -1,25 +1,27 @@
-from snek.system.service import BaseService
-import urllib.parse 
-import pathlib 
+# retoor <retoor@molodetz.nl>
+
 import mimetypes
-import uuid
+
+from snek.system.service import BaseService
+
 
 class ChannelAttachmentService(BaseService):
-    mapper_name="channel_attachment"
+    mapper_name = "channel_attachment"
 
     async def create_file(self, channel_uid, user_uid, name):
         attachment = await self.new()
         attachment["channel_uid"] = channel_uid
-        attachment['user_uid'] = user_uid
+        attachment["user_uid"] = user_uid
         attachment["name"] = name
         attachment["mime_type"] = mimetypes.guess_type(name)[0]
-        attachment['resource_type'] = "file"
+        attachment["resource_type"] = "file"
         real_file_name = f"{attachment['uid']}-{name}"
-        attachment["relative_url"] = (f"{attachment['uid']}-{name}")
-        attachment_folder = await self.services.channel.get_attachment_folder(channel_uid)
+        attachment["relative_url"] = f"{attachment['uid']}-{name}"
+        attachment_folder = await self.services.channel.get_attachment_folder(
+            channel_uid
+        )
         attachment_path = attachment_folder.joinpath(real_file_name)
         attachment["path"] = str(attachment_path)
         if await self.save(attachment):
             return attachment
         raise Exception(f"Failed to create channel attachment: {attachment.errors}.")
-

src/snek/service/channel_member.py

@@ -1,3 +1,6 @@
+# retoor <retoor@molodetz.nl>
+
+from snek.system.model import now
 from snek.system.service import BaseService
 
 
@@ -8,11 +11,12 @@ class ChannelMemberService(BaseService):
     async def mark_as_read(self, channel_uid, user_uid):
         channel_member = await self.get(channel_uid=channel_uid, user_uid=user_uid)
         channel_member["new_count"] = 0
+        channel_member["last_read_at"] = now()
         return await self.save(channel_member)
 
     async def get_user_uids(self, channel_uid):
         async for model in self.mapper.query(
-            "SELECT user_uid FROM channel_member WHERE channel_uid=:channel_uid",
+            "SELECT user_uid FROM channel_member WHERE channel_uid=:channel_uid AND deleted_at IS NULL AND is_banned = 0",
             {"channel_uid": channel_uid},
         ):
             yield model["user_uid"]
@@ -56,7 +60,6 @@ class ChannelMemberService(BaseService):
             "SELECT channel_member.* FROM channel_member INNER JOIN channel ON (channel.uid = channel_member.channel_uid and channel.tag = 'dm') LEFT JOIN channel_member AS channel_member2 ON(channel_member2.channel_uid = NULL AND channel_member2.user_uid = NULL)   WHERE channel_member.user_uid=:from_user ",
             {"from_user": from_user, "to_user": to_user},
         ):
-
             return model
 
     async def get_other_dm_user(self, channel_uid, user_uid):

src/snek/service/channel_message.py

@@ -1,22 +1,78 @@
+# retoor <retoor@molodetz.nl>
+
+import asyncio
+import json
+import logging
+import pathlib
+from concurrent.futures import ProcessPoolExecutor
+
 from snek.system.service import BaseService
+from snek.system.template import sanitize_html
+
+logger = logging.getLogger(__name__)
+jinja2_env = None
+template_path = pathlib.Path(__file__).parent.parent.joinpath("templates")
+
+
+def render(context):
+    template = jinja2_env.get_template("message.html")
+    return sanitize_html(template.render(**context))
+
 
 
 class ChannelMessageService(BaseService):
     mapper_name = "channel_message"
 
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        self._configured_indexes = False
+        self._executor_pools = {}
+        global jinja2_env
+        jinja2_env = self.app.jinja2_env
+        self._max_workers = 1
+
+    def get_or_create_executor(self, uid):
+        if not uid in self._executor_pools:
+            self._executor_pools[uid] = ProcessPoolExecutor(max_workers=self._max_workers)
+        logger.debug(f"Executors available: {len(self._executor_pools)}")
+        return self._executor_pools[uid]
+    
+    def delete_executor(self, uid):
+        if uid in self._executor_pools:
+            self._executor_pools[uid].shutdown()
+            del self._executor_pools[uid]
+
+    async def maintenance(self):
+        while True:
+            changed = 0
+            async for message in self.find(is_final=False):
+                message["is_final"] = True
+                await self.save(message)
+                changed += 1
+            async for message in self.find(is_final=None):
+                message["is_final"] = False
+                await self.save(message)
+                changed += 1
+            if not changed:
+                break
+
     async def create(self, channel_uid, user_uid, message, is_final=True):
+        logger.info(f"create: channel_uid={channel_uid}, user_uid={user_uid}, message_len={len(message) if message else 0}, is_final={is_final}")
         model = await self.new()
 
         model["channel_uid"] = channel_uid
         model["user_uid"] = user_uid
         model["message"] = message
-        model['is_final'] = is_final
+        model["is_final"] = is_final
 
         context = {}
 
         record = model.record
         context.update(record)
         user = await self.app.services.user.get(uid=user_uid)
+        if not user:
+            logger.error(f"create: user not found user_uid={user_uid}")
+            raise Exception("User not found")
         context.update(
             {
                 "user_uid": user["uid"],
@@ -25,20 +81,46 @@ class ChannelMessageService(BaseService):
                 "color": user["color"],
             }
         )
+        loop = asyncio.get_event_loop()
         try:
-            template = self.app.jinja2_env.get_template("message.html")
-            model["html"] = template.render(**context)
+            context = json.loads(json.dumps(context, default=str))
+            logger.debug(f"create: rendering html for message uid={model['uid']}")
+            model["html"] = await loop.run_in_executor(self.get_or_create_executor(model["uid"]), render,context)
         except Exception as ex:
-            print(ex, flush=True)
+            logger.error(f"create: html rendering failed: {ex}")
 
+        logger.debug(f"create: saving message uid={model['uid']}")
         if await super().save(model):
+            if not self._configured_indexes:
+                if not self.mapper.db["channel_message"].has_index(
+                    ["is_final", "user_uid", "channel_uid"]
+                ):
+                    self.mapper.db["channel_message"].create_index(
+                        ["is_final", "user_uid", "channel_uid"], unique=False
+                    )
+                if not self.mapper.db["channel_message"].has_index(["uid"]):
+                    self.mapper.db["channel_message"].create_index(["uid"], unique=True)
+                if not self.mapper.db["channel_message"].has_index(["deleted_at"]):
+                    self.mapper.db["channel_message"].create_index(
+                        ["deleted_at"], unique=False
+                    )
+            self._configured_indexes = True
+            if model['is_final']:
+                self.delete_executor(model['uid'])
+            logger.info(f"create: message created successfully uid={model['uid']}, channel={channel_uid}")
             return model
+        logger.error(f"create: failed to save message channel={channel_uid}, errors={model.errors}")
         raise Exception(f"Failed to create channel message: {model.errors}.")
 
     async def to_extended_dict(self, message):
         user = await self.services.user.get(uid=message["user_uid"])
         if not user:
             return {}
+
+        #if not message["html"].startswith("<chat-message"):
+            #message = await self.get(uid=message["uid"])
+            #await self.save(message)
+        
         return {
             "uid": message["uid"],
             "color": user["color"],
@@ -52,9 +134,13 @@ class ChannelMessageService(BaseService):
         }
 
     async def save(self, model):
+        logger.debug(f"save: starting for uid={model['uid']}, is_final={model['is_final']}")
         context = {}
         context.update(model.record)
-        user = await self.app.services.user.get(model['user_uid'])
+        user = await self.app.services.user.get(model["user_uid"])
+        if not user:
+            logger.error(f"save: user not found user_uid={model['user_uid']}")
+            return False
         context.update(
             {
                 "user_uid": user["uid"],
@@ -63,17 +149,33 @@ class ChannelMessageService(BaseService):
                 "color": user["color"],
             }
         )
-        template = self.app.jinja2_env.get_template("message.html")
-        model["html"] = template.render(**context)
-        return await super().save(model)
+        context = json.loads(json.dumps(context, default=str))
+        loop = asyncio.get_event_loop()
+        logger.debug(f"save: rendering html for uid={model['uid']}")
+        model["html"] = await loop.run_in_executor(self.get_or_create_executor(model["uid"]), render, context)
+
+        result = await super().save(model)
+        if result:
+            logger.debug(f"save: message saved successfully uid={model['uid']}")
+        else:
+            logger.warning(f"save: failed to save message uid={model['uid']}")
+        if model['is_final']:
+            self.delete_executor(model['uid'])
+        return result 
 
     async def offset(self, channel_uid, page=0, timestamp=None, page_size=30):
+        channel = await self.services.channel.get(uid=channel_uid)
+        if not channel:
+            return []
+        history_start_filter = ""
+        if channel["history_start"]:
+            history_start_filter = f" AND created_at > '{channel['history_start']}'"
         results = []
         offset = page * page_size
         try:
             if timestamp:
                 async for model in self.query(
-                    "SELECT * FROM channel_message WHERE channel_uid=:channel_uid AND created_at < :timestamp ORDER BY created_at DESC LIMIT :page_size OFFSET :offset",
+                    f"SELECT * FROM channel_message WHERE channel_uid=:channel_uid AND created_at < :timestamp {history_start_filter} ORDER BY created_at DESC LIMIT :page_size OFFSET :offset",
                     {
                         "channel_uid": channel_uid,
                         "page_size": page_size,
@@ -84,7 +186,7 @@ class ChannelMessageService(BaseService):
                     results.append(model)
             elif page > 0:
                 async for model in self.query(
-                    "SELECT * FROM channel_message WHERE channel_uid=:channel_uid WHERE created_at < :timestamp ORDER BY created_at DESC LIMIT :page_size",
+                    f"SELECT * FROM channel_message WHERE channel_uid=:channel_uid WHERE created_at < :timestamp {history_start_filter} ORDER BY created_at DESC LIMIT :page_size",
                     {
                         "channel_uid": channel_uid,
                         "page_size": page_size,
@@ -95,7 +197,7 @@ class ChannelMessageService(BaseService):
                     results.append(model)
             else:
                 async for model in self.query(
-                    "SELECT * FROM channel_message WHERE channel_uid=:channel_uid ORDER BY created_at DESC LIMIT :page_size OFFSET :offset",
+                    f"SELECT * FROM channel_message WHERE channel_uid=:channel_uid {history_start_filter} ORDER BY created_at DESC LIMIT :page_size OFFSET :offset",
                     {
                         "channel_uid": channel_uid,
                         "page_size": page_size,
@@ -104,7 +206,7 @@ class ChannelMessageService(BaseService):
                 ):
                     results.append(model)
 
-        except:
-            pass
+        except Exception as ex:
+            logger.error(f"offset query failed: {ex}")
         results.sort(key=lambda x: x["created_at"])
         return results

src/snek/service/chat.py

@@ -1,25 +1,37 @@
+# retoor <retoor@molodetz.nl>
+
+import logging
+
 from snek.system.model import now
 from snek.system.service import BaseService
 
+logger = logging.getLogger(__name__)
+
 
 class ChatService(BaseService):
 
     async def finalize(self, message_uid):
+        logger.info(f"finalize: starting for message_uid={message_uid}")
         channel_message = await self.services.channel_message.get(uid=message_uid)
+        if not channel_message:
+            logger.warning(f"finalize: message not found uid={message_uid}")
+            return
         channel_message["is_final"] = True
         await self.services.channel_message.save(channel_message)
+        logger.debug(f"finalize: message marked as final uid={message_uid}")
         user = await self.services.user.get(uid=channel_message["user_uid"])
         channel = await self.services.channel.get(uid=channel_message["channel_uid"])
         channel["last_message_on"] = now()
         await self.services.channel.save(channel)
+        logger.debug(f"finalize: broadcasting message to channel={channel['uid']}")
         await self.services.socket.broadcast(
-            channel['uid'],
+            channel["uid"],
             {
                 "message": channel_message["message"],
                 "html": channel_message["html"],
-                "user_uid": user['uid'],
+                "user_uid": user["uid"],
                 "color": user["color"],
-                "channel_uid": channel['uid'],
+                "channel_uid": channel["uid"],
                 "created_at": channel_message["created_at"],
                 "updated_at": channel_message["updated_at"],
                 "username": user["username"],
@@ -28,21 +40,43 @@ class ChatService(BaseService):
                 "is_final": channel_message["is_final"],
             },
         )
-
+        logger.info(f"finalize: completed for message_uid={message_uid}, channel={channel['uid']}")
+        await self.app.create_task(
+            self.services.notification.create_channel_message(message_uid)
+        )
 
     async def send(self, user_uid, channel_uid, message, is_final=True):
+        logger.info(f"send: user_uid={user_uid}, channel_uid={channel_uid}, message_len={len(message) if message else 0}, is_final={is_final}")
         channel = await self.services.channel.get(uid=channel_uid)
         if not channel:
+            logger.error(f"send: channel not found channel_uid={channel_uid}")
             raise Exception("Channel not found.")
-        channel_message = await self.services.channel_message.create(
-            channel_uid, user_uid, message,is_final
+        logger.debug(f"send: checking for existing non-final message in channel={channel_uid}")
+        channel_message = await self.services.channel_message.get(
+            channel_uid=channel_uid,user_uid=user_uid, is_final=False
         )
+        if channel_message:
+            logger.debug(f"send: updating existing message uid={channel_message['uid']}")
+            channel_message["message"] = message
+            channel_message["is_final"] = is_final
+            if not channel_message["is_final"]:
+                async with self.app.no_save():
+                    await self.services.channel_message.save(channel_message)
+            else:
+                await self.services.channel_message.save(channel_message)
+        else:
+            logger.debug(f"send: creating new message in channel={channel_uid}")
+            channel_message = await self.services.channel_message.create(
+                channel_uid, user_uid, message, is_final
+            )
         channel_message_uid = channel_message["uid"]
+        logger.debug(f"send: message saved uid={channel_message_uid}")
 
         user = await self.services.user.get(uid=user_uid)
         channel["last_message_on"] = now()
         await self.services.channel.save(channel)
 
+        logger.debug(f"send: broadcasting message to channel={channel_uid}")
         await self.services.socket.broadcast(
             channel_uid,
             {
@@ -59,6 +93,7 @@ class ChatService(BaseService):
                 "is_final": is_final,
             },
         )
+        logger.info(f"send: completed message_uid={channel_message_uid}, channel={channel_uid}, is_final={is_final}")
         await self.app.create_task(
             self.services.notification.create_channel_message(channel_message_uid)
         )

src/snek/service/container.py

@@ -1,9 +1,109 @@
+# retoor <retoor@molodetz.nl>
+
+from snek.system.docker import ComposeFileManager
 from snek.system.service import BaseService
 
+
 class ContainerService(BaseService):
     mapper_name = "container"
 
-    async def create(self, id, name, status, resources=None, user_uid=None, path=None, readonly=False):
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+
+        self.compose_path = "snek-container-compose.yml"
+        self.compose = ComposeFileManager(self.compose_path,self.container_event_handler)
+        self.event_listeners = {}
+
+    async def shutdown(self):
+        return await self.compose.shutdown()
+
+    async def add_event_listener(self, name, event,event_handler):
+        if not name in self.event_listeners:
+            self.event_listeners[name] = {}
+        if not event in self.event_listeners[name]:
+            self.event_listeners[name][event] = []
+        self.event_listeners[name][event].append(event_handler)
+
+    async def remove_event_listener(self, name, event, event_handler):
+        if name in self.event_listeners and event in self.event_listeners[name]:
+            try:
+                self.event_listeners[name][event].remove(event_handler)
+            except ValueError:
+                pass 
+
+    async def container_event_handler(self, name, event, data):
+        event_listeners = self.event_listeners.get(name, {})
+        handlers = event_listeners.get(event, [])
+        for handler in handlers:
+            if not await handler(data):
+                handlers.remove(handler)
+
+    async def get_instances(self):
+        return list(self.compose.list_instances())
+
+    async def get_container_name(self, channel_uid):
+        if channel_uid.startswith("channel-"):
+            return channel_uid
+        return f"channel-{channel_uid}"
+
+    async def get(self,channel_uid):
+        return await self.compose.get_instance(await self.get_container_name(channel_uid))
+
+    async def stop(self, channel_uid):
+        return await self.compose.stop(await self.get_container_name(channel_uid))
+
+    async def start(self, channel_uid):
+        return await self.compose.start(await self.get_container_name(channel_uid))
+
+    async def maintenance(self):
+        async for channel in self.services.channel.find():
+            if not await self.get(channel["uid"]):
+                print("Creating container for channel", channel["uid"])
+                result = await self.create(channel_uid=channel["uid"])
+                print(result)
+
+
+    async def get_status(self, channel_uid):
+        return await self.compose.get_instance_status(await self.get_container_name(channel_uid))
+
+    async def write_stdin(self, channel_uid, data):
+        return await self.compose.write_stdin(await self.get_container_name(channel_uid), data)
+
+    async def create(
+        self,
+        channel_uid,
+        image="ubuntu:latest",
+        command=None,
+        cpus=1,
+        memory="1024m",
+        ports=None,
+        volumes=None,
+    ):
+        name = await self.get_container_name(channel_uid)
+            
+        test = await self.compose.get_instance(name)
+        if test:
+            return test
+
+        self.compose.create_instance(
+            name,
+            image,
+            command,
+            cpus,
+            memory,
+            ports,
+            [
+                "./"
+                + str(await self.services.channel.get_home_folder(channel_uid))
+                + ":"
+                + "/root"
+            ],
+        )
+        return await self.compose.get_instance(name)
+
+    async def create2(
+        self, id, name, status, resources=None, user_uid=None, path=None, readonly=False
+    ):
         model = await self.new()
         model["id"] = id
         model["name"] = name
@@ -18,12 +118,3 @@ class ContainerService(BaseService):
         if await super().save(model):
             return model
         raise Exception(f"Failed to create container: {model.errors}")
-
-    async def get(self, id):
-        return await self.mapper.get(id)
-
-    async def update(self, model):
-        return await self.mapper.update(model)
-
-    async def delete(self, id):
-        return await self.mapper.delete(id)

src/snek/service/db.py

@@ -1,23 +1,23 @@
-from snek.system.service import BaseService
-import dataset 
-import uuid 
+# retoor <retoor@molodetz.nl>
+
+import dataset
+
+from snek.system.service import BaseService
 
-from datetime import datetime 
 
 class DBService(BaseService):
-    
+
     async def get_db(self, user_uid):
-        
+
         home_folder = await self.app.services.user.get_home_folder(user_uid)
         home_folder.mkdir(parents=True, exist_ok=True)
         db_path = home_folder.joinpath("snek/user.db")
         db_path.parent.mkdir(parents=True, exist_ok=True)
-        return dataset.connect("sqlite:///" + str(db_path)) 
-    
+        return dataset.connect("sqlite:///" + str(db_path))
+
     async def insert(self, user_uid, table_name, values):
         db = await self.get_db(user_uid)
         return db[table_name].insert(values)
-        
 
     async def update(self, user_uid, table_name, values, filters):
         db = await self.get_db(user_uid)
@@ -33,7 +33,7 @@ class DBService(BaseService):
 
     async def find(self, user_uid, table_name, kwargs):
         db = await self.get_db(user_uid)
-        kwargs['_limit'] = kwargs.get('_limit', 30)
+        kwargs["_limit"] = kwargs.get("_limit", 30)
         return [dict(row) for row in db[table_name].find(**kwargs)]
 
     async def get(self, user_uid, table_name, filters):
@@ -45,14 +45,13 @@ class DBService(BaseService):
         except ValueError:
             return None
 
-
     async def delete(self, user_uid, table_name, filters):
         db = await self.get_db(user_uid)
         if not filters:
             filters = {}
         return db[table_name].delete(**filters)
 
-    async def query(self, sql,values):
+    async def query(self, sql, values):
         db = await self.app.db
         return [dict(row) for row in db.query(sql, values or {})]
 
@@ -62,8 +61,6 @@ class DBService(BaseService):
             filters = {}
         return bool(db[table_name].find_one(**filters))
 
-    
-
     async def count(self, user_uid, table_name, filters):
         db = await self.get_db(user_uid)
         if not filters:

src/snek/service/drive.py

@@ -1,3 +1,5 @@
+# retoor <retoor@molodetz.nl>
+
 from snek.system.service import BaseService
 
 

src/snek/service/drive_item.py

@@ -1,3 +1,5 @@
+# retoor <retoor@molodetz.nl>
+
 from snek.system.service import BaseService
 
 

src/snek/service/forum.py

@@ -0,0 +1,310 @@
+# retoor <retoor@molodetz.nl>
+
+# services/forum.py
+from snek.system.service import BaseService
+import re
+import uuid
+from collections import defaultdict
+from typing import Any, Awaitable, Callable, Dict, List
+import asyncio
+import inspect
+from snek.system.model import now
+EventListener = Callable[[str, Any], Awaitable[None]] | Callable[[str, Any], None]
+class BaseForumService(BaseService):
+    """
+    Base mix-in that gives a service `add_notification_listener`,
+    an internal `_dispatch_event` helper, and a public `notify` method.
+    """
+    def get_timestamp(self):
+        return now()
+    def generate_uid(self):
+        return str(uuid.uuid4())
+
+    def __init__(self, *args, **kwargs) -> None:
+        # Map event name -> list of listener callables
+        self._listeners: Dict[str, List[EventListener]] = defaultdict(list)
+        super().__init__(*args, **kwargs)
+
+    def add_notification_listener(
+        self, event_name: str, listener: EventListener
+    ) -> None:
+        """
+        Register a callback to be fired when `event_name` happens.
+
+        Parameters
+        ----------
+        event_name : str
+            The name of the domain event, e.g. "post_created".
+        listener : Callable[[str, Any], Awaitable[None]] | Callable[[str, Any], None]
+            Your handler; can be async or sync.
+        """
+        if not callable(listener):
+            raise TypeError("listener must be callable")
+
+        self._listeners[event_name].append(listener)
+
+    async def _dispatch_event(self, event_name: str, data: Any) -> None:
+        """Invoke every listener for the given event."""
+        for listener in self._listeners.get(event_name, []):
+            if inspect.iscoroutinefunction(listener):
+                await listener(event_name, data)
+            else:
+                result = listener(event_name, data)
+                if inspect.isawaitable(result):
+                    await result
+
+    async def notify(self, event_name: str, data: Any) -> None:
+        """
+        Public method to trigger notification to all listeners of an event.
+
+        Parameters
+        ----------
+        event_name : str
+            The name of the event to notify listeners about.
+        data : Any
+            The data to pass to the listeners.
+        """
+        await self._dispatch_event(event_name, data)
+
+
+class ForumService(BaseForumService):
+    mapper_name = "forum"
+
+    async def create_forum(self, name, description, created_by_uid, slug=None, icon=None):
+        if not slug:
+            slug = self.generate_slug(name)
+        
+        # Check if slug exists
+        existing = await self.get(slug=slug)
+        if existing:
+            slug = f"{slug}-{self.generate_uid()[:8]}"
+        
+        model = await self.new()
+        model["name"] = name
+        model["description"] = description
+        model["slug"] = slug
+        model["created_by_uid"] = created_by_uid
+        if icon:
+            model["icon"] = icon
+        
+        if await self.save(model):
+            await self.notify("forum_created", model)
+            return model
+        raise Exception(f"Failed to create forum: {model.errors}")
+
+    def generate_slug(self, text):
+        # Convert to lowercase and replace spaces with hyphens
+        slug = text.lower().strip()
+        slug = re.sub(r'[^\w\s-]', '', slug)
+        slug = re.sub(r'[-\s]+', '-', slug)
+        return slug
+
+    async def get_active_forums(self):
+        async for forum in self.find(is_active=True, order_by="position"):
+            yield forum
+
+    async def update_last_post(self, forum_uid, thread_uid):
+        forum = await self.get(uid=forum_uid)
+        if forum:
+            forum["last_post_at"] = self.get_timestamp()
+            forum["last_thread_uid"] = thread_uid
+            await self.save(forum)
+
+
+# services/thread.py  
+class ThreadService(BaseForumService):
+    mapper_name = "thread"
+
+   
+
+    async def create_thread(self, forum_uid, title, content, created_by_uid):
+        # Generate slug
+        slug = self.services.forum.generate_slug(title)
+        
+        # Check if slug exists in this forum
+        existing = await self.get(forum_uid=forum_uid, slug=slug)
+        if existing:
+            slug = f"{slug}-{self.generate_uid()[:8]}"
+        
+        # Create thread
+        thread = await self.new()
+        thread["forum_uid"] = forum_uid
+        thread["title"] = title
+        thread["slug"] = slug
+        thread["created_by_uid"] = created_by_uid
+        thread["last_post_at"] = self.get_timestamp()
+        thread["last_post_by_uid"] = created_by_uid
+        
+        if await self.save(thread):
+            # Create first post
+            post = await self.services.post.create_post(
+                thread_uid=thread["uid"],
+                forum_uid=forum_uid,
+                content=content,
+                created_by_uid=created_by_uid,
+                is_first_post=True
+            )
+            
+            # Update forum counters
+            forum = await self.services.forum.get(uid=forum_uid)
+            await forum.increment_thread_count()
+            await self.services.forum.update_last_post(forum_uid, thread["uid"])
+            
+            await self.notify("thread_created", {
+                "thread": thread,
+                "forum_uid": forum_uid
+            })
+            
+            return thread, post
+        raise Exception(f"Failed to create thread: {thread.errors}")
+
+    async def toggle_pin(self, thread_uid, user_uid):
+        thread = await self.get(uid=thread_uid)
+        if not thread:
+            return None
+        
+        # Check if user is admin
+        user = await self.services.user.get(uid=user_uid)
+        if not user["is_admin"]:
+            return None
+        
+        thread["is_pinned"] = not thread["is_pinned"]
+        await self.save(thread)
+        return thread
+
+    async def toggle_lock(self, thread_uid, user_uid):
+        thread = await self.get(uid=thread_uid)
+        if not thread:
+            return None
+        
+        # Check if user is admin or thread creator
+        user = await self.services.user.get(uid=user_uid)
+        if not user.get("is_admin") and thread["created_by_uid"] != user_uid:
+            return None
+        
+        thread["is_locked"] = not thread["is_locked"]
+        await self.save(thread)
+        return thread
+
+
+# services/post.py
+class PostService(BaseForumService):
+    mapper_name = "post"
+
+    async def create_post(self, thread_uid, forum_uid, content, created_by_uid, is_first_post=False):
+        # Check if thread is locked
+        thread = await self.services.thread.get(uid=thread_uid)
+        if thread["is_locked"] and not is_first_post:
+            raise Exception("Thread is locked")
+        
+        post = await self.new()
+        post["thread_uid"] = thread_uid
+        post["forum_uid"] = forum_uid
+        post["content"] = content
+        post["created_by_uid"] = created_by_uid
+        post["is_first_post"] = is_first_post
+        
+        if await self.save(post):
+            # Update thread counters
+            if not is_first_post:
+                thread["post_count"] += 1
+                thread["last_post_at"] = self.get_timestamp()
+                thread["last_post_by_uid"] = created_by_uid
+                await self.services.thread.save(thread)
+                
+                # Update forum counters
+                forum = await self.services.forum.get(uid=forum_uid)
+                await forum.increment_post_count()
+                await self.services.forum.update_last_post(forum_uid, thread_uid)
+            
+            await self.notify("post_created", {
+                "post": post,
+                "thread_uid": thread_uid,
+                "forum_uid": forum_uid
+            })
+            
+            return post
+        raise Exception(f"Failed to create post: {post.errors}")
+
+    async def edit_post(self, post_uid, content, user_uid):
+        post = await self.get(uid=post_uid)
+        if not post:
+            return None
+        
+        # Check permissions
+        user = await self.services.user.get(uid=user_uid)
+        if post["created_by_uid"] != user_uid and not user["is_admin"]:
+            return None
+        
+        post["content"] = content
+        post["edited_at"] = self.get_timestamp()
+        post["edited_by_uid"] = user_uid
+        
+        if await self.save(post):
+            await self.notify("post_edited", post)
+            return post
+        return None
+
+    async def delete_post(self, post_uid, user_uid):
+        post = await self.get(uid=post_uid)
+        if not post:
+            return False
+        
+        # Check permissions
+        user = await self.services.user.get(uid=user_uid)
+        if post["created_by_uid"] != user_uid and not user["is_admin"]:
+            return False
+        
+        # Don't allow deleting first post
+        if post["is_first_post"]:
+            return False
+        
+        post["deleted_at"] = self.get_timestamp()
+        if await self.save(post):
+            await self.notify("post_deleted", post)
+            return True
+        return False
+
+
+# services/post_like.py
+class PostLikeService(BaseForumService):
+    mapper_name = "post_like"
+
+    async def toggle_like(self, post_uid, user_uid):
+        # Check if already liked
+        existing = await self.get(post_uid=post_uid, user_uid=user_uid)
+        
+        if existing:
+            # Unlike
+            await self.delete(uid=existing["uid"])
+            
+            # Update post like count
+            post = await self.services.post.get(uid=post_uid)
+            post["like_count"] = max(0, post["like_count"] - 1)
+            await self.services.post.save(post)
+            
+            await self.notify("post_unliked", {
+                "post_uid": post_uid,
+                "user_uid": user_uid
+            })
+            
+            return False
+        else:
+            # Like
+            like = await self.new()
+            like["post_uid"] = post_uid
+            like["user_uid"] = user_uid
+            
+            if await self.save(like):
+                # Update post like count
+                post = await self.services.post.get(uid=post_uid)
+                post["like_count"] += 1
+                await self.services.post.save(post)
+                
+                await self.notify("post_liked", {
+                    "post_uid": post_uid,
+                    "user_uid": user_uid
+                })
+                
+                return True
+        return None

src/snek/service/notification.py

@@ -1,6 +1,13 @@
+# retoor <retoor@molodetz.nl>
+
+import logging
+
+from snek.system.markdown import strip_markdown
 from snek.system.model import now
 from snek.system.service import BaseService
 
+logger = logging.getLogger(__name__)
+
 
 class NotificationService(BaseService):
     mapper_name = "notification"
@@ -33,6 +40,8 @@ class NotificationService(BaseService):
         channel_message = await self.services.channel_message.get(
             uid=channel_message_uid
         )
+        if not channel_message["is_final"]:
+            return
         user = await self.services.user.get(uid=channel_message["user_uid"])
         self.app.db.begin()
         async for channel_member in self.services.channel_member.find(
@@ -62,4 +71,20 @@ class NotificationService(BaseService):
             except Exception:
                 raise Exception(f"Failed to create notification: {model.errors}.")
 
+            if channel_member["user_uid"] != user["uid"]:
+                try:
+                    stripped_message = strip_markdown(channel_message["message"])
+                    channel_name = await channel_member.get_name()
+                    await self.app.services.push.notify_user(
+                        user_uid=channel_member["user_uid"],
+                        payload={
+                            "title": f"New message in {channel_name}",
+                            "message": f"{user['nick']}: {stripped_message}",
+                            "icon": "/image/snek192.png",
+                            "url": f"/channel/{channel_message['channel_uid']}.html",
+                        },
+                    )
+                except Exception as e:
+                    logger.warning(f"Failed to send push notification: {e}")
+
         self.app.db.commit()

src/snek/service/profile_page.py

@@ -0,0 +1,87 @@
+# retoor <retoor@molodetz.nl>
+
+import logging
+import re
+from typing import Optional, List
+from snek.system.service import BaseService
+from snek.system.exceptions import ValidationError, DuplicateResourceError
+
+logger = logging.getLogger(__name__)
+
+class ProfilePageService(BaseService):
+    mapper_name = "profile_page"
+
+    def slugify(self, title: str) -> str:
+        slug = title.lower().strip()
+        slug = re.sub(r'[^\w\s-]', '', slug)
+        slug = re.sub(r'[-\s]+', '-', slug)
+        return slug[:100]
+
+    async def create_page(self, user_uid: str, title: str, content: str = "", is_published: bool = True) -> dict:
+        slug = self.slugify(title)
+
+        existing = await self.get(user_uid=user_uid, slug=slug, deleted_at=None)
+        if existing:
+            raise DuplicateResourceError(f"A page with slug '{slug}' already exists")
+
+        pages = [p async for p in self.find(user_uid=user_uid, deleted_at=None)]
+        max_order = max([p["order_index"] for p in pages], default=-1)
+
+        model = await self.new()
+        model["user_uid"] = user_uid
+        model["title"] = title
+        model["slug"] = slug
+        model["content"] = content
+        model["order_index"] = max_order + 1
+        model["is_published"] = is_published
+
+        await self.save(model)
+        return model
+
+    async def update_page(self, page_uid: str, title: Optional[str] = None,
+                         content: Optional[str] = None, is_published: Optional[bool] = None) -> dict:
+        page = await self.get(uid=page_uid, deleted_at=None)
+        if not page:
+            raise ValidationError("Page not found")
+
+        if title is not None:
+            page["title"] = title
+            new_slug = self.slugify(title)
+            existing = await self.get(user_uid=page["user_uid"], slug=new_slug, deleted_at=None)
+            if existing and existing["uid"] != page_uid:
+                raise DuplicateResourceError(f"A page with slug '{new_slug}' already exists")
+            page["slug"] = new_slug
+
+        if content is not None:
+            page["content"] = content
+
+        if is_published is not None:
+            page["is_published"] = is_published
+
+        return await self.save(page)
+
+    async def get_user_pages(self, user_uid: str, include_unpublished: bool = False) -> List[dict]:
+        if include_unpublished:
+            pages = [p.record async for p in self.find(user_uid=user_uid, deleted_at=None)]
+        else:
+            pages = [p.record async for p in self.find(user_uid=user_uid, is_published=True, deleted_at=None)]
+
+        return sorted(pages, key=lambda p: p["order_index"])
+
+    async def get_page_by_slug(self, user_uid: str, slug: str, include_unpublished: bool = False) -> Optional[dict]:
+        page = await self.get(user_uid=user_uid, slug=slug, deleted_at=None)
+        if page and (include_unpublished or page["is_published"]):
+            return page
+        return None
+
+    async def reorder_pages(self, user_uid: str, page_uids: List[str]) -> None:
+        for index, page_uid in enumerate(page_uids):
+            page = await self.get(uid=page_uid, user_uid=user_uid, deleted_at=None)
+            if page:
+                page["order_index"] = index
+                await self.save(page)
+
+    async def delete_page(self, page_uid: str) -> None:
+        page = await self.get(uid=page_uid, deleted_at=None)
+        if page:
+            await self.delete(page)

src/snek/service/push.py

@@ -0,0 +1,269 @@
+# retoor <retoor@molodetz.nl>
+
+import base64
+import json
+import os.path
+import random
+import time
+import uuid
+from pathlib import Path
+from urllib.parse import urlparse
+
+import aiohttp
+import jwt
+from cryptography.hazmat.backends import default_backend
+from cryptography.hazmat.primitives import serialization
+from cryptography.hazmat.primitives.asymmetric import ec
+from cryptography.hazmat.primitives.ciphers.aead import AESGCM
+from cryptography.hazmat.primitives.hashes import SHA256
+from cryptography.hazmat.primitives.kdf.hkdf import HKDF
+
+from snek.system.service import BaseService
+
+# The only reason to persist the keys is to be able to use them in the web push
+
+PRIVATE_KEY_FILE = Path("./notification-private.pem")
+PRIVATE_KEY_PKCS8_FILE = Path("./notification-private.pkcs8.pem")
+PUBLIC_KEY_FILE = Path("./notification-public.pem")
+
+
+def generate_private_key():
+    if not PRIVATE_KEY_FILE.exists():
+        private_key = ec.generate_private_key(ec.SECP256R1(), default_backend())
+
+        pem = private_key.private_bytes(
+            encoding=serialization.Encoding.PEM,
+            format=serialization.PrivateFormat.TraditionalOpenSSL,
+            encryption_algorithm=serialization.NoEncryption(),
+        )
+
+        PRIVATE_KEY_FILE.write_bytes(pem)
+
+
+def generate_pcks8_private_key():
+    if not PRIVATE_KEY_PKCS8_FILE.exists():
+        private_key = serialization.load_pem_private_key(
+            PRIVATE_KEY_FILE.read_bytes(), password=None, backend=default_backend()
+        )
+
+        pem = private_key.private_bytes(
+            encoding=serialization.Encoding.PEM,
+            format=serialization.PrivateFormat.PKCS8,
+            encryption_algorithm=serialization.NoEncryption(),
+        )
+
+        PRIVATE_KEY_PKCS8_FILE.write_bytes(pem)
+
+
+def generate_public_key():
+    if not PUBLIC_KEY_FILE.exists():
+        private_key = serialization.load_pem_private_key(
+            PRIVATE_KEY_FILE.read_bytes(), password=None, backend=default_backend()
+        )
+
+        public_key = private_key.public_key()
+
+        pem = public_key.public_bytes(
+            encoding=serialization.Encoding.PEM,
+            format=serialization.PublicFormat.SubjectPublicKeyInfo,
+        )
+
+        PUBLIC_KEY_FILE.write_bytes(pem)
+
+
+def ensure_certificates():
+    generate_private_key()
+    generate_pcks8_private_key()
+    generate_public_key()
+
+
+def hkdf(input_key, salt, info, length):
+    return HKDF(
+        algorithm=SHA256(),
+        length=length,
+        salt=salt,
+        info=info,
+        backend=default_backend(),
+    ).derive(input_key)
+
+
+def _browser_base64(data):
+    return base64.urlsafe_b64encode(data).decode("utf-8").rstrip("=")
+
+
+class PushService(BaseService):
+    mapper_name = "push"
+
+    private_key_pem = None
+    public_key = None
+    public_key_base64 = None
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        ensure_certificates()
+
+        private_key = serialization.load_pem_private_key(
+            PRIVATE_KEY_FILE.read_bytes(), password=None, backend=default_backend()
+        )
+        self.private_key_pem = private_key.private_bytes(
+            encoding=serialization.Encoding.PEM,
+            format=serialization.PrivateFormat.TraditionalOpenSSL,
+            encryption_algorithm=serialization.NoEncryption(),
+        )
+
+        self.public_key = serialization.load_pem_public_key(
+            PUBLIC_KEY_FILE.read_bytes(), backend=default_backend()
+        )
+
+        self.public_key_base64 = _browser_base64(
+            self.public_key.public_bytes(
+                encoding=serialization.Encoding.X962,
+                format=serialization.PublicFormat.UncompressedPoint,
+            )
+        )
+
+    def create_notification_authorization(self, push_url):
+        target = urlparse(push_url)
+        aud = f"{target.scheme}://{target.netloc}"
+        sub = "mailto:admin@molodetz.nl"
+
+        identifier = str(uuid.uuid4())
+
+        print(
+            f"Creating notification authorization for {aud} with identifier {identifier}"
+        )
+
+        return jwt.encode(
+            {
+                "sub": sub,
+                "aud": aud,
+                "exp": int(time.time()) + 60 * 60,
+                "nbf": int(time.time()),
+                "iat": int(time.time()),
+                "jti": identifier,
+            },
+            self.private_key_pem,
+            algorithm="ES256",
+        )
+
+    def create_notification_info_with_payload(
+        self, endpoint: str, auth: str, p256dh: str, payload: str
+    ):
+        message_private_key = ec.generate_private_key(ec.SECP256R1(), default_backend())
+
+        message_public_key_bytes = message_private_key.public_key().public_bytes(
+            encoding=serialization.Encoding.X962,
+            format=serialization.PublicFormat.UncompressedPoint,
+        )
+
+        salt = os.urandom(16)
+
+        user_key_bytes = base64.urlsafe_b64decode(p256dh + "==")
+        shared_secret = message_private_key.exchange(
+            ec.ECDH(),
+            ec.EllipticCurvePublicKey.from_encoded_point(
+                ec.SECP256R1(), user_key_bytes
+            ),
+        )
+
+        encryption_key = hkdf(
+            shared_secret,
+            base64.urlsafe_b64decode(auth + "=="),
+            b"Content-Encoding: auth\x00",
+            32,
+        )
+
+        context = (
+            b"P-256\x00"
+            + len(user_key_bytes).to_bytes(2, "big")
+            + user_key_bytes
+            + len(message_public_key_bytes).to_bytes(2, "big")
+            + message_public_key_bytes
+        )
+
+        nonce = hkdf(encryption_key, salt, b"Content-Encoding: nonce\x00" + context, 12)
+        content_encryption_key = hkdf(
+            encryption_key, salt, b"Content-Encoding: aesgcm\x00" + context, 16
+        )
+
+        padding_length = random.randint(0, 16)
+        padding = padding_length.to_bytes(2, "big") + b"\x00" * padding_length
+
+        data = AESGCM(content_encryption_key).encrypt(
+            nonce, padding + payload.encode("utf-8"), None
+        )
+
+        return {
+            "headers": {
+                "Authorization": f"WebPush {self.create_notification_authorization(endpoint)}",
+                "Crypto-Key": f"dh={_browser_base64(message_public_key_bytes)}; p256ecdsa={self.public_key_base64}",
+                "Encryption": f"salt={_browser_base64(salt)}",
+                "Content-Encoding": "aesgcm",
+                "Content-Length": str(len(data)),
+                "Content-Type": "application/octet-stream",
+            },
+            "data": data,
+        }
+
+    async def notify_user(self, user_uid: str, payload: dict):
+        async with aiohttp.ClientSession() as session:
+            async for subscription in self.find(user_uid=user_uid):
+                endpoint = subscription["endpoint"]
+                key_auth = subscription["key_auth"]
+                key_p256dh = subscription["key_p256dh"]
+
+                notification_info = self.create_notification_info_with_payload(
+                    endpoint, key_auth, key_p256dh, json.dumps(payload)
+                )
+
+                headers = {
+                    **notification_info["headers"],
+                    "TTL": "60",
+                }
+                data = notification_info["data"]
+
+                async with session.post(
+                    endpoint,
+                    headers=headers,
+                    data=data,
+                ) as response:
+                    if response.status == 201 or response.status == 200:
+                        print(
+                            f"Notification sent to user {user_uid} via endpoint {endpoint}"
+                        )
+                    else:
+                        print(
+                            f"Failed to send notification to user {user_uid} via endpoint {endpoint}: {response.status}"
+                        )
+
+    async def register(
+        self, user_uid: str, endpoint: str, key_auth: str, key_p256dh: str
+    ):
+        if await self.exists(
+            user_uid=user_uid,
+            endpoint=endpoint,
+            key_auth=key_auth,
+            key_p256dh=key_p256dh,
+        ):
+            return
+
+        model = await self.new()
+        model["user_uid"] = user_uid
+        model["endpoint"] = endpoint
+        model["key_auth"] = key_auth
+        model["key_p256dh"] = key_p256dh
+
+        print(
+            f"Registering push subscription for user {user_uid} with endpoint {endpoint}"
+        )
+
+        if await self.save(model=model) and model:
+            print(
+                f"Push subscription registered for user {user_uid} with endpoint {endpoint}"
+            )
+
+            return model
+
+        raise Exception(
+            f"Failed to register push subscription for user {user_uid} with endpoint {endpoint}"
+        )

src/snek/service/repository.py

@@ -1,13 +1,19 @@
-from snek.system.service import BaseService
-import asyncio 
+# retoor <retoor@molodetz.nl>
+
+import asyncio
 import shutil
 
+from snek.system.service import BaseService
+
+
 class RepositoryService(BaseService):
     mapper_name = "repository"
 
     async def delete(self, user_uid, name):
         loop = asyncio.get_event_loop()
-        repository_path = (await self.services.user.get_repository_path(user_uid)).joinpath(name)
+        repository_path = (
+            await self.services.user.get_repository_path(user_uid)
+        ).joinpath(name + ".git")
         try:
             await loop.run_in_executor(None, shutil.rmtree, repository_path)
         except Exception as ex:
@@ -15,7 +21,6 @@ class RepositoryService(BaseService):
 
         await super().delete(user_uid=user_uid, name=name)
 
-
     async def exists(self, user_uid, name, **kwargs):
         kwargs["user_uid"] = user_uid
         kwargs["name"] = name
@@ -29,18 +34,16 @@ class RepositoryService(BaseService):
         repository_path = str(repository_path)
         if not repository_path.endswith(".git"):
             repository_path += ".git"
-        command = ['git', 'init', '--bare', repository_path]
+        command = ["git", "init", "--bare", repository_path]
         process = await asyncio.subprocess.create_subprocess_exec(
-            *command,
-            stdout=asyncio.subprocess.PIPE,
-            stderr=asyncio.subprocess.PIPE
+            *command, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE
         )
         stdout, stderr = await process.communicate()
         return process.returncode == 0
 
-    async def create(self, user_uid, name,is_private=False):
+    async def create(self, user_uid, name, is_private=False, description=None):
         if await self.exists(user_uid=user_uid, name=name):
-            return False 
+            return False
 
         if not await self.init(user_uid=user_uid, name=name):
             return False
@@ -49,4 +52,14 @@ class RepositoryService(BaseService):
         model["user_uid"] = user_uid
         model["name"] = name
         model["is_private"] = is_private
+        model["description"] = description or ""
         return await self.save(model)
+
+    async def list_by_user(self, user_uid):
+        repositories = []
+        async for repo in self.find(user_uid=user_uid):
+            repositories.append(repo)
+        return repositories
+
+    async def get_by_name(self, user_uid, name):
+        return await self.get(user_uid=user_uid, name=name)

src/snek/service/socket.py

@@ -1,12 +1,38 @@
+# retoor <retoor@molodetz.nl>
+
+import asyncio
+import logging
+from datetime import datetime
 from snek.model.user import UserModel
 from snek.system.service import BaseService
-from datetime import datetime 
-import json
-import asyncio
-import logging 
-logger = logging.getLogger(__name__)
 from snek.system.model import now
 
+logger = logging.getLogger(__name__)
+
+
+def safe_get(obj, key, default=None):
+    if obj is None:
+        return default
+    try:
+        if isinstance(obj, dict):
+            return obj.get(key, default)
+        if hasattr(obj, "fields") and hasattr(obj, "__getitem__"):
+            val = obj[key]
+            return val if val is not None else default
+        return getattr(obj, key, default)
+    except (KeyError, TypeError, AttributeError):
+        return default
+
+
+def safe_str(obj):
+    if obj is None:
+        return ""
+    try:
+        return str(obj)
+    except Exception:
+        return ""
+
+
 class SocketService(BaseService):
 
     class Socket:
@@ -14,24 +40,41 @@ class SocketService(BaseService):
             self.ws = ws
             self.is_connected = True
             self.user = user
-
+            self.user_uid = safe_get(user, "uid") if user else None
+            self.user_color = safe_get(user, "color") if user else None
+            self.subscribed_channels = set()
 
         async def send_json(self, data):
+            if data is None:
+                return False
             if not self.is_connected:
                 return False
+            if not self.ws:
+                self.is_connected = False
+                return False
             try:
                 await self.ws.send_json(data)
-            except Exception:
+                return True
+            except ConnectionResetError:
                 self.is_connected = False
-            return self.is_connected
+                logger.debug("Connection reset during send_json")
+            except Exception as ex:
+                self.is_connected = False
+                logger.debug(f"send_json failed: {safe_str(ex)}")
+            return False
 
         async def close(self):
             if not self.is_connected:
                 return True
-
-            await self.ws.close()
             self.is_connected = False
-
+            self.subscribed_channels.clear()
+            try:
+                if self.ws and not self.ws.closed:
+                    await asyncio.wait_for(self.ws.close(), timeout=5.0)
+            except asyncio.TimeoutError:
+                logger.debug("Socket close timed out")
+            except Exception as ex:
+                logger.debug(f"Socket close failed: {safe_str(ex)}")
             return True
 
     def __init__(self, app):
@@ -40,66 +83,292 @@ class SocketService(BaseService):
         self.users = {}
         self.subscriptions = {}
         self.last_update = str(datetime.now())
-        
+        self._lock = asyncio.Lock()
+
 
     async def user_availability_service(self):
         logger.info("User availability update service started.")
         while True:
-            logger.info("Updating user availability...")
-            users_updated = []
-            for s in self.sockets:
-                if not s.is_connected:
-                    continue
-                if not s.user in users_updated:
-                    s.user["last_ping"] = now()
-                    await self.app.services.user.save(s.user)
-                    users_updated.append(s.user)
-            logger.info(f"Updated user availability for {len(users_updated)} online users.")
-            await asyncio.sleep(60)
-
+            try:
+                sockets_copy = list(self.sockets)
+                users_to_update = []
+                seen_user_uids = set()
+                for s in sockets_copy:
+                    try:
+                        if not s or not s.is_connected:
+                            continue
+                        if not s.user or not s.user_uid:
+                            continue
+                        if s.user_uid in seen_user_uids:
+                            continue
+                        seen_user_uids.add(s.user_uid)
+                        users_to_update.append(s.user_uid)
+                    except Exception as ex:
+                        logger.debug(f"Failed to check user availability: {safe_str(ex)}")
+                for user_uid in users_to_update:
+                    try:
+                        if self.app and hasattr(self.app, "services") and self.app.services:
+                            user = await self.app.services.user.get(uid=user_uid)
+                            if user:
+                                user["last_ping"] = now()
+                                await self.app.services.user.save(user)
+                    except Exception as ex:
+                        logger.debug(f"Failed to update user availability: {safe_str(ex)}")
+                logger.info(f"Updated user availability for {len(users_to_update)} online users.")
+            except Exception as ex:
+                logger.warning(f"User availability service error: {safe_str(ex)}")
+            try:
+                await asyncio.sleep(60)
+            except asyncio.CancelledError:
+                logger.info("User availability service cancelled")
+                break
 
     async def add(self, ws, user_uid):
-        s = self.Socket(ws, await self.app.services.user.get(uid=user_uid))
-        self.sockets.add(s)
-        s.user["last_ping"] = now()
-        await self.app.services.user.save(s.user)
-        logger.info(f"Added socket for user {s.user['username']}")
-        if not self.users.get(user_uid):
-            self.users[user_uid] = set()
-        self.users[user_uid].add(s)
+        if not ws:
+            return None
+        if not user_uid:
+            return None
+        try:
+            if not self.app or not hasattr(self.app, "services") or not self.app.services:
+                logger.warning("Services not available for socket add")
+                return None
+            user = await self.app.services.user.get(uid=user_uid)
+            if not user:
+                logger.warning(f"User not found for socket add: {user_uid}")
+                return None
+            s = self.Socket(ws, user)
+            username = safe_get(user, "username", "unknown")
+            nick = safe_get(user, "nick") or username
+            color = s.user_color
+            async with self._lock:
+                self.sockets.add(s)
+                is_first_connection = False
+                if user_uid not in self.users:
+                    self.users[user_uid] = set()
+                    is_first_connection = True
+                elif len(self.users[user_uid]) == 0:
+                    is_first_connection = True
+                self.users[user_uid].add(s)
+            try:
+                fresh_user = await self.app.services.user.get(uid=user_uid)
+                if fresh_user:
+                    fresh_user["last_ping"] = now()
+                    await self.app.services.user.save(fresh_user)
+            except Exception as ex:
+                logger.debug(f"Failed to update last_ping: {safe_str(ex)}")
+            logger.info(f"Added socket for user {username}")
+            if is_first_connection:
+                await self._broadcast_presence("arrived", user_uid, nick, color)
+            return s
+        except Exception as ex:
+            logger.warning(f"Failed to add socket: {safe_str(ex)}")
+            return None
 
     async def subscribe(self, ws, channel_uid, user_uid):
-        if channel_uid not in self.subscriptions:
-            self.subscriptions[channel_uid] = set()
-        s = self.Socket(ws, await self.app.services.user.get(uid=user_uid))
-        self.subscriptions[channel_uid].add(s)
+        if not ws or not channel_uid or not user_uid:
+            return False
+        try:
+            async with self._lock:
+                existing_socket = None
+                user_sockets = self.users.get(user_uid, set())
+                for sock in user_sockets:
+                    if sock and sock.ws == ws:
+                        existing_socket = sock
+                        break
+                if not existing_socket:
+                    return False
+                existing_socket.subscribed_channels.add(channel_uid)
+                if channel_uid not in self.subscriptions:
+                    self.subscriptions[channel_uid] = set()
+                self.subscriptions[channel_uid].add(user_uid)
+            return True
+        except Exception as ex:
+            logger.warning(f"Failed to subscribe: {safe_str(ex)}")
+            return False
+
+    async def unsubscribe(self, ws, channel_uid, user_uid):
+        if not ws or not channel_uid or not user_uid:
+            return False
+        try:
+            async with self._lock:
+                if channel_uid in self.subscriptions:
+                    self.subscriptions[channel_uid].discard(user_uid)
+                    if len(self.subscriptions[channel_uid]) == 0:
+                        del self.subscriptions[channel_uid]
+                for s in self.sockets:
+                    if s and s.ws == ws:
+                        s.subscribed_channels.discard(channel_uid)
+                        break
+            return True
+        except Exception as ex:
+            logger.warning(f"Failed to unsubscribe: {safe_str(ex)}")
+            return False
 
     async def send_to_user(self, user_uid, message):
+        if not user_uid or message is None:
+            return 0
         count = 0
-        for s in self.users.get(user_uid, []):
-            if await s.send_json(message):
-                count += 1
+        try:
+            user_sockets = list(self.users.get(user_uid, []))
+            for s in user_sockets:
+                if not s:
+                    continue
+                try:
+                    if await s.send_json(message):
+                        count += 1
+                except Exception as ex:
+                    logger.debug(f"Failed to send to user socket: {safe_str(ex)}")
+        except Exception as ex:
+            logger.warning(f"send_to_user failed: {safe_str(ex)}")
         return count
 
-
     async def broadcast(self, channel_uid, message):
-        await self._broadcast(channel_uid, message)
+        if not channel_uid or message is None:
+            logger.debug(f"broadcast: invalid params channel_uid={channel_uid}, message={message is not None}")
+            return False
+        logger.debug(f"broadcast: starting for channel={channel_uid}")
+        return await self._broadcast(channel_uid, message)
 
     async def _broadcast(self, channel_uid, message):
+        if not channel_uid or message is None:
+            return False
         sent = 0
+        user_uids_to_send = set()
         try:
-            async for user_uid in self.services.channel_member.get_user_uids(
-                channel_uid
-            ):
-                sent += await self.send_to_user(user_uid, message)
+            if self.services:
+                try:
+                    async for user_uid in self.services.channel_member.get_user_uids(channel_uid):
+                        if user_uid:
+                            user_uids_to_send.add(user_uid)
+                    logger.debug(f"_broadcast: found {len(user_uids_to_send)} users from db for channel={channel_uid}")
+                except Exception as ex:
+                    logger.warning(f"Broadcast db query failed: {safe_str(ex)}")
+            if not user_uids_to_send:
+                if channel_uid in self.subscriptions:
+                    user_uids_to_send = set(self.subscriptions[channel_uid])
+                logger.debug(f"_broadcast: using {len(user_uids_to_send)} users from subscriptions for channel={channel_uid}")
+            send_tasks = []
+            for user_uid in user_uids_to_send:
+                send_tasks.append(self._send_to_user_safe(user_uid, message))
+            if send_tasks:
+                results = await asyncio.gather(*send_tasks, return_exceptions=True)
+                for result in results:
+                    if isinstance(result, int):
+                        sent += result
+            logger.info(f"_broadcast: completed channel={channel_uid}, total_users={len(user_uids_to_send)}, sent={sent}")
+            return True
         except Exception as ex:
-            print(ex, flush=True)
-        logger.info(f"Broadcasted a message to {sent} users.")
-        return True
+            logger.warning(f"Broadcast failed: {safe_str(ex)}")
+            return False
+
+    async def _send_to_user_safe(self, user_uid, message):
+        try:
+            return await self.send_to_user(user_uid, message)
+        except Exception as ex:
+            logger.debug(f"Failed to send to user {user_uid}: {safe_str(ex)}")
+            return 0
 
     async def delete(self, ws):
-        for s in [sock for sock in self.sockets if sock.ws == ws]:
-            await s.close()
-            logger.info(f"Removed socket for user {s.user['username']}")
-            self.sockets.remove(s)
+        if not ws:
+            return
+        async with self._lock:
+            sockets_to_remove = [sock for sock in self.sockets if sock and sock.ws == ws]
+            for s in sockets_to_remove:
+                self.sockets.discard(s)
+            departures_to_broadcast = []
+            channels_to_cleanup = set()
+            for s in sockets_to_remove:
+                user_uid = s.user_uid
+                if not user_uid:
+                    continue
+                is_last_connection = False
+                if user_uid in self.users:
+                    self.users[user_uid].discard(s)
+                    if len(self.users[user_uid]) == 0:
+                        del self.users[user_uid]
+                        is_last_connection = True
+                if is_last_connection:
+                    user_nick = None
+                    try:
+                        if s.user:
+                            user_nick = safe_get(s.user, "nick") or safe_get(s.user, "username")
+                    except Exception:
+                        pass
+                    if user_nick:
+                        departures_to_broadcast.append((user_uid, user_nick, s.user_color))
+                for channel_uid in list(s.subscribed_channels):
+                    channels_to_cleanup.add((channel_uid, user_uid))
+            for channel_uid, user_uid in channels_to_cleanup:
+                try:
+                    if channel_uid in self.subscriptions:
+                        self.subscriptions[channel_uid].discard(user_uid)
+                        if len(self.subscriptions[channel_uid]) == 0:
+                            del self.subscriptions[channel_uid]
+                except Exception as ex:
+                    logger.debug(f"Failed to cleanup channel subscription: {safe_str(ex)}")
+        for s in sockets_to_remove:
+            try:
+                username = safe_get(s.user, "username", "unknown") if s.user else "unknown"
+                logger.info(f"Removed socket for user {username}")
+                await s.close()
+            except Exception as ex:
+                logger.warning(f"Socket close failed: {safe_str(ex)}")
+        for user_uid, user_nick, user_color in departures_to_broadcast:
+            try:
+                await self._broadcast_presence("departed", user_uid, user_nick, user_color)
+            except Exception as ex:
+                logger.debug(f"Failed to broadcast departure: {safe_str(ex)}")
 
+    async def _broadcast_presence(self, event_type, user_uid, user_nick, user_color):
+        if not user_uid or not user_nick:
+            return
+        if not event_type or event_type not in ("arrived", "departed"):
+            return
+        try:
+            message = {
+                "event": "user_presence",
+                "data": {
+                    "type": event_type,
+                    "user_uid": user_uid,
+                    "user_nick": user_nick,
+                    "user_color": user_color,
+                    "timestamp": datetime.now().isoformat(),
+                },
+            }
+            sockets_copy = list(self.sockets)
+            send_tasks = []
+            for s in sockets_copy:
+                if not s or not s.is_connected:
+                    continue
+                if s.user_uid == user_uid:
+                    continue
+                send_tasks.append(s.send_json(message))
+            if send_tasks:
+                results = await asyncio.gather(*send_tasks, return_exceptions=True)
+                sent_count = sum(1 for r in results if r is True)
+                logger.info(f"Broadcast presence '{event_type}' for {user_nick} to {sent_count} users")
+        except Exception as ex:
+            logger.warning(f"Broadcast presence failed: {safe_str(ex)}")
+
+    async def get_connected_users(self):
+        try:
+            return list(self.users.keys())
+        except Exception:
+            return []
+
+    async def get_user_socket_count(self, user_uid):
+        if not user_uid:
+            return 0
+        try:
+            return len(self.users.get(user_uid, []))
+        except Exception:
+            return 0
+
+    async def is_user_online(self, user_uid):
+        if not user_uid:
+            return False
+        try:
+            user_sockets = self.users.get(user_uid, set())
+            return any(s.is_connected for s in user_sockets if s)
+        except Exception:
+            return False

src/snek/service/statistics.py

@@ -0,0 +1,95 @@
+# retoor <retoor@molodetz.nl>
+
+import sqlite3
+
+from snek.system.service import BaseService
+
+
+class StatisticsService(BaseService):
+    
+    def database(self):
+        db_path = self.app.db_path.split("///")[-1]
+        print(db_path)
+        conn = sqlite3.connect(db_path)
+        cursor = conn.cursor()
+
+        # Existing analysis code...
+        def get_table_columns(table_name):
+            cursor.execute(f"PRAGMA table_info({table_name})")
+            return cursor.fetchall()
+
+        tables = [
+            'http_access', 'user', 'channel', 'channel_member', 'broadcast',
+            'channel_message', 'notification', 'repository', 'test', 'drive',
+            'user_property', 'a', 'channel_attachment', 'push_registration'
+        ]
+
+        for table in tables:
+            print(f"\n--- Statistics for table: {table} ---")
+            columns = get_table_columns(table)
+            cursor.execute(f"SELECT COUNT(*) FROM {table}")
+            total_rows = cursor.fetchone()[0]
+            print(f"Total rows: {total_rows}")
+
+            for col in columns:
+                cid, name, col_type, notnull, dflt_value, pk = col
+                col_type_upper = col_type.upper()
+
+                cursor.execute(f"SELECT COUNT(DISTINCT '{name}') FROM {table}")
+                distinct_count = cursor.fetchone()[0]
+                print(f"\nColumn: {name} ({col_type})")
+                print(f"Distinct values: {distinct_count}")
+
+                if 'INT' in col_type_upper or 'BIGINT' in col_type_upper or 'FLOAT' in col_type_upper:
+                    cursor.execute(f"SELECT MIN('{name}'), MAX('{name}'), AVG('{name}') FROM {table} WHERE '{name}' IS NOT NULL")
+                    min_val, max_val, avg_val = cursor.fetchone()
+                    print(f"Min: {min_val}, Max: {max_val}, Avg: {avg_val}")
+
+                elif 'TEXT' in col_type_upper and ('date' in name.lower() or 'time' in name.lower() or 'created' in name.lower() or 'updated' in name.lower() or 'on' in name.lower()):
+                    cursor.execute(f"SELECT MIN({name}), MAX({name}) FROM {table} WHERE {name} IS NOT NULL")
+                    min_date, max_date = cursor.fetchone()
+                    print(f"Earliest: {min_date}, Latest: {max_date}")
+
+                elif 'TEXT' in col_type_upper:
+                    cursor.execute(f"SELECT LENGTH({name}) FROM {table} WHERE {name} IS NOT NULL")
+                    lengths = [len_row[0] for len_row in cursor.fetchall()]
+                    if lengths:
+                        avg_length = sum(lengths) / len(lengths)
+                        max_length = max(lengths)
+                        min_length = min(lengths)
+                        print(f"Avg length: {avg_length:.2f}, Max length: {max_length}, Min length: {min_length}")
+                    else:
+                        print("No data to compute length statistics.")
+
+        # New statistics functions
+        def get_time_series_stats(table_name, date_column):
+            cursor.execute(f"SELECT strftime('%Y-%m-%d', {date_column}) AS day, COUNT(*) FROM {table_name} GROUP BY day")
+            return cursor.fetchall()
+
+        def get_count_created(table_name, date_column):
+            cursor.execute(f"SELECT COUNT(*) FROM {table_name}")
+            return cursor.fetchone()[0]
+
+        def get_channels_per_user():
+            cursor.execute("SELECT user_uid, COUNT(*) AS channel_count FROM channel_member GROUP BY user_uid ORDER BY channel_count DESC")
+            return cursor.fetchall()
+
+        def get_online_users():
+            cursor.execute("SELECT COUNT(*) FROM user WHERE last_ping >= datetime('now', '-5 minutes')")
+            return cursor.fetchone()[0]
+
+        # Example usage of new functions
+        messages_per_day = get_time_series_stats('channel_message', 'created_at')
+        users_created = get_count_created('user', 'created_at')
+        channels_created = get_count_created('channel', 'created_at')
+        channels_per_user = get_channels_per_user()
+        online_users = get_online_users()
+
+        # Print or store these stats as needed
+        print("\nMessages per day:", messages_per_day)
+        print("Total users created:", users_created)
+        print("Total channels created:", channels_created)
+        print("Channels per user (top):", channels_per_user[:10])
+        print("Currently online users:", online_users)
+
+        conn.close()

src/snek/service/user.py

@@ -1,3 +1,5 @@
+# retoor <retoor@molodetz.nl>
+
 import pathlib
 
 from snek.system import security
@@ -12,6 +14,7 @@ class UserService(BaseService):
 
     async def search(self, query, **kwargs):
         query = query.strip().lower()
+        kwargs["deleted_at"] = None
         if not query:
             return []
         results = []
@@ -32,14 +35,14 @@ class UserService(BaseService):
             user["color"] = await self.services.util.random_light_hex_color()
         return await super().save(user)
 
-    def authenticate_sync(self,username,password):
+    def authenticate_sync(self, username, password):
         user = self.get_by_username_sync(username)
-          
+
         if not user:
             return False
         if not security.verify_sync(password, user["password"]):
             return False
-        return True 
+        return True
 
     async def authenticate(self, username, password):
         success = await self.validate_login(username, password)
@@ -61,14 +64,12 @@ class UserService(BaseService):
             return None
         return path
 
-
-
     async def get_template_path(self, user_uid):
         path = pathlib.Path(f"./drive/{user_uid}/snek/templates")
         if not path.exists():
             return None
         return path
-    
+
     def get_by_username_sync(self, username):
         user = self.mapper.db["user"].find_one(username=username, deleted_at=None)
         return dict(user)
@@ -79,7 +80,7 @@ class UserService(BaseService):
         if not folder.exists():
             try:
                 folder.mkdir(parents=True, exist_ok=True)
-            except:
+            except OSError:
                 pass
         return folder
 
@@ -88,7 +89,7 @@ class UserService(BaseService):
         if not folder.exists():
             try:
                 folder.mkdir(parents=True, exist_ok=True)
-            except:
+            except OSError:
                 pass
         return folder
 

src/snek/service/user_property.py

@@ -1,3 +1,5 @@
+# retoor <retoor@molodetz.nl>
+
 import json
 
 from snek.system.service import BaseService
@@ -15,6 +17,7 @@ class UserPropertyService(BaseService):
             },
             ["user_uid", "name"],
         )
+        self.mapper.db.commit()
 
     async def get(self, user_uid, name):
         try:

src/snek/service/util.py

@@ -1,3 +1,5 @@
+# retoor <retoor@molodetz.nl>
+
 import random
 
 from snek.system.service import BaseService

src/snek/sgit.py

@@ -1,76 +1,96 @@
+# retoor <retoor@molodetz.nl>
+
+import asyncio
+import base64
+import json
+import logging
 import os
-import aiohttp
+import shutil
+import tempfile
+
 from aiohttp import web
 
-import shutil
-import json
-import tempfile
-import asyncio
-import logging
-import base64
-import pathlib
-logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
-logger = logging.getLogger('git_server')
+logging.basicConfig(
+    level=logging.INFO, format="%(asctime)s - %(name)s - %(levelname)s - %(message)s"
+)
+logger = logging.getLogger("git_server")
+
 
 class GitApplication(web.Application):
     def __init__(self, parent=None):
-        #import git
-        #globals()['git'] = git
+        import git
+        globals()['git'] = git
         self.parent = parent
-        super().__init__(client_max_size=1024*1024*1024*5)
-        self.add_routes([
-            web.post('/create/{repo_name}', self.create_repository),
-            web.delete('/delete/{repo_name}', self.delete_repository),
-            web.get('/clone/{repo_name}', self.clone_repository),
-            web.post('/push/{repo_name}', self.push_repository),
-            web.post('/pull/{repo_name}', self.pull_repository),
-            web.get('/status/{repo_name}', self.status_repository),
-            #web.get('/list', self.list_repositories),
-            web.get('/branches/{repo_name}', self.list_branches),
-            web.post('/branches/{repo_name}', self.create_branch),
-            web.get('/log/{repo_name}', self.commit_log),
-            web.get('/file/{repo_name}/{file_path:.*}', self.file_content),
-            web.get('/{path:.+}/info/refs', self.git_smart_http),
-            web.post('/{path:.+}/git-upload-pack', self.git_smart_http),
-            web.post('/{path:.+}/git-receive-pack', self.git_smart_http),
-            web.get('/{repo_name}.git/info/refs', self.git_smart_http),
-            web.post('/{repo_name}.git/git-upload-pack', self.git_smart_http),
-            web.post('/{repo_name}.git/git-receive-pack', self.git_smart_http),
-        ])
-
+        super().__init__(client_max_size=1024 * 1024 * 1024 * 5)
+        self.add_routes(
+            [
+                web.post("/{username}/{repo_name}/create", self.create_repository),
+                web.delete("/{username}/{repo_name}/delete", self.delete_repository),
+                web.get("/{username}/{repo_name}/clone", self.clone_repository),
+                web.post("/{username}/{repo_name}/push", self.push_repository),
+                web.post("/{username}/{repo_name}/pull", self.pull_repository),
+                web.get("/{username}/{repo_name}/status", self.status_repository),
+                web.get("/{username}/{repo_name}/branches", self.list_branches),
+                web.post("/{username}/{repo_name}/branches", self.create_branch),
+                web.get("/{username}/{repo_name}/log", self.commit_log),
+                web.get("/{username}/{repo_name}/file/{file_path:.*}", self.file_content),
+                web.get("/{username}/{repo_name}.git/info/refs", self.git_smart_http),
+                web.post("/{username}/{repo_name}.git/git-upload-pack", self.git_smart_http),
+                web.post("/{username}/{repo_name}.git/git-receive-pack", self.git_smart_http),
+            ]
+        )
 
     async def check_basic_auth(self, request):
         auth_header = request.headers.get("Authorization", "")
         if not auth_header.startswith("Basic "):
-            return None,None
+            return None, None, None
         encoded_creds = auth_header.split("Basic ")[1]
         decoded_creds = base64.b64decode(encoded_creds).decode()
         username, password = decoded_creds.split(":", 1)
-        request["user"] = await self.parent.services.user.authenticate(
+        request["auth_user"] = await self.parent.services.user.authenticate(
             username=username, password=password
         )
-        if not request["user"]:
-            return None,None
-        request["repository_path"] = await self.parent.services.user.get_repository_path(
-            request["user"]["uid"]
+        if not request["auth_user"]:
+            return None, None, None
+
+        path_username = request.match_info.get("username")
+        if not path_username:
+            return None, None, None
+
+        if path_username.count("-") == 4:
+            target_user = await self.parent.services.user.get(uid=path_username)
+        else:
+            target_user = await self.parent.services.user.get(username=path_username)
+
+        if not target_user:
+            return None, None, None
+
+        request["target_user"] = target_user
+        request["repository_path"] = (
+            await self.parent.services.user.get_repository_path(target_user["uid"])
         )
 
-        return request["user"]['username'],request["repository_path"]
-
+        return request["auth_user"]["username"], target_user, request["repository_path"]
 
     @staticmethod
     def require_auth(handler):
         async def wrapped(self, request, *args, **kwargs):
-            username, repository_path = await self.check_basic_auth(request)
-            if not username or not repository_path:
-                return web.Response(status=401, headers={'WWW-Authenticate': 'Basic'}, text='Authentication required')
-            request['username'] = username
-            request['repository_path'] = repository_path
+            username, target_user, repository_path = await self.check_basic_auth(request)
+            if not username or not target_user or not repository_path:
+                return web.Response(
+                    status=401,
+                    headers={"WWW-Authenticate": "Basic"},
+                    text="Authentication required",
+                )
+            request["username"] = username
+            request["target_user"] = target_user
+            request["repository_path"] = repository_path
             return await handler(self, request, *args, **kwargs)
+
         return wrapped
 
     def repo_path(self, repository_path, repo_name):
-        return repository_path.joinpath(repo_name + '.git')
+        return repository_path.joinpath(repo_name + ".git")
 
     def check_repo_exists(self, repository_path, repo_name):
         repo_dir = self.repo_path(repository_path, repo_name)
@@ -80,17 +100,25 @@ class GitApplication(web.Application):
 
     @require_auth
     async def create_repository(self, request):
-        username = request['username']
-        repo_name = request.match_info['repo_name']
-        repository_path = request['repository_path']
-        if not repo_name or '/' in repo_name or '..' in repo_name:
+        auth_user = request["auth_user"]
+        target_user = request["target_user"]
+        repo_name = request.match_info["repo_name"]
+        repository_path = request["repository_path"]
+
+        if auth_user["uid"] != target_user["uid"]:
+            return web.Response(
+                text="Forbidden: can only create repositories in your own namespace",
+                status=403,
+            )
+
+        if not repo_name or "/" in repo_name or ".." in repo_name:
             return web.Response(text="Invalid repository name", status=400)
         repo_dir = self.repo_path(repository_path, repo_name)
         if os.path.exists(repo_dir):
             return web.Response(text="Repository already exists", status=400)
         try:
             git.Repo.init(repo_dir, bare=True)
-            logger.info(f"Created repository: {repo_name} for user {username}")
+            logger.info(f"Created repository: {repo_name} for user {auth_user['username']}")
             return web.Response(text=f"Created repository {repo_name}")
         except Exception as e:
             logger.error(f"Error creating repository {repo_name}: {str(e)}")
@@ -98,16 +126,22 @@ class GitApplication(web.Application):
 
     @require_auth
     async def delete_repository(self, request):
-        username = request['username']
-        repo_name = request.match_info['repo_name']
-        repository_path = request['repository_path']
+        auth_user = request["auth_user"]
+        target_user = request["target_user"]
+        repo_name = request.match_info["repo_name"]
+        repository_path = request["repository_path"]
+
+        if auth_user["uid"] != target_user["uid"]:
+            return web.Response(
+                text="Forbidden: can only delete your own repositories", status=403
+            )
+
         error_response = self.check_repo_exists(repository_path, repo_name)
         if error_response:
             return error_response
-        #'''
         try:
             shutil.rmtree(self.repo_path(repository_path, repo_name))
-            logger.info(f"Deleted repository: {repo_name} for user {username}")
+            logger.info(f"Deleted repository: {repo_name} for user {auth_user['username']}")
             return web.Response(text=f"Deleted repository {repo_name}")
         except Exception as e:
             logger.error(f"Error deleting repository {repo_name}: {str(e)}")
@@ -115,9 +149,20 @@ class GitApplication(web.Application):
 
     @require_auth
     async def clone_repository(self, request):
-        username = request['username']
-        repo_name = request.match_info['repo_name']
-        repository_path = request['repository_path']
+        auth_user = request["auth_user"]
+        target_user = request["target_user"]
+        repo_name = request.match_info["repo_name"]
+        repository_path = request["repository_path"]
+
+        repo = await self.parent.services.repository.get(
+            user_uid=target_user["uid"], name=repo_name
+        )
+        if not repo:
+            return web.Response(text="Repository not found", status=404)
+
+        if repo["is_private"] and auth_user["uid"] != target_user["uid"]:
+            return web.Response(text="Repository not found", status=404)
+
         error_response = self.check_repo_exists(repository_path, repo_name)
         if error_response:
             return error_response
@@ -126,15 +171,22 @@ class GitApplication(web.Application):
         response_data = {
             "repository": repo_name,
             "clone_command": f"git clone {clone_url}",
-            "clone_url": clone_url
+            "clone_url": clone_url,
         }
         return web.json_response(response_data)
 
     @require_auth
     async def push_repository(self, request):
-        username = request['username']
-        repo_name = request.match_info['repo_name']
-        repository_path = request['repository_path']
+        auth_user = request["auth_user"]
+        target_user = request["target_user"]
+        repo_name = request.match_info["repo_name"]
+        repository_path = request["repository_path"]
+
+        if auth_user["uid"] != target_user["uid"]:
+            return web.Response(
+                text="Forbidden: can only push to your own repositories", status=403
+            )
+
         error_response = self.check_repo_exists(repository_path, repo_name)
         if error_response:
             return error_response
@@ -142,34 +194,47 @@ class GitApplication(web.Application):
             data = await request.json()
         except json.JSONDecodeError:
             return web.Response(text="Invalid JSON data", status=400)
-        commit_message = data.get('commit_message', 'Update from server')
-        branch = data.get('branch', 'main')
-        changes = data.get('changes', [])
+        commit_message = data.get("commit_message", "Update from server")
+        branch = data.get("branch", "main")
+        changes = data.get("changes", [])
         if not changes:
             return web.Response(text="No changes provided", status=400)
         with tempfile.TemporaryDirectory() as temp_dir:
-            temp_repo = git.Repo.clone_from(self.repo_path(repository_path, repo_name), temp_dir)
+            temp_repo = git.Repo.clone_from(
+                self.repo_path(repository_path, repo_name), temp_dir
+            )
             for change in changes:
-                file_path = os.path.join(temp_dir, change.get('file', ''))
-                content = change.get('content', '')
+                file_path = os.path.join(temp_dir, change.get("file", ""))
+                content = change.get("content", "")
                 os.makedirs(os.path.dirname(file_path), exist_ok=True)
-                with open(file_path, 'w') as f:
+                with open(file_path, "w") as f:
                     f.write(content)
             temp_repo.git.add(A=True)
-            if not temp_repo.config_reader().has_section('user'):
-                temp_repo.config_writer().set_value("user", "name", "Git Server").release()
-                temp_repo.config_writer().set_value("user", "email", "git@server.local").release()
+            if not temp_repo.config_reader().has_section("user"):
+                temp_repo.config_writer().set_value(
+                    "user", "name", "Git Server"
+                ).release()
+                temp_repo.config_writer().set_value(
+                    "user", "email", "git@server.local"
+                ).release()
             temp_repo.index.commit(commit_message)
-            origin = temp_repo.remote('origin')
+            origin = temp_repo.remote("origin")
             origin.push(refspec=f"{branch}:{branch}")
-        logger.info(f"Pushed to repository: {repo_name} for user {username}")
+        logger.info(f"Pushed to repository: {repo_name} for user {auth_user['username']}")
         return web.Response(text=f"Successfully pushed changes to {repo_name}")
 
     @require_auth
     async def pull_repository(self, request):
-        username = request['username']
-        repo_name = request.match_info['repo_name']
-        repository_path = request['repository_path']
+        auth_user = request["auth_user"]
+        target_user = request["target_user"]
+        repo_name = request.match_info["repo_name"]
+        repository_path = request["repository_path"]
+
+        if auth_user["uid"] != target_user["uid"]:
+            return web.Response(
+                text="Forbidden: can only pull to your own repositories", status=403
+            )
+
         error_response = self.check_repo_exists(repository_path, repo_name)
         if error_response:
             return error_response
@@ -177,13 +242,15 @@ class GitApplication(web.Application):
             data = await request.json()
         except json.JSONDecodeError:
             data = {}
-        remote_url = data.get('remote_url')
-        branch = data.get('branch', 'main')
+        remote_url = data.get("remote_url")
+        branch = data.get("branch", "main")
         if not remote_url:
             return web.Response(text="Remote URL is required", status=400)
         with tempfile.TemporaryDirectory() as temp_dir:
             try:
-                local_repo = git.Repo.clone_from(self.repo_path(repository_path, repo_name), temp_dir)
+                local_repo = git.Repo.clone_from(
+                    self.repo_path(repository_path, repo_name), temp_dir
+                )
                 remote_name = "pull_source"
                 try:
                     remote = local_repo.create_remote(remote_name, remote_url)
@@ -192,38 +259,57 @@ class GitApplication(web.Application):
                     remote.set_url(remote_url)
                 remote.fetch()
                 local_repo.git.merge(f"{remote_name}/{branch}")
-                origin = local_repo.remote('origin')
+                origin = local_repo.remote("origin")
                 origin.push()
-                logger.info(f"Pulled to repository {repo_name} from {remote_url} for user {username}")
-                return web.Response(text=f"Successfully pulled changes from {remote_url} to {repo_name}")
+                logger.info(
+                    f"Pulled to repository {repo_name} from {remote_url} for user {auth_user['username']}"
+                )
+                return web.Response(
+                    text=f"Successfully pulled changes from {remote_url} to {repo_name}"
+                )
             except Exception as e:
                 logger.error(f"Error pulling to {repo_name}: {str(e)}")
                 return web.Response(text=f"Error pulling changes: {str(e)}", status=500)
 
     @require_auth
     async def status_repository(self, request):
-        username = request['username']
-        repo_name = request.match_info['repo_name']
-        repository_path = request['repository_path']
+        auth_user = request["auth_user"]
+        target_user = request["target_user"]
+        repo_name = request.match_info["repo_name"]
+        repository_path = request["repository_path"]
+
+        repo = await self.parent.services.repository.get(
+            user_uid=target_user["uid"], name=repo_name
+        )
+        if not repo:
+            return web.Response(text="Repository not found", status=404)
+
+        if repo["is_private"] and auth_user["uid"] != target_user["uid"]:
+            return web.Response(text="Repository not found", status=404)
+
         error_response = self.check_repo_exists(repository_path, repo_name)
         if error_response:
             return error_response
         with tempfile.TemporaryDirectory() as temp_dir:
             try:
-                temp_repo = git.Repo.clone_from(self.repo_path(repository_path, repo_name), temp_dir)
+                temp_repo = git.Repo.clone_from(
+                    self.repo_path(repository_path, repo_name), temp_dir
+                )
                 branches = [b.name for b in temp_repo.branches]
                 active_branch = temp_repo.active_branch.name
                 commits = []
                 for commit in list(temp_repo.iter_commits(max_count=5)):
-                    commits.append({
-                        "id": commit.hexsha,
-                        "author": f"{commit.author.name} <{commit.author.email}>",
-                        "date": commit.committed_datetime.isoformat(),
-                        "message": commit.message
-                    })
+                    commits.append(
+                        {
+                            "id": commit.hexsha,
+                            "author": f"{commit.author.name} <{commit.author.email}>",
+                            "date": commit.committed_datetime.isoformat(),
+                            "message": commit.message,
+                        }
+                    )
                 files = []
                 for root, dirs, filenames in os.walk(temp_dir):
-                    if '.git' in root:
+                    if ".git" in root:
                         continue
                     for filename in filenames:
                         full_path = os.path.join(root, filename)
@@ -234,50 +320,77 @@ class GitApplication(web.Application):
                     "branches": branches,
                     "active_branch": active_branch,
                     "recent_commits": commits,
-                    "files": files
+                    "files": files,
                 }
                 return web.json_response(status_info)
             except Exception as e:
                 logger.error(f"Error getting status for {repo_name}: {str(e)}")
-                return web.Response(text=f"Error getting repository status: {str(e)}", status=500)
+                return web.Response(
+                    text=f"Error getting repository status: {str(e)}", status=500
+                )
 
     @require_auth
     async def list_repositories(self, request):
-        username = request['username']
+        request["username"]
         try:
             repos = []
             user_dir = self.REPO_DIR
             if os.path.exists(user_dir):
                 for item in os.listdir(user_dir):
                     item_path = os.path.join(user_dir, item)
-                    if os.path.isdir(item_path) and item.endswith('.git'):
+                    if os.path.isdir(item_path) and item.endswith(".git"):
                         repos.append(item[:-4])
-            if request.query.get('format') == 'json':
+            if request.query.get("format") == "json":
                 return web.json_response({"repositories": repos})
             else:
-                return web.Response(text="\n".join(repos) if repos else "No repositories found")
+                return web.Response(
+                    text="\n".join(repos) if repos else "No repositories found"
+                )
         except Exception as e:
             logger.error(f"Error listing repositories: {str(e)}")
-            return web.Response(text=f"Error listing repositories: {str(e)}", status=500)
+            return web.Response(
+                text=f"Error listing repositories: {str(e)}", status=500
+            )
 
     @require_auth
     async def list_branches(self, request):
-        username = request['username']
-        repo_name = request.match_info['repo_name']
-        repository_path = request['repository_path']
+        auth_user = request["auth_user"]
+        target_user = request["target_user"]
+        repo_name = request.match_info["repo_name"]
+        repository_path = request["repository_path"]
+
+        repo = await self.parent.services.repository.get(
+            user_uid=target_user["uid"], name=repo_name
+        )
+        if not repo:
+            return web.Response(text="Repository not found", status=404)
+
+        if repo["is_private"] and auth_user["uid"] != target_user["uid"]:
+            return web.Response(text="Repository not found", status=404)
+
         error_response = self.check_repo_exists(repository_path, repo_name)
         if error_response:
             return error_response
         with tempfile.TemporaryDirectory() as temp_dir:
-            temp_repo = git.Repo.clone_from(self.repo_path(repository_path, repo_name), temp_dir)
+            temp_repo = git.Repo.clone_from(
+                self.repo_path(repository_path, repo_name), temp_dir
+            )
             branches = [b.name for b in temp_repo.branches]
             return web.json_response({"branches": branches})
 
     @require_auth
     async def create_branch(self, request):
-        username = request['username']
-        repo_name = request.match_info['repo_name']
-        repository_path = request['repository_path']
+        auth_user = request["auth_user"]
+        target_user = request["target_user"]
+        repo_name = request.match_info["repo_name"]
+        repository_path = request["repository_path"]
+
+        if auth_user["uid"] != target_user["uid"]:
+            return web.Response(
+                text="Forbidden: can only create branches in your own repositories",
+                status=403,
+            )
+
         error_response = self.check_repo_exists(repository_path, repo_name)
         if error_response:
             return error_response
@@ -285,145 +398,207 @@ class GitApplication(web.Application):
             data = await request.json()
         except json.JSONDecodeError:
             return web.Response(text="Invalid JSON data", status=400)
-        branch_name = data.get('branch_name')
-        start_point = data.get('start_point', 'HEAD')
+        branch_name = data.get("branch_name")
+        start_point = data.get("start_point", "HEAD")
         if not branch_name:
             return web.Response(text="Branch name is required", status=400)
         with tempfile.TemporaryDirectory() as temp_dir:
             try:
-                temp_repo = git.Repo.clone_from(self.repo_path(repository_path, repo_name), temp_dir)
+                temp_repo = git.Repo.clone_from(
+                    self.repo_path(repository_path, repo_name), temp_dir
+                )
                 temp_repo.git.branch(branch_name, start_point)
-                temp_repo.git.push('origin', branch_name)
-                logger.info(f"Created branch {branch_name} in repository {repo_name} for user {username}")
+                temp_repo.git.push("origin", branch_name)
+                logger.info(
+                    f"Created branch {branch_name} in repository {repo_name} for user {auth_user['username']}"
+                )
                 return web.Response(text=f"Created branch {branch_name}")
             except Exception as e:
-                logger.error(f"Error creating branch {branch_name} in {repo_name}: {str(e)}")
+                logger.error(
+                    f"Error creating branch {branch_name} in {repo_name}: {str(e)}"
+                )
                 return web.Response(text=f"Error creating branch: {str(e)}", status=500)
 
     @require_auth
     async def commit_log(self, request):
-        username = request['username']
-        repo_name = request.match_info['repo_name']
-        repository_path = request['repository_path']
+        auth_user = request["auth_user"]
+        target_user = request["target_user"]
+        repo_name = request.match_info["repo_name"]
+        repository_path = request["repository_path"]
+
+        repo = await self.parent.services.repository.get(
+            user_uid=target_user["uid"], name=repo_name
+        )
+        if not repo:
+            return web.Response(text="Repository not found", status=404)
+
+        if repo["is_private"] and auth_user["uid"] != target_user["uid"]:
+            return web.Response(text="Repository not found", status=404)
+
         error_response = self.check_repo_exists(repository_path, repo_name)
         if error_response:
             return error_response
         try:
-            limit = int(request.query.get('limit', 10))
-            branch = request.query.get('branch', 'main')
+            limit = int(request.query.get("limit", 10))
+            branch = request.query.get("branch", "main")
         except ValueError:
             return web.Response(text="Invalid limit parameter", status=400)
         with tempfile.TemporaryDirectory() as temp_dir:
             try:
-                temp_repo = git.Repo.clone_from(self.repo_path(repository_path, repo_name), temp_dir)
+                temp_repo = git.Repo.clone_from(
+                    self.repo_path(repository_path, repo_name), temp_dir
+                )
                 commits = []
                 try:
                     for commit in list(temp_repo.iter_commits(branch, max_count=limit)):
-                        commits.append({
-                            "id": commit.hexsha,
-                            "short_id": commit.hexsha[:7],
-                            "author": f"{commit.author.name} <{commit.author.email}>",
-                            "date": commit.committed_datetime.isoformat(),
-                            "message": commit.message.strip()
-                        })
+                        commits.append(
+                            {
+                                "id": commit.hexsha,
+                                "short_id": commit.hexsha[:7],
+                                "author": f"{commit.author.name} <{commit.author.email}>",
+                                "date": commit.committed_datetime.isoformat(),
+                                "message": commit.message.strip(),
+                            }
+                        )
                 except git.GitCommandError as e:
                     if "unknown revision or path" in str(e):
                         commits = []
                     else:
                         raise
-                return web.json_response({
-                    "repository": repo_name,
-                    "branch": branch,
-                    "commits": commits
-                })
+                return web.json_response(
+                    {"repository": repo_name, "branch": branch, "commits": commits}
+                )
             except Exception as e:
                 logger.error(f"Error getting commit log for {repo_name}: {str(e)}")
-                return web.Response(text=f"Error getting commit log: {str(e)}", status=500)
+                return web.Response(
+                    text=f"Error getting commit log: {str(e)}", status=500
+                )
 
     @require_auth
     async def file_content(self, request):
-        username = request['username']
-        repo_name = request.match_info['repo_name']
-        file_path = request.match_info.get('file_path', '')
-        branch = request.query.get('branch', 'main')
-        repository_path = request['repository_path']
+        auth_user = request["auth_user"]
+        target_user = request["target_user"]
+        repo_name = request.match_info["repo_name"]
+        file_path = request.match_info.get("file_path", "")
+        branch = request.query.get("branch", "main")
+        repository_path = request["repository_path"]
+
+        repo = await self.parent.services.repository.get(
+            user_uid=target_user["uid"], name=repo_name
+        )
+        if not repo:
+            return web.Response(text="Repository not found", status=404)
+
+        if repo["is_private"] and auth_user["uid"] != target_user["uid"]:
+            return web.Response(text="Repository not found", status=404)
+
         error_response = self.check_repo_exists(repository_path, repo_name)
         if error_response:
             return error_response
         with tempfile.TemporaryDirectory() as temp_dir:
             try:
-                temp_repo = git.Repo.clone_from(self.repo_path(repository_path, repo_name), temp_dir)
+                temp_repo = git.Repo.clone_from(
+                    self.repo_path(repository_path, repo_name), temp_dir
+                )
                 try:
                     temp_repo.git.checkout(branch)
                 except git.GitCommandError:
                     return web.Response(text=f"Branch '{branch}' not found", status=404)
                 file_full_path = os.path.join(temp_dir, file_path)
                 if not os.path.exists(file_full_path):
-                    return web.Response(text=f"File '{file_path}' not found", status=404)
+                    return web.Response(
+                        text=f"File '{file_path}' not found", status=404
+                    )
                 if os.path.isdir(file_full_path):
                     files = os.listdir(file_full_path)
-                    return web.json_response({
-                        "repository": repo_name,
-                        "path": file_path,
-                        "type": "directory",
-                        "contents": files
-                    })
+                    return web.json_response(
+                        {
+                            "repository": repo_name,
+                            "path": file_path,
+                            "type": "directory",
+                            "contents": files,
+                        }
+                    )
                 else:
                     try:
-                        with open(file_full_path, 'r') as f:
+                        with open(file_full_path) as f:
                             content = f.read()
                         return web.Response(text=content)
                     except UnicodeDecodeError:
-                        return web.Response(text=f"Cannot display binary file content for '{file_path}'", status=400)
+                        return web.Response(
+                            text=f"Cannot display binary file content for '{file_path}'",
+                            status=400,
+                        )
             except Exception as e:
                 logger.error(f"Error getting file content from {repo_name}: {str(e)}")
-                return web.Response(text=f"Error getting file content: {str(e)}", status=500)
+                return web.Response(
+                    text=f"Error getting file content: {str(e)}", status=500
+                )
 
     @require_auth
     async def git_smart_http(self, request):
-        username = request['username']
-        repository_path = request['repository_path']
+        auth_user = request["auth_user"]
+        target_user = request["target_user"]
+        repository_path = request["repository_path"]
+        repo_name = request.match_info.get("repo_name")
+        path_username = request.match_info.get("username")
         path = request.path
+
+        repo = await self.parent.services.repository.get(
+            user_uid=target_user["uid"], name=repo_name
+        )
+        if not repo:
+            return web.Response(text="Repository not found", status=404)
+
+        is_owner = auth_user["uid"] == target_user["uid"]
+        is_write_operation = "/git-receive-pack" in path
+
+        if is_write_operation and not is_owner:
+            logger.warning(
+                f"Push denied: {auth_user['username']} attempted to push to {path_username}/{repo_name}"
+            )
+            return web.Response(
+                text="Push denied: only repository owner can push", status=403
+            )
+
+        if not is_owner and repo["is_private"]:
+            logger.warning(
+                f"Access denied: {auth_user['username']} attempted to access private repo {path_username}/{repo_name}"
+            )
+            return web.Response(text="Repository not found", status=404)
+
         async def get_repository_path():
-            req_path = path.lstrip('/')
-            if req_path.endswith('/info/refs'):
-                repo_name = req_path[:-len('/info/refs')]
-            elif req_path.endswith('/git-upload-pack'):
-                repo_name = req_path[:-len('/git-upload-pack')]
-            elif req_path.endswith('/git-receive-pack'):
-                repo_name = req_path[:-len('/git-receive-pack')]
-            else:
-                repo_name = req_path
-            if repo_name.endswith('.git'):
-                repo_name = repo_name[:-4]
-            repo_name = repo_name[4:]
             repo_dir = repository_path.joinpath(repo_name + ".git")
-            logger.info(f"Resolved repo path: {repo_dir}")
-            return repo_dir 
+            logger.info(
+                f"Resolved repo path: {repo_dir} for user: {path_username}, repo: {repo_name}, "
+                f"auth: {auth_user['username']}, owner: {is_owner}, write: {is_write_operation}"
+            )
+            return repo_dir
+
         async def handle_info_refs(service):
             repo_path = await get_repository_path()
-        
+
             logger.info(f"handle_info_refs: {repo_path}")
             if not os.path.exists(repo_path):
                 return web.Response(text="Repository not found", status=404)
-            cmd = [service, '--stateless-rpc', '--advertise-refs', str(repo_path)]
+            cmd = [service, "--stateless-rpc", "--advertise-refs", str(repo_path)]
             try:
                 process = await asyncio.create_subprocess_exec(
-                    *cmd,
-                    stdout=asyncio.subprocess.PIPE,
-                    stderr=asyncio.subprocess.PIPE
+                    *cmd, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE
                 )
                 stdout, stderr = await process.communicate()
                 if process.returncode != 0:
                     logger.error(f"Git command failed: {stderr.decode()}")
-                    return web.Response(text=f"Git error: {stderr.decode()}", status=500)
+                    return web.Response(
+                        text=f"Git error: {stderr.decode()}", status=500
+                    )
                 response = web.StreamResponse(
                     status=200,
-                    reason='OK',
+                    reason="OK",
                     headers={
-                        'Content-Type': f'application/x-{service}-advertisement',
-                        'Cache-Control': 'no-cache'
-                    }
+                        "Content-Type": f"application/x-{service}-advertisement",
+                        "Cache-Control": "no-cache",
+                    },
                 )
                 await response.prepare(request)
                 packet = f"# service={service}\n"
@@ -435,48 +610,58 @@ class GitApplication(web.Application):
             except Exception as e:
                 logger.error(f"Error handling info/refs: {str(e)}")
                 return web.Response(text=f"Server error: {str(e)}", status=500)
+
         async def handle_service_rpc(service):
             repo_path = await get_repository_path()
             logger.info(f"handle_service_rpc: {repo_path}")
             if not os.path.exists(repo_path):
                 return web.Response(text="Repository not found", status=404)
-            if not request.headers.get('Content-Type') == f'application/x-{service}-request':
+            if (
+                not request.headers.get("Content-Type")
+                == f"application/x-{service}-request"
+            ):
                 return web.Response(text="Invalid Content-Type", status=403)
             body = await request.read()
-            cmd = [service, '--stateless-rpc', str(repo_path)]
+            cmd = [service, "--stateless-rpc", str(repo_path)]
             try:
                 process = await asyncio.create_subprocess_exec(
                     *cmd,
                     stdin=asyncio.subprocess.PIPE,
                     stdout=asyncio.subprocess.PIPE,
-                    stderr=asyncio.subprocess.PIPE
+                    stderr=asyncio.subprocess.PIPE,
                 )
                 stdout, stderr = await process.communicate(input=body)
                 if process.returncode != 0:
                     logger.error(f"Git command failed: {stderr.decode()}")
-                    return web.Response(text=f"Git error: {stderr.decode()}", status=500)
+                    return web.Response(
+                        text=f"Git error: {stderr.decode()}", status=500
+                    )
                 return web.Response(
-                    body=stdout,
-                    content_type=f'application/x-{service}-result'
+                    body=stdout, content_type=f"application/x-{service}-result"
                 )
             except Exception as e:
                 logger.error(f"Error handling service RPC: {str(e)}")
                 return web.Response(text=f"Server error: {str(e)}", status=500)
-        if request.method == 'GET' and path.endswith('/info/refs'):
-            service = request.query.get('service')
-            if service in ('git-upload-pack', 'git-receive-pack'):
+
+        if request.method == "GET" and path.endswith("/info/refs"):
+            service = request.query.get("service")
+            if service in ("git-upload-pack", "git-receive-pack"):
                 return await handle_info_refs(service)
             else:
-                return web.Response(text="Smart HTTP requires service parameter", status=400)
-        elif request.method == 'POST' and '/git-upload-pack' in path:
-            return await handle_service_rpc('git-upload-pack')
-        elif request.method == 'POST' and '/git-receive-pack' in path:
-            return await handle_service_rpc('git-receive-pack')
+                return web.Response(
+                    text="Smart HTTP requires service parameter", status=400
+                )
+        elif request.method == "POST" and "/git-upload-pack" in path:
+            return await handle_service_rpc("git-upload-pack")
+        elif request.method == "POST" and "/git-receive-pack" in path:
+            return await handle_service_rpc("git-receive-pack")
         return web.Response(text="Not found", status=404)
 
-if __name__ == '__main__':
+
+if __name__ == "__main__":
     try:
         import uvloop
+
         asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
         logger.info("Using uvloop for improved performance")
     except ImportError:

src/snek/shell.py

@@ -0,0 +1,21 @@
+# retoor <retoor@molodetz.nl>
+
+from snek.app import Application
+from IPython import start_ipython
+
+class Shell:
+    def __init__(self,db_path):
+        self.app = Application(db_path=f"sqlite:///{db_path}")
+   
+    async def maintenance(self):
+        await self.app.services.container.maintenance()
+        await self.app.services.channel_message.maintenance()
+
+
+
+    def run(self):
+        ns = {
+            "app": self.app,
+            "maintenance": self.maintenance
+        }
+        start_ipython(argv=[], user_ns=ns)

src/snek/snode.py

@@ -1,29 +1,31 @@
+# retoor <retoor@molodetz.nl>
+
 import aiohttp
 
 ENABLED = False
 
-import aiohttp
 import asyncio
-from aiohttp import web
-
+import json
 import sqlite3
 
-import dataset
+import aiohttp
+from aiohttp import web
 from sqlalchemy import event
 from sqlalchemy.engine import Engine
 
-import json 
-
 queue = asyncio.Queue()
 
+
 class State:
-    do_not_sync = False 
+    do_not_sync = False
+
 
 async def sync_service(app):
     if not ENABLED:
-        return  
+        return
     session = aiohttp.ClientSession()
-    async with session.ws_connect('http://localhost:3131/ws') as ws:
+    async with session.ws_connect("http://localhost:3131/ws") as ws:
+
         async def receive():
 
             queries_synced = 0
@@ -31,7 +33,7 @@ async def sync_service(app):
                 if msg.type == aiohttp.WSMsgType.TEXT:
                     try:
                         data = json.loads(msg.data)
-                        State.do_not_sync = True 
+                        State.do_not_sync = True
                         app.db.execute(*data)
                         app.db.commit()
                         State.do_not_sync = False
@@ -41,21 +43,25 @@ async def sync_service(app):
                         await app.services.socket.broadcast_event()
                     except Exception as e:
                         print(e)
-                    pass 
-                    #print(f"Received: {msg.data}")
+                    pass
+                    # print(f"Received: {msg.data}")
                 elif msg.type == aiohttp.WSMsgType.ERROR:
                     break
+
         async def write():
             while True:
                 msg = await queue.get()
-                await ws.send_str(json.dumps(msg,default=str))
+                await ws.send_str(json.dumps(msg, default=str))
                 queue.task_done()
 
         await asyncio.gather(receive(), write())
 
     await session.close()
 
+
 queries_queued = 0
+
+
 # Attach a listener to log all executed statements
 @event.listens_for(Engine, "before_cursor_execute")
 def before_cursor_execute(conn, cursor, statement, parameters, context, executemany):
@@ -63,7 +69,7 @@ def before_cursor_execute(conn, cursor, statement, parameters, context, executem
         return
     global queries_queued
     if State.do_not_sync:
-        print(statement,parameters)
+        print(statement, parameters)
         return
     if statement.startswith("SELECT"):
         return
@@ -71,17 +77,18 @@ def before_cursor_execute(conn, cursor, statement, parameters, context, executem
     queries_queued += 1
     print("Queries queued: " + str(queries_queued))
 
+
 async def websocket_handler(request):
-    queries_broadcasted = 0 
+    queries_broadcasted = 0
     ws = web.WebSocketResponse()
     await ws.prepare(request)
-    request.app['websockets'].append(ws)
+    request.app["websockets"].append(ws)
     async for msg in ws:
         if msg.type == aiohttp.WSMsgType.TEXT:
-            for client in request.app['websockets']:
+            for client in request.app["websockets"]:
                 if client != ws:
                     await client.send_str(msg.data)
-            cursor = request.app['db'].cursor()
+            cursor = request.app["db"].cursor()
             data = json.loads(msg.data)
             queries_broadcasted += 1
 
@@ -89,28 +96,32 @@ async def websocket_handler(request):
             cursor.close()
             print("Queries broadcasted: " + str(queries_broadcasted))
         elif msg.type == aiohttp.WSMsgType.ERROR:
-            print(f'WebSocket connection closed with exception {ws.exception()}')
+            print(f"WebSocket connection closed with exception {ws.exception()}")
 
-    request.app['websockets'].remove(ws)
+    request.app["websockets"].remove(ws)
     return ws
 
-app = web.Application()
-app['websockets'] = []
 
-app.router.add_get('/ws', websocket_handler)
+app = web.Application()
+app["websockets"] = []
+
+app.router.add_get("/ws", websocket_handler)
+
 
 async def on_startup(app):
-    app['db'] = sqlite3.connect('snek.db')
+    app["db"] = sqlite3.connect("snek.db")
     print("Server starting...")
 
+
 async def on_cleanup(app):
-    for ws in app['websockets']:
+    for ws in app["websockets"]:
         await ws.close()
-    app['db'].close()
+    app["db"].close()
+
 
 app.on_startup.append(on_startup)
 app.on_cleanup.append(on_cleanup)
 
 
-if __name__ == '__main__':
-    web.run_app(app, host='127.0.0.1', port=3131)
+if __name__ == "__main__":
+    web.run_app(app, host="127.0.0.1", port=3131)

src/snek/sssh.py

@@ -1,25 +1,32 @@
-import asyncssh
+# retoor <retoor@molodetz.nl>
+
 import logging
 from pathlib import Path
 
-global _app 
+import asyncssh
+
+global _app
+
 
 def set_app(app):
     global _app
-    _app = app 
+    _app = app
+
 
 def get_app():
     return _app
 
+
 logger = logging.getLogger(__name__)
 
 roots = {}
 
+
 class SFTPServer(asyncssh.SFTPServer):
-   
+
     def __init__(self, chan: asyncssh.SSHServerChannel):
         self.root = get_app().services.user.get_home_folder_by_username(
-            chan.get_extra_info('username')
+            chan.get_extra_info("username")
         )
         self.root.mkdir(exist_ok=True)
         self.root = str(self.root)
@@ -30,20 +37,22 @@ class SFTPServer(asyncssh.SFTPServer):
         logger.debug(f"Mapping client path {path} to {mapped_path}")
         return str(mapped_path).encode()
 
+
 class SSHServer(asyncssh.SSHServer):
     def password_auth_supported(self):
         return True
 
     def validate_password(self, username, password):
         logger.debug(f"Validating credentials for user {username}")
-        result = get_app().services.user.authenticate_sync(username,password)
+        result = get_app().services.user.authenticate_sync(username, password)
         logger.info(f"Validating credentials for user {username}: {result}")
         return result
 
-async def start_ssh_server(app,host,port):
-    set_app(app) 
+
+async def start_ssh_server(app, host, port):
+    set_app(app)
     logger.info("Starting SFTP server setup")
-    
+
     host_key_path = Path("drive") / ".ssh" / "sftp_server_key"
     host_key_path.parent.mkdir(exist_ok=True, parents=True)
     try:
@@ -63,14 +72,12 @@ async def start_ssh_server(app,host,port):
         x = await asyncssh.listen(
             host=host,
             port=port,
-            #process_factory=handle_client,
+            # process_factory=handle_client,
             server_host_keys=[key],
             server_factory=SSHServer,
-            sftp_factory=SFTPServer
+            sftp_factory=SFTPServer,
         )
         return x
-    except Exception as e:
+    except Exception:
         logger.warning(f"Failed to start SFTP server. Already running.")
-        pass 
-
-
+        pass

src/snek/static/app.js

@@ -1,16 +1,10 @@
-// Written by retoor@molodetz.nl
-
-// This project implements a client-server communication system using WebSockets and REST APIs.
-// It features a chat system, a notification sound system, and interaction with server endpoints.
-
-// No additional imports were used beyond standard JavaScript objects and constructors.
-
-// MIT License
+// retoor <retoor@molodetz.nl>
 
 import { Schedule } from "./schedule.js";
 import { EventHandler } from "./event-handler.js";
 import { Socket } from "./socket.js";
-
+import { Njet } from "./njet.js";
+import { PresenceNotification } from "./presence-notification.js";
 export class RESTClient {
   debug = false;
 
@@ -65,7 +59,7 @@ export class Chat extends EventHandler {
     }
     return new Promise((resolve) => {
       this._waitConnect = resolve;
-      console.debug("Connecting..");
+      //console.debug("Connecting..");
         
       try {
         this._socket = new WebSocket(this._url);
@@ -142,7 +136,7 @@ export class NotificationAudio {
       new Audio(this.sounds[soundIndex])
         .play()
         .then(() => {
-          console.debug("Gave sound notification");
+          //console.debug("Gave sound notification");
         })
         .catch((error) => {
           console.error("Notification failed:", error);
@@ -160,9 +154,10 @@ export class App extends EventHandler {
   typeLock = null;
   typeListener = null;
   typeEventChannelUid = null;
-  _debug = false 
+  _debug = false;
+  presenceNotification = null; 
    async set_typing(channel_uid) {
-    this.typeEventChannel_uid = channel_uid;
+    this.typeEventChannelUid = channel_uid;
   }
   debug() {
         this._debug = !this._debug;
@@ -174,9 +169,8 @@ export class App extends EventHandler {
     await this.rpc.ping(...args);
     this.is_pinging = false;
   }
-
   async forcePing(...arg) {
-    await this.rpc.ping(...args);
+    await this.rpc.ping(...arg);
   }
   starField = null
   constructor() {
@@ -184,6 +178,7 @@ export class App extends EventHandler {
     this.ws = new Socket();
     this.rpc = this.ws.client;
     this.audio = new NotificationAudio(500);
+    this.presenceNotification = new PresenceNotification(this.ws);
     this.is_pinging = false;
     this.ping_interval = setInterval(() => {
       this.ping("active");
@@ -194,7 +189,7 @@ export class App extends EventHandler {
         this.rpc.set_typing(this.typeEventChannelUid);
         this.typeEventChannelUid = null;
       }
-    });
+    }, 1000);
 
     const me = this;
     this.ws.addEventListener("connected", (data) => {

src/snek/static/base.css

@@ -9,6 +9,10 @@ html {
   height: 100%;
 }
 
+.hidden {
+  display: none;
+}
+
 .gallery {
   padding: 50px;
   height: auto;
@@ -62,11 +66,13 @@ header .logo {
 }
 
 header nav a {
-  color: #aaa;
+  color: #888;
   text-decoration: none;
   margin-left: 15px;
   font-size: 1em;
-  transition: color 0.3s;
+  padding: 4px 8px;
+  border-radius: 4px;
+  transition: all 0.2s ease;
 }
 
 .no-select {
@@ -78,6 +84,7 @@ header nav a {
 
 header nav a:hover {
   color: #fff;
+  background-color: rgba(255, 255, 255, 0.05);
 }
 
 a {
@@ -140,7 +147,7 @@ footer {
 
 .chat-messages {
   display: flex;
-  flex-direction: column;
+  flex-direction: column-reverse;
 }
 
 .container {
@@ -221,6 +228,55 @@ footer {
   hyphens: auto;
 }
 
+.message-content .spoiler {
+  background-color: rgba(255, 255, 255, 0.1);
+  /*color: transparent;*/
+  cursor: pointer;
+  border-radius: 0.5rem;
+  padding: 0.5rem;
+  position: relative;
+  height: 2.5rem;
+  overflow: hidden;
+  max-width: unset;
+}
+
+.message-content .spoiler * {
+  opacity: 0;
+  pointer-events: none;
+  visibility: hidden;
+}
+
+.spoiler:hover, .spoiler:focus, .spoiler:focus-within, .spoiler:active {
+  /*color: #e6e6e6;*/
+  /*transition: color 0.3s ease-in;*/
+  height: unset;
+  overflow: unset;
+}
+
+@keyframes delay-pointer-events {
+  0% {
+    visibility: hidden;
+  }
+  50% {
+    visibility: hidden;
+  }
+  100% {
+    visibility: visible;
+  }
+}
+
+.spoiler:hover * {
+  animation: unset;
+}
+
+.spoiler:hover *, .spoiler:focus *, .spoiler:focus-within *, .spoiler:active * {
+  opacity: 1;
+  transition: opacity 0.3s ease-in;
+  pointer-events: auto;
+  visibility: visible;
+  animation: delay-pointer-events 0.2s linear;
+}
+
 .message-content {
   max-width: 100%;
 }
@@ -315,7 +371,7 @@ input[type="text"], .chat-input textarea {
   }
 }
 
-.message:has(+ .message.switch-user), .message:has(+ .message.long-time), .message:not(:has(+ .message)) {
+.message.switch-user + .message, .message.long-time + .message, .message-list-bottom + .message{
   .time {
     display: block;
     opacity: 1;
@@ -354,34 +410,69 @@ a {
   width: 250px;
   padding-left: 20px;
   padding-right: 20px;
-  padding-top: 10px;
+  padding-top: 20px;
   overflow-y: auto;
   grid-area: sidebar;
 }
 
 .sidebar h2 {
   color: #f05a28;
+  font-size: 0.75em;
+  font-weight: 600;
+  text-transform: uppercase;
+  letter-spacing: 0.5px;
+  margin-bottom: 10px;
+  margin-top: 20px;
+}
+
+.sidebar h2:first-child {
+  margin-top: 0;
+}
+
+.sidebar-add-btn {
+  color: #f05a28;
+  text-decoration: none;
   font-size: 1.2em;
-  margin-bottom: 20px;
+  font-weight: bold;
+  margin-left: 8px;
+  opacity: 0.7;
+  transition: opacity 0.2s;
+}
+
+.sidebar-add-btn:hover {
+  opacity: 1;
 }
 
 .sidebar ul {
   list-style: none;
-}
-
-.sidebar ul li {
   margin-bottom: 15px;
 }
 
+.sidebar ul li {
+  margin-bottom: 4px;
+}
+
 .sidebar ul li a {
-  color: #ccc;
+  color: #888;
   text-decoration: none;
-  font-size: 1em;
-  transition: color 0.3s;
+  font-family: 'Courier New', monospace;
+  font-size: 0.9em;
+  display: block;
+  padding: 6px 10px;
+  border-radius: 4px;
+  border-left: 2px solid transparent;
+  transition: all 0.2s ease;
 }
 
 .sidebar ul li a:hover {
-  color: #fff;
+  color: #e6e6e6;
+  background-color: #1a1a1a;
+  border-left-color: #444;
+}
+
+.sidebar ul li a.active {
+  color: #f05a28;
+  border-left-color: #f05a28;
 }
 
 @keyframes glow {
@@ -406,31 +497,40 @@ a {
   header {
     top: 0;
     left: 0;
-    text-overflow: ellipsis;
     width: 100%;
-    display: flex;
-    flex-direction: column;
+    display: flex !important;
+    flex-direction: row !important;
+    flex-wrap: nowrap !important;
+    align-items: center !important;
+    justify-content: space-between !important;
+    gap: 8px;
+    padding: 10px 12px;
+  }
 
-    .logo {
-      display: block;
-      flex: 1;
-      text-overflow: ellipsis;
-      white-space: nowrap;
-      overflow: hidden;
+  header > * {
+    display: inline-block !important;
+    vertical-align: middle;
+  }
 
-      h2 {
-        font-size: 14px;
-      }
+  header .logo {
+    flex: 1 1 auto !important;
+    min-width: 0;
+    text-overflow: ellipsis;
+    white-space: nowrap;
+    overflow: hidden;
+  }
 
-      text-align: center;
-    }
+  header .logo h2 {
+    font-size: 14px;
+  }
 
-    nav {
-      text-align: right;
-      flex: 1;
-      display: block;
-      width: 100%;
-    }
+  header nav {
+    display: none !important;
+  }
+
+  header nav-menu,
+  header channel-menu {
+    flex: 0 0 auto !important;
   }
 
   /*
@@ -496,12 +596,22 @@ dialog .dialog-actions {
 }
 
 dialog .dialog-button {
-  padding: 8px 16px;
-  font-size: 0.95rem;
-  border-radius: 8px;
-  border: none;
+  padding: 10px 20px;
+  font-family: 'Courier New', monospace;
+  font-size: 14px;
+  font-weight: 500;
+  border-radius: 4px;
+  border: 1px solid #333;
+  background: #1a1a1a;
+  color: #e6e6e6;
   cursor: pointer;
-  transition: background 0.2s ease;
+  transition: all 0.2s ease;
+}
+
+dialog .dialog-button:hover {
+  background: #2a2a2a;
+  border-color: #444;
+  color: #fff;
 }
 
 
@@ -527,22 +637,161 @@ dialog .dialog-button {
 
 dialog .dialog-button.primary {
   background-color: #f05a28;
-  color: white;
+  border-color: #f05a28;
+  color: #fff;
 }
 
 dialog .dialog-button.primary:hover {
-  background-color: #f05a28;
+  background-color: #e04924;
+  border-color: #e04924;
 }
 
 dialog .dialog-button.secondary {
-  background-color: #f0a328;
-  color: #eee;
+  background-color: #2a2a2a;
+  border-color: #444;
+  color: #e6e6e6;
 }
 
 dialog .dialog-button.secondary:hover {
-  background-color: #f0b84c;
+  background-color: #3a3a3a;
+  border-color: #555;
+}
+dialog .dialog-button.primary:disabled,
+dialog .dialog-button.primary[aria-disabled="true"] {
+  background-color: #0a0a0a;
+  border-color: #222;
+  color: #555;
+  opacity: .55;
+  cursor: not-allowed;
+  pointer-events: none;
 }
 
+dialog .dialog-button.secondary:disabled,
+dialog .dialog-button.secondary[aria-disabled="true"] {
+  background-color: #0a0a0a;
+  border-color: #222;
+  color: #555;
+  opacity: .55;
+  cursor: not-allowed;
+  pointer-events: none;
+}
+
+dialog .dialog-button:disabled:focus {
+  outline: none;
+}
+
+dialog .dialog-form {
+  display: flex;
+  flex-direction: column;
+  gap: 12px;
+}
+
+dialog .dialog-input {
+  width: 100%;
+  padding: 10px 12px;
+  border: 1px solid #333;
+  border-radius: 4px;
+  background-color: #0f0f0f;
+  color: #e6e6e6;
+  font-family: 'Courier New', monospace;
+  font-size: 14px;
+  transition: border-color 0.2s ease, box-shadow 0.2s ease;
+  box-sizing: border-box;
+}
+
+dialog .dialog-input:focus {
+  outline: none;
+  border-color: #f05a28;
+  box-shadow: 0 0 0 2px rgba(240, 90, 40, 0.2);
+}
+
+dialog .dialog-input::placeholder {
+  color: #555;
+}
+
+dialog .dialog-input.error {
+  border-color: #8b0000;
+  box-shadow: 0 0 0 2px rgba(139, 0, 0, 0.2);
+}
+
+dialog .dialog-checkbox-label {
+  display: flex;
+  align-items: center;
+  gap: 8px;
+  color: #e6e6e6;
+  font-size: 14px;
+  cursor: pointer;
+}
+
+dialog .dialog-checkbox-label input[type="checkbox"] {
+  width: 16px;
+  height: 16px;
+  accent-color: #f05a28;
+}
+
+dialog .dialog-suggestions {
+  max-height: 150px;
+  overflow-y: auto;
+}
+
+dialog .dialog-suggestion-item {
+  padding: 8px 12px;
+  cursor: pointer;
+  border-radius: 4px;
+  transition: background-color 0.2s;
+}
+
+dialog .dialog-suggestion-item:hover {
+  background-color: #1a1a1a;
+}
+
+dialog .dialog-box-wide {
+  max-width: 500px;
+}
+
+dialog .dialog-label {
+  display: block;
+  font-size: 12px;
+  color: #888;
+  margin-bottom: 4px;
+  text-transform: uppercase;
+  letter-spacing: 0.5px;
+}
+
+dialog .dialog-label-danger {
+  color: #8b0000;
+}
+
+dialog .dialog-divider {
+  height: 1px;
+  background-color: #333;
+  margin: 16px 0;
+}
+
+dialog .dialog-danger-zone {
+  padding: 12px;
+  border: 1px solid #8b0000;
+  border-radius: 4px;
+  background-color: rgba(139, 0, 0, 0.1);
+}
+
+dialog .dialog-button.danger {
+  background-color: #8b0000;
+  border-color: #8b0000;
+  color: #fff;
+}
+
+dialog .dialog-button.danger:hover {
+  background-color: #a00000;
+  border-color: #a00000;
+}
+
+dialog .dialog-button.danger:disabled {
+  background-color: #0a0a0a;
+  border-color: #222;
+  color: #555;
+  cursor: not-allowed;
+}
 
 .embed-url-link {
   display: flex;
@@ -584,4 +833,8 @@ dialog .dialog-button.secondary:hover {
   color: #f05a28;
   text-decoration: none;
   margin-top: 10px;
-}
+}
+
+th {
+    min-width: 100px;
+}

src/snek/static/buttons.css

@@ -0,0 +1,93 @@
+.btn {
+    display: inline-flex;
+    align-items: center;
+    justify-content: center;
+    gap: 8px;
+    padding: 10px 20px;
+    font-family: 'Courier New', monospace;
+    font-size: 14px;
+    font-weight: 500;
+    text-decoration: none;
+    border: 1px solid #333;
+    border-radius: 4px;
+    cursor: pointer;
+    transition: all 0.2s ease;
+    background: #1a1a1a;
+    color: #e6e6e6;
+}
+
+.btn:hover {
+    background: #2a2a2a;
+    border-color: #444;
+    color: #fff;
+}
+
+.btn:active {
+    background: #111;
+    transform: translateY(1px);
+}
+
+.btn:disabled {
+    background: #0a0a0a;
+    color: #555;
+    cursor: not-allowed;
+    border-color: #222;
+}
+
+.btn-primary {
+    background: #f05a28;
+    border-color: #f05a28;
+    color: #fff;
+}
+
+.btn-primary:hover {
+    background: #e04924;
+    border-color: #e04924;
+}
+
+.btn-secondary {
+    background: #2a2a2a;
+    border-color: #444;
+}
+
+.btn-secondary:hover {
+    background: #3a3a3a;
+    border-color: #555;
+}
+
+.btn-danger {
+    background: #1a1a1a;
+    border-color: #8b0000;
+    color: #ff6b6b;
+}
+
+.btn-danger:hover {
+    background: #2a1515;
+    border-color: #b00;
+}
+
+.btn-success {
+    background: #1a1a1a;
+    border-color: #006400;
+    color: #6bff6b;
+}
+
+.btn-success:hover {
+    background: #152a15;
+    border-color: #0b0;
+}
+
+.btn-sm {
+    padding: 6px 12px;
+    font-size: 12px;
+}
+
+.btn-lg {
+    padding: 14px 28px;
+    font-size: 16px;
+}
+
+.btn-block {
+    display: flex;
+    width: 100%;
+}

src/snek/static/channel-menu.css

@@ -0,0 +1,123 @@
+/* retoor <retoor@molodetz.nl> */
+
+channel-menu {
+  display: none;
+  position: relative;
+}
+
+@media (max-width: 768px) {
+  channel-menu {
+    display: inline-flex !important;
+    align-items: center;
+  }
+}
+
+.channel-menu-toggle {
+  display: flex;
+  align-items: center;
+  justify-content: center;
+  padding: 4px 8px;
+  background: none;
+  color: #888;
+  border: none;
+  border-radius: 4px;
+  cursor: pointer;
+  font-size: 1.2em;
+  transition: all 0.2s ease;
+}
+
+.channel-menu-toggle:hover {
+  color: #fff;
+  background-color: rgba(255, 255, 255, 0.05);
+}
+
+channel-menu[open] .channel-menu-toggle {
+  color: #fff;
+  background-color: rgba(255, 255, 255, 0.05);
+}
+
+.channel-menu-panel {
+  position: fixed;
+  top: 50px;
+  right: 8px;
+  left: 8px;
+  background-color: #111;
+  border: 1px solid #333;
+  border-radius: 8px;
+  padding: 8px 0;
+  display: none;
+  flex-direction: column;
+  max-height: calc(100vh - 70px);
+  overflow-y: auto;
+  box-shadow: 0 4px 12px rgba(0, 0, 0, 0.4);
+  z-index: 1000;
+}
+
+channel-menu[open] .channel-menu-panel {
+  display: flex;
+}
+
+.channel-menu-section {
+  padding: 8px 16px 4px;
+  font-size: 0.75em;
+  color: #666;
+  text-transform: uppercase;
+  letter-spacing: 0.5px;
+}
+
+.channel-menu-item {
+  display: flex;
+  align-items: center;
+  justify-content: space-between;
+  gap: 12px;
+  padding: 10px 16px;
+  color: #888;
+  text-decoration: none;
+  font-size: 0.95em;
+  transition: all 0.2s ease;
+  cursor: pointer;
+  border: none;
+  background: none;
+  width: 100%;
+  text-align: left;
+}
+
+.channel-menu-item:hover {
+  color: #fff;
+  background-color: rgba(255, 255, 255, 0.05);
+}
+
+.channel-menu-item.active {
+  color: #fff;
+  background-color: rgba(255, 255, 255, 0.1);
+}
+
+.channel-menu-name {
+  flex: 1;
+  overflow: hidden;
+  text-overflow: ellipsis;
+  white-space: nowrap;
+}
+
+.channel-menu-count {
+  background: #f05a28;
+  color: #fff;
+  border-radius: 10px;
+  padding: 2px 8px;
+  font-size: 0.8em;
+  min-width: 20px;
+  text-align: center;
+}
+
+.channel-menu-empty {
+  padding: 16px;
+  color: #666;
+  font-size: 0.9em;
+  text-align: center;
+}
+
+.channel-menu-divider {
+  height: 1px;
+  background: #333;
+  margin: 8px 0;
+}

src/snek/static/channel-menu.js

@@ -0,0 +1,171 @@
+// retoor <retoor@molodetz.nl>
+
+import { app } from "./app.js";
+
+class ChannelMenu extends HTMLElement {
+  constructor() {
+    super();
+    this._isOpen = false;
+    this._channels = [];
+    this._boundClickOutside = this._handleClickOutside.bind(this);
+
+    this._container = document.createElement('div');
+    this._container.className = 'channel-menu-container';
+
+    this._toggleButton = document.createElement('button');
+    this._toggleButton.className = 'channel-menu-toggle';
+    this._toggleButton.setAttribute('aria-label', 'Toggle channel menu');
+    this._toggleButton.innerHTML = '💬';
+    this._toggleButton.addEventListener('click', (e) => {
+      e.stopPropagation();
+      this._toggle();
+    });
+
+    this._menuPanel = document.createElement('div');
+    this._menuPanel.className = 'channel-menu-panel';
+
+    this._container.appendChild(this._toggleButton);
+    this._container.appendChild(this._menuPanel);
+    this.appendChild(this._container);
+  }
+
+  async connectedCallback() {
+    await this._loadChannels();
+    app.addEventListener('channel-message', (data) => {
+      if (data.is_final && data.channel_uid) {
+        this._incrementCount(data.channel_uid);
+      }
+    });
+  }
+
+  async _loadChannels() {
+    try {
+      this._channels = await app.rpc.getChannels();
+      this._renderChannels();
+    } catch (e) {
+      this._menuPanel.innerHTML = '<div class="channel-menu-empty">Failed to load channels</div>';
+    }
+  }
+
+  _renderChannels() {
+    this._menuPanel.innerHTML = '';
+
+    const publicChannels = this._channels.filter(c => !c.is_private);
+    const privateChannels = this._channels.filter(c => c.is_private);
+
+    if (publicChannels.length > 0) {
+      const header = document.createElement('div');
+      header.className = 'channel-menu-section';
+      header.textContent = 'Channels';
+      this._menuPanel.appendChild(header);
+
+      publicChannels.forEach(channel => {
+        this._menuPanel.appendChild(this._createChannelItem(channel));
+      });
+    }
+
+    if (privateChannels.length > 0) {
+      if (publicChannels.length > 0) {
+        const divider = document.createElement('div');
+        divider.className = 'channel-menu-divider';
+        this._menuPanel.appendChild(divider);
+      }
+
+      const header = document.createElement('div');
+      header.className = 'channel-menu-section';
+      header.textContent = 'Private';
+      this._menuPanel.appendChild(header);
+
+      privateChannels.forEach(channel => {
+        this._menuPanel.appendChild(this._createChannelItem(channel));
+      });
+    }
+
+    if (this._channels.length === 0) {
+      this._menuPanel.innerHTML = '<div class="channel-menu-empty">No channels available</div>';
+    }
+  }
+
+  _createChannelItem(channel) {
+    const item = document.createElement('a');
+    item.className = 'channel-menu-item';
+    item.href = `/channel/${channel.uid}.html`;
+    item.dataset.channelUid = channel.uid;
+
+    if (window.location.pathname.includes(channel.uid)) {
+      item.classList.add('active');
+    }
+
+    const name = document.createElement('span');
+    name.className = 'channel-menu-name';
+    name.textContent = channel.name;
+    if (channel.color) {
+      name.style.color = channel.color;
+    }
+    item.appendChild(name);
+
+    if (channel.new_count > 0) {
+      const count = document.createElement('span');
+      count.className = 'channel-menu-count';
+      count.textContent = channel.new_count;
+      item.appendChild(count);
+    }
+
+    item.addEventListener('click', () => {
+      this._close();
+    });
+
+    return item;
+  }
+
+  _incrementCount(channelUid) {
+    const item = this._menuPanel.querySelector(`[data-channel-uid="${channelUid}"]`);
+    if (item && !item.classList.contains('active')) {
+      let countEl = item.querySelector('.channel-menu-count');
+      if (!countEl) {
+        countEl = document.createElement('span');
+        countEl.className = 'channel-menu-count';
+        countEl.textContent = '1';
+        item.appendChild(countEl);
+      } else {
+        const current = parseInt(countEl.textContent) || 0;
+        countEl.textContent = current + 1;
+      }
+    }
+  }
+
+  _toggle() {
+    if (this._isOpen) {
+      this._close();
+    } else {
+      this._open();
+    }
+  }
+
+  _open() {
+    this._isOpen = true;
+    this.setAttribute('open', '');
+    this._loadChannels();
+    setTimeout(() => {
+      document.addEventListener('click', this._boundClickOutside);
+    }, 0);
+  }
+
+  _close() {
+    this._isOpen = false;
+    this.removeAttribute('open');
+    document.removeEventListener('click', this._boundClickOutside);
+  }
+
+  _handleClickOutside(event) {
+    if (!this.contains(event.target)) {
+      this._close();
+    }
+  }
+
+  disconnectedCallback() {
+    document.removeEventListener('click', this._boundClickOutside);
+  }
+}
+
+customElements.define('channel-menu', ChannelMenu);

src/snek/static/chat-input.js

@@ -1,30 +1,46 @@
-import { app } from "../app.js";
+// retoor <retoor@molodetz.nl>
 
-class ChatInputComponent extends HTMLElement {
+import { app } from "./app.js";
+import { NjetComponent, eventBus } from "./njet.js";
+import { FileUploadGrid } from "./file-upload-grid.js";
+import { loggerFactory } from "./logger.js";
+import "./toolbar-menu.js";
+
+const log = loggerFactory.getLogger("ChatInput");
+
+class ChatInputComponent extends NjetComponent {
   autoCompletions = {
     "example 1": () => {},
     "example 2": () => {},
-  }
+  };
+
   hiddenCompletions = {
     "/starsRender": () => {
-        app.rpc.starsRender(this.channelUid,this.value.replace("/starsRender ",""))
+      app.rpc.starsRender(this.channelUid, this.value.replace("/starsRender ", ""))
+    },
+    "/leet": () => {
+      this.value = this.textToLeet(this.value);
+        this._leetSpeak = !this._leetSpeak;
+    },
+    "/l33t": () => {
+      this._leetSpeakAdvanced = !this._leetSpeakAdvanced;
     }
-  }
-  users = []
-  textarea = null 
-  _value = "" 
-  lastUpdateEvent = null 
-  previousValue = "" 
-  lastChange = null 
-  changed = false 
+  };
+
+  users = [];
+  textarea = null;
+  _value = "";
+  lastUpdateEvent = null;
+  queuedMessage = null;
+  lastMessagePromise = null;
+  _leetSpeak = false;
+  _leetSpeakAdvanced = false;
   constructor() {
     super();
     this.lastUpdateEvent = new Date();
     this.textarea = document.createElement("textarea");
+      this.textarea.classList.add("chat-input-textarea");
     this.value = this.getAttribute("value") || "";
-    this.previousValue = this.value;
-    this.lastChange = new Date();
-    this.changed = false;
   }
 
   get value() {
@@ -32,24 +48,27 @@ class ChatInputComponent extends HTMLElement {
   }
 
   set value(value) {
-    this._value = value || "";
+    this._value = value;
     this.textarea.value = this._value;
   }
+
   get allAutoCompletions() {
-     return Object.assign({},this.autoCompletions,this.hiddenCompletions) 
+    return Object.assign({}, this.autoCompletions, this.hiddenCompletions);
   }
-  resolveAutoComplete() {
-    let count = 0;
+
+  resolveAutoComplete(input) {
     let value = null;
-     
-    Object.keys(this.allAutoCompletions).forEach((key) => {
-        if (key.startsWith(this.value.split(" ")[0])) {
-        count++;
+
+    for (const key of Object.keys(this.allAutoCompletions)) {
+      if (key.startsWith(input.split(" ", 1)[0])) {
+        if (value) {
+          return null;
+        }
         value = key;
       }
-    });
-    if (count == 1) return value;
-    return null;
+    }
+
+    return value;
   }
 
   isActive() {
@@ -58,125 +77,313 @@ class ChatInputComponent extends HTMLElement {
 
   focus() {
     this.textarea.focus();
-  } 
-    getAuthors(){
-        let authors = []
-        for (let i = 0; i < this.users.length; i++) {
-            authors.push(this.users[i].username)
-            authors.push(this.users[i].nick)
-        }
-        return authors
-        
-    }
- extractMentions(text) {
-  const regex = /@([a-zA-Z0-9_-]+)/g;
-  const mentions = [];
-  let match;
-
-  while ((match = regex.exec(text)) !== null) {
-    mentions.push(match[1]);
   }
 
-  return mentions;
-}
-    matchMentionsToAuthors(mentions, authors) {
-  return mentions.map(mention => {
-    let closestAuthor = null;
-    let minDistance = Infinity;
-     const lowerMention = mention.toLowerCase();
-    
-    authors.forEach(author => {
-    const lowerAuthor = author.toLowerCase();
-      let distance = this.levenshteinDistance(lowerMention, lowerAuthor);
-     
+  getAuthors() {
+    return this.users.flatMap((user) => [user.username, user.nick]);
+  }
 
-        if(!this.isSubsequence(lowerMention,lowerAuthor)) {
-            distance += 10        
+  extractMentions(text) {
+    return Array.from(text.matchAll(/@([a-zA-Z0-9_-]+)/g), m => m[1]);
+  }
+
+
+  matchMentionsToAuthors(mentions, authors) {
+    return mentions.map((mention) => {
+      const lowerMention = mention.toLowerCase();
+      let bestMatch = null;
+      let bestScore = 0;
+
+      for (const author of authors) {
+        const lowerAuthor = author.toLowerCase();
+        let score = 0;
+
+        if (lowerMention === lowerAuthor) {
+          score = 100;
+        } else if (lowerAuthor.startsWith(lowerMention)) {
+          score = 90 + (5 * (lowerMention.length / lowerAuthor.length));
+        } else if (lowerAuthor.includes(lowerMention) && lowerMention.length >= 2) {
+          const position = lowerAuthor.indexOf(lowerMention);
+          score = 80 - (10 * (position / lowerAuthor.length));
+        } else if (this.isFuzzyMatch(lowerMention, lowerAuthor)) {
+          const ratio = lowerMention.length / lowerAuthor.length;
+          score = 40 + (20 * ratio);
+        } else if (this.isCloseMatch(lowerMention, lowerAuthor)) {
+          score = 30 + (10 * (lowerMention.length / lowerAuthor.length));
         }
 
-      if (distance < minDistance) {
-        minDistance = distance;
-        closestAuthor = author;
-
+        if (score > bestScore) {
+          bestScore = score;
+          bestMatch = author;
+        }
       }
 
+      const minScore = 40;
+      return {
+        mention,
+        closestAuthor: bestScore >= minScore ? bestMatch : null,
+        score: bestScore,
+      };
+    });
+  }
+
+  isFuzzyMatch(needle, haystack) {
+    if (needle.length < 2) return false;
+
+    let needleIndex = 0;
+    for (let i = 0; i < haystack.length && needleIndex < needle.length; i++) {
+      if (haystack[i] === needle[needleIndex]) {
+        needleIndex++;
+      }
+    }
+    return needleIndex === needle.length;
+  }
+
+  isCloseMatch(str1, str2) {
+    if (Math.abs(str1.length - str2.length) > 2) return false;
+
+    const shorter = str1.length <= str2.length ? str1 : str2;
+    const longer = str1.length > str2.length ? str1 : str2;
+
+    let differences = 0;
+    let j = 0;
+
+    for (let i = 0; i < shorter.length && j < longer.length; i++) {
+      if (shorter[i] !== longer[j]) {
+        differences++;
+        if (j + 1 < longer.length && shorter[i] === longer[j + 1]) {
+          j++;
+        }
+      }
+      j++;
+    }
+
+    differences += Math.abs(longer.length - j);
+    return differences <= 2;
+  }
+
+  matchMentions4ToAuthors(mentions, authors) {
+    return mentions.map((mention) => {
+      let closestAuthor = null;
+      let minDistance = Infinity;
+      const lowerMention = mention.toLowerCase();
+
+      authors.forEach((author) => {
+        const lowerAuthor = author.toLowerCase();
+        let distance = this.levenshteinDistance(lowerMention, lowerAuthor);
+
+        if (!this.isSubsequence(lowerMention, lowerAuthor)) {
+          distance += 10;
+        }
+
+        if (distance < minDistance) {
+          minDistance = distance;
+          closestAuthor = author;
+        }
+      });
+
+      if (minDistance < 5) {
+        closestAuthor = 0;
+      }
+      return { mention, closestAuthor, distance: minDistance };
+    });
+  }
+
+  levenshteinDistance(a, b) {
+    const matrix = [];
+
+    for (let i = 0; i <= b.length; i++) {
+      matrix[i] = [i];
+    }
+    for (let j = 0; j <= a.length; j++) {
+      matrix[0][j] = j;
+    }
+
+    for (let i = 1; i <= b.length; i++) {
+      for (let j = 1; j <= a.length; j++) {
+        if (b.charAt(i - 1) === a.charAt(j - 1)) {
+          matrix[i][j] = matrix[i - 1][j - 1];
+        } else {
+          matrix[i][j] = Math.min(
+            matrix[i - 1][j] + 1,
+            matrix[i][j - 1] + 1,
+            matrix[i - 1][j - 1] + 1
+          );
+        }
+      }
+    }
+
+    return matrix[b.length][a.length];
+  }
+
+  replaceMentionsWithAuthors(text) {
+    const authors = this.getAuthors();
+    const mentions = this.extractMentions(text);
+
+    const matches = this.matchMentionsToAuthors(mentions, authors);
+    let updatedText = text;
+    matches.forEach(({ mention, closestAuthor }) => {
+      if(closestAuthor){
+        const mentionRegex = new RegExp(`@${mention}`, 'g');
+        updatedText = updatedText.replace(mentionRegex, `@${closestAuthor}`);
+      }
     });
 
-    return { mention, closestAuthor, distance: minDistance };
-  });
-}
-levenshteinDistance(a, b) {
-  const matrix = [];
-
-  // Initialize the first row and column
-  for (let i = 0; i <= b.length; i++) {
-    matrix[i] = [i];
-  }
-  for (let j = 0; j <= a.length; j++) {
-    matrix[0][j] = j;
+    return updatedText;
   }
 
-  // Fill in the matrix
-  for (let i = 1; i <= b.length; i++) {
-    for (let j = 1; j <= a.length; j++) {
-      if (b.charAt(i - 1) === a.charAt(j - 1)) {
-        matrix[i][j] = matrix[i - 1][j - 1];
-      } else {
-        matrix[i][j] = Math.min(
-          matrix[i - 1][j] + 1,    // Deletion
-          matrix[i][j - 1] + 1,    // Insertion
-          matrix[i - 1][j - 1] + 1 // Substitution
-        );
-      }
+  matchMentions2ToAuthors(mentions, authors) {
+    return mentions.map(mention => {
+      let closestAuthor = null;
+      let minDistance = Infinity;
+      const lowerMention = mention.toLowerCase();
+
+      authors.forEach(author => {
+        const lowerAuthor = author.toLowerCase();
+        let distance = this.levenshteinDistance(lowerMention, lowerAuthor);
+
+        if (!this.isSubsequence(lowerMention, lowerAuthor)) {
+          distance += 10;
+        }
+
+        if (distance < minDistance) {
+          minDistance = distance;
+          closestAuthor = author;
+        }
+      });
+    if (minDistance < 5){
+        closestAuthor = 0;
     }
+      return { mention, closestAuthor, distance: minDistance };
+    });
   }
 
-  return matrix[b.length][a.length];
+
+  textToLeet(text) {
+  // L33t speak character mapping
+  const leetMap = {
+    'a': '4',
+    'A': '4',
+    'e': '3',
+    'E': '3',
+    'i': '1',
+    'I': '1',
+    'o': '0',
+    'O': '0',
+    's': '5',
+    'S': '5',
+    't': '7',
+    'T': '7',
+    'l': '1',
+    'L': '1',
+    'g': '9',
+    'G': '9',
+    'b': '6',
+    'B': '6',
+    'z': '2',
+    'Z': '2'
+  };
+  
+  // Convert text to l33t speak
+  return text.split('').map(char => {
+    return leetMap[char] || char;
+  }).join('');
 }
 
 
-
-      replaceMentionsWithAuthors(text) {
-    const authors = this.getAuthors();
-  const mentions = this.extractMentions(text);
+// Advanced version with random character selection
+textToLeetAdvanced(text) {
+  const leetMap = {
+    'a': ['4', '@', '/\\'],
+    'A': ['4', '@', '/\\'],
+    'e': ['3', '€'],
+    'E': ['3', '€'],
+    'i': ['1', '!', '|'],
+    'I': ['1', '!', '|'],
+    'o': ['0', '()'],
+    'O': ['0', '()'],
+    's': ['5', '$'],
+    'S': ['5', '$'],
+    't': ['7', '+'],
+    'T': ['7', '+'],
+    'l': ['1', '|'],
+    'L': ['1', '|'],
+    'g': ['9', '6'],
+    'G': ['9', '6'],
+    'b': ['6', '|3'],
+    'B': ['6', '|3'],
+    'z': ['2'],
+    'Z': ['2'],
+    'h': ['#', '|-|'],
+    'H': ['#', '|-|'],
+    'n': ['|\\|'],
+    'N': ['|\\|'],
+    'm': ['|\\/|'],
+    'M': ['|\\/|'],
+    'w': ['\\/\\/'],
+    'W': ['\\/\\/'],
+    'v': ['\\/', 'V'],
+    'V': ['\\/', 'V'],
+    'u': ['|_|'],
+    'U': ['|_|'],
+    'r': ['|2'],
+    'R': ['|2'],
+    'f': ['|='],
+    'F': ['|='],
+    'd': ['|)'],
+    'D': ['|)'],
+    'c': ['(', '['],
+    'C': ['(', '['],
+    'k': ['|<'],
+    'K': ['|<'],
+    'p': ['|>'],
+    'P': ['|>'],
+    'x': ['><'],
+    'X': ['><'],
+    'y': ['`/'],
+    'Y': ['`/']
+  };
   
-  const matches = this.matchMentionsToAuthors(mentions, authors);
-  let updatedText = text;
-  matches.forEach(({ mention, closestAuthor }) => {
-    const mentionRegex = new RegExp(`@${mention}`, 'g');
-      updatedText = updatedText.replace(mentionRegex, `@${closestAuthor}`);
-  });
-  
-  return updatedText;
+  return text.split('').map(char => {
+    const options = leetMap[char];
+    if (options) {
+      return options[Math.floor(Math.random() * options.length)];
+    }
+    return char;
+  }).join('');
 }
 
 
-
   async connectedCallback() {
-    this.user = null 
+    this.user = null;
     app.rpc.getUser(null).then((user) => {
-        this.user=user
-    })
+      this.user = user;
+    });
 
-
-      const me = this;
-    this.liveType = this.getAttribute("live-type") === "true";
-    this.liveTypeInterval =
-      parseInt(this.getAttribute("live-type-interval")) || 6;
+    this.liveType = this.getAttribute("live-type") == "true";
+    this.liveTypeInterval = parseInt(this.getAttribute("live-type-interval")) || 6;
     this.channelUid = this.getAttribute("channel");
-    
-    app.rpc.getRecentUsers(this.channelUid).then(users=>{
-        this.users = users    
-    })
-      this.messageUid = null;
+
+    app.rpc.getRecentUsers(this.channelUid).then(users => {
+      this.users = users;
+    });
+    this.messageUid = null;
 
     this.classList.add("chat-input");
 
+    this.fileUploadGrid = new FileUploadGrid();
+    this.fileUploadGrid.setAttribute("channel", this.channelUid);
+    this.fileUploadGrid.style.display = "none";
+    this.parentElement.insertBefore(this.fileUploadGrid, this);
+
     this.textarea.setAttribute("placeholder", "Type a message...");
     this.textarea.setAttribute("rows", "2");
 
     this.appendChild(this.textarea);
 
+    this.sttButton = document.createElement("stt-button");
+    this.ttsButton = document.createElement("tts-button");
+
     this.uploadButton = document.createElement("upload-button");
     this.uploadButton.setAttribute("channel", this.channelUid);
     this.uploadButton.addEventListener("upload", (e) => {
@@ -185,189 +392,175 @@ levenshteinDistance(a, b) {
     this.uploadButton.addEventListener("uploaded", (e) => {
       this.dispatchEvent(new CustomEvent("uploaded", e));
     });
+    this.uploadButton.addEventListener("click", (e) => {
+      e.preventDefault();
+      this.fileUploadGrid.openFileDialog();
+    });
+    this.subscribe("file-uploading", (e) => {
+      this.fileUploadGrid.style.display = "block";
+    });
 
-    this.appendChild(this.uploadButton);
+    this.toolbarMenu = document.createElement("toolbar-menu");
+    this.toolbarMenu.addButton(this.sttButton, 'stt');
+    this.toolbarMenu.addButton(this.ttsButton, 'tts');
+    this.toolbarMenu.addButton(this.uploadButton, 'upload');
+    this.appendChild(this.toolbarMenu);
 
+    this.textarea.addEventListener("blur", () => {
+      this.updateFromInput(this.value, true).then(
+          this.updateFromInput("")
+      )
+    });
+
+    this.subscribe("file-uploads-done", (data)=>{
+      this.fileUploadGrid.style.display = "none";
+      let msg =data.reduce((message, file) => {
+        return `${message}[${file.filename || file.name || file.remoteFile}](/channel/attachment/${file.remoteFile})`;
+      }, '');
+      app.rpc.sendMessage(this.channelUid, msg, true);
+    });
+
+    
+    this.textarea.addEventListener("change",(e)=>{
+        this.value = this.textarea.value;
+      this.updateFromInput(e.target.value);
+    })
     this.textarea.addEventListener("keyup", (e) => {
       if (e.key === "Enter" && !e.shiftKey) {
-        this.value = "";
+        const message = this.replaceMentionsWithAuthors(this.value);
         e.target.value = "";
+
+        if (!message) {
+          return;
+        }
+        let autoCompletionHandler = this.allAutoCompletions[this.value.split(" ", 1)[0]];
+        if (autoCompletionHandler) {
+          autoCompletionHandler();
+          this.value = "";
+          e.target.value = "";
+          return;
+        }
+
+        this.finalizeMessage(this.messageUid);
         return;
       }
-      this.value = e.target.value;
-      this.changed = true;
-      this.update();
+
+      this.updateFromInput(e.target.value);
     });
 
     this.textarea.addEventListener("keydown", (e) => {
       this.value = e.target.value;
-      let autoCompletion = null; 
+
+      let autoCompletion = null;
       if (e.key === "Tab") {
         e.preventDefault();
-        autoCompletion = this.resolveAutoComplete();
+        autoCompletion = this.resolveAutoComplete(this.value);
         if (autoCompletion) {
           e.target.value = autoCompletion;
           this.value = autoCompletion;
           return;
         }
       }
+
       if (e.key === "Enter" && !e.shiftKey) {
         e.preventDefault();
+      }
 
-        const message = me.replaceMentionsWithAuthors(this.value);
-        e.target.value = "";
-
-        if (!message) {
-          return;
-        }
-          let autoCompletionHandler = this.allAutoCompletions[this.value.split(" ")[0]];
-        if (autoCompletionHandler) {
-          autoCompletionHandler();
-            this.value = "";
-          this.previousValue = "";
-          e.target.value = "";
-
-          return;
-        }
-
-        this.updateMessage()
-        app.rpc.finalizeMessage(this.messageUid)
-        this.value = "";
-        this.previousValue = "";
-        this.messageUid = null;
+      if (e.repeat) {
+        this.updateFromInput(e.target.value);
       }
     });
 
-    this.changeInterval = setInterval(() => {
-      if (!this.liveType) {
-        return;
-      }
-      if (this.value !== this.previousValue) {
-        if (
-          this.trackSecondsBetweenEvents(this.lastChange, new Date()) >=
-          this.liveTypeInterval
-        ) {
-          this.value = "";
-          this.previousValue = "";
-        }
-        this.lastChange = new Date();
-      }
-      this.update();
-    }, 300);
-
     this.addEventListener("upload", (e) => {
       this.focus();
     });
     this.addEventListener("uploaded", function (e) {
-      let message = "";
-      e.detail.files.forEach((file) => {
-        message += `[${file.name}](/channel/attachment/${file.relative_url})`;
-      });
-      app.rpc.sendMessage(this.channelUid, message,true);
+      let message = e.detail.files.reduce((message, file) => {
+        return `${message}[${file.name}](/channel/attachment/${file.relative_url})`;
+      }, '');
+      app.rpc.sendMessage(this.channelUid, message, true);
     });
-    setTimeout(()=>{
+
+    setTimeout(() => {
       this.focus();
-    },1000)
+    }, 1000);
   }
 
   trackSecondsBetweenEvents(event1Time, event2Time) {
     const millisecondsDifference = event2Time.getTime() - event1Time.getTime();
     return millisecondsDifference / 1000;
   }
+
   isSubsequence(s, t) {
-      let i = 0, j = 0;
-      while (i < s.length && j < t.length) {
-        if (s[i] === t[j]) {
-          i++;
-        }
-        j++;
+    let i = 0, j = 0;
+    while (i < s.length && j < t.length) {
+      if (s[i] === t[j]) {
+        i++;
       }
-      return i === s.length; 
-    }
-
-
-  newMessage() {
-    if (!this.messageUid) {
-      this.messageUid = "?";
-    }
-
-    this.value = this.replaceMentionsWithAuthors(this.value);
-    this.sendMessage(this.channelUid, this.value,!this.liveType).then((uid) => {
-        if (this.liveType) {
-            this.messageUid = uid;
-        }
-    });
-    }
-
-  updateMessage() {
-    if (this.value[0] == "/") {
-      return false;
-    }
-    if (!this.messageUid) {
-      this.newMessage();
-      return false;
-    }
-    if (this.messageUid === "?") {
-      return false;
-    }
-    if (
-      typeof app !== "undefined" &&
-      app.rpc &&
-      typeof app.rpc.updateMessageText === "function"
-    ) {
-      app.rpc.updateMessageText(this.messageUid, this.replaceMentionsWithAuthors(this.value));
+      j++;
     }
+    return i === s.length && s.length > 1;
   }
 
-  updateStatus() {
-    if (this.liveType) {
-      return;
-    }
-    if (this.trackSecondsBetweenEvents(this.lastUpdateEvent, new Date()) > 1) {
+  flagTyping() {
+    if (this.trackSecondsBetweenEvents(this.lastUpdateEvent, new Date()) >= 1) {
       this.lastUpdateEvent = new Date();
-      if (
-        typeof app !== "undefined" &&
-        app.rpc &&
-        typeof app.rpc.set_typing === "function"
-      ) {
-        app.rpc.set_typing(this.channelUid, this.user.color);
-      }
+      log.debug("Flagging typing indicator", { channelUid: this.channelUid });
+      app.rpc.set_typing(this.channelUid, this.user?.color).catch((e) => {
+        log.warn("set_typing failed", { error: e, channelUid: this.channelUid });
+      });
     }
   }
 
-  update() {
-    const expired =
-      this.trackSecondsBetweenEvents(this.lastChange, new Date()) >=
-      this.liveTypeInterval;
-    const changed = this.value !== this.previousValue;
-
-    if (changed || expired) {
-      this.lastChange = new Date();
-      this.updateStatus();
+  finalizeMessage(messageUid) {
+    let value = this.value;
+    value =  this.replaceMentionsWithAuthors(value)
+    if(this._leetSpeak){
+        value = this.textToLeet(value);
+    }else if(this._leetSpeakAdvanced){
+        value = this.textToLeetAdvanced(value);
     }
+    log.info("Finalizing message", { channelUid: this.channelUid, messageLength: value.length, messageUid });
+    app.rpc.sendMessage(this.channelUid, value , true).then((result) => {
+      log.debug("Message finalized successfully", { channelUid: this.channelUid, result });
+    }).catch((e) => {
+      log.error("Failed to finalize message", { channelUid: this.channelUid, error: e });
+    });
+    this.value = "";
+    this.messageUid = null;
+    this.queuedMessage = null;
+    this.lastMessagePromise = null;
+  }
+  
 
-    this.previousValue = this.value;
 
-    if (this.liveType && expired) {
-      this.value = "";
-      this.previousValue = "";
-      this.messageUid = null;
-      return;
-    }
+  updateFromInput(value, isFinal = false) {
+    log.debug("updateFromInput called", { valueLength: value?.length, isFinal, liveType: this.liveType, channelUid: this.channelUid });
 
-    if (changed) {
-      if (this.liveType) {
-        this.updateMessage();
-      }
+    this.value = value;
+
+    this.flagTyping();
+
+    if (this.liveType && value[0] !== "/") {
+      const messageText = this.replaceMentionsWithAuthors(value);
+      log.debug("Sending live type message", { channelUid: this.channelUid, messageLength: messageText?.length, isFinal: !this.liveType || isFinal });
+      this.messageUid = this.sendMessage(this.channelUid, messageText, !this.liveType || isFinal);
+      return this.messageUid;
     }
   }
 
-  async sendMessage(channelUid, value,is_final) {
-    if (!value.trim()) {
-      return null;
+  async sendMessage(channelUid, value, is_final) {
+    log.info("sendMessage called", { channelUid, valueLength: value?.length, is_final });
+    try {
+      const result = await app.rpc.sendMessage(channelUid, value, is_final);
+      log.debug("sendMessage completed", { channelUid, result, is_final });
+      return result;
+    } catch (e) {
+      log.error("sendMessage failed", { channelUid, error: e, is_final });
+      throw e;
     }
-    return await app.rpc.sendMessage(channelUid, value,is_final);
   }
 }
 
 customElements.define("chat-input", ChatInputComponent);
+

src/snek/static/chat-window.js

@@ -1,3 +1,5 @@
+// retoor <retoor@molodetz.nl>
+
 // Written by retoor@molodetz.nl
 
 // This code defines a custom HTML element called ChatWindowElement that provides a chat interface within a shadow DOM, handling connection callbacks, displaying messages, and user interactions.

src/snek/static/container.js

@@ -0,0 +1,114 @@
+// retoor <retoor@molodetz.nl>
+
+import { app } from "./app.js";
+import { EventHandler } from "./event-handler.js";
+
+export class Container extends EventHandler{ 
+    status = "unknown"
+    cpus = 0
+    memory = "0m"
+    image = "unknown:unknown"
+    name = null 
+    channelUid = null 
+    log = false 
+    bytesSent = 0
+    bytesReceived = 0
+    _container = null
+    render(el){
+        if(this._container == null){
+        this._container = el
+        this.terminal.open(this._container)
+        
+
+        this.terminal.onData(data => {
+            if (this.ws && this.ws.readyState === WebSocket.OPEN) {
+                this.ws.send(data);
+            }
+        });
+            
+        }
+        //this.refresh()
+        this.terminal.focus()
+    }
+    refresh(){
+        //this._fitAddon.fit();
+        this.ws.send("");
+    }
+    toggle(){
+        this._container.classList.toggle("hidden")
+        this.refresh()
+    }
+    fit(){
+        this._fitAddon.fit();
+    
+    }
+    constructor(channelUid,log){
+        super()
+        this.terminal = new Terminal({ cursorBlink: true ,theme: {
+    background: 'rgba(0, 0, 0, 0)', // Fully transparent
+  }
+ });
+        this._fitAddon = new FitAddon.FitAddon();
+        this.terminal.loadAddon(this._fitAddon);
+        window.addEventListener("resize", () => this._fitAddon.fit());
+        this.log = log ? true : false 
+        this.channelUid = channelUid    
+        this.update()
+        this.addEventListener("stdout", (data) => {
+            this.bytesReceived += data.length
+            if(this.log){
+                console.log(`Container ${this.name}: ${data}`)
+            }
+            const fixedData = new Uint8Array(data);
+            this.terminal.write(new TextDecoder().decode(fixedData));
+
+        })
+        
+        const protocol = window.location.protocol === 'https:' ? 'wss:' : 'ws:';
+        const host = window.location.host;
+        const wsUrl = `${protocol}//${host}/container/sock/${this.channelUid}.json`;
+        this.ws = new WebSocket(wsUrl);
+         
+
+
+
+        this.ws.binaryType = "arraybuffer";  // Support binary data
+        this.ws.onmessage = (event) => {
+            this.emit("stdout", event.data)
+        }
+        this.ws.onopen = () => {
+            this.refresh()
+        }
+        window.container = this 
+
+    }
+    async start(){
+        const result = await app.rpc.startContainer(this.channelUid)
+        await this.refresh()
+        return result && this.status == 'running'
+    }
+    async stop(){
+        const result = await app.rpc.stopContainer(this.channelUid)
+        await this.refresh()
+        return result && this.status == 'stopped'
+    }
+    async write(data){
+        await this.ws.send(data)
+        this.bytesSent += data.length
+        return true
+    }
+    async update(){
+        
+        const container = await app.rpc.getContainer(this.channelUid)
+        this.status = container["status"]
+        this.cpus = container["cpus"]
+        this.memory = container["memory"]
+        this.image = container["image"]
+        this.name = container["name"]
+    }
+
+}
+/*
+window.getContainer = function(){
+    return new Container(app.channelUid)
+}*/

src/snek/static/dumb-term.js

@@ -1,13 +1,34 @@
-class DumbTerminal extends HTMLElement {
-  constructor() {
-    super();
-    this.attachShadow({ mode: "open" });
-  }
+// retoor <retoor@molodetz.nl>
+
+import { NjetComponent } from "/njet.js";
+class WebTerminal extends NjetComponent {
+   
+    commands = {
+      "clear": () => {
+        this.outputEl.innerHTML = "";
+        return "";
+      },
+      "help": () => {
+        return "Available commands: help, clear, date";
+      },
+      "date": () => {
+        return new Date().toString();
+      }
+    };
+    help = {
+      "clear": "Clear the terminal",
+      "help": "Show available commands",
+      "date": "Show the current date"
+    };
+
+
+
 
   connectedCallback() {
-    this.shadowRoot.innerHTML = `
+
+    this.innerHTML = `
       <style>
-        :host {
+        .web-terminal {
           --terminal-bg: #111;
           --terminal-fg: #0f0;
           --terminal-accent: #0ff;
@@ -23,7 +44,7 @@ class DumbTerminal extends HTMLElement {
           max-height: 500px;
         }
 
-        .output {
+        .web-terminal-output {
           white-space: pre-wrap;
           margin-bottom: 1em;
         }
@@ -37,7 +58,7 @@ class DumbTerminal extends HTMLElement {
           margin-right: 0.5em;
         }
 
-        input {
+        .web-terminal-input {
           background: transparent;
           border: none;
           color: var(--terminal-fg);
@@ -57,21 +78,22 @@ class DumbTerminal extends HTMLElement {
           padding: 2rem;
         }
       </style>
-
-      <div class="output" id="output"></div>
+      <div class="web-terminal">
+      <div class="web-terminal-output"  id="output"></div>
       <div class="input-line">
-        <span class="prompt">&gt;</span>
-        <input type="text" id="input" autocomplete="off" autofocus />
+        <span  class="prompt">&gt;</span>
+        <input type="text" class="web-terminal-input" id="input" autocomplete="off" autofocus />
+      </div>
       </div>
     `;
-
-    this.outputEl = this.shadowRoot.getElementById("output");
-    this.inputEl = this.shadowRoot.getElementById("input");
+    this.container = this.querySelector(".web-terminal");
+    this.outputEl = this.querySelector(".web-terminal-output");
+    this.inputEl = this.querySelector(".web-terminal-input");
 
     this.history = [];
     this.historyIndex = -1;
 
-    this.inputEl.addEventListener("keydown", (e) => this.onKeyDown(e));
+    this.inputEl.addEventListener("keyup", (e) => this.onKeyDown(e));
   }
 
   onKeyDown(event) {
@@ -116,17 +138,89 @@ class DumbTerminal extends HTMLElement {
     this.outputEl.scrollTop = this.outputEl.scrollHeight;
   }
 
+  parseCommand(input) {
+    const args = [];
+    let current = '';
+    let inSingleQuote = false;
+    let inDoubleQuote = false;
+    let inTemplate = false; // For ${...}
+    let templateBuffer = '';
+
+    for (let i = 0; i < input.length; i++) {
+      const char = input[i];
+
+      // Handle template expressions
+      if (!inSingleQuote && !inDoubleQuote && char === '$' && input[i + 1] === '{') {
+        inTemplate = true;
+        i++; // Skip '{'
+        templateBuffer = '${';
+        continue;
+      }
+
+      if (inTemplate) {
+        templateBuffer += char;
+        if (char === '}') {
+          // End of template
+          args.push(eval(templateBuffer));
+          inTemplate = false;
+          templateBuffer = '';
+        }
+        continue; // Continue to next char
+      }
+
+      // Handle quotes
+      if (char === "'" && !inDoubleQuote) {
+        inSingleQuote = !inSingleQuote;
+        continue; // Skip quote
+      }
+
+      if (char === '"' && !inSingleQuote) {
+        inDoubleQuote = !inDoubleQuote;
+        continue; // Skip quote
+      }
+
+      // Handle spaces outside quotes and templates
+      if (char === ' ' && !inSingleQuote && !inDoubleQuote) {
+        if (current.length > 0) {
+          args.push(current);
+          current = '';
+        }
+      } else {
+        current += char;
+      }
+    }
+
+    if (current.length > 0) {
+      args.push(current);
+    }
+
+    return args;
+  } 
+  executeScript(script) {
+      const scriptElement = document.createElement("script");
+      scriptElement.textContent = script;
+      this.appendChild(scriptElement);
+  }
   mockExecute(command) {
-    switch (command.trim()) {
-      case "help":
-        return "Available commands: help, clear, date";
-      case "date":
-        return new Date().toString();
-      case "clear":
-        this.outputEl.innerHTML = "";
-        return "";
-      default:
-        return `Unknown command: ${command}`;
+    let args;
+    try {
+      args = this.parseCommand(command);
+    } catch (e) {
+      return e.toString();
+    }
+      console.info({ef:this})
+      console.info({af:this})
+      console.info({gf:args})
+    let cmdName = args.shift();
+    let commandHandler = this.commands[cmdName];
+    if (commandHandler) {
+      return commandHandler.apply(this, args);
+    }
+    try {
+      // Try to eval as JS
+      return this.executeScript(command);
+    } catch (e) {
+      return e.toString();
     }
   }
 
@@ -134,7 +228,7 @@ class DumbTerminal extends HTMLElement {
    * Static method to create a modal dialog with the terminal
    * @returns {HTMLDialogElement}
    */
-  static createModal() {
+  show() {
     const dialog = document.createElement("dialog");
     dialog.innerHTML = `
       <div class="dialog-backdrop">
@@ -147,4 +241,11 @@ class DumbTerminal extends HTMLElement {
   }
 }
 
+window.showTerm = function (options) {
+  const term = new WebTerminal(options);
+  term.show();
+  return term;
+}
+
 customElements.define("web-terminal", WebTerminal);
+export { WebTerminal };

src/snek/static/editor.js

@@ -0,0 +1,499 @@
+// retoor <retoor@molodetz.nl>
+
+import { NjetComponent } from "/njet.js"
+
+class NjetEditor extends NjetComponent {
+  constructor() {
+    super();
+    this.attachShadow({ mode: 'open' });
+
+    const style = document.createElement('style');
+    style.textContent = `
+      :host {
+        display: block;
+        position: relative;
+        height: 100%;
+        font-family: 'Monaco', 'Menlo', 'Ubuntu Mono', monospace;
+      }
+      
+      #editor {
+        padding: 1rem;
+        outline: none;
+        white-space: pre-wrap;
+        line-height: 1.5;
+        height: calc(100% - 30px);
+        overflow-y: auto;
+        background: #1e1e1e;
+        color: #d4d4d4;
+        font-size: 14px;
+        caret-color: #fff;
+      }
+      
+      #editor.insert-mode {
+        caret-color: #4ec9b0;
+      }
+      
+      #editor.visual-mode {
+        caret-color: #c586c0;
+      }
+      
+      #editor::selection {
+        background: #264f78;
+      }
+      
+      #status-bar {
+        position: absolute;
+        bottom: 0;
+        left: 0;
+        right: 0;
+        height: 30px;
+        background: #007acc;
+        color: #fff;
+        display: flex;
+        align-items: center;
+        padding: 0 1rem;
+        font-size: 12px;
+        font-weight: 500;
+      }
+      
+      #mode-indicator {
+        text-transform: uppercase;
+        margin-right: 20px;
+        font-weight: bold;
+      }
+      
+      #command-line {
+        position: absolute;
+        bottom: 30px;
+        left: 0;
+        width: 100%;
+        padding: 0.3rem 1rem;
+        background: #2d2d2d;
+        color: #d4d4d4;
+        display: none;
+        font-family: inherit;
+        border-top: 1px solid #3e3e3e;
+        font-size: 14px;
+      }
+      
+      #command-input {
+        background: transparent;
+        border: none;
+        color: inherit;
+        outline: none;
+        font-family: inherit;
+        font-size: inherit;
+        width: calc(100% - 20px);
+      }
+      
+      .visual-selection {
+        background: #264f78 !important;
+      }
+    `;
+
+    this.editor = document.createElement('div');
+    this.editor.id = 'editor';
+    this.editor.contentEditable = true;
+    this.editor.spellcheck = false;
+    this.editor.innerText = `Welcome to VimEditor Component
+Line 2 here
+Another line
+Try i, Esc, v, :, yy, dd, 0, $, gg, G, and p`;
+
+    this.cmdLine = document.createElement('div');
+    this.cmdLine.id = 'command-line';
+    
+    const cmdPrompt = document.createElement('span');
+    cmdPrompt.textContent = ':';
+    
+    this.cmdInput = document.createElement('input');
+    this.cmdInput.id = 'command-input';
+    this.cmdInput.type = 'text';
+    
+    this.cmdLine.append(cmdPrompt, this.cmdInput);
+
+    this.statusBar = document.createElement('div');
+    this.statusBar.id = 'status-bar';
+    
+    this.modeIndicator = document.createElement('span');
+    this.modeIndicator.id = 'mode-indicator';
+    this.modeIndicator.textContent = 'NORMAL';
+    
+    this.statusBar.appendChild(this.modeIndicator);
+
+    this.shadowRoot.append(style, this.editor, this.cmdLine, this.statusBar);
+
+    this.mode = 'normal';
+    this.keyBuffer = '';
+    this.lastDeletedLine = '';
+    this.yankedLine = '';
+    this.visualStartOffset = null;
+    this.visualEndOffset = null;
+
+    // Bind event handlers
+    this.handleKeydown = this.handleKeydown.bind(this);
+    this.handleCmdKeydown = this.handleCmdKeydown.bind(this);
+    this.updateVisualSelection = this.updateVisualSelection.bind(this);
+    
+    this.editor.addEventListener('keydown', this.handleKeydown);
+    this.cmdInput.addEventListener('keydown', this.handleCmdKeydown);
+    this.editor.addEventListener('beforeinput', this.handleBeforeInput.bind(this));
+  }
+
+  connectedCallback() {
+    this.editor.focus();
+  }
+
+  setMode(mode) {
+    this.mode = mode;
+    this.modeIndicator.textContent = mode.toUpperCase();
+    
+    // Update editor classes
+    this.editor.classList.remove('insert-mode', 'visual-mode', 'normal-mode');
+    this.editor.classList.add(`${mode}-mode`);
+    
+    if (mode === 'visual') {
+      this.visualStartOffset = this.getCaretOffset();
+      this.editor.addEventListener('selectionchange', this.updateVisualSelection);
+    } else {
+      this.clearVisualSelection();
+      this.editor.removeEventListener('selectionchange', this.updateVisualSelection);
+    }
+    
+    if (mode === 'command') {
+      this.cmdLine.style.display = 'block';
+      this.cmdInput.value = '';
+      this.cmdInput.focus();
+    } else {
+      this.cmdLine.style.display = 'none';
+      if (mode !== 'insert') {
+        // Keep focus on editor for all non-insert modes
+        this.editor.focus();
+      }
+    }
+  }
+
+  updateVisualSelection() {
+    if (this.mode !== 'visual') return;
+    this.visualEndOffset = this.getCaretOffset();
+  }
+
+  clearVisualSelection() {
+    const sel = this.shadowRoot.getSelection();
+    if (sel) sel.removeAllRanges();
+    this.visualStartOffset = null;
+    this.visualEndOffset = null;
+  }
+
+  getCaretOffset() {
+    const sel = this.shadowRoot.getSelection();
+    if (!sel || sel.rangeCount === 0) return 0;
+
+    const range = sel.getRangeAt(0);
+    const preCaretRange = range.cloneRange();
+    preCaretRange.selectNodeContents(this.editor);
+    preCaretRange.setEnd(range.endContainer, range.endOffset);
+    return preCaretRange.toString().length;
+  }
+
+  setCaretOffset(offset) {
+    const textContent = this.editor.innerText;
+    offset = Math.max(0, Math.min(offset, textContent.length));
+    
+    const range = document.createRange();
+    const sel = this.shadowRoot.getSelection();
+    const walker = document.createTreeWalker(
+      this.editor, 
+      NodeFilter.SHOW_TEXT, 
+      null, 
+      false
+    );
+
+    let currentOffset = 0;
+    let node;
+    
+    while ((node = walker.nextNode())) {
+      const nodeLength = node.textContent.length;
+      if (currentOffset + nodeLength >= offset) {
+        range.setStart(node, offset - currentOffset);
+        range.collapse(true);
+        sel.removeAllRanges();
+        sel.addRange(range);
+        return;
+      }
+      currentOffset += nodeLength;
+    }
+    
+    // If we couldn't find the position, set to end
+    if (this.editor.lastChild) {
+      range.selectNodeContents(this.editor.lastChild);
+      range.collapse(false);
+      sel.removeAllRanges();
+      sel.addRange(range);
+    }
+  }
+
+  handleBeforeInput(e) {
+    if (this.mode !== 'insert') {
+      e.preventDefault();
+    }
+  }
+
+  handleCmdKeydown(e) {
+    if (e.key === 'Enter') {
+      e.preventDefault();
+      this.executeCommand(this.cmdInput.value);
+      this.setMode('normal');
+    } else if (e.key === 'Escape') {
+      e.preventDefault();
+      this.setMode('normal');
+    }
+  }
+
+  executeCommand(cmd) {
+    const trimmedCmd = cmd.trim();
+    
+    // Handle basic vim commands
+    if (trimmedCmd === 'w' || trimmedCmd === 'write') {
+      console.log('Save command (not implemented)');
+    } else if (trimmedCmd === 'q' || trimmedCmd === 'quit') {
+      console.log('Quit command (not implemented)');
+    } else if (trimmedCmd === 'wq' || trimmedCmd === 'x') {
+      console.log('Save and quit command (not implemented)');
+    } else if (/^\d+$/.test(trimmedCmd)) {
+      // Go to line number
+      const lineNum = parseInt(trimmedCmd, 10) - 1;
+      this.goToLine(lineNum);
+    }
+  }
+
+  goToLine(lineNum) {
+    const lines = this.editor.innerText.split('\n');
+    if (lineNum < 0 || lineNum >= lines.length) return;
+    
+    let offset = 0;
+    for (let i = 0; i < lineNum; i++) {
+      offset += lines[i].length + 1;
+    }
+    this.setCaretOffset(offset);
+  }
+
+  getCurrentLineInfo() {
+    const text = this.editor.innerText;
+    const caretPos = this.getCaretOffset();
+    const lines = text.split('
+');
+    
+    let charCount = 0;
+    for (let i = 0; i < lines.length; i++) {
+      if (caretPos <= charCount + lines[i].length) {
+        return {
+          lineIndex: i,
+          lines: lines,
+          lineStartOffset: charCount,
+          positionInLine: caretPos - charCount
+        };
+      }
+      charCount += lines[i].length + 1;
+    }
+    
+    return {
+      lineIndex: lines.length - 1,
+      lines: lines,
+      lineStartOffset: charCount - lines[lines.length - 1].length - 1,
+      positionInLine: 0
+    };
+  }
+
+  handleKeydown(e) {
+    if (this.mode === 'insert') {
+      if (e.key === 'Escape') {
+        e.preventDefault();
+        this.setMode('normal');
+        // Move cursor one position left (vim behavior)
+        const offset = this.getCaretOffset();
+        if (offset > 0) {
+          this.setCaretOffset(offset - 1);
+        }
+      }
+      return;
+    }
+
+    if (this.mode === 'command') {
+      return; // Command mode input is handled by cmdInput
+    }
+
+    if (this.mode === 'visual') {
+      if (e.key === 'Escape') {
+        e.preventDefault();
+        this.setMode('normal');
+        return;
+      }
+      
+      // Allow movement in visual mode
+      if (['ArrowLeft', 'ArrowRight', 'ArrowUp', 'ArrowDown'].includes(e.key)) {
+        return; // Let default behavior handle selection
+      }
+      
+      if (e.key === 'y') {
+        e.preventDefault();
+        // Yank selected text
+        const sel = this.shadowRoot.getSelection();
+        if (sel && sel.rangeCount > 0) {
+          this.yankedLine = sel.toString();
+        }
+        this.setMode('normal');
+        return;
+      }
+      
+      if (e.key === 'd' || e.key === 'x') {
+        e.preventDefault();
+        // Delete selected text
+        const sel = this.shadowRoot.getSelection();
+        if (sel && sel.rangeCount > 0) {
+          this.lastDeletedLine = sel.toString();
+          document.execCommand('delete');
+        }
+        this.setMode('normal');
+        return;
+      }
+    }
+
+    // Normal mode handling
+    e.preventDefault();
+    
+    // Special keys that should be handled immediately
+    if (e.key === 'Escape') {
+      this.keyBuffer = '';
+      this.setMode('normal');
+      return;
+    }
+    
+    // Build key buffer for commands
+    this.keyBuffer += e.key;
+    
+    const lineInfo = this.getCurrentLineInfo();
+    const { lineIndex, lines, lineStartOffset, positionInLine } = lineInfo;
+
+    // Process commands
+    switch (this.keyBuffer) {
+      case 'i':
+        this.keyBuffer = '';
+        this.setMode('insert');
+        break;
+
+      case 'a':
+        this.keyBuffer = '';
+        this.setCaretOffset(this.getCaretOffset() + 1);
+        this.setMode('insert');
+        break;
+
+      case 'v':
+        this.keyBuffer = '';
+        this.setMode('visual');
+        break;
+
+      case ':':
+        this.keyBuffer = '';
+        this.setMode('command');
+        break;
+
+      case 'yy':
+        this.keyBuffer = '';
+        this.yankedLine = lines[lineIndex];
+        break;
+
+      case 'dd':
+        this.keyBuffer = '';
+        this.lastDeletedLine = lines[lineIndex];
+        lines.splice(lineIndex, 1);
+        if (lines.length === 0) lines.push('');
+        this.editor.innerText = lines.join('
+');
+        this.setCaretOffset(lineStartOffset);
+        break;
+
+      case 'p':
+        this.keyBuffer = '';
+        const lineToPaste = this.yankedLine || this.lastDeletedLine;
+        if (lineToPaste) {
+          lines.splice(lineIndex + 1, 0, lineToPaste);
+          this.editor.innerText = lines.join('
+');
+          this.setCaretOffset(lineStartOffset + lines[lineIndex].length + 1);
+        }
+        break;
+
+      case '0':
+        this.keyBuffer = '';
+        this.setCaretOffset(lineStartOffset);
+        break;
+
+      case '$':
+        this.keyBuffer = '';
+        this.setCaretOffset(lineStartOffset + lines[lineIndex].length);
+        break;
+
+      case 'gg':
+        this.keyBuffer = '';
+        this.setCaretOffset(0);
+        break;
+
+      case 'G':
+        this.keyBuffer = '';
+        this.setCaretOffset(this.editor.innerText.length);
+        break;
+
+      case 'h':
+      case 'ArrowLeft':
+        this.keyBuffer = '';
+        const currentOffset = this.getCaretOffset();
+        if (currentOffset > 0) {
+          this.setCaretOffset(currentOffset - 1);
+        }
+        break;
+
+      case 'l':
+      case 'ArrowRight':
+        this.keyBuffer = '';
+        this.setCaretOffset(this.getCaretOffset() + 1);
+        break;
+
+      case 'j':
+      case 'ArrowDown':
+        this.keyBuffer = '';
+        if (lineIndex < lines.length - 1) {
+          const nextLineStart = lineStartOffset + lines[lineIndex].length + 1;
+          const nextLineLength = lines[lineIndex + 1].length;
+          const newPosition = Math.min(positionInLine, nextLineLength);
+          this.setCaretOffset(nextLineStart + newPosition);
+        }
+        break;
+
+      case 'k':
+      case 'ArrowUp':
+        this.keyBuffer = '';
+        if (lineIndex > 0) {
+          let prevLineStart = 0;
+          for (let i = 0; i < lineIndex - 1; i++) {
+            prevLineStart += lines[i].length + 1;
+          }
+          const prevLineLength = lines[lineIndex - 1].length;
+          const newPosition = Math.min(positionInLine, prevLineLength);
+          this.setCaretOffset(prevLineStart + newPosition);
+        }
+        break;
+
+      default:
+        // Clear buffer if it gets too long or contains invalid sequences
+        if (this.keyBuffer.length > 2 || 
+            (this.keyBuffer.length === 2 && !['dd', 'yy', 'gg'].includes(this.keyBuffer))) {
+          this.keyBuffer = '';
+        }
+        break;
+    }
+  }
+}
+
+customElements.define('njet-editor', NjetEditor);
+export { NjetEditor }

src/snek/static/event-handler.js

@@ -1,15 +1,178 @@
+// retoor <retoor@molodetz.nl>
+
 export class EventHandler {
   constructor() {
     this.subscribers = {};
+    this._maxListeners = 100;
+    this._warnOnMaxListeners = true;
   }
 
-  addEventListener(type, handler) {
-    if (!this.subscribers[type]) this.subscribers[type] = [];
-    this.subscribers[type].push(handler);
+  addEventListener(type, handler, options = {}) {
+    if (!type || typeof type !== "string") {
+      console.warn("EventHandler: Invalid event type");
+      return false;
+    }
+    if (!handler || typeof handler !== "function") {
+      console.warn("EventHandler: Invalid handler");
+      return false;
+    }
+    try {
+      const once = options && options.once === true;
+      if (!this.subscribers[type]) {
+        this.subscribers[type] = [];
+      }
+      if (this._warnOnMaxListeners && this.subscribers[type].length >= this._maxListeners) {
+        console.warn(`EventHandler: Max listeners (${this._maxListeners}) reached for event "${type}"`);
+      }
+      let wrappedHandler = handler;
+      if (once) {
+        const originalHandler = handler;
+        wrappedHandler = (...args) => {
+          try {
+            originalHandler(...args);
+          } catch (e) {
+            console.error(`EventHandler: Error in once handler for "${type}":`, e);
+          } finally {
+            this.removeEventListener(type, wrappedHandler);
+          }
+        };
+        wrappedHandler._original = originalHandler;
+      }
+      this.subscribers[type].push(wrappedHandler);
+      return true;
+    } catch (e) {
+      console.error("EventHandler: addEventListener error:", e);
+      return false;
+    }
   }
 
   emit(type, ...data) {
-    if (this.subscribers[type])
-      this.subscribers[type].forEach((handler) => handler(...data));
+    if (!type || typeof type !== "string") {
+      return false;
+    }
+    try {
+      const handlers = this.subscribers[type];
+      if (!handlers || !Array.isArray(handlers) || handlers.length === 0) {
+        return false;
+      }
+      const handlersCopy = [...handlers];
+      for (const handler of handlersCopy) {
+        if (typeof handler !== "function") {
+          continue;
+        }
+        try {
+          handler(...data);
+        } catch (e) {
+          console.error(`EventHandler: Error in handler for "${type}":`, e);
+        }
+      }
+      return true;
+    } catch (e) {
+      console.error("EventHandler: emit error:", e);
+      return false;
+    }
+  }
+
+  removeEventListener(type, handler) {
+    if (!type || typeof type !== "string") {
+      return false;
+    }
+    try {
+      if (!this.subscribers[type]) {
+        return false;
+      }
+      if (!handler) {
+        delete this.subscribers[type];
+        return true;
+      }
+      const originalLength = this.subscribers[type].length;
+      this.subscribers[type] = this.subscribers[type].filter((h) => {
+        if (h === handler) return false;
+        if (h._original && h._original === handler) return false;
+        return true;
+      });
+      if (this.subscribers[type].length === 0) {
+        delete this.subscribers[type];
+      }
+      return this.subscribers[type] ? this.subscribers[type].length < originalLength : true;
+    } catch (e) {
+      console.error("EventHandler: removeEventListener error:", e);
+      return false;
+    }
+  }
+
+  removeAllEventListeners(type) {
+    try {
+      if (type) {
+        if (this.subscribers[type]) {
+          delete this.subscribers[type];
+          return true;
+        }
+        return false;
+      }
+      this.subscribers = {};
+      return true;
+    } catch (e) {
+      console.error("EventHandler: removeAllEventListeners error:", e);
+      return false;
+    }
+  }
+
+  hasEventListener(type, handler) {
+    if (!type || typeof type !== "string") {
+      return false;
+    }
+    try {
+      if (!this.subscribers[type]) {
+        return false;
+      }
+      if (!handler) {
+        return this.subscribers[type].length > 0;
+      }
+      return this.subscribers[type].some((h) => {
+        if (h === handler) return true;
+        if (h._original && h._original === handler) return true;
+        return false;
+      });
+    } catch (e) {
+      return false;
+    }
+  }
+
+  getEventListenerCount(type) {
+    try {
+      if (type) {
+        return this.subscribers[type] ? this.subscribers[type].length : 0;
+      }
+      let count = 0;
+      for (const key in this.subscribers) {
+        if (Object.prototype.hasOwnProperty.call(this.subscribers, key)) {
+          count += this.subscribers[key].length;
+        }
+      }
+      return count;
+    } catch (e) {
+      return 0;
+    }
+  }
+
+  getEventTypes() {
+    try {
+      return Object.keys(this.subscribers);
+    } catch (e) {
+      return [];
+    }
+  }
+
+  once(type, handler) {
+    return this.addEventListener(type, handler, { once: true });
+  }
+
+  off(type, handler) {
+    return this.removeEventListener(type, handler);
+  }
+
+  on(type, handler) {
+    return this.addEventListener(type, handler);
   }
 }

src/snek/static/fancy-button.js

@@ -1,3 +1,5 @@
+// retoor <retoor@molodetz.nl>
+
 // Written by retoor@molodetz.nl
 
 // This JavaScript class defines a custom HTML element <fancy-button>, which creates a styled, clickable button element with customizable size, text, and URL redirect functionality.
@@ -28,21 +30,30 @@ class FancyButton extends HTMLElement {
         button {
             width: var(--width);
             min-width: ${size};
-            padding: 10px;
-            background-color: #f05a28;
-            border: none;
-            border-radius: 5px;
-            color: white;
-            font-size: 1em;
-            font-weight: bold;
+            padding: 10px 20px;
+            background-color: #1a1a1a;
+            border: 1px solid #333;
+            border-radius: 4px;
+            color: #e6e6e6;
+            font-family: 'Courier New', monospace;
+            font-size: 14px;
+            font-weight: 500;
             cursor: pointer;
-            transition: background-color 0.3s;
-            border: 1px solid #f05a28;
+            transition: all 0.2s ease;
         }
         button:hover {
-            color: #EFEFEF;
+            background-color: #2a2a2a;
+            border-color: #444;
+            color: #fff;
+        }
+        button.primary {
+            background-color: #f05a28;
+            border-color: #f05a28;
+            color: #fff;
+        }
+        button.primary:hover {
             background-color: #e04924;
-            border: 1px solid #efefef;
+            border-color: #e04924;
         }
         `;
 

src/snek/static/file-manager.js

@@ -1,4 +1,8 @@
+// retoor <retoor@molodetz.nl>
+
 /* A <file-browser> custom element that talks to /api/files */
+import { NjetComponent } from "/njet.js";
+
 class FileBrowser extends HTMLElement {
   constructor() {
     super();
@@ -6,9 +10,11 @@ class FileBrowser extends HTMLElement {
     this.path = ""; // current virtual path ("" = ROOT)
     this.offset = 0; // pagination offset
     this.limit = 40; // items per request
+    this.url = '/drive.json' 
   }
 
   connectedCallback() {
+    this.url = this.getAttribute("url") || this.url;
     this.path = this.getAttribute("path") || "";
     this.renderShell();
     this.load();
@@ -58,7 +64,7 @@ class FileBrowser extends HTMLElement {
   // ---------- Networking ----------------------------------------------
   async load() {
     const r = await fetch(
-      `/drive.json?path=${encodeURIComponent(this.path)}&offset=${this.offset}&limit=${this.limit}`,
+      this.url + `?path=${encodeURIComponent(this.path)}&offset=${this.offset}&limit=${this.limit}`
     );
     if (!r.ok) {
       console.error(await r.text());

src/snek/static/file-upload-grid.css

@@ -0,0 +1,68 @@
+.fug-root {
+    background: #111;
+    border: 1px solid #333;
+    border-radius: 8px;
+    padding: 8px;
+    margin: 0 15px 8px 15px;
+    color: white;
+    font-family: sans-serif;
+    max-width: calc(100% - 30px);
+}
+.fug-grid {
+    display: flex;
+    flex-wrap: wrap;
+    gap: 12px;
+    margin: 10px;
+    max-width: 100%;
+    overflow-x: auto;
+    justify-content: flex-start;
+}
+.fug-tile {
+    width: 120px;
+    flex-shrink: 0;
+    background: #111;
+    border: 2px solid #ff6600;
+    border-radius: 12px;
+    display: flex;
+    flex-direction: column;
+    align-items: center;
+    padding: 12px 8px 8px 8px;
+    box-shadow: 0 0 4px #333;
+    position: relative;
+}
+.fug-icon {
+    width: 48px; height: 48px;
+    margin-bottom: 8px;
+    object-fit: cover;
+    background: #222;
+    border-radius: 5px;
+}
+.fug-filename {
+    word-break: break-all;
+    margin-bottom: 8px;
+    font-size: 0.95em;
+    text-align: center;
+}
+.fug-progressbar {
+    width: 100%;
+    height: 7px;
+    background: #333;
+    border-radius: 4px;
+    overflow: hidden;
+    margin-top: auto;
+}
+.fug-progress {
+    height: 100%;
+    background: #ffb200;
+    transition: width 0.15s;
+    width: 0%;
+}
+.fug-tile.fug-done {
+    border-color: #0f0;
+}
+.fug-fileinput {
+    margin: 24px;
+    font-size: 1.1em;
+    display: none;
+}
+

src/snek/static/file-upload-grid.js

@@ -0,0 +1,182 @@
+// retoor <retoor@molodetz.nl>
+
+import { NjetComponent, NjetDialog } from '/njet.js';
+
+const FUG_ICONS = {
+    file: 'data:image/svg+xml;utf8,<svg xmlns="http://www.w3.org/2000/svg" width="48" height="48"><rect x="8" y="8" width="32" height="40" rx="5" fill="white" stroke="%234af" stroke-width="2"/></svg>'
+};
+
+class FileUploadGrid extends NjetComponent {
+    constructor() {
+        super();
+        this._grid = null;
+        this._fileInput = null;
+        this.channelUid = null ;
+        this.uploadsDone = 0;
+        this.uploadsStarted = 0;
+        this.uploadResponses = [];
+    }
+    
+  openFileDialog() {
+        if(this.isBusy){
+            const dialog = new NjetDialog({
+                title: 'Upload in progress',
+                content: 'Please wait for the current upload to complete.',
+                primaryButton: {
+                    text: 'OK',
+                    handler: function () {
+                        this.closest('njet-dialog').remove();
+                    }
+                }
+            })
+            dialog.open();
+            return false 
+        }
+
+        if (this._fileInput) {
+            this._fileInput.value = ''; // Allow same file selection twice
+            this._fileInput.click();
+        }
+      return true;
+    }
+
+    render() {
+        // Root wrapper for styling
+        this.classList.add('fug-root');
+        // Clear previous (if rerendered)
+        this.innerHTML = '';
+        // Input
+        this._fileInput = document.createElement('input');
+        this._fileInput.type = 'file';
+        this._fileInput.className = 'fug-fileinput';
+        this._fileInput.multiple = true;
+        // Grid
+        this._grid = document.createElement('div');
+        this._grid.className = 'fug-grid';
+        this.appendChild(this._fileInput);
+        this.appendChild(this._grid);
+        this.uploadsDone = 0;
+        this.uploadsStarted = 0;
+    }
+
+    reset(){
+        this.uploadResponses = [];
+        this.uploadsDone = 0;
+        this.uploadsStarted = 0;
+        this._grid.innerHTML = '';
+    }
+
+    get isBusy(){
+        return this.uploadsDone != this.uploadsStarted;
+    }
+
+    handleFiles(files) {
+        this.reset()
+        this.uploadsDone = 0;
+        this.uploadsStarted = files.length;
+        
+        [...files].forEach(file => this.createTile(file));
+    }
+    connectedCallback() {
+        this.channelUid = this.getAttribute('channel');
+        this.render();
+        this._fileInput.addEventListener('change', e => this.handleFiles(e.target.files));
+    }
+
+    createTile(file) {
+        const tile = document.createElement('div');
+        tile.className = 'fug-tile';
+        // Icon/Thumbnail
+        let icon;
+        if (file.type.startsWith('image/')) {
+            icon = document.createElement('img');
+            icon.className = 'fug-icon';
+            icon.src = URL.createObjectURL(file);
+            icon.onload = () => URL.revokeObjectURL(icon.src);
+        } else if (file.type.startsWith('video/')) {
+            icon = document.createElement('video');
+            icon.className = 'fug-icon';
+            icon.src = URL.createObjectURL(file);
+            icon.muted = true;
+            icon.playsInline = true;
+            icon.controls = false;
+            icon.preload = 'metadata';
+            icon.onloadeddata = () => {
+                icon.currentTime = 0.5;
+                URL.revokeObjectURL(icon.src);
+            };
+        } else {
+            icon = document.createElement('img');
+            icon.className = 'fug-icon';
+            icon.src = FUG_ICONS.file;
+        }
+        // Filename
+        const name = document.createElement('div');
+        name.className = 'fug-filename';
+        name.textContent = file.name;
+        // Progressbar
+        const progressbar = document.createElement('div');
+        progressbar.className = 'fug-progressbar';
+        const progress = document.createElement('div');
+        progress.className = 'fug-progress';
+        progressbar.appendChild(progress);
+
+        // Tile composition
+        tile.appendChild(icon);
+        tile.appendChild(name);
+        tile.appendChild(progressbar);
+        this._grid.appendChild(tile);
+        
+        // Start upload
+        this.startUpload(file, tile, progress);
+
+    }
+
+    startUpload(file, tile, progress) {
+
+        this.publish('file-uploading', {file: file, tile: tile, progress: progress});
+        console.info("File uploading",file)
+
+        const protocol = location.protocol === "https:" ? "wss://" : "ws://";
+        const ws = new WebSocket(`${protocol}${location.host}/channel/${this.channelUid}/attachment.sock`);
+            ws.binaryType = 'arraybuffer';
+        let sent = 0;
+
+        ws.onopen = async () => {
+            ws.send(JSON.stringify({type: 'start', filename: file.name}));
+            const chunkSize = 64*1024;
+            while (sent < file.size) {
+                const chunk = file.slice(sent, sent + chunkSize);
+                ws.send(await chunk.arrayBuffer());
+                sent += chunkSize;
+            }
+            ws.send(JSON.stringify({type: 'end', filename: file.name}));
+        };
+
+        ws.onmessage = (event) => {
+            const data = JSON.parse(event.data);
+            if (data.type === 'progress') {
+                const pct = Math.min(100, Math.round(100 * data.bytes / file.size));
+                progress.style.width = pct + '%';
+                this.publish('file-uploading', {file: file, tile: tile, progress: progress});
+            } else if (data.type === 'done') {
+                this.uploadsDone += 1;
+                this.publish('file-uploaded', {file: file, tile: tile, progress: progress});
+                progress.style.width = '100%';
+                tile.classList.add('fug-done');
+                ws.close();
+                this.uploadResponses.push({file:file, remoteFile:data.file})
+                if(this.uploadsDone == this.uploadsStarted){
+                    this.publish('file-uploads-done', this.uploadResponses);
+                    console.info({"X":this.uploadResponses})
+                    this.reset()
+                }
+
+            }
+        };
+    }
+}
+
+customElements.define('file-upload-grid', FileUploadGrid);
+
+export { FileUploadGrid };

src/snek/static/generic-form.css

@@ -2,11 +2,11 @@
     margin: 0;
     padding: 0;
     box-sizing: border-box;
-  }
-  
-  body {
-    font-family: Arial, sans-serif;
-    background-color: #1a1a1a;
+}
+
+body {
+    font-family: 'Courier New', monospace;
+    background-color: #0a0a0a;
     color: #e6e6e6;
     line-height: 1.5;
     display: flex;
@@ -15,61 +15,72 @@
     align-items: center;
     height: 100vh;
     width: 100vw;
-  }
+}
 
 generic-form {
     margin: 0;
     padding: 0;
     box-sizing: border-box;
     background-color: #000000;
-
 }
 
 .generic-form-container {
-    
-    background-color:  #0f0f0f;
+    background-color: #0f0f0f;
     border-radius: 10px;
     padding: 30px;
     box-shadow: 0 0 15px rgba(0, 0, 0, 0.5);
     text-align: center;
-
 }
+
 .generic-form-container h1 {
     font-size: 2em;
     color: #f05a28;
     margin-bottom: 20px;
-
 }
-input {
 
-    border: 10px solid #000000;
-}
 .generic-form-container generic-field {
     width: 100%;
-    padding: 10px;
+    padding: 10px 12px;
     margin: 10px 0;
     border: 1px solid #333;
-    border-radius: 5px;
-    background-color: #1a1a1a;
+    border-radius: 4px;
+    background-color: #0f0f0f;
     color: #e6e6e6;
-    font-size: 1em;
+    font-family: 'Courier New', monospace;
+    font-size: 14px;
 }
 
 .generic-form-container button {
     width: 100%;
-    padding: 10px;
-    background-color: #f05a28;
-    border: none;
-    border-radius: 5px;
-    color: white;
-    font-size: 1em;
-    font-weight: bold;
+    padding: 10px 20px;
+    background-color: #1a1a1a;
+    border: 1px solid #333;
+    border-radius: 4px;
+    color: #e6e6e6;
+    font-family: 'Courier New', monospace;
+    font-size: 14px;
+    font-weight: 500;
     cursor: pointer;
-    transition: background-color 0.3s;
+    transition: all 0.2s ease;
 }
 
 .generic-form-container button:hover {
+    background-color: #2a2a2a;
+    border-color: #444;
+    color: #fff;
+}
+
+.generic-form-container button[type="submit"],
+.generic-form-container button.primary {
+    background-color: #f05a28;
+    border-color: #f05a28;
+    color: #fff;
+}
+
+.generic-form-container button[type="submit"]:hover,
+.generic-form-container button.primary:hover {
     background-color: #e04924;
+    border-color: #e04924;
 }
 
 .generic-form-container a {
@@ -85,15 +96,13 @@ input {
     color: #e04924;
 }
 
-
 .error {
-    color: #d8000c;
-    font-size: 0.9em;
-    margin-top: 5px;
+    border-color: #8b0000;
+    box-shadow: 0 0 0 2px rgba(139, 0, 0, 0.2);
 }
 
-
 @media (max-width: 600px) {
     .generic-form-container {
+        width: 90%;
     }
 }

src/snek/static/generic-form.js

@@ -1,3 +1,5 @@
+// retoor <retoor@molodetz.nl>
+
 // Written by retoor@molodetz.nl
 
 // This code defines two custom HTML elements, `GenericField` and `GenericForm`. The `GenericField` element represents a form field with validation and styling functionalities, and the `GenericForm` fetches and manages form data, handling field validation and submission.
@@ -76,47 +78,66 @@ class GenericField extends HTMLElement {
 
       input {
         width: 90%;
-        padding: 10px;
+        padding: 10px 12px;
         margin: 10px 0;
         border: 1px solid #333;
-        border-radius: 5px;
-        background-color: #1a1a1a;
+        border-radius: 4px;
+        background-color: #0f0f0f;
         color: #e6e6e6;
-        font-size: 1em;
-        
+        font-family: 'Courier New', monospace;
+        font-size: 14px;
+        transition: border-color 0.2s ease, box-shadow 0.2s ease;
+
         &:focus {
-          outline: 2px solid #f05a28 !important;
+          outline: none;
+          border-color: #f05a28;
+          box-shadow: 0 0 0 2px rgba(240, 90, 40, 0.2);
         }
-        
+
         &::placeholder {
+          color: #555;
           transition: opacity 0.3s;
         }
-        
+
         &:focus::placeholder {
           opacity: 0.4;
         }
       }
-      
+
       button {
         width: 50%;
-        padding: 10px;
-        background-color: #f05a28;
-        border: none;
+        padding: 10px 20px;
+        background-color: #1a1a1a;
+        border: 1px solid #333;
         float: right;
         margin-top: 10px;
         margin-left: 10px;
         margin-right: 10px;
-        border-radius: 5px;
-        color: white;
-        font-size: 1em;
-        font-weight: bold;
+        border-radius: 4px;
+        color: #e6e6e6;
+        font-family: 'Courier New', monospace;
+        font-size: 14px;
+        font-weight: 500;
         cursor: pointer;
-        transition: background-color 0.3s;
+        transition: all 0.2s ease;
         clear: both;
       }
 
       button:hover {
+        background-color: #2a2a2a;
+        border-color: #444;
+        color: #fff;
+      }
+
+      button[value="submit"], button.primary {
+        background-color: #f05a28;
+        border-color: #f05a28;
+        color: #fff;
+      }
+
+      button[value="submit"]:hover, button.primary:hover {
         background-color: #e04924;
+        border-color: #e04924;
       }
 
       a {
@@ -133,17 +154,13 @@ class GenericField extends HTMLElement {
       }
 
       .valid {
-        border: 1px solid green;
-        color: green;
-        font-size: 0.9em;
-        margin-top: 5px;
+        border-color: #006400;
+        box-shadow: 0 0 0 2px rgba(0, 100, 0, 0.2);
       }
 
       .error {
-        border: 3px solid red;
-        color: #d8000c;
-        font-size: 0.9em;
-        margin-top: 5px;
+        border-color: #8b0000;
+        box-shadow: 0 0 0 2px rgba(139, 0, 0, 0.2);
       }
 
       @media (max-width: 500px) {

src/snek/static/html-frame.js

@@ -1,3 +1,5 @@
+// retoor <retoor@molodetz.nl>
+
 // Written by retoor@molodetz.nl
 
 // The following JavaScript code defines a custom HTML element `<html-frame>` that loads and displays HTML content from a specified URL. If the URL is provided as a markdown file, it attempts to render it as HTML.