Fix run command.

Browse Source
This commit is contained in:
retoor 2025-01-27 19:32:42 +01:00
parent 542ba6a3fc
commit 0dc2120cac
7 changed files with 242 additions and 107 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
Makefile
View File

@ -7,4 +7,4 @@ install:
$(PYTHON) -m pip install -e .
run:
$(PYTHON) main.py
$(PYTHON) -m dreamii

113
src/Dreamii.egg-info/PKG-INFO
View File

@ -16,37 +16,126 @@ Requires-Dist: cryptography
# Dreamii
The ultimate markdown CMS! Use the editors that you want! It can be serious article writing or just hacking something together live with VIM on a server. This CMS allows you to do both. For professionals and amateurs.
## Project description
The ultimate site builder! It's not one of the many CMS'es but its a template renderer. It renders dynamic and static content. Finally you can manage a site with the editor that you want! It can be serious article writing or just hacking something together live with VIM on a server. This project allows you to do both. Apply professional versioning / backupping of your site using a CVS. Since this system has a local database, that will be included in the CVS setup too. It's suitable for both small and big sites.
Also, consider this CMS for beginners. It's actually way easier than WYSIWYG and harder to destroy the site because everything style the markdown uses case be predefined by a designer. Anyone who knows HTML can still still switch to HTML. So you have a lot of freedom!
Also, consider Dreamii for beginners / customers. It's actually way easier than WYSIWYG. You advise the editor to the user. You can also write your own editor around it and use Dreamii only as renderer. Sky is the limit.
Writing a site using markdown is very convenient since styles are predefined. If you would use Dreamii for some one else, you can just predefine the styles so the user will keep maintaining the style. This is not the case with WYSIWYG enabled sites.
The concept is that primarely markdown will be used using Dreamii but optionally advanced features can be implemented using HTML and Python.
## Features
This project allows you to create a website using both of these technologies:
* Markdown
* HTML
This project allows you to create a site using all of these technologies:
* Jinja2 (v3.1.5+).
* Markdown.
* Syntax Highlighting within Markdown.
* HTML.
* Dynamic Python that works like PHP. Print statements are site content..
* File serving
It will resolve url's to template files what it will render.
By visiting /pony, dreamii will resolve the url and resolve template in this order and will try next one if it exists:
- pony.md
- pony.html
It will resolve / into resolving index.md or index.html. If both exist, index.md will be used. This if for every directory. If you want a file list of a directory use the file list template tag.
Markdown files have priority since they can use a html page with the same name as template to extend from.
## Installation
It will resolve root (/) into resolving index.md or index.html. If both exist, index.md will be used. This rule applies for every directory. If you want a file list of a directory, use inline Python and pathlib for that.
## Get started
These are instructions for USING Dreamii, not to modify it.
```bash
python3 -m venv .venv
source .venv/bin/activate
pip install git+https://github.com/retoor/dreamii.git
pip install git+https://molodetz.nl/retoor/dreamii.git
echo > "# My first dreamii site" > index.md
dreamii serve 7331
```
## Develop with Dreamii
Just clone the repository and run install.
```bash
git clone https://molodetz.nl/retoor/dreamii.git
# Create virtual environment and install dependencies.
make install
# Run appliction within environment.
make run
```
## Technologies used for creating this project
* python3 as programming language.
* Python3 as programming language.
* aiohttp for server.
* jinja2 for templating.
* Jinja2 for templating.
* dataset for database.
## Notes
This project runs using the default aiohttp server. If you want to use another server, see the aiohttp documentation. There are several supported. But the default is good enough for thousands for requests per seconds. Caching is not enabled with a reason. There's literally no need to for static sites.
## Performance
This project runs using the default aiohttp server. If you want to use another server, see the aiohttp documentation. There are several supported. But Dreamii is by default good enough for thousands of requests per second. For the sake of usability, caching is not enabled. Every change to the site will be live directly.
## Using markdown
Markdown files can be used as template files. They can be used for creating article content. You can also use markdown to create a static site. Rendering syntax for a specific language is done the same way as your used to at github. For more information, see [Markdown](https://www.markdownguide.org/). Markdown files will use the `content block` of the `_base.html` template if it exists.
## Python inline support
It's possible to use inline py3 into your templates. You do this by starting {% py3 %} and ending {% endpy3 %}.
It has custom usable commands like shell executing that automatically encodes to HTML entities so it will be displayed properly.
### Example shell execution
This example will print the output of the `ps aux` command. `stderr` is set to `True` so the result of the function will be `stderr` since that's the `file descriptor` that `ps aux` uses. `output` is set to `False` because it will directly render to the template if enabled. We want to print the output using one line of code between `<pre>` and `</pre>`.
```jinja2
{% py3 %}
print(
"<pre>",
system("ps aux", output=False, stderr=True),
"</pre>"
)
{% endpy3 %}
### Example database usage
See here an example of a visitor counter on your website. This is all you have to do. It's a persistent database. No configuration required! For more information how to use the database, visit the [dataset documentation](https://dataset.readthedocs.io/en/latest/).
```python
{% py3 %}
def increment(counter_name="default")
counter = dreamii.db["counter"].find_one(counter_name=counter_name)
if not counter:
counter = {"count":0,"counter_name":"main_counter"}
counter["count"] += 1
dreamii.db["counter"].upsert(counter,["counter_name"])
return counter["count"]
print(f"<p>This page had been visited {counter()} times.</p>")
{% endpy3 %}
## Hidden files
Files prefixed with `_` or `.` will not be displayed or rendered by Dreamii. This is for security and it is a convenient way to mark base templates.
## Base template
You can use any template you want as a base template but `_base.html` is special since it will be used for markdown templates by default if it exists. If you want to use it for markdown, ensure that a content block named `markdown` is available.
### Example base template
```html
<html>
<head>
<title>{% block title %}My site{% block title %}</title>
</head>
<body>
{% block html %}{% endblock %}
{% block markdown %}{% endblock %}
</body>
</html>
```
#### Use template for HTML
In this example we will be using the example base template mentioned above. We will use `_base.html` as our base template and overide the block named `html` with our content.
```html
{% extends "_base.html" %}
{% block html %}
<h1>This is the page title!</h1>
<p>This is a paragraph</p>
{% endblock %}
```
Remember, you can use any file as base template.

1
src/Dreamii.egg-info/SOURCES.txt
View File

@ -9,6 +9,5 @@ src/Dreamii.egg-info/top_level.txt
src/dreamii/__init__.py
src/dreamii/__main__.py
src/dreamii/app.py
src/dreamii/fetch.py
src/dreamii/markdown.py
src/dreamii/python.py

10
src/dreamii/__main__.py
View File

@ -1,3 +1,11 @@
# Written by retoor@molodetz.nl
# This script initializes and runs an instance of the Dreamii application.
# External import: from dreamii.app import Dreamii
# MIT License
from dreamii.app import Dreamii
def main():
@ -5,4 +13,4 @@ def main():
dreamii.run()
if __name__ == '__main__':
main()
main()

77
src/dreamii/app.py
View File

@ -1,5 +1,32 @@
from app.app import Application as BaseApplication , BaseView
import pathlib
# Written by retoor@molodetz.nl
# This source code implements a web application using the aiohttp framework. It provides a template rendering system that supports both HTML and Markdown files. The `TemplateView` class handles both GET and POST requests, resolving and rendering templates based on request paths. The `Dreamii` class sets up the application environment, adding support for markdown and python extension within jinja2 templates.
# Imports:
# 1. `aiohttp_jinja2`, `web` are part of the aiohttp framework for creating web servers and managing web requests.
# 2. `MarkdownExtension` and `PythonExtension` are custom extensions that facilitate Markdown and Python rendering within templates.
# MIT License
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
from app.app import Application as BaseApplication, BaseView
import pathlib
from aiohttp import web
from dreamii.markdown import MarkdownExtension
from dreamii.python import PythonExtension
@ -7,34 +34,28 @@ import aiohttp_jinja2
class TemplateView(BaseView):
async def resolve_template(self, path):
path = pathlib.Path(self.request.app.template_path).joinpath(str(path).lstrip('/'))
if path.exists() and path.suffix in ['.html', '.md']:
return str(path)
elif path.joinpath('.html').exists():
if path.joinpath('.html').exists():
return str(path.joinpath('.html'))
elif path.joinpath('.md').exists():
if path.joinpath('.md').exists():
return str(path.joinpath('.md'))
elif path.is_dir():
if path.is_dir():
if path.joinpath('index.html').exists():
return str(path.joinpath('index.html'))
elif path.joinpath('index.md').exists():
if path.joinpath('index.md').exists():
return str(path.joinpath('index.md'))
return None
async def render_template(self, path):
context = {}
context['request'] = self.request
context['post'] = await self.request.post()
if not context['post']:
context['post'] = None
context = {'request': self.request, 'post': await self.request.post() or None}
try:
context['json'] = await self.request.json()
except:
context['json'] = None
pass
return await super().render_template(path, self.request)
return await super().render_template(path, self.request)
async def get(self):
path = await self.resolve_template(self.request.match_info['tail'])
@ -42,8 +63,8 @@ class TemplateView(BaseView):
return await self.render_template(path)
path = pathlib.Path(self.request.app.template_path).joinpath(self.request.match_info['tail'].lstrip('/'))
if path.exists():
return await web.Response(body=path.read_bytes())
return web.Response(status=404)
return web.Response(body=path.read_bytes())
return web.Response(status=404)
async def post(self):
path = await self.resolve_template(self.request.match_info['tail'])
@ -51,30 +72,22 @@ class TemplateView(BaseView):
return await self.render_template(path)
path = pathlib.Path(self.request.app.template_path).joinpath(self.request.match_info['tail'].lstrip('/'))
if path.exists():
return await web.Response(body=path.read_bytes())
return web.Response(status=404)
return web.Response(body=path.read_bytes())
return web.Response(status=404)
class Dreamii (BaseApplication):
class Dreamii(BaseApplication):
def __init__(self, *args, **kwargs):
super(Dreamii, self).__init__(template_path=".",*args, **kwargs)
super().__init__(template_path=".", *args, **kwargs)
self.router.add_view("/{tail:.*}", TemplateView)
self.jinja2_env.enable_async = False
self.jinja2_env.globals["app"] = self
self.jinja2_env.add_extension(MarkdownExtension)
self.jinja2_env.add_extension(PythonExtension)
def run(self,port=7331):
web.run_app(self,port=port)
def run(self, port=7331):
web.run_app(self, port=port)
if __name__ == '__main__':
app = Application()
app.run()
app = Dreamii()
app.run()

39
src/dreamii/markdown.py
View File

@ -1,3 +1,30 @@
# Written by retoor@molodetz.nl
# This code provides a MarkdownRenderer class that converts markdown into HTML using Jinja2 and Mistune. It includes support for syntax highlighting through Pygments and defines a custom Jinja2 extension to process markdown blocks within Jinja2 templates.
# Imports used include mistune for markdown rendering, pygments for syntax highlighting, and jinja2 for template rendering and extension creation.
# MIT License
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
from types import SimpleNamespace
from mistune import HTMLRenderer, Markdown
from pygments import highlight
@ -8,29 +35,24 @@ from jinja2.ext import Extension
from jinja2.nodes import Const
class MarkdownRenderer(HTMLRenderer):
_allow_harmful_protocols = True
def __init__(self, app, template):
self.template = template
self.app = app
self.env = self.app.jinja2_env
formatter = html.HtmlFormatter()
self.env.globals["highlight_styles"] = formatter.get_style_defs()
def _escape(self, str):
return str ##escape(str)
return str
def block_code(self, code, lang=None, info=None):
if not lang:
lang = info
if not lang:
return f"<div>{code}</div>"
# return '\n<pre><code>%s</code></pre>\n' % escape(code)
lexer = get_lexer_by_name(lang, stripall=True)
formatter = html.HtmlFormatter(lineseparator="<br>")
return highlight(code, lexer, formatter)
@ -52,13 +74,12 @@ async def render_markdown(app, markdown_string):
return render_markdown_sync(app, markdown_string)
# Source based on: https://ron.sh/how-to-write-a-jinja2-extension/
class MarkdownExtension(Extension):
tags = {"markdown"}
def __init__(self, environment):
self.app = SimpleNamespace(jinja2_env=environment)
super(MarkdownExtension, self).__init__(environment)
super().__init__(environment)
def parse(self, parser):
line_number = next(parser.stream).lineno
@ -76,4 +97,4 @@ class MarkdownExtension(Extension):
return render_markdown_sync(self.app, caller())
async def _to_html_async(self, md_file, caller):
return await render_markdown(self.app, caller())
return await render_markdown(self.app, caller())

107
src/dreamii/python.py
View File

@ -1,13 +1,36 @@
# Written by retoor@molodetz.nl
# This code provides a Jinja2 extension for executing Python code within Jinja2 templates and rendering it.
# It also defines functions to render markdown synchronously or asynchronously using specific rendering classes.
# This document uses non-standard libraries such as Mistune for markdown rendering, Pygments for syntax highlighting,
# and Jinja2 for template processing.
# The MIT License (MIT)
#
# Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files,
# to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish,
# distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so,
# subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
# OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
# IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
from types import SimpleNamespace
from mistune import HTMLRenderer, Markdown
from mistune import Markdown
from pygments import highlight
from pygments.formatters import html
from pygments.lexers import get_lexer_by_name
from jinja2 import TemplateSyntaxError, nodes
from jinja2.ext import Extension
from jinja2.nodes import Const
import subprocess
import asyncio
import subprocess
import asyncio
def render_markdown_sync(app, markdown_string):
@ -19,32 +42,26 @@ def render_markdown_sync(app, markdown_string):
async def render_markdown(app, markdown_string):
return render_markdown_sync(app, markdown_string)
async def execute_source(source,locals):
async def execute_source(source, locals):
db = locals['db']
system = locals['system']
dreamii = locals['app']
exec(source)
async def execute_python_source(source_code):
# Create a dictionary to act as the execution context
exec_globals = {}
exec_locals = {}
# Compile and execute the source code
exec(
f"async def __temp_function__():\n" + "\n".join(f" {line}" for line in source_code.splitlines()),
exec_globals,
exec_locals
exec_locals,
)
# Run the temporary async function
return await exec_locals["__temp_function__"]()
class PythonExtension(Extension):
tags = {"py3"}
@ -66,66 +83,54 @@ class PythonExtension(Extension):
self.call_method("_to_html", md_file), [], [], body
).set_lineno(line_number)
#return self.call_method("to_html", body).set_lineno(line_number)
#return self._to_html(md_file,md_file) # self.call_method("_to_html", md_file)
def _to_html(self, md_file, caller):
print("Did it without async")
base_source = "\n".join([
"from subprocess import *",
"import asyncio",
"import pathlib",
"from pathlib import *",
"import dataset",
])
base_source = "\n".join(
[
"from subprocess import *",
"import asyncio",
"import pathlib",
"from pathlib import *",
"import dataset",
]
)
user_source = str(caller()).strip("\n").strip(" ")
print(user_source)
source = "\n".join([base_source,user_source])
import sys
source = "\n".join([base_source, user_source])
import sys
import io
import html
import html
original_stdout = sys.stdout
original_stderr = sys.stderr
original_stderr = sys.stderr
sys.stdout = io.StringIO()
sys.stderr = io.StringIO()
def system(command,output=True,stderr=False,escape=True):
def system(command, output=True, stderr=False, escape=True):
if isinstance(command, str):
command = command.split(" ")
result = subprocess.run(
command,
text=True,
capture_output=True
)
result = subprocess.run(command, text=True, capture_output=True)
if result.stdout:
original_stdout.write(result.stdout)
# Yes, Docker People, I thought about you guys!
original_stdout.flush()
if output:
if escape:
print(html.escape(result.stdout),end="")
else:
print(result.stdout,end="")
print(html.escape(result.stdout) if escape else result.stdout, end="")
if result.stderr:
original_stderr.write(result.stderr)
# Yes, Docker People, I thought about you guys!
original_stderr.flush()
if output and stderr:
if escape:
print(html.escape(result.stderr,end=""))
else:
print(result.stderr,end="")
if stderr:
return html.escape(result.stdout + result.stderr)
return result.stdout
db= self.app.db
app = self.app
dreamii = self.app
print(html.escape(result.stderr) if escape else result.stderr, end="")
return html.escape(result.stdout + result.stderr) if stderr else result.stdout
db = self.app.db
app = self.app
dreamii = self.app
context = self.context
exec(base_source)
exec(user_source)
result = sys.stdout.getvalue()
if not result:
result = sys.stderr.getvalue()
return result # nodes.CallBlock(
return result