diff --git a/Example.yaml b/Example.yaml
new file mode 100644
index 0000000..2bec029
--- /dev/null
+++ b/Example.yaml
@@ -0,0 +1,22 @@
+common:
+
+ database: "postgresql://user:password@localhost/operational_database"
+ prefix: my_project/dumps/
+ format: json
+
+exports:
+
+ - query: "SELECT id, title, date FROM events"
+ filename: "index.json"
+
+ - query: "SELECT id, title, date, country FROM events"
+ filename: "countries/{{country}}.csv"
+ format: csv
+
+ - query: "SELECT * FROM events"
+ filename: "events/{{id}}.json"
+ mode: item
+
+ - query: "SELECT * FROM events"
+ filename: "all.json"
+ format: tabson
diff --git a/docs/_themes/kr/sidebarlogo.html b/docs/_themes/kr/sidebarlogo.html
index 990f761..4631beb 100644
--- a/docs/_themes/kr/sidebarlogo.html
+++ b/docs/_themes/kr/sidebarlogo.html
@@ -8,7 +8,8 @@
diff --git a/docs/_themes/kr/static/flasky.css_t b/docs/_themes/kr/static/flasky.css_t
index 95e71c6..9879049 100755
--- a/docs/_themes/kr/static/flasky.css_t
+++ b/docs/_themes/kr/static/flasky.css_t
@@ -395,6 +395,10 @@ a:hover tt {
background: #EEE;
}
+li {
+ margin-bottom: 0.3em;
+}
+
@media screen and (max-width: 870px) {
@@ -516,6 +520,7 @@ a:hover tt {
padding: 0;
}
+
.rtd_doc_footer {
display: none;
}
@@ -551,4 +556,4 @@ div.sphinxsidebar #searchbox input[type="text"] {
.highlight .nv {
color: #C65D09!important;
-}
\ No newline at end of file
+}
diff --git a/docs/freezefile.rst b/docs/freezefile.rst
new file mode 100644
index 0000000..229e1aa
--- /dev/null
+++ b/docs/freezefile.rst
@@ -0,0 +1,95 @@
+
+Freezefiles and the ``datafreeze`` command
+==========================================
+
+``datafreeze`` creates static extracts of SQL databases for use in interactive
+web applications. SQL databases are a great way to manage relational data, but
+exposing them on the web to drive data apps can be cumbersome. Often, the
+capacities of a proper database are not actually required, a few static JSON
+files and a bit of JavaScript can have the same effect. Still, exporting JSON
+by hand (or with a custom script) can also become a messy process.
+
+With ``datafreeze``, exports are scripted in a Makefile-like description, making them simple to repeat and replicate.
+
+
+Basic Usage
+-----------
+
+Calling DataFreeze is simple, the application is called with a
+freeze file as its argument:
+
+.. code-block:: bash
+
+ datafreeze Freezefile.yaml
+
+Freeze files can be either written in JSON or in YAML.
+
+
+Example Freezefile.yaml
+-----------------------
+
+A freeze file is composed of a set of scripted queries and
+specifications on how their output is to be handled. An example could look
+like this:
+
+.. code-block:: yaml
+
+ common:
+
+ database: "postgresql://user:password@localhost/operational_database"
+ prefix: my_project/dumps/
+ format: json
+
+ exports:
+
+ - query: "SELECT id, title, date FROM events"
+ filename: "index.json"
+
+ - query: "SELECT id, title, date, country FROM events"
+ filename: "countries/{{country}}.csv"
+ format: csv
+
+ - query: "SELECT * FROM events"
+ filename: "events/{{id}}.json"
+ mode: item
+
+ - query: "SELECT * FROM events"
+ filename: "all.json"
+ format: tabson
+
+An identical JSON configuration can be found in this repository.
+
+
+Options in detail
+-----------------
+
+The freeze file has two main sections, ``common`` and ``exports``. Both
+accept many of the same arguments, with ``exports`` specifying a list of
+exports while ``common`` defines some shared properties, such as the
+database connection string.
+
+The following options are recognized:
+
+* ``database`` is a database URI, including the database type, username
+ and password, hostname and database name. Valid database types include
+ ``sqlite``, ``mysql`` and ``postgresql`` (requires psycopg2).
+* ``prefix`` specifies a common root directory for all extracted files.
+* ``format`` identifies the format to be generated, ``csv``, ``json`` and
+ ``tabson`` are supported. ``tabson`` is a condensed JSON
+ representation in which rows are not represented by objects but by
+ lists of values.
+* ``query`` needs to be a valid SQL statement. All selected fields will
+ become keys or columns in the output, so it may make sense to define
+ proper aliases if any overlap is to be expected.
+* ``mode`` specifies whether the query output is to be combined into a
+ single file (``list``) or whether a file should be generated for each
+ result row (``item``).
+* ``filename`` is the output file name, appended to ``prefix``. All
+ occurences of ``{{field}}`` are expanded to a fields value to allow the
+ generation of file names e.g. by primary key. In list mode, templating
+ can be used to group records into several buckets, e.g. by country or
+ category.
+* ``wrap`` can be used to specify whether the output should be wrapped
+ in a ``results`` hash in JSON output. This defaults to ``true`` for
+ ``list``-mode output and ``false`` for ``item``-mode.
+
diff --git a/docs/index.rst b/docs/index.rst
index 9b9c45b..3614044 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -22,12 +22,12 @@ Python a breeze:
the necessity for a full ORM model - essentially, databases can be
used like a JSON file or NoSQL store.
-* Database contents can be exported (*frozen*) using a sophisticated
- plain file generator with JSON and CSV support. Exports can be configured
- to include metadata and dynamic file names depending on the exported
- data.
+* Database contents can be exported (*frozen*) using a :doc:`sophisticated
+ plain file generator ` with JSON and CSV support. Exports can be configured
+ to include metadata and dynamic file names depending on the exported
+ data. The exporter can also be used as a command-line tool, ``datafreeze``.
-The exporter can also be used as a command-line tool, ``datafreeze``.
+A simple data loading script using **dataset** might look like this:
::
@@ -55,6 +55,8 @@ Features
* **Query helpers** for simple queries such as :py:meth:`all ` rows in a table or
all :py:meth:`distinct ` values across a set of columns.
* **Compatibility**: Being built on top of `SQLAlchemy `_, ``dataset`` works with all major databases, such as SQLite, PostgreSQL and MySQL.
+* **Scripted exports**: Data can be exported based on a scripted
+ configuration, making the process easy and replicable.
Contents
--------
@@ -63,6 +65,7 @@ Contents
:maxdepth: 2
quickstart
+ freezefile
api
Contributors
diff --git a/docs/quickstart.rst b/docs/quickstart.rst
index eacdf84..a2e65e1 100644
--- a/docs/quickstart.rst
+++ b/docs/quickstart.rst
@@ -144,9 +144,8 @@ You can create one file per row by setting ``mode`` to "item"::
# export one JSON file per user
dataset.freeze(result, 'users/{{ id }}.json', format='json', mode='item')
-
Since this is a common operation we made it available via command line
-utility ``datafreeze``. Read more about the `freezefile markup `_.
+utility ``datafreeze``. Read more about the :doc:`freezefile markup `.
.. code-block:: bash