This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<spanid="makefile"></span><h1>Makefile &<codeclass="docutils literal notranslate"><spanclass="pre">./manage</span></code><aclass="headerlink"href="#makefile-manage"title="Link to this heading">ΒΆ</a></h1>
<p>All relevant build and development tasks are implemented in the
<aclass="extlink-origin reference external"href="https://github.com/searxng/searxng/blob/master/manage">./manage</a> script and for CI or IDE integration a small
<aclass="extlink-origin reference external"href="https://github.com/searxng/searxng/blob/master/Makefile">git://Makefile</a> wrapper is available. If you are not familiar with
Makefiles, we recommend to read <aclass="reference external"href="https://www.gnu.org/software/make/manual/make.html#Introduction">gnu-make</a> introduction.</p>
<asideclass="sidebar">
<pclass="sidebar-title">build environment</p>
<p>Before looking deeper at the targets, first read about <aclass="reference internal"href="#make-install"><spanclass="std std-ref">Python environment (make install)</span></a>.</p>
<p>The usage is simple, just type <codeclass="docutils literal notranslate"><spanclass="pre">make</span><spanclass="pre">{target-name}</span></code> to <em>build</em> a target.
Calling the <codeclass="docutils literal notranslate"><spanclass="pre">help</span></code> target gives a first overview (<codeclass="docutils literal notranslate"><spanclass="pre">make</span><spanclass="pre">help</span></code>):</p>
nvm.: use nvm (without dot) to execute nvm commands directly
install : install NVM locally at /home/runner/work/searxng/searxng/.nvm
clean : remove NVM installation
status : prompt some status information about nvm & node
nodejs : install Node.js latest LTS
cmd ... : run command ... in NVM environment
bash : start bash interpreter with NVM environment sourced
webapp.:
run : run developer instance
docs.:
html : build HTML documentation
live : autobuild HTML documentation while editing
gh-pages : deploy on gh-pages branch
prebuild : build reST include files (./build/docs/includes)
clean : clean documentation build
docker.:
build : build docker image
push : build and push docker image
gecko.driver:
download & install geckodriver if not already installed (required for
robot_tests)
redis:
build : build redis binaries at /home/runner/work/searxng/searxng/dist/redis/6.2.6/amd64
install : create user (searxng-redis) and install systemd service (searxng-redis)
help : show more redis commands
py.:
build : Build python packages at ./dist
clean : delete virtualenv and intermediate py files
pyenv.:
install : developer install of SearXNG into virtualenv
uninstall : uninstall developer installation
cmd ... : run command ... in virtualenv
OK : test if virtualenv is OK
format.:
python : format Python code source using black
go.:
ls : list golang binary archives (stable)
golang : (re-) install golang binary in user's $HOME/local folder
install : install go package in user's $HOME/go-apps folder
bash : start bash interpreter with golang environment sourced
node.:
env : download & install SearXNG's npm dependencies locally
env.dev : download & install developer and CI tools
clean : drop locally npm installations
weblate.:
push.translations: push translation changes from SearXNG to Weblate's counterpart
to.translations: Update 'translations' branch with last additions from Weblate.
data.:
all : update searx/sxng_locales.py and searx/data/*
traits : update searx/data/engine_traits.json & searx/sxng_locales.py
useragents: update searx/data/useragents.json with the most recent versions of Firefox
locales : update searx/data/locales.json from babel
test.:
yamllint : lint YAML files (YAMLLINT_FILES)
pylint : lint ./searx, ./searxng_extra and ./tests
pyright : static type check of python sources (.dev or .ci)
black : check black code format
unit : run unit tests
coverage : run unit tests with coverage
robot : run robot test
rst : test .rst files incl. README.rst
clean : clean intermediate test stuff
themes.:
all : test & build all themes
test : test all themes
fix : fix JS & CSS (LESS)
live : to get live builds of CSS & JS use: LIVE_THEME=simple make run
simple.: test & build simple theme ..
pygments: build pygment's LESS files for simple theme
test : test simple theme
fix : fix JS & CSS (LESS) of the simple theme
static.build.: [build] /static
commit : build & commit /static folder
drop : drop last commit if it was previously done by static.build.commit
restore : git restore of the /static folder (after themes.all)
environment ...
SEARXNG_REDIS_URL :
----
run - run developer instance
install - developer install of SearxNG into virtualenv
uninstall - uninstall developer installation
clean - clean up working tree
search.checker - check search engines
test - run shell & CI tests
test.shell - test shell scripts
ci.test - run CI tests
</pre></div>
</div>
</div><divaria-labelledby="tab-0-YGAuL21hbmFnZWBg"class="sphinx-tabs-panel group-tab"hidden="true"id="panel-0-YGAuL21hbmFnZWBg"name="YGAuL21hbmFnZWBg"role="tabpanel"tabindex="0"><p>The Makefile targets are implemented for comfort, if you can do without
tab-completion and need to have a more granular control, use
<aclass="extlink-origin reference external"href="https://github.com/searxng/searxng/blob/master/manage">git://manage</a> without the Makefile wrappers.</p>
<spanid="make-install"></span><h2><aclass="toc-backref"href="#id14"role="doc-backlink">Python environment (<codeclass="docutils literal notranslate"><spanclass="pre">make</span><spanclass="pre">install</span></code>)</a><aclass="headerlink"href="#python-environment-make-install"title="Link to this heading">ΒΆ</a></h2>
<p>We do no longer need to build up the virtualenv manually. Jump into your git
working tree and release a <codeclass="docutils literal notranslate"><spanclass="pre">make</span><spanclass="pre">install</span></code> to get a virtualenv with a
<em>developer install</em> of SearXNG (<aclass="extlink-origin reference external"href="https://github.com/searxng/searxng/blob/master/setup.py">git://setup.py</a>).</p>
<divclass="highlight-default notranslate"><divclass="highlight"><pre><span></span>$ cd ~/searxng-clone
$ make install
PYENV [virtualenv] installing ./requirements*.txt into local/py3
<p>If you release <codeclass="docutils literal notranslate"><spanclass="pre">make</span><spanclass="pre">install</span></code> multiple times the installation will only
rebuild if the sha256 sum of the <em>requirement files</em> fails. With other words:
the check fails if you edit the requirements listed in
<aclass="extlink-origin reference external"href="https://github.com/searxng/searxng/blob/master/requirements-dev.txt">git://requirements-dev.txt</a> and <aclass="extlink-origin reference external"href="https://github.com/searxng/searxng/blob/master/requirements.txt">git://requirements.txt</a>).</p>
<divclass="highlight-default notranslate"><divclass="highlight"><pre><span></span>$ make install
<p>To get rid of the existing environment before re-build use <aclass="reference internal"href="#make-clean"><spanclass="std std-ref">clean target</span></a> first.</p>
</aside>
<p>If you think, something goes wrong with your ./local environment or you change
the <aclass="extlink-origin reference external"href="https://github.com/searxng/searxng/blob/master/setup.py">git://setup.py</a> file, you have to call <aclass="reference internal"href="#make-clean"><spanclass="std std-ref">make clean</span></a>.</p>
</section>
<sectionid="node-js-environment-make-node-env">
<spanid="make-node-env"></span><h2><aclass="toc-backref"href="#id15"role="doc-backlink">Node.js environment (<codeclass="docutils literal notranslate"><spanclass="pre">make</span><spanclass="pre">node.env</span></code>)</a><aclass="headerlink"href="#node-js-environment-make-node-env"title="Link to this heading">ΒΆ</a></h2>
<p><aclass="reference external"href="https://nodejs.org/">Node.js</a> version 16.13.0 or higher is required to build the themes.
If the requirement is not met, the build chain uses <aclass="reference external"href="https://github.com/nvm-sh">nvm</a> (Node Version
Manager) to install latest LTS of <aclass="reference external"href="https://nodejs.org/">Node.js</a> locally: there is no need to
install <aclass="reference external"href="https://github.com/nvm-sh">nvm</a> or <aclass="reference external"href="https://www.npmjs.com/">npm</a> on your system.</p>
<p>To install <aclass="reference external"href="https://github.com/nvm-sh">NVM</a> and <aclass="reference external"href="https://nodejs.org/">Node.js</a> in once you can use <aclass="reference internal"href="#make-nvm-nodejs"><spanclass="std std-ref">make nvm.nodejs</span></a>.</p>
<sectionid="nvm-make-nvm-install-nvm-status">
<spanid="make-nvm"></span><h3><aclass="toc-backref"href="#id16"role="doc-backlink">NVM <codeclass="docutils literal notranslate"><spanclass="pre">make</span><spanclass="pre">nvm.install</span><spanclass="pre">nvm.status</span></code></a><aclass="headerlink"href="#nvm-make-nvm-install-nvm-status"title="Link to this heading">ΒΆ</a></h3>
<p>Use <codeclass="docutils literal notranslate"><spanclass="pre">make</span><spanclass="pre">nvm.status</span></code> to get the current status of your <aclass="reference external"href="https://nodejs.org/">Node.js</a> and <aclass="reference external"href="https://github.com/nvm-sh">nvm</a>
</div><divaria-labelledby="tab-1-bnZtLnN0YXR1cyAodWJ1MjAwNCk="class="sphinx-tabs-panel group-tab"hidden="true"id="panel-1-bnZtLnN0YXR1cyAodWJ1MjAwNCk="name="bnZtLnN0YXR1cyAodWJ1MjAwNCk="role="tabpanel"tabindex="0"><p>Here is the output you will typically get on a Ubuntu 20.04 system which
serves only a <aclass="reference external"href="https://nodejs.org/en/about/releases/">no longer active</a>
<spanid="id1"></span><h3><aclass="toc-backref"href="#id17"role="doc-backlink"><codeclass="docutils literal notranslate"><spanclass="pre">make</span><spanclass="pre">nvm.nodejs</span></code></a><aclass="headerlink"href="#make-nvm-nodejs"title="Link to this heading">ΒΆ</a></h3>
<divclass="highlight-default notranslate"><divclass="highlight"><pre><span></span>$ make nvm.nodejs
INFO: install (update) NVM at /share/searxng/.nvm
INFO: clone: https://github.com/nvm-sh/nvm.git
...
Downloading and installing node v16.13.0...
...
INFO: Node.js is installed at searxng/.nvm/versions/node/v16.13.0/bin/node
INFO: Node.js is version v16.13.0
INFO: npm is installed at searxng/.nvm/versions/node/v16.13.0/bin/npm
INFO: npm is version 8.1.0
INFO: NVM is installed at searxng/.nvm
</pre></div>
</div>
</section>
</section>
<sectionid="make-run">
<spanid="id2"></span><h2><aclass="toc-backref"href="#id18"role="doc-backlink"><codeclass="docutils literal notranslate"><spanclass="pre">make</span><spanclass="pre">run</span></code></a><aclass="headerlink"href="#make-run"title="Link to this heading">ΒΆ</a></h2>
<p>To get up a running a developer instance simply call <codeclass="docutils literal notranslate"><spanclass="pre">make</span><spanclass="pre">run</span></code>. This enables
<em>debug</em> option in <aclass="extlink-origin reference external"href="https://github.com/searxng/searxng/blob/master/searx/settings.yml">git://searx/settings.yml</a>, starts a <codeclass="docutils literal notranslate"><spanclass="pre">./searx/webapp.py</span></code>
instance and opens the URL in your favorite WEB browser (<aclass="extlink-man reference external"href="https://manpages.debian.org/jump?q=xdg-open">xdg-open</a>):</p>
<divclass="highlight-default notranslate"><divclass="highlight"><pre><span></span>$ make run
</pre></div>
</div>
<p>Changes to themeβs HTML templates (jinja2) are instant. Changes to the CSS & JS
sources of the theme need to be rebuild. You can do that by running:</p>
<divclass="highlight-default notranslate"><divclass="highlight"><pre><span></span>$ make themes.all
</pre></div>
</div>
<p>Alternatively to <codeclass="docutils literal notranslate"><spanclass="pre">themes.all</span></code> you can run <em>live builds</em> of the theme you are
<divclass="highlight-default notranslate"><divclass="highlight"><pre><span></span>$ LIVE_THEME=simple make run
</pre></div>
</div>
</section>
<sectionid="make-format-python">
<spanid="id3"></span><h2><aclass="toc-backref"href="#id19"role="doc-backlink"><codeclass="docutils literal notranslate"><spanclass="pre">make</span><spanclass="pre">format.python</span></code></a><aclass="headerlink"href="#make-format-python"title="Link to this heading">ΒΆ</a></h2>
<p>Format Python source code using <aclass="reference external"href="https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html">Black code style</a>. See <codeclass="docutils literal notranslate"><spanclass="pre">$BLACK_OPTIONS</span></code>
and <codeclass="docutils literal notranslate"><spanclass="pre">$BLACK_TARGETS</span></code> in <aclass="extlink-origin reference external"href="https://github.com/searxng/searxng/blob/master/Makefile">git://Makefile</a>.</p>
<divclass="admonition attention">
<pclass="admonition-title">Attention</p>
<p>We stuck at Black 22.12.0, please read comment in PR <aclass="reference external"href="https://github.com/searxng/searxng/pull/2159#pullrequestreview-1284094735">Bump black from 22.12.0
to 23.1.0</a></p>
</div>
</section>
<sectionid="make-clean">
<spanid="id4"></span><h2><aclass="toc-backref"href="#id20"role="doc-backlink"><codeclass="docutils literal notranslate"><spanclass="pre">make</span><spanclass="pre">clean</span></code></a><aclass="headerlink"href="#make-clean"title="Link to this heading">ΒΆ</a></h2>
<p>Drops all intermediate files, all builds, but keep sources untouched. Before
calling <codeclass="docutils literal notranslate"><spanclass="pre">make</span><spanclass="pre">clean</span></code> stop all processes using the <aclass="reference internal"href="#make-install"><spanclass="std std-ref">Python environment (make install)</span></a> or
<spanid="id5"></span><h2><aclass="toc-backref"href="#id21"role="doc-backlink"><codeclass="docutils literal notranslate"><spanclass="pre">make</span><spanclass="pre">docs</span></code></a><aclass="headerlink"href="#make-docs"title="Link to this heading">ΒΆ</a></h2>
<p>Target <codeclass="docutils literal notranslate"><spanclass="pre">docs</span></code> builds the documentation:</p>
<spanid="make-docs-clean"></span><h3><aclass="toc-backref"href="#id22"role="doc-backlink"><codeclass="docutils literal notranslate"><spanclass="pre">make</span><spanclass="pre">docs.clean</span><spanclass="pre">docs.live</span></code></a><aclass="headerlink"href="#make-docs-clean-docs-live"title="Link to this heading">ΒΆ</a></h3>
<p>We describe the usage of the <codeclass="docutils literal notranslate"><spanclass="pre">doc.*</span></code> targets in the <aclass="reference internal"href="contribution_guide.html#contrib-docs"><spanclass="std std-ref">How to contribute /
Documentation</span></a> section. If you want to edit the documentation
read our <aclass="reference internal"href="contribution_guide.html#make-docs-live"><spanclass="std std-ref">live build</span></a> section. If you are working in your own brand,
adjust your <aclass="reference internal"href="../admin/settings/settings_brand.html#settings-brand"><spanclass="std std-ref">brand:</span></a>.</p>
</section>
<sectionid="make-docs-gh-pages">
<spanid="id6"></span><h3><aclass="toc-backref"href="#id23"role="doc-backlink"><codeclass="docutils literal notranslate"><spanclass="pre">make</span><spanclass="pre">docs.gh-pages</span></code></a><aclass="headerlink"href="#make-docs-gh-pages"title="Link to this heading">ΒΆ</a></h3>
<p>To deploy on github.io first adjust your <aclass="reference internal"href="../admin/settings/settings_brand.html#settings-brand"><spanclass="std std-ref">brand:</span></a>. For any
further read <aclass="reference internal"href="contribution_guide.html#deploy-on-github-io"><spanclass="std std-ref">deploy on github.io</span></a>.</p>
</section>
</section>
<sectionid="make-test">
<spanid="id7"></span><h2><aclass="toc-backref"href="#id24"role="doc-backlink"><codeclass="docutils literal notranslate"><spanclass="pre">make</span><spanclass="pre">test</span></code></a><aclass="headerlink"href="#make-test"title="Link to this heading">ΒΆ</a></h2>
<p>Runs a series of tests: <aclass="reference internal"href="#make-test-pylint"><spanclass="std std-ref">make test.pylint</span></a>, <codeclass="docutils literal notranslate"><spanclass="pre">test.pep8</span></code>, <codeclass="docutils literal notranslate"><spanclass="pre">test.unit</span></code>
and <codeclass="docutils literal notranslate"><spanclass="pre">test.robot</span></code>. You can run tests selective, e.g.:</p>
<divclass="highlight-default notranslate"><divclass="highlight"><pre><span></span>$ make test.pep8 test.unit test.shell
TEST test.pep8 OK
...
TEST test.unit OK
...
TEST test.shell OK
</pre></div>
</div>
<sectionid="make-test-shell">
<spanid="id8"></span><h3><aclass="toc-backref"href="#id25"role="doc-backlink"><codeclass="docutils literal notranslate"><spanclass="pre">make</span><spanclass="pre">test.shell</span></code></a><aclass="headerlink"href="#make-test-shell"title="Link to this heading">ΒΆ</a></h3>
<p><aclass="reference internal"href="../admin/buildhosts.html#sh-lint"><spanclass="std std-ref">Lint shell scripts</span></a> / if you have changed some bash scripting run this test before
commit.</p>
</section>
<sectionid="make-test-pylint">
<spanid="id9"></span><h3><aclass="toc-backref"href="#id26"role="doc-backlink"><codeclass="docutils literal notranslate"><spanclass="pre">make</span><spanclass="pre">test.pylint</span></code></a><aclass="headerlink"href="#make-test-pylint"title="Link to this heading">ΒΆ</a></h3>
<p><aclass="reference external"href="https://www.pylint.org/">Pylint</a> is known as one of the best source-code, bug and quality checker for the
Python programming language. The pylint profile used in the SearXNG project is
found in projectβs root folder <aclass="extlink-origin reference external"href="https://github.com/searxng/searxng/blob/master/.pylintrc">git://.pylintrc</a>.</p>
</section>
</section>
<sectionid="make-search-checker-engine-name">
<spanid="make-search-checker"></span><h2><aclass="toc-backref"href="#id27"role="doc-backlink"><codeclass="docutils literal notranslate"><spanclass="pre">make</span><spanclass="pre">search.checker.{engine</span><spanclass="pre">name}</span></code></a><aclass="headerlink"href="#make-search-checker-engine-name"title="Link to this heading">ΒΆ</a></h2>
<spanid="id10"></span><h2><aclass="toc-backref"href="#id28"role="doc-backlink"><codeclass="docutils literal notranslate"><spanclass="pre">make</span><spanclass="pre">themes.*</span></code></a><aclass="headerlink"href="#make-themes"title="Link to this heading">ΒΆ</a></h2>
<p>The <aclass="extlink-origin reference external"href="https://github.com/searxng/searxng/blob/master/Makefile">git://Makefile</a> targets <codeclass="docutils literal notranslate"><spanclass="pre">make</span><spanclass="pre">theme.*</span></code> cover common tasks to build the
theme(s). The <codeclass="docutils literal notranslate"><spanclass="pre">./manage</span><spanclass="pre">themes.*</span></code> command line can be used to convenient run
common theme build tasks.</p>
<divclass="highlight-text notranslate"><divclass="highlight"><pre><span></span>INFO: sourced NVM environment from /home/runner/.nvm
themes.:
all : test & build all themes
test : test all themes
fix : fix JS & CSS (LESS)
live : to get live builds of CSS & JS use: LIVE_THEME=simple make run
simple.: test & build simple theme ..
pygments: build pygment's LESS files for simple theme
test : test simple theme
fix : fix JS & CSS (LESS) of the simple theme
</pre></div>
</div>
<p>To get live builds while modifying CSS & JS use (<aclass="reference internal"href="#make-run"><spanclass="std std-ref">make run</span></a>):</p>
<spanid="id11"></span><h2><aclass="toc-backref"href="#id29"role="doc-backlink"><codeclass="docutils literal notranslate"><spanclass="pre">make</span><spanclass="pre">static.build.*</span></code></a><aclass="headerlink"href="#make-static-build"title="Link to this heading">ΒΆ</a></h2>
<p>The <aclass="extlink-origin reference external"href="https://github.com/searxng/searxng/blob/master/Makefile">git://Makefile</a> targets <codeclass="docutils literal notranslate"><spanclass="pre">static.build.*</span></code> cover common tasks to build (a
commit of) the static files. The <codeclass="docutils literal notranslate"><spanclass="pre">./manage</span><spanclass="pre">static.build..*</span></code> command line
can be used to convenient run common build tasks of the static files.</p>
<divclass="highlight-text notranslate"><divclass="highlight"><pre><span></span>INFO: sourced NVM environment from /home/runner/.nvm
static.build.: [build] /static
commit : build & commit /static folder
drop : drop last commit if it was previously done by static.build.commit
restore : git restore of the /static folder (after themes.all)
</pre></div>
</div>
</section>
<sectionid="manage-redis-help">
<spanid="id12"></span><h2><aclass="toc-backref"href="#id30"role="doc-backlink"><codeclass="docutils literal notranslate"><spanclass="pre">./manage</span><spanclass="pre">redis.help</span></code></a><aclass="headerlink"href="#manage-redis-help"title="Link to this heading">ΒΆ</a></h2>
<p>The <codeclass="docutils literal notranslate"><spanclass="pre">./manage</span><spanclass="pre">redis.*</span></code> command line can be used to convenient run common Redis
<divclass="highlight-text notranslate"><divclass="highlight"><pre><span></span>INFO: sourced NVM environment from /home/runner/.nvm
redis.:
devpkg : install essential packages to compile redis
build : build redis binaries at /home/runner/work/searxng/searxng/dist/redis/6.2.6/amd64
install : create user (searxng-redis) and install systemd service (searxng-redis)
remove : delete user (searxng-redis) and remove service (searxng-redis)
shell : start bash interpreter from user searxng-redis
src : clone redis source code to <path> and checkput 6.2.6
useradd : create user (searxng-redis) at /usr/local/searxng-redis
userdel : delete user (searxng-redis)
addgrp : add <user> to group (searxng-redis)
rmgrp : remove <user> from group (searxng-redis)
</pre></div>
</div>
</section>
<sectionid="manage-go-help">
<spanid="id13"></span><h2><aclass="toc-backref"href="#id31"role="doc-backlink"><codeclass="docutils literal notranslate"><spanclass="pre">./manage</span><spanclass="pre">go.help</span></code></a><aclass="headerlink"href="#manage-go-help"title="Link to this heading">ΒΆ</a></h2>
<p>The <codeclass="docutils literal notranslate"><spanclass="pre">./manage</span><spanclass="pre">go.*</span></code> command line can be used to convenient run common <aclass="reference external"href="https://en.wikipedia.org/wiki/Go_(programming_language)">go
(wiki)</a> tasks.</p>
<divclass="highlight-text notranslate"><divclass="highlight"><pre><span></span>INFO: sourced NVM environment from /home/runner/.nvm
go.:
ls : list golang binary archives (stable)
golang : (re-) install golang binary in user's $HOME/local folder
install : install go package in user's $HOME/go-apps folder
bash : start bash interpreter with golang environment sourced