This revise is based on the methods developed in the revise of the google engine
(see commit 410c2f9).
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
pip install -e .
...
Obtaining file:///usr/local/searx/searx-src
ERROR: Command errored out with exit status 1:
command: /usr/local/searx/searx-pyenv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/usr/local/searx/searx-src/setup.py'"'"'; __file__='"'"'/usr/local/searx/searx-src/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'rn'"'"', '"'"'n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-vzer91m2
cwd: /usr/local/searx/searx-src/
Complete output (9 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/local/searx/searx-src/setup.py", line 10, in <module>
from searx.version import VERSION_STRING
File "/usr/local/searx/searx-src/searx/__init__.py", line 19, in <module>
import searx.settings_loader
File "/usr/local/searx/searx-src/searx/settings_loader.py", line 8, in <module>
import yaml
ModuleNotFoundError: No module named 'yaml'
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Removes module searx/brand.py and creates a namespace at searx.brand.
This patch is a first 'proof of concept'. Later we can decide to remove the
brand namespace entirely or not.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Install boilerplate packages (pip, wheel, setuptools and requirements.txt) only
once, when virtualenv is created.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
In commit c2a6f145 the sphinx-doc and sphinx-live targets has been removed,
since Sphinx is already installed by the pyenvinstall target.
This commit removes the targets from the remaining prerequisites of the 'BOOK'
targets
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Makefile targets 'pyenv-install' and 'pyenv-uninstall' are unused duplicates of
'pyenvinstall' and 'pyenvuninstall'.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
This patch keeps an eye on:
1. When installing searx, the branch to install needs to be a local branch.
2. The branch to install can be different from the current branch.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
In commit a70b9b9f the SEARX_SETTINGS_TEMPLATE environment was added to the
.config file, but was not use in utils/searx.sh.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Installation and use of the command 'virtualenv' was only needed in py2 and py2
is no longer suported by searx. In py3 the command is replaced by 'python -m
venv'.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The path to the makefile.lxc is not always static and
'/share/searx/utils/makefile.lxc' can be wrong.
When target 'lxc.activate' is called within the LXC container the path to
makefile.lxc depends on LXC_SHARE_FOLDER and the basename of the folder where
searx was cloned inital on the HOST system.
This patch determines the path of 'makefile.lxc' next to 'makefile.include'
itself.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Since #2291 is merged, it is recommend to use::
use_default_settings=True
1. Add a template file use_default_settings.yml::
SEARX_SETTINGS_TEMPLATE="${REPO_ROOT}/utils/templates/etc/searx/use_default_settings.yml"
2. In Chapter "Configuration" recommend to make use of
'use_default_settings=True' and describe it
3. Rewrite of docs/admin/settings.rst
- move chapter 'settings.yml location' to the top
- update and split chapter 'Global Settings'
4. Add environment SEARX_SETTINGS_TEMPLATE to .config.sh
5. Use environment $SEARX_SETTINGS_TEMPLATE in the utils/searx.sh script
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
- strip html tags and superfluous quotation marks from content
- remove not needed cookie from request
- remove superfluous imports
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Error pattern::
Engines cannot retrieve results:
digg (unexpected crash time data '2020-10-16T14:09:55Z' does not match format '%Y-%m-%d %H:%M:%S')
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Devian's request and response forms has been changed.
- fixed title
- fixed time_range_dict to 'popular-*-***'
- use image from <noscript> if exists
- drop obsolete "http to https, remove domain sharding"
- use query URL https://www.deviantart.com/search/deviations?page=5&q=foo
- add searx/engines/deviantart.py to pylint check (test.pylint)
Error pattern::
There DEBUG:searx:result: invalid title: {'url': 'https://www.deviantart.com/ ...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Escape the all the quotation marks (" --> \") in docs/build-templates.
The scripts are *bash evaluated* (e.g)::
eval "echo \"$(< "${REPO_ROOT}/docs/build-templates/searx.rst")\""
Backslash, quotation marks and other need to be escaped.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Reloading webapp.py when a file is changed exit with error::
$ make run
....
INFO:werkzeug: * Detected change in 'searx/searx/engines/deviantart.py', reloading
INFO:werkzeug: * Restarting with stat
ERROR:searx:server.secret_key is not changed. Please use something else instead of ultrasecretkey.
make: *** [Makefile:69: run] Error 1
Patch suggested by @dalf [1]
[1] https://github.com/searx/searx/issues/2278#issuecomment-719732579
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Since 1. October 2020 google has changed the 'class' attribute of the HTML
result page.
Fix the xpath expressions and ignore <div class="g" ../> sections which do not
match to title's xpath expression.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Since there is no centos-7 documentation 'searx.sh doc' produce an error
message::
ERROR: centos-: uWSGI not yet implemented
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
## What does this PR do?
When installing searx on a centos-7 image [1] an encoding exception is raised from setup.py::
command: /usr/local/searx/searx-pyenv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/usr/local/searx/searx-src/setup.py'"'"'; __file__='"'"'/usr/local/searx/searx-src/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'rn'"'"', '"'"'n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-pidntzkq
cwd: /usr/local/searx/searx-src/
Complete output (7 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/local/searx/searx-src/setup.py", line 16, in <module>
long_description = f.read()
File "/usr/lib64/python3.6/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc9 in position 482: ordinal not in range(128)
[1] https://uk.images.linuxcontainers.org/
## Why is this change important?
README.rst uses non ASCII characters _(pronunciation səːks.)_
## How to test this PR locally?
install searx on a centos-7 image from https://uk.images.linuxcontainers.org/
## What does this PR do?
To test centos-7 on non centos systems add centos-7 image to the LXC suite.
## Why is this change important?
PR #2112 adds centos-7 support to the install scripts
## How to test this PR locally?
`sudo ./utils/lxc.sh build searx-centos7`
Python's default encoding depends on the platform, set (python) default encoding
UTF-8 in uwsgi ini files:
LANG=C.UTF-8
LANGUAGE=C.UTF-8
LC_ALL=C.UTF-8
Error pattern:
Traceback (most recent call last):
File "/usr/local/searx/searx-src/searx/webapp.py", line 74, in <module>
from searx.search import SearchWithPlugins, get_search_query_from_webapp
File "/usr/local/searx/searx-src/searx/search.py", line 32, in <module>
from searx.external_bang import get_bang_url
File "/usr/local/searx/searx-src/searx/external_bang.py", line 13, in <module>
for bang in json.load(json_file)['bang']:
File "/usr/lib/python3.8/json/__init__.py", line 293, in load
return loads(fp.read(),
File "/usr/lib/python3.8/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 31341: ordinal not in range(128)
close: https://github.com/asciimoo/searx/issues/2041
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The $DOCS_BUILD variable (like all BUILD variables) in the lxc environment::
sudo -H ./utils/lxc.sh cmd searx-archlinux make docs
is different from running build process in the HOST::
make docs
with kernel-include directive we can use the environment variables in the reST
documents to address the correct file location of the include.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The sphinx-doc-prebuilds is the single point of definition for prerequisites
each doc build needs
BTW: remove $(LXC_ENV_FOLDER) offset from BOOKS_FOLDER and DOCS_FOLDER path.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Markus Heiser