Currency engine has DuckDuckGo metadata
In the engine selector of the preferences window, the currency search engine has
the same metadata and wikidata url as duckduckgo, I'd assume there should be a
difference of some sort there clarifying what source the currency uses or, if
it's a duckduckgo service, at least clarifying that it's a currency service by
duck duck go.
Closes: https://github.com/searxng/searxng/issues/787
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Since commit cac03529 the eslint has been moved from the local nvm to the
developer packages (in `./node_modules`).
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Engine description can be configured, this is needed e.g. by custom search
engines. Here is an example of a command engine with a description in the about
section::
- name: locate
engine: command
command: ['locate', '{{QUERY}}']
disabled: true
categories: files
about:
description: local files
website: 'https://www.man7.org/linux/man-pages/man1/locate.1.html'
delimiter:
chars: ' '
keys: ['line']
Closes: https://github.com/searxng/searxng/issues/788
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Use httpx.Response.json() to avoid charset_normalizer issues:
DEBUG charset_normalizer : override steps (5) and chunk_size (512) as content does not fit (153 byte(s) given) parameters.
INFO charset_normalizer : ascii passed initial chaos probing. Mean measured chaos is 0.000000 %
DEBUG charset_normalizer : ascii should target any language(s) of ['Latin Based']
INFO charset_normalizer : ascii is most likely the one. Stopping the process.
[1] https://www.python-httpx.org/api/#response
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
This patch implements the command and Makefile target::
./manage test.pyright
make test.pyright
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Pyright [1] is in the nvm enviroment, may be you need to rebuild the nvm
environment and install nodejs in::
./manage nvm.clean
./manage nvm.nodejs
The last command installs nodejs and the packages from .nvm_packages.
You can test your pyright installation, to get a bash within the nvm run::
./manage nvm.bash
(nvm) $ which pyright
./.nvm/versions/node/v16.13.0/bin/pyright
If you have a local nvm in your HOME folder, the output from ``which`` is
different. Press ``[CTRL-D]`` to get out of this bash.
[1] https://github.com/microsoft/pyright
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Without specifying folders to check, pyright will also scan files in folders
like ./build, ./cache, ./.nvm and more.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
With Sphinx-doc update 4.4.0 we get some warnings about links that can be
replaced by already defined 'sphinx.ext.extlinks':
admin/engines/sql-engines.rst:144: WARNING: hardcoded link 'https://pypi.org/project/mysql-connector-python' could be replaced by an extlink (try using ':pypi:`mysql-connector-python`' instead)
docs/admin/installation-switch2ng.rst:10: WARNING: hardcoded link 'https://github.com/searxng/searxng/pull/446#issuecomment-954730358' could be replaced by an extlink (try using ':pull:`446#issuecomment-954730358`' instead)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
$ make test.shell
./manage line 716:
build_msg TEST "[reST markup] ${RST_FILES[@]}""
^-------------^ SC2145: Argument mixes string and array. Use * or separate argument.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The less grunt runner silently ignore missing files and continue with the build[1]::
Running "less:production" (less) task
>> Destination css/searxng.min.css not written because no source files were found.
>> 1 stylesheet created.
>> 1 sourcemap created.
Add filter function that calls grunt.fail() if the scr file does not exists.
[1] https://github.com/searxng/searxng/pull/750#discussion_r784357031
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
An ambiguous bang like `!!d` raises an exception in function get_bang_url(). A
bang is only unique when the bang_definition from get_bang_definition_and_ac() is
a string / for a ambiguous bang the returned bang_definition is a dictionary.
Reported-by: user prg at #searxng:matrix.org on 2022/01/11
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Startpage has introduced new anti-scraping measures that make SearXNG instances
run into captchas:
1. some arguments has been removed and a new `sc` has been added.
2. search path changed from `do/search` to `sp/search`
3. POST request is no longer needed
Closes: https://github.com/searxng/searxng/issues/692
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
api.openverse.engineering is a little picky and wants to have a trailing slash
in the path:
/v1/images? -->/ v1/images/?
otherwise it redirects, here is the debug log:
DEBUG searx.network.openverse : HTTP Request: GET https://api.openverse.engineering/v1/images?&page=1&page_size=20&format=json&q=foo "HTTP/2 301 Moved Permanently" (text/html; charset=utf-8)
DEBUG searx.network.openverse : HTTP Request: GET https://api.openverse.engineering/v1/images/?&page=1&page_size=20&format=json&q=foo "HTTP/2 200 OK" (application/json)
WARNING searx.engines.openverse : ErrorContext('searx/search/processors/online.py', 105, 'count_error(', None, '1 redirects, maximum: 0', ('200', 'OK', 'api.openverse.engineering')) True
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The implementation of the etools engine is poor. No date-range support, no
language support and it is broken by a CAPTCHA.
etools is a metasearch engine, the major search engines it supports (google,
bing, wikipedia, Yahoo) are already available in SeaarXNG.
While etools does support several engines we currently don't support directly,
support for them should be added directly to SearXNG if there is demand.
In practice: in SearXNG the worse etools results will be mixed with good results
from other engines we have (as long as there is no captcha).
At best case, what we win with etools is in e.g. results from de.ask.com in a
query from a german request .. in all other cases worse results are bubble up in
SearXNG's result list.
[1] https://github.com/searxng/searxng/issues/696#issuecomment-1005855499
Closes: https://github.com/searxng/searxng/issues/696
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Commit c7f27404 moves the `python -m searx.webapp` process to the background. A
background job can't open a simple python-debugger (pdb) console.
This patch moves all processes to the background except the searx.webapp.
Insert a break point somewhere in the webapp application::
import pdb
pdb.set_trace()
And start a debug session by::
make run
and test you break point.
To test that the entire 'make run' stops in the case of an error in the
themes.live [1] background process try:
make LIVE_THEME=typo-theme-name run
[1] https://github.com/searxng/searxng/pull/664#discussion_r776419585
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Add a redis connector, the default DB connector is a socket at::
unix:///usr/local/searxng-redis/run/redis.sock?db=0
To set up a redis instance simply use::
$ ./manage redis.build
$ sudo -H ./manage redis.install
A hint for developers:
To get access rights to this instance, your developer account needs to be added
to the *searxng-redis* group::
$ sudo -H ./manage redis.addgrp "${USER}"
# don't forget to logout & login to get member of group
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
A script to build & install a simple & isolated redis service, dedicated to
SearXNG and connected via Unix socket.
$ ./manage redis.help
redis.:
devpkg : install essential packages to compile redis
build : build redis binaries at /800GBPCIex4/share/SearXNG/dist/redis/6.2.6/amd64
install : create user (searxng-redis) and install systemd service (searxng-redis)
remove : delete user (searxng-redis) and remove service (searxng-redis)
shell : start bash interpreter from user searxng-redis
src : clone redis source code to <path> and checkput 6.2.6
useradd : create user (searxng-redis) at /usr/local/searxng-redis
userdel : delete user (searxng-redis)
addgrp : add <user> to group (searxng-redis)
rmgrp : remove <user> from group (searxng-redis)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Markus Heiser