[mod] set 'engine.supported_languages' from the origin python module

Browse Source 
The key of the dictionary 'searx.data.ENGINES_LANGUAGES' is the *engine name*
configured in settings.xml.  When multiple engines are configured to use the
same origin engine (e.g. `engine: google`)::

    - name: google
      engine: google
      use_mobile_ui: false
      ...

    - name: google italian
      engine: google
      use_mobile_ui: false
      language: it
      ...

    - name: google mobile ui
      engine: google
      shortcut: gomui
      use_mobile_ui: true

There exists no entry for ENGINES_LANGUAGES[engine.name] (e.g. `name: google
mobile ui` or `name: google italian`).  This issue can be solved by recreate the
ENGINES_LANGUAGES::

    make data.languages

But this is nothing an SearXNG admin would like to do when just configuring
additional engines, since this just doubles entries in ENGINES_LANGUAGES and
BTW: `make data.languages` has various external requirements which might be not
installed or not available, on a production host.

With this patch, if engine.name fails, ENGINES_LANGUAGES[engine.engine] is used
to get the engine.supported_languages (e.g. `google` for the engine named
`google mobile`).

For an engine, when there is `language: ...` in the YAML settings, the engine
supports only one language, in this case engine.supported_languages should
contains this value defined in settings.yml (e.g. `it` for the engine named
`google italian`).

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Closes: https://github.com/searxng/searxng/issues/384
This commit is contained in:
Markus Heiser 2021-10-06 18:02:29 +02:00
parent feb2d4dda5
commit 19e41c137e
2 changed files with 38 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

29
searx/engines/__init__.py
View File

@ -137,7 +137,7 @@ def update_engine_attributes(engine, engine_data):
if isinstance(param_value, str): if isinstance(param_value, str):
param_value = list(map(str.strip, param_value.split(','))) param_value = list(map(str.strip, param_value.split(',')))
engine.categories = param_value engine.categories = param_value
elif param_name != 'engine': else:
setattr(engine, param_name, param_value) setattr(engine, param_name, param_value)
# set default attributes # set default attributes
@ -147,11 +147,35 @@ def update_engine_attributes(engine, engine_data):
def set_language_attributes(engine): def set_language_attributes(engine):
# pylint: disable=protected-access
# assign supported languages from json file # assign supported languages from json file
if engine.name in ENGINES_LANGUAGES: if engine.name in ENGINES_LANGUAGES:
engine.supported_languages = ENGINES_LANGUAGES[engine.name] engine.supported_languages = ENGINES_LANGUAGES[engine.name]
elif engine.engine in ENGINES_LANGUAGES:
# The key of the dictionary ENGINES_LANGUAGES is the *engine name*
# configured in settings.xml. When multiple engines are configured in
# settings.yml to use the same origin engine (python module) these
# additional engines can use the languages from the origin engine.
# For this use the configured ``engine: ...`` from settings.yml
engine.supported_languages = ENGINES_LANGUAGES[engine.engine]
if hasattr(engine, 'language'):
# For an engine, when there is `language: ...` in the YAML settings, the
# engine supports only one language, in this case
# engine.supported_languages should contains this value defined in
# settings.yml
if engine.language not in engine.supported_languages:
raise ValueError(
"settings.yml - engine: '%s' / language: '%s' not supported" % (
engine.name, engine.language ))
if isinstance(engine.supported_languages, dict):
engine.supported_languages = {
engine.language : engine.supported_languages[engine.language]
}
else:
engine.supported_languages = [engine.language]
# find custom aliases for non standard language codes # find custom aliases for non standard language codes
for engine_lang in engine.supported_languages: for engine_lang in engine.supported_languages:
iso_lang = match_language(engine_lang, BABEL_LANGS, fallback=None) iso_lang = match_language(engine_lang, BABEL_LANGS, fallback=None)
@ -172,6 +196,7 @@ def set_language_attributes(engine):
'Accept-Language': 'ja-JP,ja;q=0.8,en-US;q=0.5,en;q=0.3', # bing needs a non-English language 'Accept-Language': 'ja-JP,ja;q=0.8,en-US;q=0.5,en;q=0.3', # bing needs a non-English language
} }
engine.fetch_supported_languages = ( engine.fetch_supported_languages = (
# pylint: disable=protected-access
lambda: engine._fetch_supported_languages( lambda: engine._fetch_supported_languages(
get(engine.supported_languages_url, headers=headers)) get(engine.supported_languages_url, headers=headers))
) )

11
searx/settings.yml
View File

@ -601,6 +601,17 @@ engines:
# additional_tests: # additional_tests:
# android: *test_android # android: *test_android
# - name: google italian
# engine: google
# shortcut: goit
# use_mobile_ui: false
# language: it
# - name: google mobile ui
# engine: google
# shortcut: gomui
# use_mobile_ui: true
- name: google images - name: google images
engine: google_images engine: google_images
shortcut: goi shortcut: goi