retoor/searxng
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
4,941 Commits 6 Branches 17 Tags
e2c8e2855cdcd0740d9e93d2e98cd9222cdb9c5b
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Alexandre Flament a582cf3d82 Merge pull request #353 from return42/stackexchange 
[mod] engines - add Stack Exchange API v2.3
 2021-09-29 10:38:50 +02:00
Alexandre Flament 7124fd1704 Merge pull request #354 from dalf/mod-searx-network-stream 
[mod] /image_proxy: fix memory leak
 2021-09-29 10:31:43 +02:00
Alexandre Flament a9c3c88cc0 [mod] searx.network.stream returns a tuple (response, stream) 2021-09-28 19:33:29 +02:00
Alexandre Flament c23aa5760c Merge pull request #343 from dalf/fix-checker-memory-issue 
[fix] checker: fix memory usage
 2021-09-28 19:32:59 +02:00
Alexandre Flament 29893cf816 [fix] searx.network.stream: fix memory leak 2021-09-28 19:28:12 +02:00
Alexandre Flament 2eab89b4ca [fix] checker: fix memory usage 
* download images using the "image_proxy" network (HTTP/1 instead of HTTP/2)
* don't cache data: URL (reduce memory usage)
* after each test: purge image URL cache then call garbage collector
* download only the first 64kb of images
 2021-09-28 15:26:02 +02:00
Alexandre Flament ca67f1555a Merge pull request #350 from MrPaulBlack/fix-engine-description-position 
[fix] engine description position in simple theme
 2021-09-28 09:22:21 +02:00
Alexandre Flament a1a53859c2 Merge pull request #348 from return42/no-arial 
[theme] simple: do not prefer arial over user's sans-serif font
 2021-09-27 18:30:17 +02:00
Alexandre Flament a1c55caa71 Merge pull request #345 from MrPaulBlack/simple-theme-changes 
simple theme css refactor
 2021-09-27 16:19:51 +02:00
Alexandre Flament cc3c54f4cd simple theme: update 2021-09-27 12:26:54 +02:00
Alexandre Flament b046322c7b Merge pull request #333 from dalf/enh-engine-descriptions 
RFC: /preferences: display engine descriptions
 2021-09-25 11:29:25 +02:00
Alexandre Flament 0c3314b338 Merge pull request #344 from dalf/osm-lang-all-fix 
[fix] openstreetmap engine: map "all" language to English
 2021-09-25 11:29:13 +02:00
Alexandre Flament ab569c1e12 [fix] openstreetmap engine: optmizer SPARQL query 
add
hint:Query hint:optimizer "None".
to the SPARQL query to keep the response time small.

It tells the optimizer to follow the path from ?item to the different property values
instead of the other way around.
See https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service/query_optimization#Property_paths
 2021-09-25 11:16:22 +02:00
Alexandre Flament 39876d9f14 [build] /static 2021-09-24 20:38:56 +02:00
Alexandre Flament 2f1384f198 [enh] themes: display the engine descriptions 2021-09-24 20:38:05 +02:00
Alexandre Flament bfd24d1226 [mod] add /engine_descriptions.json endpoint 
returns engine descriptions (JSON):
* key: engine name
* value: description in the user locale, use English description as a fallback
 2021-09-24 20:20:30 +02:00
Alexandre Flament f8d5fe0f11 [fix] fix searx_extra/update/update_engine_descriptions.py 
write ouput in searx/data/engine_descriptions.json instead of stdout
use the wikipedia engine to fetch the wikipedia descriptions
(for example use tl.wikipedia.org for the Filipino language)
 2021-09-24 20:20:30 +02:00
Alexandre Flament 8961131497 [fix] fix the about section of some engines 2021-09-24 20:20:30 +02:00
Alexandre Flament 636215b1f2 Merge pull request #340 from MrPaulBlack/simple-preferences-refactor 
Simple theme preferences refactor
 2021-09-24 20:19:03 +02:00
Alexandre Flament 6f11b61cd5 [fix] openstreetmap engine: map "all" language to English 2021-09-24 20:12:18 +02:00
Alexandre Flament e63410d2f4 Merge pull request #337 from MrPaulBlack/fix-searxjs-null-exception 
[fix] fix searx.js null pointer because of missing category div
 2021-09-22 22:09:54 +02:00
Alexandre Flament 12c90a1b47 Merge pull request #336 from MrPaulBlack/simple-search-form 
[theme] remove categories, language and time range from index.html
 2021-09-22 18:20:33 +02:00
Alexandre Flament b04b7e6fb5 Merge pull request #330 from dalf/fix-323 
Oscar theme: fix search input field on Safari
 2021-09-20 18:47:05 +02:00
Alexandre Flament e8e81c7432 [build] /static 2021-09-19 22:01:19 +02:00
Alexandre Flament 2cc6301c98 [fix] Oscar theme: fix input field on Safari 
close #323
 2021-09-19 22:00:59 +02:00
Alexandre Flament d95c46d646 Merge pull request #329 from searxng/translations_update 
Update translations
 2021-09-17 15:53:45 +02:00
Alexandre Flament 07a2633aee Merge pull request #327 from dalf/pyenv.activate 
[mod] utils/lib.sh: add pyenv.activate
 2021-09-17 15:48:51 +02:00
Alexandre Flament e5e322e8ed [mod] utils/lib.sh: add pyenv.activate 
* initialize virtualenv (pyenv.install)
* activate virtualenv

close #310
 2021-09-17 14:15:35 +02:00
Alexandre Flament dc74df3a55 Merge pull request #261 from dalf/upgrade_httpx 
[upd] upgrade httpx 0.19.0
 2021-09-17 11:48:37 +02:00
Alexandre Flament 0d9959e649 Merge pull request #325 from searxng/dependabot/pip/master/pylint-2.11.1 
Bump pylint from 2.10.2 to 2.11.1
 2021-09-17 11:36:51 +02:00
Alexandre Flament 41f6359d06 [fix] error recorder: avoid RuntimeError on some rare occasion 
httpx.RequestError (subclass of httpx.HTTPError) has a property request.
This property raises a RuntimeError if the attributes _request is None.
To avoid a cascade of errors, this commit reads directly the _request attribute.
 2021-09-17 10:06:24 +02:00
Alexandre Flament b10403d3a1 [mod] searx.network: remove redundant code 
searx.client.new_client: the proxies parameter is a dictonnary,
and the protocol (key of the dictionnary) is already normalized
(see usage of searx.network.network.PROXY_PATTERN_MAPPING)
 2021-09-17 10:06:24 +02:00
Alexandre Flament 8e73438cbe [upd] upgrade httpx 0.19.0 
adjust searx.network module to the new internal API
see https://github.com/encode/httpx/pull/1522
 2021-09-17 10:06:22 +02:00
Alexandre Flament 602cbc2c99 Merge pull request #297 from dalf/engine-logger-enh 
debug mode: more readable logging
 2021-09-14 07:06:28 +02:00
Alexandre Flament 823a2fc792 Merge pull request #317 from return42/fix-manage-usage 
[fix] manage help(): commands babel.* has been renamed to weblate.*
 2021-09-13 09:09:48 +02:00
Alexandre Flament f8793fbda0 [fix] logger per engine: make .logger is always initialized 
the openstreetmap engine imports code from the wikidata engine.
before this commit, specific code make sure to copy the logger variable to the wikidata engine.

with this commit searx.engines.load_engine makes sure the .logger is initialized.
The implementation scans sys.modules for module name starting with searx.engines.
 2021-09-13 08:47:59 +02:00
Alexandre Flament 0f43b39eac [enh] add hostname_replace plugin 
* backport of https://github.com/searx/searx/pull/2724
* allow to remove result if the replacement is the boolean value false
 2021-09-11 13:23:06 +02:00
Alexandre Flament 2de95af923 Merge pull request #313 from dalf/fix-pr-257 
[fix] PR #257: use the image_proxy network instead of the default network
 2021-09-11 11:22:29 +02:00
Alexandre Flament 2b53d718e4 [fix] PR #257: use the image_proxy network instead of the default network 2021-09-11 11:15:51 +02:00
Alexandre Flament 6abc37efc7 [mod] production mode: make sure to set the log level to WARNING 
set:
* the root logger level to WARNING
* the werkzeug logger level to WARNING (useful when there is no uwsgi)
 2021-09-11 10:13:16 +02:00
Alexandre Flament 0e42db9da1 [mod] xpath engine: remove logging of the requested URL 2021-09-11 10:13:16 +02:00
Alexandre Flament 91a6d80e82 [mod] debug mode: log HTTP requests with network name 
For example wikipedia requests use the logger name "searx.network.wikipedia"

Log is disable when searx_debug is False
 2021-09-11 10:13:14 +02:00
Alexandre Flament 577998106a [mod] debug mode: use coloredlogs package 
in debug mode, add some padding to make the output more readable
in procution mode, add the timestamp
 2021-09-10 21:49:34 +02:00
Alexandre Flament b513917ef9 [mod] searx.metrics & searx.search: use the engine loggers 
metrics & processors use the engine logger
 2021-09-10 21:49:34 +02:00
Alexandre Flament d9658a7f91 [fix] searx/results.py: strip result['content'] only if it exists 
fix PR #302
 2021-09-10 17:58:19 +02:00
Alexandre Flament 75daef8986 Merge pull request #309 from searxng/ci-python-3.10.0-rc.2 
[ci] add Python 3.10.0-rc.2
 2021-09-10 13:20:41 +02:00
Alexandre Flament ca73a37679 [ci] add Python 3.10.0-rc.2 2021-09-10 13:13:49 +02:00
Alexandre Flament 4833d85e2e [doc] update copyright notice and HTML title 2021-09-10 11:49:57 +02:00
Alexandre Flament 0b27c8698f [doc] update docs/dev/plugins.rst 2021-09-10 10:58:22 +02:00
Alexandre Flament 70c9bb6f2c [fix] remove references to transifex 2021-09-09 21:29:55 +02:00
Alexandre Flament b941763e20 [mod] ahmia_filter: use on_result instead of post_search 
see commit 6c9ae7911e9639bc46cd53af215734b4bdb61ba9
 2021-09-09 11:31:46 +02:00
Alexandre Flament 660c180170 [mod] plugin: call on_result after each engine from the ResultContainer 
Currently, searx.search.Search calls on_result once the engine results have been merged
(ResultContainer.order_results).

on_result plugins can rewrite the results: once the URL(s) are modified, even they can be merged,
it won't be the case since ResultContainer.order_results has already be called.

This commit call on_result inside for each result of each engines.
In addition the on_result function can return False to remove the result.

Note: the on_result function now run on the engine thread instead of the Flask thread.
 2021-09-09 11:31:44 +02:00
Alexandre Flament 1973e4ecf6 Merge pull request #303 from return42/fix-README 
[fix] README - fix minor typo
 2021-09-08 09:39:45 +02:00
Alexandre Flament fc20c561bf [mod] oa_doi_rewrite plugin: get_doi_resolver: remove args parameter 
doi_resolvers.get_value('preferences') already contains the value from
request.args.get('doi_resolver')
 2021-09-07 19:14:36 +02:00
Alexandre Flament a3789b3bb4 Merge pull request #295 from return42/fix-csp-2 
[fix] make HTML templates more CSP compliant
 2021-09-07 18:51:00 +02:00
Alexandre Flament e41f78b806 Merge pull request #301 from return42/fix-lintrc 
[pylint] .pylintrc - disable missing-function-docstring
 2021-09-07 15:09:40 +02:00
Alexandre Flament 9ef7f38e4f Merge pull request #300 from return42/fix-engine-pylint 
[fix] drop useless pylint: disable=undefined-variable
 2021-09-07 10:43:16 +02:00
Alexandre Flament f869e73d2f Merge pull request #299 from dalf/fix-osm-workaround 
[fix] fix openstreetmap engine
 2021-09-06 22:54:27 +02:00
Alexandre Flament ea60c03827 [fix] fix openstreetmap engine 
close #298

This is a workaround: inside engine code, any call to function in another engine can crash
since the logger won't be initialized except if it is done explicitly.
 2021-09-06 22:44:22 +02:00
Alexandre Flament b57d776edb Merge pull request #293 from dalf/fix-290 
[fix] translation: lock weblate only when there is an actual change of messages.pot
 2021-09-06 20:00:19 +02:00
Alexandre Flament 17e739a859 Merge pull request #296 from return42/engine-logger 
one logger per engine
 2021-09-06 19:50:58 +02:00
Alexandre Flament 2a5f690a71 [fix] translation: lock weblate only when there is an actual change of messages.pot 
Close #290
 2021-09-06 17:16:51 +02:00
Alexandre Flament a26e4eff10 Merge pull request #292 from return42/fix-csp 
[fix] make HTML templates more CSP compliant
 2021-09-06 11:45:15 +02:00
Alexandre Flament 33ea1d5909 Merge pull request #289 from return42/fix-xpath 
[fix] remove minimum length of content for XPath engine
 2021-09-04 15:34:18 +02:00
Alexandre Flament 39bbd15b39 Merge pull request #287 from return42/azlyrics 
[enh] add engine AZLyrics.com (XPath)
 2021-09-04 13:36:09 +02:00
Alexandre Flament 3b1acdd5c5 Merge pull request #288 from return42/enh-brave 
[enh] add suggestions to brave engine
 2021-09-04 13:25:42 +02:00
Alexandre Flament 2f348216b0 Merge pull request #283 from searxng/update-readme 
README.rst: differences with searx
 2021-09-03 16:58:55 +02:00
Alexandre Flament 94458b4881 README.rst: differences with searx 2021-09-03 16:54:24 +02:00
Alexandre Flament a14ea6396e [mod] remove searx.raise_for_httperror 
Not used anymore
 2021-09-02 16:24:25 +02:00
Alexandre Flament a7d781c49d [mod] remove remaining references to locales in settings.yml 
see #247
 2021-09-02 16:19:18 +02:00
Alexandre Flament 3c377c0aa8 [mod] searx.webapp: code clean up 2021-09-02 16:16:20 +02:00
Alexandre Flament b9c73fb697 [mod] move searx/testing.py to the tests directory 
move robot tests to tests.robot
manage calls "python -m tests.robot"
 2021-09-02 16:01:34 +02:00
Alexandre Flament d246679dbe [build] /static 2021-08-31 15:43:28 +02:00
Alexandre Flament f77f797f8c [mod] simple theme: fix eslint errors, fix VIM keys 
* VIM keys: fix the next page and previous pages (n, p keys)
* Map: Enable the wikipedia map (the layer was initialized but not included)
 2021-08-31 15:41:31 +02:00
Alexandre Flament 66a27757fd Merge pull request #244 from return42/eslint-simple-client 
[mod] simple theme - add ESLint
 2021-08-31 13:24:06 +02:00
Alexandre Flament 0ee316f3d1 eslint: grunt integration 2021-08-31 12:20:23 +02:00
Alexandre Flament 7dc370efe9 [enh] new issue: include information from searx.version 2021-08-31 11:45:33 +02:00
Alexandre Flament 4494589227 Merge pull request #272 from searxng/dependabot/pip/master/yamllint-1.26.3 
Bump yamllint from 1.26.2 to 1.26.3
 2021-08-27 12:00:05 +02:00
Alexandre Flament 6edf98216f Merge pull request #274 from searxng/translations_update 
Update translations
 2021-08-27 11:59:40 +02:00
Alexandre Flament 38e0b9360b Merge pull request #257 from dalf/mod_image_proxy 
Mod image proxy
 2021-08-25 11:24:27 +02:00
Alexandre Flament 697ebeddcc [mod] searx.utils.dict_subset: rewrite with comprehension 2021-08-24 15:28:08 +02:00
Alexandre Flament 3b0f70ed0f [mod] /image_proxy: use HTTP/1 instead of HTTP/2 
httpx: HTTP/2 is slow when a lot data is downloaded.
https://github.com/dalf/pyhttp-benchmark

also, the usage of HTTP/1 decreases the load average
 2021-08-24 14:51:20 +02:00
Alexandre Flament 43fcaa642a [fix] image_proxy: always close the httpx respone 
previously, when the content type was not an image and some other error,
the httpx response was not closed
 2021-08-24 14:51:20 +02:00
Alexandre Flament df15c655f7 [mod] /image_proxy: don't decompress images 2021-08-24 14:51:20 +02:00
Alexandre Flament dec9df28f3 Merge pull request #264 from searxng/dependabot/pip/master/pygments-2.10.0 
Bump pygments from 2.9.0 to 2.10.0
 2021-08-24 09:36:58 +02:00
Alexandre Flament 6f9e3868d6 Merge pull request #267 from dalf/backport_google 
[enh] google: add filter=0 to Google engine for more results
 2021-08-21 17:51:18 +02:00
Alexandre Flament 102c815231 Merge pull request #265 from searxng/translations_update 
Update translations
 2021-08-20 11:44:11 +02:00
Alexandre Flament 6844e3a617 Merge pull request #263 from return42/fix-config 
[fix] /config add missing GIT_BRANCH value
 2021-08-18 20:04:26 +02:00
Alexandre Flament 771691e3b0 README.rst : add weblate link 2021-08-17 10:58:26 +02:00
Alexandre Flament 9760e763f1 Merge pull request #262 from searxng/translations_update 
Update translations
 2021-08-17 10:51:44 +02:00
Alexandre Flament 28b350eda6 Merge pull request #250 from dalf/weblate_v2 
[translations] weblate integration
 2021-08-17 10:38:50 +02:00
Alexandre Flament 28ccf47ca4 [mod] searx.webapp.get_locale: read locale from the preferences 
pre_request already set the locale.
 2021-08-17 08:18:30 +02:00
Alexandre Flament a925254c3d Merge pull request #258 from dalf/plugin_minor_change 
[mod] plugins: minor change
 2021-08-16 17:10:28 +02:00
Alexandre Flament 0b0071af00 Merge pull request #259 from dalf/fix_checker 
[fix] searx.search.checker.get_result() always return a dict
 2021-08-16 14:00:17 +02:00
Alexandre Flament 2f363858b8 [fix] searx.search.checker.get_result() always return a dict 
So checker_results['status'] == 'ok' is enough to check the checker result.
See searx/webapp.py, /preferences endpoint
 2021-08-16 08:29:16 +02:00
Alexandre Flament 3f3b5d6181 [mod] plugins: minor change 
required attributes: display a different message
when the attribute has the wrong type
 2021-08-14 18:03:31 +02:00
Alexandre Flament 9667142d5c Merge pull request #255 from searxng/dependabot/pip/master/uvloop-0.16.0 
Bump uvloop from 0.15.3 to 0.16.0
 2021-08-13 09:15:42 +02:00
Alexandre Flament 7f18cbbee0 Merge pull request #253 from return42/minor-fix 
[fix] settings.yml - replace http:// URLs by https://
 2021-08-07 18:12:44 +02:00
Alexandre Flament 901ab87717 [translations] web integration 
* make babel.translations.to.master: pull weblate updates
* make babel.master.to.translations: push .pot and .po files to weblate
 2021-08-07 15:06:06 +02:00
Alexandre Flament 4e2ec9b8b3 Merge pull request #247 from dalf/remote_settings_locales 
settings.yml: remove locales section.
 2021-08-04 11:05:13 +02:00