retoor/searxng
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,946 Commits 6 Branches 17 Tags
731206d0036210b9c8d858feebeea3dbfcbd5d5e
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Alexandre Flament fd1b8f661b Update dependabot.yml 2021-02-16 18:42:44 +01:00
Alexandre Flament 3ee24b685a Create dependabot.yml 
Only pip packages and ignore pygments, related to https://github.com/searx/searx/pull/2206
 2021-02-16 18:41:34 +01:00
Alexandre Flament 44a6593c13 Merge pull request #2573 from unixfox/yggtorrent 
update yggtorrent url + add it back
 2021-02-16 08:22:07 +01:00
Alexandre Flament 09e28ae779 Merge pull request #2570 from xenrox/improve-peertube 
Improve peertube searching
 2021-02-16 08:17:42 +01:00
Alexandre Flament aef1161633 data-update.yml: run on Friday, 16:05 UTC 2021-02-12 17:19:44 +01:00
Alexandre Flament 45027765e3 Merge pull request #2566 from dalf/remove-yandex 
[remove] yandex engine
 2021-02-12 17:12:07 +01:00
Alexandre Flament 50bde93dbb data-update.yml: run on searx org 2021-02-12 16:56:00 +01:00
Alexandre Flament f59acf4138 data-update.yml: run on Friday 
give the weekend to review the PR
 2021-02-12 11:15:46 +01:00
Alexandre Flament ddab00abf4 Merge pull request #2559 from dalf/upd-duckduckgo 
[fix] duckduckgo engine: "!ddg !g" do not redirect to google
 2021-02-12 11:14:09 +01:00
Alexandre Flament c22d4c764c [fix] duckduckgo engine: "!ddg !g" do not redirect to google 
* searx understand "!ddg !g time" as : send "!g time" to DDG
* !g a DDG bang for Google: DDG return a HTTP redirect to Google

This commit adds a the allows_redirect param not to follow HTTP redirect.

The DDG engine returns a empty result as before without HTTP redirect.
 2021-02-12 11:10:08 +01:00
Alexandre Flament d76660463b Merge pull request #2562 from dalf/mod-json-engine 
[mod] json_engine: add content_html_to_text and title_html_to_text
 2021-02-12 10:58:28 +01:00
Alexandre Flament 7dcf67a47a Merge pull request #2565 from dalf/upd-wikipedia 
[upd] wikipedia engine: return an empty result on query with illegal characters
 2021-02-12 10:57:05 +01:00
Alexandre Flament 2b60d0d243 Merge pull request #2564 from dalf/fix-seznam 
[fix] fix seznam engine
 2021-02-12 10:56:53 +01:00
Alexandre Flament 7e83818879 Merge pull request #2560 from dalf/fix-duckduckgo 
Fix duckduckgo
 2021-02-12 10:56:40 +01:00
Alexandre Flament 63d6ccfbc2 Merge pull request #2557 from dalf/fix-raise_for_httperror 
Fix: activate raise_for_error by default
 2021-02-12 10:56:25 +01:00
Alexandre Flament 74c8b5606f Merge pull request #2541 from return42/mediathekviewweb 
[enh] add engine MediathekViewWeb (API)
 2021-02-11 15:11:26 +01:00
Alexandre Flament 5d9db6c2f7 [remove] yandex engine 2021-02-11 14:28:06 +01:00
Alexandre Flament 35dd069402 [fix] fix seznam engine 
no paging support
 2021-02-11 12:53:19 +01:00
Alexandre Flament 7d6e69e2f9 [upd] wikipedia engine: return an empty result on query with illegal characters 
on some queries (like an IT error message), wikipedia returns an HTTP error 400.
this commit returns an empty result instead of showing an error to the user.
 2021-02-11 12:29:21 +01:00
Alexandre Flament ff84a1af35 [mod] json_engine: add content_html_to_text and title_html_to_text 
Some JSON API returns HTML in either in the HTML or the content.
This commit adds two new parameters to the json_engine:
content_html_to_text and title_html_to_text, False by default.

If True, then the searx.utils.html_to_text removes the HTML tags.

Update crossref, openairedatasets and openairepublications engines
 2021-02-10 16:42:11 +01:00
Alexandre Flament 436d366448 Merge pull request #2544 from mrwormo/congresslibrary 
[Engine] Add Library of Congress engine
 2021-02-10 10:13:46 +01:00
Alexandre Flament eafd27f42a Merge pull request #2556 from dalf/fix-apk-mirror 
[fix] fix apk_mirror engine
 2021-02-10 10:12:37 +01:00
Alexandre Flament c40316d957 Merge pull request #2558 from dalf/remove-google-play-music 
[upd] remove google_play_music engine
 2021-02-10 10:12:21 +01:00
Alexandre Flament d2dac11392 [mod] duckduckgo engine: better support of the language preference 
After the main request, send a second to https://duckduckgo.com/t/sl_h

See https://github.com/searx/searx/issues/2259
 2021-02-09 14:36:43 +01:00
Alexandre Flament 74d56f6cfb [mod] poolrequests: for one (user request, engine) always use the same HTTPAdapter 
The duckduckgo engine requires an additional request after the results have been sent.
This commit makes sure that the second request uses the same HTTPAdapter
= the same IP address, and the same proxy.
 2021-02-09 14:33:36 +01:00
Alexandre Flament 9211cdfe9b [upd] remove google_play_music engine 
Google Play Music has been replaced by Youtube music.
 2021-02-09 11:38:50 +01:00
Alexandre Flament aedf03c0f7 Fix: activate raise_for_error by default 
Fix commit d703119d3a :
Some engines need to parse the HTTP error but
raise_for_error is always set to False in the "request" function.
 2021-02-09 11:27:41 +01:00
Alexandre Flament 5e055b069b [fix) fix apk_mirror engine 2021-02-09 11:02:12 +01:00
Alexandre Flament f03ad0a3c0 Merge pull request #2555 from dalf/fix-github-data-update 
[fix] fix github action data-update.yml
 2021-02-09 10:48:55 +01:00
Alexandre Flament 966a7a1f25 [fix] fix github action data-update.yml 2021-02-09 09:58:59 +01:00
Alexandre Flament e4cc7f13a3 Merge pull request #2542 from kvch/fix-naver-engine 
Fix XPATHs in Naver engine
 2021-02-09 08:52:38 +01:00
Alexandre Flament bec9e30fe7 Merge pull request #2554 from MarcAbonce/zh-variants-in-wikipedia 
Add support for Chinese variants in Wikipedia
 2021-02-09 08:49:59 +01:00
Alexandre Flament 6c513095e4 Merge pull request #2553 from danielhones/improve-results-highlighting-updated 
Ignore double-quotes when highlighting query parts
 2021-02-09 08:39:07 +01:00
Alexandre Flament 1e35c3ccce Merge pull request #2531 from MarcAbonce/fix-browser-locale 
[fix] Get correct locale with country from browser
 2021-02-05 10:55:37 +01:00
Alexandre Flament 321788f14a Merge pull request #2528 from dalf/mod-ci-gh-pages 
[mod] CI: minor changes
 2021-02-04 23:12:27 +01:00
Alexandre Flament 90b9d0d6a8 [mod] CI: minor changes 
* utils/makefile.python: travis-gh-pages renamed ci-gh-pages
 2021-02-02 08:53:57 +01:00
Alexandre Flament 34de715e62 Merge pull request #2500 from dalf/github-action-data 
[enh] every Sunday, call utils/fetch_*.py scripts and create a PR automatically
 2021-02-01 17:16:58 +01:00
Alexandre Flament 1742355eb8 Merge pull request #2499 from dalf/remove-language-support-variable 
[mod] dynamically set language_support variable
 2021-02-01 17:16:18 +01:00
Alexandre Flament ca93a01844 [mod] dynamically set language_support variable 
The language_support variable is set to True by default,
and set to False in only 5 engines.

Except the documentation and the /config URL, this variable is not used.

This commit remove the variable definition in the engines, and
set value according to supported_languages length: False when the length is 0,
True otherwise.

Close #2485
 2021-02-01 17:10:37 +01:00
Alexandre Flament 99244440e4 Merge pull request #2514 from return42/fix-gh-pages 
[fix] Makefile target gh-pages & flatten history of branch gh.pages
 2021-02-01 17:07:08 +01:00
Alexandre Flament 0a8799b834 Merge pull request #2517 from dalf/debug-ci 
Update pyenv pyenvinstall Make targets
 2021-02-01 17:01:34 +01:00
Alexandre Flament d70c5a621a [mod] more robust make pyenv / make pyenvinstall 
"make pyenv" ensures that ./local/py3/bin/python is an executable
 2021-02-01 16:58:04 +01:00
Alexandre Flament 806af50738 Merge pull request #2494 from return42/rm-fabfile 
[fix] remove Fabric file
 2021-02-01 15:09:35 +01:00
Alexandre Flament 71d66979c2 Merge pull request #2482 from return42/fix-google-video 
[fix] revise of the google-Video engine
 2021-01-28 11:11:07 +01:00
Alexandre Flament 0f18e885bf Merge pull request #2479 from Tobi823/master 
Document workaround for using 2 languages simultaneously #1508
 2021-01-27 21:29:42 +01:00
Alexandre Flament b661c3f5d4 Merge pull request #2509 from return42/fix-morty-key 
[doc] improve admin-docs about result proxy (morty) configuration
 2021-01-27 15:31:29 +01:00
Alexandre Flament 6047087aac [mod] utils/fetch_languages.py: write files at the right location 2021-01-24 14:25:27 +01:00
Alexandre Flament 3330cf4a46 [enh] every monday, call utils/fetch_*.py scripts and create a PR automatically 2021-01-24 13:32:39 +01:00
Alexandre Flament f4a17acb7a Merge pull request #2498 from dalf/minor-fix-google-news 
[fix] google_news: avoid one HTTP redirect except for the English results
 2021-01-24 09:13:48 +01:00
Alexandre Flament 96c2996857 Merge pull request #2497 from return42/fix-test.sh 
[fix] lxc.sh - SC2034: ubu2010_boilerplate appears unused.
 2021-01-24 09:06:11 +01:00
Alexandre Flament 8c46b767d0 [fix] google_news: avoid one HTTP redirect except for the English results 
also add
params['soft_max_redirects'] = 1
to avoid false error reporting in /stats/errors
 2021-01-24 08:53:35 +01:00
Alexandre Flament 7d24850d49 Merge pull request #2483 from return42/fix-google-news 
[fix] revise of the google-News engine
 2021-01-23 20:21:09 +01:00
Alexandre Flament 73c86f9bf2 [mod] checker: disable by default 2021-01-19 21:44:48 +01:00
Alexandre Flament 3b7b852aa8 [fix] checker: minor fix about language detection 2021-01-19 21:29:31 +01:00
Alexandre Flament aa887eb375 [mod] checker : replace pycld3 by langdetect 
pycld3 requires the native library cld3
langdetect is a pure python package
 2021-01-19 21:26:04 +01:00
Alexandre Flament 0495e15df4 Merge pull request #2476 from dalf/fix-error-recording-and-checker 
Fix error recording and checker
 2021-01-18 08:29:25 +01:00
Alexandre Flament 67a1aab0d5 [fix] /stats/checker : remove the timestamp field when the checker is disabled 2021-01-18 08:19:53 +01:00
Alexandre Flament d473407ec9 [fix] checker: fix engine statistics 
Without this commit, the URL /stats/errors shows percentage above 100% after the checker has run.
 2021-01-18 08:19:44 +01:00
Alexandre Flament ca76f3119a [fix] error_recorder: record code and lineno about the engine 
since the PR #2225 , code and lineno were sometimes meaningless
see /stats/errors
 2021-01-17 16:25:11 +01:00
Alexandre Flament 80d7411f2c Merge pull request #2452 from kvch/add-wilby-engine 
Add wiby.me engine
 2021-01-16 22:36:31 +01:00
Alexandre Flament b405646749 Merge pull request #2451 from mrwormo/invidious-engine 
[Fix] Invidious Engine
 2021-01-16 19:25:45 +01:00
Alexandre Flament 709dd960f1 Merge pull request #2473 from return42/fix-setup.py 
[fix] setup.py requires pyyaml installed
 2021-01-16 19:05:36 +01:00
Alexandre Flament 1d13ad8452 Merge pull request #2460 from dalf/engine-about 
[enh] engines: add about variable
 2021-01-16 19:05:17 +01:00
Alexandre Flament a4dcfa025c [enh] engines: add about variable 
move meta information from comment to the about variable
so the preferences, the documentation can show these information
 2021-01-14 20:57:17 +01:00
Alexandre Flament 5a511f0d62 [fix] CI: fix docker push 2021-01-14 20:35:10 +01:00
Alexandre Flament 824fe40a28 Merge pull request #2467 from dalf/fix-ci 
[fix] github actions: use ubuntu-20.04 instead of ubuntu-latest
 2021-01-14 17:14:59 +01:00
Alexandre Flament 38090daa29 [fix] github actions: use ubuntu-20.04 instead of ubuntu-latest 2021-01-14 16:49:17 +01:00
Alexandre Flament 484dc99580 Merge pull request #2419 from dalf/checker 
[enh] add checker
 2021-01-13 15:46:48 +01:00
Alexandre Flament 912c7e975c [fix] checker: don't run the checker when uwsgi is not properly configured 
Before this commit, even with the scheduler disabled, the checker was running
at least once for each uwsgi worker.
 2021-01-13 14:07:39 +01:00
Alexandre Flament 7f0c508598 [fix] checker: fix typo unknown instead of unknow 2021-01-12 11:47:17 +01:00
Alexandre Flament a0c8b413a6 [mod] searx.shared: minor tweaks 
searx.shared.shared_abstract.SharedDict inherit from abc.ABC
searx.shared.shared_uwsgi.schedule can schedule multiple functions without issue
 2021-01-12 11:47:17 +01:00
Alexandre Flament 87bafbc32b [mod] checker: add status and timestamp to the result 
for each engine: replace status by success
 2021-01-12 11:47:17 +01:00
Alexandre Flament f3e1bd308f [mod] checker: minor adjustements on the default tests 
the query "time" is convinient because most of the search engine will return some results,
but some engines in the general category will return documentation about the HTML tags <time> or <input type="time">
 2021-01-12 11:47:17 +01:00
Alexandre Flament 45bfab77d0 |mod] checker: improve searx-checker command line 
* output is unbuffered
* verbose mode describe more precisly the errrors
 2021-01-12 11:47:17 +01:00
Alexandre Flament 3a9f513521 [enh] checker: background check 
See settings.yml for the options
SIGUSR1 signal starts the checker.
The result is available at /stats/checker
 2021-01-12 11:47:17 +01:00
Alexandre Flament 6e2872f436 [enh] add searx.shared 
shared dictionary between the workers (UWSGI or werkzeug)
scheduler: run a task once every x seconds (UWSGI or werkzeug)
 2021-01-12 11:47:17 +01:00
Alexandre Flament ca0889d488 [enh] checker: wikidata & ddd: add specific tests 2021-01-12 11:47:17 +01:00
Alexandre Flament 16a889dd8f [enh] checker: add rosebud test 2021-01-12 11:47:17 +01:00
Alexandre Flament 8cbc9f2d58 [enh] add checker 2021-01-12 11:47:17 +01:00
Alexandre Flament f7e11fd722 Merge pull request #2459 from dalf/update-python 
Update python
 2021-01-12 11:02:58 +01:00
Alexandre Flament 9c55d772e9 Merge pull request #2408 from return42/rm-brand-make 
[mod] move brand options from Makefile to settings.yml
 2021-01-12 10:52:42 +01:00
Alexandre Flament 8989bc76cb [mod] remove pyopenssl dependency 
requests[security] is now deprecated since version 2.25.0
 2021-01-12 09:56:56 +01:00
Alexandre Flament d54034a5e6 [mod] add Python 3.9 support 2021-01-12 09:53:26 +01:00
Alexandre Flament f5c3cb7afa [mod] drop Python 3.5 support 2021-01-12 09:45:16 +01:00
Alexandre Flament 8d0312d014 Merge pull request #2458 from MarcAbonce/hide-links-mobile2 
Hide links panel in mobile screens
 2021-01-12 08:27:24 +01:00
Alexandre Flament e7ecf3ab42 Merge pull request #2457 from archiecodes/master 
Fixed typo
 2021-01-12 08:23:28 +01:00
Alexandre Flament 424e6abc7e [mod] settings.yml: move brand settings to a dedicated section 2021-01-11 22:59:52 +01:00
Alexandre Flament d5882147e0 [fix] utils/build_env.py: make sure to always use searx/settings.yml 
ignore SEARX_SETTINGS_PATH
 2021-01-11 22:59:35 +01:00
Alexandre Flament c2646df496 Merge pull request #2454 from MarcAbonce/fix-empty-lang-bang 
Fix empty colon in query from selecting Chinese
 2021-01-10 11:01:32 +01:00
Alexandre Flament 485380cd3e Merge pull request #2447 from unixfox/issue-template-notice 
add notice for the issue templates
 2021-01-08 09:30:23 +01:00
Alexandre Flament 5225c4f253 Merge pull request #2448 from return42/fix-1998 
[fix] Library Genesis links shifted by 1 #1998
 2021-01-08 09:29:36 +01:00
Alexandre Flament 14a395a671 Merge pull request #2445 from searx/unixfox-patch-1 
Remove voat due to its shutdown
 2021-01-06 14:06:49 +01:00
Alexandre Flament fb4e6080ed Merge pull request #2428 from dalf/update-dependencies 
[upd] update Python dependencies
 2021-01-05 16:42:51 +01:00
Alexandre Flament 5b58f6a7f8 Merge pull request #2435 from return42/fix-lxc-env 
[LXC] replace ubu1910 image by ubu2010 image
 2021-01-05 16:41:13 +01:00
Alexandre Flament 54e69d0367 [upd] update dependencies 
minor change in the oscar theme becase the last version of jinja2
respect more carefully the spaces in the templates
 2020-12-28 09:04:39 +01:00
Alexandre Flament 5fb9a5c7c1 Merge pull request #2411 from dalf/update-secret-key-check 
Update secret key check
 2020-12-27 18:04:53 +01:00
Alexandre Flament 5840499bfb Merge pull request #2423 from return42/mod1-boilerplate 
Speed up PyLint & revert obsolete CSS issue
 2020-12-27 14:12:56 +01:00
Alexandre Flament 568b9465e9 [mod] check secret_key when searx.webapp is imported 
Without this commit the module searx checks the secret_key value.

With this commit, make docs, utils/standalone_searx.py,
utils/fetch_firefox_version.py works without SEARX_DEBUG=1

For reference see https://github.com/searx/searx/pull/2386
 2020-12-27 10:30:20 +01:00
Alexandre Flament eb1b5289d5 [mod] documentation: change the jinja context doesn't depend on searx.webapp 
Before this commit, in the documentation, the jinja context is 'webapp' and contains
the global variable in the searx.webapp module.

This commit changes this to include only the mandatory variables to build the
documentation.
 2020-12-27 10:00:35 +01:00
Alexandre Flament 1ba53e6129 Merge pull request #2420 from return42/fix-sphinx-book-targets 
fix makefile `books/{name}.*` targets and add documentation about these targets
 2020-12-27 09:47:49 +01:00