retoor/searxng
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,537 Commits 6 Branches 17 Tags
53c8d945b489b1185d89e02bbc6b6ea6b60a91da
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Alexandre Flament ecb9f28869 Merge pull request #2236 from return42/fix-2234 
[fix] google engine - div classes has been renamed in HTML result
 2020-10-01 10:11:19 +02:00
Alexandre Flament fd5fe36984 Merge pull request #2232 from dalf/fix-sphinx-build2 
[fix] Sphinx 3.2.1 build: call searx.engines.initialize_engines
 2020-09-30 10:29:28 +02:00
Alexandre Flament 9a856083df [fix] Sphinx 3.2.1 build: call searx.engines.initialize_engines 
Fix admin/engines.html
Fix #2230
 2020-09-30 10:20:12 +02:00
Alexandre Flament 6c6985aaf8 Merge pull request #2230 from dalf/fix-sphinx-build 
[fix] Sphinx 3.2.1 build: add missing "engines" variable in the jinja context
 2020-09-30 09:29:09 +02:00
Alexandre Flament 9740618227 Merge pull request #2226 from dalf/fix-searx-migration 
[fix] migration from github.com/asciimoo/searx to github.com/searx/searx : fix URLs
 2020-09-29 12:35:11 +02:00
Alexandre Flament 6d56c754a7 [fix] Sphinx 3.2.1 build: add missing "engines" variable in the jinja context 
Fix the admin/engines.html page
 2020-09-29 12:30:10 +02:00
Alexandre Flament f204e4903d [fix] migration from github.com/asciimoo/searx to github.com/searx/searx : fix URLs 2020-09-28 16:44:14 +02:00
Alexandre Flament c2a6f14516 [fix] Update Sphinx package versions 
* Update call to sphinx-autobuild (remove --poll -B options, add --open-browser)
* Upgrade the Sphinx and sphinx-tabs versions.
* utils/makefile.sphinx: remove the SPHINXVERS variable.
* utils/makefile.sphinx: remove the sphinx-doc and sphinx-live targets, Sphinx is already installed by the pyenvinstall target.
 2020-09-28 13:27:50 +02:00
Alexandre Flament 93f7f7eee2 [mod] upgrade requests to version 2.24.0. use ssl instead of pyopenssl. 
requests 2.24.0 uses the ssl module except if it doesn't support SNI, in this case searx fallbacks to pyopenssl.
searx logs a critical message and exit if the ssl modules doesn't support SNI and pyOpenSSL is not installed.
searx logs a critical message and exit if the ssl version is older than 1.0.2.
in requirements.txt, pyopenssl is still required to install searx as a fallback.
 2020-09-26 19:30:27 +02:00
Alexandre Flament f2f3300bde [mod] more typing 2020-09-24 16:26:00 +02:00
Alexandre Flament 485a502b88 [mod] add typing and __slots__ 2020-09-22 19:05:05 +02:00
Alexandre Flament 678699beaf [mod] searx/webadapter.py: add get_selected_categories share common code with get_search_query_from_webapp 
Update searx/webapp.py to use get_selected_categories
Close #2142
 2020-09-22 18:59:51 +02:00
Alexandre Flament d5b5e48f04 [mod] searx/webadapter.py: each web parameter has a dedicated function (parse_pageno, page_lang,....). 2020-09-22 18:59:51 +02:00
Alexandre Flament 691d12726b [mod] check the engine tokens in searx/webadapter.py instead of searx/search.py 2020-09-22 18:59:51 +02:00
Alexandre Flament eecfff2689 [mod] check time_range parameter in searx/webadapter.py 2020-09-22 18:59:51 +02:00
Alexandre Flament 2929495112 [mod] add searx.search.EngineRef 
was previously a Dict with two or three keys: name, category, from_bang
make clear that this is a engine reference (see tests/unit/test_search.py for example)
all variables using this class are renamed accordingly.
 2020-09-22 18:59:51 +02:00
Alexandre Flament 2dbc0de0cd [mod] add searx/webadapter.py 
* move searx.search.get_search_query_from_webapp to searx.webadapter
* move searx.query.SearchQuery to searx.search
 2020-09-22 18:59:51 +02:00
Alexandre Flament e8df5d3f7f [mod] searx.search: change function declaration: online then offline then utility functions. 2020-09-22 12:36:26 +02:00
Alexandre Flament 50e717c452 [mod] searx.search: remove unused global variable number_of_searches 2020-09-22 12:36:26 +02:00
Alexandre Flament edd8dccd07 [mod] searx.query.RawTextQuery: getSearchQuery and changeSearchQuery rename to getQuery and changeQuery 
getSearchQuery is confusing, the method returns a str not a SearchQuery object
 2020-09-22 12:36:26 +02:00
Alexandre Flament c4067a9d02 [mod] searx.search.Search: split the search method into different methods 2020-09-22 12:14:46 +02:00
Alexandre Flament ad0758e52a [mod] add searx/webutils.py 
contains utility functions and classes used only by webapp.py
 2020-09-22 11:57:06 +02:00
Alexandre Flament f9664037a6 [mod] refactor searx.webapp.get_locale 
* Log each call to get_locale: display the URL, the locale and the source (browser, preferences, form).
* Rename _get_browser_language to _get_browser_or_settings_language to match the actual code.
 2020-09-22 11:49:15 +02:00
Alexandre Flament 2fd582d457 [fix] /autocompleter: return HTTP 400 when q is empty as intent. 
Fix commit c225db45c8
 2020-09-22 11:38:37 +02:00
Alexandre Flament e0142d380e Merge pull request #2211 from return42/fix-centos-7 
Bugfix of:  ERROR: centos-: uWSGI not yet implemented
 2020-09-21 09:06:20 +02:00
Alexandre Flament 4cc8644ec9 Merge pull request #1865 from dalf/uwsgi_static 
[enh] Docker image: uwsgi serves the static files directly.
 2020-09-19 16:04:07 +02:00
Alexandre Flament 375cbd12b3 Merge pull request #2205 from dalf/fix-simple-theme-autocomplete 
[fix] simple theme: fix autocomplete
 2020-09-19 16:02:01 +02:00
Alexandre Flament 530fc4bda7 Merge pull request #2190 from dalf/fix-htmltextextractor 
[fix] searx.utils.HTMLTextExtractor: invalid HTML don't raise an Exception
 2020-09-19 15:59:03 +02:00
Alexandre Flament c23401e9fa [fix] simple theme: fix autocomplete 
AJAX requests send the X-Requested-With HTTP header,
so searx.webapp.autocompleter returns the results with the expected data format.

Related to #2127
Close #2203
 2020-09-19 10:13:05 +02:00
Alexandre Flament 6deb85072a [fix] searx.utils.HTMLTextExtractor: invalid HTML don't raise an Exception 
Close #2188
 2020-09-13 10:28:11 +02:00
Alexandre Flament ae07f4a211 Merge pull request #2185 from dalf/rawtextquery-auto-parse 
[mod] searx.RawTextQuery: the constructor call parse_query
 2020-09-12 15:33:03 +02:00
Alexandre Flament df12ed6e55 [mod] searx.RawTextQuery: the constructor call parse_query 2020-09-12 15:25:58 +02:00
Alexandre Flament bdac99d4f0 Drop Python 2 (5/n): searx.utils.is_valid_lang, input parameter is a str instead of bytes 
Fix bug in translated.py and dictzone.py
 2020-09-10 10:49:42 +02:00
Dalf c225db45c8 Drop Python 2 (4/n): SearchQuery.query is a str instead of bytes 2020-09-10 10:49:42 +02:00
Dalf 7888377743 Drop Python 2 (3/n): objects 2020-09-10 10:39:04 +02:00
Dalf 78df10fb55 Drop Python 2 (2/n): templates 2020-09-10 10:39:04 +02:00
Dalf 1022228d95 Drop Python 2 (1/n): remove unicode string and url_utils 2020-09-10 10:39:04 +02:00
Alexandre Flament 272158944b [fix] make docker produces clean tag version (0.17.0 instead of 0.17.0-- for example) 
Fix #2152
 2020-09-10 10:09:15 +02:00
Alexandre Flament 8ffede6140 [fix] docker and k8s: run searx 
fix #2181
 2020-09-10 10:07:49 +02:00
Alexandre Flament d0f9778c2a [mod] results.py: code clean up (#2140) 2020-09-08 10:09:11 +02:00
Alexandre Flament 3397382754 [enh] stop searx when an engine raise an SyntaxError exception (#2177) 
and some other exceptions:
* KeyboardInterrupt
* SystemExit
* RuntimeError
* SystemError
* ImportError: an engine with an unmet dependency will stop everything.
 2020-09-07 15:39:26 +02:00
Alexandre Flament b329058c1a Revert "[enh] test: load each engine to check for syntax errors" 
This reverts commit 4fb3ed2c63.
 2020-08-31 19:00:06 +02:00
Dalf 4fb3ed2c63 [enh] test: load each engine to check for syntax errors 2020-08-28 12:12:32 +02:00
Alexandre Flament ec5adad851 [fix] requirements-dev.txt: set fixed versions for all the dependencies (#2154) 2020-08-27 22:28:51 +02:00
Alexandre Flament 093dd42bb0 Merge pull request #2134 from unixfox/fix/piratebay-engine-date-indentation 
[fix] piratebay engine date and pep8 indentation
 2020-08-11 14:56:19 +02:00
Alexandre Flament f3bfc19a55 Merge pull request #2127 from msafadieh/master 
[fix] fix the browser autocomplete feature.
 2020-08-09 19:21:09 +02:00
Alexandre Flament c3054a10a6 Merge pull request #2121 from gangural/random-sha256-fix 
Fix search error with query 'random sha256'
 2020-08-08 10:45:48 +02:00
Alexandre Flament 6d18769ccf Merge pull request #2101 from mikeri/master 
Update torrentz2 URL from .eu to .is
 2020-07-26 19:33:49 +02:00
Dalf 4a35975b42 [enh] update documentation about docker 2020-07-22 14:58:04 +02:00
Alexandre Flament 61ff5a7aea Merge pull request #2077 from dalf/bangs-no-favicon 
[mod] bangs.json: remove the favicons
 2020-07-15 10:00:12 +02:00
Dalf b339b8b6d6 [mod] bangs.json: remove the favicons 
fix #2076
 2020-07-15 09:15:26 +02:00
Dalf 718715180d [mod] external_bang.py: always read bangs.json with a UTF-8 file. 2020-07-11 12:23:08 +02:00
Alexandre Flament 040adecabf Merge pull request #2026 from return42/fix-yh 
[fix] yahoo engine - changed content_xpath
 2020-06-25 22:34:52 +02:00
Dalf 4c7b787004 [mod] don't try to proxify data URL. 
Previously only image/jpeg was not proxied.
This commit don't proxify all MIME types starting with "image/".

This is a quick fix for the PR #1985 : the google_image engine can returns some data URL.
 2020-06-22 13:57:33 +02:00
Alexandre Flament 2d4d0bdc0c Merge pull request #1947 from jjasonkal/master 
Remove cache of Dockerfile
 2020-05-09 19:25:40 +02:00
Alexandre Flament dc8d975051 Merge branch 'master' into uwsgi_static 2020-04-30 11:50:19 +02:00
Alexandre Flament fbf63f72c7 Merge branch 'master' into requirements-upgrade 2020-04-13 10:34:50 +02:00
Dalf 3973951770 [mod] requirements.txt: upgrade dependency versions. 2020-04-08 15:58:26 +02:00
Dalf a7862ba3c1 [mod] .travis.yml: add Python 3.7 and Python 3.8 2020-04-08 15:42:02 +02:00
Dalf 4dad366146 [fix] travis: fix docker tag. 
the "-dirty" ignores the searx/brand.py and utils/brand.env files.
 2020-04-02 15:06:10 +02:00
Alexandre Flament 2a1db2ac4a Merge branch 'master' into docker-build-fix 2020-03-31 20:14:40 +02:00
Dalf 4bb35e7ef5 [fix] template: fix typo in about.html 2020-03-31 16:52:52 +02:00
Dalf d425e26e80 [fix] fix travis build 2020-03-31 11:59:54 +02:00
Dalf 678d41d75b [enh] Docker image: uwsgi serves the static files directly. 
When the image is built, the static files are compressed with gzip and brotli.
The expires header is set to one day (same as Flask)
There is not etag header (Flask does add an etag header)
 2020-02-25 17:25:11 +01:00
Dalf 88f9ac58f4 [mod] move public instance list to https://searx.space 2020-02-14 13:48:44 +01:00
Dalf 0579d93783 [fix] Disable HTTP keep alive 
Fix HTTP 429 response from filtron
See https://github.com/searx/searx-docker/issues/24
 2020-02-07 19:16:15 +01:00
Dalf 0e7b6c9a03 [mod] infinite scroll: XHR requests are sent with the header "Accept: text/html". 
Fix https://github.com/searx/searx-docker/issues/16
 2019-12-11 14:34:25 +01:00
Dalf f34ac58752 [mod] tidy oscar HTML output 2019-12-10 10:58:30 +01:00
Dalf 85b3723345 [mod] speed optimization 
compile XPath only once
avoid redundant call to urlparse
get_locale(webapp.py): avoid useless call to request.accept_languages.best_match
 2019-11-15 09:33:15 +01:00
Alexandre Flament 8e62f7600f Merge pull request #1647 from dalf/no-inline-script 
No inline script for oscar and simple theme
 2019-08-06 12:25:25 +02:00
Dalf 6fbba63c83 oscar and simple themes: no inline script (allow A+ Content-Security-Policy) 2019-08-06 12:14:11 +02:00
Dalf 90b24f1060 manage.sh grunt_build: add node_modules/.bin to the path 2019-08-06 11:49:46 +02:00
Alexandre Flament 12f891da84 Merge pull request #1669 from dalf/engine-fixes 
Engine fixes
 2019-08-05 15:57:33 +02:00
Dalf 1cee2c1796 [fix] bing engine 
before this commit, sometimes there are no results
use a generic user-agent instead of one with the OS "Windows NT 6.3; WOW64"
 2019-08-05 15:46:40 +02:00
Dalf 9a378ca8c6 [fix] etymonline, voat engines 2019-08-05 15:44:22 +02:00
Dalf fcc9587ee9 [fix] fdroid engine 2019-08-05 15:44:02 +02:00
Dalf fbf6b689dd [fix] dictzone engine 2019-08-05 15:43:21 +02:00
Dalf 9ff5001816 [fix] arxiv engine 2019-08-05 15:43:01 +02:00
Alexandre Flament 91a59ff350 [fix] Docker: without -f parameter, settings.yml is not replaced by the new version (#1668) 
Before this commit, the existing settings.yml were always replaced.
 2019-08-03 13:53:14 +02:00
Alexandre Flament 333e54943d [fix] fix monkey patch in test_webapp.py (#1667) 
at the end of test_webapp.py, the monkey patch of searx.search.Search was not revert which lead to side effects on other tests
close #1663
 2019-08-03 13:23:36 +02:00
Alexandre Flament d24e7948eb Merge pull request #1665 from dalf/fix-paging 
[fix] fix paging for the oscar theme after PR #1640
Close #1664
 2019-08-02 20:38:53 +02:00
Dalf ba80a0ba38 [fix] fix paging for the oscar theme after PR #1640 2019-08-02 20:04:37 +02:00
Alexandre Flament 72029d27de [enh] Add timeout limit per request (#1640) 
The new url parameter "timeout_limit" set timeout limit defined in second.
Example "timeout_limit=1.5" means the timeout limit is 1.5 seconds.

In addition, the query can start with <[number] to set the timeout limit.

For number between 0 and 99, the unit is the second :
Example: "<30 searx" means the timeout limit is 3 seconds

For number above 100, the unit is the millisecond:
Example: "<850 searx" means the timeout is 850 milliseconds.

In addition, there is a new optional setting: outgoing.max_request_timeout.
If not set, the user timeout can't go above searx configuration (as before: the max timeout of selected engine for a query).

If the value is set, the user can set a timeout between 0 and max_request_timeout using
<[number] or timeout_limit query parameter.

Related to #1077
Updated version of PR #1413 from @isj-privacore
 2019-08-02 13:50:51 +02:00
Alexandre Flament 2179079a91 [fix] fix flickr_noapi decoding (#1655) 
Characters that were not ASCII were incorrectly decoded.
Add an helper function: searx.utils.ecma_unescape (Python implementation of unescape Javascript function).
 2019-08-02 13:37:13 +02:00
Alexandre Flament 02bbbf59d6 [fix] Dockerfile: searx version stores in org.label-schema.version (#1659) 
instead of org.label-schema.schema-version
 2019-08-01 07:51:09 +02:00
Alexandre Flament 1bed39e6cb Merge pull request #1658 from dalf/video-fixes 
Fix dailymotion, google_videos and youtube_noapi engines
 2019-08-01 07:44:30 +02:00
Dalf 0c032c8429 [fix] youtube_noapi engine: fix the title 2019-07-31 08:39:40 +02:00
Dalf 8b7ac56669 [fix] google_videos engine: some results don't a thumbnail 2019-07-31 08:38:45 +02:00
Dalf d44677e226 [fix] dailymotion engine: remove HTML tags from the description 2019-07-31 08:37:51 +02:00
Alexandre Flament 7f56c78876 Merge pull request #1650 from dalf/wikidata 
[fix] wikidata engine: faster processing, remove one HTTP redirection.
 2019-07-29 08:28:50 +02:00
Dalf 6e0285b2db [fix] wikidata engine: faster processing, remove one HTTP redirection. 
* Search URL is https://www.wikidata.org/w/index.php?{query}&ns0=1 (with ns0=1 at the end to avoid an HTTP redirection)
* url_detail: remove the disabletidy=1 deprecated parameter
* Add eval_xpath function: compile once for all xpath.
* Add get_id_cache: retrieve all HTML with an id, avoid the slow to procress dynamic xpath '//div[@id="{propertyid}"]'.replace('{propertyid}')
* Create an etree.HTMLParser() instead of using the global one (see #1575)
 2019-07-29 07:39:39 +02:00
Alexandre Flament c6ac39bcea Merge pull request #1654 from MarcAbonce/small_fixes 
[fix] Small fixes in Preferences view's text
 2019-07-28 10:31:57 +02:00
Alexandre Flament 4b56a4bfaa Merge pull request #1653 from dalf/bug-fixes 
Various bug fixes
 2019-07-27 09:11:03 +02:00
Dalf 70a01e5f02 [fix] don't merge image results if the img_src is different 2019-07-27 08:52:30 +02:00
Dalf 23611897ec [fix] make sure then engine name is lower case 
Minor fix: "%s engine initialized" display the right engine name
 2019-07-27 08:52:30 +02:00
Dalf 8645ef717e [fix] settings.yml: make sure all engine names are lower case 2019-07-27 08:52:30 +02:00
Dalf 7d8000d965 [fix] "!wp !wp test" send only one request to wikipedia. 
Note that "!general !wp test" is not fixed because the category for "!wp" is "none".
 2019-07-27 08:52:30 +02:00
Dalf 2fab23ab9a [fix] fix the debug message "starting webserver on ip:port" 
was "port:ip"
 2019-07-27 08:52:30 +02:00
Alexandre Flament f34b5cedb1 [fix] fixes google play engines (#1651) 
update commit 87baa74a86
 2019-07-25 09:31:47 +02:00
Alexandre Flament d0dd296424 Merge pull request #1642 from dalf/start-speed 
Speed up start time and decrease CPU load
 2019-07-23 13:18:29 +02:00