retoor/searxng
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,729 Commits 6 Branches 17 Tags
b167ad67a09099ce4e2a42e76ba2b836d8e9bbb8
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Alexandre Flament b167ad67a0 update AUTHORS.rst with new contributors 2020-12-14 18:01:42 +01:00
Alexandre Flament 292b73a3fc Merge pull request #2385 from joshu9h/patch-1 
[Fix] Startpage
 2020-12-14 17:56:48 +01:00
Alexandre Flament 36600118fb Merge pull request #2372 from dalf/remove-broken-engines 
[remove] remove searchcode_doc and twitter
 2020-12-13 17:11:05 +01:00
Alexandre Flament 3c4a9c1188 Merge pull request #2358 from dalf/fix-command 
[fix] command engine: SearchQuery.query is str not bytes
 2020-12-11 14:53:24 +01:00
Alexandre Flament 9f8ebeca44 Merge pull request #2375 from dalf/raise_for_errors 
[enh] add raise_for_httperror
 2020-12-11 14:52:18 +01:00
Alexandre Flament d703119d3a [enh] add raise_for_httperror 
check HTTP response:
* detect some comme CAPTCHA challenge (no solving). In this case the engine is suspended for long a time.
* otherwise raise HTTPError as before

the check is done in poolrequests.py (was before in search.py).

update qwant, wikipedia, wikidata to use raise_for_httperror instead of raise_for_status
 2020-12-11 14:37:08 +01:00
Alexandre Flament 033f39bff7 Merge pull request #2376 from dalf/fix-mojeek 
Fix mojeek
 2020-12-11 13:14:54 +01:00
Alexandre Flament 6bc6d5e9fd Merge pull request #2371 from dalf/mod-genius 
[mod) genious: return valid results even if contents are empty
 2020-12-11 13:14:03 +01:00
Alexandre Flament 0ba74cd812 [mod] results: don't crash when an engine don't have a category 
According to
https://github.com/searx/searx/blob/820b468bfe96f693d60ce06f1e78af51f00deefc/searx/engines/__init__.py#L87-L88

an engine can have no category at all.

Without this commit, searx raise an exception in searx/results.py

Note: in this case, the engine is not shown in the preferences.
 2020-12-10 10:57:07 +01:00
Alexandre Flament d41cafd5f3 [fix] xpath, mojeek: fix commit 58d72f2692 
before commit 58d72f2, category was not set in xpath.py,
so searx/engines/__init__py was setting the category to ['general']

the commit 58d72f2 set the category to [] which is not replaced by searx/engines/__init__.py
consequence: the mojeek engine is hidden in the preferences.

this commit revert the xpath.py change.

close #2368
 2020-12-10 10:52:06 +01:00
Alexandre Flament 1c9e7cef50 [remove] remove searchcode_doc and twitter 
* twitter: the API has changed. the engine needs to rewritten.
* searchcode_doc: the API about documentation doesn't exist anymore.
 2020-12-09 13:14:31 +01:00
Alexandre Flament fa73f10f11 [mod) genious: return valid results even if contents are empty 2020-12-09 13:01:34 +01:00
Alexandre Flament 42a194898b Merge pull request #2360 from dalf/update-libgen 
[mod] libgen: update the URL to http://libgen.rs/
 2020-12-08 20:33:53 +01:00
Alexandre Flament a77d8c8227 Merge pull request #2359 from dalf/update-duden 
[mod] duden engine
 2020-12-08 20:33:38 +01:00
Alexandre Flament bd4869ecd0 Merge pull request #2366 from dalf/remove-seedpeer 
[remove] seedpeer engine
 2020-12-08 20:33:23 +01:00
Alexandre Flament 35bf568b41 Merge pull request #2362 from return42/use_default_settings 
[doc] recommend to use 'use_default_settings=True'
 2020-12-08 20:33:06 +01:00
Alexandre Flament 56c64d6b64 [remove] seedpeer engine 
the website is offline.
 2020-12-07 21:02:29 +01:00
Alexandre Flament c1a9732268 Merge pull request #2364 from dalf/fix-youtube-noapi 
[fix] youtube_noapi engine
 2020-12-07 20:26:00 +01:00
Alexandre Flament 13d3004703 Merge pull request #2365 from dalf/fix-soundcloud 
[fix] soundclound: accept result without content
 2020-12-07 20:25:17 +01:00
Alexandre Flament 62073c0e1d Merge pull request #2361 from dalf/fix-1x 
[fix] 1x engine
 2020-12-07 20:24:47 +01:00
Alexandre Flament 923bc02c17 Merge pull request #2363 from dalf/fix-wikipedia-minor 
[fix] wikipedia: minor fix: return no result instead of crash in some very few cases.
 2020-12-07 18:33:37 +01:00
Alexandre Flament deb1bde20d [fix] soundclound: accept result without content 2020-12-07 17:45:36 +01:00
Alexandre Flament 34df0f7910 [fix] youtube_noapi engine 2020-12-07 17:44:31 +01:00
Alexandre Flament 58d51e082d [fix] wikipedia: minor fix: return no result instead of crash in some very few cases. 
In few cases, the JSON results doesn't contains the key 'type'.
 2020-12-07 17:42:05 +01:00
Alexandre Flament 4ec810749b [fix] 1x engine 2020-12-07 15:46:00 +01:00
Alexandre Flament 1e781863fa [fix] command engine: SearchQuery.query is str not bytes 
see c225db45c8
 2020-12-07 10:43:42 +01:00
Alexandre Flament 9bf594cbcf [mod] duden engine 
* add params['soft_max_redirects'] = 1  (when there is spelling suggestion)
* avoid try..except
* use eval_xpath_* functions
 2020-12-07 10:31:11 +01:00
Alexandre Flament b3032cf953 [mod] libgen: update the URL to http://libgen.rs/ 
https://libgen.is actually redirect to http://libgen.rs/

It seems there is no HTTPS version:
* https://www.wikidata.org/wiki/Q22017206
* https://librarygenesis.net/
 2020-12-07 10:19:34 +01:00
Alexandre Flament a458451d20 Merge pull request #2356 from dalf/fix-ddd 
[fix] duckduckgo_definitions: fix relative image URL
 2020-12-07 10:16:53 +01:00
Alexandre Flament 925bb561a2 Merge pull request #2352 from dalf/no_http 
Remove HTTP connections as much as possible
 2020-12-06 10:18:49 +01:00
Alexandre Flament 28cc644f0a [fix] duckduckgo_definitions: fix relative image URL 
ddg returns relative URL to https://duckduckgo.com/
 2020-12-06 10:14:09 +01:00
Alexandre Flament cdceec1cbb Merge pull request #2354 from dalf/fix-wikipedia 
[fix] wikipedia engine: don't raise an error when the query is not found
 2020-12-04 20:42:45 +01:00
Alexandre Flament f0054d67f1 [fix] wikipedia engine: don't raise an error when the query is not found 
Add a new parameter "raise_for_status", set by default to True.
When True, any HTTP status code >= 300 raise an exception ( #2332 )
When False, the engine can manage the HTTP status code by itself.
 2020-12-04 20:04:39 +01:00
Alexandre Flament d9494283cc Merge pull request #2353 from dalf/fix-wikidata 
[fix] wikidata: fix crash when the item has no description at all and…
 2020-12-04 18:07:24 +01:00
Alexandre Flament bef2f2efa8 [fix] wikidata: fix crash when the item has no description at all and at least one URL. 2020-12-04 17:17:20 +01:00
Alexandre Flament 38d32337e7 [mod] documentations & comments: update http://* URL to https://*. 
About http://lesscss.org see https://github.com/less/less-docs/issues/520
 2020-12-04 16:52:25 +01:00
Alexandre Flament 244e812f37 [fix] remove searx/engines/filecrop.py (dead code) 2020-12-04 16:48:15 +01:00
Alexandre Flament 0226ae69d3 [fix] dbpedia autocomplete (and use HTTPS) 2020-12-04 16:47:43 +01:00
Alexandre Flament f56e78ee80 Merge pull request #2338 from unixfox/issue-template 
add new issue template
 2020-12-03 19:16:11 +01:00
Alexandre Flament cec73b5dcf Merge pull request #2350 from dalf/mod-engines-report-captcha 
[mod] stackoverflow & yandex: detect CAPTCHA response
 2020-12-03 13:50:12 +01:00
Alexandre Flament fa909c7c02 [mod] stackoverflow & yandex: detect CAPTCHA response 2020-12-03 13:23:19 +01:00
Alexandre Flament 7905d41487 Merge pull request #2349 from dalf/fix-settings-loader 
[fix] settings_loader: don't crash when a key exists only in the user settings
 2020-12-03 11:47:53 +01:00
Alexandre Flament d0d7a3e1c2 [fix] settings_loader: don't crash when a key exists only in the user settings 
typical use case: result_proxy can be defined in the user settings,
but are not defined the default settings.yml
 2020-12-03 11:35:12 +01:00
Alexandre Flament 89fbb85d45 Merge pull request #2332 from dalf/metrology-errors 
[enh] record exception details per engine
 2020-12-03 10:31:44 +01:00
Alexandre Flament 64cccae99e [mod] various engines: use eval_xpath* functions and searx.exceptions.* 
Engine list: ahmia, duckduckgo_images, elasticsearch, google, google_images, google_videos, youtube_api
 2020-12-03 10:22:48 +01:00
Alexandre Flament ad72803ed9 [mod] xpath, 1337x, acgsou, apkmirror, archlinux, arxiv: use eval_xpath_* functions 2020-12-03 10:22:48 +01:00
Alexandre Flament de887c6347 [mod] bing_news: use eval_xpath_getindex 
remove unused function searx.utils.list_get
 2020-12-03 10:22:48 +01:00
Alexandre Flament 1d0c368746 [enh] record details exception per engine 
add an new API /stats/errors
 2020-12-03 10:22:48 +01:00
Alexandre Flament 6b5a578822 Merge pull request #2285 from return42/fix-digg 
bugfix & refactor digg engine
 2020-12-03 10:20:40 +01:00
Alexandre Flament 219af243a2 Merge pull request #2318 from dalf/add-pylint 
Add pylint
 2020-12-01 15:31:10 +01:00
Alexandre Flament 8aa2a7556a [enh] activate pylint 
There are 3 invocations:
* one with the default .pylintrc file
* one for searx/engines with some disabled checks and some additional-builtins
* one for the all the code except the engines with some disabled checks
 2020-12-01 15:21:19 +01:00
Alexandre Flament b00d108673 [mod] pylint: numerous minor code fixes 2020-12-01 15:21:19 +01:00
Alexandre Flament 9ed3ee2beb [mod] wikidata: WDGeoAttribute class: doesn't change the method signature of get_str 2020-12-01 15:21:17 +01:00
Alexandre Flament 3cfef61123 [fix] /stats: report error percentage instead of error count 
This bug exists since the PR https://github.com/searx/searx/pull/751
 2020-12-01 15:07:09 +01:00
Alexandre Flament b7f1a8424a [fix] tests: fix duplicate method name 2020-12-01 15:07:09 +01:00
Alexandre Flament a302ac5629 [mod] remove dead code 2020-12-01 15:07:09 +01:00
Alexandre Flament a1e6bc4cee Merge pull request #2291 from dalf/settings2 
[enh] user settings can relied on the default settings
 2020-12-01 14:57:12 +01:00
Alexandre Flament 0e02fee930 Merge pull request #2345 from dalf/mod-oscar 
[mod] oscar theme: added option into gruntfile.js for generate sourceMap
 2020-12-01 14:11:13 +01:00
Alexandre Flament f1e016e9ea [mod] oscar theme: added option into gruntfile.js for generate sourceMap 
Credits go to @mrwormo  (see PR #2308 )
 2020-12-01 10:07:01 +01:00
Alexandre Flament 93c2603561 Merge pull request #2336 from efectn/patch-1 
Photon API Link Update
 2020-11-28 09:06:53 +01:00
Alexandre Flament b4b81a5e1a [enh] settings.yml: add use_default_settings option (2nd version) 2020-11-27 19:40:04 +01:00
Alexandre Flament 1cfe7f2a75 [enh] settings.yml: add use_default_settings option 
This change is backward compatible with the existing configurations.

If a settings.yml loaded from an user defined location (SEARX_SETTINGS_PATH or /etc/searx/settings.yml),
then this settings can relied on the default settings.yml with this option:
user_default_settings:True
 2020-11-26 18:27:27 +01:00
Alexandre Flament 6ada5bac60 Merge pull request #2327 from renyhp/master 
Add preference for displaying advanced settings
 2020-11-26 17:37:43 +01:00
Alexandre Flament aec6a2656c Merge pull request #2319 from dalf/multiple-proxies 
[enh] Add multiple outgoing proxies
 2020-11-23 08:43:53 +01:00
Alexandre Flament 3786920df9 [enh] Add multiple outgoing proxies 
credits go to @bauruine see https://github.com/searx/searx/pull/1958
 2020-11-20 15:29:21 +01:00
Alexandre Flament 2fc3b17c85 Merge pull request #2323 from kvch/fix-unresponsive-engine-set-modified 
Fix RuntimeError during search with translated errors
 2020-11-18 09:37:29 +01:00
Alexandre Flament 963b93b543 [fix] fix github workflow: deploy documentation 2020-11-17 15:54:14 +01:00
Alexandre Flament c9fb3ac34f [fix] fix github workflow 2020-11-17 15:44:50 +01:00
Alexandre Flament 197a013d2e [fix] fix github workflow 
bug fix of 9c25cd99c4
 2020-11-17 15:38:31 +01:00
Alexandre Flament 2fc7273141 Merge pull request #2322 from dalf/github-actions 
[mod] use github actions instead of travis
 2020-11-17 15:20:45 +01:00
Alexandre Flament 9c25cd99c4 [mod] use github actions instead of travis 
fix https://github.com/searx/searx/issues/2279
 2020-11-17 15:09:06 +01:00
Alexandre Flament a30d9f2655 Merge pull request #2321 from dalf/fix-standalone 
[fix] fix a test_standalone_searx test case
 2020-11-17 13:19:55 +01:00
Alexandre Flament 230a5ecd04 [fix] fix a test_standalone_searx test case 
If test_engines_init.py runs before test_standalone_searx.py, the engine list is not empty.
It makes test_get_search_query flaky.

This commit initializes the engline list in test_standalone_searx.py
 2020-11-17 10:39:44 +01:00
Alexandre Flament 17b48ff6e8 Merge pull request #2286 from return42/fix-deviantart 
[refactor] deviantart - improve results and clean up source code
 2020-11-14 17:31:39 +01:00
Alexandre Flament 4f18faebe1 Merge pull request #2288 from dalf/lint 
[mod] remove unused import
 2020-11-14 14:24:45 +01:00
Alexandre Flament 3038052c79 [mod] remove unused import 
use
from searx.engines.duckduckgo import _fetch_supported_languages, supported_languages_url  # NOQA
so it is possible to easily remove all unused import using autoflake:
autoflake --in-place --recursive --remove-all-unused-imports searx tests
 2020-11-14 14:11:02 +01:00
Alexandre Flament 6489a560ea Merge pull request #2299 from rachmadaniHaryono/feature/pep8-to-pycodestyle 
pep8 to pycodestyle
 2020-11-14 13:49:25 +01:00
Alexandre Flament e1bd617669 Merge pull request #2296 from rachmadaniHaryono/feature/suppress-output 
suppress test output
 2020-11-14 13:45:32 +01:00
Alexandre Flament c3d9b17c2a Merge pull request #2292 from kvch/elasticsearch-engine 
New engine: Elasticsearch
 2020-11-14 13:25:08 +01:00
Alexandre Flament 102c08838b Merge pull request #2289 from dalf/pylint 
[mod] pylint: add extension-pkg-whitelist=lxml.etree
 2020-11-14 13:24:31 +01:00
Alexandre Flament 46b454277f Merge pull request #2309 from dalf/mod_search_repr 
[mod] searx.search: EngineRef, SearchQuery: add __repr__ and __eq__ methods
 2020-11-14 13:23:44 +01:00
Alexandre Flament ebed1461bc Merge pull request #2300 from dalf/fix-webapp-index 
[fix] fix of / and /search
 2020-11-14 13:23:03 +01:00
Alexandre Flament 20c95712ea Merge pull request #2301 from pierrechtux/fix_typo_virualenv 
Fix typo virualenv
 2020-11-13 17:55:54 +01:00
Alexandre Flament 8fc74d0d7b [mod] searx.search: EngineRef, SearchQuery: add __repr__ and __eq__ methods 2020-11-10 10:45:40 +01:00
Alexandre Flament b3a3ccf2db [fix] fix of / and /search 
* URL / : the index page displayed the selected or the default category.
* URL / : when the q parameter is set using the URL, the redirect includes the URL query.
* URL /search : an empty query doesn't raise an exception.
 2020-11-06 12:11:52 +01:00
Alexandre Flament 58d72f2692 [mod] pylint: minor code change to allow pylint globally 
This commit is only a step, it doesn't fix all the issues reported by pylint
 2020-11-03 11:35:53 +01:00
Alexandre Flament e28b56e262 [fix] webadapter: fix locked categories 2020-11-03 10:55:49 +01:00
Alexandre Flament eed43783f9 [fix] comamnd engine: fix import 2020-11-03 10:55:08 +01:00
Alexandre Flament a08df82574 [fix] scanr_structure engine: fix import 2020-11-03 10:54:02 +01:00
Alexandre Flament d7f21484ee [mod] pylint: add extension-pkg-whitelist=lxml.etree 2020-11-02 15:55:19 +01:00
Alexandre Flament 54afe37e18 [mod] .travis.yml: build documentation automatically on the master branch 2020-10-28 22:48:16 +01:00
Alexandre Flament 95bd6033fa [mod] wikidata engine: use one SPARQL request instead of 2 HTTP requests. 2020-10-28 08:09:25 +01:00
Alexandre Flament ca593728af [mod] duckduckgo_definitions: display only user friendly attributes / URL 
various bug fixes
 2020-10-28 08:09:25 +01:00
Alexandre Flament 382fded665 [mod] result.py: merge infobox URL and attributes when the same label or the same entity 
entity are wikidata entity (like "Q42" for "Douglas Adams", see https://www.wikidata.org/wiki/Q42 )
 2020-10-28 08:09:25 +01:00
Alexandre Flament 23f4203dfb [fix] simple theme: infobox: remove useless entity information 2020-10-28 08:09:25 +01:00
Alexandre Flament ed6696e6bf [mod] add external_urls.json and wikidata_units.json 2020-10-28 08:09:25 +01:00
Alexandre Flament 5e7060053c [mod] ahmia_filter.py: minor changes 
- use result['parsed_url']
- load ahmia_blacklist.txt in searx.datae
 2020-10-27 20:00:04 +01:00
Alexandre Flament cfd21bc475 [fix] fix duckduckgo engine 
- remove paging support: a "vqd" parameter is required between each request. This parameter is uniq for each request
- update the URL (no redirect), use the POST method
- language support: works if there is no more than request per minute, otherwise it is ignored !
 2020-10-09 16:00:42 +02:00
Alexandre Flament 8b278cbfad Merge pull request #2246 from dalf/mod-searx-data 
[mod] Add searx.data module
 2020-10-07 10:38:13 +02:00
Alexandre Flament a9dc54bebc [mod] Add searx.data module 
Instead of loading the data/*.json in different location,
load these files in the new searx.data module.
 2020-10-07 10:29:34 +02:00