Commit Graph
100 Commits
Author SHA1 Message Date
Alexandre Flament 30c950a2c7 Merge pull request #2660 from dalf/upd-translations
[mod] replace /translations.js with an embedded JSON
2021-03-21 12:39:26 +01:00
Alexandre Flament 12fa524cf7 Merge pull request #2667 from searx/dependabot/pip/master/twine-3.4.1
Bump twine from 3.3.0 to 3.4.1
2021-03-20 19:50:27 +01:00
Alexandre Flament 7a05d5c9cd Merge pull request #2669 from jhigginbotham/master
Update settings.yml to enable HTTP for yacy engine
2021-03-20 18:37:24 +01:00
Alexandre Flament 7e5553ddb6 Merge pull request #2662 from dalf/oscar-trim-space
[mod] oscar: remove space
2021-03-18 09:50:52 +01:00
Alexandre Flament 2b0dd96bd3 [mod] oscar: remove space
* reduce by 15% the uncompressed output (on average)
* dos2unix searx/templates/oscar/result_templates/files.html
2021-03-17 09:22:05 +01:00
Alexandre Flament 6553c79029 [mod] replace /translations.js by embedded JSON
In webapp.py, there is a new function "get_translations" lists available translations

Close #2064
2021-03-16 11:22:21 +01:00
Alexandre Flament cb3b379161 Merge pull request #2206 from dalf/upgrade-pygments
[mod] upgrade pygments
2021-03-16 11:06:25 +01:00
Alexandre Flament 32cd0d31b3 [mod] upgrade pygments
add searx_extra/update/update_pygments.py to update the css style of the oscar and simple themes.
2021-03-16 09:07:08 +01:00
Alexandre Flament 1664258061 Merge pull request #2655 from return42/fix-imports
[fix] remove unused import from yahoo-news engine
2021-03-15 08:38:34 +01:00
Alexandre Flament 5b176b3496 Merge pull request #2659 from MarcAbonce/onions-http-fix
Fix HTTP error in onion engines
2021-03-15 08:33:38 +01:00
Alexandre Flament 0b71d61dfc Merge pull request #2654 from return42/acgsou
[drop] Acgsou engine - www.acgsou.com no longer exists
2021-03-15 08:27:15 +01:00
Alexandre Flament 9292571304 Merge pull request #2346 from dalf/upgrade-oscar
[mod] oscar: upgrade dependencies
2021-03-13 09:29:13 +01:00
Alexandre Flament 92dd5e245e Merge pull request #2626 from mikeri/solidtorrents
Add Solid Torrents engine
2021-03-12 19:45:22 +01:00
Alexandre Flament a1a492baed Merge pull request #2641 from dalf/disable_http_by_default
[mod] by default allow only HTTPS, not HTTP
2021-03-12 19:21:46 +01:00
Alexandre Flament af3e969c5a Merge pull request #2642 from return42/fix-apkmirror
[fix] APKMirror engine - update xpath selectors and fix img_src
2021-03-11 09:48:31 +01:00
Alexandre Flament 8b650e6a2d Merge pull request #2643 from return42/fix-makefile
[fix] make targets engines.languages and useragents.update
2021-03-11 09:44:30 +01:00
Alexandre Flament cb04d42806 [mod] oscar: update README.rst 2021-03-11 09:33:04 +01:00
Alexandre Flament 86912e2272 [mod] oscar: get bootstrap and typeahead from NPM 2021-03-11 09:33:01 +01:00
Alexandre Flament 44407353ef [mod] oscar: get leaflet and jquery from NPM
easy to upgrade (package.json)
2021-03-11 09:32:22 +01:00
Alexandre Flament c7133efb12 [mod] oscar: move compiled files to the src directory 2021-03-10 19:28:51 +01:00
Alexandre Flament eda3b513ac [mod] oscar: remove polyfills for Internet Explorer 2021-03-10 19:01:16 +01:00
Alexandre Flament 1268910274 [mod] oscar: remove unused images 2021-03-10 19:01:16 +01:00
Alexandre Flament bdb41bea7b [mod] theme: remove require-2.1.15.min.js
See https://github.com/requirejs/requirejs/issues/1816

requirejs loads one file: leaflet.

This commit:
* removes requirejs
* load leaflet using <script src...> HTML tag in searx/templates/oscar/base.html
2021-03-10 19:01:15 +01:00
Alexandre Flament 2f3d5ec2af [mod] oscar: upgrade npm dependencies 2021-03-10 19:01:14 +01:00
Alexandre Flament ccf5ac9801 Merge pull request #2640 from return42/fix-yahoo-news
[fix] rewrite Yahoo-News engine
2021-03-08 19:03:41 +01:00
Alexandre Flament 99e0651cea [mod] by default allow only HTTPS, not HTTP
Related to https://github.com/searx/searx/pull/2373
2021-03-08 11:35:08 +01:00
Alexandre Flament 87f4cc4a9e Merge pull request #2631 from searx/update_data_update_languages.py
Update searx.data - update_languages.py
2021-03-06 10:03:00 +01:00
Alexandre Flament 63e696372b Merge pull request #2634 from return42/fedora
[mod] LXC switch to Fedora 33 / Fedora 31 reached its EOL
2021-03-05 21:56:18 +01:00
Alexandre Flament 70c38a93d0 Merge pull request #2635 from return42/einit
[mod] don't dump traceback of SearxEngineResponseException on init
2021-03-05 21:55:00 +01:00
Alexandre Flament 0165e14a7f Merge pull request #2632 from searx/update_data_update_wikidata_units.py
Update searx.data - update_wikidata_units.py
2021-03-05 11:59:44 +01:00
Alexandre Flament 152f6fc1da Merge pull request #2630 from searx/update_data_update_ahmia_blacklist.py
Update searx.data - update_ahmia_blacklist.py
2021-03-05 11:59:20 +01:00
Alexandre Flament d7fb55a774 Merge pull request #2633 from searx/update_data_update_currencies.py
Update searx.data - update_currencies.py
2021-03-05 11:59:05 +01:00
Alexandre Flament 10ecc303c9 Fix integration.yml
Don't run twice the workflow on PR opened from the same repository.
2021-03-05 11:54:45 +01:00
Alexandre Flament 245f8626dd Fix data-update.yml, run once a month 2021-03-05 11:12:31 +01:00
Alexandre Flament d9a35fc28e Fix data-update.yml 2021-03-05 10:24:54 +01:00
Alexandre Flament 917dff3aab Update data-update.yml
Allow to manually running the workflow

See:
* https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow
* https://docs.github.com/en/actions/reference/events-that-trigger-workflows#workflow_dispatch
2021-03-05 10:16:59 +01:00
Alexandre Flament b97273df6b Update data-update.yml
Remove cache
2021-03-05 09:49:39 +01:00
Alexandre Flament aaae9a209e Merge pull request #2600 from dalf/searx-extra
Add searx_extra package
2021-03-05 09:43:39 +01:00
Alexandre Flament b8cd326464 Add searx_extra package
Split the utils directory into:
* searx_extra contains update scripts, standalone_searx.py
* utils contains the files to build and setup searx.
2021-03-04 11:59:14 +01:00
Alexandre Flament 1d10ae175c Merge pull request #2618 from thezeroalpha/master
Fix security vulnerabilities in provided nginx configuration
2021-03-04 11:27:03 +01:00
Alexandre Flament 6ba37777f7 Merge pull request #2623 from return42/fix-centos7
[fix] add package `which` to CentOS-7 boilerplate
2021-03-04 11:19:05 +01:00
Alexandre Flament d26261c5ab Merge pull request #2619 from return42/drop-ubu1604
[mod] Drop Ubuntu 16.04 (Xenial Xerus) support
2021-03-04 11:18:21 +01:00
Alexandre Flament 4c2a8aea39 Merge pull request #2620 from return42/fix-git
[fix] support git versions <v2.22
2021-03-04 11:17:24 +01:00
Alexandre Flament aac37f288f Merge pull request #2593 from dalf/update-autocomplete
Update autocomplete
2021-03-04 10:51:09 +01:00
Alexandre Flament c748fc66cf Merge pull request #2610 from searx/dependabot/pip/master/pylint-2.7.2
Bump pylint from 2.6.2 to 2.7.2
2021-03-02 10:12:32 +01:00
Alexandre Flament 87ac6df9fc Merge pull request #2611 from return42/google-scolar
[enh] google scholar - python implementation of the engine
2021-03-02 10:05:49 +01:00
Alexandre Flament 63f17d2e4c [enh] autocomplete refactoring, autocomplete on external bangs 2021-03-01 19:12:32 +01:00
Alexandre Flament 4fa1290c11 [fix] answers: don't crash when the query is an empty string 2021-03-01 10:52:39 +01:00
Alexandre Flament e2fb500892 Merge pull request #2608 from return42/unittest2
[py2to3] use unittest from py3, remove unittest2 from py2
2021-03-01 10:05:38 +01:00
Alexandre Flament 0c663e25fc Merge pull request #2604 from searx/update_data_firefox_version
Update searx.data - firefox_version
2021-03-01 10:03:39 +01:00
Alexandre Flament f77983e174 Merge pull request #2602 from MarcAbonce/fix-bing-fetch-languages
Fix fetch_languages for Bing
2021-03-01 09:06:37 +01:00
Alexandre Flament b05f4d0664 Merge pull request #2605 from searx/update_data_currencies
Update searx.data - currencies
2021-02-26 12:56:32 +01:00
Alexandre Flament aec5188b51 Merge pull request #2606 from searx/update_data_wikidata_units
Update searx.data - wikidata_units
2021-02-26 12:55:51 +01:00
Alexandre Flament 28ed5acde5 Merge pull request #2607 from searx/update_data_ahmia_blacklist
Update searx.data - ahmia_blacklist
2021-02-26 12:55:26 +01:00
Alexandre Flament 728e096764 data-update.yml: on PR per file 2021-02-25 08:41:17 +01:00
Alexandre Flament 7c1847d5f2 [mod] add utils/fetch_external_bangs.py
Based on duckduckgo bangs
Store bangs on a trie to allow autocomplete (not in this commit)
2021-02-24 18:48:36 +01:00
Alexandre Flament 606aa79e49 Merge pull request #2584 from searx/dependabot/pip/master/sphinx-3.5.1
Bump sphinx from 3.4.1 to 3.5.1
2021-02-24 13:52:15 +01:00
Alexandre Flament 5f4a085fc4 Merge pull request #2595 from dalf/update-wikidata-units
[mod] update wikidata_units.json and fetch_wikidata_units.py
2021-02-23 17:22:37 +01:00
Alexandre Flament 94c9320c6f Merge pull request #2585 from dalf/update-currencies
[mod] update currencies.json and fetch_currencies.py
2021-02-23 17:22:05 +01:00
Alexandre Flament 46ca32c3cc [mod] update currencies.json and fetch_currencies.py
use a sparql request on wikidata to get the list of currencies.

currencies.json contains the translation for all supported searx languages.

Supersede #993
2021-02-23 16:42:28 +01:00
Alexandre Flament 93d1da4906 [mod] update wikidata_units.json and fetch_wikidata_units.py
The fetch_wikidata_units.py result won't change randomly.
See comments in the script.
2021-02-23 13:10:38 +01:00
Alexandre Flament 9ace31c829 Merge pull request #2588 from datagram1/master
Added rumble.com video search engine.
2021-02-21 14:08:02 +01:00
Alexandre Flament 640d6ce778 Update dependabot.yml 2021-02-19 13:08:57 +01:00
Alexandre Flament fd1b8f661b Update dependabot.yml 2021-02-16 18:42:44 +01:00
Alexandre Flament 3ee24b685a Create dependabot.yml
Only pip packages and ignore pygments, related to https://github.com/searx/searx/pull/2206
2021-02-16 18:41:34 +01:00
Alexandre Flament 44a6593c13 Merge pull request #2573 from unixfox/yggtorrent
update yggtorrent url + add it back
2021-02-16 08:22:07 +01:00
Alexandre Flament 09e28ae779 Merge pull request #2570 from xenrox/improve-peertube
Improve peertube searching
2021-02-16 08:17:42 +01:00
Alexandre Flament aef1161633 data-update.yml: run on Friday, 16:05 UTC 2021-02-12 17:19:44 +01:00
Alexandre Flament 45027765e3 Merge pull request #2566 from dalf/remove-yandex
[remove] yandex engine
2021-02-12 17:12:07 +01:00
Alexandre Flament 50bde93dbb data-update.yml: run on searx org 2021-02-12 16:56:00 +01:00
Alexandre Flament f59acf4138 data-update.yml: run on Friday
give the weekend to review the PR
2021-02-12 11:15:46 +01:00
Alexandre Flament ddab00abf4 Merge pull request #2559 from dalf/upd-duckduckgo
[fix] duckduckgo engine: "!ddg !g" do not redirect to google
2021-02-12 11:14:09 +01:00
Alexandre Flament c22d4c764c [fix] duckduckgo engine: "!ddg !g" do not redirect to google
* searx understand "!ddg !g time" as : send "!g time" to DDG
* !g a DDG bang for Google: DDG return a HTTP redirect to Google

This commit adds a the allows_redirect param not to follow HTTP redirect.

The DDG engine returns a empty result as before without HTTP redirect.
2021-02-12 11:10:08 +01:00
Alexandre Flament d76660463b Merge pull request #2562 from dalf/mod-json-engine
[mod] json_engine: add content_html_to_text and title_html_to_text
2021-02-12 10:58:28 +01:00
Alexandre Flament 7dcf67a47a Merge pull request #2565 from dalf/upd-wikipedia
[upd] wikipedia engine: return an empty result on query with illegal characters
2021-02-12 10:57:05 +01:00
Alexandre Flament 2b60d0d243 Merge pull request #2564 from dalf/fix-seznam
[fix] fix seznam engine
2021-02-12 10:56:53 +01:00
Alexandre Flament 7e83818879 Merge pull request #2560 from dalf/fix-duckduckgo
Fix duckduckgo
2021-02-12 10:56:40 +01:00
Alexandre Flament 63d6ccfbc2 Merge pull request #2557 from dalf/fix-raise_for_httperror
Fix: activate raise_for_error by default
2021-02-12 10:56:25 +01:00
Alexandre Flament 74c8b5606f Merge pull request #2541 from return42/mediathekviewweb
[enh] add engine MediathekViewWeb (API)
2021-02-11 15:11:26 +01:00
Alexandre Flament 5d9db6c2f7 [remove] yandex engine 2021-02-11 14:28:06 +01:00
Alexandre Flament 35dd069402 [fix] fix seznam engine
no paging support
2021-02-11 12:53:19 +01:00
Alexandre Flament 7d6e69e2f9 [upd] wikipedia engine: return an empty result on query with illegal characters
on some queries (like an IT error message), wikipedia returns an HTTP error 400.
this commit returns an empty result instead of showing an error to the user.
2021-02-11 12:29:21 +01:00
Alexandre Flament ff84a1af35 [mod] json_engine: add content_html_to_text and title_html_to_text
Some JSON API returns HTML in either in the HTML or the content.
This commit adds two new parameters to the json_engine:
content_html_to_text and title_html_to_text, False by default.

If True, then the searx.utils.html_to_text removes the HTML tags.

Update crossref, openairedatasets and openairepublications engines
2021-02-10 16:42:11 +01:00
Alexandre Flament 436d366448 Merge pull request #2544 from mrwormo/congresslibrary
[Engine] Add Library of Congress engine
2021-02-10 10:13:46 +01:00
Alexandre Flament eafd27f42a Merge pull request #2556 from dalf/fix-apk-mirror
[fix] fix apk_mirror engine
2021-02-10 10:12:37 +01:00
Alexandre Flament c40316d957 Merge pull request #2558 from dalf/remove-google-play-music
[upd] remove google_play_music engine
2021-02-10 10:12:21 +01:00
Alexandre Flament d2dac11392 [mod] duckduckgo engine: better support of the language preference
After the main request, send a second to https://duckduckgo.com/t/sl_h

See https://github.com/searx/searx/issues/2259
2021-02-09 14:36:43 +01:00
Alexandre Flament 74d56f6cfb [mod] poolrequests: for one (user request, engine) always use the same HTTPAdapter
The duckduckgo engine requires an additional request after the results have been sent.
This commit makes sure that the second request uses the same HTTPAdapter
= the same IP address, and the same proxy.
2021-02-09 14:33:36 +01:00
Alexandre Flament 9211cdfe9b [upd] remove google_play_music engine
Google Play Music has been replaced by Youtube music.
2021-02-09 11:38:50 +01:00
Alexandre Flament aedf03c0f7 Fix: activate raise_for_error by default
Fix commit d703119d3a :
Some engines need to parse the HTTP error but
raise_for_error is always set to False in the "request" function.
2021-02-09 11:27:41 +01:00
Alexandre Flament 5e055b069b [fix) fix apk_mirror engine 2021-02-09 11:02:12 +01:00
Alexandre Flament f03ad0a3c0 Merge pull request #2555 from dalf/fix-github-data-update
[fix] fix github action data-update.yml
2021-02-09 10:48:55 +01:00
Alexandre Flament 966a7a1f25 [fix] fix github action data-update.yml 2021-02-09 09:58:59 +01:00
Alexandre Flament e4cc7f13a3 Merge pull request #2542 from kvch/fix-naver-engine
Fix XPATHs in Naver engine
2021-02-09 08:52:38 +01:00
Alexandre Flament bec9e30fe7 Merge pull request #2554 from MarcAbonce/zh-variants-in-wikipedia
Add support for Chinese variants in Wikipedia
2021-02-09 08:49:59 +01:00
Alexandre Flament 6c513095e4 Merge pull request #2553 from danielhones/improve-results-highlighting-updated
Ignore double-quotes when highlighting query parts
2021-02-09 08:39:07 +01:00
Alexandre Flament 1e35c3ccce Merge pull request #2531 from MarcAbonce/fix-browser-locale
[fix] Get correct locale with country from browser
2021-02-05 10:55:37 +01:00
Alexandre Flament 321788f14a Merge pull request #2528 from dalf/mod-ci-gh-pages
[mod] CI: minor changes
2021-02-04 23:12:27 +01:00
Alexandre Flament 90b9d0d6a8 [mod] CI: minor changes
* utils/makefile.python: travis-gh-pages renamed ci-gh-pages
2021-02-02 08:53:57 +01:00
Alexandre Flament 34de715e62 Merge pull request #2500 from dalf/github-action-data
[enh] every Sunday, call utils/fetch_*.py scripts and create a PR automatically
2021-02-01 17:16:58 +01:00