retoor/searxng
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
4,274 Commits 6 Branches 17 Tags
a3a7be90ff370ec3987f29a8b3043fa2bcaedf39
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Alexandre Flament a3a7be90ff Merge pull request #50 from searxng/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2021-05-01 10:28:22 +02:00
Alexandre Flament 4558c0c027 Merge pull request #48 from searxng/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2021-05-01 10:28:11 +02:00
Alexandre Flament d16fdf662f Merge pull request #47 from searxng/update_data_update_firefox_version.py 
Update searx.data - update_firefox_version.py
 2021-05-01 10:26:51 +02:00
Alexandre Flament 96ae39c3b6 Update data-update.yml 
Call to peter-evans/create-pull-request@v3 : 
Use the default GITHUB_TOKEN
 2021-05-01 10:21:27 +02:00
Alexandre Flament 898d789e82 Update data-update.yml 
Disable fail fast: so even if a script fails, the other scripts create PR.
 2021-05-01 10:15:17 +02:00
Alexandre Flament 26b0de6843 Update data-update.yml 
- searxng brand
- fix call to ./manage
 2021-05-01 10:10:24 +02:00
Alexandre Flament 0ff14fd3ec Merge pull request #22 from searxng/docker-buildx 
[docker] multiarch build
 2021-04-30 16:20:33 +02:00
Alexandre Flament e5bcc3cbbe Merge pull request #40 from searxng/link-to-stats 
[mod] engine errors: link to the stats to create an github issue
 2021-04-29 14:37:12 +02:00
Alexandre Flament 34bced29ae [mod] engine errors: link to the stats to create an github issue 2021-04-29 11:10:38 +02:00
Alexandre Flament 3282cdca13 Merge pull request #38 from return42/filtron-rules 
[filtron.sh] make filtron rule file configurable
 2021-04-29 10:31:59 +02:00
Alexandre Flament 1e3148fea9 Merge pull request #41 from return42/fix-themes 
fix varius theme issues
 2021-04-29 10:28:53 +02:00
Alexandre Flament 20580bcbd4 [docker] multiarch support: linux/amd64,linux/arm64,linux/arm/v7 
make docker.buildx : build and push multiarch build.
(it can't be only build)

use buildx with the --cache-from and --cache-to options to cache the layers
(only the last built is cached)
 2021-04-28 14:40:00 +02:00
Alexandre Flament fe3831db1a Merge pull request #37 from return42/drop-req-idna 
[fix] drop 'idna' from requirements.txt
 2021-04-27 20:08:52 +02:00
Alexandre Flament b06a15376c Merge pull request #32 from searxng/fix-network 
[fix] searx.network: fix rare cases where LOOP is None
 2021-04-27 20:06:54 +02:00
Alexandre Flament 283ae7bfad [fix] searx.network: fix rare cases where LOOP is None 
* searx.network.client.LOOP is initialized in a thread
* searx.network.__init__ imports LOOP which may happen
  before the thread has initialized LOOP

This commit adds a new function "searx.network.client.get_loop()"
to fix this issue
 2021-04-27 17:47:36 +02:00
Alexandre Flament 5781d75ca2 [clean] drop deprecated files .codecov.yml, .landscape.yaml & tox.ini 
.codecov.yml :
  code coverage reports are attached on each build. See for example:
  https://github.com/searxng/searxng/actions/runs/788360104 (Artifacts section)

.landscape.yaml :
  landscape.io is offline

tox.ini :
  not in use
 2021-04-27 16:52:53 +02:00
Alexandre Flament 87e914e398 Merge pull request #28 from searxng/mod-processors-error-message 
[mod] processors: show identical error messages on /search and /stats
 2021-04-27 16:50:41 +02:00
Alexandre Flament f45729ae72 Merge pull request #35 from return42/fix-manage 
[fix] manage - fix miss usage of 'set -e'
 2021-04-27 14:39:43 +02:00
Alexandre Flament b1557b5443 [mod] processors: show identical error messages on /search and /stats 2021-04-27 14:20:07 +02:00
Alexandre Flament b8f5adf500 [mod] searx.metrics.error_recorder: store relative file name instead of the full absolute file name. 2021-04-27 10:42:00 +02:00
Alexandre Flament 9b482e8fcc Merge pull request #31 from return42/mod-pylint 
[pylint] get PYLINT_FILES from tag '# lint: pylint'
 2021-04-26 22:58:44 +02:00
Alexandre Flament f4ab1e1251 Merge pull request #30 from return42/brand-readme 
[brand] README.rst fix img.shields.io and searxng.github.io URLs
 2021-04-26 16:13:08 +02:00
Alexandre Flament 12bdf7146b Merge pull request #27 from return42/fix-debug-log 
[fix] debug log: UnicodeEncodeError: 'ascii' codec can't encode
 2021-04-26 13:50:29 +02:00
Alexandre Flament 51494849fc Merge pull request #23 from searxng/metrics-stats-engines 
Metrics stats engines
 2021-04-25 14:31:52 +02:00
Alexandre Flament d20f2822d0 Merge pull request #25 from return42/brand-001 
SearxNG brand
 2021-04-25 14:30:18 +02:00
Alexandre Flament df41b77121 [mod] /stats : detail per engine 
allow to submit a github issue including the technical details
(exceptions, errors, warning, checker result)
 2021-04-25 14:28:06 +02:00
Alexandre Flament eebac31ccd Merge pull request #26 from return42/brand-002 
[github] searxng does not use github's discussion
 2021-04-25 12:59:11 +02:00
Alexandre Flament 9937a908c0 Merge pull request #24 from return42/fix-doc 
[fix doc] manage.sh update_packages does not exists anymore
 2021-04-25 10:22:32 +02:00
Alexandre Flament 0603b043ce Merge pull request #16 from return42/add-core.ac 
Add a search engine for core.ac.uk
 2021-04-24 09:11:20 +02:00
Alexandre Flament 4863c1933e Rerevert c54bf42 2021-04-24 07:59:16 +02:00
Alexandre Flament 525bfe7830 Fix PEP8 2021-04-24 07:19:44 +02:00
Alexandre Flament a7b9eca98a Merge pull request #8 from return42/manage-script 
Replace Makefile boilerplate by shell scripts
 2021-04-24 07:14:35 +02:00
Alexandre Flament fe064a5c39 temporary partial revert of c54bf42cb9 2021-04-24 07:13:04 +02:00
Alexandre Flament 3cdd6a6a50 Merge pull request #12 from searxng/metrics-stats-page 
[mod] update /stats
 2021-04-24 07:02:52 +02:00
Alexandre Flament 7032f7521c [mod] /stats: simple theme implementation 2021-04-24 06:58:49 +02:00
Alexandre Flament 09e7ecdce2 [mod] /stats : add reliability column and sort by column links 2021-04-24 06:58:49 +02:00
Alexandre Flament 65c29081cc [mod] update /stats 2021-04-24 06:58:22 +02:00
Alexandre Flament 461c6fb21e Merge pull request #20 from return42/fix-commit-msg 
[CI] JamesIves/github-pages-deploy-action COMMIT_MESSAGE
 2021-04-23 14:38:07 +02:00
Alexandre Flament 1a6e3b3ed0 Merge pull request #19 from return42/fix-gh-pages 
[CI] JamesIves/github-pages-deploy-action SINGLE_COMMIT
 2021-04-23 13:00:52 +02:00
Alexandre Flament 3755cf2eba More branding 2021-04-22 21:10:22 +02:00
Alexandre Flament 04b34c70b8 More branding 2021-04-22 20:14:16 +02:00
Alexandre Flament e8c2fbc433 Merge pull request #17 from searxng/searxng 
searxng brand
 2021-04-22 20:01:02 +02:00
Alexandre Flament 9d84025e4f searxng brand 2021-04-22 19:57:02 +02:00
Alexandre Flament d01741c9a2 Merge pull request #15 from return42/add-springer 
Add a search engine for Springer Nature
 2021-04-22 13:23:31 +02:00
Alexandre Flament cdcfd74f39 Merge pull request #14 from return42/add-artic 
[enh] add engine using API from "The Art Institute of Chicago"
 2021-04-22 13:22:49 +02:00
Alexandre Flament 981cdb7acf Merge pull request #10 from return42/fix-youtube 
[fix] youtube - send CONSENT Cookie to not be redirected
 2021-04-22 12:21:31 +02:00
Alexandre Flament e9508cc897 Merge pull request #13 from return42/fix-morty-doc 
[fix] Fix typo in morty installation instruction
 2021-04-22 12:20:52 +02:00
Alexandre Flament a0e55a587a Merge pull request #11 from return42/add-doi 
[enh] add DOI resolver from sci-hub / replace default DOI
 2021-04-22 12:20:37 +02:00
Alexandre Flament f251a9718e Merge pull request #9 from return42/drop-https-rewrite 
[drop] plugin HTTPS rewrite - rule base is to old (update is to big)
 2021-04-22 11:55:58 +02:00
Alexandre Flament c6d5605d27 Merge pull request #7 from searxng/metrics 
Metrics
 2021-04-22 08:34:17 +02:00
Alexandre Flament baff1cbbab fix issues from review 2021-04-21 18:49:13 +02:00
Alexandre Flament b7848e3422 [fix] searxng fix: sjp engine 2021-04-21 16:31:29 +02:00
Alexandre Flament 7cfd8d900a [mod] oscar: /preferences , engines tab: report engine times 
* display the median time instead of the average.
* add a "Reliability" column (sum up the metrics and the checker results).
* the "selected language", "SafeSearch", "Time range" values are displayed as "broken" when the checker tests fail.
 2021-04-21 16:24:46 +02:00
Alexandre Flament c27fef1cde [mod] metrics: add secondary parameter 
Some error won't stop the engine:
* additional HTTP redirects for example
* some invalid results

secondary=True allows to flag these errors as not important.
 2021-04-21 16:24:46 +02:00
Alexandre Flament 7acd7ffc02 [enh] rewrite and enhance metrics 2021-04-21 16:24:46 +02:00
Alexandre Flament aae7830d14 [mod] refactoring: processors 
Report to the user suspended engines.

searx.search.processor.abstract:
* manages suspend time (per network).
* reports suspended time to the ResultContainer (method extend_container_if_suspended)
* adds the results to the ResultContainer (method extend_container)
* handles exceptions (method handle_exception)
 2021-04-21 16:24:46 +02:00
Alexandre Flament ae5954f2da Merge remote-tracking branch 'return42/fix-preference-save' 2021-04-21 16:24:22 +02:00
Alexandre Flament 624d3ec6db Merge remote-tracking branch 'return42/fix-url-bar-suggestion' 2021-04-21 16:23:52 +02:00
Alexandre Flament 92db0227b1 Merge remote-tracking branch 'dalf/oscar-images' 2021-04-19 09:36:54 +02:00
Alexandre Flament 48720e20a8 Merge remote-tracking branch 'searx/master' 2021-04-19 09:35:12 +02:00
Alexandre Flament 01cefffbf6 Merge pull request #1 from metasearch-lab/httpx_networks 
Httpx networks
 2021-04-12 17:34:21 +02:00
Alexandre Flament d14994dc73 [httpx] replace searx.poolrequests by searx.network 
settings.yml:

* outgoing.networks:
   * can contains network definition
   * propertiers: enable_http, verify, http2, max_connections, max_keepalive_connections,
     keepalive_expiry, local_addresses, support_ipv4, support_ipv6, proxies, max_redirects, retries
   * retries: 0 by default, number of times searx retries to send the HTTP request (using different IP & proxy each time)
   * local_addresses can be "192.168.0.1/24" (it supports IPv6)
   * support_ipv4 & support_ipv6: both True by default
     see https://github.com/searx/searx/pull/1034
* each engine can define a "network" section:
   * either a full network description
   * either reference an existing network

* all HTTP requests of engine use the same HTTP configuration (it was not the case before, see proxy configuration in master)
 2021-04-12 17:25:56 +02:00
Alexandre Flament eaa694fb7d [enh] replace requests by httpx 2021-04-10 15:38:33 +02:00
Alexandre Flament 111180705b [fix] test: avoid HTTP requests 
patch engine initialization to skip HTTP request
(engine_init function in searx.engines.initialize_engines)
 2021-04-10 15:38:27 +02:00
Alexandre Flament c09ff4faf2 [fix] fix PR 2656 
SCRIPT_NAME remove trailing slash to avoid infinite redirect
 2021-04-07 13:05:55 +02:00
Alexandre Flament 7089526723 Merge pull request #2656 from return42/fix-url_for 
[fix] url_for(..., _external=True) in templates
 2021-04-05 14:50:39 +02:00
Alexandre Flament 7a0fbdecc4 [enh] oscar: image thumbnail layout 
Adjust thumbnail sizes to fill the container width
 2021-04-05 11:29:17 +02:00
Alexandre Flament ac0fdc3b96 Merge pull request #2702 from MarcAbonce/fix-dark-infobox 
Fix dark "expand" button from infobox
 2021-04-02 10:38:11 +02:00
Alexandre Flament bb316d2698 Merge pull request #2682 from dalf/fix-checker 
[fix] checker: various bug fixes
 2021-03-27 17:44:29 +01:00
Alexandre Flament 725a69616b Merge pull request #2681 from dalf/fix-wikipedia-title 
[fix] wikipedia: remove HTML from the title
 2021-03-27 17:43:36 +01:00
Alexandre Flament 672ac91f3e Merge pull request #2684 from searx/dependabot/pip/master/linuxdoc-20210324 
Bump linuxdoc from 20210110 to 20210324
 2021-03-26 13:21:05 +01:00
Alexandre Flament e27e0c4843 Merge pull request #2683 from searx/dependabot/pip/master/lxml-4.6.3 
Bump lxml from 4.6.2 to 4.6.3
 2021-03-26 08:11:53 +01:00
Alexandre Flament ae0b621ec8 [mod] Dockerfile: remove protobuff dependency 
See https://github.com/searx/searx/pull/2481
 2021-03-25 17:09:28 +01:00
Alexandre Flament 0b45afd4d7 [fix] checker: various bug fixes 
* initialize engine_data (youtube engine)
* don't crash if an engine don't set result['url']
 2021-03-25 09:37:37 +01:00
Alexandre Flament fcfcf662ff [fix] wikipedia: remove HTML from the title 
fr.wikipedia.org (and it seems not other wikipedia websites),
adds HTML to api_result['displayTitle'].
(Search for '!wp :fr Braid' for example)

The commit uses api_result['title']
 2021-03-25 08:31:39 +01:00
Alexandre Flament 5f7056cf37 Merge pull request #2661 from dalf/pref-engine-tooltip 
[mod] preferences: a tooltip is shown when the mouse is over the engine names
 2021-03-22 13:36:52 +01:00
Alexandre Flament d648001688 [mod] preferences: a tooltip is shown when the mouse is over the engine names 2021-03-22 08:22:59 +01:00
Alexandre Flament bfd4e1b342 Merge pull request #2673 from dalf/fix-oscar-sourcemaps 
[mod] oscar: fix the sourcemap URL in *.min.css
 2021-03-22 08:16:46 +01:00
Alexandre Flament 6bd01bf81f [mod] oscar: fix the sourcemap URL in *.min.css 
Close https://github.com/searx/searx/issues/2670

Note: clean-css contains a bug:
* a multiline comment or URL adds "$stdin" to the sourcemap (see src/less/logicodev/search.less)
* in this case when the user opens the devtools, the browser fails to load this "https://.../$stdin" URL
* it is not a problem and the error appears only when the user actively tries to debug the CSS.
* seems related to https://github.com/jakubpawlowicz/clean-css/issues/593
 2021-03-21 18:03:40 +01:00
Alexandre Flament a48ec0b4bd Merge pull request #2671 from searx/update-soundcloud 
[mod] soundcloud: faster initialization
 2021-03-21 15:10:39 +01:00
Alexandre Flament 30c950a2c7 Merge pull request #2660 from dalf/upd-translations 
[mod] replace /translations.js with an embedded JSON
 2021-03-21 12:39:26 +01:00
Alexandre Flament 38c210d746 [mod] soundcloud: faster initialization 
The get_cliend_id() function:
* fetches https://soundcloud.com
* then fetches each referenced javascript URL to get the client id.

This commit fetches the javascript URLs in the reverse order: the client id is in the last javascript URL.
 2021-03-21 09:29:53 +01:00
Alexandre Flament 12fa524cf7 Merge pull request #2667 from searx/dependabot/pip/master/twine-3.4.1 
Bump twine from 3.3.0 to 3.4.1
 2021-03-20 19:50:27 +01:00
Alexandre Flament 7a05d5c9cd Merge pull request #2669 from jhigginbotham/master 
Update settings.yml to enable HTTP for yacy engine
 2021-03-20 18:37:24 +01:00
Alexandre Flament 7e5553ddb6 Merge pull request #2662 from dalf/oscar-trim-space 
[mod] oscar: remove space
 2021-03-18 09:50:52 +01:00
Alexandre Flament 2b0dd96bd3 [mod] oscar: remove space 
* reduce by 15% the uncompressed output (on average)
* dos2unix searx/templates/oscar/result_templates/files.html
 2021-03-17 09:22:05 +01:00
Alexandre Flament 6553c79029 [mod] replace /translations.js by embedded JSON 
In webapp.py, there is a new function "get_translations" lists available translations

Close #2064
 2021-03-16 11:22:21 +01:00
Alexandre Flament cb3b379161 Merge pull request #2206 from dalf/upgrade-pygments 
[mod] upgrade pygments
 2021-03-16 11:06:25 +01:00
Alexandre Flament 32cd0d31b3 [mod] upgrade pygments 
add searx_extra/update/update_pygments.py to update the css style of the oscar and simple themes.
 2021-03-16 09:07:08 +01:00
Alexandre Flament 1664258061 Merge pull request #2655 from return42/fix-imports 
[fix] remove unused import from yahoo-news engine
 2021-03-15 08:38:34 +01:00
Alexandre Flament 5b176b3496 Merge pull request #2659 from MarcAbonce/onions-http-fix 
Fix HTTP error in onion engines
 2021-03-15 08:33:38 +01:00
Alexandre Flament 0b71d61dfc Merge pull request #2654 from return42/acgsou 
[drop] Acgsou engine - www.acgsou.com no longer exists
 2021-03-15 08:27:15 +01:00
Alexandre Flament 9292571304 Merge pull request #2346 from dalf/upgrade-oscar 
[mod] oscar: upgrade dependencies
 2021-03-13 09:29:13 +01:00
Alexandre Flament 92dd5e245e Merge pull request #2626 from mikeri/solidtorrents 
Add Solid Torrents engine
 2021-03-12 19:45:22 +01:00
Alexandre Flament a1a492baed Merge pull request #2641 from dalf/disable_http_by_default 
[mod] by default allow only HTTPS, not HTTP
 2021-03-12 19:21:46 +01:00
Alexandre Flament af3e969c5a Merge pull request #2642 from return42/fix-apkmirror 
[fix] APKMirror engine - update xpath selectors and fix img_src
 2021-03-11 09:48:31 +01:00
Alexandre Flament 8b650e6a2d Merge pull request #2643 from return42/fix-makefile 
[fix] make targets engines.languages and useragents.update
 2021-03-11 09:44:30 +01:00
Alexandre Flament cb04d42806 [mod] oscar: update README.rst 2021-03-11 09:33:04 +01:00
Alexandre Flament 86912e2272 [mod] oscar: get bootstrap and typeahead from NPM 2021-03-11 09:33:01 +01:00
Alexandre Flament 44407353ef [mod] oscar: get leaflet and jquery from NPM 
easy to upgrade (package.json)
 2021-03-11 09:32:22 +01:00