Xray: Online DB sync - using Nginx Forward Proxy

Online Synchronization

• Online: In online mode, Xray synchronizes with the global database server automatically on a daily basis through an internet connection

In general, we recommend that you use online synchronization when the Internet is accessible and offline synchronization when the Internet is restricted;
Otherwise, you might see an error like this, and get ERROR messages in the xray-server-service.log

Caused by: Get "https://jxray.jfrog.io/api/v1/updates/onboarding?version=3.79.11": dial tcp: lookup jxray.jfrog.io on 198.18.xx.xx:53: dial udp 198.18.xx.xx:53: connect: network is unreachable

How to handle network isolation?

In corporate environments, it is often required to go through a proxy server to access remote resources, such as a remote repository or Federated repository, or Xray DB sync. So you can setup a proxy for Xray DB sync.
If you do not have a proxy here, you can use the Nginx to set up a proxy in your DMZ region.
 

Follow the below steps to set Nginx forward proxy

1. Download & Install Nginx + ngx_http_proxy_connect_module
Note: The module version needs to match the Nginx version 



 

$ wget https://nginx.org/download/nginx-1.25.2.tar.gz 
$ tar -xf nginx-1.25.2.tar.gz
$ cd nginx-1.25.2
$ wget https://github.com/chobits/ngx_http_proxy_connect_module/archive/refs/tags/v0.0.5.tar.gz 
$ tar -xf v0.0.5.tar.gz 
$ patch -p1 < ngx_http_proxy_connect_module-0.0.5/patch/proxy_connect_rewrite_102101.patch
$ yum install -y pcre pcre-devel openssl openssl-devel gd gd-devel zlib patch libxml2-devel libxslt-devel perl-devel perl-ExtUtils-Embed perl zlib-devel patch redhat-rpm-config-9.1.0-88.el7.centos.noarch gcc-c++ gcc gcc-devel unzip
$ ./configure --prefix=/usr/local/nginx-1.25.2 --add-module=/opt/jfrog/nginx-1.25.2/ngx_http_proxy_connect_module-0.0.5
$ make -j2 && make install

2. Create nginx config file (in China, we use resolver as 223.5.5.5)
Add a server to forward the requests of Xray DB Sync, like the server to listen 3128 in this example.

$ cd /usr/local/nginx-1.25.2/conf/
$ vim nginx.conf
worker_processes  1;
events {
    worker_connections  1024;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;
    server {
        listen       80;
        server_name  localhost;
        location / {
            root   html;
            index  index.html index.htm;
        }
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
     server {
         listen                         3128;

         # dns resolver used by forward proxying
         resolver                       8.8.8.8;

         # forward proxy for CONNECT request
         proxy_connect;
         proxy_connect_allow            443 563;
         proxy_connect_connect_timeout  10s;
         proxy_connect_read_timeout     10s;
         proxy_connect_send_timeout     10s;

         # forward proxy for non-CONNECT request
         location / {
             proxy_pass http://$host;
             proxy_set_header Host $host;
         }
    }

}

3. Start Nginx 

$ /usr/local/nginx-1.25.2/sbin/nginx -t
nginx: the configuration file /usr/local/nginx-1.25.2/conf/nginx.conf syntax is ok
nginx: configuration file /usr/local/nginx-1.25.2/conf/nginx.conf test is successful
$ /usr/local/nginx-1.25.2/sbin/nginx

4. Config the proxy on Artifactory UI



5. Run Xray DB sync again