©2015 -
Information Technology
Sample Sync Gateway Installation And Configuration Procedure
TABLE OF CONTENTS
System Configuration Requirements
Required Kernel Parameter Changes
Starting And Stopping Sync Gateway
Create Couchbase User For Sync Gateway
Sample Sync Gateway Configuration
Guide To Uninstall Sync Gateway
I am re-
- Maintains up-
to- date copies of documents where users need them. On mobile devices for instant access and on servers in data centers for reasons such as synchronizing documents, sharing documents, and loss- protection. Mobile apps create, update, and delete files locally, Sync Gateway takes care of the rest. - Provides access control, ensuring that users can only access documents to which they should have access.
NOTE: The Sync gateway goes beyond just having access control for mobile users but also it allows you to create virtual databases for specific group of users.
- Ensures that only relevant documents are synced. Sync Gateway accomplishes this by examining document and applying business logic to decide whether to assign the documents to channels. Access control and ensuring that only relevant documents are synced are achieved through the use of channels and the sync function.
SYSTEM CONFIGURATION REQUIREMENTS
REQUIRED KERNEL PARAMETER CHANGES
Modify /etc/sysctl.conf to include the following parameters:
#
### Recommended kernel parameters for Couchbase:
#
# Engage swapping only when memory is exhausted
vm.swappiness = 0
# the interval between subsequential keepalive probes
# Increase the frequency of keepalive probes, identifying and timing out dead connections faster.
net.ipv4.tcp_keepalive_intvl = 30
# the number of unacknowledged probes to send before considering the connection dead and notifying the application layer
net.ipv4.tcp_keepalive_probes = 4
# the interval between the last data packet sent (simple ACKs are not considered data)
# and the first keepalive probe; after the connection is marked to need keepalive, this counter is not used any further
net.ipv4.tcp_keepalive_time = 900
# Enables reuse of TIME-
# This setting is safe for all Couchbase protocols.
net.ipv4.tcp_tw_reuse = 1
# Limits the maximum memory used to 200MB before pdflush is involved.
# The default 20% of total system memory can overwhelm the storage system
# once flushed.
vm.dirty_bytes = 209715200
# The lower threshold for vm.dirty_bytes
vm.dirty_background_bytes = 104857600
# Do not set dirty ratio if dirty bytes are defined. Use one or the other parameter.
#vm.dirty_ratio = 10
#vm.dirty_background_ratio = 3
# Increases the rate at which data is flushed to disk
vm.dirty_expire_centisecs = 300
vm.dirty_writeback_centisecs = 100
# Increases the max open file handles.
fs.file-
# Disables NUMA zone reclaim algorithm. This tends to decrease read latencies.
vm.zone_reclaim_mode = 0
Adjust Open Socket range if heavy traffic is anticipated.
Default system settings:
[root@lxnode15 ~]# cat /proc/sys/net/ipv4/ip_local_port_range
32768 61000
[root@lxnode15 ~]#
Modify to increase open socket range:
vi /proc/sys/net/ipv4/ip_local_port_range
5000 64000
Open sockets allowed in a system is the difference of the above port ranges (59,000).
Reboot the Linux hosts once system changes are applied.
[root@lxnode15 ~]# groupadd -
[root@lxnode15 ~]#
[root@lxnode15 ~]# useradd -
[root@lxnode15 ~]#
[root@lxnode15 ~]# passwd couchbase
Changing password for user couchbase.
New password:
Retype new password:
passwd: all authentication tokens updated successfully.
[root@lxnode15 ~]#
vi /etc/security/limits.conf
couchbase soft nofile 71000
couchbase hard nofile 71000
sync_gateway soft nofile 51200
sync_gateway hard nofile 51200
NOTE: For couchbase user, 70000 is the minimum required open file descriptor limit. But here we set at 71000.
Download the sync gateway installer for your platform.
[chad@lxnode15 ~]$
[chad@lxnode15 ~]$ su -
Password:
[root@lxnode15 ~]# rpm -
sync_gateway start/running, process 7572
You have successfully installed Couchbase sync_gateway.
You can find sample sync_gateway configuration files in the /opt/couchbase-
You can control the Couchbase sync_gateway service by using the following command:
initctl start sync_gateway
That's it! sync_gateway is now running on port 4984. We've setup a simple in-
which works great for exploring sync_gateway's capabilities. A limited console is available
by opening your browser to http://localhost:4985/_admin/.
The command-
-
-
-
-
-
-
-
-
-
-
By using this software you agree to the End User License Agreement.
See /opt/couchbase-
[root@lxnode15 ~]#
There are 2 users created by the installer:
[chad@lxnode15 ~]$ tail -
couchbase:x:801:800:Couchbase system user:/opt/couchbase:/bin/bash
sync_gateway:x:802:802::/home/sync_gateway:/bin/bash
[chad@lxnode15 ~]$
The couchbase user is the owner of the binaries (/OPT/COUCHBASE) while the sync_gateway user is the runtime user and has the ownership of the config file ($HOME/sync_gateway.json).
The default configuration of Sync Gateway primarily connects to its local Walrus database:
[chad@lxnode15 ~]$ su -
Password:
[root@lxnode15 ~]# su -
[sync_gateway@lxnode15 ~]$ pwd
/home/sync_gateway
[sync_gateway@lxnode15 ~]$ ls -
total 12
drwxr-
drwxr-
-
[sync_gateway@lxnode15 ~]$
[sync_gateway@lxnode15 ~]$ cat sync_gateway.json
{
"log": ["HTTP+"],
"adminInterface": "127.0.0.1:4985",
"interface": "0.0.0.0:4984",
"databases": {
"db": {
"server": "walrus:data",
"users": {
"GUEST": {"disabled": false, "admin_channels": ["*"] }
},
"allow_conflicts": false,
"revs_limit": 20
}
}
}
[sync_gateway@lxnode15 ~]$
The port 4984 is the standard port for client connections (also known as PUBLIC REST API port) and normally is the only port you'll have to open in the firewall.
Port 4985 is for administrative purpose.
To test that your Sync Gateway port is reachable, a version response to should be displayed:
[sync_gateway@lxnode15 ~]$ curl http://localhost:4984/
{"couchdb":"Welcome","vendor":{"name":"Couchbase Sync Gateway","version":"2.0"},"version":"Couchbase Sync Gateway/2.0.0(832;2d8a6c0)"}
[sync_gateway@lxnode15 ~]$
STARTING AND STOPPING SYNC GATEWAY
CentOS/RedHat/Oracle 6
initctl start sync_gateway
initctl stop sync_gateway
CentOS/RedHat/Oracle 7
systemctl start sync_gateway
systemctl stop sync_gateway
Note: You need sudo privs to run the command from a user other than root.
If not already available, you need to create a bucket dedicated for sync gateway.
For this exercise, I have created a bucket -
CREATE COUCHBASE USER FOR SYNC GATEWAY
Create a user for Sync Gateway with RBAC roles -
In my case, I've created demoSyncData_user.
SAMPLE SYNC GATEWAY CONFIGURATION
As mentioned earlier, a newly installed Sync Gateway has a default config connecting to its local Walrus database.
The procedures below is for us to create a sample configuration connecting to a Couchbase cluster.
1. Shutdown the Sync Gateway.
[root@lxnode15 ~]# initctl status sync_gateway
sync_gateway start/running, process 8597
[root@lxnode15 ~]#
[root@lxnode15 ~]# initctl stop sync_gateway
sync_gateway stop/waiting
[root@lxnode15 ~]#
2. Backup the original config file.
[sync_gateway@lxnode15 ~]$ pwd
/home/sync_gateway
[sync_gateway@lxnode15 ~]$
[sync_gateway@lxnode15 ~]$ ls
data logs sync_gateway.json
[sync_gateway@lxnode15 ~]$ mv sync_gateway.json sync_gateway.json.orig.bak
[sync_gateway@lxnode15 ~]$
[sync_gateway@lxnode15 ~]$ ls
data logs sync_gateway.json.orig.bak
[sync_gateway@lxnode15 ~]$
3. Create a new log directory.
Sync Gateway logs all changes and errors and tends to be high in volume in production implementations.
The default log path is located on sync_gateway user $HOME/logs directory which in most implementations had smaller disk space.
It is best that you create a separate directory on a mount point with more disk space and set your log path.
[root@lxnode15 ~]# mkdir /app
[root@lxnode15 ~]# chmod 777 /app
[root@lxnode15 ~]# ls -
drwxrwxrwx. 2 root root 4096 Jul 10 17:55 /app
[root@lxnode15 ~]# su -
[sync_gateway@lxnode15 ~]$
[sync_gateway@lxnode15 ~]$ mkdir -
[sync_gateway@lxnode15 ~]$ ls -
total 4
drwxrwxr-
[sync_gateway@lxnode15 ~]$
3. Create a sync gateway config file connecting to your Couchbase cluster.
file: sync_gateway.json
{
"interface":":4984",
"adminInterface":"lxnode15.vlabs.net:4985",
"log": ["*"],
"logging":{
"default": {
"logFilePath": "/app/syncgateway/log/sync_gateway_error.log",
"rotation": {
"maxsize": 250,
"maxage": 14,
"maxbackups": 30
}
}
},
"databases": {
"syncdb": {
"server": "http://lxnode11.vlabs.net,lxnode12.vlabs.net:8091",
"bucket": "demoSyncData",
"username": "demoSyncData_user",
"password": "demo123",
"enable_shared_bucket_access": true,
"import_docs": "continuous",
"users": { "GUEST": { "disabled": false, "admin_channels": ["*"] } },
"sync": `function (doc, oldDoc) {
if (doc.sdk) {
channel(doc.sdk);
}
}`
}
}
}
NOTE: When naming sync gateway database, use only lowercase characters and numbers; otherwise, an "error 400 Illegal database name" occurs.
logging.default.rotation
maxsize (integer, default 100) The maximum size in MB of the log file before it gets rotated.
maxage (integer) The maximum number of days to retain old log files.
maxbackups (integer) The maximum number of old log files to retain.
databases.db
import_docs (boolean) If set to true, causes the Sync Gateway to scan the bucket on startup looking for documents
that have no _sync metadata, and adding that metadata. You only want to have this on when you do the first import,
because it will slow down the startup of the Sync Gateway. You also only want to run a single Sync Gateway while
in this mode, otherwise they'll step on each other's toes trying to generate the _sync metadata at the same time.
In Sync Gateway 1.5, the behaviour of this property has changed to work in tandem with the enable_shared_bucket_access
property. When shared bucket access is enabled, setting this property to continuous ("import_docs": "continuous")
indicates that this Sync Gateway node should perform import processing. If shared bucket access is not enabled,
Sync Gateway continues to support the previously existing import functionality.
In a deployment with shared bucket access enabled and Sync Gateway Accelerator nodes, this property is not required
because the import process is always shared across Sync Gateway Accelerator.
enable_shared_bucket_access (boolean, default false)
Starting in Sync Gateway 1.5, Couchbase Server and Mobile applications can now read from and write to the same bucket.
This property specifies whether to enable the interopability between Couchbase Mobile and Couchbase Server.
On start-
Couchbase Server bucket. From then on, documents can be inserted on the Server directly (with N1QL or SDKs) or through
the Sync Gateway REST API.
Mobile applications require additional metadata in order to manage security and replication. In previous versions of
Sync Gateway, this information has always been stored in the document body. Sync Gateway 1.5 utilizes a new feature of
Couchbase Server 5.0 called XATTRs (x-
For a complete description of configuration parameters, please refer to the online guide -
4. Startup the Sync Gateway with the new config.
This will then create the initial meta info in the bucket.
[root@lxnode15 ~]# initctl start sync_gateway
sync_gateway start/running, process 9692
[root@lxnode15 ~]#
Sample log snippet:
[sync_gateway@lxnode15 log]$ tail -
2018-
2018-
2018-
2018-
2018-
2018-
2018-
2018-
2018-
2018-
2018-
2018-
2018-
2018-
2018-
2018-
2018-
2018-
2018-
2018-
2018-
2018-
2018-
2018-
2018-
2018-
2018-
2018-
At this point your sync gateway is now ready to accept document inputs and changes.
[chad@lxnode15 ~]$ curl http://lxnode15.vlabs.net:4984/syncdb/
{"committed_update_seq":1,"compact_running":false,"db_name":"syncdb","disk_format_version":0,"instance_start_time":1531265732497424,"purge_seq":0,"state":"Online","update_seq":1}
[chad@lxnode15 ~]$
If you’d like to learn the basics of Sync Gateway data updates, please refer to my article -