Sunday, May 4, 2014

Meteor, load balancing and sticky sessions

Meteor clients establish a long-lived connection with the server that is uniquely identified by a session identifier to support the DDP protocol.  The DDP protocol allows Meteor clients to make RPC calls and also allows the server to keep the client updated with changes to data, i.e., Mongo documents. Meteor uses sockjs, which provides a cross-browser web-socket like API which falls back on long-polling when web sockets are not supported by the server.

Meteor with sockjs long-polling

Consider the following setup: Nginx is acting as a load balancer that is load-balancing 2 or more meteor servers.  The Nginx server configuration looks like this:
upstream meteor_server_lp {
   server localhost:3000;
   server localhost:3001;
}
server {
        listen       8084;
        server_name  localhost;

        location / {
            proxy_pass  http://meteor_server_lp;
        }

}
This configuration of Nginx does not support web-sockets, so the Meteor clients will use long polling.  Since long polling re-establishes connections every so often due to connection timeouts, such a configuration will require sticky sessions to ensure that client is directed to the same server that they have previously established a connection with. A sockjs connection that is directed to the wrong server by the Nginx load balancer will fail with a 404 Not Found.  The solution is to compile Nginx with the sticky module, and modify the configuration to be sticky like this:
upstream meteor_server_lp {
    sticky;
   server localhost:3000;
   server localhost:3001;
}
server {
        listen       8084;
        server_name  localhost;

        location / {
            proxy_pass  http://meteor_server_lp;
        }

}
When using this setup with AWS load balancers, sticky session needs to be enabled on the load balancer - app stickiness using the "route" cookie setup by Nginx's sticky module.

Meteor with web sockets

Nginx since 1.3.13, supports web-sockets using the protocol switch mechanism in HTTP/1.1. The configuration file looks like this:
upstream meteor_server {
   server localhost:3000;
   server localhost:3001;
}
server {
        listen       8082;
        server_name  localhost;

        location / {
            proxy_pass  http://meteor_server;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
        }
}
This works as is without the need for any sticky sessions. The websocket connection between the client and any one of the two servers will be established when the client first connects or reconnects or if one of the servers go down.  AWS load balancers don't support websockets with http listeners, but it works with a tcp listener setup.  But this means any SSL termination must occur on the instances being load balanced. 

Friday, March 16, 2012

HAProxy and SSL

HAProxy does not have support for SSL. Common solution is to use Stud to handle SSL and send un-encrypted data to the backends.
Terminating SSL in the load balancer is not considered a good idea because it does not scale.
It is considered better to use webservers like Nginx with session caching enabled.
Good benchmark comparing Nginx, Stud and Stunnel is here- http://vincent.bernat.im/en/blog/2011-ssl-benchmark.html.
Another benchmark comparing stud,stunnel and nginx: http://matt.io/entry/uq and the follow up which establishes Nginx to be just as performant as Stud - the key is picking the right cipher.
http://matt.io/technobabble/hivemind_devops_alert:_nginx_does_not_suck_at_ssl/ur

Sunday, February 19, 2012

Tomcat with HAProxy/Nginx

Tomcat is usually fronted with a http server for various reasons - security, load balancing and additional functionality like URL-rewriting. Most common options for the proxy include: HTTPD, HAProxy and NGINx.

Compile HAProxy from source
$ make
$ make TARGET=generic
$ sudo make install

Resources:
http://www.tomcatexpert.com/blog/2010/07/12/trick-my-proxy-front-tomcat-haproxy-instead-apache
http://www.mulesoft.com/tomcat-proxy-configuration
http://haproxy.1wt.eu/download/1.2/doc/architecture.txt

Tuesday, January 31, 2012

Comet technology

Server Push, long polling, Good descriptions here: http://code.google.com/p/google-web-toolkit-incubator/wiki/ServerPushFAQ
Maturity of Comet implementations: http://cometdaily.com/maturity.html
Best Comet/Streaming server: Caplin Liberator (http://www.caplin.com/caplin_liberator.php)

Sunday, August 14, 2011

Building C++/.NET apps with MSBuild 4.0

In .NET 4.0/VS 2010, Microsoft replaced vcbuild.exe with msbuild.exe.
To build both .NET managed as well as native C++ apps, you only need .NET 4 along with Windows 7 SDK:
http://www.microsoft.com/download/en/details.aspx?displayLang=en&id=8279
There is no need to install VS 2010.
Here is a walkthrough for a simple hello world C++ app:
http://msdn.microsoft.com/en-us/library/dd293607.aspx
With VS 2010, vcbuild.exe is no longer used to build C++ projects.

For VS 2008 solution files, you will need Microsoft Windows 7 SDK and .NET 3.5:
http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=3138
After installation, use the CMD shell (Programs->Windows 7 SDK->Cmd) to invoke msbuild on solution files.
This version of MSBuild (3.xx) uses vcbuild.exe to build C++ projects.

There is no need to install VS 2008.

Thursday, July 28, 2011

Event loop approach to concurrency

Event loop approach to concurrency as an alternative to threading - everything is non-blocking and executed via callbacks. The event loop is executing a queue of callbacks forever. This works well as long as the callbacks complete quickly! If a callback is going to take long it should fork another process. The primary application is networking - non-blocking I/O.
Douglas Crockford's presentation on event loop approach to concurrency

Libraries that use this approach include node.js, Ruby's Event Machine and Python's Twisted. and Java's new JDK7 Asynchronous IO and the older NIO library. The main difference between new Asychronous I/O and the older NIO - for NIO you are notified when the read operation is ready to start (data is available); while in Asychronous I/O - you are notified only when the read is completed (all data is read).

The design pattern being employed in all of these is the Reactor Design pattern. The Reactor pattern allows for the activation of handlers when events occur (e.g. activates handler to read data from socket when the data is available)

Wednesday, July 27, 2011

Five minute rule

The new five-minute rule
Compares the cost of holding data in memory vs disk I/O. With flash memory prices becoming cheaper, you can now pool memory from different machines to provide an ocean of RAM with low-latency.
RAM -> Flash memory -> Disk