Throttling Apache HTTP Client Requests

Working with the Shopify API our application had to throttle the number of HTTP requests we make to the API. A brief background; our integration later would retrieve products from a 3rd party system, perform a diff to see what has changed and publish to a queue the products that need to be updated. Apache Camel route would listen to the queue and do the necessary updates on Shopify Products or Variants.

As the messages are read of the queue we would end up sending too many requests per second due to concurrent processing by Apache Camel. Its works, that how JMS is supposed to work.

From Shopify;

The API call limit operates using a “leaky bucket” algorithm as a controller. This allows for infrequent bursts of calls, and allows your app to continue to make an unlimited amount of calls over time. The bucket size is 40 calls (which cannot be exceeded at any given time), with a “leak rate” of 2 calls per second that continually empties the bucket. If your app averages 2 calls per second, it will never trip a 429 error (“bucket overflow”). To learn more about the algorithm in general, click here.

To fix this Shopify suggests limiting to 2 requests per seconds

Enter Google Guice!

Google Guice provides a RateLimiter class. From the javadocs;

A rate limiter. Conceptually, a rate limiter distributes permits at a configurable rate. Each acquire() blocks if necessary until a permit is available, and then takes it. Once acquired, permits need not be released.

Rate limiters are often used to restrict the rate at which some physical or logical resource is accessed. This is in contrast to java.util.concurrent.Semaphore which restricts the number of concurrent accesses instead of the rate (note though that concurrency and rate are closely related, e.g. see Little’s Law).

A RateLimiter is defined primarily by the rate at which permits are issued. Absent additional configuration, permits will be distributed at a fixed rate, defined in terms of permits per second. Permits will be distributed smoothly, with the delay between individual permits being adjusted to ensure that the configured rate is maintained.

creates a rate limiter that will make sure only 2 permits are issued per second. Other threads are blocked until they can acquire a permit.


Using versioned third party libraries on Wildfly 10

Wildfly comes with the the best module management for managing classloading of your deployed application.

1. Create new folder structure for the new driver, $WILDFLY_HOME/modules/org/postresql/9.4

2. Create the corresponding module.xml as below.

3. Add a driver based on the new module

Note the module is specified as org.postgresql:9.4 to specify the specific slot/version we want to use. If not specified Wildfly will user the default slot/version, main.

4. Use the driver in your datasource by specifying the driver as postgresql94


Setting application/system properties for WildFly 10

You can set application/system properties on Wildfly 10 (it should be the same with previous version of Wildfly). In this particular case we will set the property for the folder that will be used by logback to save logs.

Below is the logback configuration.

We need to set the application variable application.logs.dir so that logback can use the correct folder. To achieve this we add the system-properties in the configuration file, in our case standalone.xml. This should be immediately after the extension tag.

Don’t forget to restart the application server for the changes to take effect.

Tagged with:  

Install Root CA on Android

I have an SSL certificate issued by RapidSSL, via Geotrust. The CA is RapidSSL SHA256 CA – G3. It turns out the root CA is not installed by default in Android (5.0.2 in my case). I was getting the error; ‘..Certificate cannot be verified’. Android applications using SSL to the site would also fail, the Alfresco Mobile application was failing.


The solution is to install the CA onto Android. I managed to get the CA cert here; I downloaded the CA certificate in PEM format on Android and it was immediately available for installation. I was prompted for a name and saved.

Note that the installation Lock screen device policy, i.e., for the CA certificate to install the lock screen must be secured.

Tagged with:  

Install a Linux Dev machine

So i’m creating a developer machine, HP Probook 6560b. 8GB RAM, 500GB HDD. I will be running LinuxMint 17. I installed it using bootable USB. Below are the steps to get it ready for work, i’m a Java developer dabling in Python, Django.

Step 1
– change the Firefox search engine to Google.
– Go to here
– Click on the search engine drop down and add. Remove the rest.

Step 2
– lets get the system upto date.

Step 3
– install Ubuntu Tweak so that we can be able to keep the system clean.

Step 4
– install Oracle JDK.

– and remove Open JRE that is installed by default

Step 5
– install Git and Svn for source control.

Step 6
– install Google Chrome

Step 7
– install MySQL

Step 8
– install PostgreSQL

Step 9
– let make sure the Python environment is ready


Install Alfresco WCM from source

If you building Alfresco from source you you might want to add the WCM module to the base installation after

was meant to give project help but elects to skip the WCM modules part. The commands

will install the WCM module to alfresco.war and share.war files respectivily

Obvious confirmation will be in the logs at startup


Download Oracle JDK using wget

We are all used to downloading the JDK from Oracle using the browser. You will have to accept the licence to proceed with the download.

This makes it tricky to download on a Linux installation without or you don’t have access to its GUI. Trying to use wget will download an HTML page insisting you accept the licence. There are a couple of ways to get round ‘Accept Licence’ issue including the plain old download on local machine and upload to the remote machine. The simplest I got was to add the authentication parameter that’s used by the download server to verify if you have accepted the license to the JDK download URL used by wget.

The solution to get the precious AuthParam cookie, in this post I use Google Chrome to retrieve it. On the download page right click and select “Inspect Element”. Select the “Network” tab on the debugging window. After accepting the licence on the browser click on the file to download it. Quickly cancel the download and check the Network output. Right click the last result and copy the link.

wget will download the JDK.


Work, Motivation and Rewards

Interesting video on work, motivation and rewards in the workplace.

© 2011 - Pass me the Mic