Monitoring app performance

Front end performance: Speedtracer - can tell you how much time was spent on - DOM processing, garbage collection in the browser.
New relic tracks both front-end and backend performance by injecting javascript into the brower:
http://blog.newrelic.com/2011/05/17/how-rum-works/

Spring Roo 1.1.5 with GWT & GAE

mkdir rooapp
cd rooapp
start roo shell
roo> project --topLevelPackage com.xxx.rooapp --java 6
roo>persistence setup --provider DATANUCLEUS --database GOOGLE_APP_ENGINE
roo>entity --class ~.model.Product --testAutomatically
roo>field string --fieldName name --notNull
roo>field string --fieldName id --notNull
roo>field date --fieldName dateIntroduced --type java.util.Date --notNull
roo>field number --type java.lang.Float --fieldName unitPrice --notNull
roo>field string --fieldName description --notNull
roo>web gwt setup
I encountered a number of problems ...
The POM file it generated setup gae:home to be in the maven repo. Had to change it for things to work.
Compile failures with gwt:compile:
[INFO] [ERROR] Line 3: The import com.xxx.rooapp.server.gae.UserServiceLocator cannot be resolved
[INFO] [ERROR] Line 4: The import com.xxx.rooapp.server.gae.UserServiceWrapper cannot be resolved
This because GWT needs access to these sources - these files were not in the GWT source path!
Ran into lots of other errors around the generated classes for GAE and the GWT DesktopInjector ...
[ERROR] Generator 'com.google.gwt.inject.rebind.GinjectorGenerator' threw an exception while rebinding 'com.xxx.rooapp.client.scaffold.ioc.DesktopInjector'

Deploying apps with Puppet

Use Puppet master/agent

I want to deploy a simple application that installs a file in /tmp on a single box.

Here is the puppet module definition:
/etc/puppetlabs/puppet/modules/myapp/manifests/init.pp:
class myapp {
file { 'testfile' : path => '/tmp/testfile-local', ensure => 'present', content => 'Test', mode => 0640}
}

Typically, the puppet master uses a site.pp file for node definitions:
/etc/puppet/manifests/site.pp:

node development {
include "myapp"
}

node staging {
include "myapp"
}

Start the puppet master:
puppet master --no-daemonize --verbose
notice: Starting puppet master version 2.6.4

This is telling puppet master that myapp needs to be installed on the development box. If I have a puppet agent running on the client (which in this case happens to be the same box), then it will apply the latest configuration from the server automatically when it polls the next time.

I could run the agent manually onetime by ssh'ing into the box and invoking the agent:
[root@learn ~]# puppet agent --no-daemonize --onetime --server puppet --verbose
info: Retrieving plugin
info: Caching catalog for puppet
info: Applying configuration version '1310559260'
notice: /Stage[main]/Myapp/File[testfile-local]/ensure: created
notice: Finished catalog run in 0.02 seconds

CONS:
The main problem here is that I am not sure how you to achieve this via puppet master itself .. have not checked the UI - but surely there must be a way to update specific nodes - i.e. dev nodes only ?

You can always automate this using a capistrano script:
set :user, "root"
task :deploy
role :app, ‘development’
run 'puppet agent --no-daemonize --onetime --server puppet --verbose'
end

The problem with using a script like this is that the environment information has to be maintained in two places – in the script and the site.pp file. One way to avoid this is to generate this script from the site.pp file.

PROS:
You are using Puppet to install the application, just like a sysadmin would to ensure that machines were setup correctly. Easy to sell to ops.

Server-less puppet

Use capistrano to run puppet “apply” on the relevant nodes. Here is a Capfile for the app. It assumes that you have manifests checked out on the clients.

set :user, "root"

task :development do
role :app, “development”
end

task :staging do
role :app, "staging"
end

task :deploy do
# TODO: checkout manifests to module path
run "puppet apply -e \"include myapp\""
end

To deploy to development environment, you would run cap for that environment:

rg6977:puppet Thoughtworks$ cap development deploy
* executing `development'
* executing `deploy'
* executing "puppet apply -e \"include myapp\""
servers: ["192.168.56.101"]
Password:
[192.168.56.101] executing command
command finished

PROS:
Node definitions are now in Capistrano and not in puppet. Puppet is used only to install the application in a given environment. Puppet does a poor job of managing environment specific information. See http://docs.puppetlabs.com/guides/environment.html.

CONS:

Node definitions are now in Capistrano and not in puppet. Puppet is used only to install the application in a given environment.
This is more code than the previous approach. Puppet “apply” will only apply manifests to the local machine. So, if your application must be installed on a web server and db server, your Capistrano script needs to invoke puppet apply on the appropriate manifest (db vs web).

Amazon ec2

Command line api:
ec2-describe-images -o amazon
ec2-run-instances -k
ec2-describe-instances
ec2-authorize default -p 22 -open port for ssh
- ssh into the box
ssh -i root@xxx.amazonaws.com
ec2-terminate-instances
Good tutorial here

Puppet with Nagios

Puppet : Resources, aggregate resources using "defines" and "classes", organize using "modules" (see Puppet Language Guide)
Run-Stages used to control the order of resource management. The "sigils" - magical operators - <| and @ when doubled up - are very powerful.
Good article on configuring nagios with puppet. Also see the original Puppet example it references.

SNA Projects Blog at LinkedIn

Building a terabyte-scale data cycle at LinkedIn with Hadoop and Project Voldemort:
http://project-voldemort.com/blog/2009/06/building-a-1-tb-data-cycle-at-linkedin-with-hadoop-and-project-voldemort/
There are many other interesting articles at http://sna-projects.com/blog/

Amazon Dynamo
http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html

Running JBehave tests with xvfb

- Install xvfb
- Start xvfb: Xvfb :1 -screen 0 1024x768x24
- export DISPLAY=:1
- firefox (should start without errors)
- install vnc server and use a VNC client like ChickenOfVnc to connect and see your tests running.