Docker in OSX via boot2docker or Vagrant: getting over the hump

There are bucket loads of fantastic Docker related blog posts out there. This is yet another tldr guide to get the readers over the hump, particularly illustrating how you can mount local file systems onto a Docker container with Vagrant.

Before we begin, Docker is a client/daemon application. Docker daemon can listen to commands from the Docker client. The client is thin and all containers are hosted on the daemon. This means if you point Docker client to another daemon endpoint, the usual available list of images and containers will change. This is a good thing.

Installing Docker on OSX

As stated on the official Docker installation guide, install via Homebrew is highly recomended.

brew install boot2docker
brew install docker

Note that brew only installs the Docker client on OSX. boot2docker is the VirtualBox wrapper that will fire up Docker daemon. This is important as at the time of writing, commands performed on Docker clients are executed and takes place on the Docker daemon. For example, mounting files to a container really refers to mounting files from the server that the Docker daemon is running to the desired container. This is a cause of many confusion.

Running Docker with boot2docker

The installation guide is really good, and here is a tldr version. Note that the following script uses a different boot2docker port. This is to avoid port conflict with the Docker daemon via Vagrant later.

echo “export DOCKER_PORT=14243” >> ~/.boot2docker/profile 
boot2docker init
boot2docker start
export DOCKER_HOST=tcp://localhost:14243
docker version

If all goes well, doing docker version will yield the following.

docker_version. all good

docker_version. all good

Setting up Docker with Vagrant

The steps are based on John Zanchetta’s blog post, with all of the Vagrant steps trimmed and one network routing step found redundant. This is the tldr version and it assumes you are comfortable with Vagrant

Step 1 – Provision the Docker daemon box
Using the following Vagrant file, fire up the VM called docker

# -*- mode: ruby -*-
# vi: set ft=ruby :

VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
    config.vm.define :docker do |docker|
        docker.vm.box = "precise64"
        docker.vm.box_url = "http://files.vagrantup.com/precise64.box"
        docker.vm.network "forwarded_port", guest: 80, host:58080
        docker.vm.network "forwarded_port", guest: 4243, host: 4243
        $script = <<SCRIPT
wget -q -O - https://get.docker.io/gpg | apt-key add -
echo deb http://get.docker.io/ubuntu docker main > /etc/apt/sources.list.d/docker.list
apt-get update -qq 
apt-get install -q -y --force-yes lxc-docker
usermod -a -G docker vagrant
sed -e 's/DOCKER_OPTS=/DOCKER_OPTS=\"-H 0.0.0.0:4243\"/g' /etc/init/docker.conf > /vagrant/docker.conf.sed
cp /vagrant/docker.conf.sed /etc/init/docker.conf
rm -f /vagrant/docker.conf.sed
service docker restart
SCRIPT
       docker.vm.provision :shell, :inline => $script
    end
 end

Step 2 – Vagrant or boot2docker, an easy switch
Point the DOCKER_HOST to the Vagrant-based Docker daemon. Note that by simply changing the DOCKER_HOST, you can point your Docker client to whichever Docker daemon you so desire; boot2docker or Vagrant-based for example.

export DOCKER_HOST=tcp://localhost:4243
docker version

If all goes well, you are ready to build!

Step 3 – Build your first Docker container and mount volumes in Docker with Vagrant
Once you have a container built, mounting the /vagrant is no different from mounting any other directories to the container from the Docker daemon. For example

docker run -t -i --name test --volume /vagrant:/vagrant:rw example:test

Easy!

Step 4 – Make your first push, my only advice
This blog post would not be complete without recommending you to perform your first Docker push.

If like me, you have created your container with “example” as the repository, and “test” as the tag based on a Dockerfile

docker build -t example:test .

And upon pushing (assuming the username is “yves”), for example

docker login
docker push yves/example

You will notice the following response:

The push refers to a repository [yves/example] (len: 0)
2014/05/31 00:46:35 No such id: yves/example

This is because the push command needs username, which has to be part of the repository name. However, the above example, the repository name is merely just “example”.

The work around here, based on scouring issues #720, is to create a new tag that includes the username as part of the repository name. For example:

docker tag  yves/example:test_tag
docker push yves/example

then watch the magic unfold in front of your very eyes.

The big take away is to always prefix your repository name with your docker.io username, followed by a /. For example, yves/ubuntu, yves/centos yves/whatever.

My docker image, yves/yves, is built based on the Vagrant file mentioned in https://macyves.wordpress.com/2014/05/20/release-management-part1-test-automation-with-vagrant-and-jenkins-build-pipeline/

Why Vagrant over boot2docker?

At the time of writing, mounting directories from your OSX machine to a container hosted by boot2docker is quite a mind field. See issues #3957. boot2docker is very well supported, but I feel more at ease with Vagrant.

Side note

Docker is really resilient. Simply retry the same command when encountering failures during image building or pushing to docker.io. This is in part thanks to the untagged images that floats around Docker daemon and that an image is really a repository containing a bunch of changesets performed, akin to Git.

However, if you wish to clean up all untagged images, see this blog post.

Advertisements

siege and TCP TIME_WAIT

Note to self form installing siege via brew on OSX.

brew install siege

Thanks clever people in brew and siege! 🙂

Mac OS X has only 16K ports available that won't be released until socket
TIME_WAIT is passed. The default timeout for TIME_WAIT is 15 seconds.
Consider reducing in case of available port bottleneck.

You can check whether this is a problem with netstat:

    # sysctl net.inet.tcp.msl
    net.inet.tcp.msl: 15000

    # sudo sysctl -w net.inet.tcp.msl=1000
    net.inet.tcp.msl: 15000 -> 1000

Run siege.config to create the ~/.siegerc config file.

Dirrty Hax0r in Perl 5.12 Pid.pm for varnish-agent

Right, so I am not particularly proud of this but again, more of a note to self than anything. Secondly, I am most certainly not a Perl expert. This little hack should probably never be used or put into any production Mac setup. If you know of a proper fix, please do let me know.

Lastly, this is varnish-agent and not varnish-agent2. The former is written in Perl while the latter is written in C and uses Varnish API directly.

OK! Now that the disclaimer is out of the way, onward with the dodgy! So when running varnish-agent on OSX 10.8.2, one may encounter the following issue:

Can't kill a non-numeric process ID at /Library/Perl/5.12/File/Pid.pm line 124.

The quick fix is proposed here involves adding this line to the the Pid.pm file on line 124.

return undef unless $pid > 0;

So your subroutine in Pid.pm will end up looking more like this.

sub running {
   my $self = shift;
   my $pid  = $self->_get_pid_from_file;
   return undef unless $pid > 0;
   return   kill(0, $pid)
      ? $pid
      : undef;
}

libcurl and pkg-config on Homebrew – Mountain Lion OSX 10.8.2

OSX ships with curl and installing parallel versions via Homebew can cause all kinds of trouble.

Homebrew comes with the .pc meta file for libcurl in both Mountain Lion and Leopard and it is advisable to just reuse them where possible.

A really simple approach would be to include the Homebrew’s meta file as part of the search path for pkg-config. So in addition to the default searches in /usr/lib/pkgconfig, /usr/share/pkgconfig, /usr/local/lib/pkgconfig and /usr/local/share/pkgconfig, pkg-config will now include your new paths.

export PKG_CONFIG_PATH=/usr/local/Library/ENV/pkgconfig/mountain_lion/

UML CASE Tools for the Mac

Unified Modeling Language (UML) is a visual language for communicating software design, architectures, workflows etc.

Traditionally commercial Computer-aided software engineering (CASE) tools are abundant on the Windows platforms, with ArgoUML being the only open source UML tool back in day.

There are three categories of CASE tools. First up is the a graphical tool with stencils supporting UML syntax, a glorified Microsoft Paint if you will. The second is the traditional CASE tool with reverse engineering, round-trip development and team collaboration. Last but not least, CASE tools in the clouds. These are hosted UML tools for drawing and often have integration with wikis.

Selection criteria

The criteria for me for a CASE tool for the Mac are as follow:

  • Suitable for start up company, small team
  • Agile and fast learning curve
  • A desktop application
  • Simple to use
  • Includes the basic diagrams
  • Uses UML standards, not stencils
  • No need to use Wine, Cross-over, Parallel or VMware to run the tool

If I used Windows…

My CASE tool of choice traditionally is Enterprise Architect by Sparx System. It matches most of the criteria with added flexibility catering for larger teams, with model check-in and check-out, feature matrices, linking features to requirements to implementation and even to sprints. It is both an excellent Scrum tool and project management tool for large enterprise teams. The only downside to Enterprise Architect is that it runs in Windows and requires virtualisation to run on the Mac.

CASE tools summarised

Mac CASE tools I have tested out are Visual-Paradigm, Poseidon for UML, ArgoUML, Omnigraffle, and Astah community edition.

Visual-Paradigm, as the name suggests, is quite big on the visuals. It is a solid UML tool supporting wide variety of diagrams outside of UML. For example, Mindmap is supported by Visual-Paradigm.

Poseidon for UML is traditionally built on ArgoUML, the open-source UML project. It is clean, well supported, simple and quick to use. It follows the UML standards to its fullest and thus your models can be exported out to another tool such as Rational Rose or Enterprise.

ArgoUML is an open source UML project. It is also relatively simple to get going and it maintains the classic Java Swing look and feel.

Omnigraffle is similar to Visio – a graphical tool for drawing. UML is supported as stencils.

Astah is another UML tool, simple to get going and can be upgraded to a business edition for added features such as collaboration.

For a cloud-based solution, Gliffy is pretty decent. It came with our Atlassian OnDemand setup and it works beautifully with Confluence. A special mention to websequencediagram.com for providing such a cool idea.

And the winner is….

So which one to choose? For us, a start up company with small teams, I have chosen Astah community edition for its simplicity and cost with Poseidon coming in close second. Astah is fast and easy to get going. It can be extended to suit team collaboration if needed. If we ever become the next Google, I will consider Enterprise Architect again and live with VMware running Windows XP.

A newbie’s guide to Setting up a Java dev env on a mac

As a newly baptised MacBook Pro user, I was keen on getting started with Java development.

As a precursor, I have been a Java developer since 2003 and have used a MS environment all this time. I am an avid user of Linux but by no means an expert. This amounts to quite a lot of fiddling about when setting up a BSD environment for proficient software development. This Java Developer’s Environment guide will hopefully address my initial barrier and help other newbies there.

My dev env targets Java development with emphasis on following software development methodologies. The latter means that an UML, or any other graphical design language for that matter, ranks fairly high on my software design and implementation processes. My daily work evolves around web development, so setting up web servers or web containers are essential.

This guide targets Mac OSX 10.6.4.

1. Install XCode

XCode is the developer environment for Macs. It includes GNU packages like gcc, gdb and etc. Sign up to Apple and get XCode. This will install the essentials for iPhone development, ObjectC, libraries and IDEs.

http://developer.apple.com/

2. Install MacPorts

MacPorts is essential for installing additional packages like MySQL, Apache and etc onto a Mac. This application is dependent on XCode. As the name suggests, it is a ported version of these packages and it behaves similarly to apt-get.

MacPorts install applications in the /opt/local/ directory. It separates itself from the core BSD directory structures thus ensuring that packages installed using MacPorts do not overlap with existing executables.

http://www.macports.org/

Easiest way is to download the dmg package and then run the folowing command to update MacPorts to the latest version.

sudo port -v selfupdate

3. Install Source Control

I use SVN and Git for various different projects. I would recommend setting up both. Naturally CVS would be nice but it seems a bit out of fashion these days. For new projects, perhaps consider Git for the distributed source control.

svnX is a nice free GUI for SVN on a Mac. http://code.google.com/p/svnx/

Gitty is a free Git implementation for Macs. http://macendeavor.com/gity

4. Install IDE

Naturally this is entirely up to you. I switch between Netbeans and Eclipse with the latter being my main preference.

Netbeans 6.9.1 – http://netbeans.org/downloads/
Eclipse Hellios 3.6 – http://www.eclipse.org/downloads/

For ease of development, install plugins for SVN and Git on your IDE of choice. In addition, I usually install GWT plugins, Glassfish and other supporting packages that helps improve efficiency.

5.  Install Apache

The process of installing Apache is rather simple with MacPorts. However configuring it may require some finesse and knowledge. The steps are derived from Technie Corner [1] and MacNN [2]. Please note that Apache comes installed so feel free to use it, but feel free to install your very own using Apache. The following is from MaccNN [2]:

To install the latest version of Apache 2, type the following:

sudo port install apache2

When that is done, you have to create a conf file (which you can edit to configure Apache 2):

sudo cp /opt/local/apache2/conf/httpd.conf.sample /opt/local/apache2/conf/httpd.conf

5.1. Preserve Old Apache Directories

Please perform the following steps if you wish for your “Sites” directories (i.e. /Users/user/Sites) to be accessed viahttp://your-server/~user, as is the case with Apache 1.3).

Open /opt/local/apache2/conf/httpd.conf, and change:
DocumentRoot "/opt/local/apache2/htdocs"
to
DocumentRoot "/Library/WebServer/Documents"

and change
<Directory “/opt/local/apache2/htdocs”>
to
<Directory "/Library/WebServer/Documents">

Then, uncomment this line:

Include conf/extra/httpd-userdir.conf

5.2. Some Apache 2 extras

You may also want to uncomment the following lines in /opt/local/apache2/conf/httpd.conf:

Include conf/extra/httpd-autoindex.conf (Fancy directory listing)
Include conf/extra/httpd-default.conf (Some default settings)

5.3. Make Apache 2 access a little easier

Warning: Remember that OS X has a preinstalled version of Apache (1.3), which can turned on and off via “Personal Web Sharing” in Preferences, or via the “apachectl” command. The command points to that version of Apache, and not the one we are installing!

To make starting and stopping Apache 2 a little easier, we will construct an alias to Apache 2’s apachectl. Add the following line to ~/.profile (Panther/Tiger) and restart the terminal to gain access to it:

alias apache2ctl='sudo /opt/local/apache2/bin/apachectl'

5.4. Start Apache 2 on reboot

Warning: When I perform the command below, I eventually get the following error: Workaround Bonjour: Unknown error: 0. I’ve read that this is harmless, but I really don’t know for sure. I assume that everything ends up in proper order after a reboot.

To start Apache 2 now and whenever the system reboots, type the following:

sudo launchctl load -w /Library/LaunchDaemons/org.macports.apache2.plist

5.5. Notes

We added “sudo” to the alias apache2ctl, so if prompted for a password, enter your root password.

To run Apache 2: apache2ctl start
To stop Apache 2: apache2ctl stop

6. Install MySQL

MySQL is installed using MacPorts. The following is from MaccNN [2]:

Note: MySQL has a very nice binary installation of MySQL, so feel free to go ahead and use that instead of a port. I, however, like having my entire system (Apache 2, PHP 5, MySQL 5) installed together in the same fashion.

To install MySQL 5, we will first get the port. If you want MySQL 5 to run automatically after system startup, perform the following commands:

Warning: When I perform the launchctl command below, I eventually get the following error: Workaround Bonjour: Unknown error: 0. I’ve read that this is harmless, but I really don’t know for sure. I assume that everything ends up in proper order after a reboot.

sudo port install mysql5 +server
sudo launchctl load -w /Library/LaunchDaemons/org.macports.mysql5.plist

Otherwise, run the following:

sudo port install mysql5

6.1. Make MySQL 5 access a little easier

When done, you may want to add the following lines to ~/.profile (Panther/Tiger) and restart the Terminal to gain access to them:

Warning: You will need to type sudo -v before using mysqlstart, to avoid being prompted for a password in the background (thus preventing MySQL from starting).

alias mysqlstart='sudo mysqld_safe5 &'
alias mysqlstop='mysqladmin5 -u root -p shutdown'

The “-p” assumes that you will be using a password for root, which you of course should be.

6.2. Set up DB

Then, run the following to set up the database:

sudo -u mysql mysql_install_db5

You will now be able to use MySQL 5 (and you can use mysqlstart and mysqlstop for convenience).

I also recommend immediately adding a password to your root account:

mysqladmin -u root password [yourpw]

7. Install PHP

Again, using MacPort, PHP is supported with the aforementioned Apache. The following is from MaccNN [2]:

(Make sure Apache 2 is stopped before proceeding.)

To install the latest version of PHP 5, type either:

sudo port install php5 +apache2 +mysql5 +pear (With PEAR) or sudo port install php5 +apache2 +mysql5 (Without PEAR)

When that is done, register PHP 5 with Apache 2:

cd /opt/local/apache2/modules
sudo /opt/local/apache2/bin/apxs -a -e -n "php5" libphp5.so

And create a php.ini file (which you can edit to configure PHP 5):

cp /opt/local/etc/php.ini-dist /opt/local/etc/php.ini

Then open /opt/local/apache2/conf/httpd.conf again, and add this line:

Include conf/extras-conf/*.conf

You will probably want to register index.php with Apache 2 as a directory index page (automatically loaded as the directory index). If you would like to do this, also change:

DirectoryIndex index.html to DirectoryIndex index.html index.php

8. Install phpMyAdmin

This package is really easy to use for manipulating and managing MySQL databases. The following is from MaccNN [2]:

The port is not updated regularly, and installation is ridiculously easy for phpMyAdmin, so download directly from

http://www.phpmyadmin.net/

Next, untar the downloaded file. Rename the directory to “phpMyAdmin” and place it in your document root (/Library/WebServer/Documents if you have preserved the old Apache directories).

Create a php file called config.inc.php and place it in the root of the phpMyAdmin folder. This file will overwrite settings defined by phpMyAdmin/libraries/config.default.php. The following is my config.inc.php file. Some of the lines are not completely necessary, but I like readability and consistency:


<?php
$cfg['blowfish_secret'] = ‘[A passphrase for internal use]‘;
$i = 0; $i++;
$cfg['Servers'][$i]['host'] = ‘localhost’;
$cfg['Servers'][$i]['connect_type'] = ‘tcp’;
$cfg['Servers'][$i]['extension'] = ‘mysqli’;
$cfg['Servers'][$i]['auth_type'] = ‘cookie’;
$cfg['Servers'][$i]['user'] = ”;
$cfg['Servers'][$i]['password'] = ”;
?>

That’s it! You should now be able to access it at http://your-server/phpMyAdmin. You can log in as root, add more users, create databases, etc.

9. Installing Enterprise Architect

Though this is entirely debatable, but enterprise UML tools are often only Windows based. Its nice to have StarUML or ArgoUML, but it really lacks the project maangement and requirement analysis functionalities. My preference is Enterprise Architect from Sparx Systems.

In order to run Windows applications correctly on Mac, we need a virtualisation software. Parellel and VMWare is fine but I believe it requires Windows installations, which can be costly. I recommend the following:

Winebottler – http://winebottler.kronenberg.org/
CrossOver – http://www.codeweavers.com/products/

Winebottler is free and is supported by the open-source community. CrossOver is proprietary. Currently I prefer CrossOver due to some display bug that Winebottler has when running Enterprise Architect.

10. That’s it!

Thats pretty much it to get coding right off the bat.

Reference

[1] Techie Corner, available at http://www.techiecorner.com/174/how-to-install-apache-php-mysql-with-macport-in-mac-os-x/

[2] MacNN Forum, available at http://forums.macnn.com/79/developer-center/322362/tutorial-installing-apache-2-php-5-a/