Encrypting data with Chef

31-Aug, 2014

I know that this may be covered elsewhere, but I struggled to find a single resource that detailed clearly how to do this. What I can work out is that there appeared to be a number of ways to do this but I detail the one that worked for me.

The first step in encrypting your data is to create an encryption key:

openssl rand -base64 4096 | tr -d '\r\n' > encrypted_data_bag_secret

I placed this encryption key in the root of my Chef repository, but in truth it can be stored anywhere.

The next step is to create the data to encrypt. This is a json file in a similar format to the following:

    "id": "mysql",
    "username": "jira",
    "password": "password"

The data can be any data that is in a json format. To encrypt the data and load it into Chef run the following command:

knife data bag from file passwords data_bags/mysql.json --secret-file encrypted_data_bag_secret

In the above command the “data_bags/mysql.json” is the json file created above and the encrypted_data_bag_secret is the encryption key.

Successful encryption can be shown by running the following command:

knife data bag show passwords mysql

Which should show something similar to the following:

Chef Encrypted Databag

Obviously encrypted databags aren’t that useful if we can’t decrypt it, in order to decrypt it the key needs to be added to the above command:

knife data bag show --secret-file encrypted_data_bag_secret passwords mysql

And it should output the following:

id:       mysql
password: password
username: jira

Encrypted content can also be viewed via the Chef UI:

Chef UI Encrypted Databag

Lastly, the host itself needs to be able to access the decrypted contents. This is done by dropping the encryption key onto the host and then access the data bag as follows:

mysql_secret = Chef::EncryptedDataBagItem.load_secret("/etc/.encrypt_key")
mysql_creds = Chef::EncryptedDataBagItem.load("passwords", "mysql", mysql_secret)
username = mysql_creds["username"]
password = mysql_creds["password"]
file "/tmp/output.txt" do
    content username + ":" + password
    mode 0600

The contents of the file is as follows:

[root@jboss01 tmp]# cat output.txt 

And password encryption with Chef works!


Bootstrapping OSX with Chef

04-Aug, 2014

I have already moved much of my development infrastructure to Chef and thought that I would save a lot of time and learn more about Chef by moving my workstation configuration to Chef as well. I run a MacBook Pro and in order to bootstrap it you run the following command:

knife bootstrap titanium.justnudge.com -x mransley --sudo

You need to use the sudo command together with the username (-x) because Mac OSX doesn’t really use the root account that much.

titanium:Desktop mransley$ knife bootstrap titanium.justnudge.com -x mransley --sudo
Connecting to titanium.justnudge.com
titanium.justnudge.com Starting first Chef Client run...
titanium.justnudge.com Starting Chef Client, version 11.12.8
titanium.justnudge.com Creating a new client identity for titanium.justnudge.com using the validator key.
titanium.justnudge.com resolving cookbooks for run list: []
titanium.justnudge.com Synchronizing Cookbooks:
titanium.justnudge.com Compiling Cookbooks...
titanium.justnudge.com [2014-08-04T16:21:36+10:00] WARN: Node titanium.justnudge.com has an empty run list.
titanium.justnudge.com Converging 0 resources
titanium.justnudge.com Running handlers:
titanium.justnudge.com Running handlers complete
titanium.justnudge.com Chef Client finished, 0/0 resources updated in 9.399649 seconds
titanium:Desktop mransley$

And the OSX host is successfully bootstrapped.


Querying JVM memory with JMX

30-Jun, 2014

Monitoring memory utilisation of JMX is an important part of monitoring runtime infrastructure. There are a number of methods for achieving this such as:

  • Parsing the verbose GC logs.
  • Querying the JVM using remote JMX calls.
  • Querying the JVM using local JMX calls outside of the running JVM.

The first gives you great information including timings for the GC runs but can be a little bit verbose (minor issue I know). The second (remote JMX) provides some issues in terms of security, either you leave it off and it is insecure or you turn it off and it becomes overly complex (firewalls, SSL and usernames and passwords). The last option has the following good points:

  • Does not need an agent installed into the JVM.
  • Can query the JVM while it is running.
  • Very light weight.

We are triggering this using NRPE calls and using Nagios as the triggering mechanism. To make the connection to the JVM you need to be running as the user that is running the VM (i.e. if you are running the VM as tomcat you must make the call as the tomcat user, you cannot use another user or root or it will fail). The process for connecting to a JVM is as follows:

VirtualMachine vm = VirtualMachine.attach(Integer.toString(processID));
String address = vm.getAgentProperties().getProperty(CONNECTOR_ADDRESS);

Now, if the connector address has never been loaded then it will return null, to load it run the following code:

String javaHome = (String) vm.getSystemProperties().get("java.home");
String agent = javaHome + "/lib/management-agent.jar";

The address can then be gathered by re-running the vm.getAgentProperties().getProperty(CONNECTOR_ADDRESS); again.

Now that you have a VM connection you can run JMX queries to get runtime information from the system:

List<MemoryPoolMXBean> list = ManagementFactory.getPlatformMXBeans(connection, MemoryPoolMXBean.class);

From here you can query much of the JVM’s runtime settings.

I hope that this helps.


More Posts..