Random links from Operability.IO conference Part 1

These are my random links from the excellent 2 day conference Operability.IO. These are all links mentioned in different talks.

Worse is Better
Much to the shock of my collages I had not read about this before. Now I have!

OSV / Capstan
I’ve been using hyper.sh which is very similar except it uses Docker images inside the hypervisor. OSV looks very interesting especially the REST API for inspecting a running image.

This is a unikernel for compiling OCaml to Xen hypervisor

Google Borg Paper
Interesting paper but I’m not working in this space so hard to appreciate all the details.

Notes on Distributed Systems for Young Bloods
Lots of great bits of advice. Question: When would knowing the 50th percentile ever be valuable?

Common Ground and Coordination in Joint Activity

Engineering for the Long Game – Astrid Atkinson

Antifragile – Things That Gain From Disorder

How to connect to CloudFoundry’s Loggregator directly via WebSockets

Firstly make sure you can run the standard CloudFoundry cli:

cf logs [APP_NAME]

Then turn on tracing:

export CF_TRACE=true

You will see a HTTP request and then it will switch to a secure WebSocket:

WEBSOCKET REQUEST: [2015-04-22T11:27:22+01:00]
GET /tail/?app=[APP_GUID] HTTP/1.1
Host: wss://loggregator.[YOUR_CF_DOMAIN]:443
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Version: 13
Sec-WebSocket-Key: [HIDDEN]
Origin: http://localhost
Authorization: [PRIVATE DATA HIDDEN]

As you can see the Authorisation header is hidden, but fear not if look in ~/.cf/config.json under “AccessToken” thats all you need ("jq .AccessToken ~/.cf/config.json"). Combine this with wscat and you are good to go:

wscat -c wss://loggregator.[YOUR_CF_DOMAIN]:443/tail/?app=[APP_GUID] -H Authorization:$(jq .AccessToken ~/.cf/config.json)

Intel Graphics Installer on Ubuntu 14.10

So if you just downloaded the latest Intel Graphics Installer and just found that it doesn’t support Ubuntu 14.10.

Fear not you can trick it into installing by doing the following:

First backup

sudo cp /etc/lsb-release /etc/lsb-release.backup

Then edit the file

sudo nano /etc/lsb-release

And put the following in there


This will allow the installer to proceed but you will also want to add the public key so updates work correctly:

wget --no-check-certificate https://download.01.org/gfx/RPM-GPG-KEY-ilg -O - | sudo apt-key add -
wget --no-check-certificate https://download.01.org/gfx/RPM-GPG-KEY-ilg-2 -O - | sudo apt-key add - 

Original articles:
“Distribution not supported” when trying to install Intel Graphics Installer in 14.10
Intel Linux Graphic Drivers

Updated power savings settings for Samsung Series 9 (NP900X3C-A05UK) and Ubuntu 14.10

Here is my updated /etc/rc.local for Ubuntu 14.10

#!/bin/sh -e

# Sleep so all services have started before we change settings
sleep 5

# Set Intel Audio to power save 
echo '1' > '/sys/module/snd_hda_intel/parameters/power_save';

# Temp disable ethernet port
modprobe -r r8169

# Wireless Power Saving for interface wlan0
iw dev wlan0 set power_save on

# VM writeback timeout
echo '1500' > '/proc/sys/vm/dirty_writeback_centisecs';

# Temp disable bluetooth
modprobe -r btusb

# Adjust backlight to start much lower
echo 800 > '/sys/class/backlight/intel_backlight/brightness'

# - NMI Watchdog (turned off)
echo 0 > '/proc/sys/kernel/nmi_watchdog';

# - SATA Active Link Power management
for i in `find /sys/class/scsi_host/*/link_power_management_policy`; do echo 'min_power' > $i; done;

# - USB Autosuspend (after 2 secs of inactivity)
for i in `find /sys/bus/usb/devices/*/power/control`; do echo auto > $i; done;
for i in `find /sys/bus/usb/devices/*/power/autosuspend`; do echo 2 > $i; done;

# - Device Power Management
echo 'auto' | tee /sys/bus/i2c/devices/*/power/control > /dev/null;
echo 'auto' | tee /sys/bus/pci/devices/*/power/control > /dev/null;

# - CPU Scaling (power saving scaling governor for all CPU's
for i in `find /sys/devices/system/cpu/*/cpufreq/scaling_governor`; do echo 'powersave' > $i; done;

exit 0

Getting the generic signature of a Java 8 lambda

Posted to StackOverflow

This is currently possible to solve but only in a pretty hackie way, but let me first explain a few things:

When you write a lambda, the compiler inserts a dynamic invoke instruction pointing to the LambdaMetafactory and a private static synthetic method with the body of the lambda. The synthetic method and the method handle in the constant pool both contain the generic type (if the lambda uses the type or is explicit as in your examples).

Now at runtime the LambdaMetaFactory is called and a class is generated using ASM that implements the functional interface and the body of the method then calls the private static method with any arguments passed. It is then injected into the original class using Unsafe.defineAnonymousClass
(see John Rose post) so it can access the private members etc.

Unfortunately the generated Class does not store the generic signatures (it could) so you can’t use the usual reflection methods that allow you to get around erasure

For a normal Class you could inspect the bytecode using Class.getResource(ClassName + “.class”)
but for anonymous classes defined using Unsafe you are out of luck. However you can make the LambdaMetaFactory
dump them out with the JVM argument:

java -Djdk.internal.lambda.dumpProxyClasses=/some/folder

By looking at the dumped class file (using javap -p -s -v), one can see that it does indeed call the static method. But the problem remains how to get the bytecode from within Java itself.

This unfortunately is where it gets hackie:

Using reflection we can call Class.getConstantPool and then access the MethodRefInfo to get the type descriptors. When can then use ASM to parse this and return the argument types. Putting it all together:

Method getConstantPool = Class.class.getDeclaredMethod("getConstantPool");
ConstantPool constantPool = (ConstantPool) getConstantPool.invoke(lambda.getClass());
String[] methodRefInfo = constantPool.getMemberRefInfoAt(constantPool.size() - 2);

int argumentIndex = 0;
String argumentType = jdk.internal.org.objectweb.asm.Type.getArgumentTypes(methodRef[2])[argumentIndex].getClassName();
Class<?> type = (Class<?>) Class.forName(argumentType);

Now ideally the classes generated by LambdaMetaFactory should store the generic type signatures (I might see if I can submit a patch to the OpenJDK) but currently this is the best we can do. The code above has the following problems:

  • It uses undocumented methods and classes
  • It is extremely vulnerable to code changes in the JDK
  • It doesn’t preserve the generic types, so if you pass List into a lambda it will come out as List

Where did I download that file from?

Assuming you used Chrome and a modern Linux file system…

$ attr -g xdg.origin.url Downloads/google-chrome-stable_current_x86_64.rpm 
Attribute "xdg.origin.url" had a 74 byte value for /home/dan/Downloads/google-chrome-stable_current_x86_64.rpm:

Bash function to switch Java versions

setjava() {
	if [ "$1" = "-q" ]; then
                local quiet=true

	local jdk=~/Applications/Java/jdk1.$1
        if [ ! -d "${jdk}" ]; then
                echo Jdk not found: ${jdk}
                return 1
	export JAVA_HOME=${jdk}
        export PATH=${JAVA_HOME}/bin:${PATH}
        if [ -z "${quiet}" ]; then
                java -version

export -f setjava

I have symlinks for all major versions of Java so that in IntelliJ and the command line I can upgrade minor Java versions just by changing the symlink:

$ ls -la ~/Applications/Java/
total 20
drwxrwxr-x. 5 dan dan 4096 Mar 27 10:12 .
drwxrwxr-x. 8 dan dan 4096 Mar 27 10:13 ..
lrwxrwxrwx. 1 dan dan   11 Mar 27 10:12 jdk1.6 -> jdk1.6.0_45
drwxr-xr-x. 8 dan dan 4096 Mar 26  2013 jdk1.6.0_45
lrwxrwxrwx. 1 dan dan   11 Mar 18 13:55 jdk1.7 -> jdk1.7.0_51
drwxr-xr-x. 8 dan dan 4096 Dec 19 03:24 jdk1.7.0_51
lrwxrwxrwx. 1 dan dan    8 Mar 27 10:12 jdk1.8 -> jdk1.8.0
drwxr-xr-x. 8 dan dan 4096 Mar  4 11:18 jdk1.8.0

Then I can switch version by either just saying the major version:

$ setjava 8
java version "1.8.0"
Java(TM) SE Runtime Environment (build 1.8.0-b132)
Java HotSpot(TM) 64-Bit Server VM (build 25.0-b70, mixed mode)

or I can be specific about minor version:

$ setjava 6.0_45
java version "1.6.0_45"
Java(TM) SE Runtime Environment (build 1.6.0_45-b06)
Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode)

I’ve also started adding the following to my build and execution scripts so they can specify the Java version themselves:

type -t setjava > /dev/null && setjava -q 8 || if [ -n "${JAVA_HOME}" ]; then PATH=${JAVA_HOME}/bin:${PATH}; fi

Currently this is designed to be optional and will fallback to JAVA_HOME etc.

Dan's technical ramblings