Joe Ray

Running a Game Day

Sat, 29 May 2021 00:00:00 UTC

Recently at Kooth we ran a game day. We assembled all the engineers, and some of the product team, to practice what we would do in case of a production incident and learn how we and our systems might respond.

There are a few reasons we wanted to do this:

Not everyone has experience dealing with incidents and we want to ensure that all our engineers feel confident responding to incidents themselves.
Everyone can improve how they handle incidents, especially understanding how best to work collaboratively to solve them.
It allowed us to explore our systems and better understand how they function, as well as how they can fail, in a safe environment.
We wanted to see if we could come up with user stories or processes to mitigate failure that we hadn’t considered.
We haven’t yet had to deal with a major security incident—thankfully!—but we wanted to be more prepared should that happen.
It gave us an opportunity to work with people we wouldn’t normally work with and form better bonds.

How did we plan?

Three of us volunteered to plan and run the game day and we started by figuring out what we wanted to achieve from the day. Since very few of us had experience with participating in a game day we decided that we would start by coming up with a set of scenarios we could control, rather than doing something like red team/blue team.

We came up with a list of scenarios we could run that would help achieve one or more of our goals. We wanted to ensure that people could practice incident response collaboratively and in as real a way as possible. Examples included:

A data breach is discovered.
A third party provider we heavily rely on is down.
One of our databases is unavailable.

We also thought about who would be participating and split people up into teams. We tried to spread out people with more experience as well as splitting up people who normally work together.

One question we discussed was how we would run the scenarios: what environment should we use? Should we resize it to production scale? Should we try and simulate production levels of traffic? Should each team do the same scenario or each take a different one?

In the end, we decided to keep it as simple as possible and just use one of our non-production environments without any modifications. We figured that if it was successful, we could do something more complex at a future game day. We also decided to allocate each team a different scenario, partly because some of the scenarios might result in teams interfering with each other and partly because we felt we would learn more that way.

Finally, we discussed how to structure the day. We wanted to make sure we left time for teams to showcase what they’d learned, as well as hold a retrospective at the end of the day on how the day went.

What did we learn?

Overall the day was a great success! People had a lot of fun exploring the scenarios and we gained plenty of useful insights.

The biggest thing that came out of the retro was that the day had highlighted people’s knowledge gaps which will allow us to focus training and future exercises.

We also learned about where our processes, documentations and systems are lacking and now have ideas of how to improve them.

Finally, people really liked spending time working with people from outside their teams. This sparked a lengthy discussion about team rotations.

Remote working: tools and tips

Sat, 02 Feb 2019 00:00:00 UTC

Over the past six months or so, I’ve been working in a team that’s almost entirely remote. Working in a team that is at least somewhat remote seems to be an increasingly common pattern in ThoughtWorks so it’s something that’s being discussed more.

There’s been a lot of useful discussion on how to make it successful. Here are some of the highlights.

Meetings

Retros/Workshops

Cross-government design systems workshop at GDS, December 2017 CC By 2.0

We love using stickies in meetings at ThoughtWorks and you don’t have to go far on the ThoughtWorks website to find this out!

There are some tools which attempt to recreate this digitally.

funretro.io and Ideaboardz: Used by lots of teams to facilitate remote retros.
RealTimeBoard: Recommended by many teams, allows more flexibility than tools dedicated to retros, for example to draw circles around groups of stickies etc.
Mural: An alternative to RealTimeBoard.
Google Docs/Sheets: My team uses Google Docs to facilitate retros at the moment which works pretty well. Andy Yates also suggested using Google Sheets for workshops where each cell could represent a place to put a sticky. We tried Google Draw but it was too clunky to be useful in the context of a retro.

Diagramming

Jennifer drawing ideas CC By 2.0

You don’t have access to a whiteboard when you’re all remote so here are some suggestions for alternatives:

Some video conferencing software (e.g. Zoom) allows drawing on shared screens for quick collaborations.
Jim Barritt had success sharing Omnigraffle over video conferencing software when needing to explain things.

Audio/Visual

Accessibility

Some tips were about how to make remote working accessible to those with hearing impairments but really, given how difficult it can sometimes be to follow what people are saying on video conferencing calls, these apply to anyone:

Don’t allow more than one person to talk at once.
Encourage people to screen share and use diagrams, charts, documents etc. to back up what they are saying.
Ask people to send links to supporting materials (diagrams, docs, etc.) in advance of the meeting and then refer to them during the call.
Ava and Google Docs text-to-speech can be used for doing text-to-speech in real-time, though experiences with this were mixed.
Take minutes for meetings (using a Google Doc means it can be real-time and people can follow along).

Audio

It was universally agreed that good headsets are a must. As Korny Sietsma put it, “I think having a boom mic is quite critical - no amount of smart technology can quite compare with just putting a small microphone quite close to your mouth.”

Some recommendations were:

Jabra UC Voice 550 Duo.
Gaming headsets, e.g. HyperX HX-HSCS-BK/EM.
- Often these will have features such as raising the mic to mute.
Jabra speakers are good when you have some members of the team clustered around the same laptop, but only when you’re in a quiet room.
Directional (karaoke-style) mics were recommended by one colleague for groups of people as it forces people to speak one at a time and only the person with the mic can speak.
krisp.ai: Some reported installing this improved sound quality, but someone else found they experienced a lot of cut-outs when they did and had to revert.
Push-to-talk features of video conferencing software are useful to prevent background noise from leaking in when you’re not talking, though as Korny said, “you’re on mute” must be the most common phrase in video conferencing meetings. Shush provides the same functionality on a Mac but system-wide.

Pairing

wocintech (microsoft) - 62 CC By 2.0

I love pairing but doing it remotely can be a challenge. There are a couple of tools we’ve come across to help:

Slack has good screen-sharing capabilities and allows others to control/draw on your screen.
Zoom/other video conferencing software can also be used for this.
Some were suggesting using the collaborative features in Visual Studio Code.
For remote pairing in a terminal environment, tmux can be used to allow two or more people to connect to the same session. tmate helps with the setup of this.

Dealing with Problems

Technical problems can really hamper remote teams.

One important point, highlighted by Clare Sudbery, was that “people will just put up with remote communication problems when they shouldn’t, so anticipate that and be deliberate and proactive about addressing issues with hardware, software and connectivity. Put time aside to triage any problems as a matter of urgency and be prepared to be quite pushy about demanding investment and support in decent tools to facilitate remote meetings.”
If you do encounter problems, having an alternative channel such as the chat window in your video conferencing software, the team Slack or a Google Doc, can help mitigate them. (See also the Accessibility section above.)

Culture

The team culture is really key in making a remote team successful. There are a number of aspects to this:

If even one person is remote, everyone should be remote (use headsets, even be in separate rooms).
Invest in building relationships with remote team members:
- Make sure that the team meets regularly in person.
- Try and also meet in person when new people join/at the start of the project.
- ‘Water-cooler’ conversations have to happen intentionally. You could e.g. have a rule where the first ten minutes of every meeting is officially set aside for non-meeting-related chat.
- Check in and out in your team chat when you arrive at/leave work.
Where a significant part of the team is co-located, or there are multiple teams in different locations, you could try having big TVs set up by the team(s) with webcams and mics. This allows you to see the other groups of people working, and if you have an impromptu question you can walk up and use the mic to get the other location’s attention.

Dynamically loading classes from modules in Python

Fri, 20 Jan 2017 00:00:00 UTC

I was working on a bit of code today where I wanted to be able to dynamically load a class from a module that’s specified by some external configuration.

Let’s say you have a package called widgets and within that, a series of modules (e.g. mail, sms, irc etc.) that contain classes which all inherit from the superclass WidgetAPI.

To load the widgets.mail.WidgetAPI class when given the string 'mail' by the user, you could do:

import inspect
import importlib

from widgets import WidgetAPI

def find_widget(widget_type):
    try:
        module = importlib.import_module('widgets.{0}'.format(widget_type))
        for x in dir(module):
            obj = getattr(module, x)

            if inspect.isclass(obj) and issubclass(obj, WidgetAPI) and obj is not WidgetAPI:
                return obj
    except ImportError:
        return None

if __name__ == '__main__':
    print(find_widget('mail'))
    print(find_widget('blah'))

This will produce the output:

<class 'widgets.mail.MailWidget'>
None

PGP Keys with a Yubikey and GPGTools

Tue, 05 Jul 2016 00:00:00 UTC

I finally got around to getting a Yubikey NEO last week and along with setting it up for 2FA for some of my online accounts, I took the opportunity to generate a new PGP key.

I mostly followed Simon Josefsson’s excellent post on the subject and ended up with a new private key and set of subkeys.

The one snag I had was getting it to work with GPGTools for Mac OSX – I kept getting the message “card error” when trying to access it. I upgraded to the latest version (with GPG 2.0.30) today and it seemed to work, though I also noticed you can’t use the Yubico Authenticator at the same time as using the GPG tool so that may have explained my problems.

My new key ID is 804EFECC, a copy of which can be found here and on Keybase.io.

The environmental case for eating less meat

Wed, 29 Jun 2016 00:00:00 UTC

Unless there is a "radical shift in meat and dairy consumption", it's unlikely we'll be able to keep global temperature rises to two degrees Celsius.

The world eats a lot of meat and we’re eating more and more each year. The supply of meat per person across the globe almost doubled in the 50 years between 1961 and 2011.

Growth in countries like China is even more pronounced – in 2011 the supply of meat to China was almost 16 times as much per person as in 1961. Even in countries such as the US and UK, supply has increased.

Fig. 1: Meat supply per person 1961-2011

Source: Meat supply data from Food and Agriculture Organization of the United States and population data from The World Bank

The problems with meat

The problem with meat production is that it’s so inefficient: livestock consume 4.6 billion tonnes of carbon per year in feed but the produce (meat and dairy) ends up as only 2.6% of that, even before losses are taken into account.¹

What’s more, agriculture represents a significant proportion of global emissions, exceeding those from the transport sector. Experts estimate that the total global greenhouse gas emissions for livestock alone account for at least 14.5%,² and possibly up to 51%.³

And one paper predicts that if current population and consumption trends continue, greenhouse gas emissions for livestock would increase by around 77%.⁴

Gideon Mendel/WSPA

With meat production being so inefficient, huge amounts of land are given over just to feeding animals – 75% of agricultural land and 23% of arable land worldwide⁵ – and increasing demand means that cattle farming also plays a major role in deforestation, both directly and indirectly.

In 2011, 90% of deforestation in Columbia was caused by cattle farming.⁶ 75% of soy production, also a major contributor to deforestation, goes to feeding animals.⁷ Not only does deforestation destroy habitats and reduce biodiversity, it also destroys a valuable source of carbon storage and is a primary cause of carbon emissions in the agricultural sector.⁸

How will stopping eating meat help?

In principle, if everyone moved to a purely vegan diet we could feed 4 billion extra people.⁹ A reduction in the production of meat will also result in there being more land available for other uses – planting forests, bioenergy etc.

But perhaps most importantly, a 2014 report states that unless there is a “radical shift in meat and dairy consumption”, it’s unlikely we’ll be able to keep global temperature rises to two degrees Celsius.¹⁰

Not all land that is used for grazing would be suitable for growing crops – about 14% of feed by carbon mass comes from land like this – but its use for grazing “can still have negative consequences for carbon storage and biodiversity.”¹¹

However, it is difficult to envisage a transition to entirely plant-based diets in all areas of the globe: many regions rely on livestock for food security as they can be fed on plants that aren’t suitable for human consumption or graze on land that couldn’t support crops.¹²

What’s your impact?

By transitioning from a high-meat to a vegetarian diet, your carbon emissions could decrease by about 3.38 kgCO2e per day (that’s 47% less). That’s roughly the equivalent of eating 3,084 loaves of bread per year.

If everyone in the UK aged 15-64 did the same thing, we could save about 38,605,845 tonnes of CO2e per year. That’s roughly the equivalent of taking 8,392,575 return flights from the UK to Hong Kong.

And don’t forget this just shows the estimated potential carbon savings, without taking into account the extra benefits that come with more land being available.

Explaining the diet calculations

A high-meat diet in the UK equates to 7.19 kgCO2e per day, or 2,624.35 kgCO2e per year. A vegetarian diet in the UK equates to 3.81 kgCO2e per day, or 1,391 kgCO2e per year.¹³
As a comparison, loaves of bread equates to about 0.40 kgCO2e, and 1 return flight from the UK to Hong Kong equates to about 4.60 tonnes CO2e.¹⁴
There are an estimated 41,972,000 people (UK) in the 15-64 age bracket as of 2016.¹⁵

Bajželj, Bojana, Keith S. Richards, Julian M. Allwood, Pete Smith, John S. Dennis, Elizabeth Curmi, and Christopher A. Gilligan. 2014. Importance Of Food-Demand Management For Climate Mitigation pp. 925-926 ↩︎
Bailey, Rob, Antony Frogatt, and Laura Wellesley. 2014. Livestock – Climate Change’s Forgotten Sector ↩︎
Anhang, Jeff, Robert Goodland. 2009. Livestock and Climate Change: What if the key actors in climate change are…cows, pigs, and chickens? ↩︎
Bajželj et al. 2014. pp. 925-926 ↩︎
Bailey et al. 2014. ↩︎
Kissinger, Gabrielle, Martin Herold, Veronique De Sy. 2012. Drivers of Deforestation and Forest Degradation ↩︎
WWF. Soy is everywhere ↩︎
“The AFOLU sector is responsible for just under a quarter (~10–12 GtCO2eq/yr) of anthropogenic GHG emissions mainly from deforestation and agricultural emissions from livestock, soil and nutrient management.” IPCC. 2014. Climate Change 2014: Agriculture, Forestry and Other Land Use p816 ↩︎
Ravilios, Kate. 2014. Cutting meat consumption could feed 10 billion ↩︎
Bailey et al. 2014. ↩︎
Bajželj et al. 2014. ↩︎
“livestock can be fed on plants not suitable for human consumption or growing on land with high soil carbon stocks not suitable for cropping; hence, food production by grazing animals contributes to food security in many regions of the world.” IPCC. 2014. Climate Change 2014: Agriculture, Forestry and Other Land Use p838 ↩︎
Scarborough, Peter, Paul N. Appleby, Anja Mizdrak, Adam D. M. Briggs, Ruth C. Travis, Kathryn E. Bradbury, and Timothy J. Key. 2014. Dietary greenhouse gas emissions of meat-eaters, fish-eaters, vegetarians and vegans in the UK ↩︎
Berners-Lee, Mike. 2010. How Bad Are Bananas?: The Carbon Footprint of Everything ↩︎
World Bank. World DataBank ↩︎

GNU Parallel or How to list millions of S3 objects

Tue, 24 May 2016 00:00:00 UTC

GNU Parallel is a great tool for parallelising command line tasks and the AWS CLI is a great tool for interacting with S3.

Put them together and you have everything you might need to list millions of objects in an S3 bucket efficiently.

The problem: Listing a lot of objects in an S3 bucket takes a seriously long time when using aws s3 ls

The solution: GNU parallel can parallelise the process

Let’s say your bucket contains a set of log files named by date and you’ve followed the S3 performance tips by prefixing each key with a hexadecimal character.

We can use parallel to split up the object listing into separate tasks which can then run simultaneously:

$ parallel aws s3api list-objects \
    --bucket my-bucket \
    --prefix {1} \
    ::: {a..f} {0..9}

What we’re doing here is performing 16 list-objects commands, one for each of the hexadecimal digits. s3api list-objects is more efficient than s3 ls as it just does the raw API call and returns the result to you. The {1} is a placeholder for each of the digits and we generate the digits using brace expansion. We tell parallel that the hexadecimal digits are arguments using the ::: syntax.

If we had a two-character prefix, we could further parallelise the work by using the following command:

$ parallel aws s3api list-objects \
    --bucket my-bucket \
    --prefix {1}{2} \
    ::: {a..f} {0..9} \
    ::: {a..f} {0..9}

This uses the same technique as before but will now parallelise all combinations of the hexadecimal digits, producing commands like this:

aws s3api list-objects --bucket my-bucket --prefix ab
aws s3api list-objects --bucket my-bucket --prefix ac
aws s3api list-objects --bucket my-bucket --prefix ad
aws s3api list-objects --bucket my-bucket --prefix ae
aws s3api list-objects --bucket my-bucket --prefix af
aws s3api list-objects --bucket my-bucket --prefix a0
aws s3api list-objects --bucket my-bucket --prefix a1
aws s3api list-objects --bucket my-bucket --prefix a2
aws s3api list-objects --bucket my-bucket --prefix a3
aws s3api list-objects --bucket my-bucket --prefix a4
aws s3api list-objects --bucket my-bucket --prefix a5
aws s3api list-objects --bucket my-bucket --prefix a6
aws s3api list-objects --bucket my-bucket --prefix a7
aws s3api list-objects --bucket my-bucket --prefix a8
aws s3api list-objects --bucket my-bucket --prefix a9
aws s3api list-objects --bucket my-bucket --prefix ba
aws s3api list-objects --bucket my-bucket --prefix bb
...

You could combine this technique with the jq and awk commands to count the number of objects and sum the total size:

$ parallel aws s3api list-objects \
    --bucket my-bucket \
    --prefix {1}{2} \
    ::: {a..f} {0..9} \
    ::: {a..f} {0..9} | \
  jq '.Contents[].Size' --raw-output | \
  awk '{ count += 1; size += $1 } END
       { print "Count:", count, "Size:", size }'

Or use parallel again to copy the objects locally:

parallel aws s3api list-objects \
    --bucket my-bucket \
    --prefix {1}{2} \
    ::: {a..f} {0..9} \
    ::: {a..f} {0..9} | \
  jq '.Contents[].Key' --raw-output | \
  parallel -j20 aws s3 cp s3://my-bucket/{} .

Here we use the -j flag to increase the level of parallelisation. By default, parallel will create as many parallel tasks as you have cores on your CPU but if your workload involves waiting for network IO (as in the case of accessing S3) then you can probably safely increase the level of parallelisation to speed things up.

Analysing Oyster data dumps with Clojure

Tue, 07 Apr 2015 00:00:00 UTC

Having moved to London last December, I’m using my Oyster card a lot more than I used to. In March I discovered that if you’ve registered your Oyster card with TfL you can get them to send you CSV dumps of your usage. And what’s a programmer to do with CSV dumps if not analyse them somehow?

I’ve been interested in Clojure for a while and started learning it just before Christmas so it seemed a natural fit to write the analyser in Clojure.

The result is a command line application which takes a set of the CSV files and outputs some figures like the average journey cost, total journey time etc. Here’s my output for the first three months of 2015:

 From 01 Jan 2015 to 31 Mar 2015

    Total Duration  21 hrs 2 mins
     Avg. Duration        23 mins
  Shortest Journey        10 mins
   Longest Journey   1 hrs 6 mins
        Total Cost        £213.99
         Avg. Cost          £1.88
          Journeys            114
 Most popular mode      bus (51%)

As you can see, depressingly I’ve spent almost a whole day on public transport since January and that only includes journeys that were timed – 51% of my journeys were by bus which don’t have end times!

You can find the code on GitHub – improvements and comments welcome.

Thoughts on Clojure

I’m really enjoying programming in Clojure. I’ve not done any functional programming before and the approach is making a lot of sense to me, especially in the context of the Alan Perlis quote:

“It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures.”

Clojure for the Brave and True has been an unusual introduction to the language, the Exercism.io and 4Clojure exercises are useful for practicing and I’ve used jafingerhut’s Grimoire cheatsheet a lot for reference.

Inclusivity and devops

Mon, 01 Dec 2014 00:00:00 UTC

One of the themes that came out of the recent devopsdays Ghent conference was empathy, and Bridget Kromhout wrote afterwards about making the first rule of devops, ‘welcome to the club’.

Inclusivity is something I’ve been increasingly thinking about over the last couple of years, especially with regards to women and minorities in tech. I have just about all kinds of privilege going for me – white, heterosexual, university educated male from a nuclear family with a middle-class background and supportive, open parents – and so understanding the difficulties that others face when trying to enter and thrive in the industry has been a challenge.

This post is not an attempt to provide solutions to the problems facing women and minorities in tech; I wouldn’t presume to know better than the people it actually affects and inclusivity is only one part of the problem in any case. Rather it is a synthesis of the thoughts I’ve had around inclusivity and the people and posts that have inspired me. Making tech an inclusive space is incredibly important to me and I hope to show you why.

Many of the examples in this post are about gender as that is what I have been mostly reading about recently. However, my desire for inclusivity extends to everyone, not just women, as I hope you’ll see.

Why should we care about inclusivity?

Before talking about inclusivity, I’d like to talk about diversity.

If you have a homogeneous, white/Asian male workforce (and statistically you do, at least in the more technical roles) then your teams will all have had similar experiences, they’ll likely lead similar lives, have had a similar education. Whilst this makes collaboration easy, the flip side is that we all think in similar ways and so come up with similar solutions.

By introducing people who don’t fit the mould you introduce new ideas, new ways of approaching problems and thus improve the quality of your work. Research has shown that a more socially diverse environment increases performance.

But quite apart from the benefits for the organisation, I just don’t want to work in a homogeneous environment: it’s boring. I want to speak with people not like me, hear their stories and learn about their lives. Surrounding ourselves with people that are different from us enriches our own lives as well as producing better outcomes for everyone else.

Of course, introducing people who don’t fit in can be painful, as illustrated by Erica Joy Baker in her recent post The other side of diversity. This is supported by research, and not only that cited in her article: whilst a more socially diverse environment was shown to increase performance, the same study also showed it increased relationship conflict.

The reason posited for the increase in relationship conflict comes from social identity theory: “group members establish positive social identity and confirm affiliation by showing favouritism to members of their own social category.”

For me, this is where inclusivity comes in. By creating an inclusive environment where individuals are accepting of others, and by creating a communal social identity which is inherently inclusive, hopefully we can decrease the risk of conflict.

I also want to work in an environment where acceptance is the norm. Inclusivity is so often associated with gender, race or sexuality, and those things are all incredibly important, but it’s also about the smaller things such as understanding that not everyone wants to drink alcohol – people are different from us in many different ways, and that’s OK; we can still share things with them.

We have a habit in the tech community of creating separations between us based on perceived knowledge gaps, where we judge and exclude others because they’re too nerdy or not nerdy enough. I really want to break down this ’nerd hierarchy’ where someone more or less ‘beardy’ than you is not seen as ‘other’ (and while we’re at it, can we stop using beard length as a measure of nerdiness please?) so that we can work together, respecting everyone’s skill level.

What’s it got to do with devops?

For me, devops is fundamentally about collaboration and human interaction. The movement attempts to address the problems of separate teams working towards shared goals (in our case, producing quality, functional software that is easily deployed and scaled) and so, whilst the tools associated with devops help us achieve some of that, our primary concern should be culture, not technology.

The classic situation that devops hopes to address is the organisation that has separate development and operations teams where the development team rarely takes responsibility for their software once it is deployed in production, ’throwing their software over the wall’ for the operations team to deal with. This problem won’t be solved by throwing a configuration management tool at it.

Given devops culture depends so heavily on the ability of individuals to collaborate, and if we want a diverse workforce (which hopefully you’re convinced we do), then we need to find ways to minimise conflict.

One of the ways in which we can achieve an improved culture of collaboration is to increase the quality of our communication. For me, allowing people the ability to express themselves free from constraints (in a constructive, respectful manner of course) is key and so building an environment where team members feel comfortable is crucial. Creating a culture where everyone is welcome goes a long way towards this and, as Bridget stated, this should extend not only to the development and operations teams but also to the other people we work with to produce software.

But I’m already inclusive!

The short answer to this is that even if you think you are, you’re probably not, at least not entirely.

When attending an interview recently, I was talking with my interviewer about inclusivity. The company had recently moved offices and my interviewer mentioned that the landlord had installed a sign on the unisex toilets with a man and a woman on the front. In order to be more inclusive of transgender people the company had requested the sign be removed, which would have never occurred to me before now.

The problem is that we are shaped by the world around us and we adopt the traits and biases of the culture we grow up in. Take gender bias, for example. There is no country in the world that has achieved perfect gender equality (the World Economic Forum puts Iceland at the front and according to the index they’re still 14 percent off total equality) and so we are all influenced by stereotypically associating men with breadwinning work and women with childcare.

Even if we work to redress this imbalance in our conscious thoughts, our actions are still influenced by our unconscious biases and we can often do things which exclude people without thinking.

There is plenty of anecdotal evidence to suggest unconscious (or perhaps, sadly, even conscious) bias when hiring. Take the post describing a black woman who pretended to be white, resulting in a marked increase in job offers.

Research backs this up, both for racial discrimination and gender discrimination. Given two CVs of fictional candidates for an academic job which differ only in the name (Dr. Karen Miller vs. Dr. Brian Miller), the male candidate is either perceived as more hireable than the female candidate or reviewers are four times more likely to express reservations about the female candidate even if they think she has the required expertise.

So what do we do?

Firstly, I think it’s really important for people in a position of privilege to acknowledge this and constantly challenge it. I do this by having great people around me who inspire me and call me out on my speech and actions (thanks sis!) and by reading as much as I can. For example, for more information on the research into gender differences, I can’t recommend Cordelia Fine’s Delusions of Gender enough.

I know I said I wouldn’t propose solutions but it would be silly of me not to at least point you in the direction of other people with suggestions. A lot of this is related to gender, as that’s been my main focus for my reading, but some of it equally applies to including other groups and individuals.

If you’re a man, read about ways to be a male advocate for women.
Be aware of microaggressions and what effect they can have.
If you’re in a position where you’re hiring people, learn about how to overcome unconscious bias by, for example, watching how Google is trying to do that in their hiring process or consider implementing a quota system.
Think about the parts of your culture that might not be inclusive and read Model View Culture, a great resource on diversity and inclusivity issues.
Examine the language you use and consider avoiding terms that are not inclusive, such as addressing a mixed-gender group as ‘guys’.
Take lessons from conferences that have successfully implemented a code of conduct.
Try and empathise with others more, even if that takes you to uncomfortable places.

Puppet: 'could not find class' and metadata.json

Fri, 10 Oct 2014 00:00:00 UTC

This week, when writing some custom Puppet modules, I was getting the following error when trying to use them:

Could not find class <class> for <node> on <node>

I could not for the life of me figure out what the problem was – the module was in the correct place, with the correct name and layout.

I finally realised it was a problem with the metadata.json file in the root of the module’s directory. It turns out that if there’s a syntax error in this file, Puppet will refuse to load the module without letting you know why. And it’s a known bug.

In my case it wasn’t even a syntax error that was the problem – it was caused by one of the required fields being missing from the manifest. For reference, the PuppetLabs docs specify the required fields for the metadata.json file.

Ten months travelling: UK to India overland by train

Fri, 16 May 2014 00:00:00 UTC

On 18th October 2013 I set off on an overland adventure, travelling from my home town of Frome, UK all the way to Kerala in the south of India. Here’s how I did it.

Prior to experiencing the post-university world of jobs and houses and responsibility I’d never really had the travel bug. My younger sister interrailed throughout Europe in her gap year before university; I did the ‘sensible’ thing and got a job.

Although enjoying my fledgeling career, thoughts of travel started to surface. I wanted to see totally different cultures and, inspired by my parents’ own travels, I started setting my sights on India.

I like to think of myself as environmentally conscious and so I was fairly set on not flying – train travel can reduce the carbon emissions of a trip by a factor of 5 to 10 compared to flying. I occasionally perused Seat61’s UK to India Overland page with nothing more than vague plans about how I would get there.

But one evening in August 2013, personal circumstances led me to decide that now was the time. The next day I handed in my notice and foolishly decided I needed only six weeks to plan the trip that would take me out of Europe for the first time in my life.

The Route

The Seat61 site suggested two possible routes. One, detailed in the UK to India Overland page linked to above, went via Turkey, Iran and Pakistan; the other via Russia, China and Tibet.

I agonised over the choice as well as briefly considering flying, given the considerable challenges of both routes. Both offered adventure but both also came with difficulties: the first, getting through Iran and Pakistan overland and dealing with potential security issues; the second, being able to get into Tibet at all given the complicated political situation there.

In the end I decided that the Russia/China/Tibet option was the one for me – I figured that getting from the UK to Beijing wouldn’t be too challenging and if all else failed I could just get a flight from China to Kathmandu to skip out Tibet.

I’d also always been interested in taking the Trans-Siberian train and this would be my opportunity.

The Details

Tickets

Because of the number of changes involved, and my desire to get to Tibet and Nepal as soon as possible to avoid visiting in the harsh winter, I decided I would book as much of my trip as I could in advance (preferably to Nepal).

Using a spreadsheet I planned out each train I’d need to take and booked them in three separate segments: London to Frankfurt, Frankfurt to Moscow (both with RailEurope, now Voyages SNCF) and Moscow to Beijing (with Real Russia).

Visas

Going through so many countries, I needed to apply for my visas pretty quickly. I did not think this through before deciding when to leave and I would not recommend applying for five visas (Belarus, Russia, Mongolia, China and India) with only six weeks to go.

Fortunately I had just enough time. I applied for the India visa independently first (before I had made my mind up on my route), then applied for the rest through Real Russia who were incredibly helpful and got my passport back to me with one day to go before leaving.

The only downside was that it cost me much more than it otherwise would have done, both to get the visas processed through Real Russia instead of doing it myself and to get some of them fast-tracked so I would receive them in time.

The visa for Nepal was simple for me: just turn up at the border with 40 USD (or a friend who has it – thanks Sara!) and pay the border guards.

Tibet

This was the trickiest part of the planning. Once I’d decided on the route I started researching tour companies. Due to travel restrictions, foreigners can only enter Tibet as part of an organised tour. I then started sending out a few enquiries. I also posted on the Tibet section of the Lonely Planet forum, asking whether anyone would be willing to join me.

Luckily I was able to find two travelling companions interested in doing the same trip as me (Lhasa to Nepal) and with less than 48 hours before I left home, we requested our tour with Snow Lion Tours. This Tibetan-owned operator had come highly recommended by other travellers and having now travelled with them I would also not hesitate to recommend them.

However, I couldn’t pay my deposit until I got to Moscow and even then I wasn’t 100% sure it would all go through. I finally received confirmation half-way through my Trans-Mongolian journey. Phew!

The Timings

All-in-all, travelling from Frome, UK to Cochin, Kerala, India took me about five months. That includes three months around India and a month in Nepal.

London to Moscow took the best part of two days.
The Trans-Mongolian took six and a half days.
I spent seven days in and around Beijing.
Beijing to Lhasa took less than 24 hours.
Our Tibet tour was nine days.

The Costs

Well, it certainly would have been a lot cheaper to fly directly to India, but where would the fun have been in that?!

My spreadsheet details the ticket costs which totalled £856 to Beijing. In addition I spent about £600 on the visas and ¥5730 (approx £560) for the Tibet tour. As I said before, the visas would have been cheaper had I started the application process sooner and the tickets would have undoubtedly also been cheaper if booked further in advance.

If you’re thinking of taking any trains from Moscow east and fancy taking a bit of a risk, you can buy your tickets at the station in Moscow for a fraction of the cost you get them for through an agency. However, don’t expect the process to be straightforward (I spoke to a couple on the train who did this and it took them several attempts), or that the tickets you want will necessarily be available.

The Reality

In short, I had an amazing time. The Trans-Siberian was fantastic, partly due to the experience and the scenery and partly due to my fellow passengers, China and in particular Tibet were eye-opening and Nepal was the highlight of my trip.

In fact, I had such a great time that once I got to India I decided I didn’t want to stop so I flew to Australia (I agonised over this decision and spent a long time trying to figure out how to get there overland; alas, it was just too difficult) and spent five months there and in New Zealand working and travelling (but mostly travelling).

Scratching an itch: why I like programming

Thu, 01 May 2014 00:00:00 UTC

The Indian train system is amazing considering the number of passengers it hauls around the country but it can be frustrating as a foreigner trying to book tickets, especially at the last minute.

Certain trains have tickets put aside for foreign tourists. This is fantastically useful given that most trains sell out well in advance but discovering which trains have foreign tourist quota tickets available is problematic as online booking agents don’t have the information – you can only buy them at stations. The process invariably involves the following steps:

Navigate to Cleartrip site.
Enter journey details and submit.
Discover no trains run between the two chosen stations (Cleartrip only searches direct routes).
Repeat step 2 until you get some results.
Note down the train number and figure out the short codes for the departing and arriving stations.
Navigate to the seat availability form on the Indian Rail website.
Enter journey details and submit.
Find you’ve entered something wrong – very likely given how exacting the form is.
Go back, amend details and submit form.
Find the site has reset the date of travel you put in.
Repeat step 9.
Repeat steps 8 through 11 once or twice.
No foreign tourist quota tickets for this train.
Go back to step 5.

If there are 20 trains on a given day for your route, checking each one manually becomes unfeasible.

Instead, I wrote a Greasemonkey script that automates the Indian Rail website portion of the process and reports the results right in my Cleartrip results page. What might have taken thirty minutes for each journey (and as I have no firm travel plans I’ve been checking a lot of journeys) now takes seconds. Itch scratched.

Another little automation project I recently completed was prompted by one of my guesthouses’ ISPs. To use the WiFi you were first required to log into a web page. Unfortunately, unless you kept the login page open, the session would frequently time out requiring another visit to the login page. Rather than doing this manually every time I discover I’d been logged out, I wrote a simple Python script to perform the task for me.

All it does is mimics the keepalive request that the login page performs once you’ve logged in. If the script detects the session has timed out anyway, it posts the login details to the ISP. This may not sound like a big win but when you’re working all day and the session is timing out several times an hour, it saved me going prematurely bald.

These two scripts only took me a couple of enjoyable hours to build, taught me more about the languages and domains I wrote them in and, most importantly, unlike the experience in the above XKCD comic, have saved me a lot of time.

Teaching the world to scratch their itches

A lot has been said of late about the value of teaching schoolchildren to code as the UK Government launched its Year of Code initiative earlier this year, although not without some controversy.

One of the reasons I love being a programmer is having the ability to solve problems by bashing out a few lines of code. Not only does this make my life easier but I get to enjoy the process as well.

My skills mean I’m able to recognise and understand a problem and come up with the solution, not just because I know a handful of programming languages (although that obviously helps) but because of the knowledge programming has given me about the world of technology and how it works.

My hope for the Year of Code is not that a whole generation decides they want to become the next Zuckerburg but that youth are given the tools to understand the technology that the world is increasingly reliant on and be able to make it serve their needs.

If that also results in someone being more likely to create the next Facebook, so much the better.

Why TDD isn't dead

Wed, 19 Feb 2014 00:00:00 UTC

Rails creator David Heinemeier Hansson (DHH) recently wrote a post denouncing TDD. Cue mass outrage/support from the web. I’m a bit of a TDD convert so I wanted to put my thoughts down on why that was.

DHH seems to conflate unit testing and TDD and ends up hating on them both so I’ll address each topic separately. Sure, the emphasis with TDD is generally on testing the unit because that’s the focus of your short-term efforts but you don’t have to be limited by that – I sometimes use TDD to create system/integration tests before I start writing the underlying units.

Unit testing

Encourages Single Responsibility in your design

If you’re writing your unit tests properly, each unit is tested in isolation from the other units it interacts with. This encourages the developer to design with the Single Responsibility principle in mind – each unit only being responsible for one aspect of the system.

This seems to be DHH’s main argument against TDD, that it somehow leads to an overly complex architecture where you have a huge set of objects just to enable you to test each one in isolation. Whilst I can sympathise with this somewhat I think there’s a tradeoff to be made.

For me, the advantages of having unit-level test coverage generally outweigh the disadvantage of introducing a few extra objects. And in my experience it is just a few; most separation of functionality into units is not merely contrived for the purposes of testing if following the Single Responsibility principle correctly.

Pinpoints a problem’s source

By testing each unit in isolation it’s much easier to pinpoint the source of a problem – you should have a failing test which tells you the rough, if not exact, point in your unit which is misbehaving. With a system-level test you just know what the problem is, not where it resides.

Also if you’re correctly isolating your units then the unit tests you write will be testing your code and your code alone. This means that when a failing unit test occurs, you can rule out any bugs with collaborators (such as framework code).

Makes unit-level refactoring a breeze

Once you’ve got a good level of coverage for a unit’s public interface, you can easily change the underlying implementation of the unit without worrying about affecting other units relying on that interface. If all your unit tests pass, you have a good degree of confidence that your refactor was successful.

Makes for reliably reproducible tests

If you’re isolating your units away from other systems – a database, the filesystem etc. – you can rely on the fact that your test will run the same every time. If you have tests which do interact with an external system then you have to have a way of controlling their state, else you can’t ensure that they won’t give your system under test inconsistent responses. Furthermore, by having control over collaborators you can ensure your unit handles edge-cases such as network timeouts correctly.

This also has the advantage of tests not affecting persistent state systems such as an external API – you don’t want your tests asking MailChimp to send a campaign to your customers every time they run!

Test-first

Tests actually get written

Let’s face it, despite lots of tooling, literature and evidence that automated testing is a Good Thing™, the web development industry isn’t particularly good at writing tests. I’ve seen tonnes of production code from many different projects that for various reasons didn’t have any automated test coverage whatsoever. If you encourage a test-first mentality it’s harder to let the tests slide.

You get to check your code works before integrating

By writing the tests first, you get to test your code actually works before you even make it interface with anything else. This works both at unit-level and higher levels: I recently used a test-first approach to write integration tests for an API that didn’t yet have a consumer. I was able to write the API to a specification and check it matched that specification without writing the client.

A bit of pragmatism

It’s not a good idea to adopt TDD, blanket mocking or any other technique blindly without analysing why you’re using it and how it affects the code you write. Having a test-first attitude can lead you to create wasteful tests just to ensure every line you code works (something I’ve definitely been guilty of), just as wanting perfectly isolated unit-level coverage can lead to an overly complex system design.

What’s needed with any approach to testing is pragmatism – as Gary Bernhardt says in his reaction piece TDD, Straw Men, and Rhetoric, “TDD is useful and test isolation is useful, but they both involve making trade-offs.” Let’s not discard unit testing or TDD simply because they can be harmful or wasteful when used in extremis.

Framework scalability

Sat, 08 Feb 2014 00:00:00 UTC

Whenever I start a project with a new tool or framework I spend a lot of time seeking out its best practices, especially with regards to code organisation. I want to make sure that the habits I adopt early in the project won’t come back to bite me later on as the project grows to avoid refactoring swathes of code.

So when I started looking into Angular.js, I did the same thing. As it’s a Google-backed project and is one of the more popular Javascript libraries of its ilk in the development community I expected to find it easy to discover how to scale an Angular application – after all, there should be plenty of people who have developed large projects with it.

However, as I went through the tutorial and searched for example projects it became apparent that this would be harder than I expected. Even to my beginner’s eye, it was obvious the code in the tutorial would not scale – it creates single files for each ’type’ (controllers, services etc.) in the application and each one is referenced individually in the HTML file.

I knew that even a modest application with more than a few pages would likely require several of each type and that organising each one into a separate file would make maintenance much simpler. By doing this, referencing them manually in the HTML quickly becomes unfeasible as you have lots of script tags to maintain and reduced client performance when loading the application.

And yet I could not find a project template which addressed these concerns and made it simple to start building:

angular-seed, the official project bootstrap, has the same problems
angular-app is a step in the right direction but lacks documentation on how it works rather than just how to run it and would need lots of work to get a new project up and running
Whilst more what I was looking for, I found AngularFun very complicated and if you’re not writing your project in CoffeeScript then you won’t be able to use all the components in the build script.

Finally, I came across Radian. It provides an interactive generator for Angular projects, loads modules with RequireJS, sets up a CSS preprocessor if you want; in short, does everything I was looking for.

It’s not perfect – its documentation was a bit lacking and setting it up required a little bit of tinkering – but otherwise I’m very pleased with the results. I’ve already contributed to the documentation which will hopefully improve the experience for other people wanting to use it.

I can understand why the Angular tutorial avoids the complexity of build tools and script loaders – it’s not straightforward to explain itself – but for anything other than a very simple application its own complexity necessitates using ancillary tools to manage it. Surely it wouldn’t be too hard to provide a build script and explain it iteratively through the tutorial?

This process highlighted to me just how important it is to provide users with tools to get started quickly and remain productive as their projects scale. Django does this well by bundling a project generator with the framework and the default structure is fairly scalable. Sites such as Django best practices are an excellent complement.

Hopefully as Angular matures and more people document their experiences of using it in larger projects it will become easier for newcomers to start building scalable applications. Tools like Radian will fill the gaps in the meantime.

Mocking files and file storage for unit testing Django models

Sat, 08 Feb 2014 00:00:00 UTC

When writing unit tests, avoiding interaction with external services (for example the filesystem or network) is considered best practice because you can’t rely on external services’ availability, nor do they present a consistent, predictable state when running your tests. Normally, the way to achieve this is to mock out the particular service your code depends upon but when your code uses a large framework such as Django it becomes difficult to ascertain what to mock and how to mock it.

In my case, I was trying to unit test some code which interacted with a Model with a FileField on it. To create an instance of the Model I needed to populate the FileField with a value but the field expects a django.core.files.File instance.

This presents two problems:

If I give the Model an actual File instance I need a file to read which would involve touching the filesystem
Upon saving the Model the file will be processed and stored using the file storage system, again touching the filesystem (or the network if you have a custom storage system to e.g. upload to S3)

My solution involved mocking both the File instance and the file storage system. For my unit tests, I use the Python library mock.

First we need to mock the File instance. There is only one thing we really need to mock – the name attribute – because it is primarily the file storage system which accesses the File’s methods:

import mock
from django.core.files import File

file_mock = mock.MagicMock(spec=File, name='FileMock')
file_mock.name = 'test1.jpg'

This tells the mock library to create a mock object based on Django’s File class and assigns the name attribute. Using this is as simple as assigning the mock to the file attribute on our model:

asset = Asset()
asset.file = file_mock

The file storage system is a little more tricky. We need to create the mock and then get Django to use it. Luckily there’s one way in which Django retrieves the default file storage system. We can use mock’s patch function to force it to return our mock instead of the default:

from django.core.files.storage import Storage

storage_mock = mock.MagicMock(spec=Storage, name='StorageMock')
storage_mock.url = mock.MagicMock(name='url')
storage_mock.url.return_value = '/tmp/test1.jpg'

with mock.patch('django.core.files.storage.default_storage._wrapped', storage_mock):
    # The asset is saved to the database but our mock storage
    # system is used so we don't touch the filesystem
    asset.save()

We need to patch _wrapped because default_storage is a lazy-loaded object and _wrapped is the property which it uses to determine if it’s been loaded yet or not. Note: this won’t work if you’re manually setting the storage property of the file field when creating the Model; you must be relying on the DEFAULT_FILE_STORAGE setting.

Edit: A previous version of this article recommended patching get_storage_class. @Paul_Collins spotted that this would lead to Django caching the first test’s mock rather than replacing each time. This new method of patching _wrapped should get around that.