Learning

What’s wrong with this code? A bit of background for this example; `driver` is a Selenium::WebDriver object, the Selenium server is remote. Also, this technique is applicable to all expensive code that does the same thing over and over.

1
2
3
4
5
6
7
8
9
10
11
def first_name_field
  (driver.find_elements :id, "name")[0]
end

unless first_name_field.nil?
  if first_name_field.displayed?
    if first_name_field.enabled?
    first_name_field.send_keys "Dylan"
    end
  end
end

Apart from being terribly contrived, this code is bad because every single interaction with first_name_field causes a call to find_elements, which in turn reaches out to the Selenium server. You’re adding a network round trip every single time, and for no reason. Instead of asking the browser to do this:

  1. Find the “name” element (line 5)
  2. Check if the name element is displayed (line 6)
  3. Check if the name element is enabled (line 7)
  4. Send the string “Dylan” to the name element (line 8)

You’re asking it to do this:

  1. Find the “name” element (line 5)
  2. Find the “name” element (line 6)
  3. Check if the name element is displayed (line 6)
  4. Find the “name” element (line 7)
  5. Check if the name element is enabled (line 7)
  6. Find the “name” element (line 8)
  7. Send the string “Dylan” to the name element (line 8)

That’s nearly twice as many commands, effectively doubling the length of your test, for no gain at all. You only need to find an element once and you can interact with it until Selenium’s reference to that object is invalidated. This only happens when the page you’re on mutates to another (either directly via URL commands, through clicks on buttons that navigate, or by Javascript).

This doesn’t just occur through Selenium commands (although Page Object Model code and libraries are often prone working this way). Loading configurations, accessing web services, validating data. There’s many places where you’ll burn up cycles (and correspondingly incinerate coal and warm the planet dooming us all just a liiiiitle sooner. Yeah, I actually do think wasted compute matters.) and, maybe more sellable to your Product Owner (who clearly doesn’t care about the environment), this will make your code faster.

Are you writing this down, and taking a memo?

No, like, literally. Take a memo. This kind of memo. Memoization is where you save the result of an expensive operation and return the saved result every time the operation is requested. Like responding to everything your Mum says on the phone with “Uhuh”, Memoization swaps one resource (computing time or brain-cells) for another (memory or engagement). Memory is cheap. You can probably throw more into your server if you need it.  (But you don’t need it. Just… You don’t, trust me.)

If you use Ruby, you’re probably already taking advantage of Memoization:

def proscuitto_pizza
  @proscuitto_pizza ||= Pizza.new(:delicious)
end

That’s right, the ||= operator in Ruby effectively does Memoization; If the variable exists, it does nothing, else it sets it to the code on on the right of the operator. Wrap the whole lot up in a method name and you’ve got a memoized value.

For example, we could re-write the code above as so:
def first_name_field
  @first_name_field ||= (driver.find_elements :id, "name")[0]
end

That single line change means that Ruby will remember the field in question and we won’t need to ask the Selenium server for it again, saving us a heap of time.

(Selenium Sidenote – Unless the page changes of course. How do you know if the page changes? Well, every navigation will do it (including those involving submitting forms) and some AJAX requests do it. You need to understand how your site is implemented… But then, you should anyway so you know what you’re actually testing).

(Some more reading on the ||= operator)

 

So, the Latte; The One True Unit Of Value, representing all that’s wasteful about Millenials, preventing us from becoming rich. The single way to convey something’s monetary value.

Entirely beside the point of whether you consider Lattes to have value, I assume you consider your career does? Maybe you’re working up to a glorious technocratic empire, constant spreads in Inc., Johnny Ive begging you to please, please come back to Apple. Perhaps you’d just like to not go bankrupt.

Avdi Grimm, Generally Awesome Person and Excellent Ruby Developer, would also like this. He posted this article explaining that, while he’s not in imminent danger of dying in debtors prison, he’s not liking his income trend.

This was probably a rough thing to admit because talking about money makes people weird and evidence of failure is often taken as evidence of deserved failure. To me it was rough to read because Avdi is an excellent developer who makes fantastic resources. I’ve loved reading Confident Ruby. I’ve been a subscriber to Ruby Tapas (his short-form screencast series) for over two years. If you’re wanting to invest some cash in your career, and have that go directly to a literal Ruby Hero, go read the blog post and then buy a book or subscribe to Ruby Tapas. You’ll get value out of them. And yes, I think they’re good for a developer’s work as at least two cups of coffee.

(I’m not being paid for this post; I don’t think I’ve ever met Avdi, I just think he’s made some nifty stuff and this was a good chance to turn you all onto it.)

 

Git (and its ilk) are tools for working collaboratively (especially with hosted services). Your commit messages are your best weapon for communicating context, and every collaborator, including Future You, depends on context to be productive. It’s not opinion, it’s Science!

See, human memory is so bad it can be erased by passing through a doorway. We can defeat doorway-induced time wasting (not to mention the change of screaming “what the hell is this?!” at code someone, maybe Past You, wrote) with great commit messages.

Randall from http://www.xkcd.com knows what's up.
(Image courtesy of XKCD because Randall is a champ.)

A great commit message:

  • Is never ‘left behind’ by code changes (unlike comments)
  • Gives all the info you need (ticket numbers, reasons for change, expected behaviour changes)
  • Shows relevant changes around your change
  • Summarises everything in 50 characters or less as the ‘subject line’

All this info, the context of the change, makes problem solving (read: coding) more efficient.  Good context helps any collaborator; there’s evidence that context-based approaches to problem solving are more effective.

Continue Reading

 

So I worked all weekend this week. I won’t get paid. I won’t even get a nod from my boss, although I was there for even longer for a standard work day. Jerks. Time to burn the office down. Get my stapler back.

OK, so they’re not jerks, and I wasn’t working at work. I was at Brisbane’s Global Day of Code Retreat, taking generous advantage of the Thoughtworks office’s fridge and wifi.

The idea is that you spend a day focusing on implementing the same thing over and over, so you can hone your professional skills. In the same way as a sculptor might practice straight lines over and over, we implemented Conway’s Game of Life using TDD techniques in 45 minutes. Every 45 minutes you had to delete your code and tests, debrief, and start again with a new constraint. Some of the constraints felt more annoying then educational, but on the whole I felt I got something out of every session. I also got to teach a couple of people something about TDD & RSpec, and I love helping others improve, so I got a kick out of being able to contribute like that.

I’m going to post how I found each session today, then mull over the others and post some observations a little bit later. I think it’s best to see how learning effects you in other contexts (like actually at work) before you jump into critiquing it, so I’m going to see how the next work week goes.

Session 1

Language: Ruby
Testing Framework: RSpec

This session just booted people off into implementing Conway’s life. I worked with a young Thoughtworker whose name eluded me, and we got about halfway into an implementation before time was up.

We spent the majority of this session just discussing what way to do things. I was a big fan of using a “Cell” object which maintained a list of its own neighbours, without directional information because it doesn’t matter. Doing it like this allows for arbitrary topologies (And I suspected that might be one of the constraints.)

Potentially tricky issues with this approach include implementing the tick for every cell at once(If you duplicate the cells, you have to duplicate the network and ensure you’ve duplicated every cell with its new state. If you mutate the network in place, each cell needs to know if it’s neighbours have ticked, and if so, what their state was last tick.

At the end of the session we had a discussion about what techniques people tried. Having a “Cell” object was quite popular (OO is like herpes, it spreads with contact), there was one “have a universe holding an array of cells” solution, and the one which I liked most, which was using the co-ordinates of the cells as a key for a hash of booleans, representing whether the cell is alive or dead. I really liked this… It’s unbounded (unlike an array in many languages) and *doesn’t* require a hojillion objects.

Session 2 – Ping Pong

Language: Ruby
Testing Framework: RSpec
Constraint: Each time you write a test, you ensure it tests properly, then you pass it to your pair and they have to do all the implementation. Then they write the next test, and you implement that

This session we decided to try the hash technique one of the groups suggested last time. I say”we decided” but actually mean “I decided”, because prevarication frustrates me, my partner had no strong opinions and I thought it sounded interesting so I made a snap decision.

The Ping Pong criteria is how I’d probably prefer to do pairing in the future, although usually it irks me (I need to be doing something and my “I’m not doing anything” behaviors usually irritate my pair). It meant that your tests had to be higher quality, and there was a defined place where you had to hand over to the other member of your pair — No long stints at the keyboard or just watching.

We didn’t get an implementation finished though, although I worked out how I’d like to derive neighbours. I proceeded to teach people a tiny bit of matrix math all day (more on that later)

Session 3 – Silent Ping Pong

Language: Ruby
Testing Framework: RSpec
Constraint: The same Ping Pong TDD as before, but this time you’re not allowed to talk

I paired again with a Thoughtworker this time, a gent from China called Hiyun (I hope, sorry if I’m wrong mate >.>). Wow. Dude was a shortcut pro. I learned a shittonne of RubyMine shortcuts and tricks I’d always wanted too but never devoted time to (Which is obviously my fault). It was really cool, probably my favorite session of the day.

We went back to the “Cell as self aware object” paradigm (drink). With the exception of a couple of inadequate tests, this session worked remarkably well. The inadequate tests required a lot of gesticulating. Once you’ve written something it’s hard to prove that the test is wrong rather then the code.

I think not being able to talk focused our attentions on communicating *only* our requirements through tests. There was no discussions of where you worked, what you thought of Test::Unit VS RSpec, why’s DHH’s hair like that; it was purely a “get shit done” event.

I think the most surprising thing was that we finished. It was almost like being unable to design architecture by consensus made one arise naturally out of requirements, which I am… skeptical of. I’m not sure how well our solution would have accommodated change, but maybe most software projects are simple enough that it doesn’t matter.

Session Four – No Primatives

Language: Ruby
Testing Framework: RSpec
Constraint: Rather then using native types to store results, cells and so on, use a wrapper class or object. No Arrays of cells, no Object.live? returns true

Oh god so hard. Getting away from primitives in Ruby is hard. Every time I wanted to wrap something I heard a tiny voice in my head stop talking about bees, games, cooking and Alice in Wonderland for a minute to scream “NEEdleSS ComPleXItY!“. All I wanted to do was use built in objects so I could rely on methods that Someone Else Has Tested™.

This session, we did something horrific. We decided that, if we weren’t going to use primitives, we’d use something as far removed as possible. Something so non-primative it’d make your eyes haemorrhage.
Files.

Yup. Files. To check if there was a cell at (4,5), you checked the directory ./data/4 for file 5.cell. If it existed, it was alive. Touch a location to make a cell alive, rm to delete it. We were planning to store the current generation number in the file to allow for ticking.

How’s that for non-primitives, bitches?

Session Five – Tests must pass in 3 minutes

Language: Java
Testing Framework: JUnit
Constraint: Once you start writing a test, you must write code to make it fail in three minutes or less. If you get to three minutes without it passing, you must revert all you changes and delete your test

The intent of this constraint, I believe, was to force you to test small amounts of functionality rather then huge sweeping changes. The idea is that small tests are good tests, and it’s tempting to write large tests that try to do too much. When they fail, you’re not actually sure why, and you lose much of the advantage of a full test suite.

My pair (Hi, Mike!) and I chose Java because it was our lingua franca (Scala was preferred but it’s quite hard to use a language without a dev environment, oddly enough). This may not have been the wisest choice, because of one simple truism:

Fuck me Java is Verbose!

It’s not just the punchline to the “what’s wrong with other people’s languages” joke. It was never really bought home to me just how verbose Java was until I had to spend so much time setting up each test. I’d just spent most of the day writing Ruby and now I’m having to give all my tests annotations and a typed, static method signature, and set up my variables types, and then set up and type the method and ensure to return the right sort of thing, and just making so much boilerplate it nearly made my eyes bleed (I propose we call this Stigjava.)

Even Mike, who writes Java for a living, was surprised. He said it bought Java’s verbosity into stark relief after a day of writing Ruby and Haskell. It forced us into doing true TDD because otherwise we simply couldn’t make a test that tested anythingin time. We had to implement the minimal amount every single test pass. Here’s how we had to test that we could get a list of neighbours for a cell:

  • Write a test to ensure that there is a “getNeighbours” method. Just that one exists. Pass it
  • Write a test to ensure the method returns an (empty) ArrayList. Pass it.
  • Write a test to ensure the method returns an ArrayList containing eight things
  • Run out of time because it took so long to learn how much code we could write at once

It was just astonishing. I’m not Java bashing (so passé), just really surprised. Writing the code to pass things, and even the tests, wash’t too hard, just getting to a point where we could do something took forever. And Mike is an IntelliJ wizard – Otherwise we’d have been screwed. He was making sweet combo love to the keyboard to bust out methods and variables and implement the signatures correctly and we were still fighting the clock.

Session Six – No Returns

Language: Ruby
Testing Framework: RSpec
Constraint: None of our methods could return values.

Woah. OK. Globalling it up. During this session I had a bit of an idea how to use events to generate a grid of neighbors, but didn’t implement it cleanly. Actually, this session was embarrassing because I forgot that Ruby is pass-by-reference, so resorted to using Globals. Don’t tell my mother. The solution wasn’t really productive, just because every time I manipulated the global objects I felt dirty and wanted to cry.

During this session, the pair using Haskell gave up on the constraint and just started trying to get it working at all. Globals were popular (at least we weren’t alone wallowing in filth) and only one group used callbacks. The idea I’d had was callback/eventing based, but wasn’t async, so obviously I need to do a bit more work about doing that kind of programming at need, rather then when forced.

Then, we had beer, a debrief and a powwow. Roughly half the room left straight away and the rest wanted to hang for a bit, which I always enjoy. I’m looking forward to the next one. I’m thinking this time, we use the presence of machines on a network as live cells….

 

A quick tip. It’s relatively easy to build a test, or an entire suite, and never actually ensure it gets run. Which makes all the effort that goes into it worthless AND gives you a false sense of security. It’s like having a photo of a security camera feed instead of the feed: “Of course everything’s safe, look!”

So, every time you create a new test file, assert that false == true. And run it. And make sure it fails.

Otherwise you’re wasting everyone’s time, especially your own.

 

I’ve been playing with Devise for an authentication system for a project I’m working on (More on that when there’s something more solid), and I stumbled upon a heisenbug in omniauth-facebook, specifically in \lib\omniauth\facebook.rb, in the callback_url method:

def callback_url
        if @authorization_code_from_cookie
          ''
        else
          if options.authorize_options.respond_to?(:callback_url)
            options.authorize_options.callback_url
          else
            super
          end
        end
        #THIS LINE HERE IS IMPORTANT
      end

When I’d add ‘puts "callback_url finished"‘ in the commented line, the defect would go away (And leave me with another, but that’s another show). Remove the line, and it’d be back. Now, this is proof that the intelligence of the observer effects if they count AS an observer. A smarter man then me would have been able to collapse the defect’s waveform and figure out what was happening. Can you see it?

Continue Reading

 

My favorite stupid mistake is the kind you catch quickly and fix just as fast. Even when I make them. Like when I was logging today.

Yup. I can even screw up logging. I have a very special talent.

See, I was writing some logging for a daemon using The AMQP Gem and I want to mention when stuff is bound, that connections opened successfully, and so on. This log will be mostly used for after-the-fact defect diagnosis. I went ahead and wrote some logging statements after events happen such as

"Queue Created: Raywood, durable => true"
"Queue Raywood subscribed to exchange with routing key Brisbane"

Those log lines accurately describe what’s happened just prior to that line. What’s the catch?

Those log lines only accurately describe what’s happened just prior to that line, for now, assuming no weirdness with libraries.

Sure, right now I’m setting the Raywood queue to be durable, but what if (for some reason) it could be created non-durable and that declaration didn’t throw an error? That could easily happen with a library bug, or even by design. I could also change the line and forget to change the log, with the same result.

Logging should always reflect reality.

The best way to ensure that is by directly interrogating objects you’re logging about for their properties, rather then relying on what you’ve asked to happen, happening.

"Queue Created: #{queue.name}, durable => #{queue.opts.durable}"

If it’s optional for an operation to succeed, the logging should be tailored to reflect that possibility.

"Queue #{queue.name} subscribed to Exchange #{exchange.name} #{queue.subscribed? ? "succeeded" : "failed"}

 

Now, I’m sure when it was created, the entire block VS inline & “lets have different means of aligning shit” mustn’t have seemed like a terrible idea. After all, surely having different formatting paradigms is a GREAT idea! CONTEXT SENSITIVE SHORTCUTS FOR EVERYONE! Still, our chances of getting a new, easy, consistent format are precisely fuckall, and will remain there until I can force people to upgrade their PC and browser by fiat, including all the empire building fuckwit device browsers (Yes, Fapple and Windows Mobile Phone 9 Excel Ultra Slimline Fantastico included).

I don’t have the time or energy to work out aligning shit by first principles, by sucking up to Tim Berners-Lee (’cause he’s got his own shit to do, like riding the mighty moon-worm) or by trying to interpret 407 different crappy “How does I CSS” posts.

So I just refered to this one, which is well written, easy to understand, and is slightly patronizing, which makes me feel less bad for not knowing already, even if it is NEEDLESSLY COMPLEX MINUTIA.

Seriously, Go Check It Out.

 

Since writing your own UI elements from scratch is for chumps, I’ve been using jQuery to construct our new booking systems.  Lots of shiny there, all the better to Web-2.0 your social media with, my dear.

*{vomits a little in his mouth}*

I have a sortable list where each li element has an attribute storing object data.  An AJAX lookup supplies the data from a 3rd party source, and there can be 1..n li elements.  Calling serialize on that list allows me to get the ids of the list objects in their currently sorted order:

>>> $("#waypoints").sortable("serialize")

"waypoints[]=5&waypoints[]=8"

That’s OK, but I want to access the json data, not the ids.  I could put that data in a javascript array and then simply re-order it (or post the ids and the data separately and use the controller to order them), and I also could gouge my eyeballs out with a rusty spoon.  Thankfully, according to the docs, I can pass a hash to serialize specifying which attribute to return:

>>> $("#waypoints").sortable("serialize" {attribute: "json_data"})

"{"des_id":"a","des_name":"r","fleet[]=id":"5"}&  "{"des_id":"a","des_name":"r","fleet[]=id":"8"}

…Well that’s not what we what.  That’s weird, and what the fuck is going on with that inserted []?  Hmm, perhaps the key parameter is being clobbered and needs to be set as well (thus specifying what the first part should be):

>>> $("#waypoints").sortable("serialize" {attribute: "json_data", key: "waypoints"})

"waypoints[]=id":"5"}&waypoints=name":"8" }"

Nope, still fucked.  And weirdly escaped now, also.  I could use expression to create my own parsing regex, but I also could gouge my eyes out with a rusty spoon.  Into the source code, then, to see what the hell’s going on.

 serialize is defined on line 3365 of my copy of jquery-ui.js, and has this snippet:

3365
3366
3367
3368
$(items).each(function() {
    var res = ($(o.item || this).attr(o.attribute || 'id') || '').match(o.expression || (/(.+)[-=_](.+)/));
    if(res) str.push((o.key || res[1]+'[]')+'='+(o.key && o.expression ? res[1] : res[2]));
});

Ah.  That explains why I was getting that inserted [] in the last part of each value, and also why I wasn’t getting the full thing when I changed the key.  When I don’t pass an expression in, it chunks up the value by discarding the first group, and using the second.  In this case, everything up to the last underscore is discarded and the key used is everything before the last underscore, with a [] appended.  

When I supplied a key, it simply discarded the entire first matched group and returned the second.

So now I can solve my problem.  To parse an attribute containing underscores (or – or =) with serialize, I simply need to provide the attribute name, the key and a catchall expression, say, (.+):

>>> $("#waypoints").sortable("serialize" {attribute: "json_data", key: "waypoints", expression: "(.+)"})

"waypoints[]={"des_id":"a","des_name":"r","fleet_id":"5"}&  "{"des_id":"a","des_name":"r","fleet_id":"8"}"

 

Somewhere along the line I decided to use SubSonic for my ORM.  It might have something to do with the sense of humor of the major contributing author, Rob Conery, or its flexibility, or perhaps it’s just that it talks to me like I like to be talked too, and many many ORMs don’t.  As for MVC’s Entity Framework… I’d just rather not.

So, I’m using SubSonic and I’ve got a wee bit of a problem.  Validation for MVC3 is powered by annotations, as is their wont.  Adding an annotation is a simple matter of placing it above the property definition, as well you know:

public class LookingGlass{

  [GetsJam("Tomorrow")]
  public LookingGlassCharacter Alice {get; set;}

  [Required]
  public string AllMadHere{get; set;}

}

So adding a validation is simply a matter of typing it in, and away you go.  Now, SubSonic uses T4 templates to generate model classes from the database and… Oh.  Every time we re-generate we’re going to have to re-add our validations.  Shit.  But wait!  SubSonic generates partial classes!  When two partial classes have different annotations, the annotations are stacked when compiled! …But only for the class definition, not properties.  Shit, again.

So now what?  Well, now we need to use the MetaDataTypeAttribute. This tells a class to use another classes’ members’ metadata as its own.  Where this gets useful is when you consider that you can roll custom classes to provide metadata on a partial.  Since partial class attributes are stacked, your metadata attributes are applied to members of the composite class once created. You can then provide the metadata needed in “stub” properties in your metadata provider class:

[MetadataType(typeof(LookingGlassMetadata))

partial class LookingGlass {}


partial class LookingGlassMetadata {

  [GetsJam("Tomorrow")]
  public LookingGlassCharacter Alice {get; set;}

  [Required]
  public string AllMadHere {get; set;}

}

So now you’ve got annotations which can applied to properties in partial classes, and will survive a re-generation of a partial from a template. Sure, there’s replication, but that’s something we’ll address next time.

Neat huh?