Journeyman weeks - week four @ msgGillardon

Read here about last week at soundcloud...

Nicole Rauch had been a strong supporter even before I actually came up with the idea of doing a journeyman tour. The German Softwerkskammer network has been playing with the idea of craftsmen swaps and Nicole worked hard to make that possible in the company she works for. Hopefully the example will inspire others to do similar things.

During SoCraTes Nicole and her partner Andreas Leidig had offered to host me and eventually got the OK from their employer msgGillardon to let me work there for a week. The company is set in the small town of Bretten, close to Karlsruhe. It's a very beautiful little corner of Germany and I was quite happy with the contrast of small town life compared to frantic Berlin the week before.

View from the top of the office

The part of msgGillardon I was working at makes software doing forecasting for finacial institutions. Most of that is still in C++, with newer parts now being written in Java. I was a little too intimidated by C++ to work on that, which in hindsight might have been unnecessarily self-limiting. But I actually enjoyed working in Java again, since it's been quite a while. The framework used (Eclipse RAP) and the specifications written in FitNesse provided enough things that were new to me and gave opportunity to learn.

I remain skeptical of frameworks that try to hide the complexity of visually representing an application. In the case of Eclipse RAP it at least seems to be thorough and consistent in hiding representation, which makes it more acceptable then Wicket or JSF (*shudder*). If you're happy to stick with what the framework provides and think in terms of desktop applications, as in this case, RAP seems to be helpful. But one of the drawbacks showed itself when we started to write Selenium tests for the application. The effort required to select elements to interact with made automating them too expensive.

FitNesse seems a little crude in some places but I liked working with it. I was missing support to generate code snippets, as Cucumber/JBehave/SpecFlow all do. Aside from that, writing examples in Slim tables fit very nicely for the domain we were working on. There was a lot of combinatorial complexity in the inputs and it looked a lot more comprehensible to have these in FitNesse rather than in java unit tests. It also allowed for a slightly easier active conversation with the product owner about the required functionality.

In terms of culture, msgGillardon is quite different to the start-up companies I've visited in previous weeks (and also quite different compared to ThoughtWorks and its clients). As a more traditional medium sized company working for a lot of customers in banking, things were a little bit more formal and the technology less bleeding edge.

Nevertheless, I was very positively surprised by the willingness to try out new things. (And not just the obvious experiment of letting some random guy show up there and work there for a week. With very convenient and simple organisation.) There seems to be a genuine interest in changing and improving and that is not something I'm taking for granted anymore.

The other thing that definitely stuck out was the diversity of the teams, at least in age and gender. I have no idea how that came to be but it was refreshing to see. I'd be curious if people there have found an explanation for why they are doing so much better than everyone else in Germany.

During the week I also had the opportunity to take part in the Softwerkskammer Karlsruhe meet-up. It was nice to see so many familiar faces from the SoCraTes conference. Nicole and Andreas were running a legacy refactoring workshop that was a very nice alternative to the legacy code retreat format. Like all good workshops, it left me with a lot of things to think about on how I would do things differently if I were to do it again.

This ends my journeyman weeks for now. I feel incredibly grateful for the privilege of having been able to do this. I am in the process of summarising the different experiences and compare and contrast them. I will hopefully also have some useful information for others who want to try something similar. If you have any specific questions or feedback, please do leave a comment, contact me on twitter or write me an email. I'd love to hear from you.

Pictures from the past couple of weeks are on picasa. Also check out Peter Kofler's blog about doing something similar in Vienna and Andy Waite's for something remarkably global.

Journeyman weeks - week three @ soundcloud

To find where I left off go here...

Leaving Bucharest behind

The break in Bucharest was really awesome. The ALE conference was as good as I remembered from Berlin in 2011. There's a certain vibe to this conference that is hard to describe. People generally seem to be very open and trusting and thus the conversations to be had are very interesting and candid. It felt very nice to be around friends old and new. And people were very encouraging of what I'm currently doing and very supportive. Which reminds me to point out the following. If you like what I'm doing please let me know in comments or on Twitter. It means a lot to me.

I stayed at quite historic grounds

I was still processing the impressions from Bucharest when I set out for Berlin. Arriving there I found my way to the interim SoundCloud office in Prenzlauer Berg. I was greeted by my old friend and former colleague Duana Stanley who had arranged my week there. We had never got to work on the same team while we were in ThoughtWorks, so I was looking forward to finally getting to pair up with her.

The interim SoundCloud office

We had briefly talked about what I could do wile I was there and one of the things I said I was interested in doing, was learning more about Android development. The Android team had been working on the jenkins CI build of their SoundCloud client and were happy to have Duana and me help out on some things there.

Their build consists of a fairly normal maven configuration that gets instrumented by a rake build file. That seemed like a much nicer solution to wrap filesystem and git tasks then trying to do that in maven. As we set to work, I was quite surprised by how much maven knowledge was still in my head. Since I haven't really worked with maven in two years and have been trying to forget its existence for longer than that, this was not what I had expected. (Just to be clear: this is not an implicit endorsement of other build tools.)

We managed to get our initial goal done fairly quickly, which clearly showed the benefits of pairing. Since some of that work was the usual drudgery of trial and erroring our way through different maven settings it was nice to have someone to share in the suffering as we pushed each other forward.

After that we tried to move parts of the build out into its own project. This took more time than I would have liked but eventually reduced the build time by a good minute. Duana quickly calculated that we would need about 1200 builds to make up for the time we invested in doing this. As we calculated a bit more I was surprised by how quickly this would pay back, if you consider that multiple people are running the build, many times a day. This led me to realise that I never really thought much about this in other instances of trying to improve build times and it seems painfully obvious now.

Sure everyone feels the pain when the build duration passes that magical 5 minutes barrier. And we feel compelled to improve things. And even 5 minutes is pretty awfully slow. But if you think about it in more concrete terms it gets highlighted. Improve the build by a minute with 3 pairs trying to build 10 times a day and you already get half an hour. By month's end you have already gained more then a full day. The very unscientific and fuzzy way of coming up with this number is balanced by the fact that you really should be building more than 10 times a day anyway.

Anyway, back to my week. Finally we got to work on a little feature for SoundCloud's android client. Since Android 4.1 notifications can have a different, expanded view when there is enough space in the notification menu. Getting this working for the SoundClound client was a nice, small fearure that cut through a lot of basic Android concepts. I got to do the layout for the big view and, while refactoring, learned how the notifications fit into the concept of Android's services. We decided not to add any extra functionality to it yet, because we felt that we wouldn't be able to finish it by the end of the week.

We didn't get around to writing much tests (shame on me) but learned that Robotium wouldn't be able to test the notifications anyway. But I think we managed to get the feature to a good starting position.

On Friday afternoons, SoundCloud has a demo session where people from across teams show what they've been working on. This seems to have most of the employees present and is followed by drinks and mingling. Duana and myself presented our results and I talked very briefly about what I am doing with this journeyman tour.

Large paintings of your favourite meme

I quite liked this get together and SoundCloud's company culture in general. SoundCloud is decidedly bigger than the other two startups I was at (and they're still growing) and I think it's at a critical stage where it becomes challenging to keep the existing close-knit culture. People were very open and helpful to me and I think there's still that sense of playfulness that makes startups fun. Also, they provide free pistachios and cashews (among other things) which meant I got pretty sick quickly because I have terrible impulse control when it comes to eating those.

While I was there I also got to meet some of the women working for the rails girls summer of code project. They all seemed really eager and doing fine so I was pretty happy to see that the project seems to be going along well.

I also went to another session of xtc Berlin. I think there were about eight people present and I was happy to see that it still lives on. It was also really nice to catch up with some people I hadn't seen in a while. Among which was Stefan Hübner, who reminded me about Euro Clojure, which I promptly bought a ticket for.

Next week I'm heading south to Bretten near Karlsruhe. Nicole Rauch and Andreas Leider have invited me to work with them at msgGillardon. I've never spent any time in that region of Germany so it should be interesting.

Journeyman weeks - week two @ vaamo.de

Wonder what happened last week?

My travel continued as I left Berlin behind and headed out to Frankfurt. During SoCraTes 2013 I spent some time talking about my little journeyman project with Benjamin Reitzammer. He was one of the first that came to my mind as someone I could be working with and learning from. He had recently started working at a start-up called vaamo and so all of that seemed like a good fit and thankfully he was willing to organise getting me there.

Arriving

I had spent the weekend at my parents' house in Düsseldorf and so the short Monday morning train ride to Frankfurt went by pretty quickly. I had good directions and easily found vaamo's offices in the university campus' "House of Finance". Everyone was quick to welcome me and it was a matter of minutes to get set up for GitHub, Travis and HipChat. It took a little bit of installing random stuff (like a JDK... ahem) and I was up and running. I was able to reuse my postgres docker image from the week before, which I thought was pretty cool.

Goethe University Frankfurt

Vaamo is working on an interesting product around making saving money more accessible. I say interesting because it is something I can probably make use of, once it gets released. The tech stack is based on Scala & the Play framework. There's already quite a few people working on the product, including another of my SoCraTes buddies, Johannes Seitz. During the week I stayed in the huge apartment of one of the founders, conveniently located in walking distance of the office. 

Technology

I am happy I finally had some time to actually work with a JVM language other than Java. I've had some basic idea of how Scala works but now I could actually write useful code in it.

I especially enjoyed having Johannes show me a few ways to solve common web app problems in a more functional way. We made heavy use of Option (and Either) monads, which felt a lot more straight-forward in Scala than anything I had experimented with in Java, quite some time ago. (Hopefully it was also easier because I've learned a bit more since that time. :))

"Show me"

In terms of tooling, I was less impressed. IntelliJ was struggling very much on my (admittedly underpowered) laptop while adding very little benefit in terms of refactoring support. Being used to how well the JetBrains stuff works for other languages I have hope that this will improve but was very disappointed with the current state. Working with vim or sublime might have been the better option.

I'm also pretty skeptical about the Play framework (and maybe sbt, if that's what's under the covers). Opinionated frameworks are good for the 80% case but then seem to always end up fighting you when you trail off the beaten path. In this particular case I was unable to get jasmine tests integrated into the build. (I'm certain it's possible to do that if you invest time or know more about Scala/Play/sbt but it shouldn't be this hard to get started.) This was sad because the support for TDD in Play seems to otherwise be pretty excellent.

I did enjoy a different set of tooling very much though. Vaamo uses Travis for continuous integration and I really loved the integration of that with git(hub) and HipChat. As I mentioned here I really like chat rooms to keep a history of ephemeral status information. It's easy to find when you need the information and it's easy to ignore if you don't. And it also makes it viable to have people work in a distributed fashion.

Extra curricular activities

I also enjoyed hanging out with people and seeing more of Frankfurt. Due to a prior bad experience I had a pretty low opinion of the city and I was glad to find out that it's actually not really that bad. :)

Team Vaamo (with Lasers!)

I enjoyed philosophizing about our industry with Benjamin and similarly I was also lucky to run into Olaf Lewitz and Meike Mertsch who just happened to be in Frankfurt at the same time. Once again I feel extremely grateful for the opportunities that make these serendipitous encounters possible!

This is of course also still the case for the overall journeyman weeks tour. I'm really glad I get to be able to do this. Although I did wake up very disoriented on Tuesday morning, not knowing where I was. :)

Next up

As I'm finally finishing up this post I'm sitting in the offices of Mozaic Works in Bucharest. My old friend Adi Bolboaca was kind enough to host me and show me around this beautiful city. In the end it was a little bit too short notice to find someone to pair with for two days but I'm enjoying having some time to walk around the city. And starting tomorrow I'll be busy enjoying catching up with people and hopefully meeting new ones during this year's ALE conference.

Down the street from Mozaic Works

Next week I will be returning to Berlin again to work with Duana Stanley at Soundcloud. They use some pretty interesting and diverse technologies so I'm sure that will be another great week for learning. For my final journeyman week (as of now) I will head south to msgGillardon in Bretten, where Nicole Rauch and Andreas Leidig are kind enough to host me.

Read about my week at SoundCloud here.

Journeyman weeks - week one @ bitbond.net

After publishing my initial post and advertising it on Twitter I was quite overwhelmed with the positive reaction and offers of support. I also realised I had very little clue about how to do this from a logistical perspective.

Nevertheless, I quickly found the first two companies willing to host me. And so I decided to just set out and figure the rest out along the way. This will probably eventually involve having to consult with a tax accountant but that's for another day.

For the first week I returned to Berlin once more. My ex-colleague Gregor Russbuelt connected me to one of his former colleagues, Jürgen Walter, who heads up the technical side of a little startup at https://www.bitbond.net. I was looking forward to do a little Rails work in earnest and possibly also learn more about bitcoin. The latter part didn't actually happen but I'm quite happy with what other stuff I picked up along the way.

Balconia Berlin

Jürgen put me up in a room in his nice apartment in Prenzlauer Berg and we went out to a co-working space nearby each day to do our work. As a first problem we tried to change bitbond's capistrano scripts to work on docker images rather than straight on the remote server. Coincidentally, both Jürgen and myself had recently been pointed at docker.io and thought it was worth looking into further. It turned out that assessment was correct. It's a really neat little toolset. Although it did take us a while to get a clear understanding of how repositories, images, containers, tags and volumes fit together. (Now there's a new tutorial available. Maybe that makes things easier.)

Docker

Docker lets you spin up preconfigured ubuntu instances and run commands inside of them. The idea being that you have a little vm for each of your components. Spin up an image to run the rails app, one to run redis and maybe more for postgres and nginx. Each image is essentially disposable and changes in state to the image are only persisted if you use docker to commit the image. There is a concept of ephemeral, potentially persistent volumes that can be mounted and shared between images, allowing you to store e.g. the data of your postgres server.

Each image keeps a history of the commits made to it and you can run commands against each of these versions or base new images or repositories on them. A repository is maybe best described as a branch. Lastly, new images can also be built off of existing repositories and there is a public directory containing a lot of useful readily configured ones.

There is a limitation to the size of an image's history though, so you cannot run more than 42 (doubt that's unintentional) commands within an image while committing the results. This should usually not be an issue but while we were toying around with our deployments we committed quite a few changes and then were quite surprised when we ran into this problem. It didn't help that the error message is far from helpful.

Our plan then was to

• build new images based on the default docker ubuntu repository, containing the debian packages necessary for rails/postgres/redis/nginx
• in the case of rails, run each capistrano step inside of the docker image, committing the image after each step 
• for the other images, run the respective service, making sure addresses and ports of related services are passed in as environment variables and passed on correctly
• start rails from its image, also passing in information about related services 

(There is another project called coreos that would make the configuration a little more scalable by using etcd to handle service discovery but we didn't look into that)

We wrote a bunch of little shell scripts to help with the individual steps and also experimented a lot until we sort of figured out how docker volumes work. We spent some time working on creating a capistrano plugin to make this available to others. Hopefully it won't take too long to get that ready.

Working on the application

We eventually got to a point where we had a redis and postgres image serving data and a rails and nginx image serving up dynamic and static content, respectively, all working together. At this point we stopped and went to work on the actual application, going through the list of open bugs and features for bitbond.

I was quite impressed with the level of testing being done on bitbond and with Jürgen's TDD workflow and good naming of things. This last bit might sound like a minor thing until you've actually worked on code where people are diligent about keeping things appropriately named. It makes working with the code just so much easier.

And so when we were working on a little bug, we wrote a test that reproduced that behaviour and then fixed the code. When we worked on a new feature, we started with a small capybara script that described how that feature was supposed to work and then implemented against that. It's a very focused, quick and rewarding way of working and I always appreciative when I get to do it.

So we did that and then deployed to production, thereby also making sure we didn't screw up the current production deployment scripts too bad. Ah, the joy of releasing. I can't understand why so many people artificially limit themselves from having that experience more often.

I also got a glimpse of more of coffee script and haml for working on the client side of things. I always liked both of them but never had the opportunity to do any real work with them. At least in the case of coffee script I definitely need to finally change that. Hopefully my recent experiences with lineman will help with that.

Success

So I'm considering the first week of my little journeyman tour a smashing success. I learned new things, got to meet new people and helped ship code. I feel extremely privileged and happy that I have the opportunity to go on this little adventure and do so at such short notice. This community is really great. (And I'm not even sure what "this community" means, exactly.)

If you think what I'm doing here is interesting, please spread the word or invite me over. You can still contact me at dtemme@gmail.com or @dtemme on Twitter. My calendar is still available at this link as well as embedded at the end of this post.

Next up

I'm staying in the broader domain of financial services but moving to Frankfurt, where vaamo is hosting me. I'm really looking forward to working with old twitter and socrates friend Benjamin Reitzammer.

And if you're in Frankfurt and want to grab a beer, please do let me know!

(Write up for week 2 is up now.)

Personal Kanban - auf deutsch

Wie an anderen Stellen schon erwähnt, bin ich ein grosser Freund von Personal Kanban und bin dankbar für die Hilfe, die ich daraus ziehen konnte. Einige der hektischeren Phasen in den letzten Jahren wären mir ohne dieses Werkzeug bedeuten schwerer gefallen.

Nun gibt es das Buch endlich auch in einer deutschen Übersetzung von Meike Mertsch und ich bin froh, dass ich auch diese Version ohne Vorbehalt empfehlen kann.

Journeyman weeks

Having just quit my job and not really knowing what I want to do next I've decided to take to the road. I've always enjoyed Corey Haines' programming tour (wasn't there supposed to be a book?) and recently I came across Rob Ashton's post here. This gave me the idea to just travel around for a few weeks and see what I can learn from other people and hopefully help others build cool stuff.

At the SoCraTes conference I mentioned this to a few people and was generally encouraged. During the code retreat that happened on the Sunday I also realized that I need more practice and that there is so much stuff out there that I'm clueless about and want to learn (most recently Elixir). I was also discouraged a little, as the organisational overhead seems a little daunting.

But anyway, I'm currently homeless and jobless and while I wait for the "next step" I wouldn't mind learning more and keeping my operating cost down. So here is my offer: I will work with you if you can provide a roof over my head and can somehow get me to said roof (I am currently based in Düsseldorf, Germany). I've mostly been working with Java, JavaScript and C# and would very much appreciate the chance to expand more into the functional world. I am very opinionated about writing maintainable software and even more opinionated about what software not to write at all.

I hear you say:

Daniel, that's great! How can I help?

Thanks! There's three things, really: 

• If you would like to work with me for a week or two please contact me at dtemme@gmail.com or @dtemme on twitter
• If you have any tips on being self-employed in Germany and how to arrange something like this, I would like your input
• If you think you might know anyone that is interested in pairing with me, please spread the word

I will initially run this little experiment until the middle of September. I've created a calendar showing my availability here.

Updates

• Updates for week one
• Week two at Vaamo
• Week three at SoundCloud
• Week four at msgGillardon

Three tales of browser-based test suites [Part 2] - test harder

In Part 1 I wrote about my initial experience testing with Selenium. In this part I will write about what I learned about browser-based tests on my first project working for ThoughtWorks. It was a step up in terms of coverage and complexity and that also brought its own set of problems.

2012, a year

This thing proved to take quite a bit of time to write. But 2012 was a very eventful, challenging, rewarding, entertaining and educational year.

Like last year I'm overcome by massive gratefulness and joy for having been able to connect to so many nice people throughout the year. 

So, in very brief stats: I worked on 3 projects, spoke at 2 conferences, did more code retreats, and moved to South Africa.