The idempotent command

The great thing about the server architectures we have these days is that everything is scalable (if you set it up correctly). The hard thing about the server architectures we have these days is that everything is scalable (if you set it up correctly). Yeah, I know.

One of the things you may run into these days (and that I had to solve this week) is that these days we provision all servers similarly (or according to their role). This may also mean that you provision several of your servers to run the same cronjobs at the same time. However, some tasks may not be run multiple times, and especially not at the same time. In a symfony project I'm working on, I was tasked with making sure some of the cronjobs would only be run once, even if started on several servers at the same time.

Adding the locking

My initial idea was to add a locking system to all commands that had to be idempotent, but I felt this was a bad idea: Having to add similar code to several different classes did not really make sense to me.

While looking for a different option by searching for Symfony Command classes and events I came by this blogpost by Matthias Noback. While his specific use case in that blogpost is different, it inspired me: I simply needed to use the events console.command and console.terminate. I would be able to hook into those events to initially set the lock and then on termination release the lock.

The lock library

The next step was to find the right type of locking. I looked around for libraries that could do locking. In that process I came by the very recently pushed symfony/lock. Unfortunately that was a bit too fresh for me to use. Eventually, I settled on arvenil/ninja-mutex, a nice and simple library that can do Mutex locks on a variety of backends. We went with the Redis backend for our locks.

Deciding when to lock

The thing is: I don't need all commands to do locking, I only need specific commands to add and release locks. My initial plan was to simply create an array of class names in the listener, but that did not feel right. This meant that every time we'd add a new command that needs locking, we'd have to update the listener.

Another option would be to keep track of a list in the configuration, but that similarly did not feel right.

I ended up going for an implementation with an interface. The interface, that I called IdempotentCommand, contains just a single method. The method that needs to be implemented is getIdentifier(): string, which would return the identifier used for the lock.

The listener

Time to write the listener. The listener needs to listen to two events:

  • console.command is the event triggered by starting a Symfony Command. This is where I need to create the lock.
  • console.terminate is the event triggered by a Symfony Command ending execution. This is where I need to release the lock.

The listener is pretty simple. It gets the MutexFabric class from arvenil/ninja-mutex as a constructor argument that it can use internally. It then implements two methods, one for the first event and one from the second event.

Creating the lock

public function onConsoleCommand(ConsoleCommandEvent $event)
{
    if ($this->shouldBeHandledIdempotently($event->getCommand())) {
        $this->acquireLock($this->getLockName($event->getCommand()->getIdentifier()));
    }
}

private function shouldBeHandledIdempotently(Command $command)
{
    return $command instanceof IdempotentCommand;
}

private function acquireLock(string $name)
{
    $result = $this->mutexPool->get($name)->acquireLock(1000);
    if (false === $result) {
        throw new ProcessLocked('Process '.$name.' is locked and can not be executed');
    }

    $this->acquired = true;
}

private function getLockName(string $commandName): string
{
    return 'command-'.$commandName;
}

The onConsoleCommand() method is linked to the console.command using a service tag:

- { name: kernel.event_listener, event: console.command, method: onConsoleCommand, priority: 1 }

First we check whether this command is required to be locked. If so, we try to acquire a lock. If we succeed, we keep track of that by setting a local property (yay, we are the actually executing process). The purpose of this property is to prevent a second (or third) process that is started to release the lock when it ends before the initial process is ended. If we can not acquire a lock we throw a ProcessLocked exception to quit execution immediately.

Releasing the lock

Once the main process has ended, it needs to release the lock. To do that, we have a second method in our listener class:

public function onConsoleTerminate(ConsoleTerminateEvent $event)
{
    if ($this->shouldBeHandledIdempotently($event->getCommand()) && $this->acquired === true) {
        $this->releaseLock($this->getLockName($event->getCommand()->getIdentifier()));
    }
}

private function releaseLock(string $name)
{
    $this->mutexPool->get($name)->releaseLock();
}

Here we simply check whether this command is supposed to lock and whether the current process has acquired the lock. If so, it releases the lock. That's all.

Making things lock

Now the only step left is to find the right Command classes that need to be locked, and make them implement the IdempotentCommand interface I defined at the start. These are now automatically picked up by the listener to set and release a lock accordingly.

If I now start the same command twice at the same time, only one of the commands will actually run, the other one will be stopped by the exception and won't run at all.

A small extra lesson

During the process of building the second listener method, some weird things were happening. I would get an error at the end of the Command execution about the locks, and whatever I did it seemed the second listener was never triggered. After a lot of searching I found the error to be a single missing comma. I had accidentally typed:

- { name: kernel.event_listener, event: console.terminate method: onConsoleTerminate, priority: 1 }

This is still valid YAML, so Symfony did not complain about it, but because of the missing comma between console.terminate and method: it did not pick it up to be a listener. The devil is in the details, and it took me a while to figure this one out.


What is WeCamp all about?

Recently I got an email from someone who was interested in coming to WeCamp, but needed some more information to help convince their manager. I wrote a big email trying to describe WeCamp and our ideas of what WeCamp is (or should be).

Since this may actually be useful to more people that want to come to WeCamp but need to convince their manager, I'm posting that email here as well.

The idea behind WeCamp is that attendees will go through the whole process of a software development project. When the attendees come to the island they will be assigned in teams with 4 other people that they most probably not know (we try to split up people who work for the same company or live in the same region). Every team of 5 people will be assigned one of the coaches, and this coach will support the team in finding their own way in the project. These coaches are specifically NOT teachers, but instead people who will support the team, because we want the team to find their own way most of the time.

Once a team is formed and assigned a coach, they find a working place on the island, and they'll have to think of a project to work on. The team will have to think of a project themselves and decide on what is realistic to build given the limited amount of time (usually about 3 days of development). Once they've decided on their MVP and created their initial planning, they start working on their project.

Each team is free to choose a methodology for development. While we've so far seen no waterfall-style development, we've had several different methodologies from full scrum to kanban and variations on both because the team decided that was the best approach. One thing all teams have though: A central stand-up right before lunch where each team can share their progress, the problems they've run into and their lessons learned. In that way we try to get everyone to learn from everyone else. During the fifth day, each team will present their project as well as their lessons learned while doing the project.

To balance out all the hard work, we have some social events during the week. We have a game night (with board- and card games, no electronic games!), we have a BBQ and we have a "pirate game", an activity where a pirate comes to the island with some friends to have everyone do some assignments. When there's no special events at night, there's always the option for a drink with fellow attendees around the camp fire. Everyone sleeps in tents with beds in 'em, on the island. The price for the ticket is all-in: It includes all drinks and meals so attendees don't have to worry about anything during the 5 days on the island.

When we first started WeCamp 4 years ago, we meant to start a technical event. We had a lot of experience with conferences as an inspiring place for new tech, but felt we never had time to actually play with that new tech. That's what we initially aimed for with WeCamp: A place to actually play around with new tech. It quickly turned out though that all the tech is cool, but WeCamp was about more than just tech. It was about personal development. About learning a lot of soft skills next to all the tech: communication, teamwork, planning, making decisions, presenting.

The coaches will have private conversations with all team members, and together they will create a personal development plan with attendees. Some time after WeCamp coaches follow up with their team members to see if the goals set in the personal development plan were reached, or at least are being pursued.


To Exception or not to Exception

I recently found myself in a discussion on whether or not exceptions could be used to control program flow in software (specifically in PHP applications). That triggered me to post a tweet:

Exceptions should not be used to control expected application flow. Discuss.... @skoop

This triggered quite a bit of discussion, which gave me a lot of input on this topic. I want to thank everyone who joined that discussion for their input, which was really valuable. In this blogpost I'll do a summary of the different arguments in the discussion, and give my opinion on this.

What is program flow?

First of all, what exactly do I mean with program flow and using exceptions to control program flow. The reason for the discussion was an exception thrown in a persistence layer in the situation that there were no results in a findBy* method. This is a slightly different situation from for instance errors with database connections or API connections, things that can be expected but are not meant to be happening. When you do a findBy*, you're effectively searching, meaning 0 results would be a valid situation. This also was reflected in the discussion.

@mvriel @skoop @rdohms Find implie Search. Search implies zero results is a valid output. Excepting in a Find-er seems odd to me.Make it a Get-er or wrap in Option @n0x13

This triggered a whole discussion on whether for instance findById is actually searching and whether it would be a valid use case that this would return 0 results. For instance:

@n0x13 @skoop @rdohms Finding one item specifically by ID implies that it exists; non-existance is equal to an error 400. @mvriel

Why exceptions?

A great definition of what an exception is and when it should be using was given by Chris:

@thomas_shone @skoop Exceptions are only where the code throwing the exception cannot deal with it. Calling code can always expect the exception and cope! @choult

I quite agree with this definition of exceptions. If your code can not deal with a certain situation anymore (such as a missing connection to the database, or an error 500 response from an API) then it should throw an exception. Basically, when something should be there and is not there. In all other situations, your program flow should handle "errors" by itself.

What is the intent of your code?

I had not really expected this when I first asked the question, but eventually I think the discussion stopped being about exceptions, and started being about naming and intent. Now, we all know there are two things that are extremely hard in software programming:

  • Cache invalidation
  • Naming things
  • Off-by-one errors

Let us focus on the middle one: Naming things.

Naming things is hard. It is extremely hard. It is so hard people do talks about the subject. But in essence, naming things is easy. Names should be clear, descriptive and describe the intent of the code. Yeah, that sounds easy, but once you start to think of what the right name is, it gets harder. Actually finding the right name is extremely hard.

So, let's go back to the example of findById(). Given several tweets in the discussion, different people interpret this method and its intent differently. There's basically two interpretations:

  • find implies search, which means you're going to search for a record with the given ID. When you search, one of the options would be that no results are found
  • ById implies that you're asking for a specific record, because ID is usually a unique key. If you know that key, then the record must exist. If it does not exist, this is an exceptional situation

And both interpretations are valid. Which basically means the naming is off. Jaap has a good solution for that:

@mvriel @skoop @rdohms Find can return null, getbyid should throw an exception @jvotterdijk

I like this idea; when you search (represented in this tweet by find) 0 results is a valid situation, but when you getById() you expect it to be there, so that may result in an exception if it is not there. Nicolas later confirms this as well:

@skoop Agreed, exceptions shouldn't be used for flow control. Although valid when ex: has(): bool, get(): Object (throws Exception) @nicholasruunu

So, naming things...

... is still very hard. But it is extremely important. So please think hard before you name things, and avoid ambiguity. Always choose names that are clear and that communicate the intent of the code you are writing. Because it isn't wrong to throw an exception when you getById() and you find nothing, but when you searchById() it is OK to get an empty result which should not result in an exception.


Sculpin and Docker

I've been running this blog on Sculpin for quite a while now, and I'm quite happy with how that works. We've been in a process of migrating some of our websites off a standard VPS towards a setup with Docker, Gitlab CI and Rancher. I've now migrated some websites, which is relatively easy, but most of those sites were dynamic PHP websites. Migrating a site that generates static HTML and running that is a slightly different thing. Here's how I ended up doing it.

The old setup

First, let me quickly describe my old setup. I had set up Sculpin on my webserver and put only the blogposts into a Git repository that was hosted on Github. After committing and pushing a new blogpost, I'd manually run a shell command that would generate the new static version:

ssh -t [email protected] 'cd /var/www/vhosts/leftontheweb.com/site/source/_posts;git pull origin master;cd /var/www/vhosts/leftontheweb.com/site;php /var/www/vhosts/leftontheweb.com/site/bin/sculpin.phar generate --env=prod --url=http://leftontheweb.com'

This worked, but would be a bit harder with a new setup. Besides, why would I want to do it manually when I could have it all be done automatically after pushing a new blogpost?

The new setup

First, let's have a look at the global setup that we have:

  • Gitlab for Git repository hosting, we're using Gitlab pipelines to build and push changes
  • We've got droplets on Digital Ocean which contains our Docker/Rancher setup
  • Our production setup is managed by Rancher

So once we push some changes to master (I mean: merge a merge request) a Gitlab pipeline is triggered that builds the Docker container, and pushes the new container to Rancher. All of a sudden, the website is updated.

Now that we've got that basic setup described, let's have a look at how I've set this up with Sculpin.

My initial attempt

In my initial attempt I started with the base Nginx container and started working from there.

FROM nginx

I started running into some issues with installing PHP, so I decided to approach it from the opposite side.

My second attempt

I started over by using the base PHP7 container

FROM php:7.0-cli

So, first things first: I'll need to install Nginx, because eventually I need to serve my static website to anyone wanting to visit my site.

RUN apt-get update
RUN apt-get install -y nginx

For installing Sculpin I'll need Git and the Zlib library, so I'll also install that.

RUN apt-get install -y git
RUN apt-get install -y zlib1g-dev && docker-php-ext-install zip

Now, let's install Composer so I can use that for installing Sculpin.

ADD https://getcomposer.org/installer /tmp/composer-installer.php
RUN php /tmp/composer-installer.php --install-dir=/usr/local/bin --filename=composer && \
    rm /tmp/composer-installer.php

OK, all prerequisites are installed, let's install Sculpin.

RUN git clone https://github.com/sculpin/sculpin.git /usr/lib/sculpin && \
    cd /usr/lib/sculpin && \
    composer install && \
    ln -s /usr/lib/sculpin/bin/sculpin /usr/local/bin/sculpin

Now I have a basic Sculpin install in /usr/lib/sculpin. I now want to customize that installation with my own custom information. I've structured my Git repository in such a way that I can easily copy my custom configuration into this base installation. My repository layout is:

/app
/posts
/source

app/ contains a config/ directory that contains the Sculpin configuration files (sculpin_kernel.yml which contains the URL structure and sculpin_site.yml which contains some basic site information, my Google Analytics ID and the Disqus configuration).

posts/ contains all my blogposts. This is basically what I imported from my old Git repository. All the blogposts are in here with any images that may be needed for the blogposts.

source/ is my own source directory for Sculpin. It contains my theme and custom pages. As we'll see later, this is also where the contents of posts/ will end up being placed, but I wanted the blogposts to be more easily accessible, so I've seperated the posts/ directory.

So, given that structure, I can now copy the contents of those directories to my container.

COPY app /usr/lib/sculpin/app
COPY source /usr/lib/sculpin/source
COPY posts /usr/lib/sculpin/source/_posts

I copy the app/ directory to the Sculpin installation, I copy source/ to the Sculpin directory, and now I copy the posts to the _posts/ to the source/ directory. Now I've got everything I need to generate the static website using Sculpin.

RUN cd /usr/lib/sculpin && bin/sculpin generate --env=prod --url=https://leftontheweb.com

This will call Sculpin to generate the new static version of my blog. The new version is generated in the output_prod/ directory in my Sculpin installation. Of course, this is not the Nginx document root, so I need to make sure I can expose the static site using Nginx.

RUN rm -rf /var/www/html
RUN ln -s /usr/lib/sculpin/output_prod /var/www/html

Firstly, I remove the default Nginx document root. After that, I create a symlink to the output_prod/ directory. Now I can serve my static site. The only thing that is left is to ensure Nginx gets started.

CMD ["nginx", "-g", "daemon off;"]

This starts Nginx and makes sure the site is now being served. Everything is up and running!

Credits

I need to give some credits of course, because I have used some sources for inspiration. First of all, I used the gitlab runner sculpin to check some of steps in the Dockerfile in my second attempt. Also, my standard "HELP! IT NO WORK!" person Mike gave me some insights in to how to solve some of the problems I encountered along the way.


Upcoming conferences and usergroups

It's been a while since I've done a post like this, but given the focus of the upcoming events I thought I'd give a small overview of my upcoming speaker appearances.

Why?

For 2017 I want to once again focus a bit more on building bridges. I've done this a couple of years ago but limited myself again to mostly generic PHP conferences in the previous couple of years. Others, such as Jenny Wong and Larry Garfield, are trying to build bridges between the generic PHP community and their respective communities (WordPress and Drupal). Inspired by the work some of these people are doing, I wanted to join in again.

No more generic PHP conferences then?

Of course I'll be at generic PHP conferences and usergroups. But I want to also focus a bit more on what is out there beyond generic PHP and the popular frameworks.

Events

So let's have a look at the upcoming events where I'll be speaking

WordPress Meetup Enschede

On Match 14th I'll be speaking at the WordPress Meetup Enschede, a local WordPress usergroup for the Dutch city of Enschede. I'll be doing my 'Level Up Your Team' talk on ways to improve your knowledge.

Dutch Joomla!Days

On April 2nd I'll travel all the way to Zeist (which, really, is just around the corner from where I live ;) ) for the Dutch Joomla!Days. I've spoken at this event a couple of times already and have always enjoyed being there. This year, I'll be doing two talks: I'll be doing the opening keynote on Sunday on the need to keep learning, and right before the closing keynote I'll be doing my 'Level Up Your Team' talk on ways to improve knowledge.

Dutch Web Alliance freelancers meetup

On Monday April 3rd I'll be helping organize and host a free meetup for people in The Netherlands who either just started freelancers or want to start freelancing. I'll be one of several Dutch Web Alliance members that will be sharing their knowledge and experience about freelancing.

PHPKonf

On May 20 I will travel to Turkey for PHPKonf. In Istanbul I will speak about working with components and how you can build your own framework using components with my 'Build your framework like Constructicons' talk.

More?

If you run a usergroup or conference that focusses not just on generic PHP but on specific applications, such as WordPress, Joomla!, Drupal, Magento or anything else and you are interested in a speaker from the generic PHP community to come give a talk, please do get in touch! For usergroups within the Netherlands, I should be able to travel to you without any issues. For events outside of The Netherlands, I'm OK with a travel reimbursement and a simple hotel. I'd love to make some more connections outside of my PHP bubble, and look at the problems other developers are solving and share with them some of my own knowledge and experience.


Pragmatic coding

Can you write me a simple script that fetches some information from an RSS feed and displays the titles? Like, just write me that script in a couple of minutes. I don't care about tests, quality, etc. Just get me the information, quickly.

I'm curious how you would handle this request. A couple of years ago I was asked a similar question. Not in a professional role, but for a hobby project that has nothing to do with PHP. There was a website, a simple website, and it needed some information displayed. Nothing fancy right? I can do that.

But as I sat down to code this little script, I came to the conclusion that it took me way too much effort to start thinking in a pragmatic, simplistic way. I started thinking...

I'll want to use a library

and...

If I want to use a library, I'll need to use composer to install it

and...

If I install that library then it would be useful to add some other components, to easily manage configuration, dependency injection, etc.

Stop!

Wait, what is the actual question here? There is a pre-existing website, a codebase that I have nothing to do with so far. It is not built using any of the standards I use in my dayjob, and really, I should not care too much about it. It is a simple website, for a hobby. They ask me to simply add something simple. Should I really want to introduce libraries and components, tools like Composer?

Hammertime!

As it turns out, I seemed to have forgotten how to do simple scripting. I was so used to installing a full-stack framework, adding all kinds of libraries, that the simple task of writing a script that would just do one simple thing was getting too hard for me. Instead of just getting a hammer to get a nail in the wall, I wanted to rebuild the whole wall because it was not initially built using the same type of wood I was used to. Do I really want to take on a huge reconstruction project in my house just to hang a painting? Why is it so hard to open my toolbox and grab that hammer?

Back to the basics

Since that moment I've tried to challenge myself every once in a while by writing simple scripts instead of full applications. For instance to automate simple tasks on my laptop (see: automation in this blogpost). Other challenges were simply to write something I had no use for. Just to get back to the basics of writing scripts instead of applications.

It wasn't easy, but I've learned a couple of nice lessons.

First of all, PHP as a language actually has a lot of stuff that we don't really need userland code for. Yes, the components and frameworks will make the task easier, but there are so many functions in PHP that can help you achieve a task that you don't really need a library for. RSS, for instance, can easily be parsed using SimpleXML. And I really don't need to have domain objects or entities per RSS item. Need a quick database connection to get or save some information. There really is no need for Doctrine or Propel, PDO will do what you want. And that's just two examples. There's so much more.

Second: Bootstrapping a project using Composer and frameworks works well for big projects, but if all you need to do is write a simple script, the overhead is just way too big. I had somehow forgotten that, or was just ignoring the option of just starting. I'm sure this had to do with the fact that I have been mostly working on big projects in the past 10+ years, but still... why did I forget?

And last but not least: Being pragmatic can be really useful. Sure, we all want to write the absolute best and most beautiful code ever, but sometimes you really shouldn't care. Sometimes you just need to be pragmatic, make it work, and forget about it.

Build from the basics

One of the "simple scripts" I wrote a couple of years ago ended up becoming bigger. I had written it quickly, just to make it work, but as it grew, I decided to refactor it a bit, add a bit more structure, and introduce some libraries along the way.

An important lesson I learned from this approach: We don't need the full-stack for everything. Components can by itself be really useful. Not every project I do automatically gets a full-stack framework installation these days. You can easily build your own framework for your project using components. I even have a talk about that these days, and I plan to write more specifically about that at some point.

Challenge yourself

So let's go back to the original question:

Can you write me a simple script that fetches some information from an RSS feed and displays the titles? Like, just write me that script in a couple of minutes. I don't care about tests, quality, etc. Just get me the information, quickly.

I'd say: go for it! Challenge yourself to do this, without libraries, in an hour. You can do it.


Best practices on bundles in Symfony

I know I'm late with this, but I've only recently really been discovering the podcast as a way of keeping informed. My daily commute is quite long because of traffic jams and a good way to keep yourself occupied with something useful is to listen to podcasts.

On one of my recent commutes I started listening to the Sound of Symfony podcast. As I had just discovered that one, I decided to listen to their most recent episode, which is on best practices for bundles. I quite disagreed with what was being said in the podcast. I started voicing my disagreement on Twitter but quickly decided that 140 characters is not enough to really explain my disagreement. So here's a blogpost.

The Best Practices documentation

First of all, I need to say something about the Symfony best practices documentation. At some point the symfony project released an "official best practices book", and in the podcast they refer quite a bit to this book. The book is being referred to as "this is how you're supposed to be doing Symfony" and "In 95% of the cases, this is how you do it". Let me state that I sincerely disagree with this. I think the practices described in the Symfony best practices book are not really best practices, but instead they are a good starting point. In my opinion, the Symfony best practices should've been called the "Getting started with Symfony" book.

I also disagree with the fact that you should be setting up all your Symfony projects in the same way "because it's easier to understand for new developers". If you do a lot of similar projects it makes sense to set up your projects the same way, but it really depends on the project. Look at the project, not the developers, to determine how you set up your Symfony installation. Code organization is, to me, much more important than how long the on-boarding process for new developers is. On-boarding is a one-time investment in a project. A good code structure will help you in the long run as well.

The AppBundle

All Symfony documentation (and also the Sound of Symfony podcast) assume your project contains a single bundle: The AppBundle. This bundle is supposed to contain all your application code. ALL your application code. When you're working on a very small application, this perhaps makes sense, but just about any project I've ever worked on, this is not realistic. There's a lot of code in most applications. Also, putting it all in a single bundle makes it very unstructured and bloated. I personally really dislike that.

Uncle Bob

While I was at ZendCon this year, I attended the keynote by Robert C. Martin. His keynote went into a lot of different programming best practices, and one thing really resonated with me: Code structure.

At some point during the keynote the slide contained a directory structure of a project. It was pretty clear from the directory structure that this was a project built on a specific framework, but looking at that structure, there was no way to see what the project was actually about. One of the statements Uncle Bob made was that the structure of your code should not show too much about your framework of choice, but should communicate to you what the code is actually doing.

Structure in a Symfony project

Now, the basic directory structure for a Symfony project will always show you that it is a Symfony project. However, you could argue that the project root is a basic filesystem, and the directories that a Symfony project have reflect the purpose of the underlying directories. app/ will contain application-specific stuff, web/ is your document root, and src/ will contain your source code. As such, I think it's fair to apply the lesson Uncle Bob was teaching specifically to the src/ directory. If that directory only has an AppBundle directory, you have no idea what the project is about. The only thing you know is that you're looking at the code for an application. Well, duh.

Bundles to structure your code

So the first thing I'd suggest is to use bundles in Symfony to add a bit more structure to your code. Name them after what it does and your code is already a lot easier to understand. If you have a webshop, you could have a ProductBundle, a CartBundle and a CheckoutBundle for instance. If you build a CMS-based website, having an AdminBundle, a PageBundle and perhaps a ForumBundle makes sense. I can take a look at your src directory and immediately see what goes where.

What about your domain code?

Now there is a slight issue with this approach. Because what do you do with your domain code? The code that contains your business logic, that is not tied to your framework but to your business. Because we don't want to tie our business logic to the framework, putting the domain code into the bundle feels wrong. We want to have it seperated. Since the src/ directory is the place to put all custom code, this would mean we'd also have to put that in the src/ directory. Following that logic, we'd now have the following directory structure:

/Cart
/CartBundle
/Checkout
/CheckoutBundle
/Product
/ProductBundle

While not necessarily wrong, this feels wrong. This feels like duplicating things, or splitting things that somehow belong together. There's a solution for that.

Adding more structure to your project

In a project I recently worked on I was introduced to the concept of Bounded Contexts. The structure of this project was really nice, because we wouldn't even leak the framework implementation in the first level of the src/ directory. Taking the webshop example I had because, the src/ directory would look something like this:

/Cart
/Checkout
/Product

If you're new to this project, you'd immediately understand that this is a webshop, and that it has code in there for a cart, the checkout process and products.

Now, within every directory, the same structure could be found:

/Cart
    /Bridge
    /Bundle
    /Domain
    /Tests

Bridge

The Bridge/ directory and namespace would contain code that would connect to external systems. It provides the bridge between our domain objects and the storage/persistence systems we were using. This code is not part of our domain code, but not really part of our application either.

Bundle

The Bundle/ directory and namespace would contain our actual Symfony bundle. It contains all the code that one would expect in a bundle: Controllers, configuration, templates and any other code required to tie our domain code to our framework implementation.

Domain

Of course, the Domain/ directory contains all our domain code. This may contain interfaces for certain code, domain objects (usually POPO's that reflect data structure), specific exceptions for our code, etc.

Tests

This directory will contain all the tests that we've written. This could be unit tests, but also integration tests. Anything that tests our code will be here.

The importance of structure

As a consultant and freelance developer I go into a lot of companies. I either develop with an existing team, make changes to an existing (and sometimes unmaintained) codebase, I do code reviews or help in improving a codebase. I agree with everyone who says that having a common structure in a codebase will help with getting to know the codebase, and as such I understand people going for the standard structure that a framework provides. While I prefer Symfony in many situations, I am not tied to a single framework. Over the past two years I've worked on projects built on top of Symfony, Zend Framework, Laravel, Slim, Code Igniter and Silex. I've worked on projects that adhere to their framework default structure and also on projects that have a custom structure (such as the structure above). For me, it is a lot easier when the structure reflects what the application is about instead of what tool it is built on top of. I am therefore extremely happy to have worked on a project with the above structure. It has taught me a new way of structuring code that makes more sense than anything I've done before. It was a good confirmation to hear Uncle Bob describe a similar structure in his keynote at ZendCon.

I can understand if you opt to go for the default structure of the framework you're working with, but perhaps it is good to think a bit longer about it. While the initial learning curve may be a bit more steep when you go for another structure, it may actually make things easier in the long run. And if you document your choices well, everyone will have a place to check for reference.

Keep on learning

I've said it before and I'll say it again: It is important to keep on learning. One of the things I've learned this year is a new way of structuring code, which is part of the Domain-Driven Design approach. I've been learning about DDD a lot this year and I already realize that I'll need to learn even more about this. If the concepts in this blogpost are new to you, then you probably want to learn about this as well.


24PullRequests 2016

Last year I already wrote about different initiatives in the period leading up to Christmas. In one of my talks this year, Level Up Your Team I've been discussing many ways of learning, and for this year I want to highlight one of the initiatives that I participated in last year, and again am trying again this year: 24 Pull Requests.

What is it?

The idea is that you send (at least) one pull request to an open source project in the days leading up to Christmas, starting on December 1st. The website will keep track of your progress and give you a list of suggested projects.

Why would you do it?

The most obvious reason is to contribute something to open source. If you, like me, earn most of your income by using open source code in your projects, this is most certainly a good reason to contribute to open source. But there's more...

When do you learn most? I learn most when I am challenged, when I am pushed out of my comfort zone. This is one of the reasons we get so much positive feedback for WeCamp, because we push people out of their comfort zone in every possible way. That same thing applies with 24 Pull Requests: You are forced to send a pull request to an open source project 24 days in a row. While they can be 24 PRs to the same project, it works better to have a look at the suggested projects and try to contribute to multiple projects, preferably ones that you don't know (that well).

Is it easy?

No. Not every project has a low hanging fruit or easy pick tag, and even if they have them, it may not be easy for someone new to the project/codebase. Also: Actually writing a PR every single day for 24 days is not easy. At least if your calendar is as busy as mine, it's not easy.

Actually last year, I completely failed. I can't find the actual stats anymore on the 24PullRequests website, but I think I sent maybe 5 or 6 pull requests. Time was mostly the reason for this failure, but I tried. And even though I didn't succeed in sending 24 pull requests, I did push myself to contribute to open source, and I did learn about some projects. For instance, I had not looked at Disco until last year. And this year, I've already contributed a simple typo to RMT, which I did not know until yesterday, and I've contributed a bugfix to Bolt, a CMS that I've been using a lot in the past year.

Join in!

So go ahead, start contributing!


Get rid of -m

I can't remember where I actually first read about this little developer's lifehack, but I've been using this for a while now and really like it: Not using the -m option anymore when commiting with git.

What I used to do...

I use Git nearly exclusively on the command-line. My main reason for that is being burned by a version control integration of an IDE somewhere in the remote past. Since then I don't really trust any version control integration anymore and I want to have the full control over my version control tool of choice, these days mostly Git.

Committing changes had a certain never-changing process:

git status

To check which files I've touched.

git diff

I'd diff each file separately to see if the changes in that file belong to the stuff I currently want to commit.

git add

I'd add every file that belongs to the current change.

git commit -m ""

Then I'd commit the staged changes with a small comment about what the change contains

The main problem with the above flow was that the -m promoted short and simple messages. And tempts a bit too much to write bugfix or just a little change in the controller. It is just too tempting. Also, even though I'd just changed on which changes there are in the files I'm committing, I wouldn't have it all in my head anymore.

Get rid of -m

Some time ago I'd read a tip from someone that said "just don't use -m. It took some time to process the usefulness of this tip in my head, but eventually I decided to try it. It works!

The last step in my flow now is:

git commit

This opens up (in my case) vi, where I see a list of the staged files and I can type a commit message. Because I'm not on a command-line anymore but in an editor, I'm less tempted to write only a short commit message. Instead, I can describe what is in the changeset. And since I have the list of changed files there as well, it's easier to remember what I changed and for what reason.

It's a simple lifehack for a developer to write better commit messages, and it works (for me)!


The Serendipity Machine

I recently finished a book that has been in my possession for quite some time already, The Serendipity Machine by Sebastian Olma. Subtitled A Disruptive Business Model for Society 3.0 sounds very buzzword-y and startup-y, but despite the subtitle it was a very good book.

I've been a big fan of the Seats2Meet concept for quite a long time. The idea of getting a free working place including free lunch in exchange for sharing your knowledge and experience with others really appeals to me. I was first introduced to the concept quite a while ago by a colleague. Despite my business model causing me to work on-site with customers most of the time I've worked at Seats2Meet on several occassions, and with PHPAmersfoort we also organized meetups at one of their locations.

Some of the things discussed in the book really already describe what happens naturally in the PHP community. Especially things like sharing knowledge is happening a lot. One of the things where I think we can still improve though is the facilitating meetings. On the higher level, I mean to combat the isolation of the "islands" inside the greater PHP community, such as Drupal, Joomla! and all the different framework communities. But on a lower level, I also mean the people attending usergroup meetings and conferences. In a way, initiatives such as FIG provide a way of doing this, but on that smaller, lower level, we fail to do this, perhaps also because we fail to facilitate this. The fact that the majority of PHP developers is pretty introvert does not help, but all the more reason to facilitate these meetings.

So how do we facilitate this? you may ask. This is where we can have a look at how Seats2Meet does this. Their platform, where all visitors of the Seats2Meet locations sign in, allows people to "tag" themselves. In their profile, they list the topics they're expert in. For instance, my profile for Seats2Meet contains the following:

Tags in Seats2Meet profile

When you've book yourself a seat, you'll see the tags of other people that are there together with you, and you can look for the specific knowledge that you need. This is an active facilitation of meetings between people that have something in common. If at some point during the day while working at Seats2Meet you find yourself in need of some specific knowledge, all you have to do is check their system to see if someone with that knowledge is also at the location you are at.

The hallway track

This is where the hallway track at conferences becomes useful. While most people are in a session, staying in the hallway is a great way of meeting other people that made the same choice of a skipping a session. These kinds of random meetings are harder during breaks: There are so many people in the hallway at such a moment that it's hard to really decide who to talk to outside of the people you already know, also because everyone else is also talking to people they know. During a session though, there is little choice. So next time you skip a session, just talk to someone else that made the same choice. Even if you can't help eachother at that moment, you've expanded your network (or value network as Sebastian calls it in the book).

Conferences may facilitate these meetings by allowing "tagging" of people. Either through an online platform or by allowing people to add "tags" to their name badge. I can't tell you exactly how this should be done, but I do feel this is something that can be done in a better way, to facilitate these "random" meetings. To enhance the serendipity.

If all this sounds good, I would recommend you picking up a copy of The Serendipity Machine. It is an inspiring read, especially for those that are active in the PHP community. And if you're near a Seats2Meet location, book a seat for a day every once in a while. It's free and it's awesome.

The Serendipity Machine (ISBN 978-90-8169, November 2012, Creative Commons BY-NC-ND) is also available as free PDF download.