Below is an extract of the continuous deployment strategy that I and my team used at Hulu over the last year and a half or so. It was extremely successful for us and allowed us to do continuous development, where as soon as we have something production ready, it hits production with little friction. The results of us doing this were that every developer was empowered to ship at their own pace, as often as possible. And because developers were in charge of the shipping process, it forced all of us to rely a lot more on automation & testing for pre-verification.

We’ve had days where we shipped to production more than 30 times in a 24 hour period, and we rarely had bad push incidents. Even when we did have bad releases, the fix was always just a push away :).

Deployment model 

Branching model

We are using a branching model inspired by ( http://nvie.com/posts/a-successful-git-branching-model/). We use development integration branches and release branches. The integration branches are shared development branches for sharing code, and release branches are branches which get deployed to our various environments. Code changes should originate only from integration branches, and then be propagated to the deployment branches. Both branch types are quickly outlined below.

Integration branches

origin/develop

The develop branch is a branch for integrating feature-complete code or fixes. It is considered unstable, but that does not mean broken . Run tests before committing to the develop branch. Most of us use develop for their day-to-day development. Most changes should originate from develop or if small production bugfixes are done off of master, they should quickly be integrated back into develop.

origin/master

The master branch is a stable branch. All commits to master go through CI server, and tests are run after every push. The master branch is just an integration branch, which means no deployments are done from it. Changes should be pushed to master, generally only after being code reviewed and pushed to the test environment and tested there.  

origin/hotfixes

The hotfixes branch is a stable branch off of master for integrating small bugfixes that need to make it to production without tagging all of our development changes with them. All commits to the branch go through CI server, and tests are run after every push. The hotfixes branch is just an integration branch, which means no deployments are done from it. Changes should be pushed to hotfixes, generally only after being code reviewed and pushed to the test environment and tested there.  

origin/cr

The cr branch is an auxiliary branch, used for pushing changes pending code reviews. This branch is useful in cases, for example, when remote teams would make changes during the night, and would then submit for code review, and which the next day, the core team can code review, rebase to head of develop and merge in, while remote team is sleeping. This is very useful, as during the day, there are number of changes that happen in the integration branches, and not integrating the code early, could cause merge conflicts. The use of this branch tries to reduce turnaround of code-changes merged into the integration branches.

Release branches

origin/test

The test branch is a release branch for our test environment. The test environment is generally for testing development changes in a real environment, and should not be used/referenced outside of the team. The environment should be deployed via CI, which will automatically run all available tests before deployment. Code changes should not originate from this branch. All changes in this branch should come from a merge of the origin/develop branch.

origin/stage

The stage branch is a release branch for our stage environment. The stage environment should be used for final verification of changes ready to be pushed to production. The branch should be kept as stable at all times, as its deployed environment is being used and is a dependency to other staging services and clients. The environment should be deployed via CI, which will automatically run all available tests before deployment. Code changes should not originate from this branch. All changes in this branch should come from a merge of the origin/master integration branch or from origin/develop for verifying more unstable changes.

origin/prod

The prod branch is a release branch for our production environment. The prod environment is obviously running production traffic. There are a variety of monitoring tools hooked up to the prod environment. In case you push broken changes to origin/prod, you’d likely hear from them. The prod branch should be kept stable at all times. The environment should be deployed via CI (or git push), which will automatically run all available tests before deployment. Code changes should not originate from this branch. All changes in this branch should come from a merge of the origin/master integration branch, after first pushing them to origin/stage and testing them in the staging environment.

A change workflow

Here is how to propagate a change from being ready ( in develop ) to production.

Integrate in develop:

git push origin develop

Send for code review to the core team:

rake codereview origin/develop

Merge and push to test. This will also deploy the change there , after running tests.

git checkout -b test origin/test

git merge develop

git push origin test

Test the change

Merge into master

git checkout -b master origin/master

git merge develop

git push origin master

Merge and deploy staging. git push will deploy you.

git checkout -b stage origin/stage

git merge master

git push origin stage

Test the change. Generally try and wait at least half an hour before deploying to prod to weed out any issues. There is a log-replay that takes production traffic and replays to stage, so if something is broken, you should be able to quickly spot it.  

Merge and deploy to prod. git push will deploy you.

git checkout -b prod origin/prod

git merge master

git push origin prod

This short talk by Scott Belsky, founder and CEO of Behance, is a true insight into the creative process behind idea selection and ensuring their longitivtiy. A definite must watch for anybody in the startup community.

Scott Belsky: How to Avoid the Idea Generation Trap from 99% on Vimeo.

I spent some time thinking about the angle of this post.  And mostly, what is it that will give me the credibility to write it and maybe convince some people that it is not just a wasted 10 minutes of their lives. At the end, the only thing that I could think of is that it was extremely effective at shaping my thinking about the future and startups in general. This combined with the fact that I had very positive response from the some 20 or so people with whom I’ve shared my thinking on the topic, made me think that it is good enough of a start.

The anxiety

Being a founder and an entrepreneur are some of the ultimate words nowadays. It seems like you can’t go wrong using such words in a sentence or defining yourself by them. Just like when you say ‘cloud computing’ or ‘social media’, it all sounds… righteous and elevating. And I don’t argue differently. In many ways being an entrepreneur and a founder represent the ultimate career satisfaction award.

But this strive towards entrepreneurship creates an enormous amount of anxiety in the entrepreneurs-to-be, forcing an almost binary switch of whether you are working for yourself or not. Furthermore, this is reinforced by the fact that usually people only hear about the few success stories out there, and tend to ignore the numerous failed attempts at startups. As a result, many people jump all in, only to find themselves unable to stir the ship in the right direction, run out of fuel, get demotivated, fail, quit, get completely frustrated, and go back to square 1.

For me this was also the case. I spent endless sleepless nights recently, trying to figure out how to become a full-blown successful founder and entrepreneur overnight. Having been a founder before, not going back to square 1 was a top priority for me. Having seen the many failures around me, made me even more paranoid about the steps I take going forward. I wanted my path going forward to be shaped by positive rather than negative reinforcement (for clarification, negative reinforcement is when you for example administer painful electric shock to an animal, to make them cease a certain behavior like peeing on your carpet).

As I was talking to people about my tentative plan to leave my full-time job at Microsoft and start my own company, I caught a very saddle piece of information – “ the average age of entrepreneurs is 37”. 37? Wow. The profile in my mind was a dropout college kid in their early twenties that busts out code for a few weeks and arrives with the next big thing. Well, be it or not the case, the reason why for me this was such a crucial piece of information, is that it removed a lot of the anxiety that had built up over time. I recently turned 26, and that meant that by now and the time I reach the 37-year average there are about 11 years to go. It didn’t mean that I have more time to slack, quite the opposite. It meant that I could take a step back and strategically use some of that time to close the gap between where I am and where I want to be. The goal being to reduce risk of failure by being surrounded and learning from others’ success, while positioning for taking next steps. This was the basis for me coming up with the four stages of a startup, which helped me chart a logical path to achieving my ultimate goal and also helped me evaluate where a prospective company lies on the path.

The Four Stages of a Startup

The four stages of a startup is my breaking down of the stages between which a technology company moves, and some of the things that can be learned at each stage.

Stage 4: The enterprise

The enterprise, or the post-startup, is the stage a startup enters after it stops being a startup and becomes an established enterprise. That’s a company with a few thousand employees. Some of the companies that I’d consider being Stage 4 are Microsoft, Google, Facebook, and Apple. They are great and successful engineering companies that sometime in the past have experienced success and continue to rely on solid engineering practices to guide them to sustain and expand their influence in their respective sectors. One can learn a lot about what it means to be in the industry by joining and spending some time at a Stage 4 company, especially when it comes to engineering.  

That’s essentially where I was recently.

Stage 3: The successful startup

The way I would classify Stage 3 startups would be, a startup that has not been around for a very long time, and has experienced a pretty rapid recent growth and success. In essence, it still operates and behaves like a startup in every possible way. It’s a company you have heard of, like, and get excited about. There is a lot that can be learned and observed from being at such Stage 3 startup, especially with respect to successful company execution. It also has the added benefit of providing a lot of leadership and broad company-wide impact opportunities. As far as Stage 3 companies go, I consider Twitter, Scribd, and Hulu being prime examples in this category.

For me, a few months back, while still at Microsoft, Stage 3 was an immediate next step that I wanted to get a shot at, as it would get me closer to my ultimate goal, while giving me more time to figure out my positioning for Stage 2 or Stage 1. Every minute that I spent at Stage 4 onwards felt like a waste. Luckily, I got the right opportunity relatively quick, and became the second hire of the new Hulu Seattle office. So far, it’s been an amazing experience there.

Stage 2: The prospective successful startup

Stage 2 startup, I would classify as a company of 3-5 people that has formed and has locked down on a potential future value proposition and has started executing on it. That would likely be a company that you haven’t heard of yet, but which team members have a proven track record of success. I think the latter is extremely important, since having people who have been successful in the past, more often than not would mean that when times get tough, they’d know what to do. That’s not to say that you should make them change your diapers. No. This means that you’d be confident in their decision making to a degree that you can trust them to do the right thing and focus on what you’d be extraordinarily good at.

Stage 1: The (co-)founder

Stage 1 is the ultimate goal – being a founder or a co-founder. You are responsible for conceiving and turning an idea into reality, forming an A+ team, being a guiding force, finding funding, getting great set of advisors, and having unprecedented execution. It is, in my opinion, what good founders are able to achieve. And to a large degree this requires a lot of prior play in the field. It is not to say that it can’t be done on first try, but the odds of failure, I feel, are much higher.

Jumping head first into Stage 1 from Stage 4 for me felt like too much of a jump at the time. So, whereas being a founder remains my ultimate goal, taking some time to close the gap, and establish my play felt like something that’s worth spending some time on.

~~~

That’s what my thought process of what lays in the near future looks like. Life is full of surprises though, and that’s just one of the possible ways to get to Stage 1 — certainly not the only one. However, for me the biggest benefit of this framework (which if you follow closely, reassembles a mirror image of Crossing The Chasm, or how you’d see Crossing the Chasm if you were a constant, and all companies are moving targets) is that it elevated some of my frustrations and anxiety with the decision of how and when to shoot at founding a company.

I must’ve watched this video at least a half a dozen of times. It is definitely one of the must-see videos out there that gives a pretty good insight in the strategic workings of a startup. Do see.

Unfortunately the video is no longer available for embedding into other sites, but you can head out to http://vimeo.com/16098382 and check it out

When is the best time for quitting what you are currently doing and taking on new challenges? That’s a tough question! Answers can vary from anytime to never. It really depends.

I, personally, like to operate by principles. Having done the quitting thing a couple times now, I think I’ve figured a set of very important guidelines in the process.

1) It is vital to have an idea what you want to do next.

• Can’t stress enough on this. I always try to think what’s the next thing that I want to do and seek opportunities in that direction. Figuring this out is critical before taking the leap, because otherwise precious time is lost after you quit when the clock has started ticking and you are pressured to make a decision. In essence, just like with a startup, I think of it as pivoting. And before you pivot, you need to know where you are trying to go. If you are interested in services, look for opportunities in upcoming cloud computing teams. If you are interested in startups, look for successful startups that you like and can relate to. And so on, and so on…

2) Always leave on a high streak

• I try to quit when at my highest and strongest position. I know it probably sounds counterintuitive, but I’ve figured that this is really critical. Quitting while on a high streak feels good, makes you feel appreciated, and leaves the door open in case things go horribly wrong and you need a safe shelter to evade to. It is naturally hard to forgo the short-term future benefits in the current position, but collectively the experience from the diversification of skills and cultures adds a new perspective that is truly invaluable. Even if you decided to return, you would have experienced a supernatural jump in excellence and abilities that you wouldn’t be able to achieve in any other way. Just like with stock - sell high, rather than low. Don’t linger too much on the top. Quitting on a high streak to me always feels like a successful exit strategy no matter what happens next. 

3) Set a deadline

Well for better or worse, I decided that today will be the day that I will create my blog. You must be thrilled, I know …

And so, thinking about something, followed by setting a deadline, seemed to have resulted in something tangible here. This is  http://www.ludofriend.com, which will be my official place to share my (mostly) random tech thoughts with the world (or the two random people that will eventually stumble upon my blog)! 

I am not determined to be a hardcore blogger. I will try and use this space to share useful bits and pieces that I collect along the way. My interest is to mostly solicit opinion, in attempt to dismiss or reinforce my thinking about the current technology climate and the rapid changes occurring within it.

Enjoy!