Outsoucing and Scrum: Retrospective

Retrospectives are fantastic opportunities where the team can discuss any issues regarding the project at the end of a sprint. I aim to share some of the experiences and findings written from the viewpoint of different roles in a software project. Together, it highlights the difficulties of outsourcing.

The Architect

Open source tools and API used in the project should be well documented and maintained.
This point is often neglected as architects pick high performance API, with little or no documentation. Remember that it is the team that has to live with the daily struggle of using the API. With poor documentation, the team loses focus on coding instead of squandering about on Stackoverflow, looking for a possible solution to their problems. The team needs to be comfortable with the packages, and poor documentation will always negatively impact the velocity.

Examples and clear documentation of a vertical slice of the system, a prototype, will help the team scale out horizontally. Having a clear structure and well defined interactions between components will help the team burn down faster.

Quality of code should be constantly scrutinised. Static code analysis tools will help.

Knowledge sharing and training is encouraged. Pairing up the architect with other developers in pair-programming will help share the knowledge and gives the architect an opportunity to set the coding standards.

Write interfaces and couple them with test cases. This is a great way to ensure that the code that is checked in will actually perform as it should as it would be independent from the development cycle. There are a few mock APIs that will help this approach. Naturally test-driven development is highly recommended.

The Test Manager

Understanding both the requirements and the problem domain is critical to the success of the Test Manager, and the project.

Requirements will be written badly at times. It is therefore important to hunt down the Product Owner and flesh out these missing pieces. Ensure that each tests conducted is repeatable and well documented. Ideally, automated testing is highly recommended.

A creative, technical, thorough and methodical Test Manager might be hard to find. Ideally you would want a person that is an expert in the domain of your application and have a knack of finding bugs in any system. The scope of the system must extend to that of the customer setup and OS/hardware variation.

The Scrum Master

The Scrum Master lives to help the team, both resolving any blockers and help run the team on a daily basis. Ideally you want a Scrum Master that has the best interests of the team in mind and is at the same locale as the team. Always remember that the Scrum Master works for the people and this position is not suited for an aggressive tyrannical individual.

For our outsourced project, we rotated the Scrum Master role between the team members in Bangladesh with varying degree of success. We deliberately assigned this role to a promising Junior developer as an attempt to break the hierarchical mold in Bangladesh. This did not work well as the concept of seniority is so ingrained that it backfired with a senior developer threatening to quit because he felt that he was undermined by a junior and that his talent was not nearly recognised. Naturally we also tried assigning this role to this particular senior developer but his style was so tyrannical and turned the role of Scrum Master into a mini authoritative dictatorship.

Scrum Master should challenge the Product Owner. Ensuring that the team can finish all the tasks in a given sprint is important. Nothing is more demotivating than constantly pushing sprint backlogs to the next. In stark contrast, nothing is more empowering than being able to finish the sprint backlog plus additional tasks.

One difficult aspect is dealing with bug fixes. Bug fixes can interrupt the whole sprint. That is unavoidable. However, scheduling a couple of bugs each sprint will help drive down the bug lists. A zero-bug policy is admirable. Personally, an alarm bell goes off when there is absolutely no bugs in the system.

The Product Owner

The number one advice is to be available for the team. Answer their questions on time and flesh out your ideas clearly and ensure each user story is well thought out. Communication is key. Ensuring that the team understands the deliverables for each user story will improve their estimations and save a lot of heartaches.

Be interested in the technical details. Participate in testing and consider participating in development. Write test cases, unit tests or anything that will help you answer your number one question: Can I release this product this sprint?

Expect a good product demo at all times. Ask hard questions and never settle for anything less than you deserve.

Having a solid release process also helps. Releasing is surprising difficult and challenging. As Steve Job puts it, “Real artists ship.” I also recommend looking at Chrome`s release cycle – every 6 weeks.

Outsourcing and Scrum: our tools, our process, and some pitfalls

Part 2 of the outsourcing series on Bangladesh.

Last blog we touched on the difficulties in hiring and managing an outsourced team. This blog entry will highlight the tools we used, our processes and some of the pitfalls.

Cloud everything

Due to the time difference and the distributed nature of outsourcing, solid communication channels are critical. The general lack of stable infrastructure in Bangladesh meant that we could not have tools hosted there due to the high risks of down time. Furthermore, power outages would frequently plague the Dhaka office until a backup generator was installed.

We used Google for our emails and documentation, at times Dropbox. We used Jira Studio, particularly Jira, Confluence and Greenhopper for bug tracking, Wiki, Scrum tool respectively. In addition, we use Balsamiq and Gliffy plugins. The Jira Studio packages seamlessly integrates with our Google accounts, which made access control easy. The one thing I loved about Jira Studio is that all the tools are integrated; this includes our source control system. Note that Jira Studio is now migrated to Atlassian OnDemand.

Github is where we store the code. It allows for local cache copies and caters for your very own push process. i.e. Setup a local git repo in Dhaka, team pushes there. A pull request can be submitted upstream and reviewed for example. It can be as simple or as sophisticated as you want.

For sprint demos, we used Webex and TeamViewer for remote access.

Last but not least, Instant Messages and video conferences such as Skype or GTalk was very useful. Did I mention that Voice-over-IP is blocked in Bangladesh? One needs a government permit to get through the great firewall of Dhaka.

Planning and running sprints

Before any sprint planning occurs, the team needs to agree on a “definition of done”. This is the notion that a task is deemed complete. The team needs to come up with this themselves and agree to it. For example, unit testing, documentation and compile-error-free code is pushed to repo as part of the definition of done.

For this project, requirements were written as epics initially, then broken down in user stories. Each user stories are estimated in user story points by the technical team, as well as attaching a business value from our steering group.

Each user story is then further dissected into into technical tasks and estimated in man hours no more than 2 days worth of work. A user story is completed when all associated tasks is deemed complete.

The product owner will prioritise the user stories and provide a product backlog that can be potentially accepted as sprint backlog.

During sprint planning, we used an online planning poker provided by Mountain Goat Software and each task has an initial estimate and a remaining estimate field. These two fields allows us to get a good overview of the burndown and thus the associated velocity in each sprint. Note that an estimate is only accepted if it is a unanimous decision. You will be surprised how people will come to an agreement eventually.

With this approach, you will get burndown in user story points, business value and man hours. The benefit is that after a few sprint, you will get a sense of the complexity of a user story through user story points, and the team`s general performance via man hours.

At the end of each sprint, both a sprint demo and a retrospective is conducted and any unfinished user story is pushed to the following sprint. Note that re-estimation is not performed on delayed user story. In fact, re-estimation is something I do not recommend at all. For example, it is a known fact that estimation is quite inaccurate in the start of any project. Not to mention that there will be a varying degree of competency and perhaps confidence level. As the team progresses, estimation becomes a closer reflection of the complexity of the task at hand, as well as team`s proficiency with the problem domain. Thus the estimation will converge as individuals` own competency improves. In order to fully illustrate this and correctly reflect the increase in velocity, the old crappy estimates must remain. If the project is intended to be very short, then re-estimation *might* make sense.

Sprint demo is the chance where the team can really shine, or fail miserably. Though the Scrum Master should try to protect the team, if the sprint is faced with a ton of road blocks, it is inevitable that the demo will not be their proudest moment. However, that should improve with time. The Scrum Master should be the best communicator in English and host these demos. Anybody from the organisation can attend these demos and encourage them to do so. I have heard that Google often wheel out cake and coffee during sprint demos to set a celebratory tone. And indeed it should be a celebration.

At the end of each sprint, the product needs to be in a shape where it is potentially releasable. This is extremely important as I believe it is the central concept of any agile processes, particularly Scrum. Even if the product does not have a packing or release process, you can still convey to the team that test users needs to be able to check out the code and run the program via some simple scripts. Being able to potentially release at any moment is critical to the success and pacing of the project.

Some pitfalls

  • Estimation. Some team members estimate in perfect days (7,5 hours without any interruption), some include lax time in their estimates. The unanimous estimation aims to find a balance between the two. In time, the team itself will understand and identify correctly the difference between a 2 hour vs 8 hour tasks.
  • Business value estimation is of no use to the business. This is a typical case where an idea is good, but have no real life use usage. Establishing a steering group to iron out these type of features is great to avoid erroneous requirements will hopefully ensure the invested features are with the support of marketing and sales. Once this becomes a routine, showing how much business value deliver per sprint is as easy as looking at the burndown.
  • Inherent hierarchies in Bangladesh. There is an intricate and ingrained notion of hierarchies in the Dhaka team. An interesting observation for me is that a junior engineer cannot be seen having a cigarette when a senior engineer member is in the vicinity. The definition of a junior is merely somebody who has joined the company later, or is of a younger age. Nothing to do with their technical skills. To further exacerbate this symptom, the junior is rarely seen correcting the seniors. This is a huge problem because I would like the team to operate in a flat hierarchy, where ideas and innovation can be freely discussed. There is not easy cure for this as it is part of establishing a local company culture that promotes innovation and discussion, while leaving the old world baggage behind. Our approach was to build a good team rapport and encourage the juniors to speak their mind. One last suggestion is to talk to the juniors directly on a 1-1 basis to find out their thoughts and suggestions. This is quite time consuming of course, but it can be rewarding and often serve as an early indicator if there is something fishy going on with the team.
  • User stories are vague and thus form a horrible basis for requirements. This is the general pitfall of Scrum, not particularly an outsourcing issue. Establish concrete tasks and implementation based on an idea is never straightforward. A vision can be as ambiguous as “As a sysadmin, I want to create a user in this system, so that this newly created user and login and work with the system.” This is a typical user story that any CRUD application will support. But what does that user story really entails? How much work is this? How much business value does this actually give? These are questions that both the team and the steering group will need to iron out for themselves. However, I cannot recommend the use of condition of satisfaction enough for user stories. Be as meticulous as possible when writing condition of satisfaction. Your QA will thank you. If you are the borderlining the formal methods type, then throwing in a UML design methodology would not hurt. It will bolster the understanding of both the business requirements and the implementation itself.
  • The team used powerpoint slides for their sprint demo. This is generally not encourage as the idea of having a potentially releasable product is obviously not met. However, there are at times where this is perfectly ok. I encourage good documentation throughout the development cycle and if extracting power point slides from a few wiki pages will do the trick. So be it. Be sure to not make this a habbit.
  • Certain tasks such as refactoring will render the product in a potentially unshippable state. Create a new branch and merge it back to the release branch is your solution. No excuses that a product cannot be shipped.

Outsourcing & Scrum: Hire the right team

This blog series is more of a note to self and a chance to share my experience with running a Scrum team in an outsourcing environment for a software project.

An opinion piece

Outsourcing or offshoring, which ever way you want to spin it, is essentially a distributed team environment. A virtual office if you will. The members of the team have to deal with added complexities to any projects be it difference in time zones, cultural misunderstandings, and communication problems on a daily basis. The culmination of all the added hurdles may result in the detriment of product quality or delayed releases. Digging a little deeper, one might find that the code quality could be so lacking that it becomes a chore to maintain or even expand.

I am not having a go at outsourcing in general. Quite the contrary. I believe outsourcing works. I enjoy overseeing these types of projects and the associated challenges. I am a firm believer that it can be managed successfully.

Committing to long term outsourcing is akin to your top brass management buying over their favourite development house somewhere else in the world during their summer vacation, whilst trying to sell this idea back home. Cynical as it sounds, this whole shenanigan can function like a well oiled machine. However, lets not kid ourselves about the amount of oil needed to lubricate the bad boy which is another topic altogether.

The get up

Here`s the get up: we are a start-up company. We approached a local company with a strong connection to Bangladesh. They too, were starting up. Together we were tasked to find the smartest and brightest to work on our project in Dhaka.

Hiring is tricky

Now comes the dilemma, hiring. During the interview process, one of the applicant informed us that engineers in Bangladesh were more willing to work for a well established, household-name I.T. companies as a cleaner than for a start-up company as a senior engineer. It seems branding was everything. Indeed this was shocking news because the word on the street was that more and more big names was going to set up shop in Dhaka too.

Hiring was indeed difficult because brand awareness was apparently the key attraction in Dhaka. A high salary simply will not attract the right people. Luckily for us, we were an up and coming open source company and our product was already well received in the community. Our shining star was that it was going to be a brand new project with an existing prototype built with leading edge technologies.

Communication is key

Eventually we settled on five bright minds with varying degrees of English competency. Communication proved difficult. Here`s a precursor before I continue. I am an Australian working in Norway. Our working language is English and we are based in Oslo. The outsourced team at times needed to interact with my Norwegian colleagues, whom are not native English speakers. Norwegians in general speak fantastic English. It is second nature to them. However accents were difficult for my Norwegian colleagues, which was both frustrating and blatantly counter productive.

To add insult to injury, internet is unpredictable and bandwidth was inadequate. Not to mention there are frequent power outages in Dhaka. This resulted in horrible daily Scrum meetings and end of sprint demos where none of the attended parties could understand one and other. The situation improved as a backup generator was installed and more bandwidth was provided by the local ISP in Dhaka. Note that the internet cables in Dhaka are often exposed and lies jumbled up right next to the road at times. Not entirely sure if they are weather proof even.

Team building

Travels to Dhaka to meet the team was scheduled at least twice a year. This was an excellent chance for meet and greet, and some team building. Noticeable motivation and communication level was improved with these outings. I strongly recommend a make-shift karaoke session with jugs of long island ice tea to break the ice.

Last but not least, I am a firm believer of knowledge sharing and encourage the Dhaka team to have knowledge sharing sessions. This included hosting a developer`s day, where each of the team members presented something of their interests or something new to the team. Companies often encourages sharing of knowledge but they stop short of hosting an annual event or actually schedule time in a week where techies can roam free and just be techies.

A little checklist

In summary, I have learnt the following:

  • Hiring good engineers is the same back home. You will be competing against well established software houses.
  • Good brand awareness will improve the chance of hiring the right team in an outsourcing environment.
  • Frequent visit either to or fro helps build team morale and motivation.
  • Communication plays an even bigger part in outsourcing. Ensure the infrastructure is there for web meetings.
  • High level of English competency ensures that the outsourced team can be understood by everybody.
  • Build rapport with individual team members. Your team members need to be comfortable enough to really tell you their blockers during daily Scrum.
  • Participate in daily Scrum along with the team.