Thursday, June 23, 2011

Services, practices and tools that should exist in any software development house / department - Part 1.

I truly believe that it's part of our job, some soft of duty, to evangelize the correct way of working within the company / department we work for. Apart from doing our regular tasks analysis, design and implementation we must work efficiently under certain practices and using tools that have been there for years and will continue to exist. It is not an uncommon thing to see while hopping around on different IT companies especially smaller firms, that the software development practice is not functioning properly or is not established at all.

We could elaborate around the term 'practice'  - what I am really eager to write though is about the minimum tools, services and practices that every developer should be aware of and every new manager or it software company owner/manager should install setup and maintain so that he/she can help his team(s).

1. Code repository/Versioning system : Eventually it is a central place where we have to save and share code. It manages aspects like version-ing and distribution through out the team. It is amazing but even today you may see around developers sharing code using shared folders, emails or setting up their own code repositories per project (waste of resources). These are practices that we should strongly discourage at all cost. There are some basic concerns around selecting a code versioning system.
  • (Type)Which one of them? Indeed we have quite a few, VSS, CVS, SVN, Git, Merculiar etc. In genera, SVN and Git tend to trend in these recent years. Eventually you pick one depending on your existing know how and your requirements. In case you completely lack previous experience pick one that is trending  (SVN or Git). In case you have your own personal preferences and you know what is good and bad you are free to make your choice and support it. SVN would be a good and safe choice at the moment (IMHO).
  • ($$ )Most of these services/servers are open source and free so you wont have to spent any extra money on downloading and installing them (SVN, CVS, GIt).
  • (Backup) After installing the service and using it, we should be aware that we need to setup a backup strategy and procedure, usually assigning this task to any of our administrators.So proper back-up and maintenance is required!
  • (People) No matter if we manage to buy or install the best code repository/versioning system in our company if the majority of people do not follow the it is another wast of resource. Bringing the tool inside the corporate context is the first step the second one is to make the people start using it - this is where you gain all the benefits!
  • (Remote Access)A central code repository gives you the ability to enable remote working (if it is required or you are willing to do so). People may download parts of work code at home or at the clients premises and contribute hours - not being restricted to local corporate resources.
2. Issue & Bug Tracker : An issue tracker is a software service (usually a web application) that helps us create, monitor manage and filter the different issues and bus found during development, operations or maintenance of a software solution. Any software product or service can be considered as a living organism, from the very early stages of its creation will need special attention producing errors or problems. Even when we manage to make it work when you release it in the production we always discover new problems, new issues or new ideas emerge. 

In order to manage this long lasting process of finding, fixing, prioritizing and filtering all these issues and problems we usually fall back to the use of an issue and bug tracker - that can be access from all the related parties. In such a system we are engaging both developers, managers even the end customer to support the issue and problem life cycle of the end product. The developers have tool to associate problems with tasks of work, preserve a history of their fix work and probably able to reactively fix things as they show up. Managers are able to monitor the 'health' of the system by being aware at any time the maturity of the solution. That way they are capable to plan efficiently, give better estimates and provide efficent cost estimates for future releases and change requests. Last but not least the end customer is able to monitor the progress of work, get involved with the product at early stages and being able to log and push on the priority list things that he/she consider that require immediate attention.

I need to point that an issue tracker usually can be used an efficient way of logging work hours - something that usually companies do using interfaces with other systems like SAP or siebel etc. Logging hours and creating weekly monthly reports you get valuable budgeting information regarding your teams / department. (for free). 

There is no system with 0 bugs especially in it's initial life and we can not log, archive, search and keep history of this evolution (through the fixes, issues or change requests) with no issue and bug tracker help.

There are some basic concerns around selecting an issue tracker:
  • (Type) Which one of them? We still have a variety of solutions. Their features more or less are the same in terms of what the system is doing. Each product though differentiates by providing a lot more o certain areas (eg. better reporting, better ui, price, compatibility and integration with other systems). My personal favorite and the one that I have seen in most companies is JIRA from Atlassian. I personally consider it as one stop shop and a great and decent product. I would also have a look on IDEA's YouTrack which features great functionality in a cheaper price. Trac or bugzzilla are interesting free open source alternatives.
  • ($$)The situation is a bit mixed on this category. Great proprietary products and a list of free open source. It is really a matter of money I guess in this. I would strongly though recommend to invest on something good rather than use just for the sake of it a non usuable or handy solution.
  • (Backup) After installing the service and using it, we should be aware that we need to setup a backup strategy and procedure, usually assigning this task to any of our administrators.So proper back-up and maintenance is required!
  • (People)  As already elaborated - using an issue tracking service requires a change of work mentality on teams that have not seen this before. I think it is change that pays back very soon + the company is rapidly reducing man hours on its work log by automating procedures and tasks that were manual in the past.
  • (Remote Access) Remote access is a great asset, anyone (you want to) may have control & access to different parts of this service (per project, per company per client) and remotely can handle important work - regarding the end product.

3. Corporate - central Wiki: Creating software means exchanging information either related to the end product or the technologies used. The information flow within a software department or even within software development teams is a great problem even today. Problematic information flow at any level may lead to inconsistencies, problems or bad quality work for any team and product. We need to ensure that we have available those tools and practises that will enable all involved parties during software development to easily and flexible enough share information, document findings and share content together. We live in the era of blogs, twitter and web services. Tons of content is being loged, saved and archived through web applications. Google is indexing a searching web pages and related resources. So since we are doing it for our every day life why not follow the same pattern within our company?

From my personal experience having a central wiki service available is enabling people to easily document and share knowledge either related to a specific project or something general far easier comparing to our old style of Writing multi page and multi megabyte word documents. How many of you have received in your inbox huge word files with the title ' System Specifica version1' having track changes enabled with thousand of comments and a ever changing structure. Word/ Office documents in general are formats that aim to the printed outcome of the document and do not encourage viewing or working on the electronic format it self. I am not against word documents but I am really against writting information, sharing information with other people and having to share the same instance of the same resource, merging changes or keeping track of them while I would just share a virtual page/ place on a wiki (like ablog) and start writting and sharing immidiately.

I strongly engourage all of you to start experimenting moving your technical analysis or business analysis  documents to such a service by transforming them to wiki pages and web resources. You will be amazed how easier would be for all the related parties (Analysts- Developers- Managers - The Customer) to find, read through efficiently and even comment of add information comparing to the old office sharing way! It is really a change and I always remember proposing such a thing to different management teams while getting back some weird eye- you are proposing something our of our comfort zone thing. But truth to be said - start capturing information in a wiki and stop misusing word or open office - produce word documents or pdfs only at the final stage.

  • (Type) ($$)Which one of them? Mainly most of them do the same thing. Plenty of choices. Proprietary solutions like Confluence are trending a lot but you can easily cover your needs with a free solution like MediaWiki etc.
  • (Backup) The same as above!
  • (People)  As already elaborated -(yet again) - you need to persuade people understand the significance of information flow within an organization.Lots of people are used of the old way, messing around with excel and word documents. Eventually we have we have a solution for this uncomfortable way of working.
  • (Remote Access) Like blogs or any web resource wikis can be easily accessed  shared (or limit them if you wish so) to a variety of people or groups.

This is the first part illustrating 3 main tools /services that will get you started. In the second part we will elaborate more on other tools or practices. Bare in mind that in some cases the combination of these services - provides - functions and automation levels that have not been covered in this blog post. For example, an issue tracker system may automatically associated issues /bugs with parts of the code as retrieved automatically by the versioning system. Or you may extend the design/solution or an explanation of a problem using a link to an issue that targets your wiki!
Feel free to comment and provide proposals on the above 3 categories (for example products or services you have used and you consider worth buying or installing).

Thanks for your may continue with the second part here.


  1. Really nice summary so far, but i expect to see something about security in part 2!!!

  2. Hi thanks for the comment, I am trying to set the most basic stuff (i expect senior and experienced software architects already to have such an environment established - or make sure that they have one)

  3. by the way when you - refer to security you mean firewalls, proxies and vpns?

  4. SVN, Jira and Confluence.

    case: enterprises.

    Department Level: It is essential to define the methodology/process of work on each system (depending the user role). The tool by itself won't help, you will need expertise on each system like you do on Eclipse or Intellij (not only you, but all the team. ok different level of expertise depending the user role)... of course these are lot more simpler systems to learn and use, but they require discipline for efficient long term of use on lot's of information/data.

    Company Level: You will need dedicated people resources for the administration on each system and of course dedicated hardware resources.

    Tools Integration: As you mentioned the integration of these systems, will boost the teams productivity.
    SVN with Jira, Jira with Confluence, Java IDE with SVN, Java IDE with Jira.

    Java IDE Plugin: Don't miss the Atlassian's Jira connector for Intellij and Eclipse!!!
    The jira issues are always up to date (in progress, resolution, comments... even the log work is set automatically) without the developer need to open the jira on a browser.
    The Intellij's Change List and Shelve Changes functionality help a lot on this!

    Finally, a serious tool for PMs:
    Jira is not only an excellent issue/bug tracker system... it is the best tool for project management.

    If you apply Scrum methodologies on your development team, you can try the GreenHopper plugin (by Atlassian) for Jira.

  5. Tip: Don't give full user permissions control on all the team on a Confluence Space. In a couple of months you will end up to have a space full of waste information.
    A review process before publish any info is mandatory. Example: let the users insert the info on comments on the pages and next permitted user review the comments and add them on the pages. Or let the users enter the information on pages on their personal space and once review you move them on the "official" space.

  6. @Paris,

    no, i don't mean infrastructure security controls, i mean application (code development) controls

    Some examples could be:
    - Static code analysis: Fortify (now HP), Ounce Labs (now IBM)
    - Automated dynamic analysis: For web applications you can have "web application scanning" tools, IBM, HP, Accunetix, and a lot of open source ones. For other kind of apps, fuzzing tool (like JBroFuzz)
    - Manual static analysis: MITM proxy tools (Burp, ZED, etc)

  7. @Dimitris
    Ok got it, would be happy to add your comments in the related section of IDE plugins (when it comes to quality) most probably to part 3. would that be ok with you?

  8. @Paris

    sure, feel free to do so, and if you need any help to write that section (since i am working a lot with application security) do drop me an email!

  9. I was looking for a good article about increasing my firm’s turnover and tracking employee efficiency, If you have added the fact about implementing time management systems like that of timesheet in your article it would have been great any way it is a good article ...Keep it up and GO ahead with your work..

  10. Hello timSheet thanks for your comment.

    That would be a good idea (I admit it is still missing) but I think there is still room for part 5.

    Eventually I've found that you can do an excellent time tracking practise using tools like JIRA. You can associate your developer's tasks and issue with direct man hour logged by them.

  11. Software Development Company providing end to end software solutions. As a software development company based out of India, Acetech offers a range of quality services that include Custom Software Development, Offshore Software Development, Open Source Development, Web Design & Development Services, Software Testing, Technical Maintenance and Support services