Software Development Macro and Micro Process

If you think that in year 2012 all companies which produce software and IT divisions in our world have already their optimized software development process, you are wrong. It seems that we - software architects, software developers or whatever your title is - still need to optimize the software development process in many software companies and IT divisions.

So what do you do if you enter a software company or IT division and you see following things:

1. There is a perfect project management process to handle all those development of software but it is a pure project management without a context to software development. So basically you only take care of cost, time, budget and quality factors. In the software development you still use the old fashioned waterfall process.

2. From the tooling point of view: you have a project management planning and controlling tool but you are still in the beginning of Wiki (almost no collaboration tool) and you don't use issues tracking system to handle all the issues for the development of your software components and applications. You use Winword and Excel to define your requirements and you cannot transform them to your software products since you don't have any isssues tracking system. No chance to have traceability from your requirements down to your issues to be done in your software components and applications.

3. Maven is already used but with a lot customization and not intuitively used. The idea of using a concrete already released version of dependencies was not implemented. Instead you always open all the dependently projects in Eclipse. You can imagine how slow Eclipse works since you need to open a lot of projects at once although you only work for one project. Versioning in Maven is also not used correctly e.g.: no SNAPSHOT for development versions.

4. As you work with webapp you always need to redeploy to the application server. No possibility to hot deploy the webapp. Use ctrl-s, see your changes and continue to work without new deployment is just a dream and not available.

Luckily as an experienced software architect and developer we know that we can optimize the two main software development processes:

1. Software Development Macro Process (SDMaP): this is the overall software development lifecycle. In this process model we define our requirements, we execute analysis, design, implementation, test and we deploy the software into production. Waterfall process model and agile process model like RUP and Scrum are examples of SDMaP.

2. Software Development Micro Process (SDMiP): this is the daily work of a software developer. How a software developer works to develop the software. A software developer codes, refactors, compiles, tests, runs, debugs, packages and deploys the software.

More information on SDMaP and SDMiP:

• You can find the definition of SDMaP and SDMiP in the context of analysis and design in the book  Object-Oriented Analysis and Design with Applications from Grady Booch, et. al.
• Unifying Microprocess and Macroprocess Research 
• Effects of Architecture and Technical Development Process on Micro-Process

The picture below shows the SDMaP and SDMiP in combination. The macro (SDMaP) and micro (SDMiP) process meet at the implementation phase and activity. So changing and optimizing one has definitely side effects on the other one and vice versa.

At the example of organization mentioned above it is important that we optimize both processes since they work hand in hand. So how can the optimization for macro and micro process looks like?

1. SDMaP:

• Introduce Wiki for IT divisions and software companies. You can use WikIT42 to make the structure of your Wiki and use Confluence as your Wiki platform.
• Introduce Wiki with issue tracking like JIRA and combine both of them to track your requirements.
• Refine the requirements into issues (features, tasks, bugs, etc.) to the level of the software components and applications, because at the end you will implement all the requirements using your software components and applications.
• Introduce iterative software development lifecycle instead of waterfall process. This is a long way to go since you need to change the culture of the company and you need a full support from your management.

2. SDMiP

• Update the Maven projects to use the standard Maven mechanism and best practices with no exception. Transform the structure of the old Maven to the new standard Maven using frameworks like MoveToMaven. 
• Use Maven release plugin to standardize the release mechanism of all Maven projects.
• Use m2e Eclipse plugin to optimize your daily work as a software developer under Eclipse and Maven.
• Use Mylyn to integrate your issue tracking system like JIRA into your Eclipse IDE.
• Introduce JRebel to be able to hot deploy quickly your webapps into the application server.

Optimizing macro and micro process for software development is not an easy task. In the macro process you need to handle all those relationships with other divisions like Business Requirements, Quality Assurance and Project Management divisions. You need to convince them that your SDMaP optimization is the best way to go. This is more an organizational challenge and changes than the micro process optimization.

The micro process is also not easy to optimize, since you need to convince all developers that they can be more productive with the new way of working than before. You need to show them that it is a lot more faster if you don't open a lot of Java projects within your Eclipse workspace. Also using JRebel to deploy your webapp to your application server is the best way to go. Normally developers are technical oriented, so if you can show them the cool things to make, they will join your way.