EA - Enterprise Architecture in Action

- Jerry Larivee

At this risk of sounding like a management, motivational speaker there is one consistent piece of advice that I give to IT organizations over and over and over again when undertaking large initiatives: start small and build in incremental steps.  This is particularly true for Enterprise Architecture groups.  It is in this spirit that I recommend the adoption of agile methods within the Enterprise Architecture groups, even if agile is not used within the application development groups of the organization.

Agile methods can be successfully applied to all sorts of Enterprise Architecture activities, but to illustrate its applicability let’s look at how agile applies to the development of a standard technology platform for application development.  What I mean by a technology platform is a set of standard technology components and reference architectures that can be used by application development projects within the organization, such as a platform for e-commerce applications, or a platform for Java-based, web development, etc.

While we’re on the subject of using agile development, I’ll also cover what considerations one might take into account while defining a technology platform to maximize the benefits of agile methods in application development.

First let me level-set with quick a definition of agile.

If you are familiar with agile, feel free to skip forward.  If you’d like to read a more thorough explanation of agile I suggest reading “Extreme Programming Explained: Embrace Change” by Kent Beck.  Even though it’s about one specific agile method, XP, it’s a great one-evening-read that covers the core of agile.

The definition I use is made up of eight simple ideas:

1. Small release and rapid feedback cycle – By moving stuff into production use as quickly as possible you start to realize benefits quicker and you get feedback sooner.  Most importantly you don’t wait until months have passed to realize the project is failing.
2. Don’t be afraid to change the requirements – As you put releases out and get feedback, you’ll refine your understanding of what the customer wants and the customer will refine their understanding of what is required.  This can be a good thing, embrace it.
3. Continuous customer interaction – By keeping the customer involved you allow for that rapid feedback.
4. Define releases based on a functionally complete set of features - Each release should demonstrate a function of the complete system from the perspective of the customer.  I.e. the customer should be able to look at the release and test it against their expectations without someone explaining why this or that doesn’t work yet.  This goes against a lot of old school engineering practices that say build your entire infrastructure first and then build the functionality.  The reality is sometimes we build infrastructure that we never use.  A corollary to this is that before we start building we should know how to test, so when the customer defines the requirement we need to also define a test case.
5. Don’t be afraid to refactor – One of the reasons we tend to build out infrastructure before functionality is because we like to write a piece if code, test it, call it done, lock it away in a source control system and don’t change it again.  This works fine if you follow the approach of build all the infrastructure first and then the functionality and you don’t change any of the requirements mid-stream.  Since we’ve already accepted the fact that requirements will change and we’re not building out infrastructure until we need it we have to be flexible and allow for the refactoring of code that was already released.
6. Automate testing as much as possible – The main reason we fear refactoring is we’re afraid we might break something by mistake.  The solution: automated regression testing.   If it’s easy to test if broke something you’ll be more confident in your refactoring.
7. Continuous integration – Since most projects have multiple developers and we’re allowing them to refactor code as required we need to synchronize as often as possible.  Practically this means a shared environment with all the automated build and test capabilities.  Ideally the integration build and test process should be push button.
8. Keep it simple – Underlying everything in agile is the basic belief that the simplest solution is the best.  Don’t implement a general purpose spreadsheet if all you need is the ability to sum a column of numbers.  Don’t build out infrastructure if you don’t need it.  Don’t design complete abstractions when a simple abstraction will do.  Don’t write hooks for 15 features you haven’t thought of yet into something. Etc.

So let’s look at how each of these ideas can be applied to defining a platform:

Small releases and rapid feedback – I always recommend to organizations starting to defined standards to start slow.  Don’t define more standards than the organization needs, has the ability to absorb or has the ability to support.

• If you have only one application that uses a rules engine in your entire portfolio, there’s no sense defining a standard around rules engines and certainly no sense is holding up the development of a reference architecture until you’ve figured out how to incorporate the rules engine into it.
• Organizations absorb standards by writing new applications that use them, if there aren’t any applications in the pipe for development to use a standard it won’t get used and by the time it does it might be out of date.
• If you tell someone they have to use AJAX you also need to tell them how to use AJAX.  If you don’t have the bandwidth to provide the support to a standard think twice about defining it.

As you put out standards see how they are used and get feedback from actual application developers.  If they are also using agile methods you shouldn’t have to wait long for feedback.

Changing Requirements – Platforms change whether we want them to or not: technology evolves, vendors update their software, and new applications have new requirements.  No matter what methodology you use for platform development you have to plan for change.  This is often called the “technology adoption lifecycle” every company that is serious about managing standards has some version of it.

Continuous customer interaction – This should be even easier for platform development than application development since the customers are members of IT who speak the same language as the platform developers.  There’s little reason why platform development should not be getting constant input from the application development community.

Functionally complete feature sets – A platform is essentially infrastructure so defining a functionally complete feature set is probably something a platform developer will have an easier time of than an application developer.  A platform should address specific needs of the application development, maintenance and operations.  Some thought should be put into what those needs are and what the minimum feature set required to develop those releases are.  One complication for platforms is that often times they are build on vendor products which have lots of features you don’t really need.  Part of defining a platform is setting the rules around which features of a vendors product should be encouraged, which ones allowed and which ones discouraged or even banned.  This is something that is often missed in platform development; the results are applications that are irrevocably tied to a specific vendor’s product.

Refactoring – Not that I want to discourage the use of refactoring, but a word of caution here is prudent.  Good platforms are designed to be modular with well defined interfaces between the modules and most important clean Application Programmer Interfaces (API) exposed for the developer using the platform.  When refactoring within a platform it is ok to change the interfaces between modules in the platform, but be very judicious when changing the externally exposed APIs.

Automated testing – This is just good practice, I don’t care what you’re developing.  And the infrastructure guys will love you if you develop an automated test for validating a deployed platform.

Continuous integration – Another good practice, and again the infrastructure guys will take whatever assistance they can get in automating the deployment of a platform.

Keep it simple – There’s an old saying about the US Supreme Court that says the court can withstand anything except ridicule.  Basically it means that if the court makes decisions and nobody follows them, they become meaningless.   As a result the court tends to look for the simplest way to resolve any issue figuring that it will also be the most readily accepted by all parties.  The same is true of platform developers.  If no one uses the platform it becomes meaningless, thus a good platform is both modular and minimalist, thus allowing for piecemeal adoption as an alternative to wholesale rejection.

As promised, while we’re on the subject let’s look at how a standardized technology platform can support agile application development.

Small releases and rapid feedback – Part of a platform definition should include the release mechanism.  If releasing a new application onto a platform is a four week task, then the idea of small, rapid releases will not be very efficient.  The platform should be designed to allow for releases as quickly and effortlessly as possible.

Changing Requirements – One of the most contentious parts of any application development project is defining non-functional requirements (NFR).  Project shouldn’t be focusing on NFRs that much in the first place; most NFR should already be defined at the enterprise-level (i.e. availability, security, monitoring, etc.).  By building enterprise-level NFR compliance into the platform, the application development team can focus on functional requirements.

Continuous customer interaction – The platform needs to support development.   The development group shouldn’t have to figure out how to make a “beta” version of the application available to the customer, the platform should have that mechanism already.

Functionally complete feature sets – This is one of the fundamental reasons why we build platforms.  A lot of the infrastructure issues will be common across applications and to the extent that the platform is complete out of the box the application developers can jump straight into functionality development knowing all the basic infrastructure is there already.

Refactoring – If the platform is modular it will support refactoring the application in smaller increments.  Often times the scope of a given increment will be influenced by the scope of the modules within the underlying platform.

Automated testing – The platform can do a lot to support automated testing; this should be included as a requirement in the platform development.  Also, there should be a simple way to validate a platform has not been corrupted as well as mechanisms to support the regression testing of an application when the underlying platform changes.  How many times do we defer taking vendor updates because we’re afraid it’ll break something?

Continuous integration – Again the platform can do a lot to support continuous integration.  Most immediately if the platform can easily be virtualized that solves a host of resource issues.  Also, the easier it is to deploy the platform the easier it is to refresh our environment if we’re afraid we corrupted something during testing.

Keep it simple – Developers are assumed to have a certain base knowledge set, but they’ll need to learn a few things about the platform if it’s the first time they use it or if it’s been updated.  The platform developer should try to minimize this learning curve.  It’s just good practice.

A final word about the adoption of agile for Enterprise Architecture work, this one’s for the finance guys.  One complaint about agile application development is that it doesn’t fit well within generally accepted practices for deciding when it is acceptable to capitalize application development and when to treat it as an expense.  To make matter worse the general practice is: when in doubt treat it as an expense.  Unfortunately this has the result that when comparing agile to waterfall, agile looks less attractive from a financial perspective.  Fortunately since Enterprise Architecture work is treated an expense in most cases anyway, this isn’t really a problem.  In fact, agile’s focus on small releases, put into production quickly will allow your EA effort to deliver return on investment more quickly; that’s always a good thing.

Posted by Jerry Larivee at 12:47 PM | Permalink | Comments (0) | TrackBacks (0)

- Jerry Larivee

In the recent Garner report “Ten Criteria for Choosing an External Service Provider for Your EA Effort", 3 of the 10 criteria related directly to the architects from the External Service Provider (ESP):

6.5 Does the ESP Have a Standard Approach for Identifying and Developing Architects Within Its Pool of Consultants?

6.6 Does the ESP Have a Standard Way of Describing Architects' Levels of Competence?

6.7 Does the ESP Have a Consistent Way of Staffing the Right Architect With the Right Skills on the Right Project?

Gartner is quite right to include these in their criteria.  No issue has nagged the Architecture profession in general and the Enterprise Architecture discipline in particular as the challenge of identifying and developing architects.

I’ve often said that we play a cruel joke on most architects at some point in their career.  Since most architects start as developers they spend years learning to take large problems and efficiently break them down into smaller and smaller pieces until finally they get to a level of detail that they can readily translate into code.  Then after having proved their analysis skills sufficiently they are promoted to architect and told to do everything they used to do except backwards.

The programmer’s skill is analysis, but the core architect’s skill is synthesis.  This is not to say that architecture doesn’t require a great deal of analysis, but the critical step in architecture is often re-assembling the pieces into new higher-level constructs in order to get to that which is “architecturally significant”.  This is synthesis.

In his book “A Whole New Mind: Why Right Brainers will Rule the Future” Daniel Pink points out that analysis is a left-brain-directed skill, but synthesis is a right-brain-directed skill.

This starts to explain the challenge that most organizations face in trying to “grow” architects.  The traditional developer evolutionary path focuses on training the left-brain, but a good architect requires both a well developed right-brain as well as left-brain.

(This is an EA blog and not a neuroscience blog, so if you want to understand all the differences between the left-brain and the right-brain I suggest reading “A Whole New Mind”, but suffice it to say that the left-brain is sequential and logical, but the right-brain is holistic or non-sequential and intuitive.)

Diving deeper into “A Whole New Mind”, Mr. Pink describes six essential “senses”, as he calls them, which are all right-brain-directed aptitudes.  He argues that these senses complement our left-brain-directed reasoning and are essential to success in the new “conceptual era” that we now find ourselves entering.

Whereas he describes the value of these senses in a generic context, I’ll describe their value to the architect.

1. Design – In order for a system to be successful it must first of all be usable and second of all it must be used.  A system that merely delivers function may not meet either of these criteria.  Particularly when implementing a system that changes the way people work, it is important to design a system that compels the user to explore the new functionality and incorporate it into their normal way of working.  One could argue the Blackberry and the iPhone have very similar functionality, but whereas as the Blackberry took nearly a decade to become ubiquitous, the iPhone achieved the same within its first year.  Why?  Design.
2. Story – Architects often have the skill of being able to hold an enormous amount of complexity in their heads at one time, but the successful architect can convey this complexity to others in a way that is simple.  The unsuccessful architects is often frustrated by the fact that their audience cannot appreciate or even understand the elegant models they develop to manage complexity no matter how many times they take them through the details.  But the successful architect uses metaphor and story to convey their message.
3. Symphony – Can anyone doubt that architects must be big picture thinkers?  The sense of symphony is to see the whole composition, to understand how a change in each component will affect the whole.
4. Empathy – As any fan of Star Trek knows, humans are not known for being logical.  An architect’s ability to really put himself/herself in the shoes of the user is enormously important.  This ability is a form of empathy, feeling with the user not just feeling for the user.
5. Play – The notion of play has lost much of its frivolous taboo in recent years.  Much research points to the effectiveness of games at teaching problem solving skills.  In some case games can themselves be used as problem solving techniques.  And last but certainly not least, the value of “fun” has been shown to be significant in boosting productivity, facilitating adoption of new ideas and ways of working, and even enhancing the retention of knowledge.  When the LoJack system was first deployed one factor leading to its adoption was that police officer found it fun to use the system; tracking down a stolen car took on a video game-like quality and thus the police embraced it whole heartedly.  An Architect should be able to incorporate the right mix of seriousness and play in the systems he/she designs.
6. Meaning – Although the current “Great Recession” has caused many people and corporations to focus more on survival, this will likely prove to be a minor blip in a trend that has seen the developed world search out for meaning in life as it feels more and more comfortable that its basic material needs will be met.  This applies to both individuals as well as corporations; how many corporations today have adopted corporate value statements that include the responsibility of the corporation to society as a whole.  Enterprise Architects are well position to help balance the fiduciary responsibility of a corporation to its shareholder with these socially-conscious corporate value statements.

All this might lead one to conclude what many have always feared, that architects cannot be grown; they must be found.  Fortunately as Mr. Pink points out one can develop their right-brain skills just as we have learned to develop our left-brain skills.  He offers exercises for developing the six senses in his book.

So I believe architects can be grown, but to do so requires a program that develops both left-brain and right-brain skills.  And whereas experience does count for a lot in the discipline of architecture, I’m not so convinced that the experience one acquires from years of being a developer or a project manager is what one needs to be a good architect.

It is my hope that the architect profession evolves to the point where architect training starts much earlier in the career of individuals.  In fact I would hope (and I believe there are schools working on this) that architecture evolves to the point where colleges and universities offer IT and Enterprise Architecture degrees just as they offer building architecture degrees.  And these degrees will develop both left-brain and right-brain skills just as current day design school do.

Posted by Jerry Larivee at 6:57 AM | Permalink | Comments (2) | TrackBacks (0)

Changes are taking shape. Some are evident, many are like undercurrents, yet not prominently visible but slowly moving the needle, like global warming. I'm talking about the changes in the industry with respect to future of IT and IT organizations. Where businesses stand today and what they need in future, how much ready are IT organizations to make it happen for them?

While today we mostly ask the questions in terms of new technologies as far as the IT organizations are concerned for their readiness, I think its going to be far more fundamental that we will need to worry about going forward since the changes happening around will make many of the existing models, methods and patterns of IT services irrelevant. Lot of this has to be do with the changing expectations of how IT will be used in future, how products and technologies in IT are going to be like and how businesses are going to structure themselves to make business happen on IT driven business operation eco-system. Needless to say, that this eco-system is not just applications but digital devices, sensors and all that you can think of. In recent past I had quite diverse conversations with organization leadership teams, especially in the context of Business value strategy. Using that as a background and seeing what else is happening around, here is what I think should be top 5 worry for the IT organizations that are providing services to their business owners.

1. What is the cost that your organization is paying for your constraints? - constraints could be your service levels in terms of response time, could be quality, could be ability to resolve business challenges. Most important thing is that you as IT organization, are you aware of cost that business pays for every constraint that you bring on the table? You may not be watching but trust me, one who is paying the cheque for your services is watching it. Check it out.
2. Given a chance of better ‘service provider’ today, will Business owners replace you? - Don't get surprised. Many organizations treat their internal IT organization as yet another IT services provides and pretty much deal with them like one. Organizations have kind of got into mindset of looking for better options if their internal IT organization is not able to perform upto the mark. Options are always there, its just matter of making the decision. Its all going to be 'business value' game and every one will have to earn the cheque through business value delivery and not just by 'doing things'. Serious stuff, I think.
3. Will you remain relevant ‘tomorrow’ with changing patterns of technology evolution? - Changes are happening rapidly and some of them will change the game for businesses. Linking back to previous point, you as service organization, how much are you ready to enable your businesses to leverage the new technologies without reading putting much to stake in terms of risk? If you do not evolve, new technologies will make you irrelevant in terms of your servicing capabilities and point# 2 will make it even more serious.
4. Are you busy solving your internal problems or are you solving business problems? - Watch out where are the leaders and managers of your IT organizations spending their time on. If they are busy sorting out their internal issues and not really providing leadership in resolving business challenges through IT capabilities, there is lot of wastage happening. Its not more affordable.
5. How much can your business organization rely on you for their future bets? - Probably THE most crucial question that IT organizations need to think of. Can their businesses rely on them to capably provide support for the future business bets of the organization? Businesses are in transition and hence no one really knows how things are going to be turned out in future. But they are picking up their best bets and playing on it. If IT organizations are not able to support the future bets, that's lot to lose, big stake. So first, as IT organization, are you aware what those big business bets are for your organization and secondly, are you ready to make those bets happen from your end?

Is this scary? It should be. But again, its meant to show the good news also. While it is scary, it also means that current way of doing is not going to sustain so you have option to change the game. That's exciting part of it. You get to be part of the change and not just being part of the change but leading the change. Can you ask for more?

Posted by Rakesh Mishra at 8:57 AM | Permalink | Comments (5) | TrackBacks (0)