IBM Automates the decision? I think not.

Wow, this article really triggered some emotions in me. The gist of it is that IBM sees a trend of companies becoming more fact based than intuition based. IBM sees a potential market for a server that can automate management decisions based upon analytical metrics.

It conjured up some emotions in me, of projects past. You know the ones, where a team member raises a risks or issue, then are challenged as to the facts that drove their concern. They couldn't talk in terms of facts, as it was their gut and past experience which was causing them unease (the classic Han Solo "I've got a bad feeling about this").

Understandably, companies are moving towards fact based decision making. After all, gut calls and intuition is humanly fallible, right? That's exactly what was the topic of a recent book I read called Predictably Irrational. It was a great book about the decision traps humans fall into because we are pretty good at comparing apples to apples, but not so good at comparing apples to oranges. I recommend the book.

However before we go and replace Architects and other Leaders with an oversized laptop, consider for a minute what it means to be an expert in your field. One famous model of skill acquisition is known as the Dryfus Model. It was conceived within the healthcare industry, when studies were conducted to determine how physicians diagnose ailments. This model defines experts as those who have had so much hands on /real world experience that they have internalized a formal process, and execute/iterate over that process at an incredible speed in their subconscious. Because of the speedy subconscious process, it appears (even to them) to be relying on intuition, or gut feels. However their instinct was found to be uncannily accurate. The study continued and found that one sure fire way to decrease the accuracy of the diagnosis, was to force them through a rigid process of explicit rules.

Will IBM succeed in automating decisions? I am not sure, but I for one am scared at thinking expert PEOPLE won't be making the tough calls.

It's about the Cathedral, man..

The past year or two I've really been struggling to define myself as an architect. As a developer, I defined myself by the code I wrote, and the solutions I crafted. The level of pride I brought to my work was akin to the love a master carpenter brings to their furniture making.

But more and more I feel like an outsider when socializing with my brethren technicians. Don't get me wrong, I bask in their genius and creativity and love for their work. Their passion alone fills me with memories of long forgotten times when I too was like them. And I hope one day I can regain the level of comfort and confidence that comes with knowing your purpose.

Peter Drucker wrote of a story which I've suddenly found meaning in, about three stonecutters. When the first worker was asked what he was doing, he responded "I am making a living". The second worker responded "I an doing the best job of stonecutting in the entire country". The third stonecutter replied "I am building a cathedral".

Drucker goes on to explain that the second stonecutter, the craftsman, can become a real problem in an organization.

This statement was in direct violation to everything I believed. I had defined myself as a craftsman for years, and may have thrown the book away in disgust if it wasn't written by a leading authority on leadership.

The problem, as Drucker puts it, is that a craftsman can too easily be working on tasks which subvert the organization. Polishing stones, or other fine detailed work that does not offer a benefit other than an increase in the feeling of pride of the worker. I was happy to see Drucker go out of his way to mention that craftsmanship and pride in one's work is a required trait of a successful organization. While the organization must contain craftsman, they must also realize they exist to serve a much grander purpose.

Bingo. A book written before I was born puts into words my feelings of disconnect with my organization. Here I was a craftsman, trying to be the best stonecutter in the country... And then suddenly I had been given a glimpse of the cathedral I was building. And like an addict, I've been doing whatever I can to keep the cathedral in my focus ever since.

I've realized that my once attention to craftsmanship was really just a form of purpose. I needed a reason to do what I do, and it manifested itself in the form of a code artisan. When working on larger concepts that drive business value, I am able to have an even stronger feeling of purpose then when I was a codeslinger. I just currently lack a way to put this new purpose into words and action, such that it can be sustainable... and that, is the source of my struggle to define myself.

The Software Factory

The Homework Machine - A Poem by Shel Silverstien

The Homework Machine , oh the Homework Machine,
Most perfect contraption that’s ever been seen.
Just put in your homework, then drop in a dime,
Snap on the switch, and in ten seconds time,
You homework comes out, quick and clean as can be.
Here is is – “nine plus four?” and the answer is “three”.
Three?
Oh me…
I guess it’s not as perfect
As I thought it would be.

It is an old concept, but has really garnered a lot more mindshare with the nationwide financial crisis. The Software Factory. I'm not talking about the Factory design pattern, but the concept that you can have a rigid set of process, guidelines and governance such that you can feed work to the factory and get results which are repeatable and predictable.

Anyone who has been accountable for delivering business value understands the boon predictability brings to software development. Understanding exactly how long, and exactly how much a software effort will cost would mean the business could make better decisions and make promises to clients and customers which can be honored.

Developers have pushed back strongly with the mantra that software development is an art, not a science. After all, effective engineers and architects require a certain amount of creativity in order to get through roadblocks, or perform the necessary steps to troubleshoot a problem. Consistency does not come naturally within the realm of software development. Giving the same problem to multiple teams will deliver multiple solutions.

Thinking of engineers and architects as cogs in a complex factory machine is problematic. First, those with experience with the software development lifecycle understand that developers aren't universally replaceable. Seriously, Fred Brooks even spoke about this back in 1975 in his infamous book 'The Mythical Man Month', where it is mentioned that research proved experienced developers saw a dramatic increase in effectiveness over unseasoned peers.

So if the conventional model of software factories (with the analogy as developers workers on an assembly line) is flawed, is there a model which might work? Perhaps.

Some folks in my company have been speaking for a few years about a slight twist on the factory model which might have some legs to stand on. The difference is first it all starts with a highly effective team. Erase from your mind the dingy gray coveralls of a faceless interchangeable factory worker. Now picture..... the A-team. A highly effective experienced unit capable of putting their unique blend of talents to work against unexpected challenges.

I love using the A-team as an analogy of a high performing team. When some folks mention high performing teams, they think of a team in which everyone is highly skilled at everything. But I think the A-team is closer to reality. While the A-team was a high performing TEAM, the individuals all had glaring weaknesses. Individuals on a high performing team don't have to be genetically engineered specimens with god-like coding ability, as long as the team remains effective. Mr T. was a horrible communicator, you'd never find him sweet talking the team out of trouble... that's what Faceman was for.

With a high performing team, you can flip the factory model on it's head and bring the work to the team. Once teams have been given a chance to be seeded with the correct mix of people (or better said, the correct mix of strengths and weaknesses), and given a chance to gel and find their rhythm.... why tear down the team once the project is complete?

Instead, if the team has become an A-team, simply send them out on another tour of duty somewhere else in the organization. Instead of interchangeable developers, you now have teams that can be dovetailed with projects. I think it may just work, if we take the time to properly seed the teams and foster the next B.A. Baracus.

Coaxing the organization to change

At a forum for the local Ruby users group, a member spoke of the symptom that many large organizations adopt: a resistance to change. In this case, the change being resisted is the Ruby programming language.

I was happy to see an acknowledgment that the industry the organization is in makes a huge difference in the context of the discussion. Technology companies (like IBM, Oracle, or Sun) require an influx of new languages (e.g. Ruby) in order to retain their status of product leadership. Whereas organizations in other industries only require enough technology to enable their business, they often don't need industry leading cutting edge tech.

However, a few questions were posed:

• What causes it?
• Is it worth changing? (or should we just deal)
• How do we change it?
  

I'll offer my own opinions assuming we are in the context of a non-technical industry.

What causes it?
Simple. The organization is in business to make money, not to make change. The early years of IT was like the industrial revolution. Mechanized computers entered the workplace and began streamlining and creating efficiencies like the cotton gin did for the American textile industry. But times have changed. All large organizations now utilize technology to drive efficiencies over manual labor, gone is the Industrial Age of computing. Technology no longer creates business value on its own, it is instead an enabler to the business.

To embrace change, an organization must tolerate failure. Rather than kill an obviously failing project, organization which do not tolerate failure push forward and drive these projects into a costly and failed implementation. Worse, some organizations cause those responsible for a failed project to disappear (quietly, or in some cases in the public square).

Likewise, a big challenge facing a technician or architect wishing to sell change to their organization is the past experiences of that company. Regardless how unfair, you inherit the sins of your forefathers... any technical improvement that was over sold and under delivered can be an albatross hanging around the neck of a new idea.

If those that came before us had well thought out and executed plans, there is always the chance that we are short sighted. Did we see all the nuances the change would have on the organization? An enlightened architect needs to remove their technical bias and ensure they are looking at the problem from all perspectives. Those you sell your idea too may politely respond to your idea with a 'no thanks' but internally be thinking 'this person has no ideas on the complexities of the organization'.

Is it worth changing? (or should we just deal)
Every organization needs a change agent. Someone who is pushing the boundaries, a few steps ahead of everyone else. Executives fancy themselves to be these heralds of the new era... but change can come from all levels of the organization, as the 'goodness' of your idea is not proportional to the level you sit in your organization.

Ask yourself why you wish to sell your new idea. Are you just excited with the newness of it? The novelty? Is it just 'fun', 'hip', or 'cool'? Do you honestly feel deep in your heart that it will improve the organization in a significant way? Make sure you aren't fooling yourself with techno-babble or buzzwords. Check out my blog posts on The Spin Factor and How To Communicate Your Ideas.

My advice is to avoid pitching your idea to a loved one or friend. After all, they think you are the best thing since sliced bread. Throw it at one of your critics, and see how they digest it. Be sure to be open minded and don't simple throw away their critique as hurtful venom. If you don't have good responses to the feedback, then your idea may not survive the intense focus an executive in your organization would give it.

How do we change it?
Change isn't too hard, you just need to potentially get out of your comfort zone and follow fhree of my maxims:

MAXIM #1 - understand what the driving factor of your company is. Is the biggest factor keeping total cost of ownership at an all time low? Then you need to explain on those terms why your new idea will save the company big bucks. Resist the temptation to assign more than one business benefit to your idea, as it will just water down your message. If your idea can't stand on it's own with a single benefit, it may not be worth doing. If you say your idea will save money, it better eliminate physical assets the company owns (people, hardware, software licenses, etc). Otherwise it is often not classified as a 'savings' as much as an 'operating efficiency'. Understanding how your organization deals with the financials helps tremendously.

MAXIM #2 - don't even think about over promising and under delivering. Stop talking about how your new idea will save millions of dollars, increase the efficiency of the organization, and cure world hunger. And you better not downplay the cost of the change. Ensure your calculation (yes, here is the comfort zone thing... you need to come with some numbers relating to the cost in dollars) includes not only the amount of time it will take, but also includes any re-tooling needed to be done (of hardware, software and PEOPLE!). If you have gotten to this step, don't fall into the other pitfall which is to assume everyone is as smart or efficient as you. You need a plan to bring everyone in your organization along for the change... yes, this includes Jim... that developer who can barely code his way out of a wet paper bag. You better have a plan for training him too, because letting him go isn't an option.

MAXIM #3 - personally take on the risk of the change. All organizations are driven by people, and most people are resistant to risk. There is fear and uncertainty, and doubt. Let me remind you that the question was posed by a member of a local Ruby users group... a group whom I could easily state are early adopters which makes them more naturally prone to accept the risk that accompanies new things. But to these types of people, problem solving is a skill that comes naturally. The rest of your organization is most likely not like you. Therefore, state very openly and explicitly at your personal commitment and willingness to take on the accountability of the change. This means you should be shouldering the brunt of the training of the newbies, or personally working very long hours to get through a trouble spot. If you are still not clear on what I mean by accountability... then let's just say you job is on the line, and you'll be fired if this idea doesn't pan out successful.

The Spin Factor

When I clean out my file cabinet, I sometimes imagine myself as Indiana Jones. The files in the cabinet are akin to dusty tomes of knowledge from a long lost civilization. Long forgot assignments and projects that upon seeing the once familiar powerpoints and specifications come flooding back into my mind.

One of the most educational finds was documentation of an aborted attempt to migrate an internal application to J2EE. In one hand I held a presentation made in early 2004 called 'Leveraging J2EE' which was a pitch to migrate our legacy application to a new technology. In my other hand I held a powerpoint made just 6 months later by the same team as a plea to kill the project.

The details of the project are irrelevant, but the wording of the competing presentations is enlightening:

'Levereging J2EE'	6 Months Later
It's component based architecture offers: Ease of Application Development	Extensive coding required to achieve comparable results [to legacy code and techniques]
J2EE helps us meet our End-to-End performance needs.	...multiple page refreshes, which leads to usability and potential Performance issues.
Rapid development to meet changing business needs	[Legacy Coding Techniques] require relatively less coding due to RAD features [compared to J2EE]
Higher Scalability at lower costs	Scalability: not a major concern for [this application] due to small user base
Java based technology offers platform independence	[Interoperability is] not a major concern for [this application] due to small and internal user base.
[Java allows] business logic to be organized into reusable components	Within the context of [a system such as this] reusability is very minimal

---

The thought of these competing arguments coming from the same project team can be troubling to some. How could the same team flip to the opposite point of view in 6 months? Some may conclude they were not authentic in their proposal of their first presentation. Others may suggest that teams which undertake conversion projects such as this can't possibly predict the reality of unknown risks encountered after initiation. However I believe this to instead be a good example of how architects must be very careful in how we spin strategic efforts.

Cast your dispersions elsewhere, as I can vouch for the presenters. They were genuine and sincere in their belief of the benefits of J2EE. They likewise were armed with enough detailed knowledge of the system to understand the potential pitfalls at play.

Instead my opinion is that the effort sulfured from what I have begun to call 'kitchen sink' strategies, in which every benefit but the kitchen sink is attached to our next great idea. Being engineers at heart, we have a habit of attaching too many business benefits to our sales pitch (faster time to market, lowered total cost of ownership, and world peace). This is why me and my team challenge those around us to only attach a single benefit to strategic efforts. While it may be logically feasible to reap the rewards of secondary and tertiary benefits, my experience has been that when a project team attempts to deliver on multi-benefit efforts all of the benefits end up watered down to the point that none of them are delivered.

I feel this strain even on my smaller and more tactical efforts. This is due to the fact that some business benefits are in conflict with one another. Increased performance and availability often comes at the cost of an increased total cost of ownership and complexity as more servers, caching, or redundancy is added to the solution. Having a single pinpoint focused benefit will help guide project teams when decisions are to be made that may negatively impact the primary desired business benefit.

I decided to keep the presentations in my cabinet rather than toss them, as they are also an excellent reminder of how as an architect I must be careful in how I attach business benefit buzzwords to my own strategic efforts. It is too easy to claim a newly emerging technology will dramatically impact Reusability, Performance, or Time to Market. I've learned the hard way that new technologies can't give you ANY of these results. Instead, good design produces these results. Success is possible regardless of the technology you have at hand (new/old, COBOL/Java), but a good design is a requirement for success. Therefore we must cease assigning these buzzwords to technologies, and begin assigning them to designs.

Behold, the power of the spreadsheet

Excel often receives an upturned nose from techno-files. Our deep love for technology has allowed up to appreciate the richness of relational data. Our need to manipulate information has transcended SUM() and AVG(). However, the power of the spreadsheet has been mastered by many of our business partners.

On my travels, I have encountered business users with the ability to manipulate information within a spreadsheet with as much effectiveness as a developer. The spreadsheet remains a means for them to perform powerful transformations of data without the need for a deep technical background. Code turns them off, but advanced excel formulas don't. Anyone with a basic understanding of math can comprehend Excel formulas, reducing it's threat level.

Despite the ease of use within the business community, many enterprise architectures ignore this fact and do not offer easy ways for Excel to be utilized to both export and import data by the business users. Personally, I've been burned by the misuse of tools like Excel, since data can be manipulated without the audit trails and validations of the typical system... but it is without question that Excel is a more efficient front end data entry tool for massive amounts of data than many enterprise class systems I have encountered.

How to Communicate Your Ideas

An enlightened architect has to be well versed in getting their point across. The critical ability to drive people and technology towards a common vision requires a knack for conveying intent and meaning. Often our years in corporate America or our mastery of technology robs us of our ability to speak coherently... filling our mouths with technobabble words like factories, engines, and services and corporate-speak like leverage, synergy and paradigm.

I'll never forget in high school, when a friend of mine proudly proclaimed they had started a new job. "I'm a fuel transference technician", they boasted. "What is that?", everyone asked with a puzzled expression. "I pump gas". Amazingly, our daily lives as architects and developers are riddled with similar technology colloquialisms.

On a recent project, a developer had spent some time building a new component which I felt was architecturally significant. I had requested that they spend extra time putting good comments in the code, since the component contained complex logic... and I felt it was extra important that the intent of the component be known to future maintenance programmers. What followed was a paragraph of buzzword filled jargon that was difficult for even ME to understand what the component did. Pulling in another developer who was not familiar with the component confirmed that the documentation didn't do an adequate job describing the intent. I asked to original developer to tell the review what the component's purpose was. They did a wonderful job summarizing the details of the component, to which I replied "That was awesome, I want you to remove the existing comment and have you replace it with the story you just told.... word for word".

Part of the trouble is that we are surrounded by examples of poor communication all of the time. Corporate memos, articles, and many textbooks overuse fluffy language. I've lost count of the business meetings I've attended whose single purpose was to 'wordsmith' a document. Wordsmithing is just a fancy way of saying you are cracking out the thesaurus and injecting industry buzzwords to give the document an extra air of credibility. Surely if a document is chock full of complex language it is more believable, right? wrong.

Corporate-speak and technobabble need to be fully eradicated from your language. Ok... even the best of us slip from time to time. The goal should be to stop judging your worth by the complexity of your ideas and statements. In fact, it is the reverse... the more simple your idea, the more simple your communication the more enlightened you truly are.