Confusing Business Complexity with Programming Complexity

Responses to my recent posting "Why Occasional Programmers Suck" were mostly positive, but I did get a few expected disgreements from offended "full time" programmers such as the following from Mark:

"I don't agree. What we doing is not simple. It is more than just putting parts together or drawing a diagram. I think our tooling has gotten better and has made it much easier. But Tools like VB and MS Access have done much to hurt our industry as it allows people to THINK they can program. So we end up with piles of unmaintainable code and code that can't deal with the complexity of business and the need for systems to integrate. Advancements of tooling should be to help developers develop, not end users." 

Mark misunderstood a bit what I mean by "Occasional Programmers"... It's not really the end users of business applications that need better development tools.  Developing Business Applications is an art that requires a proper mindset and many skills that you would not expect most end users to possess.Occasional Programmers are people who understand the concepts of programming.  They are folks who understand process flow, how to express rules, how to logically organize functions... but their full time jobs are something other than "writing code".


Where Mark's comment really missed the mark (pun intended) is with his tacit assumption that the "complexity of business" has much to do with the complexity of programming.

For example... I was recently working with some very experienced programmers who were completely aghast that IBM's Lombardi BPM doesn't provide support for rolling back database transactions that spawn multiple process activities.  In database programming, there is the concept of a "transaction" that spawns multiple database updates.  Many records can be "written" to the database as part of a single "transaction".  If for some reason you need to cancel the "transaction", you can roll back  (undo) all of the changes that have been made to the database.  The programmers I was working with naturally assumed that they'd be able to define database transaction boundaries in the Business Process that we were working on, to handle rolling back any changes in the event that the Business Process was canceled.

Programmers don't work on reality, they work on models of reality.  Canceling a process was, to these programmers, as simple as undoing any changes to their model.  They'd missed the fact that each process activity takes place in the real world.  People perform tasks, messages are sent, related activities and processes are started... "real" changes are made by the process.  "Rolling back" a Business Process when it's canceled has very little to do with restoring the state of a database.

The programmers building this application didn't fully understand the "business complexity" generated by canceling the Business Process in-flight.  In contrast, the Business SME on the project did know the necessary steps... and the Business SME was able to use the tools to diagram and define the necessary steps.

Here's the bottom line - programming tools intended for building Business Applications need to focus on dealing with common patterns of Business Complexity.  The underlying Business Machine worries about programming complexity... not an easy task by any means... but not the concern of the Occasional Programmers.

As an analogy, think of your car.  Your interface to your car is the steering wheel, the gas pedal, and the brake pedal.  With these three controls you are able to drive anywhere you want to (until you run out of gas). The complexity of the machine that accepts your input - what's "under the hood" - is somebody else's problem.  You just handle the complexities of driving the car.

This is what Object Oriented Design and Service Oriented Architecture is all about - providing the proper interfaces for those who build higher level (business) applications.  With the right interfaces, and the right tools, our Occasional Programmers can focus on dealing with those business complexities that Mark's worried about.

The Business Programmer Gap

I had lunch today with a good colleague who's a Rules Guy... Shocking, I know, to think of a Process Guy having lunch with a Rules Guy... but it's good to socialize with these folks in the hope that they'll wise up and "see the light" ;-)

Process Guys and Rules Guys have a common problem.  Our products can change the world... Improving efficiency, cutting through bottle-necks, eliminating fraud and waste... but only if governments make widespread use of them.  Governments kind of like the idea of using our tools, but they can't use our tools unless they can find a lot of people who know how to use our tools.  A lot of people.

Every Federal, State, County and City government needs our tools.  Big, huge or tiny... it doesn't matter the scale... governments need our tools to manage their procedures and policies - and they need people who know how to use our tools.

My colleague and I live in Santa Fe, New Mexico... not the smallest state capital, but by far not the largest.  Not the worst state government, but not the most stellar either.  I'm convinced that widespread use of Business Process Management and Business Rules systems within our state government could help dramatically improve our state's operations... but there simply aren't enough BPM and BRM folks in the state to tackle the problems.  Case in point, there's been an unfilled BPM consultant posting on the Santa Fe Craigslist for months.

I travel the world helping people master our BPM suite.  I would never say that there's "no coding required"... that's simply not true... but the level of programming skill that's required is way below that needed for truly custom programming.  You need to know some JavaScript and some SQL, and it really helps if you know a bit of HTML and how to parse XML, but you don't need much more than that.  Our BPM suite handles most of the heavy lifting on "the front end" of your processes (Truth in advertising caveat: Your traditional IT folks still need to do your back-end integrations).

Folks who have the right aptitude can pick up the skills that they need in a very short time.  Months, not years... less if they have the right guidance.  I know because I've helped them.

The problem is one of classic "Chicken and Egg".  I'd love to teach a class at our local community college (the Santa Fe Community College is awesome), but nobody in their right mind would sign up for the course.  There's no work for the graduate of a BPM program here in town.

We could solve that problem by convincing the State to launch some BPM projects... but they'd be insane to do so... There aren't any BPM folks available to staff those projects.

This "Chicken and Egg" conundrum isn't unique to Santa Fe.  Off the top of my head I can only think of a dozen cities in the country where you wouldn't have a problem staffing a BPM or BRM project.  Business programmers are a rare breed outside the urban centers... as I imagine database programmers were a generation ago.

We can fix that. We can close "the Business Programmer Gap"... but only if the "Chicken" or the "Egg" goes first.

Project Leadership versus Management

Projects need Leaders more than Managers.

I've probably just offended many Project Managers, but please read on...

My job lets me work with talented programmers and business people all over the planet.  These folks all know their jobs - the programmers know how to write good code and the business folks know how to run their business - but when you bring them together they often can't get a project finished.

The objective of the projects that I work on is to deploy Business Solutions. Requirements grow up to be Plans, Plans grow up to be Projects, but many Projects just don't grow up to be Solutions. Like the Energizer Bunny, these Projects just keep going and going and going... but they never get anywhere.

I blame a lack of leaders.

Look back at the American Civil War and you'll see that Generals who were great managers, such as George McClellan, were able to build great armies... but they weren't able to win the war.  Great leaders, such a US Grant, won the war.  Good management is crucial... but without a real leader to follow the army doesn't win.

I fear that leaders are born, not made (although training certainly helps improve them).  When a project is stalled, a hero can't just "take charge" and get a project back on track.  Leaders aren't leaders without followers.

People follow someone who knows how to get there.  Experience is crucial - we'll follow someone who has been there before, or someone who's been in similar situations. This is why the Technical resources on your project team will want to follow a Technical Lead.

Most Technical Leads will fight any attempts to turn them into Project Managers - they're leaders, not managers.  Management requires a lot of drudgery... Leading is much more exciting.

The problem with Technical Leads is that they tend to focus on the Technical battle, not the Business war.  You may have some really happy programmers, but a bunch of disgruntled Business folks sitting around griping that the techies are building the wrong things. Technology is a means, not an end... Tech Leads sometimes forget that.

What we need are real Project Leads... People who command the respect of both the Business and Technical folks.  We need people who have met both Business and Technical challenges in their careers, and who can show all of us "how to get there"...

That's why we lack leaders for our projects.  There simply aren't many Business people in the world who are respected by Techies, and not many Techies in the world who are respected by Business people.

This is not by any means an easy problem to solve - but if we want success for more of our projects, we need to find more of these leaders.