Friday 18 January 2008

Why a lot of large corporation software projects fail

I have worked with a lot of large corporations, and as a lot of others reading this might understand all too well, getting anything done always takes far longer than doing the same thing for a small business (SME)

Usually the people who hold the purse strings don't talk to the people who want the system, and there are company procedures that have to be completed before a design is signed off as ready to implement. In fact I remember an old job where the pressure really came on (from the accounts department) to get the code written and delivered. Then once we spent all hours hitting the deadline, we found out that the IT department weren't ready for the system for another 4 months, and it sat on the shelf until they were ready. Aaah well, you have to laugh (or cry).
Well a few years ago, I was working through another software house similar to mine, which in turn works for a very large corporation.
Because my contact wasn't the major company then I can sit back and laugh at the procedure for getting this product to market. Don't get me wrong, I feel sorry for my customer because he is now going through what we have gone through before, and you can just see the frustration building up.

I am not going to name customers or reference these on my website as a customer I have provided a solution for, so there is no need for you to try and find out who it is. The fact that I have my main customer as another SME shields me from the fact that the larger corporations can recognise me.
Some large companies are fine (in case any of my repeat customers are reading this - it's not you heheh). But this project has the main paying customer (Company A), then another IT security company handles, well the security (Company B), and another large IT company handles the hosting, and server (Company C).
We - well basically we do all the work.

I wanted to just catalog the list of errors, because it may entertain some, and it may also show some, the pitfalls to avoiding these situations in future.
So this is a system that has moved servers (decided by Company B and handled by Company C) and had an upgrade to server technologies, fully rewritten by me. Obviously being way down the chain, I cant get access to the server (there are many servers, ranging from development, to testing, to final server). We get an ftp to the development server, and there are some fancy replication things going on to get it to the other servers when signed off.

So, major balls up number 1:
The system gets written, and then goes to the development server and in turn the testing server, it goes through testing and some pages aren't accessing the database correctly. Instead of Company C seeing the blatant error message on the screen (Invalid database connection string) and sorting that out, the information gets passed down to my customer who then passes the problem to me. Except I cant get access to the server to find out the database user accounts.
So I have to write a simple page with a textbox to enter a connection string, and then attempt to read a record from the database. I put that on the server, and allow Company C to enter connection strings, or set up database accounts until my test page says (Connection Successful).
Then I can take that string and add it to the system. So what should of been a case of a network admin seeing the obvious error, changing the string and fixing it in ten minutes, this took a week to finally get fixed. But fair play to them, they managed to get the problem fixed and can tell their boss that they never compromised security doing so.

Major Balls up number 2:
The system finished testing and goes live. However the paying customer (Customer A) was never told about this, and now the public can use it, but the office admin hasn't been told about new procedures or account changes, and so get locked out of this very expensive system they have just paid for and don't know why.
Looking into this a bit more we find that Company B and Company C have somehow managed to work this so that there is no one person that you can point to to say "They are responsible for this whole system". Each decision and small change has got a committee with a leader, but that gives a list of people of whom Company A has no clue of which one to contact to say why cant our admin use the system any more?

Balls up number 3:
The development server is uncompiled code because it is development and small bits always need changing and you don't want to have to compile the whole site and upload everything every time a little change is made. Now because the rest of the system is hidden from us lowly developers, then this development code is replicated up until it gets to the live server. So now the live server is running uncompiled code, which isn't really the most secure or fastest way to serve a system. By keeping the developers out of the top end, they have really invited the hackers in. Anyway that is a small thing, and as you are thinking, at the end why don't I upload the compiled code to the development server to get replicated. Well if and when this job ever finishes then I intend to :)



Balls up number 4:
This one made me laugh because this kind of sums up the whole process.
There is a new small change that needs to be made to the system. An update. A very simple update that can take the newest entered record in a certain table (about 10 fields), and send its details somewhere in a standard formatted CSV file. Nice and simple, I can write that with my eyes closed to take the record and send all fields in the file. They can then import it and use the fields they want, and ignore the ones they don't. The update would take less than a day to do and get on to the server.
Well, I just heard that there has been a committee assigned with 4 days to decide which fields they need and which they don't want exporting. This committee is more than likely 3 people, and they are going to take at least 4 working days, so instead of paying one person (me) for one days work, this update is now going to cost the customer (Company A) 4 days for paying 3 consultants, added to my day to implement it (that's 13 man days) so were talking at least 1300% markup. Now knowing that I am probably the lowest paid in the chain, I can hazard a guess that in reality it is way more than a 1300% markup.


Conclusion:
Like I say I can sit back this time and laugh, because over the last 15 years I can give many examples of how large corporations who want control over something end up over controlling it to the point where it becomes far too expensive. There ends up being too many committees all pulling in different directions, until you get a product that is nothing like what the customer ever wanted. The argument is it has been done securely and everybody got their say all through the development process, but my word, it would be nice to be able to cut through all of the red tape, sit down with the customer (actually sit down with the customer's employees who use the system), and build something that the end user wants.
Aaah well, if I could perfect that business model to keep everybody happy then I would be a rich man.


Oh dear, this isn't so much as a rant because I am writing it with a smile on my face. I also know if my immediate customer reads this he should see the funny side.
But I am glad to get this documented. Please feel free to add your comments and similar experiences if you have any.

Cheers,

H

2 comments:

Anonymous said...

Thanks for your valuable information.

It was really of use to me.

-Prabakaran.K

www.usjobcareer.com

The No.1 Job and Career Search Portal
pren

Lindsay said...

A shining example.

Government UK offender tracking system a shanbles

It would be nice to say lessons will be learnt. But at what point through this project which overran from an estimated £234 MILLION up to £690 MILLION do you think someone stopped and asked if this was going to plan?
Which Committee stood up and said I am not going to blame another committee I am going to take charge...
It seems when it got to October 2007 someone actually looked at it and couldnt work out what £161 MILLION had already been spent on.

These are of course just numbers. But really, say it out loud to yourself one last time.
Some software company / IT contractors, and committees had spent £161 MILLION that they couldnt account for. They couldnt say what it was spent on.

The major skill of this group is their sales team who were able to blind the paying customers with science for 3 years while everyone spent and spent. But of course the sales team arent project leaders so they will tell you it isnt there respoinsibility to make sure you actually get what they have sold to you.

Ho Humm, I need to stop this comment before it turns into a rant and a claim of (I could of done that better)

Take care all...