Heong, when is the prototype going to be finished?

I learned long ago, never to wrestle with a pig. You get dirty, and besides, the pig likes it.
- George Bernard Shaw

Blast from the past: the Macsym 150, running Macbasic was an early embedded computer

Back when I was at my first job designing SCADA systems for Oil & Gas installations, a new system perhaps took two to three years to complete. You got from the customer what he wanted in detail, the 'requirements', produced a massive document, the 'specifications' and then proceeded with the 'design', and then the hellish part, 'testing' and hopefully the delivery and installation. These days it is called the waterfall model.

This was so with my first project 1985-1987, a SCADA system to monitor 4 Daniel 2234 flow computers. It was based on the ancient MACSYM 150 by Analog Devices, both now extinct. While wallowing in the now-familiar post-project blues, I asked to tag along for the big one- a building automation system for a whole building complex comprising tower block, museum, theater and library. It was critical to the company- in fact the Technical Director handled it himself. Being the FNG, I had to promise not to touch anything.

Heavy metal- PDP-11/23+

It was based on a Digital Equipment PDP11/23, heavy metal in those days. It ran Ultrix. A BAS is not that different from a SCADA. It required microprocessor-based RTUs.

By 1987 the project was in trouble. The company by then could not afford to import the rest of the RTUs nor pay for the software, but was still bound by contract to deliver the BAS. The performance penalties alone would spell the end. Desperate times required desperate people- going three months without pay does things to you.

We had the nucleus of the BAS- the PDP11/23+, one MTU and 6 RTU, so we knew how it should work. There was no choice- we produced the remaining 4 MTUs and 25 RTUs, copied it lock stock and barrel. We installed it and got a shock reject from the customer - the system was too slow. Despite the agreed-upon 'Requirements', and 'Specifications'.

There was little time to think. All I had on hand were IBM PCs. I got the newest one, an IBM PC/AT. The only choice of OS was SCO Xenix/286 and I pretty much wrote the software on the fly, alternating between customer site and office. The result was a 4X improvement in speed, and we got it accepted by the skin of our teeth.

Small beer- IBM PC/AT

 We had taken a properly engineered system, replaced major components willy-nilly and got away with it! It was a one-off, we had dodged a bullet and it was time to go back to the straight and narrow, the waterfall method. I could not have been more wrong.

Over the years working at Westinghouse, HP, Motorola, then ON Semiconductor this pattern was to be repeated again and again. The waterfall method does not work.

So, what worked?

Being close to the customers did. You visited them often. Better yet get a desk there too. This is because often the customers does not know what they want. They know what they don't want- if they can see it and use it they will soon tell you.

You then need to whip up a prototype to show the customer. The faster the better. It does not have to meet all the requirements - the customers are certain to reject even the perfect ones. But nearly every time they will tell you why they hate it.

You then repeat the process - rush back to the office, and repeat the entire process and rush back to the customer with a new prototype. Eventually they will relent and you will succeed. Most customers are flattered you are willing to develop a bespoke system for them. In fact most times the customers do not get to meet the designers. They just keep hearing the mantra from the sales or operations guys: "You have agreed to the specifications and this is what we will deliver".

But what of the cost, the failed prototypes, the components discarded? There is no choice but to reduce cost throughout the process. The man-hours are about the same, the delivery times are often fixed in advance. You start with the components on hand. Failing that, off-the-shelf parts. You use the minimum of custom parts. Software is a godsend. There is little material cost and it can be changed very fast, often right there in front of the customer.

And that is why prototypes are never going to be finished. For there is always the next customer. Sometimes over many customers the design may settle to a happy equilibrium, and you get to declare victory and move on, but don't bet on it.

You will see this pattern repeated in this blog- the prototypes sort of work but there is always something lacking. If I am lucky this results in fresh insight on the design. It helps if you like design and prototyping. For me, this is pachinko - something done for fun. Like wrestling in the mud.

Over the years, especially in software, this method has been given respectability and names, 'iterative development', 'extreme programming'. You automate it up the wazoo and apply it to large teams(indeed the whole company), and it becomes 'DevOps'.

That used to be just my fangs out hair on fire approach, a beast to be let out only when necessary.

Happy Trails, and see you at the pachinko.