If you have read any of the interviews here and my or Russ’s articles in the past, it should be obvious how important knowledge of the domain is when you are going to automate something. Before I came into IT, I worked in the automotive industry with safety products such as airbags, belts etc. I would like to tell a story from that time that relates to what we are doing today with network automation. Sometimes viewing something from another perspective can help clarify concepts in another domain.
Companies like to make money. And save money. That’s kind of the point of running a business. What we would do at the company I worked for was to assemble the airbag into the correct steering wheel, packaging protective curtains that protect against glass splitter in a crash etc. Most of this work is not very complex and is therefore subject to be automated. Before automating though there are many things to consider.
Does it make sense to automate? How many products are we shipping? Can we ship more when we automate? Can we lower the cost by automating? Can we decrease the number of faults when automating?
The number of products we would ship would depend on the number of cars ordered for the car vendors we were supplying to. Producing more airbags won’t do anything good unless more cars get ordered. Using automation to produce more is not an issue in that case as we were able to meet the demands by doing it manually. The same holds true in a network. If the networking team can keep up today, then automating for speed is not the main goal.
So what about cost? How much does it cost to produce one airbag? This depends on materials used, number of staff involved in producing it, salaries, number of faulty units, people on sick leave and a lot of factors. If the cost of producing one unit is made up of 95% materials and 5% other costs, then automation may not do a lot. If on the other hand the cost is 20% materials and 80% other costs, there’s a lot go be gained by automating. In network automation, the goal is generally to decrease the OPEX by having fewer people operating the network. Going through the same thing that happened in server virtualization where a server admin today can handle a lot more servers.
So before implementing automation in the automative industry, someone had to calculate the benefit of doing it. A person would visit the factory and time different sequences. How much time does it take to pick up that part from the belt? How much time does it take to screw the screws? How much time does it take to package it and scan the bar code? What’s the total time of producing one unit? This data would then be used to compare both costs and time saved by building a machine to do the same thing. With network automation this would be how long it takes to provision a VLAN? How much time to update an ACL? How much time to update devices with a NTP server? Keep in mind that this may be quick on a few devices but can take a lot of time in a large network.
When thinking about automation, it’s easy to visualize the benefits and sometimes the challenges are not given as much consideration. Going back to the example above. The human is far superior than machines in some areas, especially if the machine hasn’t been “trained”. One example is that a human can see if the finish on the airbag is poor because it got scraped by a sharp item. A machine can’t see anything. It needs a camera and it needs software that says which one’s are good and which one’s are bad. This is far more complex than it sounds and often generates a lot of false positives, especially in the beginning. This must be taken into consideration because shipping faulty items is a risk and can increase the cost of doing business, not to mention that shipping faulty products could risk human lives. Another example is that a human can screw a screw and know when the screw is in place. A machine needs to measure the momentum in Nm to know when it’s done. It would be better for the human to also do this and this was normally done but the point is that the human has builtin senses it can use to know when something is done while the machine does not. The same holds true in network automation. A human while logged in can notice if something is not behaving as it normally does. If a command didn’t “take”. If something got logged to the TTY. A script running needs additional logic to capture this.
The point of this post is that automating networks is not much different than automating other things. We tend to develop tunnel vision and don’t learn the lessons of the past. We have been automating things for a long time and there’s nothing special about automating networks. I’m not against automating networks. Not at all. But we have to be intelligent about what we do. Does it make sense to automate this part? Can we move faster? Can we be more accurate? Can we free time for our engineers to work on more challenging work? If the ROI is there, go for it! But don’t ignore lessons of the past. Don’t think that automating networks is a special snow flake. Don’t automate if the use case is not there.