Complexity knows only one direction: Getting more complex
Complexity, in general usage, tends to be used to characterize something with many parts in intricate arrangement.
Following this disambiguation, and assuming that “many” means N > 2, all systems with at least two or more components are complex. But that would be an exaggeration, right?
Why is information technology complex?
Most systems in information technology (IT) are complex. Almost everything we are working with, consists of two or more components, regardless if it is hardware or software. But it’s a question of the perspective. If you look at a system from a higher level, you will only be able to identify some of the greater components. If you look closer at it, you will be able to identify more, smaller components. Every system consists of hardware and software. Hardware is nothing without software. Think of a storage system, with all those disks, controllers, disk enclosures, firmware etc. Or think bigger: A complete infrastructure based on a VMware vSphere cluster with multiple servers, network switches, SAN switches, storage systems, synchronous mirroring between data centers etc. The system can be split into its components and sub-components. And each component and sub-component is more or less important of the operational function of the whole system. Adding more features makes a system even more complex. With each added feature or modified feature, the probability rises, that something breaks or doesn’t work as expected.
IT systems and infrastructure tends to act like a nonlinear system.
Matter of opinion
In fact, there is no uniqe definition of complexity. What complexity means, is in the eye of the beholder. If you understand each component of a complex system very well, the whole system isn’t complex for you. But the same system would be a “complexity hell” for someone else.
I’ve talked to many customers in the last months. Most of them don’t try to understand each aspect of their infrastructure. All they want, and all they need, to know is how to keep it running. And most of them have accepted that they need external help (like me), even for “simple tasks” e.g. adding a vSphere ESXi host, create new LUNs, adding VLANs, recovering a database etc. This also includes planned site failovers, even if the failover is done with a few clicks. It’s the order of the necessary tasks and the knowledge about dependencies, that prevents customers to do this on their own.
Interestingly, this is not only a problem of smaller companies, or smaller IT teams. I also observed this in bigger companies and IT teams. The only difference is, that you have more specialized staff in bigger organisations. One-track specialists. Knowledge is distributed over more individuals. Each and everything has to be discussed in bigger teams, meetings etc, because each individual knows only a small part of a bigger picture.
A typical reaction is to source less known, or “complex”, systems out. Out of sight, out of mind. I hear you say “Migrate to public cloud / IaaS / PaaS etc”. But does this make it more simple? No. Complexity is only shifted. Automation can reduce complexity! No, automation can hide complexity. You should only automate, what you have fully understood. A nice GUI can reduce complexity! No, a nice GUI can hide complexity. So there is no way out?
Solution?
One possible way out of the “complexity hell” is to try to understand most (not all… that’s nearly impossible) of the components, and how they are interacting with each other. This seems to be the best way, right? No, it’s not the best way. To achieve this, you would need to invest much more time in building up knowledge. Sure, that might be a way for someone who has time, or someone who is being paid for his knowledge. But this seems to be not the best way for most customers.
Another possible way out of the “complexity hell” is to try to reduce components. Keep it simple. Focus on a lean design. Focus on the problem. Focus on the requirements. Lean thinking and a A scientific approach during the solution design, can help to build less complex systems. A customer doesn’t need a synchronous mirrored storage or replication, only because he has two datacenters. Sometimes, distributing primary storage and backup into different datacentes is sufficient. Stop using iSCSI or Fibre Channel for three or four node VMware or Hyper-V clusters. Focus on SAS for storage interconnect. Skip 24x7 support contacts if the customer only works from 9 to 5. Design the backup concept from the recovery perspective.
You get no prize for the prettiest solution. IT isn’t a beauty contest. Build simple and robust solutions. Complexity can’t be reduced with specific products. It’s a question of the design.