Questioning assumptions and being willing to deviate from your boss’ instructions can be scary.
It was about ten years ago, and the director of engineering took a chance on me. She assigned me to work on a mega-multi-million dollar program. This program also had several million dollars in incentives if the project achieved milestones ahead of schedule. This project was a big deal, and I was assigned to be the first team member of the networking engineering team.
The team only had the manager and the technical lead. The team needed to build a large multi-site and resilient network for this important customer. I had never worked with these two leads before. Yet, the technical lead told me I would be responsible for designing and building the voice over IP (VOIP) phone system.
I had never built a VOIP phone system before. I was still relatively new to networking engineering too. Now another person was taking a chance on me.
Usually, I would have been too afraid and chickened out. This was a big responsibility with a lot at stake.
I did not realize at first that this customer relied more on the phone systems than on the computer systems to do their work. The phone system had to be more resilient and better designed than the computer systems. (Of course, the network had to be just as resilient to support the VOIP phone system too.)
To top it off, I would be the only person designing and building the phone system. In contrast, the manager would bring in several people to work on the network.
How could such an important system be assigned to someone who had never even built one?
Rather than chicken out, I went against my nature and took on the challenge.
I spent two months reading the customer requirements and the VOIP system technical manuals. I made sure I read every page of those manuals and the customer requirement documents. I ensured every requirement could trace back to the VOIP functionality presented in the manuals.
When it came time to meet with the technical lead to discuss design, I answered most of the questions. I took action to research those I could not answer.
The technical lead had built phone systems with bare-metal systems. Learning about his experience over the past several decades was eye-opening. A lot went into powering and supporting a “simple” phone call.
I typically would have followed the same patterns and the guidance from my technical lead. Why not? He had several decades of experience when I did not. Yet, it did not play out that way.
I had several meetings with the sales engineers from the company from which we would be buying the VOIP technology. Those people were very knowledgeable.
I had remembered reading about virtualization technology, and I inquired about it. They assured me this new technology would achieve the same results as the bare metal solutions. I took a note and researched it some more.
The more I read about it, the more I was impressed. This seemed like the future. I asked some fellow network engineers about this. They had not used this yet in any previous system. Would I dare take the plunge and propose a new technique for this high-value customer?
I eventually proposed the virtualization design to the technical lead. He was pretty upset and wary. This was new. This was different. This seemed very risky.
I remember he told me something like, “We will try it and present it to the project’s technical director. If he doesn’t like it, it’ll be on you.”
This seemed acceptable. I agreed.
The technical lead presented the design to the technical director. He had some questions and seemed pensive. He asked some questions on design items I had not considered. After all, he did not become a technical director without thoroughly assessing systems for resiliency.
I took note of the technical director’s concerns.
What happened next shook me and stuck with me.
My technical lead turned to me and said, “I knew I never should have listened to your virtualization idea.”
I felt I messed up royally.
As I researched the answers to the technical director’s questions, I realized that the questions were good. He was not questioning the design but making sure the design was resilient. There were no negative comments from the director — only from my lead. Maybe the director could accept the new virtualization design.
I found the answers and submitted them to my lead, who presented the findings to the technical director. The technical director liked the explanations and approved the design.
When it came to procuring hardware six months later, we did not have to procure separate hardware for the VOIP system. We could simply use the servers that we already needed to buy for the data center we were building. Furthermore, we could simplify the network design too.
By adopting virtualization for the VOIP systems, we were able to save about $1 million in hardware costs.
(As an added bonus: the network engineers and the technical lead had a much easier time installing the VOIP system at all the customer sites. It would have been nice to have been there too, but I could not travel for personal reasons.)
Preparation was essential to achieve this good outcome. I spent a lot of time working on my assignment. With the knowledge I had acquired, I could spot an opportunity that I would have missed otherwise. I respected my technical lead’s and the technical director’s concerns, and made sure I answered any questions as best as I could. Having the preparation in place, I was able to take a calculated risk that paid off.
Don’t be afraid to take a risk, especially when you’ve done your homework.
Miguel is a Principal Engineer and the author of the “Serverless Security” book. He has worked on multiple serverless projects as a developer and security engineer, contributed to open-source serverless projects, and worked on large military systems in various engineering roles.
Originally published on Medium