Hello my friends

Its been a while since I last posted. Even though we have quite a few drafts ready, but since each article is 100% written by me only, and I value quality more than quantity, it takes its own sweet time. And being a passionate techie, a founder, content creator & social entrepreneur cum coach has my calendar quite booked these days. If you want a glimpse of where I live, and what else do I do or like outside of tech, you may check this Linkedin post to get an idea (and maybe visit us someday too! :)

About this blog

In this blog we will discuss the role of engineering infrastructure for a company to be 10X from now to long term. This blog is for CTOs, founders, tech leaders but as well aspiring engineers and future leaders to know how awesome tech orgs are made :)

As always, let me add the disclaimer that here I am presenting only my personal views as proposals. You are urged to think upon and verify yourself as discussed in my notes on scientific enquiry in the primer blog for the 10X Engineering series.

Let's take today's dynamic technological landscape, where software development stands as the backbone of industries. Every Chief Technology Officer (CTO), tech leader or founder grapples with a set of KPIs and critical questions. These queries serve as the compass, guiding the creation of highly scalable, high quality engineering machinery, which can sustain 10X velocity, maintainability & agility over the course of time.

A company's KPIs in tech ops

Overarching goal

Shipping products with highest possible quality, long term maintainability and agility, in least possible time and cost, with least experienced engineers.

Pure Tech KPIs

Some can be directly measured through automation and processes. Some can be indirectly measured.

• Story Throughput Per Developer Per Sprint - the higher the better

• Bugs and performance issues encountered every month (Bug Introduction Rate aka B.I.R.) - the lower the better

• Average turn around time to debug and resolve an issue - (Mean time to resolution M.T.T.R.) - the lower the better

• Average time, cost and resource (developer) per story point shipped - the lower the better

• Average experience of team members - lower the better

• Average time to onboard a new team member on existing projects - lower the better

• Average lines of code per story point (shipped till date) - lower the better

• Maintainability, Agility, Reliability and Performance - higher the better

• Technical debt - lower the better

• Cost of infra and entire engineering ops - lower the better

• Compliance and security - lesser gaps are better

• Timely releases - Higher the better

And this has to be shown consistently over years, not just first six months (or first lap) of preparing a demo, launching a new product or a new startup where one may even cut corners to get the MVP rolled out fast.

A CTO (or tech org's) need is of striking intricate balance between business needs & constraints with the KPIs.

Questions of every Tech Leader

As the goals and situations of your business evolve constantly and as members come and go, each with their own experience and style of development, how can you ensure perpetual productivity, effectiveness and reliability of your software delivery?
Is your approach cost-effective, making optimal use of resources, while simultaneously removing obstacles for the rising stars – the younger developers who represent the future of your team?

How can your infrastructure adapt and scale six months from now and remain so for long term future? Can the foundations and buildings you lay today withstand the test of time, remaining robust, adaptable and effective two, five, ten years into the future?

As I discussed in one of my previous blogs on 10X Engineering Mindset, the code we authored for iXigo 16 years ago is still running its core, as the 4 engineer startup from then has scaled to being a highly profitable OTA in India, about to launch its initial public offering in June' 24. Here is a snippet from the same.

Can you imagine the meaning and implication of "code once shipped remaining valid forever", from the growth standpoint of a tech centred business?

These questions are not mere reflections; they are the cornerstone of a thriving software development journey. By the end of this blog we will mention the pillars of developer infrastructure, each representing a crucial aspect of the software development lifecycle. These pillars are not just standalone elements but interconnected components that shape the efficiency, collaboration, and adaptability of the entire development & delivery process.

Now where is your company?

In all typical API and event driven systems, the business logic, schemas and configurations are less than 10% (tip of the iceberg) of what it otherwise takes to run the software development and delivery operations from end to end. But till date, most of the global teams whether startup or enterprise, own also the rest of the iceberg (the 90% infra part).

A common engineer doing things which are more than business logic, without guardrails, makes companies sluggish even though they may say - it works! And it is the definition of "what works", which makes the difference between the average and the great (or 10X) businesses.

The more more work your teams owns, and the less infra and guardrails you have, more is the

• Complexity: Which the team is expected to handle

• Time to market: Time taken to ship features and products

• Design mistakes and chaos: More complexity and more heads to think means increased probability of design mistakes & diversified implementations across projects

• Bugs and performance issues: More work and less guardrails means more chances of errors and technical debt.

• Incident resolution time: Finding the root cause of error could turn out to be finding a needle in a haystack. And fixing it may be a nightmare depending on your technical debt and design choices in the given project.

• Skill level and learning curve: More expertise needed to learn, develop and debug the more complex stuff

• Technical debt and chaos: When there are more code and design decisions to do, and given each team's unique design sense, preferences and perspectives, each project gets implemented differently. This means its scaffolding & solutioning, technical debt and challenges are also unique to it. So basically the organisation's code bases diversity, complexity & problems grow proportionately with the number of projects, features and members it adds over time.

• Tribal Knowledge: Only the developers who develop a particular project know how it is implemented.

• Higher switching cost: Varying implementations, designs, problems and tribal knowledge in every project increase higher switching costs

• Lack of manintainability and agility: it becomes more and more difficult to refactor or enhance your codebase or replace integrations or cloud. This also means more work to do for compliance and security checks. This reduces your agility as a company.

• Bottlenecks: All of this may lead to various bottlenecks for growth, in form of software's lacklustre performance, increased technical debt, lack of maintainability, talent bottleneck, tribal knowledge bottleneck etc.

• Cost to org: More time taken and higher level of expertise required means shooting the development and delivery cost by many times.

• Competitive risks to org: You may soon have competitor(s) who are faster, leaner and better than you in shipping products and features.

• Loss to org: More costs, more time to market, more bugs, more downtimes, more migration efforts, more risks - all of these also mean - lost customers and revenue opportunities, making you less competitive.

The Two Paths to 10X

In this post shared on Linkedin recently I talked about which strategy has better chances to make a company 10X tech company.

A. Striving to maintain a 100% 10X engineer team?

B. Reduce the complexity and scope of work by 10X (and use guardrails)?

Let's visualise the iceberg of software development and delivery

Should your application teams work on

A. The actual business logic (tip of the iceberg) or

B. Business logic + rest (the whole iceberg) - rest includes everything which is not business logic but is the infra or boilerplate required to develop and deliver your software at scale.

Now visualise the guardrails too!

Guardrails protect us from going in the wrong direction (astray).

Should various teams be allowed to take their own path in every project, or follow an org wide standard with best practices baked in?

Absolutely Essential Guardrails for 10X

Here are some guardrails that I really believe are absolutely essential for any software delivery well done. You can not remove even one of them and expect to be 10X.

• Schema Driven Development

• Modular Architecture

• Security & Compliance

• Configure Over code

• Scaffolding, IDE and Developer environment setup

• Automation - in infra setup, development, delivery and reporting

Guardrails can be baked in at these four stages

1. Infra setup (First time setup of the infra and system layer shown below)

2. Develop and commit code (scaffolding & framework used + pre-commit hooks)

3. Code merging and integration (CI pipeline with automated scans for testing, security and compliance) - can be configured using Github, Gitlabs, Jenkins etc.

4. Deployment (Deployment, migration and rollback process) - Part of your CD.

5. Post deployment - Continuous security and compliance scans, monitoring, auto-scalability and self-healing)

The layers of typical software delivery

And how a rightly baked infra can help manage them. The business dimension is the coding (business logic and data) layer - requires fourth generation frameworks. The System and Infra dimension requires a platform driven automation thinking.

System and Infra layer solutioning includes

A. Infra: Setting up hardware & network infra plus common system services on cloud of choice. System services cover CD service, monitoring and other shared services like messaging, IAM, policy engine etc

B. Application deployments through CI/CD/CT with testing, compliance/security checks, monitoring and auto-scaling - Automated setup pipeline and checks for delivering domain services over K8s.

Goal

Imagine you had everything available to get started with the choice of your microservices in 30 minutes, for any new product or business line, with costs, data, no lockin & maintainability!

The 7 Pillars of 10X Developer Infrastructure

When designing infra for 10X, here are the 7 pillars which I believe are absolutely essential and can be brought in at different layers and steps of your software delivery. Without even one pillar working well, your temple of 10X engineering work will falter (or is already faltering).

In my view, every CTO, engineering lead or founder needs to ensure these pillars are robust and working fine in their organisation

1. Standardization and Maintainability (published)

2. Productivity and Democratization

3. Collaborative Excellence

4. Testing and Quality Assurance

5. Deployment, Monitoring and Scalability

6. Development Environment

7. Security and Compliance checks

In future entries, we will deep dive further into these pillars of the organisation's infra.

The Dilemma of a Tech Leader

It is not uncommon to see tech leaders grapple with the needs of the hour, and never getting time to solve the needs of all the future hours of the company. This means they are constantly reacting to situations with short term measures addressing immediate needs. As for the future, they make do with faith - "that things will somehow work well".

There are two main types of situations with leaders here

• A. The current state of affairs is acceptable to them, even with the multitude of challenges. I believe this is because

  • Ignorance - They do not have information or can not visualise their gaps and how much better their company could be tech wise

  • Lack of motivation for bold steps - This could be for N number of reasons, but is truly detrimental for such an organisation.

• B. The current state is not acceptable to them but,

  • They don't have resources to modernise and enhance their tech ops

  • They have the resources, but their developers are resisting bringing anything new because they are comfortable with whatever is existing today. The case where you are comfortable with "a known devil" and don't wish to step out of the "comfort zone" to try something new. This is detrimental not only for the engineers career, but as well the tech leader and organisation, if they let it be.

Its critically important for you as a tech leader to see your company as an integral being. This being exists, will exist for many years and probably outlast you and all the teams and developers working for it today. When you see this integral being's needs and challenges from the birds eye view, you will gain the right perspective to solve problems from the being's standpoint. If you do not have the being's view, you will be dealing from situation to situation, project to project and team to team - never working with a cohesive and deterministic company strategy, never achieving your fullest potential, compromising and saying to yourself for the next few years "It works for now!"

Is "it works" acceptable to a 10X mindset, when they know things can work better?

Every CTO and tech founder should contemplate building a company whose projects are scalable and maintainable from day one - She should fight if needed, to prevent technical debt. The questions of scalability, ease of adding features, and making developers productive over time should be at the forefront.

Being 10X is not a choice. Its a need. You better realise it.

Question Yourself

So, is your company and project teams working on the tip of iceberg today or the whole iceberg today?

Lets set aside the notion of 10X, even if you achieve 2X, you will not only save half resources, half time to market, half costs, but you will increase your chance of business success by manifold.

Would you like to settle for less when you can have much more by a little effort and investment?

As well feel free to reach out to me or us in case you wish to have a friendly discussion on any of the aspects mentioned here.

The Conclusion

Taking the analogy of iceberg. In order to be 10X, tech companies need to reduce their scope of work by 10X and introduce guardrails based infra. This means you need to ensure your teams work only on the pure business logic, configurations and schemas. Everything else below your business logic should be left to a pre-tested, reliable infra and process automation setup.

You can adopt proper infra, processes and practices which handle or abstract 70-90% work of the software delivery lifecycle. You can have a strategy to get high quality work done by fresh engineering graduates from a tier B town, or even aspiring school dropouts with 1-3-6 month training program. Even if people switch between projects, cloud or language ecosystems, they can expect to see a standardised implementation, making the switch fast and easy. As a CTO or tech leader I will take special efforts to establish infrastructure & processes with guardrails and automation, to ensure we constantly move with 10X velocity from the short to the long term.

Our Mission at Godspeed

At Godspeed we envision to assemble together a stack once and for all for tech startups and enterprises and make it available in an equitable fashion so as to help them solve great engineering problems with ease, with tech as their enabler.

The Meta Framework already covers few of these guardrails when developing apps currently over Nodejs. Its polyglot implementation is expected to release this year with couple of more languages. We are also adding stack for automating infra and application deployment with guardrails.

In case you do not use Godspeed's stack for whatever reason, you will need to think and develop your infra on the same lines.

Reference

Curious about Godspeed's Meta Framework?

Check out

A. Getting started section and guide in documentation

B. Video playlist introducing the concepts of Godspeed along with these guardrails