Infrastructure and capacity planning

When you hear the word “(information) infrastructure”… what comes to mind? Your servers, routers and switches, and networks? What about software and dependencies to other applications, and related Service Level Agreements (SLAs)? It is all of that. Plus it is people too, and the instructions for those people (procedures) about the infrastructure itself.

When it comes to infrastructure, you must do your homework and invest on infrastructure and capacity planning…

How heavy or light the infrastructure needs to be will all depend on your application's availability and capacity requirements.

If anticipating large amounts of users and transactions, and/or consumption of large amount of storage, you have to be ready to pay the big bucks in support for such infrastructure. The math is simple: the higher the availability and the capacity that is needed, the more equipment is needed to support such, from servers to routers… and storage and power consumption.

To reduce such costs of operations, and to ensure that your operations will run smoothly during high-volume peak times, you must spend time doing infrastructure and capacity planning. I'm not going to explain here all this in detail, but I will leave you with some pointers:

• Begin by making volume, capacity, utilization projections: on number of users, transactions, storage and power consumption you think you might need to support for the next year, broken down on a monthly basis. Note that to project deltas, you must know your current numbers. If you don't, then it is a good time to implement such monitoring and metrics gathering.
• Classify your application(s), based on their availability (uptime, disaster recovery) and/or other impact requirements. This classification is what is going to help you understand how heavy your infrastructure needs to be, and how much more it is going to cost. Include in your classification all upstream and downstream application dependencies.
• Understand the horsepower of your existing servers, and the available storage.
• Understand the current utilization numbers delivered by your infrastructure (servers, storage, network, …)
• Understand your current connectivity and bandwidth usage.
• Understand your current power consumption (if applicable).
• Understand dependencies to other applications, their availability requirements and past-record, and how it will impact you ($) if it goes down.
• Based on the projections, extrapolate equipment needed. Note that availability is solved by introducing clustering and load-balancing (use of more BigIP/3DNS and servers), but transactional capacity is solved by adding more and more servers horizontally. And obviously, to increase storage capacity, you need to add more physical storage.

  The mapping of projections to actual equipment needed could be an art or a science. You can either approximate (the art), or accurately map (the science) by understanding your existing equipment and software utilization (including resources consumed by your application) and their output capacity, and it also means you must do a very good job with the your volume and capacity projections, which means you need to know well your space or domain business-wise. Use your business folks to come up with accurate projection numbers.

• For procedures, there are SLAs, backup and disaster recovery, (users) guides, and so on.
• Understand what it will take to run your infrastructure people-wise. Keep in mind backups, disaster-recovery, support, and general system administration.
• Monitor system performance over time including actual server, memory, storage, and network utilization, and re-adjust as appropriate. Do this at least once a year, perhaps more often, and definitely
  when anticipating high-volumes due to a “special event”.
• Shop around for hosting vendors, and make sure they can support your infrastructure requirements. Compare cost. Also, their physical location might be important as you may want them close to you in case of emergency. One day, if your business/application makes it big, you will have your own data center…

Note that if you are developing a web-based application or Mashup today, there already are quite a bit of items in your infrastructure checklist – the above list gives you a head-start. If you are in the early stages of your product development, don't worry much about this, but the infrastructure and capacity planning itself must start early (pre-Alpha stage, so that you can start raising the cash early in support of such infrastructure) so by the time you enter Beta you already have a good idea of what your infrastructure will look like, and how much it will cost. For some companies, such infrastructure must be in place early, by Beta.

OK, so, from the Mobile Applications perspective, which is the main topic of my website, why do I write about infrastructure and capacity planning? Because mobile applications typically are end-to-end, connected, on the network – unless you are writing a standalone application, such as a casual game.

Infrastructure is very important and without it, the Internet, our (mobile) Internet-based applications, and our piece or presence in the cloud wouldn't be possible.

ceo