Web-Scale IT

With a constantly increasing amount of data and more complex application requirements, talk about so-called Web-scale IT architecture is on the rise. But what exactly does Web-scale IT mean?

The research firm Gartner introduced the term Web-scale IT in an effort to describe what the fine folks at Internet giants like Facebook, Google, LinkedIn, etc. have achieved in agility and scalability by applying new processes, architectures, and practices. These companies exceed the “scale in terms of sheer size to also include scale as it pertains to speed and agility,” according to Gartner. The research firm also named it one of the top ten strategic technology trends of 2014.

The term, Web-scale IT, is often used in the context of DevOps but it also applies to the underlying IT infrastructure – the system needs to be in a “known good state” to achieve agility at scale. And by now you probably know where we are going with this (hint: we pride ourselves to be the leaders in infrastructure automation).

Can Organizations of All Sizes Benefit From Web-Scale IT Methodology?

Good question and let’s think about this. While most organizations don’t reach the scale of a Google or the scale of a Facebook, they will still benefit from increased velocity that comes with the Web-scale IT approach (if done right). But let’s go even further, thanks to the availability of powerful open source tools like Hadoop, OpenStack, etc., Big Data and cloud techniques are no longer the privileges of hyperscale web properties and became available to enterprises of all sizes. That being said, with all these new tools and capabilities, many sub web-scale enterprises today run some form of Big Infrastructure and that brings new challenges. 

Disruption of the IT Infrastructure As We Know It

The shift to Web-scale IT represents a radical departure from the old ways of doing things in the IT world and as with every disruptive movement, it can be a scary transition. Web-scale IT requires IT professionals to be able to move faster than ever to deploy and manage Big Infrastructure. Infrastructure has become increasingly heterogeneous with commodity hardware, open source software, and home-grown provisioning and management software that make infrastructure difficult to manage at scale. Many steps are still done manually, are inefficient, and error prone.

To do Web-scale IT right, organizations must move to the next level of infrastructure automation, the level that understands the requirements of applications and responds to those requirements in real time – a software defined environment. In a software-defined environment, IT becomes simplified, as well as responsive to shifting requirements and adaptive through automation. Building and managing these systems is the “secret sauce” but it isn’t easy to achieve.

Hyperscale websites have the capital to build their own management tools that automate the management, configuration, and deployment, but that takes resources that enterprise IT infrastructure managers just don’t have. However, they still need automation if they want to achieve their goals. They need a solution that will harness the collection of commodity hardware and open source software, and make it work like a turnkey solution (from bare metal all the way to the applications layer) without the price tag of a proprietary system.

 What Would Such Solution Look Like?

Here at StackIQ we spend a lot of time thinking about how to make the lives of infrastructure managers easier. How can we help enterprises of all sizes to get to the next level of infrastructure automation and benefit from a Web-scale IT approach?

What do you think? To get the discussion started, here are a few characteristics that we believe are key to be successful in this new IT world:

-        Heterogeneous support for all commodity hardware and open source software

-        Capability to build the entire stack from bare metal to the applications layer

-        Modular extensibility of the stack to keep up with ever changing business requirements

-        Simplified deployment with script free configuration