How to build a high performance network appliance like routers using commodity hardware and off the shelf components

You can assemble a server from off the shelf components that will be able to sent and receive traffic in multi Gigabit speed. Here is an example of an 10Gps net card from Intel.

But can we turn this server into a high performance network appliance? Do we still need a dedicated hardware like for example ASIC, FPGA, low latency RAM and TCAM RAM in network devise so they can efficiently switch and forward packets with maximum wire speed.

Router hardware design plan

Looking at this presentation from 2012 https://ripe64.ripe.net/presentations/18-ripe-64-router-architecture-challenges.pdf you would think that yes. These would be the obvious reasons (screenshots taken from the presentation):

Network processing unit (NPU) and new hardware design

The key points listed above still hold. But the next generation network appliances will be rather build with a help of a multicore generic NPU using the power of parallel processing than expensive and purposely design ASIC. With the right software (OS - often Linux, drivers, firmware, SDK, and API libraries) you will be able to turn a conventional x86 server with a modern PCIe data bus into a high performance, low latency and high speed network appliance.

Netronome Network Cards Accelerate SDN and NFV Designs
100Gps FlowNIC-6xxx network card
Hardware reference designs for FlowProcessor NPU chips

Network operating system architecture

In my previous posts we took a look at how hardware networking vendors design and build modern systems. We concentrated mainly on the OS and software integration on the appliance:

• Arista network operating system switch architecture
• Network appliance architecture

Today I found another good article on Arista blog that takes this one step further and again advocates an open operating system design: Linux as a Switch Operating System: Five Lessons Learned. This is what Arista says:

• It’s okay to leave the door unlocked - get net admin access to the underlying Linux operating system tools
• Preserve the integrity of the Linux core - keep your product specific changes as small as possible to allow integration with already existing software out there
• Focus on state, not messages - being only maybe an average programmer I will not comment on this as this is clearly not my area of expertise
• Keep your hands out of the kernel - why to complicate your code if it doesn't bring any revolutionary benefit; besides it is much easier to find developers who know how to program in linux/libc than in Linux kernel.
• Provide familiar interfaces to ease adoption - everyone knows the IOS CLI so why to invent something new.

Lesson learned from Arista: reuse, integrate and trust your customers.

The company success would not be possible without a clear and consistent OS and tools set design. From my experience they are the second company (I saw this first on BigIP - F5 load balancers).