A Vanilla Solution to Network Quality Assurance

Case Studies

The following examples illustrate how we have successfully used CustardWare to analyse and manage problems, usually resulting in defining a clear way forward to resolve issues.


Case 1 - Major Broadcaster

Project Requirements
This major broadcast customer required an enterprise level diagnostic tool. This would summarize the vital health signs of all servers and routers on the their network. It would also provide detailed diagnostic information of the WAN routers at the OSPF level, to provide a full enterprise detailed view of the network. Further requirements included the need to integrate with existing asset management system to validate host network addresses and system configuration information.

The Solution
Layer3 created a state of the art diagnostic tool which satisfied all the above requirements and integrated seamlessly with their Asset Management system.

The system comprises a web interface to give secure and simple access to reports and configuration of the system. Details of servers and routers are automatically imported to the system from the Asset Management system. This information is verified against the devices themselves to ensure the integrity of both the local database and the Asset Management system data. Exception reports are generated and the feedback is returned to both the Network Management and Asset Management/Change Control teams on a daily basis.

The system uses SNMP to monitor the state of all the above servers and routers. Configuring a device to be monitored is as simple as adding an IP address to either the Asset Management system, or to the system itself. The system automatically takes care of assessing new devices and calculating their position and function within the customers Enterprise.

A polling engine built into the the system contacts all systems on a regular basis. The polling gathers vital health signs (configuration, memory, CPU, disk and interface utilization) and collates the information into a series of online graphs and tables.

Reports are generated monthly,weekly or daily on a programmable basis. These are used to measure success of operation against the required SLA as defined in the Asset Management system. Real time summaries are also available via the web front end.

• £2,500,000 per year saved in network downtime. 
• Full administration for multiple users (Administrator, Network Manager, or View Only) 
• Complements and integrates with existing customer systems, such as, Spectrum, Concorde Network Health, which all work at Layer 1 and Layer 2, where as Reithmon works at Layer 3.
• Low bandwidth network usage.


Case 2 - Global Network Redesign

A large and worldwide cable broadcaster who produce high quality documentary and educational programs for a wide audience. They work closely with a large number of other broadcasters and depend on their systems and networks to provide close support to both the office environment and also their broadcast chain.

Project Requirements
The complexity of networking within their local and global networks has grown over the last few years as an increasing number of business partners have required a connection into the their network. All of these links have required extra firewall functionality and some of these have been implemented with low cost application level gateways. This growing level of network and systems complexity needed to be consolidated into a world class design, reducing single point failure risk and simplifying support across multiple time zones. The requirement was to address the above issues and where possible to simplify the overall design without compromising security or reliability, increasing robustness where possible. 

The Solution
Layer3 used CustardWare to document and to aid understanding of the current configuration of all European inter-network devices and brought this together into a single document to summarise operation. The configuration was analysed to identify the need for and types of security present in the overall design. This was then translated into a generic design that could be modified to provide a solution for all European centres. A design was then developed for each of the European centres and extended to cover requirements for the rest of the world. The design once implemented was checked again using CustardWare to show that it was correctly configured and operating reliably.

• £350,000 saving in network links, support and maintenance.
• Great simplification and standardisation of design
• Cost saving design can be fully managed globally 24x7
• Scalable and consistent approach
• Fully secured and can be made highly available according to local requirement

Case 3 - Global Mobile Phone Operator

Our customer is one of the world's largest mobile telephone operators. Much of their radio network is managed, configured and maintained by the company. A number of critical software packages and large databases have been developed in house. These are used to support operations such as frequency allocation to radio masts or coverage planning and radio network maintenance.

Project Requirements 
The company needed to decommission over one hundred UNIX systems that were approaching their end of life. This was potentially a complex problem as each machine could be supporting a large number of applications, some of which were still in use and mission critical, others that were only occasionally needed and a small number that could be simply abandoned. Each system had a population of users each with their own login, each system had a list of printers and print queues. Many systems shared file system space either from other systems or shared space out to other systems. Some systems had licensed software which needed to be carefully asset maintained. All users and applications needed to be migrated to a new Sun cluster, and a high degree of continuity needed to be maintained with no downtime experienced by the user.

The Solution
Using core components from CustardWre, Layer3 developed a series of scripts that could be remotely run by the admin teams on each system. This produced an analysis of the remote systems. Further scripts were then written that could look for indications of applications and find associated users, files, file shares, databases and print queues. This process generated a web site that had a per system breakdown of results and a summary page. Finally procedures and automated processes were designed to carry out an automated migration of the applications, users and their facilities onto the a large Sun Cluster.


• Consistent systematic approach.
• Solving migration issues for a single application could be applied across all systems.

• Saving £10,000-£35,000 per system
• Decommission process was automatically and fully documented.
• Reduced manual searching and physical visits with automated network technique.
• Provided accurate measurement of progress at any given point.
• Decommission process was carried out on time and under budget. (Saved £15,000/day on project overrun)
• Systematic decommission guaranteed no expensive losses.
• Systematic license recovery - £850,000 in Unix application and development licenses.