Data Center Audit Strategy By and For System Admins

Why Stress? This is How to Properly Audit Your Data Center

The Time Has Come

It’s time to audit your data center. Perhaps it’s time for a data center project. You may be decommissioning, retiring legacy equipment, migrating to the cloud, co-locating, relocating to a different space, or performing a core network migration. Maybe someone in a suit needs to evaluate hardware and software assets to project future cost-effectiveness. Regardless, properly auditing and documenting the data center is a very crucial task. 

We Have All Been There

As administrators, we have all been there: most veteran sysadmins have been laid off a time or two. Data center auditing is critical for projecting and bargaining the budget. If upper-management doesn’t see a full and clear picture of the network topology and use-cases, they can’t adequately communicate the need to maintain not only the hardware and software assets, but also the budget that helps keep the sysadmins employed. A proper data center audit shows your value and efficacy as an administrator, and ensures the company receives the best value in return for the cost of their disposable assets. 

Why It’s Important

Middle management can be a tough job. Most administrators are savvy to this, and have no interest in being sandwiched into the stress of management roles, or the related tasks of documenting and sending emails all day. However, sysadmins are held accountable for many management-related matters. 

Data center auditing is one of the most overlooked yet, undeniably, most critical functions of a systems administrator. Network documentation is the foundation of a strategic asset management plan. An accurate audit is central to the security, reliability, quality of support, cost-efficiency, and disaster recovery of a data center.

Perhaps you work with other sysadmins as part of a cross-functional team. Without a proper data center audit, it can be a hassle onboarding new team members. Even for a seasoned administrator, it’s a steep learning curve without an accurate view of the network topology. 

The Monkey Wrench

Poor data center audit documentation leads to other pains we have all experienced as sysadmins: the naming convention doesn’t make sense; the Canonical Data Model is hiding; past administrators didn’t document software; passwords are difficult to find; upper management is concerned about the exorbitant energy costs; or, most haunting of all, data security is sketchy, not up to industry standard, and leads to a breach of company or, even worse, customer data.

When a critical priority incident occurs, we tend to be the monkey in the middle handling not only the critical outage but also the disconnected communication between executives, middle-management, other team-members, and frustrated end-users. It can be a headache when the Exchange admin needs information privy to the database admin; or we need a quick retrieval of sensitive customer data; or if you’re trying to figure out which patches were applied during the last ‘Patch Tuesday’ because it needs to be rolled-back. 

The lack of a proper audit comes full circle when it’s time to perform a data center decommissioning  – a scenario in which nobody wants to be found. Much diligence is required to perform a proper audit, so here is a rundown of the different types of data center audits, what should be audited, as well as some tools that can be used to help facilitate the process. 

Security Data Center Audit

The most crucial feature of a data center is its security. Not only do companies rely on their mission-critical data to be safely contained, but one breach could easily sink a business. As stated by IBM, the average total cost of a data breach in the United States is $8.19 million, with an average of 25,575 records compromised per breach. 

That’s not the only scary statistic. According to the Thales 2019 Data Threat Report, 65% of businesses in the United States say they have been breached at any point in time – and 35% reported having experienced a breach within the past year alone.

Cybersecurity risk grows progressively, becoming more aggressive and frequent over time as the complexity of hybrid-cloud and IoT technologies grow. The most vulnerable times for data breaches are during implementations, migrations, and data center decommissions. A proper security audit will help organize and strategize security efforts, and ensure the appropriate measures are in place to avoid breaches, downtime, and other data center disasters. 

44% of respondents to the Data Threat Report said that complexity is a perceived barrier to implementing proper data security. Taking that into consideration, the foundation of a proper security plan, and a safe data center decommissioning, is identifying holes in standards-compliance by properly auditing and documenting the data center on a regular basis. 

Access Control

AKA, screening of employees, contractors, and vendors who have access to infrastructure and software. Consider biometric scanners, and multi-factor authentication methods used to grant access to users.

For many organizations and technical teams, password management can be a messy ordeal. Encrypted password managers such as LastPass and Dashlane can take much of the burden and risk out of the realm of password storage and management. Unfortunately, it’s still critical to evaluate how well employees are being trained against social engineering.

Physical Security – High levels of physical security around assets is one of the main features of a quality data center. Consistent audits of the ever-moving parts of physical security help ensure disaster prevention protocols such as fire suppression is in a consistent state of readiness. It also keeps a tight lock on the doors, and a thumb on the pulse of the most vulnerable aspect of a data center: human error. 

Video Surveillance – how many cameras, their locations, and adherence to Routine Activities Theory, and other theories that best explain crime and deviance activities.

The DMZ – The importance of documenting the Demilitarized Zone and perimeter of a network is overlooked far too often. Where does your network end and the Internet begin? Where are the virtual doors? This is a very important consideration – especially when decommissioning a data center or migrating to a hybrid cloud infrastructure. Properly auditing the IP addresses, servers, server roles, usernames and passwords in the perimeter and DMZ of your network is absolutely critical to the security of your data center before the decommissioning process.

Standards-Compliance Data Center Audit

According to weekly data breach headlines at least, data security is difficult. Thankfully, there are many data center compliance standards within the data center industry. A third-party auditor can evaluate your data center’s compliance against legal requirements for essential security certifications. Some examples of industry-recognized security compliance certifications are ISO, SSAE 18, SOC2 Type II, PCI-DSS (Payment Card Industry Data Security Standard) and – in the healthcare industry where data breaches are the most expensive on average – HIPAA (Health Insurance Portability and Accountability Act). 

Without proper auditing procedures, a data center will have a difficult time not only competing, but also keeping their data safe in a progressively hostile cybersecurity world. 

Infrastructure Audit

Data centers are complex, constantly changing, and evolving digital organisms. Cabling management, power supplies and redundant power backups, cooling systems, fire and flood management systems, server racks and clusters. 

The two most important considerations in an Infrastructure Audit are: The Network Topology Map and Asset Audit.

Network Topology Map

There are many ways to skin a cat. This is true when it comes to creating a Network Topology Map. For this critical task, there are many paid and free tools available: e.g. Visio, Gliffy, Lucidchart, Spiceworks, and PDQ Inventory.

 Asset Audit 

The main consideration for auditing the assets in a data center are the server farms. These come with an array of considerations: 

• How many server racks? 
• What is the RU (Rack Unit) position of each server? 
• The NetBIOS, IP address, and DNS configurations each server.
• Server clustering configurations
• What is the role/function of each server and server cluster?
• Age, make, and model of each server. 
• Physical specifications of each server. This includes: Hard drives, disk quota usage, RAM, processors, RAID cards, DRAC/iLo cards, NIC configurations, connectivity (e.g., fiber)
• Soft specifications of each server, including: bare-metal or virtual server, databases in use, permissions (e.g. Active Directory security groups), vendor support information, and miscellaneous information such as patch levels
• Routers, Switches, and Firewalls – Make, model, IP configuration, subnet names, and virtual configuration files of each router, switch, firewall, or load balancer.

Power Efficiency Audit

According to Forbes, astonishingly, data centers are responsible for about 3% of the world’s total energy consumption. As technology continues to evolve, this is a problem that will only continue to rise over time. When it comes to data center decommissioning, auditing energy efficiency is key for planning the forward strategy of your network. How much of it needs to be decommissioned?

A standard way to metric the energy consumption of a data center is to calculate the Power Usage Effectiveness (PUE). This is calculated for each asset by dividing the total power usage by average equipment power. This will allow benchmarking and burn-rate that can be tracked over a duration of time.

Design Audit

Finally: what is the blueprint of your data center? A design audit focuses on the facility’s birdseye-view design and applicable industry standards. What is the cost-benefit to redesigning suboptimal aspects of the layout?