In this article, I’ll show some devices can use for ESXi installation.(Ex. Hard drive/Solid state drive, USB memory stick and etc…)
You need think some side of character for each devices. (cost, endurance, monitoring)
How to choose install device for vSphere ESXi
vSphere ESXi install Requirement
First of all, check “Storage Requirements for ESXi 6.7 Installation or Upgrade”
VMware ESXi Installation and Setup
Points from the document
- At least 1GB is minimum for installation
- Scratch partition, that is log directory for ESXi, and need at least 5.2GB and more for an installation device when administrators want to set it on same device with ESXi.
- SD cards and USB memory sticks are not used for device for scratch partition, because these devices are low endurance. to store logs. (To put log, a lot of write operation will be send on it)
- When administrators use M.2 devices as installation device, ESXi installer create VMFS on it automatically. But administrators must not use it to store run VMs. It will be root cause to break your M.2 device.
From vSphere 6.7 document, M.2 device information is written as below.
Dell EMC released BOSS(Boot Optimized Storage Solution), that is M.2 device to install operating system and you can access more detail information from following links.
vSphere ESXiのスクラッチ パーティションについて
Dell EMC | BOSSってなんだろう？～PartnerSEつぶやきブログ～
Now, we have these devices to install ESXi, let’s compare these devices and check their character, pros, cons and consideration.
- USB Memory stick
- SD Card
- Hard disk drives / Solid State Drives
- M.2 device(In Dell EMC, BOSS)
- SAN Boot
I evaluate each devices from these view.
- Deploy difficulty
- Cost
- Endurance and monitoring
USB Memory stick
- Deploy difficulty：★(Easy)
- Cost：★（Reasonable）
- Endurance and monitoring：★（Low）
An USB Memory stick is very useful device, reasonable, easy. Just insert it to USB port on servers, we can use it. There are two considerations for this device, “No monitoring process” and “Need plan additional storage space for logging“.
When I talked to my friend in Technical support, they had customer situation that is “Only reboot a server, can’t boot ESXi after the reboot due to no boot device found”
Normally, general servers have hardware failure detection/monitoring feature but it does not support USB memory stick.
This is general flow why this situation is happen.
- Administrators boot a server with ESXi installed USB memory stick.
- ESXi is loaded on memory(RAM) from the USB memory devices.
- The server starts to work in production environment
- During running servers, the USB memory stick is broken.
- ESXi is running in spite of USB memory physical failure. (Because it is on RAM)
- Administrators reboot the host
- The USB memory stick is already broken, so can’t detect it. so Boot will be failure.
From administrators view, Only just reboot the physical hosts, ESXi is broken by the operation. But this reboot is not root cause for the memory stick break.
SD card
- Deploy difficulty：★(Easy)
- Cost：★★（Low ~ Middle）
- Endurance and monitoring：★★（A little bit high）
You need SD cards slot on your servers to choose this way. In Dell EMC PowerEdge Servers case, you can buy “Internal Dual SD Card Module(a.k.a IDSDM) as option slot. (This option was released from 11th generation PowerEdge Server)
Internal Dual SD Card Module – Dell
You can order this option when you configure your server with your Dell EMC Sales or after the order) Same point with USB memory stick is, “Easy to use”, “Low Cost”, “Easy to deploy”
But can’t set “Scratch partition” on this, this is same point with USB memory sticks too.
On the other hands, IDSDM has two different point from USB memory sticks.
- Failure detection feature
- Need open chassis cover when you repair the SD.
Administrators can use integrated Dell EMC Remote Access Controller(a.k.a iDRAC) to monitor/check SD cards status as below. So you can’t miss failure like USB memory stick.
One of consideration point is, when you need change existing SD cards to new to repair, need open the server. So you may need plan unmount your server from your rack.
Hard disk drive / Solid State drive
- Deploy Difficulty：★★(Easy)
- Cost：★★★（Middle）
- Endurance and monitoring：★★★（Standard）
This is traditional way to install your operating system for your servers. But for ESXi, it needs only 1GB for vmkernel, so drive spaces on these devices is very excessive drive spaces when you choice this way. You can use RAID to protect failure scenario and can set scratch partition on it. These two points are benefit. So this is very simple design for ESXi and log.
On the other hands, two considerations on this way this is.
- Need follow this KB when you use vSAN on your environment.
Best practices when using vSAN and non-vSAN disks with the same storage controller (2129050) - Can’t use every drive slots for “Data”. (One or two drive slots will be reserved for ESXi and log)
Let’s see more detail about 1. (vSAN case)
For example, Dell EMC PowerEdge R640 is very popular server and we have multiple drive slots option as below.
Dell EMC Virtual Rack – PowerEdge R640
This is not supported scenario.
Key thing is, if you have only one storage adapter like this, you must use only one drive operation mode on it. (RAID mode or Non-RAID mode). If you use multiple mode on single controller, it may be root cause incorrect vSAN I/O Handling.
If you want to hardware RAID for your ESXi, please prepare two storage controller as below.(One is for ESXi, another is for vSAN)
And for topic 2(Slot utilization), if you use RAID1 for your ESXi, it means two physical drive slots will be use for it. In other word, you can never use two slots for vSAN.
When you remember ESXi foot print size is very small, these two types of drive are very excessive. Of course you already understand this, it’s no problem.
If customer needs simple installation plan, this is very good way. If you will use vSAN, need additional PCI Slot for storage adapter. Even one PCI Slot is very valuable on 1U server.(Because 1U servers scalability is limited than 2U and more.
M.2 device
- Deploy difficulty：★★(Easy)
- Cost：★★★（Middle）
- Endurance and monitoring：★★★（Standard）
BOSS is new device from Dell EMC PowerEdge 14th generation, that is PCI card based M.2 device. I explained some devices before, and they have some consideration points.
- When you use USB memory sticks and SD cards, can’t set log partition on it(Scratch partition)
- When you use HDD or SSD, can’t use slots for data. It will use only for ESXi and log.
M.2 device will fix these challenge.
BOSS is PCI card based storage, but it will not use physical drive slot on the chassis. On the PCI card, one or two M.2 SSD will be put. and you can configure Mirror between two M.2 SSD.(RAID1) And you can monitor health status using some solutions, iDRAC is one of it.
Dell EMC Boot Optimized Server Storage-S1 User’s Guide
Addition to say, we can set scratch partition on it. (But, our documents said remove VMFS on BOSS. This mean is “don’t run VMs on the BOSS”)
These are consideration points for BOSS.
- This solution may not be general solution, it maybe vendor locked in solution.
- This is new technology based solution, so if administrators prefer proven, high stability solution, this solution may not be fit for him or her. This solution is for administrators who loves advanced new solution on IT.
- Need use one PCI Slot
Appendix
Dell EMC Boot Optimized Server Storage-S1 User’s guide
SAN Boot
- Deploy Difficulty：★★★★(Difficulty)
- Cost：★★★★★（High）
- Endurance and monitoring：★★★★★（High）
On SAN Boot, ESXi will be loaded from SAN LUN. So user must have SAN storage and infrastructure. If you don’t have SAN, this option will not applied for your environment.
And you need check whether your SAN supports SAN Boot on the document.
For example, Dell EMC PowerVault MD 3 storage does not support SAN Boot. Link (Please check Page 11)
This is SAN Boot benefit. Boot from SAN Benefit – vSphere 6.7
Before we discussed/compared endurance, cost, hardware monitoring among devices. But SAN boot benefit is mainly centralized management.
Consideration Boot device on vSAN
When you will use vSAN, you need check amount of memory on each servers. Depends on sum of memory on a host, supported install devices will be change.
If your host has less than 512GB, you can use USB memory stick and SD cards for ESXi installation.
If your host has more than 512GB, you can use SATA DOM, HDD or SSDUSB memory stick and SD cards for ESXi installation.
Please access following URL to get more detail.
Using Boot Devices and vSAN – vSphere 6.7
Conclusion
On this article, we learned multiple devices different point for ESXi installation.
There is no “best device”, it always depends on customer.(Cost, manageability and etc..)
Most important point is, understand your device character and plan how to cover consideration point for the device in your environment.
This is my one point advice based on my support experience. Most customer usually forget set “Scratch partition” on their environment. Most case, logs are already gone after host failure.
Please don’t forget how you keep your logs. Scratch partition is not only way to keep it, you can use syslog service too.
コメント