In the first post of this series, I explained how to install and configure Splunk. In this second post, I will show you how to ship ESXi logs to a Splunk server.
If a ESXi server has a local storage, logs are automatically saved in /var/log. If however a ESXi server is installed on a SD or USB media, there is no local storage to be used, and logs are saved into the ram disk; thus these logs are not retained during a reboot process. Usually, you end up with this warning:
In both cases, the configuration to send logs to Splunk will be the same.
First, into the Splunk server. Splunk can receive logs arriving on every TCP or UDP port you want, but initially there is no configured “receiver”. In order to enable a syslog you need to go in Manager -> Data inputs -> UDP -> Add new, and configure it as follows.
You need to configure the port you want to use, usually UDP:514, and you choose “syslog” as log format; also, even if is not mandatory, I prefer to convert hosts record from IP to DNS so they are more readable in the logs.
Once you saved the new receiver, let’s move into the ESXi configuration. Go into Advanced System Settings, search for parameter Syslog.global.logHost and set a value like udp://splunk_ip:514:
Once the configuration is saved, the warning message disappears, and ESXi logs are shipped to Splunk.
Finally, you can configure the same parameter also on servers with local storage, so all the ESXi logs are saved inside the same log platform.