vSphere Content Pack Log Insight Dashboards

Vsphere – All vSphere events

The total number of events received over time. An increase in events may point to a change in the environment. A large increase in events may be an indication of an issue in the environment.

All vSphere events by hostname

The total number of events received by hostname. A larger number of events on a subset of hosts may point to a change within the environment or an issue on the subset of hosts.

vSphere error events

The total number of error events received over time. An increase in events may point to a change in the environment. A large increase in events may be an indication of an issue in the environment.

vSphere error events by cluster

The total number of error events received over time. An increase in events may point to a change in the environment. A large increase in events may be an indication of an issue in the environment.

vSphere warning events

The total number of warning events received over time. An increase in events may point to a change in the environment. A large increase in events may be an indication of an issue in the environment.

vSphere warning events by cluster

The total number of warning events received over time. An increase in events may point to a change in the environment. A large increase in events may be an indication of an issue in the environment.

Vsphere General – Problems

ESXi events by hostname

All events from all ESXi hosts.

vSphere problem events by type

Some common problems you may see include:

• coredump.unconfigured2
• scratch.partition.unconfigured
• storage.apd.start (alert available)
• storage.connectivity.devicepor (alert available)
• storage.connectivity.pathpor (alert available)
• visorfs.ramdisk.full (alert available)
• vmsyslogd.remote.failure

vSphere connectivity lost by component

These events point to connectivity issues that lead to reduced redundancy, potential performance issues, and potential outages.

For network connectivity lost, see: 

• Network Connectivity Lost Due to Physical Link Down

For storage connectivity lost, see:

• Troubleshooting LUN connectivity issues on ESXi/ESX hosts
• Lost or degraded connectivity to storage device
• Understanding the messages: Connectivity to NFS Server Lost and Connectivity to NFS Server Restored

Physical hardware event detected

• Advanced Programmable Interrupt Controller (APIC)
• Machine Check Exception (MCE)
• Non-Maskable Interrupt (NMI) 

In the case of a “Memory Controller Read Error” MCE, run memtest on the impacted hosts and replace any bad DIMMs.

For more information, see:

• vHBAs and other PCI devices may stop responding in ESXi 6.0.x, ESXi 5.x and ESXi/ESX 4.1 when using Interrupt Remapping
• Identifying and addressing Non-Maskable Interrupt events on an ESXi host
• Decoding Machine Check Exception (MCE) output after a purple screen error

Configuration problems by hostname

Note: In healthy environments this widget should return no results.
Average ESXi SCSI latency >1s over time by device Chart
Important: In healthy environments this widget should return no results.

SCSI latency is a good indicator of performance issues within an environment. Latency over one second (one million microseconds) is noticeable and unexpected in most environments. Unless you are experiencing high storage latency within your environment, this widget should return no results.

For more information, please see: Storage device performance deteriorated

ESXi NFS problem events by status

For more information, please see:

• Connectivity to NFS Server Lost and Connectivity to NFS Server Restored
• Connectivity to the NFS server is lost.

Vsphere DRS imbalance by cluster and vCenter Server

DRS has detected in imbalance above the configured threshold for a cluster. DRS was unable to correct the imbalance, which typically indicates a lack of available resources. Manual intervention is necessary to correct this issue.

Vsphere General – Performance

vCenter Server: API invocations by source and user

The number of api request per source and user. This information is helpful in determining potential performance issues.
vSphere: Slow SOAP RPC communication by vCenter Server Chart
If vCenter Server and ESXi host communication time takes longer than 2 seconds, vCenter prints a log message to indicate a potential performance problem. It may or may not indicate real performance issue. If there is a spike in such messages, you may want to investigate your network performance.

vCenter Server: Slow SQL commands by table

The vCenter Server SQL database can become slow for a variety of reasons. When the vCenter Server database execution time is longer than 3 seconds, vCenter prints a log message showing this information indicating a potential performance issue. Database latency may depend on your environment. It may or may not indicate real performance issue. If there is a spike in such messages,database maintenance such as truncating tables or rebuilding indexes may be necessary.

For more information, please see: Rebuilding indexes to improve the performance of SQL Server and Oracle vCenter Server databases

VMware DRS executed vMotions by ESXi host

This chart shows the count of DRS fully-automated migration operations by source host and destination host. This indicates the amount of load balancing work that DRS performed. A large number of DRS operations on a particular host may indicate a subset of VMs causing performance issues. Multiple hosts in the same cluster performing a large number of DRS operations may indicate that additional resources are needed in the cluster.
vCenter Server: Database execution time by hostname Chart
Abnormally long execution time of database events are a good indication of vCenter performance issues. To validate whether the execution time is abnormally long, be sure to use a time of the last 24 hours.

vSphere: vMotion precopy stun time

For more information, see VMware vSphere vMotion Architecture, Performance and Best Practices in VMware vSphere 5 Performance Study
ESXi: SCSI latency >1s over time by device Chart
SCSI latency is a good indicator of performance issues within an environment. Latency over one second (one million microseconds) is noticeable and unexpected in most environments. Unless you are experiencing high storage latency within your environment, this widget should return no results.

For more information, please see: Storage device performance deteriorated

vSphere: All tasks over time

Spikes during a performance issue may provide information on where to focus efforts.
ESXi: VMware VMFS reservation times by datastore Chart.

vSphere: VM tasks over time by task

Spikes of a particular task during a performance issue may provide information on where to focus efforts.

Vsphere General – Licensing

vCenter Server: Powered on events by VM Chart

IMPORTANT: This widget requires vSphere integration and more specifically the collection of vCenter Server events, task, and alarms to display any results.

Some VM licensing requirements enforce paying for all hosts on which a VM ran over a given period of time.

This widget shows all VM deploy operations as reported from vCenter Server. The widget to the right shows all VM deploy operations as reported from ESXi. You can drill-down into a particular VM by selecting it in this widget or manually adding it to the dashboard filters at the top.

ESXi: Power on events by VMX name Chart

IMPORTANT: ESXi only provides the VMX name of the VM, which may or may not be the same as the actual VM name.

Some VM licensing requirements enforce paying for all hosts on which a VM ran over a given period of time.

This widget shows all VM deploy operations as reported from ESXi. The widget to the right shows all VM deploy operations as reported from vCenter Server. You can drill-down into a particular VM by selecting it in this widget or manually adding it to the dashboard filters at the top.

vCenter Server: vMotion events by VM Chart

IMPORTANT: This widget requires vSphere integration and more specifically the collection of vCenter Server events, task, and alarms to display any results.

Some VM licensing requirements enforce paying for all hosts on which a VM ran over a given period of time.

VMs may move from host to host for a variety of reasons, but all lead to a vMotion operation. This widget shows all vMotion operations as reported from vCenter Server. The widget to the right shows all vMotion operations as reported from ESXi. You can drill-down into a particular VM by selecting it in this widget or manually adding it to the dashboard filters at the top.

ESXi: vMotion events by VM Chart

IMPORTANT: ESXi only provides the VMX name of the VM, which may or may not be the same as the actual VM name.

Some VM licensing requirements enforce paying for all hosts on which a VM ran over a given period of time.

VMs may move from host to host for a variety of reasons, but all lead to a vMotion operation. This widget shows all vMotion operations as reported from ESXi. The widget to the left shows all vMotion operations as reported from vCenter Server. You can drill-down into a particular VM by selecting it in this widget or manually adding it to the dashboard filters at the top.

Hosts Chart As you drill down to particular VMs you are concerned with licensing this widget will tell you which hosts hosted the VM over the given time range. This makes it auditing and license compliance easy.

vMotion operations by source and destination host Chart

IMPORTANT: This widget does not provide any value unless you have drilled down to the particular VMs you wish to perform auditing on.

As you drill down to particular VMs you are concerned with licensing this widget will tell you which hosts hosted the VM over the given time range due to migrations. This makes it auditing and license compliance easy.

vCenter Server: datacenters Chart

As you drill down to particular VMs you are concerned with licensing this widget will help narrow down where in your environment the VMs live.

vCenter Server: clusters Chart

IMPORTANT: This widget does not provide any value unless you have drilled down to the particular VMs you wish to perform auditing on.

As you drill down to particular VMs you are concerned with licensing this widget will help narrow down where in your environment the VMs live.

Vsphere General – Inventory

ESXi hosts logging

If ESXi hosts were configured to forward logs to Log Insight using Log Insight’s vSphere integration then this number should match the number of ESXi hosts configured under under vSphere integration. If the number does not match then ensure the hosts are configured properly by selecting the View details option under each vCenter Server instanced defined.

Note: There is no way to guarantee whether an ESXi host was configured via vSphere integration or not so if ESXi hosts were configured to log to this Log Insight instance through other means they will also appear in this total.

ESXi datastores found

Datastores for which logs were generated. All datastores can generate logs including, local, (i)SCSI, and NFS.

ESXi portgroups found

Portgroups for which logs were generated. Typically, portgroup logs are only generated if changes to portgroups are made or a VM’s networking is reconfigured.

VMs found

vSphere logs typically contain logs pertaining to specific VMs from time-to-time. This widget indicates the number of unique VM names found in vSphere logs. 

Vsphere Security – Auditing

ESXi audit events by type

ESXi audit events indicate security-related configuration changes, These changes should be audited to ensure a secure environment.
Count of Administrator roles configured Chart – The number of times an administrator role is assigned to a user.

ESXi firewall configuration changes by ruleset and operation

The most commonly changed rulesets. This information can be very important from an auditing/security perspective.

Snapshot events by VM and operation

fact are that Snapshots are not backups and should be deleted within 72 hours to prevent unnecessary disk consumption and possible performance implications. If more creates then removes are seen then this may mean that snapshots are not being cleaned up. To accurately compare creates to removed be sure to set the time range for at least 72 hours.

The following operations may be observed: consolidate disks – delta VMDK needs to be combined and removed to save disk space.

ESXi service enabled events by hostname and service

Services within the ESXi Host need to be monitor to apply best Security Practice, this Widget provide you info on the number of host that have this service Enable.

Remote console events by host and user

IMPORTANT: This widget requires vSphere integration and more specifically the collection of vCenter Server events, task, and alarms to display any results.

ESXi shell commands FieldTable

Any shell (i.e. CLI) commands run on ESXi are logged. This widget returns the most recent run commands by user.

Vsphere Security – Authentication

Authentication requests over time by vCenter Server Chart

Authentication requests over time by ESXi host Chart

Failed log in attempts by source and vCenter Server Chart

Failed logins can be a good indication of a security issue within your environment. This widget makes it easy to determine which source is generating the most failed login attempts for auditing purposes.

Failed log in attempts by source and ESXi host

Failed logins can be a good indication of a security issue within your environment. This widget makes it easy to determine which source is generating the most failed login attempts for auditing purposes.

vCenter Server administrator logins

Administrator logins to vCenter with an associated alert. Ideally there should be no Administrator logins unless there is a compelling reason for the login.

ESXi administrator logins
vCenter Server logins by type

Several types of logins are possible including:

• axis = VMware vSphere Web Services SDK
• gSOAP = VMware vSphere Web Services SDK
• jax-ws = Java API for XML Web Services
• ms web services client protocol= PowerCLI
• vim-java = Java connection to vSphere API
• vi perl = Perl connection to vSphere API
• vixdisklib = Virtual Disk Development Kit (VDDK) API
• vmware client = vSphere client

Several types of logins are possible including:

• axis = VMware vSphere Web Services SDK
• gSOAP = VMware vSphere Web Services SDK
• jax-ws = Java API for XML Web Services
• ms web services client protocol = PowerCLI
• vim-java = Java connection to vSphere API
• vi perl = Perl connection to vSphere API
• vixdisklib = Virtual Disk Development Kit (VDDK) API
• vmware client = vSphere client

Leave a Reply

Your email address will not be published. Required fields are marked *