LogZilla

Overview

VMware vSphere is a virtualization and cloud computing platform that enables organizations to create, run, and manage virtual machines (VMs) and cloud-based services. It provides a complete virtualization infrastructure, including virtualized computing, networking, storage, and security resources.

vSphere enables multiple operating systems and applications to run on a single physical server or cluster of servers, allowing organizations to consolidate IT infrastructure and reduce hardware costs. Key features include High Availability (HA), Distributed Resource Scheduler (DRS), and Fault Tolerance (FT) for increased reliability and availability of virtualized applications.

App Function

The VMware vSphere app includes customized rules, dashboards, and triggers designed specifically for vSphere environments.

Performance Configuration Required:

The app can extract 123 possible user tags from vSphere log messages. While all tags are enabled by default, it is strongly recommended to enable only the tags of interest and disable the remainder, as each tag has a performance cost.

Configuration Steps:

Edit /etc/logzilla/apps/vmware/config/vmware-config.yaml
In the DESIRED_USER_TAGS section, list only the tags you need
Remove unused tag entries to optimize performance
Refer to USER_TAG_DEFINITIONS section for tag descriptions

Vendor Documentation

Incoming Log Format

VMware vSphere uses standard syslog format with two distinct message structures used simultaneously:

Format 1:

Date-timestamp
Severity level
Process name and ID
Key-value pairs
Message text

Format 2:

Date-timestamp
Thread ID
Key-value pairs
Event ID
Message text

The VMware app can parse both formats to extract user tag information.

Parsed Metadata Fields

These are the fields / user tags that are parsed from the log messages:

vSphere User Tag Name	vSphere User Tag Name
`vmw_config_type`	`vmw_device_changed`
`vmw_esx_shell_command`	`vmw_esx_shell_user`
`vmw_esxi_audit`	`vmw_esxi_auth_failed_source`
`vmw_esxi_auth_failed_user`	`vmw_esxi_auth_source`
`vmw_esxi_auth_type`	`vmw_esxi_auth_user`
`vmw_esxi_cli_command`	`vmw_esxi_connect_source`
`vmw_esxi_connectivity_component`	`vmw_esxi_drs_from`
`vmw_esxi_drs_to`	`vmw_esxi_drs_vm`
`vmw_esxi_esxupdate_command`	`vmw_esxi_firewall_operation`
`vmw_esxi_firewall_ruleset`	`vmw_esxi_hostd_auth_user`
`vmw_esxi_iscsi_server`	`vmw_esxi_nfs_datastore`
`vmw_esxi_nfs_server`	`vmw_esxi_nfs_status`
`vmw_esxi_nsx_severity`	`vmw_esxi_permission_event`
`vmw_esxi_portgroup`	`vmw_esxi_problem`
`vmw_esxi_problem_datastores`	`vmw_esxi_scsi_additional_sense_code`
`vmw_esxi_scsi_additional_sense_code_qualifier`	`vmw_esxi_scsi_device_status`
`vmw_esxi_scsi_host_status`	`vmw_esxi_scsi_latency`
`vmw_esxi_scsi_plugin_status`	`vmw_esxi_scsi_sense_code`
`vmw_esxi_scsi_sense_data`	`vmw_esxi_severity`
`vmw_esxi_snapshot_operation`	`vmw_esxi_snmp_trap_name`
`vmw_esxi_snmp_trap_oid`	`vmw_esxi_sub`
`vmw_esxi_uptime`	`vmw_esxi_vim_datastore`
`vmw_esxi_vmdowntime`	`vmw_esxi_vmfs_heartbeat_datastore`
`vmw_esxi_vmfs_volume_guid`	`vmw_esxi_vmk_component`
`vmw_esxi_vmk_world`	`vmw_esxi_vmkernel_net_vm_name`
`vmw_esxi_vmotion_bandwidth`	`vmw_esxi_vmotion_opid`
`vmw_esxi_vmotion_type`	`vmw_esxi_vmotiondst_opid`
`vmw_esxi_vmprecopybandwidth`	`vmw_esxi_vmprecopystuntime`
`vmw_esxi_vms`	`vmw_fdm_state`
`vmw_ha_component`	`vmw_ha_component_operation`
`vmw_ha_guesthb`	`vmw_ha_slave`
`vmw_hatask`	`vmw_hostd_vmotion_id`
`vmw_opid`	`vmw_recordop`
`vmw_recordop_action`	`vmw_rsv_source`
`vmw_rsv_time`	`vmw_scsi_path_state`
`vmw_scsideviceio_pid`	`vmw_task_status`
`vmw_user`	`vmw_vc_alarm_source`
`vmw_vc_alarm_status`	`vmw_vc_alarm_type`
`vmw_vc_api_invocations`	`vmw_vc_auth_failed_source`
`vmw_vc_auth_failed_user`	`vmw_vc_auth_source`
`vmw_vc_auth_type`	`vmw_vc_auth_user`
`vmw_vc_custom_field_name`	`vmw_vc_custom_field_on_vm`
`vmw_vc_drs_migrate_cluster`	`vmw_vc_drs_migrate_datastore`
`vmw_vc_duplicate_ip_vm1`	`vmw_vc_duplicate_ip_vm2`
`vmw_vc_fdm_state`	`vmw_vc_file_action`
`vmw_vc_mks_host`	`vmw_vc_msg_info`
`vmw_vc_msg_vm`	`vmw_vc_power_host`
`vmw_vc_power_status`	`vmw_vc_power_vm_name`
`vmw_vc_reconfig_on`	`vmw_vc_rhttpproxy_error`
`vmw_vc_task_method`	`vmw_vc_task_object`
`vmw_vc_task_operation`	`vmw_vc_task_status`
`vmw_vc_task_type`	`vmw_vc_vmodl_fault`
`vmw_vc_vmotion_from`	`vmw_vc_vmotion_precopystuntime`
`vmw_vc_vmotion_to`	`vmw_vc_vpxd_clientip`
`vmw_vc_vpxd_hearbeat_host`	`vmw_vc_vpxd_username`
`vmw_vim_fault_type`	`vmw_vm_heartbeat_source`
`vmw_vm_heartbeat_status`	`vmw_vm_state_transition_post`
`vmw_vm_state_transition_pre`	`vmw_vm_vmx_name`
`vmw_vmfs_heartbeat_status`	`vmw_vmkernel_vmotion_id`
`vmw_vmotion_status`	`vmw_vob_component`
`vmw_vob_event_type`	`vmw_volume_name`
`vmw_vsphere_op_time`

Log Examples

Log Format 1 - Process-based

text
- - 2022-10-07T01:31:23.561Z info vpxd[06166] [Originator@6876 sub=Default opID=sps-Main-670825-661-106283-91] [VpxLRO] -- ERROR session[52209c62-c72c-38b9-47de-a9cde9fc032f]521ac939-f711-0fa9-41fc-62fad40b3af9 -- CatalogSyncManager -- vim.vslm.vcenter.CatalogSyncManager.queryCatalogChange: vmodl.fault.NotSupported

Log Format 2 - Thread-based

text
- - 2022-10-07T01:31:23.868Z [pool-24-thread-1] INFO  opId=sps-Main-670825-661 com.vmware.vim.storage.common.util.OperationIdUtil - OperationID present in invoker thread, adding suffix and re-using it sps-Main-670825-661-106283