> For the complete documentation index, see [llms.txt](https://docs.vectra.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.vectra.ai/deployment/traffic-engineering-and-validation/traffic-visibility-drop-alerting.md). # Traffic visibility drop alerting Configure health checks and alerts for drops in traffic visibility, including IP counts, bandwidth, and packet counts. ## Overview As of the v9.11 release, Brain appliances monitor the below metrics for abnormality: * **The number of observed IP addresses** - Aggregate for entire deployment * **The observed traffic bandwidth (per Sensor**) * **The observed packet counts (per Sensor)** For each of these metrics, the system will collect data, determine a baseline, and create a system health alert when it is determined that any of these metrics have crossed a critical threshold. The exact cause will not be known to the system but it is likely a situation where an external network event (e.g. switch reconfiguration) has caused a reduction in network observability. Vectra expects a high correlation between losses in network observability and the system health alerts related to traffic visibility drops, and advises customers receiving these notifications that they investigate their network traffic monitoring to determine what event(s) may have caused a loss in visibility. Alerts can be received by email, webhook, retrieved via API call, or received via Syslog (QUX deployments only). ## Alerting Configuration Traffic visibility drop alerts are a sub category of system health alerts and if you are receiving system health alerts by any supported method, you will receive these traffic visibility drop alerts. An alert is created when a metric becomes critical and when the metric returns to normal. There are also alerts related to the quality of the traffic being ingested. For detail on those alerts, please see [Traffic validation (ENTV) alerting](/deployment/traffic-engineering-and-validation/traffic-validation-entv-alerting.md). ### System Health Alert Notification Methods **Email notification:** Please see [System alerts](/configuration/response/notifications/system-alerts.md) for details. **Webhook notification:** Please see [External app alerts (webhook)](/configuration/response/notifications/external-app-alerts-webhook.md) for details. {% hint style="info" %} **Please Note:** For Webhook notifications, an alert will be sent when a metric becomes critical, but there will be no alert sent when the metric returns to normal. This is expected behavior and is not cause for alarm. {% endhint %} **API:** * Perform a `GET` against the `/events/health` endpoint * Example URL: [https://VECTRA\_PORTAL\_URL/api/v3.4/events/health/](https://vectra_portal_url/api/v3.4/events/health/) * RUX API details please see [https://apidocs.vectra.ai](https://apidocs.vectra.ai/) * Example URL: `https://VECTRA_PORTAL_URL/api/v3.4/events/health` * QUX API details please see [v2.5 API guide (QUX)](/configuration/access/api-qux/v25-api-guide-qux.md) * Example URL: `http://your_QUX_URL_or_IP//api/v2.5/events/health` **Syslog** (for QUX deployments only): Please see the [Syslog Guide (QUX)](/configuration/response/notifications/syslog-guide-qux.md) ## Metric Thresholds There are no specific thresholds shared because the system calculates what is normal for each system automatically. In general, the system monitors traffic levels over 3-6 weeks and builds a baseline from that. It then periodically compares the previous 8 hours to the baseline. {% hint style="info" %} **Please Note:** Your system will not create any alerts until the system has confidence in a generated baseline. Vectra may change the thresholds and parameters discussed above in future updates. {% endhint %} Customers with extremely noisy traffic graphs (wide fluctuations vs predictable patterns) or very low bandwidth may create alerts when not desired or fail to create alerts when desired. If this happens in your deployment, please open a support ticket. There are some underlying parameters that Vectra can alter to attempt to optimize for your deployment that the system may not be able to calculate automatically. ## Example Alert Messages Below are some example email alert messages for a system crossing the critical threshold for observed IPs and another message for when the observed IP count returns to normal (a "recovery" alert). {% hint style="info" %} **Please Note:** Recovery alerts are sent via most channels (see [alerting configuration](#alerting-configuration) above for details) when the traffic level was abnormal but is now normal again. This can occurr when: * The observed metric is fluctuating and transitioning in and out of the calculated critical threshold. * The traffic level has stabilized, and the system has determined a new baseline. {% endhint %} Alerts for other visibility drops (bandwidth, packet counts) would look similar. Alerts through other channels such as Webhook, API, or syslog (QUX only) would also contain similar language.
## Additional Details When you have received an alert, you may wish to gather additional information or check at the CLI what current status the checks are in. Please see [SSH login process for CLI](/deployment/appliance-operations/ssh-login-process-for-cli.md) for details on how to login to the Brain CLI. `show system-health --verbose` The above command can be used to see the Sensor Bandwidth. Alternatively you can perform a `GET` against the `/events/health` endpoint using the API. Example: ``` Client : Sensor Traffic Sensor Traffic exceeds critical threshold for Potential traffic anomaly detected. Please investigate recent traffic pattern changes in your environment. ``` This alert is an indicator of change and does not by itself confirm an issue. The system can also sends a recovery notification. The sentence `Potential traffic anomaly cleared` is the indicator that the alert is the recovery or clearing of the previous alert. ``` Client : Sensor Traffic Sensor Traffic is normal for xx sensor(s); unable to determine status for , , ..., and xx more Potential traffic anomaly cleared. ``` The recovery notification can also contain U`nable to determine status for xxxx` when there are unknown values. This usually means that there isn't enough traffic data to generate a baseline. This can happen if the traffic levels are very low, or if the sensor is newly deployed. `Unable to determine status` should not be considered a problem unless those sensors have been receiving good quantities for traffic for a long time (over a month). --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.vectra.ai/deployment/traffic-engineering-and-validation/traffic-visibility-drop-alerting.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.