Skip to content

OSDOCS-10877: Virtualization in Network Observability #83105

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 17, 2024
Merged

OSDOCS-10877: Virtualization in Network Observability #83105

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions _topic_maps/_topic_map.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2922,6 +2922,8 @@ Topics:
File: network-observability-operator-monitoring
- Name: Scheduling resources
File: network-observability-scheduling-resources
- Name: Secondary networks
File: network-observability-secondary-networks
- Name: Network Observability CLI
Dir: netobserv_cli
Topics:
Expand Down
4 changes: 2 additions & 2 deletions modules/network-observability-SRIOV-configuration.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// Module included in the following assemblies:
//
// * network_observability/configuring-operator.adoc
// * observability/network_observability/network-observability-secondary-networks.adoc

:_mod-docs-content-type: PROCEDURE
[id="network-observability-SR-IOV-config_{context}"]
Expand All @@ -16,7 +16,7 @@ In order to collect traffic from a cluster with a Single Root I/O Virtualization
. Under the *Provided APIs* heading for the *NetObserv Operator*, select *Flow Collector*.
. Select *cluster* and then select the *YAML* tab.
. Configure the `FlowCollector` custom resource. A sample configuration is as follows:

+
.Configure `FlowCollector` for SR-IOV monitoring
[source,yaml]
----
Expand Down
87 changes: 87 additions & 0 deletions modules/network-observability-virtualization-configuration.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
// Module included in the following assemblies:
//
// * observability/network_observability/network-observability-secondary-networks.adoc

:_mod-docs-content-type: PROCEDURE
[id="network-observability-virtualization-config_{context}"]
= Configuring virtual machine (VM) secondary network interfaces for Network Observability
You can observe network traffic on an OpenShift Virtualization setup by identifying eBPF-enriched network flows coming from VMs that are connected to secondary networks, such as through OVN-Kubernetes. Network flows coming from VMs that are connected to the default internal pod network are automatically captured by Network Observability.

.Procedure
. Get information about the virtual machine launcher pod by running the following command. This information is used in Step 5:
+
[source,terminal]
----
$ oc get pod virt-launcher-<vm_name>-<suffix> -n <namespace> -o yaml
----
+
[source,yaml]
----
apiVersion: v1
kind: Pod
metadata:
annotations:
k8s.v1.cni.cncf.io/network-status: |-
[{
"name": "ovn-kubernetes",
"interface": "eth0",
"ips": [
"10.129.2.39"
],
"mac": "0a:58:0a:81:02:27",
"default": true,
"dns": {}
},
{
"name": "my-vms/l2-network", <1>
"interface": "podc0f69e19ba2", <2>
"ips": [ <3>
"10.10.10.15"
],
"mac": "02:fb:f8:00:00:12", <4>
"dns": {}
}]
name: virt-launcher-fedora-aqua-fowl-13-zr2x9
namespace: my-vms
spec:
# ...
status:
# ...
----
<1> The name of the secondary network.
<2> The network interface name of the secondary network.
<3> The list of IPs used by the secondary network.
<4> The MAC address used for secondary network.

. In the web console, navigate to *Operators* -> *Installed Operators*.
. Under the *Provided APIs* heading for the *NetObserv Operator*, select *Flow Collector*.
. Select *cluster* and then select the *YAML* tab.
. Configure `FlowCollector` based on the information you found from the additional network investigation:
+
[source,yaml]
----
apiVersion: flows.netobserv.io/v1beta2
kind: FlowCollector
metadata:
name: cluster
spec:
# ...
ebpf:
privileged: true <1>
processor:
advanced:
secondaryNetworks:
- index: <2>
- MAC <3>
name: my-vms/l2-network <4>
# ...
----
<.> Ensure that the eBPF agent is in `privileged` mode so that flows are collected for secondary interfaces.
<.> Define the fields to use for indexing the virtual machine launcher pods. It is recommended to use the `MAC` address as the indexing field to get network flows enrichment for secondary interfaces. If you have overlapping MAC address between pods, then additional indexing fields, such as `IP` and `Interface`, could be added to have accurate enrichment.
<.> If your additional network information has a MAC address, add `MAC` to the field list.
<.> Specify the name of the network found in the `k8s.v1.cni.cncf.io/network-status` annotation. Usually <namespace>/<network_attachement_definition_name>.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even though you have omitted index by Interface here, but if users see this example and use Interface as indexing field they will not get enrichment since when the flows are captured it matches interface name partially just for the hex portion of it between what's published in annotation and what ebpf sees. Therefore it's important we only mention MAC index as an example and add a note about IP and Interface.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it and addressed as part of your other comment on this. I was initially thinking I could try to make it clear in the callouts that they would need to be careful to omit any of the index fields that weren't relevant to their secondary interface, but I see your point about the risk if they copy our sample and use it without actually reading all the callouts.

. Observe VM traffic:
.. Navigate to the *Network Traffic* page.
.. Filter by *Source* IP using your virtual machine IP found in `k8s.v1.cni.cncf.io/network-status` annotation.
.. View both *Source* and *Destination* fields, which should be enriched, and identify the VM launcher pods and the VM instance as owners.
5 changes: 0 additions & 5 deletions observability/network_observability/configuring-operator.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,5 @@ For more information about specifying flow format, see xref:../../observability/

include::modules/network-observability-configuring-FLP-sampling.adoc[leveloffset=+1]
include::modules/network-observability-configuring-quickfilters-flowcollector.adoc[leveloffset=+1]
include::modules/network-observability-SRIOV-configuration.adoc[leveloffset=+1]
[role="_additional-resources"]
.Additional resources
For more information about creating the `SriovNetwork` custom resource, see xref:../../networking/hardware_networks/configuring-sriov-device.adoc#cnf-creating-an-additional-sriov-network-with-vrf-plug-in_configuring-sriov-device[Creating an additional SR-IOV network attachment with the CNI VRF plugin].

include::modules/network-observability-resource-recommendations.adoc[leveloffset=+1]
include::modules/network-observability-resources-table.adoc[leveloffset=+2]
2 changes: 1 addition & 1 deletion observability/network_observability/installing-operators.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ Once you create the `FlowCollector` instance, you can reconfigure it, but the po

* xref:../../observability/network_observability/configuring-operator.adoc#network-observability-flowcollector-kafka-config_network_observability[Configuring the Flow Collector resource with Kafka]
* xref:../../observability/network_observability/configuring-operator.adoc#network-observability-enriched-flows_network_observability[Export enriched network flow data to Kafka or IPFIX]
* xref:../../observability/network_observability/configuring-operator.adoc#network-observability-SR-IOV-config_network_observability[Configuring monitoring for SR-IOV interface traffic]
* xref:../../observability/network_observability/network-observability-secondary-networks.adoc#network-observability-SR-IOV-config_network-observability-secondary-networks[Configuring monitoring for SR-IOV interface traffic]
* xref:../../observability/network_observability/observing-network-traffic.adoc#network-observability-working-with-conversations_nw-observe-network-traffic[Working with conversation tracking]
* xref:../../observability/network_observability/observing-network-traffic.adoc#network-observability-dns-tracking_nw-observe-network-traffic[Working with DNS tracking]
* xref:../../observability/network_observability/observing-network-traffic.adoc#network-observability-packet-drops_nw-observe-network-traffic[Working with packet drops]
Expand Down
4 changes: 2 additions & 2 deletions ...ability/network_observability/network-observability-operator-release-notes.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -249,7 +249,7 @@ For more information, see xref:../../observability/network_observability/observi

[id="SR-IOV-configuration-1.4"]
==== SR-IOV support
You can now collect traffic from a cluster with Single Root I/O Virtualization (SR-IOV) device. For more information, see xref:../../observability/network_observability/configuring-operator.adoc#network-observability-SR-IOV-config_network_observability[Configuring the monitoring of SR-IOV interface traffic].
You can now collect traffic from a cluster with Single Root I/O Virtualization (SR-IOV) device. For more information, see xref:../../observability/network_observability/network-observability-secondary-networks.adoc#network-observability-SR-IOV-config_network-observability-secondary-networks[Configuring the monitoring of SR-IOV interface traffic].

[id="IPFIX-support-1.4"]
==== IPFIX exporter support
Expand All @@ -266,7 +266,7 @@ Network Observability Operator can now run on `s390x` architecture. Previously i
=== Bug fixes
* Previously, the Prometheus metrics exported by Network Observability were computed out of potentially duplicated network flows. In the related dashboards, from *Observe* -> *Dashboards*, this could result in potentially doubled rates. Note that dashboards from the *Network Traffic* view were not affected. Now, network flows are filtered to eliminate duplicates before metrics calculation, which results in correct traffic rates displayed in the dashboards. (link:https://issues.redhat.com/browse/NETOBSERV-1131[*NETOBSERV-1131*])

* Previously, the Network Observability Operator agents were not able to capture traffic on network interfaces when configured with Multus or SR-IOV, non-default network namespaces. Now, all available network namespaces are recognized and used for capturing flows, allowing capturing traffic for SR-IOV. There are xref:../../observability/network_observability/configuring-operator.adoc#network-observability-SR-IOV-config_network_observability[configurations needed] for the `FlowCollector` and `SRIOVnetwork` custom resource to collect traffic.
* Previously, the Network Observability Operator agents were not able to capture traffic on network interfaces when configured with Multus or SR-IOV, non-default network namespaces. Now, all available network namespaces are recognized and used for capturing flows, allowing capturing traffic for SR-IOV. There are xref:../../observability/network_observability/network-observability-secondary-networks.adoc#network-observability-SR-IOV-config_network-observability-secondary-networks[configurations needed] for the `FlowCollector` and `SRIOVnetwork` custom resource to collect traffic.
(link:https://issues.redhat.com/browse/NETOBSERV-1283[*NETOBSERV-1283*])

* Previously, in the Network Observability Operator details from *Operators* -> *Installed Operators*, the `FlowCollector` *Status* field might have reported incorrect information about the state of the deployment. The status field now shows the proper conditions with improved messages. The history of events is kept, ordered by event date. (link:https://issues.redhat.com/browse/NETOBSERV-1224[*NETOBSERV-1224*])
Expand Down
24 changes: 24 additions & 0 deletions observability/network_observability/network-observability-secondary-networks.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
:_mod-docs-content-type: ASSEMBLY
[id="network-observability-secondary-networks"]
= Secondary networks
include::_attributes/common-attributes.adoc[]
:context: network-observability-secondary-networks

toc::[]
You can configure the Network Observability Operator to collect and enrich network flow data from secondary networks, such as SR-IOV and OVN-Kubernetes.

// Note to tech review:
// Is the existing SR-IOV example we have, "Configuring monitoring for SR-IOV interface traffic", an example of secondary network? If so, it is not through a VM, right?

[discrete]
[id="network-observability-secondary-network-prerequisites_{context}"]
== Prerequisites
* Access to an {product-title} cluster with an additional network interface, such as a secondary interface or an L2 network.

include::modules/network-observability-SRIOV-configuration.adoc[leveloffset=+1]

[role="_additional-resources"]
.Additional resources
*xref:../../networking/hardware_networks/configuring-sriov-device.adoc#cnf-creating-an-additional-sriov-network-with-vrf-plug-in_configuring-sriov-device[Creating an additional SR-IOV network attachment with the CNI VRF plugin].

include::modules/network-observability-virtualization-configuration.adoc[leveloffset=+1]