OSDOCS-10877: Virtualization in Network Observability #83105

skrthomas · 2024-10-04T20:50:34Z

Version(s):

Merge only to no-1.7. On GA 10/21/24, Sara will open a branch to integrate no-1.7 with main, and that branch will go to 4.12+.
Issue:

https://issues.redhat.com/browse/OSDOCS-10877
Link to docs preview:

https://83105--ocpdocs-pr.netlify.app/openshift-enterprise/latest/observability/network_observability/network-observability-secondary-networks
QE review:

QE has approved this change.

Additional information:

openshift-ci-robot · 2024-10-04T20:50:38Z

@skrthomas: This pull request references OSDOCS-10877 which is a valid jira issue.

In response to this:

Version(s):

Issue:

Link to docs preview:

QE review:

QE has approved this change.

Additional information:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

ocpdocs-previewbot · 2024-10-04T20:55:19Z

🤖 Wed Oct 16 14:58:06 - Prow CI generated the docs preview:

https://83105--ocpdocs-pr.netlify.app/openshift-enterprise/latest/observability/network_observability/configuring-operator.html
https://83105--ocpdocs-pr.netlify.app/openshift-enterprise/latest/observability/network_observability/installing-operators.html
https://83105--ocpdocs-pr.netlify.app/openshift-enterprise/latest/observability/network_observability/network-observability-operator-release-notes.html
https://83105--ocpdocs-pr.netlify.app/openshift-enterprise/latest/observability/network_observability/network-observability-secondary-networks.html

openshift-ci-robot · 2024-10-07T17:32:29Z

@skrthomas: This pull request references OSDOCS-10877 which is a valid jira issue.

In response to this:

Version(s):

Issue:

https://issues.redhat.com/browse/OSDOCS-10877
Link to docs preview:

https://83105--ocpdocs-pr.netlify.app/openshift-enterprise/latest/observability/network_observability/configuring-operator.html#network-observability-virtualization-config_network_observability
QE review:

QE has approved this change.

Additional information:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

skrthomas · 2024-10-07T17:33:37Z

@memodi @jotak can you PTAL at this PR? I left some comments for you both inline // NOTE for tech review

jpinsonneau · 2024-10-08T07:28:06Z

modules/network-observability-virtualization-configuration.adoc

+= Configuring virtual machine (VM) secondary network interfaces for Network Observability
+You can observe networking patterns on an OpenShift Virtualization setup by identifying eBPF-enriched network flows coming from VMs that are connected to additional network interfaces.


Should we add a mention that default pod network are covered out of the box ?

+1 we should

@jpinsonneau do you mean that if a user has this configured, the flows from this default pod network are automatically captured by Network Observability? Or that the default pod networks are automatically configured out of the box? I assumed the former but just want to make sure.

yes the flows are automatically captured by netobserv

jpinsonneau · 2024-10-08T07:31:27Z

modules/network-observability-virtualization-configuration.adoc

+= Configuring virtual machine (VM) secondary network interfaces for Network Observability
+You can observe networking patterns on an OpenShift Virtualization setup by identifying eBPF-enriched network flows coming from VMs that are connected to additional network interfaces.
+// NOTE to tech review: 
+// do we need to mention anything about CNI here? I noted this in our convo, but I'm not sure if its relevant in the context of this procedure. Maybe its a prerequisite?  


The virt docs explicitly mention OVN-Kubernetes secondary network so I would suggest to only refer to that in our doc too as it's what we tested 😸

Okay, so I changed the intro sentence to:
You can observe networking patterns on an OpenShift Virtualization setup by identifying eBPF-enriched network flows coming from VMs that are connected to an Open Virtual Network (OVN)-Kubernetes secondary network.

jpinsonneau · 2024-10-08T07:48:48Z

modules/network-observability-virtualization-configuration.adoc

+
+.Prerequisites
+* Access to an {product-title} cluster with an additional network interface, such as a secondary interface or an L2 network.
+* Any other prerequisites?


As same as SRIOV case, we need to have spec.agent.ebpf.privileged field to true

cc @msherif1234

I have this listed as a FlowCollector config: https://github.com/openshift/openshift-docs/pull/83105/files#diff-2e10b29803e092e329b07fcd472f336c98d04f3eed531135e2afb892b7b22db0R111. I could also call it out in the prerequisites though.

yes, that's right, we should probably have dedicated menu item for "secondary interfaces" and list common pre-requisites there.

Ok I added this. I feel that the intro text for "secondary interfaces" with Network Observability is lacking, and I'm wondering about potentially mixing up SR-IOV VMs and then SR-IOV interfaces. Are those two separate things? I left a //comment to you guys there trying to parse my confusion 🥴 Just trying to better understand the bigger Network Observability/secondary networks picture

jpinsonneau · 2024-10-08T08:10:39Z

modules/network-observability-virtualization-configuration.adoc

+. Get information about the virtual machine by running the following command. This information is used in Step 2: 
+
+[source,terminal]
+----
+oc get network-attachment-definitions.k8s.cni.cncf.io/netdevice67619-1 -o yaml
+----
+// NOTE to tech review: 
+// does any of this need to be anonymized for the customer? Or made to a <user_replaceable_value>? and is it necessary to use a different oc get command for different interfaces? Like I know the one above is from Mehul's SR-IOV example. But then in the QE test, its `$ oc get pod/virt-launcher-test-vm1-bsfb4 -o jsonpath='{.metadata.annotations.k8s\.v1\.cni\.cncf\.io/network-status}' | jq`. I can split this up and put both, but just not sure and want to verify. 
+. Investigate each additional network interface to find relevant pod annotations. These annotations differ depending on the kind of network interface you have: 
+.. For SR-IOV, look for the annotated fields:
+
+[source,yaml]
+----
+apiVersion: k8s.cni.cncf.io/v1
+kind: NetworkAttachmentDefinition
+metadata:
+  annotations:
+    k8s.v1.cni.cncf.io/resourceName: openshift.io/netpolicy67619
+  creationTimestamp: "2024-10-03T20:40:48Z"
+  generation: 1
+  name: netdevice67619-1
+  namespace: openshift-sriov-network-operator
+  resourceVersion: "159492"
+  uid: 13131f06-e7b1-4b42-8d95-ac7c53e908cb
+spec:
+  config: |-
+    {
+        "cniVersion": "1.0.0",
+        "name": "netdevice67619-1", <1>
+        "type": "sriov",
+        "vlan": 0,
+        "vlanQoS": 0,
+        "logLevel": "info",
+        "ipam": {
+            "type": "static",
+            "addresses": [
+                {
+                    "address": "192.168.122.71/24" <2>
+                }
+            ]
+        }
+----
+<1> Name of the virtual virtual machine connected to the additional network interface.
+<2> The MAC address to identify the virtual machine.
+
+.. For L2, look for the annotated fields:
+
+[source,yaml]
+----
+ # ...
+  {
+    "name": "ovn-kubernetes",
+    "interface": "eth0",
+    "ips": [
+      "10.129.2.39"
+    ],
+    "mac": "0a:58:0a:81:02:27",
+    "default": true,
+    "dns": {}
+  },
+  {
+    "name": "my-vms/l2-network",   <1>
+    "interface": "podc0f69e19ba2",
+    "mac": "02:fb:f8:00:00:12",    <2>
+    "dns": {}
+  }
+ # ... 
+----
+<1> The name of the virtual machine connected to the additional network interface.
+<2> The MAC address to identify the virtual machine.
+
+. Configure `FlowCollector` based on the information you found from the additional network investigation. 


This whole section is not needed to me.

You can directly configure the secondary network discovery without checking at these.

However, we can recommend to check for k8s.v1.cni.cncf.io/network-status annotation in launcher pods to let the user understand how it work:

Suggested change

. Get information about the virtual machine by running the following command. This information is used in Step 2:

+

[source,terminal]

----

oc get network-attachment-definitions.k8s.cni.cncf.io/netdevice67619-1 -o yaml

----

// NOTE to tech review:

// does any of this need to be anonymized for the customer? Or made to a <user_replaceable_value>? and is it necessary to use a different oc get command for different interfaces? Like I know the one above is from Mehul's SR-IOV example. But then in the QE test, its `$ oc get pod/virt-launcher-test-vm1-bsfb4 -o jsonpath='{.metadata.annotations.k8s\.v1\.cni\.cncf\.io/network-status}' | jq`. I can split this up and put both, but just not sure and want to verify.

. Investigate each additional network interface to find relevant pod annotations. These annotations differ depending on the kind of network interface you have:

.. For SR-IOV, look for the annotated fields:

+

[source,yaml]

----

apiVersion: k8s.cni.cncf.io/v1

kind: NetworkAttachmentDefinition

metadata:

annotations:

k8s.v1.cni.cncf.io/resourceName: openshift.io/netpolicy67619

creationTimestamp: "2024-10-03T20:40:48Z"

generation: 1

name: netdevice67619-1

namespace: openshift-sriov-network-operator

resourceVersion: "159492"

uid: 13131f06-e7b1-4b42-8d95-ac7c53e908cb

spec:

config: |-

{

"cniVersion": "1.0.0",

"name": "netdevice67619-1", <1>

"type": "sriov",

"vlan": 0,

"vlanQoS": 0,

"logLevel": "info",

"ipam": {

"type": "static",

"addresses": [

{

"address": "192.168.122.71/24" <2>

}

]

}

----

<1> Name of the virtual virtual machine connected to the additional network interface.

<2> The MAC address to identify the virtual machine.

.. For L2, look for the annotated fields:

+

[source,yaml]

----

# ...

{

"name": "ovn-kubernetes",

"interface": "eth0",

"ips": [

"10.129.2.39"

],

"mac": "0a:58:0a:81:02:27",

"default": true,

"dns": {}

},

{

"name": "my-vms/l2-network", <1>

"interface": "podc0f69e19ba2",

"mac": "02:fb:f8:00:00:12", <2>

"dns": {}

}

# ...

----

<1> The name of the virtual machine connected to the additional network interface.

<2> The MAC address to identify the virtual machine.

. Configure `FlowCollector` based on the information you found from the additional network investigation.

. Get information about the virtual machine launcher pod by running the following command. This information is used in Step 2:

+

[source,terminal]

----

$ oc get pod virt-launcher-<vm-name>-<random-suffix> -n <namespace> -o yaml

----

+

[source,yaml]

----

$ oc get pod virt-launcher-fedora-aqua-fowl-13-zr2x9 -n my-vms -o yaml

apiVersion: v1

kind: Pod

metadata:

annotations:

k8s.v1.cni.cncf.io/network-status: |-

[{

"name": "ovn-kubernetes",

"interface": "eth0",

"ips": [

"10.129.2.39"

],

"mac": "0a:58:0a:81:02:27",

"default": true,

"dns": {}

},

{

"name": "my-vms/l2-network", <1>

"interface": "podc0f69e19ba2", <2>

"ips": [

"10.10.10.15"

], <3>

"mac": "02:fb:f8:00:00:12", <4>

"dns": {}

}]

name: virt-launcher-fedora-aqua-fowl-13-zr2x9

namespace: my-vms

spec:

...

status:

...

----

<1> The name of the secondary network.

<2> The network interface name of the secondary network.

<3> The list of IPs used by the secondary network.

<4> The MAC address used for secondary network.

. Configure `FlowCollector` based on the information you found from the additional network investigation.

You can directly configure the secondary network discovery without checking at these.

just configuring name of secondary network wouldn't be enough, right? I think some sort of indexing would definitely be needed for enrichment, no? cc @jpinsonneau

Yes @memodi, you need to specify the fields that define the index. However, you don't really care about the values here.

Either we let the user check the k8s.v1.cni.cncf.io/network-status and figure out by themself which fields are reliable or we directly tell them for each type of secondary network which fields to use as index.

jpinsonneau · 2024-10-08T08:18:43Z

modules/network-observability-virtualization-configuration.adoc

+[source,yaml]
+----
+apiVersion: flows.netobserv.io/v1beta2
+kind: FlowCollector
+metadata:
+  name: cluster
+spec:
+# ...
+ebpf:
+    privileged: true       <1>
+  processor:
+    advanced:
+      secondaryNetworks:
+      - index:
+        - MAC "02:fb:f8:00:00:12"  <2>
+        - IP  "192.168.122.71/24"  <3>
+        name: my-vms/l2-network    <4>
+# ...
+----
+<1> Ensure that the ebpf agent is in `privileged` mode so that the flows are enriched according to the MAC address.
+<2> If your additional network information has a MAC address, specify add it here.
+<3> If your additional network information has an IP address, specify add it here.
+<4> Specify the name of the virtual machine. 
+
+. Ping from one VM to another for secondary interface IP


Suggested change

[source,yaml]

----

apiVersion: flows.netobserv.io/v1beta2

kind: FlowCollector

metadata:

name: cluster

spec:

# ...

ebpf:

privileged: true <1>

processor:

advanced:

secondaryNetworks:

- index:

- MAC "02:fb:f8:00:00:12" <2>

- IP "192.168.122.71/24" <3>

name: my-vms/l2-network <4>

# ...

----

<1> Ensure that the ebpf agent is in `privileged` mode so that the flows are enriched according to the MAC address.

<2> If your additional network information has a MAC address, specify add it here.

<3> If your additional network information has an IP address, specify add it here.

<4> Specify the name of the virtual machine.

. Ping from one VM to another for secondary interface IP

[source,yaml]

----

apiVersion: flows.netobserv.io/v1beta2

kind: FlowCollector

metadata:

name: cluster

spec:

# ...

ebpf:

privileged: true <1>

processor:

advanced:

secondaryNetworks:

- index: <2>

- Interface

- MAC

- IP

name: my-vms/l2-network <3>

# ...

----

<1> Ensure that the ebpf agent is in `privileged` mode so that the flows are enriched according to the MAC address.

<2> Define the fields to use for indexing the virtual machine launcher pods. These should form a unique identifier across the cluster according to the fields available in `k8s.v1.cni.cncf.io/network-status` annotation.

<3> Specify the name of the network found in `k8s.v1.cni.cncf.io/network-status` annotation. Usually <namespace>/<network-attachement-definition-name>.

Discussed this with @skrthomas on slack - that we should keep general recommendation such as indexing by MAC only. For other fields we could simply add a note saying, indexing is also supported for IP and Interface fields in combination with MAC if they’re published in annotations.

jpinsonneau · 2024-10-08T08:29:23Z

modules/network-observability-virtualization-configuration.adoc

+.Verification
+. Navigate to Netflow traffic page, Filter by *Source* IP the 10.10.10.15.
+. View both *Source* and *Destination* fields should enriched identifying VM Pods and VM as Owners


Suggested change

.Verification

. Navigate to Netflow traffic page, Filter by *Source* IP the 10.10.10.15.

. View both *Source* and *Destination* fields should enriched identifying VM Pods and VM as Owners

.Verification

. Navigate to Netflow traffic page, Filter by *Source* IP using your virtual machine IP found in `k8s.v1.cni.cncf.io/network-status` annotation.

. View both *Source* and *Destination* fields should enriched identifying VM launcher Pods and VM instance as Owners

memodi

@skrthomas I think we could have separate menu item for "Secondary Networks" under Network Observability here and list pre-requisites and enrichment configuration there.

Examples of annotations could be added there. Between SRIOV and VM's secondary interface there's nothing specific that we do to capture their traffic or enrichment, so we could simply generalize them and remove "sriov" text from them to make it sound that it's not sriov specific.

memodi · 2024-10-08T13:22:58Z

modules/network-observability-virtualization-configuration.adoc

+= Configuring virtual machine (VM) secondary network interfaces for Network Observability
+You can observe networking patterns on an OpenShift Virtualization setup by identifying eBPF-enriched network flows coming from VMs that are connected to additional network interfaces.


+1 we should

memodi · 2024-10-08T18:16:42Z

modules/network-observability-virtualization-configuration.adoc

+= Configuring virtual machine (VM) secondary network interfaces for Network Observability
+You can observe networking patterns on an OpenShift Virtualization setup by identifying eBPF-enriched network flows coming from VMs that are connected to additional network interfaces.
+// NOTE to tech review: 
+// do we need to mention anything about CNI here? I noted this in our convo, but I'm not sure if its relevant in the context of this procedure. Maybe its a prerequisite?  


memodi · 2024-10-08T18:17:07Z

modules/network-observability-virtualization-configuration.adoc

+
+.Prerequisites
+* Access to an {product-title} cluster with an additional network interface, such as a secondary interface or an L2 network.
+* Any other prerequisites?


yes, that's right, we should probably have dedicated menu item for "secondary interfaces" and list common pre-requisites there.

memodi · 2024-10-08T18:28:25Z

modules/network-observability-virtualization-configuration.adoc

+. Get information about the virtual machine by running the following command. This information is used in Step 2: 
+
+[source,terminal]
+----
+oc get network-attachment-definitions.k8s.cni.cncf.io/netdevice67619-1 -o yaml
+----
+// NOTE to tech review: 
+// does any of this need to be anonymized for the customer? Or made to a <user_replaceable_value>? and is it necessary to use a different oc get command for different interfaces? Like I know the one above is from Mehul's SR-IOV example. But then in the QE test, its `$ oc get pod/virt-launcher-test-vm1-bsfb4 -o jsonpath='{.metadata.annotations.k8s\.v1\.cni\.cncf\.io/network-status}' | jq`. I can split this up and put both, but just not sure and want to verify. 
+. Investigate each additional network interface to find relevant pod annotations. These annotations differ depending on the kind of network interface you have: 
+.. For SR-IOV, look for the annotated fields:
+
+[source,yaml]
+----
+apiVersion: k8s.cni.cncf.io/v1
+kind: NetworkAttachmentDefinition
+metadata:
+  annotations:
+    k8s.v1.cni.cncf.io/resourceName: openshift.io/netpolicy67619
+  creationTimestamp: "2024-10-03T20:40:48Z"
+  generation: 1
+  name: netdevice67619-1
+  namespace: openshift-sriov-network-operator
+  resourceVersion: "159492"
+  uid: 13131f06-e7b1-4b42-8d95-ac7c53e908cb
+spec:
+  config: |-
+    {
+        "cniVersion": "1.0.0",
+        "name": "netdevice67619-1", <1>
+        "type": "sriov",
+        "vlan": 0,
+        "vlanQoS": 0,
+        "logLevel": "info",
+        "ipam": {
+            "type": "static",
+            "addresses": [
+                {
+                    "address": "192.168.122.71/24" <2>
+                }
+            ]
+        }
+----
+<1> Name of the virtual virtual machine connected to the additional network interface.
+<2> The MAC address to identify the virtual machine.
+
+.. For L2, look for the annotated fields:
+
+[source,yaml]
+----
+ # ...
+  {
+    "name": "ovn-kubernetes",
+    "interface": "eth0",
+    "ips": [
+      "10.129.2.39"
+    ],
+    "mac": "0a:58:0a:81:02:27",
+    "default": true,
+    "dns": {}
+  },
+  {
+    "name": "my-vms/l2-network",   <1>
+    "interface": "podc0f69e19ba2",
+    "mac": "02:fb:f8:00:00:12",    <2>
+    "dns": {}
+  }
+ # ... 
+----
+<1> The name of the virtual machine connected to the additional network interface.
+<2> The MAC address to identify the virtual machine.
+
+. Configure `FlowCollector` based on the information you found from the additional network investigation. 


You can directly configure the secondary network discovery without checking at these.

just configuring name of secondary network wouldn't be enough, right? I think some sort of indexing would definitely be needed for enrichment, no? cc @jpinsonneau

memodi · 2024-10-08T18:29:26Z

modules/network-observability-virtualization-configuration.adoc

+[source,yaml]
+----
+apiVersion: flows.netobserv.io/v1beta2
+kind: FlowCollector
+metadata:
+  name: cluster
+spec:
+# ...
+ebpf:
+    privileged: true       <1>
+  processor:
+    advanced:
+      secondaryNetworks:
+      - index:
+        - MAC "02:fb:f8:00:00:12"  <2>
+        - IP  "192.168.122.71/24"  <3>
+        name: my-vms/l2-network    <4>
+# ...
+----
+<1> Ensure that the ebpf agent is in `privileged` mode so that the flows are enriched according to the MAC address.
+<2> If your additional network information has a MAC address, specify add it here.
+<3> If your additional network information has an IP address, specify add it here.
+<4> Specify the name of the virtual machine. 
+
+. Ping from one VM to another for secondary interface IP


Discussed this with @skrthomas on slack - that we should keep general recommendation such as indexing by MAC only. For other fields we could simply add a note saying, indexing is also supported for IP and Interface fields in combination with MAC if they’re published in annotations.

openshift-ci-robot · 2024-10-08T21:38:39Z

@skrthomas: This pull request references OSDOCS-10877 which is a valid jira issue.

In response to this:

Version(s):

Issue:

https://issues.redhat.com/browse/OSDOCS-10877
Link to docs preview:

https://83105--ocpdocs-pr.netlify.app/openshift-enterprise/latest/observability/network_observability/network-observability-secondary-networks
QE review:

QE has approved this change.

Additional information:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

modules/network-observability-virtualization-configuration.adoc

memodi · 2024-10-09T18:47:31Z

modules/network-observability-virtualization-configuration.adoc

+:_mod-docs-content-type: PROCEDURE
+[id="network-observability-virtualization-config_{context}"]
+= Configuring virtual machine (VM) secondary network interfaces for Network Observability
+You can observe networking patterns on an OpenShift Virtualization setup by identifying eBPF-enriched network flows coming from VMs that are connected to secondary networks, such as through Open Virtual Network (OVN)-Kubernetes, or SR-IOV CNI plugins. Network flows coming from VMs that are connected to the default internal pod network are automatically captured by Network Observability.


Suggested change

You can observe networking patterns on an OpenShift Virtualization setup by identifying eBPF-enriched network flows coming from VMs that are connected to secondary networks, such as through Open Virtual Network (OVN)-Kubernetes, or SR-IOV CNI plugins. Network flows coming from VMs that are connected to the default internal pod network are automatically captured by Network Observability.

You can observe network traffic on an OpenShift Virtualization setup by identifying enriched network flows coming from VMs that are connected to secondary networks, such as through Open Virtual Network (OVN)-Kubernetes, or SR-IOV CNI plugins. Network flows coming from VMs that are connected to the default internal pod network are automatically captured by Network Observability.

do we want to mention here about ebpf must be in privileged mode to be inline with what you have SRIOV? I see you have already mentioned as pre-requisite.

Suggested change

You can observe networking patterns on an OpenShift Virtualization setup by identifying eBPF-enriched network flows coming from VMs that are connected to secondary networks, such as through Open Virtual Network (OVN)-Kubernetes, or SR-IOV CNI plugins. Network flows coming from VMs that are connected to the default internal pod network are automatically captured by Network Observability.

You can observe networking patterns on an OpenShift Virtualization setup by identifying eBPF-enriched network flows coming from VMs that are connected to secondary networks. Network flows coming from VMs that are connected to the default internal pod network are automatically captured by Network Observability.

I would remove the mention of SRIOV here as it's not the goal in VMs

do we want to mention here about ebpf must be in privileged mode to be inline with what you have SRIOV? I see you have already mentioned as pre-requisite.

Since its a prerequisite and also a code callout, i think mentioning it in two places is good enough.

memodi · 2024-10-09T18:51:14Z

modules/network-observability-virtualization-configuration.adoc

+    advanced:
+      secondaryNetworks:
+      - index:                    <2>
+        - Interface


In example, I'd suggest only show MAC index and add IP and Interface as field as simply note.

+1, it will be more obvious for the user

Okay I added IP and Interface as part of the callout <2> text and removed them from the example yaml + corresponding callouts.

memodi · 2024-10-09T18:52:10Z

modules/network-observability-virtualization-configuration.adoc

+        name: my-vms/l2-network   <5> 
+# ...
+----
+<.> Ensure that the ebpf agent is in `privileged` mode so that the flows are enriched according to the MAC address.


Suggested change

<.> Ensure that the ebpf agent is in `privileged` mode so that the flows are enriched according to the MAC address.

<.> Ensure that the ebpf agent is in `privileged` mode so that the flows so that flows are collected for secondary interfaces

memodi · 2024-10-09T18:56:50Z

modules/network-observability-virtualization-configuration.adoc

+# ...
+----
+<.> Ensure that the ebpf agent is in `privileged` mode so that the flows are enriched according to the MAC address.
+<.> Define the fields to use for indexing the virtual machine launcher pods. These should form a unique identifier across the cluster according to the fields available in `k8s.v1.cni.cncf.io/network-status` annotation. Not all secondary networks have all the index fields in this example. Ensure that only the fields that are annotated in your secondary interface are listed here and if they are not annotated, they are removed.


I don't understand existing sentence. I'd suggest something like below instead

Suggested change

<.> Define the fields to use for indexing the virtual machine launcher pods. These should form a unique identifier across the cluster according to the fields available in `k8s.v1.cni.cncf.io/network-status` annotation. Not all secondary networks have all the index fields in this example. Ensure that only the fields that are annotated in your secondary interface are listed here and if they are not annotated, they are removed.

<.> Define the fields to use for indexing the virtual machine launcher pods. It is recommended to use MAC address as indexing field to get network flows enrichment for secondary interfaces. If you have overlapping MAC address between pods then additional indexing fields such as IP and Interface could be added to have accurate enrichment.

Addressed, thanks for the suggestion.

memodi · 2024-10-09T19:02:06Z

modules/network-observability-virtualization-configuration.adoc

+<.> If your additional network information has a MAC address, specify it here.
+<.> If your additional network information has an IP address, specify it here.
+<.> Specify the name of the network found in `k8s.v1.cni.cncf.io/network-status` annotation. Usually <namespace>/<network_attachement_definition_name>. 
+


Even though you have omitted index by Interface here, but if users see this example and use Interface as indexing field they will not get enrichment since when the flows are captured it matches interface name partially just for the hex portion of it between what's published in annotation and what ebpf sees. Therefore it's important we only mention MAC index as an example and add a note about IP and Interface.

Got it and addressed as part of your other comment on this. I was initially thinking I could try to make it clear in the callouts that they would need to be careful to omit any of the index fields that weren't relevant to their secondary interface, but I see your point about the risk if they copy our sample and use it without actually reading all the callouts.

memodi · 2024-10-09T19:28:42Z

modules/network-observability-virtualization-configuration.adoc

+<.> If your additional network information has an IP address, specify it here.
+<.> Specify the name of the network found in `k8s.v1.cni.cncf.io/network-status` annotation. Usually <namespace>/<network_attachement_definition_name>. 
+
+. Ping from one VM to another for secondary interface IP.


I think this is not required for users to do. We do that for testing.

Got it, will remove.

memodi · 2024-10-09T19:36:26Z

modules/network-observability-virtualization-configuration.adoc

+.Verification
+. Navigate to the *Network Traffic* page.
+. Filter by *Source* IP using your virtual machine IP found in `k8s.v1.cni.cncf.io/network-status` annotation.
+. View both *Source* and *Destination* fields should enriched identifying VM launcher Pods and VM instance as Owners


Instead of "verification" could we say something like "Observe VM Traffic" and after Navigate to the Network Traffic page bullet:

filter by Source/Destination Owner name == VM name to view traffic from/to those VMs

This could even be a new section, more generic, giving the summary of any scenario (SRIOV & VMs)

Yeah I can make it a step with substeps. For the sake of time, I'll just do that instead of making a new section for both bc that would technically need to be a new module

jpinsonneau · 2024-10-10T08:21:55Z

modules/network-observability-virtualization-configuration.adoc

+<.> If your additional network information has a MAC address, specify it here.
+<.> If your additional network information has an IP address, specify it here.


Suggested change

<.> If your additional network information has a MAC address, specify it here.

<.> If your additional network information has an IP address, specify it here.

<.> If your additional network information has a MAC address, add `MAC` to the field list.

<.> If your additional network information has an IP address, add `IP` to the field list.

The specify it here mentions are confusing to me. I feel it requires to fill the values found in the pod annotation instead of the field names.

Makes sense. Added your suggestion

stleerh · 2024-10-11T00:35:47Z

observability/network_observability/network-observability-secondary-networks.adoc

+
+.Prerequisites
+* Access to an {product-title} cluster with an additional network interface, such as a secondary interface or an L2 network.
+* The `spec.agent.ebpf.privileged` field must be set to `true`.


This shouldn't be considered a prerequisite since it's providing the instructions to do this in the two scenarios below.

openshift-ci-robot · 2024-10-14T18:42:34Z

@skrthomas: This pull request references OSDOCS-10877 which is a valid jira issue.

In response to this:

Version(s):

Merge only to no-1.7. On GA 10/21/24, Sara will open a branch to integrate no-1.7 with main, and that branch will go to 4.12+.
Issue:

https://issues.redhat.com/browse/OSDOCS-10877
Link to docs preview:

https://83105--ocpdocs-pr.netlify.app/openshift-enterprise/latest/observability/network_observability/network-observability-secondary-networks
QE review:

QE has approved this change.

Additional information:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

bergerhoffer · 2024-10-15T14:57:27Z

/label peer-review-in-progress

bergerhoffer

Nice job, just a few things!

observability/network_observability/network-observability-secondary-networks.adoc

modules/network-observability-virtualization-configuration.adoc

bergerhoffer · 2024-10-15T16:14:00Z

/label peer-review-done
/remove-label peer-review-needed
/remove-label peer-review-in-progress

memodi

/label qe-approved

openshift-ci-robot · 2024-10-15T20:32:12Z

@skrthomas: This pull request references OSDOCS-10877 which is a valid jira issue.

In response to this:

Version(s):

Merge only to no-1.7. On GA 10/21/24, Sara will open a branch to integrate no-1.7 with main, and that branch will go to 4.12+.
Issue:

https://issues.redhat.com/browse/OSDOCS-10877
Link to docs preview:

https://83105--ocpdocs-pr.netlify.app/openshift-enterprise/latest/observability/network_observability/network-observability-secondary-networks
QE review:

QE has approved this change.

Additional information:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci · 2024-10-16T15:00:15Z

@skrthomas: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Oct 4, 2024

openshift-ci bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Oct 4, 2024

skrthomas force-pushed the OSDOCS-10877 branch 5 times, most recently from e0d2704 to 4d8f1a2 Compare October 7, 2024 15:44

skrthomas requested review from jotak and memodi October 7, 2024 17:32

skrthomas requested a review from jpinsonneau October 7, 2024 17:37

jpinsonneau reviewed Oct 8, 2024

View reviewed changes

memodi reviewed Oct 8, 2024

View reviewed changes

skrthomas force-pushed the OSDOCS-10877 branch 2 times, most recently from 4dbaa8c to 0dfcb55 Compare October 8, 2024 21:37

skrthomas force-pushed the OSDOCS-10877 branch 4 times, most recently from 8868895 to a1293cb Compare October 9, 2024 01:58

ocpdocs-vale-bot reviewed Oct 9, 2024

View reviewed changes

modules/network-observability-virtualization-configuration.adoc Outdated Show resolved Hide resolved

skrthomas force-pushed the OSDOCS-10877 branch from a1293cb to 9e961f4 Compare October 9, 2024 02:25

skrthomas force-pushed the OSDOCS-10877 branch from 6a414b3 to a0d2180 Compare October 9, 2024 18:00

memodi suggested changes Oct 9, 2024

View reviewed changes

memodi reviewed Oct 9, 2024

View reviewed changes

jpinsonneau reviewed Oct 10, 2024

View reviewed changes

skrthomas force-pushed the OSDOCS-10877 branch 2 times, most recently from aa3b3f9 to 4360107 Compare October 10, 2024 18:51

stleerh reviewed Oct 11, 2024

View reviewed changes

skrthomas force-pushed the OSDOCS-10877 branch from 4360107 to 9efabd6 Compare October 14, 2024 13:53

jpinsonneau approved these changes Oct 14, 2024

View reviewed changes

skrthomas added the peer-review-needed Signifies that the peer review team needs to review this PR label Oct 14, 2024

openshift-ci bot added the peer-review-in-progress Signifies that the peer review team is reviewing this PR label Oct 15, 2024

bergerhoffer reviewed Oct 15, 2024

View reviewed changes

openshift-ci bot added peer-review-done Signifies that the peer review team has reviewed this PR and removed peer-review-needed Signifies that the peer review team needs to review this PR peer-review-in-progress Signifies that the peer review team is reviewing this PR labels Oct 15, 2024

skrthomas force-pushed the OSDOCS-10877 branch from 9efabd6 to 6c640cc Compare October 15, 2024 20:25

memodi approved these changes Oct 15, 2024

View reviewed changes

openshift-ci bot added the qe-approved Signifies that QE has signed off on this PR label Oct 15, 2024

OSDOCS-10877: Virtualization in Network Observability

7c8b771

skrthomas force-pushed the OSDOCS-10877 branch from 6c640cc to 7c8b771 Compare October 16, 2024 14:42

skrthomas added the merge-review-needed Signifies that the merge review team needs to review this PR label Oct 17, 2024

jab-rh merged commit e20e0aa into openshift:no-1.7 Oct 17, 2024
2 checks passed

jab-rh removed the merge-review-needed Signifies that the merge review team needs to review this PR label Oct 17, 2024

skrthomas added this to the Continuous Release milestone Oct 21, 2024

		= Configuring virtual machine (VM) secondary network interfaces for Network Observability
		You can observe networking patterns on an OpenShift Virtualization setup by identifying eBPF-enriched network flows coming from VMs that are connected to additional network interfaces.

	You can observe networking patterns on an OpenShift Virtualization setup by identifying eBPF-enriched network flows coming from VMs that are connected to secondary networks, such as through Open Virtual Network (OVN)-Kubernetes, or SR-IOV CNI plugins. Network flows coming from VMs that are connected to the default internal pod network are automatically captured by Network Observability.
	You can observe network traffic on an OpenShift Virtualization setup by identifying enriched network flows coming from VMs that are connected to secondary networks, such as through Open Virtual Network (OVN)-Kubernetes, or SR-IOV CNI plugins. Network flows coming from VMs that are connected to the default internal pod network are automatically captured by Network Observability.

	<.> Ensure that the ebpf agent is in `privileged` mode so that the flows are enriched according to the MAC address.
	<.> Ensure that the ebpf agent is in `privileged` mode so that the flows so that flows are collected for secondary interfaces

		<.> If your additional network information has a MAC address, specify it here.
		<.> If your additional network information has an IP address, specify it here.

OSDOCS-10877: Virtualization in Network Observability #83105

OSDOCS-10877: Virtualization in Network Observability #83105

Uh oh!

Conversation

skrthomas commented Oct 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci-robot commented Oct 4, 2024 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ocpdocs-previewbot commented Oct 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci-robot commented Oct 7, 2024 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

skrthomas commented Oct 7, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

skrthomas Oct 8, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jpinsonneau Oct 8, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

memodi Oct 8, 2024 • edited by skrthomas Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jpinsonneau Oct 8, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

memodi left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

memodi Oct 8, 2024 • edited by skrthomas Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

openshift-ci-robot commented Oct 8, 2024 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

skrthomas commented Oct 4, 2024 •

edited

Loading

openshift-ci-robot commented Oct 4, 2024 •

edited by openshift-ci bot

Loading

ocpdocs-previewbot commented Oct 4, 2024 •

edited

Loading

openshift-ci-robot commented Oct 7, 2024 •

edited by openshift-ci bot

Loading

skrthomas Oct 8, 2024 •

edited

Loading

jpinsonneau Oct 8, 2024 •

edited

Loading

memodi Oct 8, 2024 •

edited by skrthomas

Loading

jpinsonneau Oct 8, 2024 •

edited

Loading

memodi Oct 8, 2024 •

edited by skrthomas

Loading

openshift-ci-robot commented Oct 8, 2024 •

edited by openshift-ci bot

Loading