Adding rsyslog topics back to 4.2 #15951

mburke5678 · 2019-07-19T18:14:23Z

https://jira.coreos.com/browse/PROD-810
https://jira.coreos.com/browse/OSDOCS-623

openshift-docs-preview-bot · 2019-07-19T18:16:45Z

The preview will be available shortly at:

https://logging-rsyslog--ocpdocs.netlify.com/

mburke5678 · 2019-07-19T20:51:28Z

modules/efk-logging-fluentd-json.adoc

@@ -41,8 +45,8 @@ If you set the `MERGE_JSON_LOG` and `CDM_UNDEFINED_TO_STRING` enviroment variabl



@richm Does this problem affect Rsyslog as it affected Fluentd?
If you set the MERGE_JSON_LOGandCDM_UNDEFINED_TO_STRINGenviroment variables totrue, you might receive an Elasticsearch *400* error.

mburke5678 · 2019-07-19T20:53:21Z

modules/efk-logging-fluentd-throttling.adoc

@@ -42,7 +42,7 @@ $ oc edit configmap/fluentd
 The format of the *_throttle-config.yaml_* key is a YAML file that contains
 project names and the desired rate at which logs are read in on each
 node. The default is 1000 lines at a time per node. For example:
-
+
 ----
 throttle-config.yaml: |
  - opensift-logging:


@richm Where in the config map is this code added? The console reported the CM is updated, but the code disappears.

Throttling is not supported (yet) by rsyslog.

mburke5678 · 2019-07-19T21:01:28Z

modules/efk-logging-fluentd-external.adoc

@@ -13,6 +13,9 @@ hosted Fluentd has processed them.
 ifdef::openshift-origin[]
 The `secure-forward` plug-in is provided with the Fluentd image as of v1.4.0.
 endif::openshift-origin[]
+ifdef::openshift-enterprise[]
+The `secure-forward` plug-in is supported by Fluentd only.
+enddef::openshift-enterprise[]



@richm Is the secure-forward plug-in for Fluentd only?

Yes. We'll have to figure out some other way for rsyslog to forward logs off of the cluster.

mburke5678 · 2019-07-19T21:02:34Z

@richm This is a WIP. However, I have a few questions if you would be so kind.

richm · 2019-07-30T16:55:20Z

logging/efk-logging-eventrouter.adoc

@@ -9,6 +9,11 @@ The Event Router communicates with the {product-title} and prints {product-title

 If Cluster Logging is deployed, you can view the {product-title} events in Kibana. 

+NOTE:
+====
+The Event Router is not supported for the Rsyslog log collector.


should add that to the list of differences between fluentd and rsyslog

richm · 2019-07-30T16:56:38Z

modules/efk-logging-fluentd-log-rotation.adoc

 [cols="3,7",options="header"]
 |===
 |Parameter
 |Description

-| `LOGGING_FILE_SIZE` | The maximum size of a single Fluentd log file in Bytes. If the size of the *fluentd.log* file exceeds this value, {product-title} renames the *fluentd.log.** files and creates a new *fluentd.log*. The default is 1024000 (1MB).
-| `LOGGING_FILE_AGE` | The number of logs that Fluentd retains before deleting. The default value is `10`.
+| `LOGGING_FILE_SIZE` | The maximum size of a single log collector file in Bytes. If the size of the *fluentd.log* file exceeds this value, {product-title} renames the log files and creates a new file. The default is 1024000 (1MB).


size of the fluentd.log file or the rsyslog.log file

mburke5678 · 2019-08-08T18:33:18Z

@QiaolingTang Is this something you can review?
I reverted the current 4.1 docs to include both Rsyslog and Fluentd.

Changed Fluentd to log collector or Fluentd or Rsyslog as appropriate.
Added examples for Rsyslog as approptiate.
Made a few other file names more generic.
Other required changes.

The might be some issues that are being followed in other PRs that apply to these files. I just want to make sure the files here look good.

richm · 2019-08-08T23:07:49Z

modules/efk-logging-fluentd-envvar.adoc

 configuration of the log collector, Fluentd or Rsyslog.

+See the link:https://github.com/openshift/origin-aggregated-logging/blob/master/fluentd/README.md[Fluentd README] in Github or the 
+link:https://github.com/openshift/origin-aggregated-logging/blob/master/rsyslog/README.md[Rsyslog README] for lists of the


openshift/origin-aggregated-logging#1722

QiaolingTang · 2019-08-09T07:47:47Z

modules/efk-logging-about-components.adoc


 * logStore - This is where the logs will be stored. The current implementation is Elasticsearch.
-* collection - This is the component that collects logs from the node, formats them, and stores them in the logStore. The current implementation is Fluentd.
+* collection - This is the component that collects logs from the node, formats them, and stores them in the logStore. The current implementation is Fluentd, default, or Rsyslog.


Seems we don't have an default option here.

@QiaolingTang Thank you. I will correct this throughout. There is no default. The user needs to specify the collector in the CR.

QiaolingTang · 2019-08-09T07:54:10Z

modules/efk-logging-configuring-image-about.adoc

-* *RSYSLOG_IMAGE* deploys Rsyslog, by default `docker.io/viaq/rsyslog:latest`. <1>
-
-<1> The image used for RSYSLOG when deployed. You can change this value using an environment variable. You cannot change this value through the Cluster Logging CR.
+<6> *RSYSLOG_IMAGE* deploys Rsyslog, by default `docker.io/viaq/rsyslog:latest`. You can change this value using an environment variable. You cannot change this value through the Cluster Logging CR.


I don't think the default value of RSYSLOG_IMAGE will be docker.io/viaq/rsyslog:latest after the rsyslog's GA. And the RSYSLOG_IMAGE is written in the CSV, it's better not to change the value. Setting environment variable won't make any change.

QiaolingTang · 2019-08-09T08:06:21Z

modules/efk-logging-fluentd-limits.adoc

+          requests:
+            cpu: 250m
+            memory: 1Gi
+----


@mburke5678 I checked the default values of rsyslog's resources, they are:

name: rsyslog resources: limits: memory: 358Mi requests: cpu: 100m memory: 358Mi name: logrotate resources: limits: memory: 358Mi requests: cpu: 100m memory: 358Mi

QiaolingTang · 2019-08-09T08:07:28Z

modules/efk-logging-fluentd-limits.adoc

@@ -39,3 +39,27 @@ spec:
            memory: 1Gi
 ----
 <1> Specify the CPU and memory limits as needed. The values shown are the default values.


Users can specify the limits and requests.

QiaolingTang · 2019-08-09T08:15:10Z

modules/efk-logging-fluentd-log-location.adoc


-Fluentd writes logs to a specified file or to the default location, `/var/log/fluentd/fluentd.log`, 
+The log collector, Fluentd or Rsyslog, writes logs to a specified file or to the default location, `/var/log/fluentd/fluentd.log`, 


For fluentd, it's /var/log/fluentd/fluentd.log , for rsyslog, it is /var/log/rsyslog/rsyslog.log

QiaolingTang · 2019-08-09T08:24:54Z

modules/efk-logging-fluentd-log-rotation.adoc

+rsyslog.log.2  1Mb
+rsyslog.log.3  1Mb
+rsyslog.log.4  1Mb
+----


The log files for rsyslog is looks like:
-rw-r--r--. 1 root root 467 Jul 17 03:40 logrotate.log
-rw-r--r--. 1 root root 1007 Jul 17 03:25 rsyslog.log
-rw-r--r--. 1 root root 10382396 Jul 17 02:49 rsyslog.log-20190717-1563331501
-rw-r--r--. 1 root root 12649621 Jul 17 03:02 rsyslog.log-20190717-1563332101
-rw-r--r--. 1 root root 11345789 Jul 17 03:11 rsyslog.log-20190717-1563332701
-rw-r--r--. 1 root root 12271294 Jul 17 03:21 rsyslog.log-20190717-1563333302
-rw-r--r--. 1 root root 23122678 Jul 17 03:40 rsyslog.log-20190717-1563333901

Thank you. I forgot that I wanted an overly-simple example without the full naming to show the rotation. I'm not sure the fluentd log file names I have are accurate for what the user will see.
Do you think this will work to show the rotation, or would using real log file names make more sense?

The following example shows logs in a cluster where the maximum log size is 1Mb and four logs should be retained. When the `log`reaches 1Mb, {product-title} deletes the current `log.4`, renames the each of the Fluentd logs in turn, and creates a new `log`. ---- log 0b log.1 1Mb log.2 1Mb log.3 1Mb log.4 1Mb ----

QiaolingTang · 2019-08-09T08:29:42Z

modules/efk-logging-fluentd-log-rotation.adoc

 [cols="3,7",options="header"]
 |===
 |Parameter
 |Description

-| `LOGGING_FILE_SIZE` | The maximum size of a single Fluentd log file in Bytes. If the size of the *fluentd.log* file exceeds this value, {product-title} renames the *fluentd.log.** files and creates a new *fluentd.log*. The default is 1024000 (1MB).
-| `LOGGING_FILE_AGE` | The number of logs that Fluentd retains before deleting. The default value is `10`.
+| `LOGGING_FILE_SIZE` | The maximum size of of the fluentd.log file or the rsyslog.log file in Bytes. If the size of the *fluentd.log* file exceeds this value, {product-title} renames the log files and creates a new file. The default is 1024000 (1MB).


The log rotation of rsyslog is different from fluentd. There has a configmap logrotate-crontab controlls the rsyslog's log rotation. You can get some information from: https://bugzilla.redhat.com/show_bug.cgi?id=1730211#c1

Yes, there is a configmap with a crontab, and one with the logrotate config, but the user doesn't need to use those. Most users will be able to set these environment variables and not need to know the implementation details. I suppose we should document those configmaps, but we should tell users that they probably don't need to touch them.

@QiaolingTang @nhosoi In order to change the log rotation parameters, does the user still edit the rsyslog daemonset or edit the logrotate configmap?

oc set env daemonset/rsyslog LOGGING_FILE_AGE=30 LOGGING_FILE_SIZE=1024000 oc set env cm/logrotate LOGGING_FILE_AGE=30 LOGGING_FILE_SIZE=1024000

This one:

oc set env daemonset/rsyslog LOGGING_FILE_AGE=30 LOGGING_FILE_SIZE=1024000

If an advanced user needs to change the cron or logrotate settings, they would do

oc edit configmap logrotate-bin or oc edit configmap logrotate-crontab

@richm Thanks. I didn't see your comment above as I didn't refresh the page.

mburke5678 · 2019-08-23T16:12:36Z

@QiaolingTang Are you satisfied with this PR?

QiaolingTang · 2019-08-26T09:12:02Z

modules/efk-logging-configuring-image-about.adoc

+KIBANA_IMAGE=registry.redhat.io/openshift4/ose-logging-kibana5:v4.2 <3>
+CURATOR_IMAGE=registry.redhat.io/openshift4/ose-logging-curator5:v4.2 <4>
+OAUTH_PROXY_IMAGE=registry.redhat.io/openshift4/ose-oauth-proxy:v4.2 <5>
+RSYSLOG_IMAGE=quay.io/openshift/origin-logging-rsyslog:latest <6>


I thought the rsyslog image would be something like registry.redhat.io/openshift4/ose-logging-rsyslog:v4.2 when the 4.2 GA.

Correct. In 4.2, rsyslog is in the redhat registry like everything else.

@QiaolingTang Thank you. RSYSLOG_IMAGE=quay.io/openshift/origin-logging-rsyslog:latest is what appears in 4.1. I must have gotten this string from 4.1. Fixed!

QiaolingTang · 2019-08-26T09:13:48Z

modules/efk-logging-fluentd-alerts.adoc


-How you view logs depends upon the `LOGGING_FILE_PATH` setting.
+How you view logs collected by the log collector, Fluentd or Rsyslog, depends upon the `LOGGING_FILE_PATH` setting.


These logs are not collected by the log collector, they are generated by the log collector.

mburke5678 · 2019-08-28T19:11:52Z

/cherrypick enterprise-4.2

openshift-cherrypick-robot · 2019-08-28T19:12:18Z

@mburke5678: new pull request created: #16419

In response to this:

/cherrypick enterprise-4.2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

mburke5678 · 2019-09-13T17:21:42Z

Moving to 4.3. Reverting changes added to 4.2

#16597

openshift-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Jul 19, 2019

openshift-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jul 19, 2019

mburke5678 commented Jul 19, 2019

View reviewed changes

mburke5678 force-pushed the logging-rsyslog branch 2 times, most recently from 5f64b6a to a1de25e Compare July 25, 2019 21:46

richm reviewed Jul 30, 2019

View reviewed changes

mburke5678 changed the title ~~Adding rsyslog topics back to 4.2~~ Rsyslog GA Aug 6, 2019

mburke5678 changed the title ~~Rsyslog GA~~ Adding rsyslog topics back to 4.2 Aug 6, 2019

richm reviewed Aug 8, 2019

View reviewed changes

QiaolingTang reviewed Aug 9, 2019

View reviewed changes

mburke5678 force-pushed the logging-rsyslog branch 2 times, most recently from ffc97de to 336f5cc Compare August 23, 2019 16:11

QiaolingTang reviewed Aug 26, 2019

View reviewed changes

Adding rsyslog topics back to 4.2

9cdeee5

mburke5678 force-pushed the logging-rsyslog branch from 57deee5 to 9cdeee5 Compare August 28, 2019 17:14

mburke5678 added the branch/enterprise-4.2 label Aug 28, 2019

mburke5678 modified the milestones: Next Release, Future Release Aug 28, 2019

mburke5678 merged commit 314185f into openshift:master Aug 28, 2019

mburke5678 deleted the logging-rsyslog branch August 28, 2019 19:11

openshift-cherrypick-robot mentioned this pull request Aug 28, 2019

[enterprise-4.2] Adding rsyslog topics back to 4.2 #16419

Merged

mburke5678 removed the branch/enterprise-4.2 label Sep 13, 2019

		@@ -41,8 +45,8 @@ If you set the `MERGE_JSON_LOG` and `CDM_UNDEFINED_TO_STRING` enviroment variabl


		Fluentd writes logs to a specified file or to the default location, `/var/log/fluentd/fluentd.log`,
		The log collector, Fluentd or Rsyslog, writes logs to a specified file or to the default location, `/var/log/fluentd/fluentd.log`,


		How you view logs depends upon the `LOGGING_FILE_PATH` setting.
		How you view logs collected by the log collector, Fluentd or Rsyslog, depends upon the `LOGGING_FILE_PATH` setting.

Adding rsyslog topics back to 4.2 #15951

Adding rsyslog topics back to 4.2 #15951

Uh oh!

Conversation

mburke5678 commented Jul 19, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-docs-preview-bot commented Jul 19, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mburke5678 Jul 19, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mburke5678 commented Jul 19, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mburke5678 commented Aug 8, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

QiaolingTang Aug 9, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mburke5678 commented Aug 23, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mburke5678 commented Aug 28, 2019

Uh oh!

openshift-cherrypick-robot commented Aug 28, 2019

Uh oh!

mburke5678 commented Sep 13, 2019

Uh oh!

mburke5678 commented Jul 19, 2019 •

edited

Loading

mburke5678 Jul 19, 2019 •

edited

Loading

mburke5678 commented Aug 8, 2019 •

edited

Loading

QiaolingTang Aug 9, 2019 •

edited

Loading