Skip to content

Conversation

alexandraBara
Copy link
Collaborator

@alexandraBara alexandraBara commented Aug 19, 2025

Reading journal logs. Sample run:

node-scraper run-plugins JournalPlugin
(venv) (base) alexbara@pp-128-a5-1:~/node-scraper$ node-scraper run-plugins JournalPlugin
  2025-08-19 14:28:53 CDT       INFO               nodescraper | Log path: ./scraper_logs_pp_128_a5_1_2025_08_19-02_28_53_PM
  2025-08-19 14:28:53 CDT       INFO               nodescraper | System Name: pp-128-a5-1
  2025-08-19 14:28:53 CDT       INFO               nodescraper | System SKU: None
  2025-08-19 14:28:53 CDT       INFO               nodescraper | System Platform: None
  2025-08-19 14:28:53 CDT       INFO               nodescraper | System location: SystemLocation.LOCAL
  2025-08-19 14:28:53 CDT       INFO               nodescraper | Initializing connection manager for InBandConnectionManager with default args
  2025-08-19 14:28:53 CDT       INFO               nodescraper | --------------------------------------------------
  2025-08-19 14:28:53 CDT       INFO               nodescraper | Running plugin JournalPlugin
  2025-08-19 14:28:53 CDT       INFO               nodescraper | Initializing connection: InBandConnectionManager
  2025-08-19 14:28:53 CDT       INFO               nodescraper | Using local shell
  2025-08-19 14:28:53 CDT       INFO               nodescraper | Checking OS family
  2025-08-19 14:28:53 CDT       INFO               nodescraper | OS Family: LINUX
  2025-08-19 14:28:53 CDT       INFO               nodescraper | Running data collector: JournalCollector
  2025-08-19 14:28:53 CDT       INFO               nodescraper | Collected journal: /var/log/journal/599d998a7309484ab6a3e6d6e4de9ea8/system@3cfd64813ae2436f8e3d68d8c4a81075-0000000000709fce-00063b46e07c5796.journal
  2025-08-19 14:28:54 CDT       INFO               nodescraper | Collected journal: /var/log/journal/599d998a7309484ab6a3e6d6e4de9ea8/system@3cfd64813ae2436f8e3d68d8c4a81075-0000000000729e29-00063b54b647ac53.journal
  2025-08-19 14:28:54 CDT       INFO               nodescraper | Collected journal: /var/log/journal/599d998a7309484ab6a3e6d6e4de9ea8/system@3cfd64813ae2436f8e3d68d8c4a81075-000000000074a021-00063b628a8dfb54.journal
  2025-08-19 14:28:55 CDT       INFO               nodescraper | Collected journal: /var/log/journal/599d998a7309484ab6a3e6d6e4de9ea8/system@3cfd64813ae2436f8e3d68d8c4a81075-000000000076a406-00063b70a3965fe0.journal
  2025-08-19 14:29:09 CDT       INFO               nodescraper | Collected journal: /var/log/journal/599d998a7309484ab6a3e6d6e4de9ea8/system.journal
  2025-08-19 14:29:09 CDT       INFO               nodescraper | (JournalPlugin) journalctl logs collected
  2025-08-19 14:29:09 CDT       INFO               nodescraper | Closing connections
  2025-08-19 14:29:09 CDT       INFO               nodescraper | Running result collators
  2025-08-19 14:29:09 CDT       INFO               nodescraper | Running TableSummary result collator
  2025-08-19 14:29:09 CDT       INFO               nodescraper |

+-------------------------+--------+-----------------------------+
|  Connection              | Status | Message                     |
+-------------------------+--------+-----------------------------+
|  InBandConnectionManager | UNSET  | task completed successfully |
+-------------------------+--------+-----------------------------+

+---------------+--------+-------------------------------------+
|  Plugin        | Status | Message                             |
+---------------+--------+-------------------------------------+
|  JournalPlugin | OK     | Plugin tasks completed successfully |
+---------------+--------+-------------------------------------+

  2025-08-19 14:29:09 CDT       INFO               nodescraper | Data written to csv file: ./scraper_logs_pp_128_a5_1_2025_08_19-02_28_53_PM/nodescraper.csv

Comment on lines 42 to 51
def _shell_quote(self, s: str) -> str:
"""single-quote fix.

Args:
s (str): path

Returns:
str: escaped path
"""
return "'" + s.replace("'", "'\"'\"'") + "'"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be moved into utils since it is now being used in multiple places

Comment on lines 96 to 106
list_res = self._run_sut_cmd(self.CMD, sudo=True)
paths = [p.strip() for p in (list_res.stdout or "").splitlines() if p.strip()]

if not paths:
self._log_event(
category=EventCategory.OS,
description="No /var/log/journal files found (including rotations).",
data={"list_exit_code": list_res.exit_code},
priority=EventPriority.WARNING,
)
return []
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should not be necessary for journalctl logs. The journalctl command will automatically merge and display available logs, including rotated ones. We do not want to read log files individually and should instead rely on journalctls management of these files.

str|None: name of local journal log filed, or None if log was not read
"""
qp = self._shell_quote(path)
cmd = f"journalctl --no-pager --system --all --file={qp} --output=json"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should not need to read from a file. Please also add '-o short-iso' for iso timestamps.

@alexandraBara alexandraBara changed the base branch from development to alex_syslog September 5, 2025 14:20
Base automatically changed from alex_syslog to development September 5, 2025 15:34
@alexandraBara alexandraBara merged commit c53a68d into development Sep 13, 2025
5 checks passed
@alexandraBara alexandraBara deleted the alex_journal_log branch September 13, 2025 19:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants