Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

INFO level logs disappear in caclmgrd #21290

Closed
ZhaohuiS opened this issue Dec 27, 2024 · 0 comments
Closed

INFO level logs disappear in caclmgrd #21290

ZhaohuiS opened this issue Dec 27, 2024 · 0 comments
Assignees
@ZhaohuiS

Comments

@ZhaohuiS
Copy link
Contributor

ZhaohuiS commented Dec 27, 2024 •
edited
Loading

Description

Steps to reproduce the issue:

  1. Load the latest 20240510 image, only WARN and ERR log found in syslog, INFO level logs disappear for caclmgrd

No info level log found for systemctl caclmgrd service in syslog file in 202405 images, only warning log found in /var/log/syslog.

202405:
admin@str2-7050cx3-acs-12:~$ sudo systemctl status caclmgrd
● caclmgrd.service - Control Plane ACL configuration daemon
Loaded: loaded (/lib/systemd/system/caclmgrd.service; enabled; preset: enabled)
Active: active (running) since Thu 2024-12-26 01:49:19 UTC; 4h 36min ago
Main PID: 1485 (caclmgrd)
Tasks: 1 (limit: 9458)
Memory: 31.8M
CGroup: /system.slice/caclmgrd.service
└─1485 /usr/bin/python3 /usr/local/bin/caclmgrd

Dec 26 01:49:19 sonic systemd[1]: Started caclmgrd.service - Control Plane ACL configuration daemon.
Dec 26 01:49:22 str2-7050cx3-acs-12 caclmgrd[1485]: Unable to determine if ACL table 'NTP_ACL' contains IPv4 or IPv6 rules. Skipping table... ==> those are warning level logs
Dec 26 01:49:22 str2-7050cx3-acs-12 caclmgrd[1485]: Unable to determine if ACL table 'SNMP_ACL' contains IPv4 or IPv6 rules. Skipping table...
Dec 26 01:49:22 str2-7050cx3-acs-12 caclmgrd[1485]: Unable to determine if ACL table 'SSH_ONLY' contains IPv4 or IPv6 rules. Skipping table...
Dec 26 01:49:25 str2-7050cx3-acs-12 caclmgrd[1485]: Unable to determine if ACL table 'NTP_ACL' contains IPv4 or IPv6 rules. Skipping table...
Dec 26 01:49:25 str2-7050cx3-acs-12 caclmgrd[1485]: Unable to determine if ACL table 'SNMP_ACL' contains IPv4 or IPv6 rules. Skipping table...
Dec 26 01:49:25 str2-7050cx3-acs-12 caclmgrd[1485]: Unable to determine if ACL table 'SSH_ONLY' contains IPv4 or IPv6 rules. Skipping table...

202311:
admin@str3-7260cx3-acs-14:~$ sudo systemctl status caclmgrd
● caclmgrd.service - Control Plane ACL configuration daemon
Loaded: loaded (/lib/systemd/system/caclmgrd.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2024-12-26 06:14:10 UTC; 1h 40min ago
Main PID: 1310 (caclmgrd)
Tasks: 1 (limit: 9465)
Memory: 14.7M
CGroup: /system.slice/caclmgrd.service
└─1310 /usr/bin/python3 /usr/local/bin/caclmgrd

Dec 26 06:14:14 str3-7260cx3-acs-14 caclmgrd[1310]: ip6tables -A INPUT -d fc00::4c/128 -j DROP==> those are info level logs
Dec 26 06:14:14 str3-7260cx3-acs-14 caclmgrd[1310]: iptables -A INPUT -d 10.0.0.40/32 -j DROP
Dec 26 06:14:14 str3-7260cx3-acs-14 caclmgrd[1310]: ip6tables -A INPUT -d fc00::50/128 -j DROP
Dec 26 06:14:14 str3-7260cx3-acs-14 caclmgrd[1310]: iptables -A INPUT -d 10.0.0.8/32 -j DROP
Dec 26 06:14:14 str3-7260cx3-acs-14 caclmgrd[1310]: ip6tables -A INPUT -d fc00::8/128 -j DROP
Dec 26 06:14:14 str3-7260cx3-acs-14 caclmgrd[1310]: iptables -A INPUT -d 10.0.0.42/32 -j DROP
Dec 26 06:14:14 str3-7260cx3-acs-14 caclmgrd[1310]: ip6tables -A INPUT -d fc00::54/128 -j DROP
Dec 26 06:14:14 str3-7260cx3-acs-14 caclmgrd[1310]: iptables -A INPUT -m ttl --ttl-lt 2 -j ACCEPT
Dec 26 06:14:14 str3-7260cx3-acs-14 caclmgrd[1310]: ip6tables -A INPUT -p tcp -m hl --hl-lt 2 -j ACCEPT
Dec 26 06:14:15 str3-7260cx3-acs-14 caclmgrd[1310]: Issuing the following iptables commands:
admin@str3-7260cx3-acs-14:~$

Describe the results you received:

It's caused by #17171

Even it set min log priority to INFO, but INFO still disappear in syslog file.
https://github.com/sonic-net/sonic-host-services/blob/master/scripts/caclmgrd

    # Log all messages from INFO level and higher
    caclmgr.set_min_log_priority_info()

Describe the results you expected:

Need to check INFO level logs in syslog

Output of show version:

(paste your output here)

Output of show techsupport:

(paste your output here or download and attach the file here )

Additional information you deem important (e.g. issue happens only occasionally):
@ZhaohuiS ZhaohuiS mentioned this issue Dec 27, 2024
Closed
12 tasks
@ZhaohuiS ZhaohuiS self-assigned this Jan 8, 2025
@ZhaohuiS ZhaohuiS mentioned this issue Jan 9, 2025
Merged
mssonicbld added a commit to mssonicbld/sonic-host-services that referenced this issue Jan 9, 2025
@mssonicbld
Fix the issue of no info log in syslog for caclmgrd
bb326a8 
### **Issue**
Fix the issue sonic-net/sonic-buildimage#21290
No info log found in syslog on 202405 image for caclmgrd

### **Work item tracking**
- Microsoft ADO **(number only)**:
30611546

### Why did it happen
RP sonic-net/sonic-buildimage#17171, introduced a new Class SysLogger, DaemonBase will choose SysLogger by default
PR sonic-net/sonic-buildimage#19232, it added noticed level and make it to be default level which suppresses INFO logs.

`caclmgr.set_min_log_priority_info()` it sets min log priority to info, this function is in Logger class, SysLogger doesn't have this function. But DaemonBase still inherits Logger which implements set_min_log_priority_info, that's why even caclmgrd called this function, it didn't throw exception. But it didn't make INFO level effect in SysLogger which is actually used in caclmgrd

Even change to use Logger by setting `use_syslogger=False`, it still doesn't work.
The root cause is that it added a new instance for logger, `self.logger_instance`, any instance inherited from DaemonBase class can't change the debug level, the level they changed is their own instance, not the self.logger_instance's level.

### **How to fix**
The solution here for caclmgrd is to choose logger.Logger class instead of DaemonBase.

### **How to verify it**
Test it on 202405
@mssonicbld mssonicbld mentioned this issue Jan 9, 2025
Merged
bingwang-ms pushed a commit to sonic-net/sonic-host-services that referenced this issue Jan 9, 2025
@mssonicbld
Fix the issue of no info log in syslog for caclmgrd (#201)
bae9dbd 
### **Issue**
Fix the issue sonic-net/sonic-buildimage#21290
No info log found in syslog on 202405 image for caclmgrd

### **Work item tracking**
- Microsoft ADO **(number only)**:
30611546

### Why did it happen
RP sonic-net/sonic-buildimage#17171, introduced a new Class SysLogger, DaemonBase will choose SysLogger by default
PR sonic-net/sonic-buildimage#19232, it added noticed level and make it to be default level which suppresses INFO logs.

`caclmgr.set_min_log_priority_info()` it sets min log priority to info, this function is in Logger class, SysLogger doesn't have this function. But DaemonBase still inherits Logger which implements set_min_log_priority_info, that's why even caclmgrd called this function, it didn't throw exception. But it didn't make INFO level effect in SysLogger which is actually used in caclmgrd

Even change to use Logger by setting `use_syslogger=False`, it still doesn't work.
The root cause is that it added a new instance for logger, `self.logger_instance`, any instance inherited from DaemonBase class can't change the debug level, the level they changed is their own instance, not the self.logger_instance's level.

### **How to fix**
The solution here for caclmgrd is to choose logger.Logger class instead of DaemonBase.

### **How to verify it**
Test it on 202405
@ZhaohuiS ZhaohuiS mentioned this issue Jan 16, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ZhaohuiS ZhaohuiS

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ZhaohuiS