catalinux / ninedogs (public) (License: GPLv3) (since 2022-10-21) (hash sha1)
LD_PRELOAD interceptor for performance and problems reporting
Clone URLs: https://rocketgit.com/user/catalinux/ninedogs ssh://rocketgit@ssh.rocketgit.com/user/catalinux/ninedogs git://git.rocketgit.com/user/catalinux/ninedogs
main release
List of commits:
Subject Hash Author Date (UTC)
Mostly documentation updates fadbfe517eac4b0de3783e79d9779af57791833d Catalin(ux) M. BOIE 2023-03-17 07:17:12
Another json_object_get_uint64 replaced with int64 21f272b86ed57916d0fb7cc05de20170c36cf1cb Catalin(ux) M. BOIE 2023-03-15 20:16:13
make chown optional to not get errors when building rpm 2541f217b4ee77748d03f553ce1f6576c43c7106 Catalin(ux) M. BOIE 2023-03-15 19:00:49
Wrong path to Makefile.common in Makefile 82d474d4e825fd1afaf225518512f075ca25295f Catalin(ux) M. BOIE 2023-03-15 18:16:58
DESTDIR installation fix 5b5b7917a4b26daed48f1bb6dee826533997b518 Catalin(ux) M. BOIE 2023-03-15 12:59:22
RockyLinux 8 does not have json_object_new_uint64 function ce66611027547ca6740f36efa4f9da60f47c6132 Catalin(ux) M. BOIE 2023-03-15 12:33:18
More functions added 920cc4824df01ccfc0b04450fdfa43568910cf85 Catalin(ux) M. BOIE 2023-03-15 06:17:29
Lots of changes 8c803765a4dd6d79e7f61927c47a5f1a19e3db31 Catalin(ux) M. BOIE 2023-03-13 05:51:28
Record full path of the cert for openssl 66e89d4e0eaf638859b39c791d82d233e2d0c2c2 Catalin(ux) M. BOIE 2023-03-03 18:41:57
Improve presentation 0600fd3c6e5fcb0dd152d0e680ea85a8ae89bd86 Catalin(ux) M. BOIE 2022-12-09 17:46:09
More curl tracing info 04013af7a216e6cf7f363b3895ffcebe78859a65 Catalin(ux) M. BOIE 2022-12-09 17:45:46
Added curl support c374ad15d2d5548d168084c0e8506929803e9bf0 Catalin(ux) M. BOIE 2022-11-30 11:28:48
More Latex modules were missing ff5bec2cfba2cf77e0115f89124cdeb83dfbc73d Catalin(ux) M. BOIE 2022-11-24 16:52:48
Forgot to add beamer 1080ddc6497af77b66fd57eb7b9da0abaf75cb2a Catalin(ux) M. BOIE 2022-11-24 16:39:30
Mispelled docs 463d312e5ddc8b237e03a4d9b4cfb33524393fb0 Catalin(ux) M. BOIE 2022-11-24 16:11:45
More docs updated 83876ce8fcf9ae03c189e4fc7f354bf67f328916 Catalin(ux) M. BOIE 2022-11-24 15:59:05
Record bytes read/written on fds 428c3905bb135725ac3bd2ecfe5949e434b179b5 Catalin(ux) M. BOIE 2022-11-24 15:58:25
A lot of small fixes 9a5841916f3f0870ca091f457102c7512b21808a Catalin(ux) M. BOIE 2022-11-23 07:28:06
Added pthread_join + small fixes a5206c335f8c858d791f114da7361190196ad4a4 Catalin(ux) M. BOIE 2022-11-21 14:05:31
Lots of updates 5583e8dbd607b684a14ced92b4d2ffdacbcf57e1 Catalin(ux) M. BOIE 2022-11-18 05:40:05
Commit fadbfe517eac4b0de3783e79d9779af57791833d - Mostly documentation updates
Author: Catalin(ux) M. BOIE
Author date (UTC): 2023-03-17 07:17
Committer name: Catalin(ux) M. BOIE
Committer date (UTC): 2023-03-17 07:17
Parent(s): 21f272b86ed57916d0fb7cc05de20170c36cf1cb
Signer:
Signing key:
Signing status: N
Tree: f3223096f17a0fc1948416838291c425a21bf7e2
File Lines added Lines deleted
agent/TODO 3 0
debian/control 1 1
docs/pre1.tex 103 3
docs/pre1.txt 0 18
ingestd/Makefile 1 1
ingestd/ninedogs-ingestd.service 1 1
ninedogs.spec 1 0
tools/.gitignore 1 1
tools/Makefile 2 2
tools/nd-cert-notify.c 0 0
tools/ninedogs-cert-notify.service 1 1
trace/nd-trace.c 1 1
webd/ninedogs-webd.service 1 1
File agent/TODO changed (mode: 100644) (index d1dc5bd..d0eca21)
... ... send q2 str
16 16 close conn close conn
17 17
18 18 == Random == == Random ==
19 [ ] I can record when a high level starts and when ends - show total time taken!
20 Add this to the presentation.
21 [ ] Add meta nformation for curl calls.
19 22 [ ] mysqli_stmt_execute could show the value of the parameters [ ] mysqli_stmt_execute could show the value of the parameters
20 23 associated with mysqli_stmt_bind_param. associated with mysqli_stmt_bind_param.
21 24 [ ] Add an identification for the pids attached, to store them in the log. [ ] Add an identification for the pids attached, to store them in the log.
File debian/control changed (mode: 100644) (index 4f8a92e..a162c65)
... ... Source: ninedogs
2 2 Maintainer: Catalin(ux) M. BOIE <catab@embedromix.ro> Maintainer: Catalin(ux) M. BOIE <catab@embedromix.ro>
3 3 Section: misc Section: misc
4 4 Priority: optional Priority: optional
5 Build-Depends: debhelper (>= 9), libgnutls28-dev, json-c-dev, libcap-dev, texlive-latex-base, texlive-latex-extra, libcurl-dev, catalinux+Conn
5 Build-Depends: debhelper (>= 9), libgnutls28-dev, json-c-dev, libcap-dev, texlive-latex-base, texlive-latex-extra, texlive-graphics, libcurl-dev, catalinux+Conn
6 6 Standards-Version: 3.9.6 Standards-Version: 3.9.6
7 7 Homepage: https://rocketgit.com/user/catalinux/ninedogs Homepage: https://rocketgit.com/user/catalinux/ninedogs
8 8 Vcs-Git: git://rocketgit.com/user/catalinux/ninedogs Vcs-Git: git://rocketgit.com/user/catalinux/ninedogs
File docs/pre1.tex changed (mode: 100644) (index 2ae00f1..995b692)
1 \documentclass[10pt]{beamer}
1 \documentclass[9pt,aspectratio=169]{beamer}
2 2
3 3 \usetheme{default} \usetheme{default}
4 4
 
6 6 \usepackage{hyperref} \usepackage{hyperref}
7 7 \usepackage{url} \usepackage{url}
8 8 \usepackage[utf8]{inputenc} \usepackage[utf8]{inputenc}
9 \usepackage{graphicx}
9 10
10 11 \title{ninedogs project} \title{ninedogs project}
11 12 \subtitle{Augmented high and low-level tracing} \subtitle{Augmented high and low-level tracing}
 
49 50 \item A log collection tool (future). \item A log collection tool (future).
50 51 \item An alerting tool for certificate imminent expiration and security issues alerting. \item An alerting tool for certificate imminent expiration and security issues alerting.
51 52 \item A tool for live patching of applications (log4j etc.) (future). \item A tool for live patching of applications (log4j etc.) (future).
52 \item It is free software (license is AGPL, the most developer and user friendly possible).
53 \item It is free software (license is AGPL, the best for both contributors and users).
54 \item No change or recompile required for your programs to be traced.
55 \item Written in plain C.
53 56 \end{itemize} \end{itemize}
54 57 \end{block} \end{block}
55 58 \end{frame} \end{frame}
 
57 60 \begin{frame} \begin{frame}
58 61 \begin{block}{How ninedogs is working?} \begin{block}{How ninedogs is working?}
59 62 \begin{itemize} \begin{itemize}
60 \item It uses LD\_PRELOAD mechanism to hook all interesting shared libraries
63 \item It uses LD\_PRELOAD mechanism to hook most interesting shared libraries
61 64 calls and write the events in a shared memory area. calls and write the events in a shared memory area.
62 65 \item The tracing tool (nd-trace) will attach to the target process' shared \item The tracing tool (nd-trace) will attach to the target process' shared
63 66 memory area and exfiltrate the data. memory area and exfiltrate the data.
 
67 70 \end{block} \end{block}
68 71 \end{frame} \end{frame}
69 72
73 \begin{frame}
74 \begin{block}{What "augmented" tracing means?}
75 \begin{itemize}
76 \item ninedogs will not just blindly output functions which are executing.
77 It also improves the output with more information to help the user
78 spot the problems.
79 \item Some examples:
80 \begin{itemize}
81 \item When generating an image with gd, ninedogs outputs how much time
82 it took to generate the image.
83 \item When an Oracle clients executes a prepared query, ninedogs will
84 remind you what the query was. Else, it is hard to figure out which
85 query will be executed [in progress].
86 \item When curl executes a request, the total elapsed time is shown [in progress].
87 \item When an Oracle client runs a query the values of IN bindings are
88 shown before the query and the values of OUT bindings are shown
89 after the query.
90 \end{itemize}
91 \item Other augmented information will be added in the next releases
92 \end{itemize}
93 \end{block}
94 \end{frame}
95
70 96 \begin{frame} \begin{frame}
71 97 \begin{block}{Why strace/ltrace are not good enough?} \begin{block}{Why strace/ltrace are not good enough?}
72 98 \begin{itemize} \begin{itemize}
 
... ... Segmentation fault (core dumped)
440 466 \end{block} \end{block}
441 467 \end{frame} \end{frame}
442 468
469 \begin{frame}[fragile]{Colors}
470 \begin{block}{colored output, time sensitive}
471 \includegraphics[scale=0.35]{colors1.png}
472 \end{block}
473 \end{frame}
474
443 475
444 476 \subsection{Demo} \subsection{Demo}
445 477
 
... ... $ nd-trace -p 1 # pid 1 is usualy the main program
465 497 \end{frame} \end{frame}
466 498
467 499
500
501 \section{Tools}
502
503 \subsection{nd-cert-notify}
504
505 \begin{frame}[fragile]{nd-cert-notify}
506 \begin{itemize}
507 \item A tool to report the close to expiration certificates used by applications
508 running under ninedogs
509 \item Zero configuration needed
510 \end{itemize}
511 \begin{block}{Sample e-mail:}
512 \small
513 \begin{verbatim}
514 Subject: Expiring certificates (1)
515
516 Subject: /C=RO/CN=expiring.example.com/L=Brasov/O=openssl
517 Issuer: /C=RO/CN=expiring.example.com/L=Brasov/O=openssl
518 Path: /date/sync/no-crypt/sync1/Dev/ninedogs/test/openssl/expiring-certificate.crt
519 Serial: 112233ee
520 Not before: 2023-03-03 18:39:41
521 Not after: 2023-03-16 18:39:41
522 Users:
523 Command: [openssl] [x509] [-in] [expiring-certificate.crt] [-text] [-noout]
524 Last start of command: 2023-03-03 18:39:47
525 Command: [nginx: master process /usr/sbin/nginx]
526 Last start of command: 2023-03-03 18:39:52
527 \end{verbatim}
528 \end{block}
529 \end{frame}
530
531
532 \subsection{nd-info}
533
534 \begin{frame}[fragile]{nd-info}
535 \begin{itemize}
536 \item A tool to report information and statistics about the file descriptors
537 of a traced process.
538 \item Better reporting than /proc/pid/fd
539 \end{itemize}
540 \begin{block}{Sample:}
541 \tiny
542 \begin{verbatim}
543 1025713: peer version is 1
544 nd-info: waiting for data...
545 Process started 2s ago at Thu Mar 16 07:47:33 2023
546 File descriptors:
547 socket fd=6 stream/tcp bind=[n/a] peer=[ipv6/::1/443/flow=0/scope=0] backlog=0 accepts=0 ops=0/0 bytes=0/0 errs=0/0 [r/w]
548 file fd=7 path=[/dev/null] flags=0x0 mode=0x0 off=0 ops=0/1 bytes=0/0 errs=0/1 [r/w]
549 file fd=13 path=[1.tmp] flags=0x42 mode=0x1b6 off=3 ops=0/1 bytes=0/5 errs=0/0 [r/w]
550 socket fd=14 stream/ip bind=[ipv4/127.0.0.1/65432] peer=[n/a] backlog=10 accepts=1 ops=0/0 bytes=0/0 errs=0/0 [r/w]
551 socket fd=15 stream/ip bind=[n/a] peer=[ipv4/127.0.0.1/65432] backlog=0 accepts=0 ops=0/0 bytes=0/0 errs=0/0 [r/w]
552 socket fd=16 stream/ip bind=[n/a] peer=[n/a] backlog=0 accepts=0 accept_fd=15 ops=0/0 bytes=0/0 errs=0/0 [r/w]
553 socket fd=17 seqpacket/ip bind=[unix/'1.unix.tmp'] peer=[n/a] backlog=50000 accepts=1 ops=0/0 bytes=0/0 errs=0/0 [r/w]
554 socket fd=18 seqpacket/ip bind=[n/a] peer=[unix/'1.unix.tmp'] backlog=0 accepts=0 ops=0/0 bytes=0/0 errs=0/0 [r/w]
555 socket fd=19 seqpacket/ip bind=[n/a] peer=[n/a] backlog=0 accepts=0 accept_fd=18 ops=0/1 bytes=0/3 errs=0/0 [r/w]
556 mem fd=20 path=[bla] flags=0x3|cloexec|allow_sealing off=4 size=200 ops=0/1 bytes=0/4 errs=0/0 [r/w]
557 timer fd=21 clock=realtime flags=0x800|nonblock ops=0/0 bytes=0/0 errs=0/0 [r/w]
558 \end{verbatim}
559 \end{block}
560 \end{frame}
561
562
563
468 564 \section{Misc} \section{Misc}
469 565
470 566 \subsection{What is coming next?} \subsection{What is coming next?}
 
... ... $ nd-trace -p 1 # pid 1 is usualy the main program
475 571 \item Send logs to the server (intercept any open of filenames containing '.log' or '/logs?/'). \item Send logs to the server (intercept any open of filenames containing '.log' or '/logs?/').
476 572 \item Adding a cron to report used certificates close to expiration date. \item Adding a cron to report used certificates close to expiration date.
477 573 \item nd-trace: report statistics (number of calls and average time spent per function). \item nd-trace: report statistics (number of calls and average time spent per function).
574 \item More notification channels (Matrix, Slack etc.).
478 575 \end{itemize} \end{itemize}
479 576 \end{frame} \end{frame}
480 577
 
... ... $ nd-trace -p 1 # pid 1 is usualy the main program
488 585 \vspace{2ex} \vspace{2ex}
489 586 Download/history/artifacts: \href{https://rocketgit.com/user/catalinux/ninedogs}{https://rocketgit.com/user/catalinux/ninedogs} Download/history/artifacts: \href{https://rocketgit.com/user/catalinux/ninedogs}{https://rocketgit.com/user/catalinux/ninedogs}
490 587
588 \vspace{2ex}
589 Latest version of this document: \href{https://rocketgit.com/user/catalinux/ninedogs/artifacts/download/docs/pre1.pdf}{https://rocketgit.com/user/catalinux/ninedogs/artifacts/download/docs/pre1.pdf}
590
491 591 \vspace{2ex} \vspace{2ex}
492 592 We need sponsors, please contact us if you want to become one. We need sponsors, please contact us if you want to become one.
493 593 \end{frame} \end{frame}
File docs/pre1.txt changed (mode: 100644) (index 3743946..62e1a85)
... ... Ideas for the first presentation
2 2
3 3 - Who is the target of the presentation. - Who is the target of the presentation.
4 4 - Why? - Why?
5 - Difference between strace and nd-trace
6 - Why strace cannot be used in cloud in containers
7 - Talk about high-level decoding
8 - Show some examples
9 - Performance impact
10 - Sponsors?
11 - License!
12 - Multiple slides, show how to use it.
13 - Insist on free software
14 - One tool to rule them all.
15 - Explain "augmented".
16 - Language: C
17 - Where you can find the latest version of this presentation?
18 - Switch to 16:9 format
19 - No re-compilation, no hard deployment procedure => easy all over
20 5 - Package management security updates not done. - Package management security updates not done.
21 - add color1.png
22 - Add cert-notify page
23 - Add nd-info page
24 6
25 7 - Ce probleme vrei sa le rezolvi? - Ce probleme vrei sa le rezolvi?
26 8 - Cum te ajuta produsul meu sa-ti rezolvi problemele? - Cum te ajuta produsul meu sa-ti rezolvi problemele?
File ingestd/Makefile changed (mode: 100644) (index d4dc875..850db5e)
... ... ninedogs-ingestd: ninedogs-ingestd.c $(OBJS)
30 30
31 31 install: ninedogs-ingestd install: ninedogs-ingestd
32 32 @mkdir -p $(I_USR_SBIN) @mkdir -p $(I_USR_SBIN)
33 @cp ninedogs-ingestd $(I_USR_SBIN)
33 @cp -v ninedogs-ingestd $(I_USR_SBIN)
34 34 @mkdir -pv $(I_USR)/lib/systemd/system @mkdir -pv $(I_USR)/lib/systemd/system
35 35 cp -vd *.service $(I_USR)/lib/systemd/system/ cp -vd *.service $(I_USR)/lib/systemd/system/
36 36 @systemctl daemon-reload @systemctl daemon-reload
File ingestd/ninedogs-ingestd.service changed (mode: 100644) (index 6ac6cab..002c7a9)
... ... Description = Ninedogs ingestion daemon
3 3
4 4 [Service] [Service]
5 5 Type = exec Type = exec
6 ExecStart = /date/sync/no-crypt/sync1/Dev/ninedogs/ingestd/ninedogs-ingestd
6 ExecStart = /usr/sbin/ninedogs-ingestd
7 7 PrivateTmp = true PrivateTmp = true
8 8 Restart = on-failure Restart = on-failure
9 9 RestartSec = 10 RestartSec = 10
File ninedogs.spec changed (mode: 100644) (index ea74dbb..e2e9af3)
... ... BuildRequires: libcurl-devel, catalinux+Conn
21 21 # To generate the pdfs: # To generate the pdfs:
22 22 BuildRequires: texlive-latex, texlive-beamer, texlive-adjustbox BuildRequires: texlive-latex, texlive-beamer, texlive-adjustbox
23 23 BuildRequires: texlive-babel-english, texlive-hyperref, texlive-url BuildRequires: texlive-babel-english, texlive-hyperref, texlive-url
24 BuildRequires: texlive-graphics
24 25 # For tests - TODO - postgres # For tests - TODO - postgres
25 26 # Seems is not available on RockyLinux 8! # Seems is not available on RockyLinux 8!
26 27 #BuildRequires: mysql-connector-java #BuildRequires: mysql-connector-java
File tools/.gitignore changed (mode: 100644) (index e30f9ac..1907fa6)
1 cert-notify
1 nd-cert-notify
File tools/Makefile changed (mode: 100644) (index 4424a54..106077c)
... ... OBJS := \
9 9 ../common/tools.o ../common/info.o ../common/decode_text.o \ ../common/tools.o ../common/info.o ../common/decode_text.o \
10 10 ../common/sctools.o ../common/bin2struct.o ../common/sctools.o ../common/bin2struct.o
11 11
12 ALL_TOOLS := cert-notify
12 ALL_TOOLS := nd-cert-notify
13 13
14 14 all: $(ALL_TOOLS) all: $(ALL_TOOLS)
15 15
16 cert-notify: cert-notify.c Makefile $(COMMON_H) $(OBJS)
16 nd-cert-notify: nd-cert-notify.c Makefile $(COMMON_H) $(OBJS)
17 17 $(CC) $(CFLAGS) $@.c -o $@ $(OBJS) $(CURL_LIBS) $(CC) $(CFLAGS) $@.c -o $@ $(OBJS) $(CURL_LIBS)
18 18
19 19 .PHONY: clean .PHONY: clean
File tools/nd-cert-notify.c renamed from tools/cert-notify.c (similarity 100%)
File tools/ninedogs-cert-notify.service changed (mode: 100644) (index 96ce6ec..720c434)
... ... Description = Notify users about certificates problems (about to expire etc.)
3 3 Documentation = https://rocketgit.com/user/catalinux/ninedogs Documentation = https://rocketgit.com/user/catalinux/ninedogs
4 4
5 5 [Service] [Service]
6 ExecStart = /usr/bin/cert-notify
6 ExecStart = /usr/bin/nd-cert-notify
7 7 Type = simple Type = simple
8 8 User = ninedogs User = ninedogs
9 9 Group = ninedogs Group = ninedogs
File trace/nd-trace.c changed (mode: 100644) (index 375f3e5..0ff6e40)
... ... static void decode_func(const uint32_t parent, unsigned char *d)
1336 1336 snprintf(rest, sizeof(rest), "'%s'", s); snprintf(rest, sizeof(rest), "'%s'", s);
1337 1337 } else { } else {
1338 1338 snprintf(rest, sizeof(rest), snprintf(rest, sizeof(rest),
1339 "[cannot dump yet because size was not set yet]");
1339 "[cannot dump because size was not set yet]");
1340 1340 } }
1341 1341 } else if (type == 5) { // slist } else if (type == 5) { // slist
1342 1342 decode_string_array(rest, sizeof(rest), d, &i); decode_string_array(rest, sizeof(rest), d, &i);
File webd/ninedogs-webd.service changed (mode: 100644) (index 2bf43a8..7382e42)
... ... Description = Ninedogs Websocket daemon
3 3
4 4 [Service] [Service]
5 5 Type = exec Type = exec
6 ExecStart = /date/sync/no-crypt/sync1/Dev/ninedogs/webd/ninedogs-webd
6 ExecStart = /usr/sbin/ninedogs-webd
7 7 PrivateTmp = true PrivateTmp = true
8 8 Restart = on-failure Restart = on-failure
9 9 RestartSec = 10 RestartSec = 10
Date/time (UTC) Type Misc Labels
2023-03-29 20:26 build fedora-rawhide-x86_64 worker/r1 builder/color=fff worker_elap/553s wait_time/1083573s date/2023-03-17 time/07:17
2023-03-29 21:19 build fedora-37-x86_64 worker/r1 builder/color=fff worker_elap/668s wait_time/1086603s date/2023-03-17 time/07:17
Hints:
Before first commit, do not forget to setup your git environment:
git config --global user.name "your_name_here"
git config --global user.email "your@email_here"

Clone this repository using HTTP(S):
git clone https://rocketgit.com/user/catalinux/ninedogs

Clone this repository using ssh (do not forget to upload a key first):
git clone ssh://rocketgit@ssh.rocketgit.com/user/catalinux/ninedogs

Clone this repository using git:
git clone git://git.rocketgit.com/user/catalinux/ninedogs

You are allowed to anonymously push to this repository.
This means that your pushed commits will automatically be transformed into a merge request:
... clone the repository ...
... make some changes and some commits ...
git push origin main