catalinux / ninedogs (public) (License: GPLv3) (since 2022-10-21) (hash sha1)
LD_PRELOAD interceptor for performance and problems reporting
Clone URLs: https://rocketgit.com/user/catalinux/ninedogs ssh://rocketgit@ssh.rocketgit.com/user/catalinux/ninedogs git://git.rocketgit.com/user/catalinux/ninedogs
main release
List of commits:
Subject Hash Author Date (UTC)
Improve presentation 0600fd3c6e5fcb0dd152d0e680ea85a8ae89bd86 Catalin(ux) M. BOIE 2022-12-09 17:46:09
More curl tracing info 04013af7a216e6cf7f363b3895ffcebe78859a65 Catalin(ux) M. BOIE 2022-12-09 17:45:46
Added curl support c374ad15d2d5548d168084c0e8506929803e9bf0 Catalin(ux) M. BOIE 2022-11-30 11:28:48
More Latex modules were missing ff5bec2cfba2cf77e0115f89124cdeb83dfbc73d Catalin(ux) M. BOIE 2022-11-24 16:52:48
Forgot to add beamer 1080ddc6497af77b66fd57eb7b9da0abaf75cb2a Catalin(ux) M. BOIE 2022-11-24 16:39:30
Mispelled docs 463d312e5ddc8b237e03a4d9b4cfb33524393fb0 Catalin(ux) M. BOIE 2022-11-24 16:11:45
More docs updated 83876ce8fcf9ae03c189e4fc7f354bf67f328916 Catalin(ux) M. BOIE 2022-11-24 15:59:05
Record bytes read/written on fds 428c3905bb135725ac3bd2ecfe5949e434b179b5 Catalin(ux) M. BOIE 2022-11-24 15:58:25
A lot of small fixes 9a5841916f3f0870ca091f457102c7512b21808a Catalin(ux) M. BOIE 2022-11-23 07:28:06
Added pthread_join + small fixes a5206c335f8c858d791f114da7361190196ad4a4 Catalin(ux) M. BOIE 2022-11-21 14:05:31
Lots of updates 5583e8dbd607b684a14ced92b4d2ffdacbcf57e1 Catalin(ux) M. BOIE 2022-11-18 05:40:05
mysql, sqlite3, depth d9d83cafbd372d1055fa2e51cdf3deb2396f5d08 Catalin(ux) M. BOIE 2022-11-07 05:58:04
More spec fixes 3867b17137b5397c3eeabdcbbc232417ad212a27 Catalin(ux) M. BOIE 2022-10-25 19:04:28
More spec fixes 68abc4ce485fbbcc18a232e57b956f9e2b016d72 Catalin(ux) M. BOIE 2022-10-25 18:48:22
Fixed installation paths in Makefiles c98920bd55f7cd2ebcec7083e2360009d8c16b29 Catalin(ux) M. BOIE 2022-10-25 18:19:49
Debian support added, create correctly the paths for install 512c13ca06dda005504cdeab831431c943dde74a Catalin(ux) M. BOIE 2022-10-25 06:19:50
Bump version to 0.6 for a stupid reason (mass with dupdump) c4d10510ddf944965a80d9a3e5f4ac62b84a103e Catalin(ux) M. BOIE 2022-10-25 06:04:33
History update c83cd94af87636bc67bf9526ca0a65734ce96fac Catalin(ux) M. BOIE 2022-10-22 10:34:01
TODO update b1cbf0845dada017466763126d8569f62ace4cb4 Catalin(ux) M. BOIE 2022-10-22 08:20:20
Need libcap 4cdcd084fcf0e089e074aa0b090bc6f569a5d625 Catalin(ux) M. BOIE 2022-10-22 07:32:51
Commit 0600fd3c6e5fcb0dd152d0e680ea85a8ae89bd86 - Improve presentation
Author: Catalin(ux) M. BOIE
Author date (UTC): 2022-12-09 17:46
Committer name: Catalin(ux) M. BOIE
Committer date (UTC): 2022-12-09 17:46
Parent(s): 04013af7a216e6cf7f363b3895ffcebe78859a65
Signer:
Signing key:
Signing status: N
Tree: 89bee18b3e4d9e26aa8c14068b3442ea2210fc6d
File Lines added Lines deleted
docs/pre1.tex 20 18
docs/pre1.txt 3 0
File docs/pre1.tex changed (mode: 100644) (index 98e1c33..5e966f5)
8 8 \usepackage[utf8]{inputenc} \usepackage[utf8]{inputenc}
9 9
10 10 \title{ninedogs project} \title{ninedogs project}
11 \subtitle{Augumented high and low-level tracing}
11 \subtitle{Augmented high and low-level tracing}
12 12 \author{\small Catalin(ux) M. BOIE - ninedogs@embedromix.ro} \author{\small Catalin(ux) M. BOIE - ninedogs@embedromix.ro}
13 13 \date{\small 2022-11-09} \date{\small 2022-11-09}
14 14 %\titlegraphic{\includegraphics[width=2.5cm]{ninedogs.svg}} %\titlegraphic{\includegraphics[width=2.5cm]{ninedogs.svg}}
 
57 57 \begin{frame} \begin{frame}
58 58 \begin{block}{How ninedogs is working?} \begin{block}{How ninedogs is working?}
59 59 \begin{itemize} \begin{itemize}
60 \item It uses LD\_PRELOAD mechanism to hook all interesting shared libraries calls.
61 \item Examples of hooked functions (not syscalls! it does not use ptrace): recv, send, open,
62 mysqli\_fetch\_array, pg\_pconnect, sqlite3\_open\_v2, syslog etc.
63 \item Many more will come in each release.
60 \item It uses LD\_PRELOAD mechanism to hook all interesting shared libraries
61 calls and write the events in a shared memory area.
62 \item The tracing tool (nd-trace) will attach to the target process' shared
63 memory area and exfiltrate the data.
64 \item Examples of hooked functions (not syscalls, it does not use ptrace):
65 recv, send, open, mysqli\_fetch\_array, pg\_pconnect, sqlite3\_open\_v2, syslog etc.
64 66 \end{itemize} \end{itemize}
65 67 \end{block} \end{block}
66 68 \end{frame} \end{frame}
 
70 72 \begin{itemize} \begin{itemize}
71 73 \item They cannot be used in GCP and Azure (probably also AWS) containers for \item They cannot be used in GCP and Azure (probably also AWS) containers for
72 74 security reasons. security reasons.
73 \item They are low level tools; e.g. if a connection to a database is encrypted,
74 you cannot see anything interesting.
75 \item They are low level tools; e.g. if a connection to a database or web
76 server is encrypted, you cannot see anything interesting.
75 77 \item They are way too verbose. \item They are way too verbose.
76 78 \end{itemize} \end{itemize}
77 79 \end{block} \end{block}
 
81 83 \begin{block}{Why strace is better (if it can be used)?} \begin{block}{Why strace is better (if it can be used)?}
82 84 \begin{itemize} \begin{itemize}
83 85 \item strace can be used on static linked binaries, ninedogs cannot. \item strace can be used on static linked binaries, ninedogs cannot.
84 In practice, this is rarely encountered.
86 One notable exceptions are Go programs which are statically linked.
85 87 \end{itemize} \end{itemize}
86 88 \end{block} \end{block}
87 89 \end{frame} \end{frame}
 
92 94 \item It uses shared memory as a ring buffer to pass information to the tracer. \item It uses shared memory as a ring buffer to pass information to the tracer.
93 95 \item The biggest cost is copying from memory to memory (negligible). \item The biggest cost is copying from memory to memory (negligible).
94 96 \item It does not block if the ring is full. \item It does not block if the ring is full.
95 \item It uses a small amount of memory compared with other solutions.
97 \item It uses a small amount of memory and CPU compared with other solutions.
96 98 \end{itemize} \end{itemize}
97 99 \end{block} \end{block}
98 100 \end{frame} \end{frame}
 
... ... dnf install catalinux+ninedogs
128 130 \begin{block}{Usage in a systemd service file} \begin{block}{Usage in a systemd service file}
129 131 \small \small
130 132 \begin{verbatim} \begin{verbatim}
131 TODO
133 /etc/systemd/system/bla.service.d/ninedogs.conf:
134 Environment=LD_PRELOAD=ninedogs.so
132 135 \end{verbatim} \end{verbatim}
133 136 \end{block} \end{block}
134 137 \end{frame} \end{frame}
 
... ... curl_easy_cleanup(curl);
229 232 \tiny \tiny
230 233 \begin{verbatim} \begin{verbatim}
231 234 curl_easy_init() = 0x1c2b960 curl_easy_init() = 0x1c2b960
232 curl_easy_setopt(0x1c2b960, CURLOPT_URL, 'https://localhost') = ok
233 curl_easy_setopt(0x1c2b960, CURLOPT_POST, 1) = ok
234 curl_easy_setopt(0x1c2b960, CURLOPT_POSTFIELDS, 0x402010) = ok
235 curl_easy_setopt(0x1c2b960, CURLOPT_POSTFIELDSIZE, 10) = ok
236 curl_easy_setopt(0x1c2b960, CURLOPT_HTTPHEADER, {'Content-Type: application/json', 'X-ninedogs-bla: 1'}) = ok
235 curl_easy_setopt(0x1c2b960, URL, 'https://localhost') = ok
236 curl_easy_setopt(0x1c2b960, POST, 1) = ok
237 curl_easy_setopt(0x1c2b960, POSTFIELDS, '{"a": "1"}') = ok
238 curl_easy_setopt(0x1c2b960, POSTFIELDSIZE, 10) = ok
239 curl_easy_setopt(0x1c2b960, HTTPHEADER, {'Content-Type: application/json', 'X-ninedogs-bla: 1'}) = ok
237 240 curl_easy_perform(0x1c2b960) curl_easy_perform(0x1c2b960)
238 241 curl_easy_cleanup(0x1c2b960) curl_easy_cleanup(0x1c2b960)
239 242 \end{verbatim} \end{verbatim}
 
... ... Segmentation fault (core dumped)
441 444 \subsection{Demo} \subsection{Demo}
442 445
443 446 \begin{frame}[fragile]{Entering a container and do a trace} \begin{frame}[fragile]{Entering a container and do a trace}
444 We assume that a container named c1 is already runing and it has
447 We assume that a container named c1 is already running and it has
445 448 the ninedogs tools installed and that a program is already running with the ninedogs tools installed and that a program is already running with
446 449 \verb|LD_PRELOAD=ninedogs|. \verb|LD_PRELOAD=ninedogs|.
447 450 \small \small
448 451 \begin{verbatim} \begin{verbatim}
449 452 $ docker exec -it c1 /bin/bash # enter container $ docker exec -it c1 /bin/bash # enter container
450 $ nd-trace -o my_trace_file.nd -p 1
451 Saving output to [my_trace_file.nd]
453 $ nd-trace -p 1 # pid 1 is usualy the main program
452 454 1: shm_open returned 4 1: shm_open returned 4
453 455 1: Attached 1: Attached
454 456 16692xxxxx.054 1 getrandom(buf, 1, 0x1) 16692xxxxx.054 1 getrandom(buf, 1, 0x1)
File docs/pre1.txt changed (mode: 100644) (index 8eb2369..a3035c0)
... ... Ideas for the first presentation
12 12 - Multiple slides, show how to use it. - Multiple slides, show how to use it.
13 13 - Insist on free software - Insist on free software
14 14 - One tool to rule them all. - One tool to rule them all.
15 - Explain "augmented".
16 - Language: C
17 - Where you can find the latest version of this presentation?
15 18 - -
16 19
17 20 - Ce probleme vrei sa le rezolvi? - Ce probleme vrei sa le rezolvi?
Date/time (UTC) Type Misc Labels
2022-12-09 17:57 build fedora-37-x86_64 worker/r1 builder/color=fff worker_elap/464s wait_time/88s date/2022-12-09 time/17:47
2022-12-09 18:09 build fedora-rawhide-x86_64 worker/r1 builder/color=fff worker_elap/599s wait_time/611s date/2022-12-09 time/17:47
Hints:
Before first commit, do not forget to setup your git environment:
git config --global user.name "your_name_here"
git config --global user.email "your@email_here"

Clone this repository using HTTP(S):
git clone https://rocketgit.com/user/catalinux/ninedogs

Clone this repository using ssh (do not forget to upload a key first):
git clone ssh://rocketgit@ssh.rocketgit.com/user/catalinux/ninedogs

Clone this repository using git:
git clone git://git.rocketgit.com/user/catalinux/ninedogs

You are allowed to anonymously push to this repository.
This means that your pushed commits will automatically be transformed into a merge request:
... clone the repository ...
... make some changes and some commits ...
git push origin main