Preface¶
It has been several months since the last introduction of the ptcpdump project. Over these past months, I have been continuously developing this project. This article will introduce the major changes released from the previous v1.16 version to the latest v1.26 version in chronological order.
Main changes¶
Compatible with older versions of TencentOS/OpenCloudOS system¶
Base on user feedback the older versions were not compatible with previous editions of TencentOS/OpenCloudOS. However, the current update now includes support for the following TencentOS/OpenCloudOS releases: OpenCloudOS 7/8/9 and TencentOS Server 2.4/2.6/3.1/3.2.
Add parent process information to the output¶
Old version:
13:44:41.529003 eth0 In IP (tos 0x4, ttl 45, id 45428, offset 0, flags [DF], proto TCP (6), length 52)
139.178.84.217.443 > 172.19.0.2.42606: Flags [.], cksum 0x5284, seq 3173118145, ack 1385712707, win 118, options [nop,nop,TS val 134560683 ecr 1627716996], length 0
Process (pid 553587, cmd /usr/bin/wget, args wget kernel.org)
Container (...)
Pod (...)
Latest update:
13:44:41.529003 eth0 In IP (tos 0x4, ttl 45, id 45428, offset 0, flags [DF], proto TCP (6), length 52)
139.178.84.217.443 > 172.19.0.2.42606: Flags [.], cksum 0x5284, seq 3173118145, ack 1385712707, win 118, options [nop,nop,TS val 134560683 ecr 1627716996], length 0
Process (pid 553587, cmd /usr/bin/wget, args wget kernel.org)
ParentProc (pid 553296, cmd /bin/sh, args sh)
Container (...)
Pod (...)
Support using an older version of Kubernetes environment with Dockershim¶
The earlier release couldn't gather Pod data in legacy Kubernetes setups running Dockershim (specifically CRI v1alpha2). The latest update is now compatible with such environments.
Fix for filtering by Pod name containing periods¶
In the past, when filtering by Pod name, names with periods were not supported. The latest version now allows names with periods to be filtered:
--pod-name foo.bar.default
Fix the scenario where filtering by Pod does not support Pods with multiple containers¶
In previous versions, filtering by Pod did not support scenarios where Pods contain multiple containers. This issue has been resolved in the new version.
Supporting the Concurrent Execution of Multiple ptcpdump Instances¶
Previously, only one ptcpdump process could run at a time. Running multiple ptcpdump processes simultaneously would cause the old process to malfunction (capture no traffic). The latest version has resolved this issue. In the new version, multiple ptcpdump processes can run concurrently without interfering with each other or causing problems with packet capture for other processes.
Support Filtering Multiple Process PID Simultaneously¶
Now the --pid parameter supports filtering multiple PIDs simultaneously:
--pid pid1 --pid pid2
New parameters --micro, --nano, --time-stamp-precision added to control the time format in the output¶
New parameters --micro, --nano, --time-stamp-precision have been added to control the time format in the output, mirroring the functionality and usage of tcpdump.
--micro, --time-stamp-precision=micro:
13:36:05.701978 IP 10.0.2.15.22 > 10.0.2.2.59874: Flags [P.], seq 1370707216:1370707292, ack 4569736, win 62780, length 76
--nano, --time-stamp-precision=nano:
13:36:05.701978488 IP 10.0.2.15.22 > 10.0.2.2.59874: Flags [P.], seq 1370707216:1370707292, ack 4569736, win 62780, length 76
Add parameters -A, -x, -xx, -X, -XX to control the output format of the data¶
Add functionality and usage comparison of tcpdump flags -A, -x, -xx, -X, -XX.
-A:
14:36:38.159559 ens33 curl.244103 Out IP 10.0.2.15.53478 > 203.205.254.157.80: Flags [P.], seq 3293023896:3293023966, ack 1986159173, win 64240, length 70, ParentProc [ptcpdump.244094] E..nHL@.@... ..........P.G..vbbEP.......GET / HTTP/1.1 Host: qq.com User-Agent: curl/7.81.0 Accept: */*
-x:
14:36:38.159559 ens33 curl.244103 Out IP 10.0.2.15.53478 > 203.205.254.157.80: Flags [P.], seq 3293023896:3293023966, ack 1986159173, win 64240, length 70, ParentProc [ptcpdump.244094]
0x0000: 4500 006e 484c 4000 4006 1bc4 0a00 020f
0x0010: cbcd fe9d d0e6 0050 c447 8e98 7662 6245
0x0020: 5018 faf0 d6da 0000 4745 5420 2f20 4854
0x0030: 5450 2f31 2e31 0d0a 486f 7374 3a20 7171
0x0040: 2e63 6f6d 0d0a 5573 6572 2d41 6765 6e74
0x0050: 3a20 6375 726c 2f37 2e38 312e 300d 0a41
0x0060: 6363 6570 743a 202a 2f2a 0d0a 0d0a
-xx:
14:36:38.159559 ens33 curl.244103 Out IP 10.0.2.15.53478 > 203.205.254.157.80: Flags [P.], seq 3293023896:3293023966, ack 1986159173, win 64240, length 70, ParentProc [ptcpdump.244094]
0x0000: 0050 56eb bc4e 000c 298e 31f3 0800 4500
0x0010: 006e 484c 4000 4006 1bc4 0a00 020f cbcd
0x0020: fe9d d0e6 0050 c447 8e98 7662 6245 5018
0x0030: faf0 d6da 0000 4745 5420 2f20 4854 5450
0x0040: 2f31 2e31 0d0a 486f 7374 3a20 7171 2e63
0x0050: 6f6d 0d0a 5573 6572 2d41 6765 6e74 3a20
0x0060: 6375 726c 2f37 2e38 312e 300d 0a41 6363
0x0070: 6570 743a 202a 2f2a 0d0a 0d0a
-X:
14:36:38.159559 ens33 curl.244103 Out IP 10.0.2.15.53478 > 203.205.254.157.80: Flags [P.], seq 3293023896:3293023966, ack 1986159173, win 64240, length 70, ParentProc [ptcpdump.244094]
0x0000: 4500 006e 484c 4000 4006 1bc4 0a00 020f E..nHL@.@.......
0x0010: cbcd fe9d d0e6 0050 c447 8e98 7662 6245 .......P.G..vbbE
0x0020: 5018 faf0 d6da 0000 4745 5420 2f20 4854 P.......GET / HT
0x0030: 5450 2f31 2e31 0d0a 486f 7374 3a20 7171 TP/1.1..Host: qq
0x0040: 2e63 6f6d 0d0a 5573 6572 2d41 6765 6e74 .com..User-Agent
0x0050: 3a20 6375 726c 2f37 2e38 312e 300d 0a41 : curl/7.81.0..A
0x0060: 6363 6570 743a 202a 2f2a 0d0a 0d0a ccept: */*....
-XX:
14:36:38.159559 ens33 curl.244103 Out IP 10.0.2.15.53478 > 203.205.254.157.80: Flags [P.], seq 3293023896:3293023966, ack 1986159173, win 64240, length 70, ParentProc [ptcpdump.244094]
0x0000: 0050 56eb bc4e 000c 298e 31f3 0800 4500 .PV..N..).1...E.
0x0010: 006e 484c 4000 4006 1bc4 0a00 020f cbcd .nHL@.@.........
0x0020: fe9d d0e6 0050 c447 8e98 7662 6245 5018 .....P.G..vbbEP.
0x0030: faf0 d6da 0000 4745 5420 2f20 4854 5450 ......GET / HTTP
0x0040: 2f31 2e31 0d0a 486f 7374 3a20 7171 2e63 /1.1..Host: qq.c
0x0050: 6f6d 0d0a 5573 6572 2d41 6765 6e74 3a20 om..User-Agent:
0x0060: 6375 726c 2f37 2e38 312e 300d 0a41 6363 curl/7.81.0..Acc
0x0070: 6570 743a 202a 2f2a 0d0a 0d0a ept: */*....
Create Docker Images for Compiling and Running ptcpdump¶
Create a Docker image for compiling programs: quay.io/ptcpdump/develop:latest and a Docker image for running ptcpdump through Docker: quay.io/ptcpdump/ptcpdump:latest
Compile eBPF programs and ptcpdump programs as needed using make build-bpf-via-docker and make build-via-docker.
You can run ptcpdump through Docker using a command similar to the following:
docker run --privileged --rm -t --net=host --pid=host \ -v /sys/fs/cgroup:/sys/fs/cgroup:ro \ quay.io/ptcpdump/ptcpdump:latest ptcpdump -i any -c 2 tcp
Experimental New Feature: Generating Corresponding TLS Key Log File When Capturing Go Program's Network Traffic¶
A new experimental feature has been implemented in the latest version: generating the corresponding TLS Key Log file when capturing Go program's network traffic.
The TLS Key Log file, also known as SSLKEYLOGFILE , corresponds to the pre-master secret configuration in Wireshark. Third-party programs can decrypt captured TLS traffic using the information recorded in this file.
Please note: This feature is experimental and is only supported by Go programs compiled with version 1.18 or above. It also requires running the target Go program via ptcpdump.
You can try out this feature using the following two methods:
Use the --write-keylog-file argument or the SSLKEYLOGFILE environment variable to specify the location where the SSLKEYLOGFILE file will be saved:
sudo ptcpdump -i any --write-keylog-file /tmp/keylogfile.txt -w /tmp/go.pcapng -- ./gohttpapp
By specifying the --embed-keylog-to-pcapng parameter, embed the contents of the SSLKEYLOGFILE into the saved data in the pcapng format file:
sudo ptcpdump -i any --embed-keylog-to-pcapng -w /tmp/gotls.pcapng -- ./gohttpapp
Later, you can decrypt the saved TLS data using the recorded SSLKEYLOGFILE content with Wireshark or tshark:
$ sudo tshark -r /tmp/go.pcapng -o tls.keylog_file:/tmp/keylogfile.txt | grep HTTP -B 2 Running as user "root" and group "root". This could be dangerous. 18 0.518380586 10.0.2.15 → 13.35.238.63 TLSv1.3 118 Change Cipher Spec, Finished 19 0.519208290 13.35.238.63 → 10.0.2.15 TCP 60 443 → 47606 [ACK] Seq=5508 Ack=340 Win=64240 Len=0 20 0.519914720 10.0.2.15 → 13.35.238.63 HTTP 179 GET /foo/bar HTTP/1.1 $ tshark -r /tmp/gotls.pcapng | grep HTTP -B 2 20 0.525662563 10.0.2.15 → 13.35.238.114 TLSv1.3 118 Change Cipher Spec, Finished 21 0.526138582 13.35.238.114 → 10.0.2.15 TCP 60 443 → 37618 [ACK] Seq=5987 Ack=340 Win=64240 Len=0 22 0.526977836 10.0.2.15 → 13.35.238.114 HTTP 179 GET /foo/bar HTTP/1.1
ptcpdump plans to eventually have built-in support for decrypting TLS data using SSLKEYLOGFILE and to automatically decrypt TLS data in real-time while capturing packets. Currently, this can only be achieved using third-party tools for decrypting TLS data.
Output TCP options by default¶
The new version will by default output TCP Options information:
Old version:
14:09:54.324433 ens33 curl.26570 Out IP (..).43772 > (..).443: Flags [S], seq 1674193846, win 64240, length 0, ParentProc [ptcpdump.26560]
Latest version:
14:09:54.324433 ens33 curl.26570 Out IP (..).43772 > (..).443: Flags [S], seq 1674193846, win 64240, options [mss 1460,sackOK,TS val 2107137325 ecr 0,nop,wscale 7], length 0, ParentProc [ptcpdump.26560]
Moreover, the latest update includes support for TCP SACK (Selective Acknowledgment) and TFO (TCP Fast Open).
SACK:
19:03:36.220872 IP6 dead:beef:2::2.35288 > dead:beef:2::1.10029: Flags [.], seq 731670714, ack 2274465610, win 201, options [nop,nop,TS val 1253137130 ecr 837820024,nop,nop,sack 1 {2274467010:2274483378},mptcp 12 dss ack 16301812255838552430], length 0
TFO:
19:22:26.586851 IP6 dead:beef:1::2.54040 > dead:beef:1::1.10056: Flags [S], seq 271661201, win 64800, options [mss 1440,sackOK,TS val 2947503028 ecr 0,nop,wscale 7,tfo cookiereq,nop,nop,mptcp 4 capable v1 flags [H]], length 0
19:22:26.591736 IP6 dead:beef:1::1.10056 > dead:beef:1::2.54040: Flags [S.], seq 1229575956, ack 271661202, win 64260, options [mss 1440,nop,nop,sackOK,nop,wscale 7,tfo cookie 29b3cc66639d427d,nop,nop,mptcp 12 capable v1 flags [H] {0xc87438912bc26cb7}], length 0
Output the MPTCP information in the TCP Options¶
According to the requirement from the MPTCP (MultiPath TCP) community, support for MPTCP should be added:
15:31:51.696224 IP 10.0.1.2.60958 > 10.0.1.1.10004: Flags [S], seq 3019570341, win 64240, options [mss 1460,sackOK,TS val 1007819908 ecr 0,nop,wscale 7,mptcp 4 capable v1 flags [H]], length 0
15:31:51.696346 IP 10.0.1.1.10004 > 10.0.1.2.60958: Flags [S.], seq 2367868313, ack 3019570342, win 65160, options [mss 1460,sackOK,TS val 162498895 ecr 1007819908,nop,wscale 7,mptcp 12 capable v1 flags [H] {0x8ea1df6e0d588003}], length 0
15:31:51.696587 IP 10.0.1.2.60958 > 10.0.1.1.10004: Flags [.], seq 3019570342, ack 2367868314, win 502, options [nop,nop,TS val 1007819909 ecr 162498895,mptcp 20 capable v1 flags [H] {0x465bcd01b5d78120,0x8ea1df6e0d588003}], length 0
Add a new parameter --netns to support capturing network interfaces in other network namespaces¶
The previous version only supported capturing network interfaces in the current network namespace. The new version now supports capturing network interfaces in other network namespaces by adding the --netns parameter:
sudo ptcpdump -i lo --netns /run/netns/foobar sudo ptcpdump -i any --netns /run/netns/foobar sudo ptcpdump -i any --netns /proc/26/ns/net
PcapNg format enhancement: reading/writing network interface names and Inbound/Outbound information¶
In previous versions, using ptcpdump -r <file.pcapng> to read a pcapng file did not display the recorded network interface names and Inbound/Outbound information. This issue has been resolved in the new version:
$ ptcpdump -r demo.pcapng 14:36:35.880947 ens33 curl.244103 Out IP 10.0.2.15.37668 > 114.114.114.114.53: 44427+ A? qq.com. (24), ParentProc [ptcpdump.244094] 14:36:35.882099 ens33 curl.244103 Out IP 10.0.2.15.37668 > 114.114.114.114.53: 31415+ AAAA? qq.com. (24), ParentProc [ptcpdump.244094] 14:36:35.954613 ens33 curl.244103 In IP 114.114.114.114.53 > 10.0.2.15.37668: 44427 3/0/0 A 203.205.254.157, A 113.108.81.189, A 123.150.76.218 (72), ParentProc [ptcpdump.244094]
When saving the captured traffic information as a PcapNg format file, the new version will automatically write Inbound/Outbound information (previous versions already wrote network interface information), thus supporting the epb_flags (Enhanced Packet Block Flags Word ) flag for the PcapNg format.
If you have any additional improvements or new feature suggestions for ptcpdump, feel free to leave a comment in the comments section or project issues.
Comments