Capturing clear-text network exchanges from the proprietary HP driver

Allow sending scan from device to computer for some HP All-in-One Printers - Scan to computer

Capturing clear-text network exchanges from the proprietary HP driver

Addresses the task of issue #1019: document how to capture and analyze in the clear the network exchanges between the proprietary HP driver (Windows) and the printer, in particular when the communication is HTTPS-encrypted.

Prerequisites (Linux)

  1. A Linux host (Debian-based, as used here) with Docker supporting bridge + macvlan networks and the NET_ADMIN capability.
  2. The HP driver files on disk (Windows install media / files):
    • the interactive installer webpack (Full_Webpack-*.exe), and/or
    • an extracted DriverStore (for static analysis via fetch-driver.sh), and/or
    • the real driver config ProgramData/HP/<Product>/NetworkDevices/<SERIAL>.ini. Mount the Windows drive (e.g. /mnt/win) and pass its root to prepare.sh.
  3. The printer reachable on the LAN.
  4. Everything else (WineHQ wine-devel, mono, tcpdump, tshark, gcc/libgnutls) is built inside the container by prepare.sh.

Why it is needed

The proprietary HP protocols (WalkupScanToComp, WalkupScan, ScanJob, eSCL) are documented in protocol_doc/ from captures made on Windows machines over plain HTTP (ports 80/8080). On newer printers (e.g. HP Smart Tank Plus 570 series) the driver code also references a “secure” variant — the path constant /WalkupScanToComp/SecureWalkupScantoCompDestinations (and the schema wus:/SecureWalkupScantoCompDestinations) — which would be used over HTTPS. Note that on the tested device this endpoint is not exposed (it returns 404 on ports 80/443/8080); it is a code constant, not a confirmed device endpoint. To keep reverse-engineering these protocols, the HTTPS traffic still needs to be readable in the clear.

The problem: SSLKEYLOGFILE does not work under Wine

The classic “Wireshark + SSLKEYLOGFILE” method (proposed in the issue) does not work for a Windows driver running under Wine:

As a result, a classic MITM (mitmproxy) is not enough either, because Wine’s WinHTTP certificate validation does not read the crypt32 certificate store (ERROR_WINHTTP_CERT_NOT_TRUSTED even with the printer’s real self-signed certificate imported as a root).

The validated solution: enable the GnuTLS keylog via LD_PRELOAD

Wine loads the system libgnutls library for its schannel. libgnutls exposes gnutls_session_set_keylog_function() (exported in libgnutls.so.30). Intercepting gnutls_init() with a small LD_PRELOAD shim is enough to register that callback on every TLS session: all session keys are then written in NSS format (CLIENT_RANDOM <client_random_hex> <master_secret_hex>), which Wireshark / tshark can decrypt.

The capture is done without MITM: the driver talks directly to the printer, and one combines (a) the traffic pcap and (b) the key log file.

Diagram

HP Windows driver (under Wine, dedicated uid, inside a container)
   │  direct HTTPS to <printer-ip>:443
   ▼
gnutls_init() interception ──►  SSLKEYLOGFILE (session keys)
   │
   ▼
tcpdump (in the container) ──►  pcap
   │
   └──► tshark -o tls.keylog_file:keys.log  →  plaintext

Everything happens inside an isolated Docker container (private network namespace): no host network change, no system proxy, no certificate in the host store.

Components

1. LD_PRELOAD shim (sslkeylog-gnutls.c.txt)

See protocol_doc/capture/tools/sslkeylog-gnutls.c.txt for the full source. Compile:

gcc -shared -fPIC -x c -o sslkeylog-gnutls.so sslkeylog-gnutls.c.txt -ldl -lgnutls

Note: the wrong ELF class warning at startup (the 32-bit Wine process cannot preload the 64-bit shim) is harmless — the 64-bit process (the one running the x64 HP binaries) does load the shim.

2. Isolated Docker Wine environment

Base: Debian bookworm + Wine from the WineHQ repository (wine-devel; pin amd64/i386 to the same version, e.g. 11.10, because the bookworm repo can temporarily have a newer amd64 than i386), plus Xvfb, tcpdump, tshark, mono, gcc/libgnutls. See protocol_doc/capture/tools/Dockerfile and protocol_doc/capture/tools/prepare.sh for the full rebuild recipe.

Launch (default bridge network, no --network=host):

docker run -d --name wine-capture --network bridge --cap-add=NET_ADMIN \
  -v "$PWD/capture:/capture" wine-capture sleep 3600

3. Capture an HTTPS call (end-to-end validation)

The HP driver uses WinHTTP/WinINet. To validate the method without a full install (the complete driver needs an interactive installer), a minimal WinHTTP client is enough (protocol_doc/capture/tools/winhttp-get.c.txt). Important points:

export SSLKEYLOGFILE=/capture/keys.log
export LD_PRELOAD=/opt/sslkeylog-gnutls.so
tcpdump -i any -w /capture/capture.pcap 'tcp port 443' &
wine winhttp-get.exe https://<printer-ip>/DevMgmt/DiscoveryTree.xml
kill %1
# Decrypt:
tshark -r /capture/capture.pcap -o 'tls.keylog_file:/capture/keys.log' \
  -Y http -T fields -e http.request.method -e http.host -e http.request.uri -e http.response.code

Verified result on an HP Smart Tank Plus 570 series:

GET <printer-ip>  /DevMgmt/DiscoveryTree.xml   200

Multicast is blocked by the Docker bridge

Printer location uses multicast. A Docker container on the default bridge network does not send/receive LAN multicast: the driver cannot locate the printer that way. Two options:

Tooling (all files in protocol_doc/capture/tools/)

File Role
Dockerfile Wine image (WineHQ repo, wine-devel 11.10) + Xvfb, tcpdump, tshark, gcc/libgnutls, mono. Isolated container (no --network=host).
prepare.sh Full rebuild recipe: creates the macvlan+bridge networks, the container, the Wine prefix (mono), the GnuTLS shim, and copies the HP driver + printer config (Windows source path passed as an argument).
drive-installer.sh Drives the interactive HP webpack installer under Wine (OCR + xdotool): clicks “Continue”, checks the EULA checkbox, clicks “Accept”. Experimental — the WebView EULA checkbox is fragile.
fetch-driver.sh Copies a complete HP DriverStore (driver DLLs) from a Windows install (path as argument) into driver-store/ (git-ignored).
sslkeylog-gnutls.c.txt LD_PRELOAD shim enabling SSLKEYLOGFILE for GnuTLS (Wine’s schannel backend) — the key to decrypt the driver’s HTTPS.
winhttp-get.c.txt Minimal WinHTTP client (certificate validation disabled) to validate the decryption chain without installing the complete driver.

Rebuild + driving the installer:

cd protocol_doc/capture/tools
docker build -t wine-capture .
./prepare.sh "/mnt/win" "HP Smart Tank Plus 570 series/NetworkDevices/CNXXXX.ini" \
  192.168.1.50 192.168.1.0/24 192.168.1.1 192.168.1.200/28 enp2s0
./drive-installer.sh "/downloads/Full_Webpack-50.2.4593_1-ST570_Full_Webpack.exe"

Fetching a real DriverStore for static analysis:

./fetch-driver.sh "/mnt/win/Program Files/HP/HP Smart Tank Plus 570 series/DriverStore"
./fetch-driver.sh "/mnt/win/Program Files/HP/HP Scan/DriverStore"
# → driver-store/<Product>/... (git-ignored, not committed)

The scan-engine DLL

In any HP DriverStore, the code that actually talks to the printer is:

<DriverStore>/NGScanDriver/drivers/scanner/x64/HPScanTEDrv_x64.dll

It exposes the WalkupScanToComp / ScanJob / eSCL endpoints and the event-wait loop. For another HP printer, copy its DriverStore with fetch-driver.sh and strings-ing HPScanTEDrv_x64.dll immediately reveals the supported endpoints (1st-gen WalkupScan vs 2nd-gen WalkupScanToComp, eSCL, etc.).

Status and limits

Validated:

Remaining limit: