A thread dump is a snapshot of the state of all threads that are part of the process.
In Xray we have 4 main processes: Server, Indexer, Analysis, Persist, which require collecting information from each separately.
Starting from Xray version 3.103 a new process has been introduced, called Policy Enforcer.
To collect Xray thread dumps, run the following cURL commands:
Note
The profiling endpoint is only accessible via localhost.
curl "localhost:8000/debug/pprof/goroutine?debug=2" > "server.dump.$(date +%Y%m%d%H%M%S).td"
curl "localhost:7000/debug/pprof/goroutine?debug=2" > "analysis.dump.$(date +%Y%m%d%H%M%S).td"
curl "localhost:7002/debug/pprof/goroutine?debug=2" > "indexer.dump.$(date +%Y%m%d%H%M%S).td"
curl "localhost:7003/debug/pprof/goroutine?debug=2" > "persist.dump.$(date +%Y%m%d%H%M%S).td"
—---- In Xray version 3.103c and above:
curl "localhost:7009/debug/pprof/goroutine?debug=2" > "policyEnforcer.dump.$(date +%Y%m%d%H%M%S).td"
Another useful debugging tool is CPU profiling, which can be collected by:
curl "localhost:8000/debug/pprof/profile?seconds=60" > "server.cpu-profile-60s.$(date +%Y%m%d%H%M%S).out"
curl "localhost:7000/debug/pprof/profile?seconds=60" > "analysis.cpu-profile-60s.$(date +%Y%m%d%H%M%S).out"
curl "localhost:7002/debug/pprof/profile?seconds=60" > "indexer.cpu-profile-60s.$(date +%Y%m%d%H%M%S).out"
curl "localhost:7003/debug/pprof/profile?seconds=60" > "persist.cpu-profile-60s.$(date +%Y%m%d%H%M%S).out"
—---- In Xray version 3.103c and above:
curl "localhost:7009/debug/pprof/profile?seconds=60" > "policyEnforcer.cpu-profile-60s.$(date +%Y%m%d%H%M%S).out"
To collect a Heap Memory profile:
curl "localhost:8000/debug/pprof/heap" > "server.heap.$(date +%Y%m%d%H%M%S).out"
curl "localhost:7000/debug/pprof/heap" > "analysis.heap.$(date +%Y%m%d%H%M%S).out"
curl "localhost:7002/debug/pprof/heap" > "indexer.heap.$(date +%Y%m%d%H%M%S).out"
curl "localhost:7003/debug/pprof/heap" > "persist.heap.$(date +%Y%m%d%H%M%S).out"
—---- In Xray version 3.103c and above:
curl "localhost:7009/debug/pprof/heap" > "policyEnforcer.heap.$(date +%Y%m%d%H%M%S).out"