IT极限技术分享汇py-spy
安装
# pip install py-spy -i https://mirrors.aliyun.com/pypi/simple/输出
sh-4.2# pip install py-spy -i https://mirrors.aliyun.com/pypi/simple/
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Looking in indexes: https://mirrors.aliyun.com/pypi/simple/
Collecting py-spy
Downloading https://mirrors.aliyun.com/pypi/packages/c9/c1/5e012669ebb687e546dc99fcfc4861ebfcf3a337b7a41af945df23140bb5/py_spy-0.4.0-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (2.7 MB)
|████████████████████████████████| 2.7 MB 1.1 MB/s
Installing collected packages: py-spy
Successfully installed py-spy-0.4.0使用
用top 找到占用cpu高的程序
# top输出
top - 10:38:44 up 287 days, 12:34, 0 users, load average: 12.57, 10.92, 10.09
Tasks: 27 total, 4 running, 23 sleeping, 0 stopped, 0 zombie
%Cpu(s): 20.3 us, 3.9 sy, 0.0 ni, 73.1 id, 0.0 wa, 0.0 hi, 2.7 si, 0.0 st
KiB Mem : 26381324+total, 6515296 free, 21381326+used, 43484692 buff/cache
KiB Swap: 0 total, 0 free, 0 used. 46831644 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
19 root 20 0 1469452 164264 10836 R 101.0 0.1 3:18.40 python
28 root 20 0 1488212 152292 12160 R 101.0 0.1 3:19.60 python
29 root 20 0 1504032 166216 16440 R 100.3 0.1 3:17.54 python
445 root 20 0 1811540 148344 16556 S 12.3 0.1 0:00.76 python
21 root 20 0 1504656 42840 16204 S 8.3 0.0 0:16.91 python
30 root 20 0 2797244 34032 10720 S 8.3 0.0 0:17.09 python
24 root 20 0 1487544 39276 12320 S 8.0 0.0 0:14.84 python
25 root 20 0 2602332 38744 15152 S 8.0 0.0 0:15.25 python
27 root 20 0 1496984 38784 14920 S 8.0 0.0 0:16.31 python
22 root 20 0 1469752 33660 11168 S 5.0 0.0 0:09.68 python
26 root 20 0 1487900 35288 11876 S 5.0 0.0 0:09.80 python
13 root 20 0 1750800 50184 16708 S 1.3 0.0 0:02.37 python
14 root 20 0 1750800 47872 16728 S 1.3 0.0 0:02.25 python
16 root 20 0 1750800 49836 16628 S 1.3 0.0 0:02.39 python
17 root 20 0 1750800 48208 16580 S 1.3 0.0 0:02.33 python
15 root 20 0 1750800 52168 16932 S 1.0 0.0 0:02.30 python
11 root 20 0 122540 19120 3388 S 0.7 0.0 0:03.97 supervis+ 上面进程号 19，28， 29都很高, py-spy 出场
# py-spy top --pid 19输出
Collecting samples from '/usr/local/bin/python -u message_consumer.py action_message mygroupid111' (python v2.7.17)
Total Samples 4281
GIL: 98.00%, Active: 100.00%, Threads: 1
%Own %Total OwnTime TotalTime Function (filename)
47.00% 47.00% 20.52s 20.52s raw_decode (json/decoder.py)
21.00% 100.00% 10.34s 42.81s <module> (message_consumer.py)
20.00% 20.00% 6.54s 6.54s decode (encodings/utf_8.py)
9.00% 56.00% 4.21s 24.73s decode (json/decoder.py)
3.00% 59.00% 1.20s 25.93s loads (json/__init__.py)
Press Control-C to quit, or ? for help.上面可以看到性能瓶颈在json decoder上
把输出塞给AI, 加上源代码，马上得到优化方按，用ujson替代json, 加上其他的缓存，问题解决！